upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/how-to
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2025-11-04 21:58:23 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2025-11-04 21:58:23 +0000
commit652c5913f695ba7e8dfd78cd0cbe5cc3de67fa59 (patch)
treee76bc0bcdc9eeec466ddf6e5e75a7f63a9d6650e /docs/how-to
parentc2c0cdba4af434043f3fa707231d8f5a7e3fd882 (diff)
test: migrate to TestRelay fixture pattern and add compliance docs
- Remove unnecessary 'nix' dev dependency (Unix syscalls crate, not needed) - Migrate announcement tests to new TestRelay fixture pattern - Delete legacy test files (announcement_tests.rs, test_relay.sh) - Add comprehensive test documentation (docs/how-to/test-compliance.md) - Update README.md with new test commands - All 18 integration tests passing (NIP-01 + NIP-34) Benefits: - Automatic relay lifecycle management - No manual setup required - Pure Rust integration tests - Better developer experience - CI/CD ready
Diffstat (limited to 'docs/how-to')
-rw-r--r--docs/how-to/test-compliance.md440
1 files changed, 440 insertions, 0 deletions
diff --git a/docs/how-to/test-compliance.md b/docs/how-to/test-compliance.md
new file mode 100644
index 0000000..370fb9a
--- /dev/null
+++ b/docs/how-to/test-compliance.md
@@ -0,0 +1,440 @@
1# How to Test GRASP Compliance
2
3**Purpose:** Guide for running compliance tests against ngit-grasp relay
4**Audience:** Developers, contributors, CI/CD maintainers
5**Category:** How-To (task-oriented)
6
7---
8
9## Overview
10
11This guide shows you how to run GRASP protocol compliance tests for the ngit-grasp relay. We have two test suites:
12
131. **Integration Tests** - Built into ngit-grasp, test core functionality
142. **GRASP Audit Tool** - Standalone compliance checker for any GRASP relay
15
16---
17
18## Quick Start
19
20```bash
21# Run all integration tests (automatic relay management)
22nix develop -c cargo test --test nip01_compliance --test nip34_announcements
23
24# Run NIP-01 compliance tests only
25nix develop -c cargo test --test nip01_compliance
26
27# Run NIP-34 announcement tests only
28nix develop -c cargo test --test nip34_announcements
29
30# Run with detailed output
31nix develop -c cargo test --test nip01_compliance -- --nocapture
32```
33
34**No manual setup needed!** Tests automatically start and stop relay instances.
35
36---
37
38## Integration Tests
39
40### What They Test
41
42**NIP-01 Compliance (`tests/nip01_compliance.rs`)**
43- Basic WebSocket connectivity
44- Event publishing and subscription
45- REQ/EVENT/CLOSE message handling
46- Filter-based event queries
47- Relay connection lifecycle
48
49**NIP-34 Announcements (`tests/nip34_announcements.rs`)**
50- Repository announcement acceptance (kind 30617)
51- Repository state event acceptance (kind 30618)
52- Clone URL validation
53- Relay URL validation
54- Domain matching
55- Multi-branch state events
56- Event queries by kind and tags
57
58### Test Architecture
59
60All integration tests use the **TestRelay fixture pattern**:
61
62```rust
63use crate::common::relay::TestRelay;
64
65#[tokio::test]
66async fn test_something() {
67 // Automatic relay startup on random port
68 let relay = TestRelay::start().await;
69
70 // Test code here
71 // ...
72
73 // Automatic cleanup when relay drops
74}
75```
76
77**Benefits:**
78- ✅ Automatic relay lifecycle management
79- ✅ Random port allocation (no conflicts)
80- ✅ Isolated test environments
81- ✅ Automatic cleanup on test completion
82- ✅ No manual relay management needed
83
84### Running Specific Tests
85
86```bash
87# Run a specific test by name
88nix develop -c cargo test --test nip01_compliance test_nip01_smoke
89
90# List all tests without running
91nix develop -c cargo test --test nip34_announcements -- --list
92
93# Run tests matching a pattern
94nix develop -c cargo test --test nip34_announcements test_accepts
95```
96
97### Test Output
98
99```bash
100$ nix develop -c cargo test --test nip01_compliance
101
102running 6 tests
103test test_nip01_smoke ... ok
104test test_subscription ... ok
105test test_event_publishing ... ok
106test test_filter_queries ... ok
107test test_connection_lifecycle ... ok
108test test_relay_lifecycle ... ignored
109
110test result: ok. 5 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out
111```
112
113---
114
115## GRASP Audit Tool
116
117### What It Does
118
119The `grasp-audit` tool is a standalone compliance checker that can test **any** GRASP relay (local or remote).
120
121**Located:** `grasp-audit/` subdirectory (separate Rust project)
122
123### Running the Audit Tool
124
125```bash
126# Enter the grasp-audit directory
127cd grasp-audit
128
129# Run against local relay
130nix develop -c cargo run -- --url ws://127.0.0.1:7000
131
132# Run against remote relay
133nix develop -c cargo run -- --url wss://relay.example.com
134
135# Run with verbose output
136nix develop -c cargo run -- --url ws://127.0.0.1:7000 --verbose
137```
138
139### What It Tests
140
141- NIP-01 basic relay functionality
142- NIP-34 repository announcement handling
143- GRASP-01 core service requirements
144- Domain validation
145- Event acceptance/rejection rules
146
147### Example Output
148
149```bash
150$ cd grasp-audit
151$ nix develop -c cargo run -- --url ws://127.0.0.1:7000
152
153GRASP Compliance Audit
154======================
155Relay: ws://127.0.0.1:7000
156
157✅ NIP-01: Basic Connectivity
158✅ NIP-01: Event Publishing
159✅ NIP-01: Subscriptions
160✅ NIP-34: Repository Announcements
161✅ NIP-34: State Events
162✅ GRASP-01: Domain Validation
163
164Summary: 6/6 tests passed
165Status: COMPLIANT
166```
167
168---
169
170## Testing Workflow
171
172### For Development
173
174**1. Quick Validation (after code changes)**
175```bash
176# Run all integration tests
177nix develop -c cargo test --test nip01_compliance --test nip34_announcements
178```
179
180**2. Deep Compliance Check (before release)**
181```bash
182# Start your relay
183nix develop -c cargo run
184
185# In another terminal, run audit tool
186cd grasp-audit
187nix develop -c cargo run -- --url ws://127.0.0.1:8080
188```
189
190### For CI/CD
191
192**Recommended CI pipeline:**
193
194```yaml
195# .github/workflows/test.yml example
196test:
197 runs-on: ubuntu-latest
198 steps:
199 - uses: actions/checkout@v3
200 - uses: cachix/install-nix-action@v22
201 - name: Run integration tests
202 run: nix develop -c cargo test --test nip01_compliance --test nip34_announcements
203```
204
205**Why this works:**
206- No external relay needed
207- Tests manage their own relay instances
208- Fast parallel execution
209- Clean isolation
210
211---
212
213## Test Configuration
214
215### Environment Variables
216
217Tests use these environment variables (set automatically by TestRelay):
218
219- `NGIT_DOMAIN` - Domain for clone URL validation (auto-set to bind address)
220- `NGIT_RELAY_DATA_PATH` - Temporary directory for relay data
221- `RUST_LOG` - Logging level (optional, for debugging)
222
223**Example: Enable debug logging**
224```bash
225RUST_LOG=debug nix develop -c cargo test --test nip01_compliance -- --nocapture
226```
227
228### Test Data Locations
229
230Integration tests use temporary directories:
231
232```
233/tmp/ngit-test-XXXXXX/ # Relay data (auto-cleaned)
234 ├── events/ # Nostr events
235 └── git/ # Git repositories (if tested)
236```
237
238**Cleanup:** Automatic when test completes (or on failure).
239
240---
241
242## Troubleshooting
243
244### Test Hangs or Times Out
245
246**Problem:** Test hangs waiting for relay to start
247
248**Solution:**
249```bash
250# Check if port is already in use
251lsof -i :7000
252
253# Kill any stray relay processes
254pkill -f ngit-grasp
255
256# Re-run test
257nix develop -c cargo test --test nip01_compliance
258```
259
260### Connection Refused
261
262**Problem:** `Connection refused` error in tests
263
264**Cause:** Relay failed to start (check for port conflicts)
265
266**Solution:**
267```bash
268# Tests use random ports, but check for system issues
269netstat -tuln | grep LISTEN
270
271# Check relay logs
272RUST_LOG=debug nix develop -c cargo test --test nip01_compliance -- --nocapture
273```
274
275### Tests Pass Locally but Fail in CI
276
277**Problem:** CI environment differences
278
279**Common causes:**
280- Network restrictions (WebSocket blocked)
281- Insufficient resources (slow startup)
282- Missing dependencies
283
284**Solution:**
285```bash
286# Ensure Nix is installed in CI
287# Use longer timeouts for slow systems
288# Check CI logs for specific errors
289```
290
291### Audit Tool Can't Connect
292
293**Problem:** `grasp-audit` fails to connect to relay
294
295**Checklist:**
2961. Is the relay running? (`ps aux | grep ngit-grasp`)
2972. Is the URL correct? (ws:// for local, wss:// for remote)
2983. Is the port accessible? (`telnet 127.0.0.1 7000`)
2994. Check firewall rules
300
301---
302
303## Writing New Tests
304
305### Integration Test Pattern
306
307**1. Create test file in `tests/` directory**
308
309```rust
310// tests/my_new_tests.rs
311mod common;
312
313use common::relay::TestRelay;
314use tokio_tungstenite::connect_async;
315
316#[tokio::test]
317async fn test_my_feature() {
318 // Start relay
319 let relay = TestRelay::start().await;
320
321 // Connect
322 let (mut ws, _) = connect_async(relay.ws_url())
323 .await
324 .expect("Failed to connect");
325
326 // Test your feature
327 // ...
328
329 // Cleanup automatic when relay drops
330}
331```
332
333**2. Run your test**
334```bash
335nix develop -c cargo test --test my_new_tests
336```
337
338### Adding to Audit Tool
339
340**1. Edit `grasp-audit/src/main.rs`**
341
342Add your test function following existing patterns.
343
344**2. Test it**
345```bash
346cd grasp-audit
347nix develop -c cargo run -- --url ws://127.0.0.1:7000
348```
349
350---
351
352## Test Coverage
353
354### Current Coverage
355
356**NIP-01 (Nostr Relay):**
357- ✅ WebSocket connectivity
358- ✅ Event publishing
359- ✅ Subscriptions (REQ/EVENT/EOSE/CLOSE)
360- ✅ Filter queries
361- ✅ Connection lifecycle
362
363**NIP-34 (Git Stuff):**
364- ✅ Repository announcements (kind 30617)
365- ✅ Repository state events (kind 30618)
366- ✅ Clone URL validation
367- ✅ Relay URL validation
368- ✅ Domain matching
369- ✅ Multi-branch support
370- ✅ Event queries
371
372**GRASP-01 (Core Service):**
373- ✅ Nostr relay at `/`
374- ✅ NIP-34 event acceptance
375- ✅ Domain validation
376- ⏳ Git HTTP backend (planned)
377- ⏳ Push authorization (planned)
378
379### Gaps (TODO)
380
381- Git Smart HTTP protocol tests
382- Push authorization validation
383- Multi-maintainer scenarios
384- PR reference handling (`refs/nostr/<event-id>`)
385- CORS headers
386- NIP-11 relay info document
387
388---
389
390## Performance Testing
391
392### Load Testing (Future)
393
394```bash
395# Planned: Load test with multiple concurrent connections
396# TODO: Add load testing tools
397```
398
399### Benchmarking (Future)
400
401```bash
402# Planned: Benchmark event processing throughput
403# TODO: Add criterion benchmarks
404```
405
406---
407
408## Related Documentation
409
410- **[Test Strategy](../reference/test-strategy.md)** - Overall testing approach
411- **[Architecture](../explanation/architecture.md)** - System design
412- **[Getting Started](../tutorials/getting-started.md)** - Initial setup
413- **[Nix Flakes](./nix-flakes.md)** - Nix development environment
414
415---
416
417## Summary
418
419**For quick validation:**
420```bash
421nix develop -c cargo test --test nip01_compliance --test nip34_announcements
422```
423
424**For deep compliance check:**
425```bash
426cd grasp-audit
427nix develop -c cargo run -- --url ws://127.0.0.1:8080
428```
429
430**Key points:**
431- ✅ No manual relay management needed
432- ✅ Automatic cleanup and isolation
433- ✅ Fast parallel execution
434- ✅ Works in CI/CD
435- ✅ Tests both local and remote relays
436
437---
438
439**Last Updated:** November 4, 2025
440**Status:** ✅ Complete and current