upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/learnings/grasp-audit.md
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2025-11-04 09:31:57 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2025-11-04 09:31:57 +0000
commit22557f15d6a7b77f72d4597fc05aa06346495a33 (patch)
treee31e0cdecfc4cb1e28246227a7ef295b71687b09 /docs/learnings/grasp-audit.md
parentb3031800cd95601c2d9cd2d24034364d1496b073 (diff)
docs: major cleanup and reorganization
- Archive 30 completed session documents to docs/archive/ - Extract learnings to docs/learnings/ (nix-flakes, nostr-sdk, grasp-audit) - Create CURRENT_STATUS.md as single source of truth - Create AGENTS.md with documentation guidelines - Create docs/archive/README.md for archive organization - Clean root directory: 32 files → 4 files Root directory now contains only: - README.md (project overview) - AGENTS.md (documentation guidelines) - CURRENT_STATUS.md (current state) - CLEANUP_SUMMARY.md (cleanup report) All historical documents preserved in docs/archive/ with proper dating. All reusable knowledge extracted to docs/learnings/. Benefits: - Easy to find current information - Clear document lifecycle - No more documentation sprawl - Learnings are accessible and reusable - Better onboarding for new developers/agents File counts: - Root: 4 (was 32) - Permanent docs: 7 - Learnings: 3 (new) - Archive: 32 (new) - Total: 49 well-organized docs
Diffstat (limited to 'docs/learnings/grasp-audit.md')
-rw-r--r--docs/learnings/grasp-audit.md498
1 files changed, 498 insertions, 0 deletions
diff --git a/docs/learnings/grasp-audit.md b/docs/learnings/grasp-audit.md
new file mode 100644
index 0000000..531ebda
--- /dev/null
+++ b/docs/learnings/grasp-audit.md
@@ -0,0 +1,498 @@
1# GRASP Audit Tool - Patterns and Learnings
2
3**Purpose:** Document grasp-audit architecture, patterns, and lessons learned
4**Last Updated:** November 4, 2025
5
6---
7
8## Overview
9
10`grasp-audit` is a compliance testing tool for GRASP (Git Relays Authorized via Signed-Nostr Proofs) protocol implementations. It tests both Nostr relay compliance (NIP-01) and GRASP-specific functionality.
11
12---
13
14## Architecture Decisions
15
16### Separate Crate Strategy
17
18**Decision:** Build `grasp-audit` as a separate crate from `ngit-grasp`
19
20**Why:**
211. **Parallel Development**: Can build tests before implementation
222. **Isolated Testing**: Tests run in isolation (CI/CD safe)
233. **Production Auditing**: Can audit live production services
244. **Reusability**: Other GRASP implementations can use it
25
26**Location:** `grasp-audit/` subdirectory with own `Cargo.toml` and `flake.nix`
27
28---
29
30### Audit Event Tagging Strategy
31
32**Problem:** Test events pollute the relay and need cleanup without deletion events.
33
34**Solution:** Use special tags to mark audit events:
35
36```rust
37// Every audit event includes these tags
38[
39 ["t", "grasp-audit-test-event"], // Marker
40 ["t", "audit-{run-id}"], // Run isolation
41 ["t", "audit-cleanup-after-{timestamp}"] // Cleanup time
42]
43```
44
45**Benefits:**
46- ✅ **Queryable**: Can find all audit events via tag filter
47- ✅ **Isolated**: Each test run has unique run ID
48- ✅ **Self-cleaning**: Cleanup timestamp indicates when to delete
49- ✅ **No deletion events**: Direct database cleanup, no KIND 5 events
50- ✅ **Production safe**: Won't interfere with real events
51
52**Reference:** See `docs/archive/2025-11-04-tag-migration.md`
53
54---
55
56### Standard "t" Tags vs Custom Tags
57
58**Evolution:**
591. **Original**: Custom single-letter tags (`g`, `r`, `c`)
602. **Current**: Standard NIP-01 "t" tags with prefixed values
61
62**Why we changed:**
63- ❌ Custom tags could conflict with other systems
64- ✅ "t" tag is standard for categorization/topics
65- ✅ Multiple "t" tags are expected and supported
66- ✅ Self-documenting values (`audit-{run-id}` vs just `{run-id}`)
67- ✅ Better namespacing with prefixes
68
69**Migration:** Completed November 4, 2025
70
71---
72
73## Code Patterns
74
75### Audit Configuration
76
77```rust
78use grasp_audit::audit::AuditConfig;
79
80// CI mode - isolated test runs
81let config = AuditConfig::ci();
82// Generates UUID run ID: "ci-{uuid}"
83// Cleanup after 1 hour
84
85// Production mode - persistent run ID
86let config = AuditConfig::production("prod-server-1");
87// Uses provided run ID
88// Cleanup after 24 hours
89```
90
91**When to use:**
92- **CI mode**: Automated testing, parallel runs, temporary
93- **Production mode**: Manual audits, monitoring, persistent
94
95---
96
97### Creating Audit Events
98
99```rust
100use grasp_audit::audit::{AuditConfig, AuditEventBuilder};
101use nostr_sdk::prelude::*;
102
103let config = AuditConfig::ci();
104let keys = Keys::generate();
105
106// Create audit event
107let event = AuditEventBuilder::new(&config, Kind::TextNote, "test content")
108 .build(&keys)?;
109
110// Event automatically includes:
111// - Audit marker tag
112// - Run ID tag
113// - Cleanup timestamp tag
114```
115
116---
117
118### Querying Audit Events
119
120```rust
121use grasp_audit::client::AuditClient;
122use grasp_audit::audit::AuditConfig;
123
124let config = AuditConfig::ci();
125let client = AuditClient::new(config, keys);
126
127// Connect to relay
128client.add_relay("ws://localhost:7000").await?;
129client.connect().await;
130
131// Query audit events for this run
132let events = client.query().await?;
133
134// Events are filtered by:
135// - "grasp-audit-test-event" marker
136// - Current run ID
137```
138
139---
140
141### Test Isolation
142
143**Each test run is isolated by unique run ID:**
144
145```rust
146// CI mode generates unique UUID per run
147let config1 = AuditConfig::ci();
148let config2 = AuditConfig::ci();
149
150// config1.run_id != config2.run_id
151// Tests won't interfere with each other
152```
153
154**Benefits:**
155- ✅ Parallel CI/CD runs don't conflict
156- ✅ Can run multiple test suites simultaneously
157- ✅ Easy to identify which run created which events
158- ✅ Cleanup can target specific runs
159
160---
161
162### Cleanup Strategy
163
164**Two-phase cleanup:**
165
1661. **Automatic expiry** via cleanup timestamp tag
1672. **Manual cleanup** by querying and deleting
168
169```rust
170// Events include cleanup timestamp
171["t", "audit-cleanup-after-1730707200"]
172
173// Cleanup process:
174// 1. Query events with expired cleanup timestamp
175// 2. Delete from database directly (no KIND 5)
176// 3. Avoid deletion event pollution
177```
178
179**Implementation:** To be built in relay (not in audit tool)
180
181---
182
183## Testing Strategy
184
185### Test Organization
186
187```
188grasp-audit/src/specs/
189├── nip01_smoke.rs # NIP-01 basic functionality
190├── grasp_01_relay.rs # GRASP-01 relay requirements (planned)
191└── mod.rs # Test suite registry
192```
193
194### Unit vs Integration Tests
195
196**Unit Tests** (no relay required):
197```rust
198#[cfg(test)]
199mod tests {
200 #[test]
201 fn test_audit_config() {
202 let config = AuditConfig::ci();
203 assert!(config.run_id.starts_with("ci-"));
204 }
205}
206```
207
208**Integration Tests** (relay required):
209```rust
210#[cfg(test)]
211mod tests {
212 #[tokio::test]
213 #[ignore] // Requires relay
214 async fn test_smoke_tests_against_relay() {
215 // Test against real relay
216 }
217}
218```
219
220**Running tests:**
221```bash
222# Unit tests (fast, no dependencies)
223cargo test --lib
224
225# Integration tests (requires relay)
226docker run --rm -p 7000:7000 scsibug/nostr-rs-relay
227cargo test -- --ignored
228```
229
230---
231
232### Test Result Reporting
233
234```rust
235use grasp_audit::result::AuditResult;
236
237// Run tests
238let results = vec![
239 AuditResult::pass("websocket_connection", "Connected successfully"),
240 AuditResult::fail("invalid_event", "Expected rejection, got acceptance"),
241];
242
243// Report
244for result in &results {
245 println!("{}", result);
246}
247
248// Summary
249let passed = results.iter().filter(|r| r.is_pass()).count();
250let total = results.len();
251println!("Results: {}/{} passed ({:.1}%)",
252 passed, total, (passed as f64 / total as f64) * 100.0);
253```
254
255---
256
257## CLI Design
258
259### Command Structure
260
261```bash
262grasp-audit audit [OPTIONS]
263
264Options:
265 --relay <URL> Relay to test (required)
266 --mode <MODE> ci or production (default: ci)
267 --run-id <ID> Custom run ID (production mode only)
268 --spec <SPEC> Test spec to run (default: all)
269 --verbose Detailed output
270```
271
272### Usage Examples
273
274```bash
275# CI mode - quick smoke test
276grasp-audit audit \
277 --relay ws://localhost:7000 \
278 --mode ci \
279 --spec nip01-smoke
280
281# Production mode - full compliance audit
282grasp-audit audit \
283 --relay wss://relay.example.com \
284 --mode production \
285 --run-id "audit-2025-11-04" \
286 --verbose
287
288# Test all specs
289grasp-audit audit --relay ws://localhost:7000
290```
291
292---
293
294## Lessons Learned
295
296### 1. Tag Migration is Breaking
297
298**Lesson:** Changing tag structure breaks event queries.
299
300**Impact:** Events created with old tags won't be found by new queries.
301
302**Mitigation:**
303- ✅ Accept breaking changes in alpha stage
304- ✅ Document migration clearly
305- ✅ Old events auto-expire via cleanup
306- ✅ No production deployments affected
307
308**Reference:** `docs/archive/2025-11-04-tag-migration.md`
309
310---
311
312### 2. Test Data Lifecycle Matters
313
314**Lesson:** Test events accumulate and pollute relay.
315
316**Solution:** Built-in cleanup strategy from day one.
317
318**Implementation:**
319- Every event has cleanup timestamp
320- Relay can cleanup expired events
321- No deletion event pollution (direct DB cleanup)
322
323---
324
325### 3. Isolation Enables Parallel Testing
326
327**Lesson:** Unique run IDs enable parallel test execution.
328
329**Benefit:** CI/CD can run multiple test suites simultaneously.
330
331**Pattern:**
332```rust
333// Each CI run gets unique ID
334let config = AuditConfig::ci();
335// run_id = "ci-{uuid}"
336
337// Tests isolated by run ID
338let events = client.query().await?;
339// Only returns events for this run
340```
341
342---
343
344### 4. Standards Compliance Reduces Friction
345
346**Lesson:** Using standard NIP-01 "t" tags instead of custom tags.
347
348**Benefits:**
349- ✅ No conflicts with other systems
350- ✅ Standard relay filtering works
351- ✅ Better interoperability
352- ✅ Self-documenting
353
354---
355
356## Future Enhancements
357
358### Planned Features
359
360- [ ] **GRASP-01 Test Suite**: Repository announcement and state event tests
361- [ ] **Test Report Generation**: JSON/HTML output for CI/CD
362- [ ] **Performance Benchmarks**: Measure relay performance
363- [ ] **Relay Comparison**: Side-by-side compliance comparison
364- [ ] **Continuous Monitoring**: Periodic production audits
365
366---
367
368### Possible Improvements
369
370- [ ] **Parallel Test Execution**: Run specs in parallel
371- [ ] **Retry Logic**: Handle transient failures
372- [ ] **Custom Assertions**: Domain-specific test helpers
373- [ ] **Event Diff Tool**: Compare expected vs actual events
374- [ ] **Cleanup Automation**: Auto-cleanup after tests
375
376---
377
378## Common Issues
379
380### Issue: Integration Tests Fail
381
382**Symptoms:** Tests timeout or fail to connect
383
384**Causes:**
3851. No relay running
3862. Wrong relay URL
3873. Firewall blocking connection
388
389**Solution:**
390```bash
391# Start relay
392docker run --rm -p 7000:7000 scsibug/nostr-rs-relay
393
394# Verify relay is running
395curl http://localhost:7000
396
397# Run tests
398cargo test -- --ignored
399```
400
401---
402
403### Issue: Events Not Found in Query
404
405**Symptoms:** Query returns empty even though events were sent
406
407**Causes:**
4081. Wrong run ID (querying different run)
4092. Connection timing (query before event propagated)
4103. Tag mismatch (uppercase vs lowercase)
411
412**Solution:**
413```rust
414// Use same config for send and query
415let config = AuditConfig::ci();
416
417// Wait for event to propagate
418tokio::time::sleep(Duration::from_millis(500)).await;
419
420// Verify tags match exactly
421let t_tag = SingleLetterTag::lowercase(Alphabet::T); // Lowercase!
422```
423
424---
425
426### Issue: Build Fails in CI
427
428**Symptoms:** `cargo build` fails with dependency errors
429
430**Cause:** Not in Nix dev environment
431
432**Solution:**
433```bash
434# Enter Nix environment first
435cd grasp-audit
436nix develop
437
438# Then build
439cargo build
440```
441
442---
443
444## Quick Reference
445
446### Configuration
447
448```rust
449// CI mode
450let config = AuditConfig::ci();
451
452// Production mode
453let config = AuditConfig::production("run-id");
454```
455
456### Event Creation
457
458```rust
459let event = AuditEventBuilder::new(&config, kind, content)
460 .build(&keys)?;
461```
462
463### Client Usage
464
465```rust
466let client = AuditClient::new(config, keys);
467client.add_relay("ws://localhost:7000").await?;
468client.connect().await;
469let events = client.query().await?;
470```
471
472### Running Tests
473
474```bash
475# Unit tests
476cargo test --lib
477
478# Integration tests
479cargo test -- --ignored
480
481# CLI
482cargo run -- audit --relay ws://localhost:7000
483```
484
485---
486
487## References
488
489- **GRASP Protocol**: https://gitworkshop.dev/danconwaydev.com/grasp
490- **NIP-01**: https://github.com/nostr-protocol/nips/blob/master/01.md
491- **NIP-34**: https://github.com/nostr-protocol/nips/blob/master/34.md
492- **grasp-audit README**: `grasp-audit/README.md`
493- **Tag Migration**: `docs/archive/2025-11-04-tag-migration.md`
494
495---
496
497*Last updated: November 4, 2025*
498*Status: Living document - update as grasp-audit evolves*