upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/learnings/grasp-audit.md
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2025-12-04 12:34:20 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2025-12-04 13:02:59 +0000
commitd9bc5ed7fddef3a26de8e69a7124e1dbe5b8602f (patch)
treec76ffcbf246c8bef7545337316c0afb90433bbf5 /docs/learnings/grasp-audit.md
parent40831a9025d05fa354b7d8386eeebd902092ea86 (diff)
docs: update based on current implementation
Diffstat (limited to 'docs/learnings/grasp-audit.md')
-rw-r--r--docs/learnings/grasp-audit.md209
1 files changed, 71 insertions, 138 deletions
diff --git a/docs/learnings/grasp-audit.md b/docs/learnings/grasp-audit.md
index 14e5a2b..f4620d9 100644
--- a/docs/learnings/grasp-audit.md
+++ b/docs/learnings/grasp-audit.md
@@ -1,13 +1,13 @@
1# GRASP Audit Tool - Patterns and Learnings 1# GRASP Audit Tool - Patterns and Learnings
2 2
3**Purpose:** Document grasp-audit architecture, patterns, and lessons learned 3**Purpose:** Document grasp-audit architecture, patterns, and lessons learned
4**Last Updated:** November 4, 2025 4**Last Updated:** December 4, 2025
5 5
6--- 6---
7 7
8## Overview 8## Overview
9 9
10`grasp-audit` is a compliance testing tool for GRASP (Git Relays Authorized via Signed-Nostr Proofs) protocol implementations. It tests both Nostr relay compliance (NIP-01) and GRASP-specific functionality. 10`grasp-audit` is a **fully implemented** compliance testing tool for GRASP (Git Relays Authorized via Signed-Nostr Proofs) protocol implementations. It tests both Nostr relay compliance (NIP-01) and GRASP-specific functionality.
11 11
12--- 12---
13 13
@@ -32,10 +32,10 @@
32 32
33**Problem:** Test events pollute the relay and need cleanup without deletion events. 33**Problem:** Test events pollute the relay and need cleanup without deletion events.
34 34
35**Solution:** Use special tags to mark audit events: 35**Solution:** Use special tags to mark audit events (implemented in [`grasp-audit/src/audit.rs`](grasp-audit/src/audit.rs)):
36 36
37```rust 37```rust
38// Every audit event includes these tags 38// Every audit event includes these tags (added automatically)
39[ 39[
40 ["t", "grasp-audit-test-event"], // Marker 40 ["t", "grasp-audit-test-event"], // Marker
41 ["t", "audit-{run-id}"], // Run isolation 41 ["t", "audit-{run-id}"], // Run isolation
@@ -78,6 +78,8 @@
78 78
79### Audit Configuration 79### Audit Configuration
80 80
81From [`grasp-audit/src/audit.rs`](grasp-audit/src/audit.rs):
82
81```rust 83```rust
82use grasp_audit::audit::AuditConfig; 84use grasp_audit::audit::AuditConfig;
83 85
@@ -101,100 +103,41 @@ let config = AuditConfig::shared();
101 103
102### Creating Audit Events 104### Creating Audit Events
103 105
104```rust 106From [`grasp-audit/src/client.rs`](grasp-audit/src/client.rs):
105use grasp_audit::audit::{AuditConfig, AuditEventBuilder};
106use nostr_sdk::prelude::*;
107
108let config = AuditConfig::isolated();
109let keys = Keys::generate();
110
111// Create audit event
112let event = AuditEventBuilder::new(&config, Kind::TextNote, "test content")
113 .build(&keys)?;
114
115// Event automatically includes:
116// - Audit marker tag
117// - Run ID tag
118// - Cleanup timestamp tag
119```
120
121---
122
123### Querying Audit Events
124 107
125```rust 108```rust
126use grasp_audit::client::AuditClient; 109use grasp_audit::client::AuditClient;
127use grasp_audit::audit::AuditConfig; 110use grasp_audit::audit::AuditConfig;
128 111
129let config = AuditConfig::isolated(); 112let config = AuditConfig::isolated();
130let client = AuditClient::new(config, keys); 113let client = AuditClient::new("ws://localhost:8080", config).await?;
131
132// Connect to relay
133client.add_relay("ws://localhost:7000").await?;
134client.connect().await;
135
136// Query audit events for this run
137let events = client.query().await?;
138
139// Events are filtered by:
140// - "grasp-audit-test-event" marker
141// - Current run ID
142```
143
144---
145
146### Test Isolation
147
148**Each test run is isolated by unique run ID:**
149
150```rust
151// CI mode generates unique UUID per run
152let config1 = AuditConfig::isolated();
153let config2 = AuditConfig::isolated();
154
155// config1.run_id != config2.run_id
156// Tests won't interfere with each other
157```
158
159**Benefits:**
160 114
161- ✅ Parallel CI/CD runs don't conflict 115// Create and send an event - cleanup tags are added automatically
162- ✅ Can run multiple test suites simultaneously 116let event = client.event_builder()
163- ✅ Easy to identify which run created which events 117 .kind(Kind::TextNote)
164- ✅ Cleanup can target specific runs 118 .content("test content")
165 119 .build(&keys)?;
166---
167
168### Cleanup Strategy
169
170**Two-phase cleanup:**
171
1721. **Automatic expiry** via cleanup timestamp tag
1732. **Manual cleanup** by querying and deleting
174
175```rust
176// Events include cleanup timestamp
177["t", "audit-cleanup-after-1730707200"]
178 120
179// Cleanup process: 121client.send_event(event).await?;
180// 1. Query events with expired cleanup timestamp
181// 2. Delete from database directly (no KIND 5)
182// 3. Avoid deletion event pollution
183``` 122```
184 123
185**Implementation:** To be built in relay (not in audit tool)
186
187--- 124---
188 125
189## Testing Strategy 126### Test Suites
190 127
191### Test Organization 128From [`grasp-audit/src/specs/grasp01/mod.rs`](grasp-audit/src/specs/grasp01/mod.rs):
192 129
193``` 130```
194grasp-audit/src/specs/ 131grasp-audit/src/specs/grasp01/
195├── nip01_smoke.rs # NIP-01 basic functionality 132├── mod.rs # Module exports
196├── grasp_01_relay.rs # GRASP-01 relay requirements (planned) 133├── nip01_smoke.rs # NIP-01 basic functionality
197└── mod.rs # Test suite registry 134├── nip11_document.rs # NIP-11 document tests
135├── event_acceptance_policy.rs # GRASP-01 event rules
136├── cors.rs # CORS header tests
137├── git_clone.rs # Git clone operations
138├── push_authorization.rs # Push validation tests
139├── repository_creation.rs # Repository lifecycle
140└── spec_requirements.rs # Requirement definitions
198``` 141```
199 142
200### Unit vs Integration Tests 143### Unit vs Integration Tests
@@ -229,17 +172,18 @@ mod tests {
229 172
230```bash 173```bash
231# Unit tests (fast, no dependencies) 174# Unit tests (fast, no dependencies)
232cargo test --lib 175cd grasp-audit && nix develop -c cargo test --lib
233 176
234# Integration tests (requires relay) 177# Integration tests (requires relay via test-ngit-relay.sh)
235docker run --rm -p 7000:7000 scsibug/nostr-rs-relay 178cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test
236cargo test -- --ignored
237``` 179```
238 180
239--- 181---
240 182
241### Test Result Reporting 183### Test Result Reporting
242 184
185From [`grasp-audit/src/result.rs`](grasp-audit/src/result.rs):
186
243```rust 187```rust
244use grasp_audit::result::AuditResult; 188use grasp_audit::result::AuditResult;
245 189
@@ -255,7 +199,7 @@ for result in &results {
255} 199}
256 200
257// Summary 201// Summary
258let passed = results.iter().filter(|r| r.is_pass()).count(); 202let passed = results.iter().filter(|r| r.passed).count();
259let total = results.len(); 203let total = results.len();
260println!("Results: {}/{} passed ({:.1}%)", 204println!("Results: {}/{} passed ({:.1}%)",
261 passed, total, (passed as f64 / total as f64) * 100.0); 205 passed, total, (passed as f64 / total as f64) * 100.0);
@@ -291,7 +235,7 @@ grasp-audit audit \
291grasp-audit audit \ 235grasp-audit audit \
292 --relay wss://relay.example.com \ 236 --relay wss://relay.example.com \
293 --mode production \ 237 --mode production \
294 --run-id "audit-2025-11-04" \ 238 --run-id "audit-2025-12-04" \
295 --verbose 239 --verbose
296 240
297# Test all specs 241# Test all specs
@@ -366,25 +310,23 @@ let events = client.query().await?;
366 310
367--- 311---
368 312
369## Future Enhancements 313## What's Implemented
370 314
371### Planned Features 315### Completed Features
372
373- [ ] **GRASP-01 Test Suite**: Repository announcement and state event tests
374- [ ] **Test Report Generation**: JSON/HTML output for CI/CD
375- [ ] **Performance Benchmarks**: Measure relay performance
376- [ ] **Relay Comparison**: Side-by-side compliance comparison
377- [ ] **Continuous Monitoring**: Periodic production audits
378 316
379--- 317- ✅ **GRASP-01 Test Suites**: All NIP-01, NIP-11, CORS, event acceptance tests
318- ✅ **Spec Requirements Database**: Machine-readable requirements in [`spec_requirements.rs`](grasp-audit/src/specs/grasp01/spec_requirements.rs)
319- ✅ **Automatic Cleanup Tags**: Production-safe event tagging
320- ✅ **Test Isolation**: UUID run IDs for parallel execution
321- ✅ **AuditClient**: Nostr client wrapper with audit features
322- ✅ **Fixture Helpers**: Event creation helpers in [`fixtures.rs`](grasp-audit/src/fixtures.rs)
380 323
381### Possible Improvements 324### Future Enhancements
382 325
383- [ ] **Parallel Test Execution**: Run specs in parallel 326- [ ] **GRASP-02 Test Suite**: Proactive sync tests
384- [ ] **Retry Logic**: Handle transient failures 327- [ ] **HTML Report Generation**: Rich CI/CD reports
385- [ ] **Custom Assertions**: Domain-specific test helpers 328- [ ] **Performance Benchmarks**: Measure relay performance
386- [ ] **Event Diff Tool**: Compare expected vs actual events 329- [ ] **Relay Comparison**: Side-by-side compliance comparison
387- [ ] **Cleanup Automation**: Auto-cleanup after tests
388 330
389--- 331---
390 332
@@ -403,14 +345,12 @@ let events = client.query().await?;
403**Solution:** 345**Solution:**
404 346
405```bash 347```bash
406# Start relay 348# Use test-ngit-relay.sh for automated relay management
407docker run --rm -p 7000:7000 scsibug/nostr-rs-relay 349cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test
408
409# Verify relay is running
410curl http://localhost:7000
411 350
412# Run tests 351# Or manually:
413cargo test -- --ignored 352docker run --rm -p 18081:8081 ghcr.io/danconwaydev/ngit-relay:latest
353RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib -- --ignored
414``` 354```
415 355
416--- 356---
@@ -471,46 +411,39 @@ let config = AuditConfig::isolated();
471let config = AuditConfig::shared(); 411let config = AuditConfig::shared();
472``` 412```
473 413
474### Event Creation
475
476```rust
477let event = AuditEventBuilder::new(&config, kind, content)
478 .build(&keys)?;
479```
480
481### Client Usage 414### Client Usage
482 415
483```rust 416```rust
484let client = AuditClient::new(config, keys); 417let client = AuditClient::new("ws://localhost:7000", config).await?;
485client.add_relay("ws://localhost:7000").await?; 418assert!(client.is_connected().await);
486client.connect().await;
487let events = client.query().await?;
488``` 419```
489 420
490### Running Tests 421### Running Tests
491 422
492```bash 423```bash
493# Unit tests 424# Unit tests (from grasp-audit/)
494cargo test --lib 425nix develop -c cargo test --lib
495 426
496# Integration tests 427# Integration tests with ngit-relay
497cargo test -- --ignored 428nix develop -c bash test-ngit-relay.sh --mode test
498 429
499# CLI 430# CLI audit
500cargo run -- audit --relay ws://localhost:7000 431nix develop -c cargo run -- audit --relay ws://localhost:7000
501``` 432```
502 433
503--- 434### Key Files
504
505## References
506 435
507- **GRASP Protocol**: https://gitworkshop.dev/danconwaydev.com/grasp 436| File | Purpose |
508- **NIP-01**: https://github.com/nostr-protocol/nips/blob/master/01.md 437|------|---------|
509- **NIP-34**: https://github.com/nostr-protocol/nips/blob/master/34.md 438| [`grasp-audit/src/lib.rs`](grasp-audit/src/lib.rs) | Public API |
510- **grasp-audit README**: `grasp-audit/README.md` 439| [`grasp-audit/src/client.rs`](grasp-audit/src/client.rs) | AuditClient implementation |
511- **Tag Migration**: `docs/archive/2025-11-04-tag-migration.md` 440| [`grasp-audit/src/audit.rs`](grasp-audit/src/audit.rs) | AuditConfig, cleanup tags |
441| [`grasp-audit/src/specs/grasp01/mod.rs`](grasp-audit/src/specs/grasp01/mod.rs) | Test suite registry |
442| [`grasp-audit/src/specs/grasp01/spec_requirements.rs`](grasp-audit/src/specs/grasp01/spec_requirements.rs) | Requirement database |
512 443
513--- 444---
514 445
515_Last updated: November 4, 2025_ 446## Related Documentation
516_Status: Living document - update as grasp-audit evolves_ 447
448- [Test Strategy](../reference/test-strategy.md) - Overall testing approach
449- [GRASP-01 Implementation](grasp-01-implementation.md) - Main project learnings \ No newline at end of file