diff options
| author | DanConwayDev <DanConwayDev@protonmail.com> | 2025-11-04 09:31:57 +0000 |
|---|---|---|
| committer | DanConwayDev <DanConwayDev@protonmail.com> | 2025-11-04 09:31:57 +0000 |
| commit | 22557f15d6a7b77f72d4597fc05aa06346495a33 (patch) | |
| tree | e31e0cdecfc4cb1e28246227a7ef295b71687b09 /docs/learnings/grasp-audit.md | |
| parent | b3031800cd95601c2d9cd2d24034364d1496b073 (diff) | |
docs: major cleanup and reorganization
- Archive 30 completed session documents to docs/archive/
- Extract learnings to docs/learnings/ (nix-flakes, nostr-sdk, grasp-audit)
- Create CURRENT_STATUS.md as single source of truth
- Create AGENTS.md with documentation guidelines
- Create docs/archive/README.md for archive organization
- Clean root directory: 32 files → 4 files
Root directory now contains only:
- README.md (project overview)
- AGENTS.md (documentation guidelines)
- CURRENT_STATUS.md (current state)
- CLEANUP_SUMMARY.md (cleanup report)
All historical documents preserved in docs/archive/ with proper dating.
All reusable knowledge extracted to docs/learnings/.
Benefits:
- Easy to find current information
- Clear document lifecycle
- No more documentation sprawl
- Learnings are accessible and reusable
- Better onboarding for new developers/agents
File counts:
- Root: 4 (was 32)
- Permanent docs: 7
- Learnings: 3 (new)
- Archive: 32 (new)
- Total: 49 well-organized docs
Diffstat (limited to 'docs/learnings/grasp-audit.md')
| -rw-r--r-- | docs/learnings/grasp-audit.md | 498 |
1 files changed, 498 insertions, 0 deletions
diff --git a/docs/learnings/grasp-audit.md b/docs/learnings/grasp-audit.md new file mode 100644 index 0000000..531ebda --- /dev/null +++ b/docs/learnings/grasp-audit.md | |||
| @@ -0,0 +1,498 @@ | |||
| 1 | # GRASP Audit Tool - Patterns and Learnings | ||
| 2 | |||
| 3 | **Purpose:** Document grasp-audit architecture, patterns, and lessons learned | ||
| 4 | **Last Updated:** November 4, 2025 | ||
| 5 | |||
| 6 | --- | ||
| 7 | |||
| 8 | ## Overview | ||
| 9 | |||
| 10 | `grasp-audit` is a compliance testing tool for GRASP (Git Relays Authorized via Signed-Nostr Proofs) protocol implementations. It tests both Nostr relay compliance (NIP-01) and GRASP-specific functionality. | ||
| 11 | |||
| 12 | --- | ||
| 13 | |||
| 14 | ## Architecture Decisions | ||
| 15 | |||
| 16 | ### Separate Crate Strategy | ||
| 17 | |||
| 18 | **Decision:** Build `grasp-audit` as a separate crate from `ngit-grasp` | ||
| 19 | |||
| 20 | **Why:** | ||
| 21 | 1. **Parallel Development**: Can build tests before implementation | ||
| 22 | 2. **Isolated Testing**: Tests run in isolation (CI/CD safe) | ||
| 23 | 3. **Production Auditing**: Can audit live production services | ||
| 24 | 4. **Reusability**: Other GRASP implementations can use it | ||
| 25 | |||
| 26 | **Location:** `grasp-audit/` subdirectory with own `Cargo.toml` and `flake.nix` | ||
| 27 | |||
| 28 | --- | ||
| 29 | |||
| 30 | ### Audit Event Tagging Strategy | ||
| 31 | |||
| 32 | **Problem:** Test events pollute the relay and need cleanup without deletion events. | ||
| 33 | |||
| 34 | **Solution:** Use special tags to mark audit events: | ||
| 35 | |||
| 36 | ```rust | ||
| 37 | // Every audit event includes these tags | ||
| 38 | [ | ||
| 39 | ["t", "grasp-audit-test-event"], // Marker | ||
| 40 | ["t", "audit-{run-id}"], // Run isolation | ||
| 41 | ["t", "audit-cleanup-after-{timestamp}"] // Cleanup time | ||
| 42 | ] | ||
| 43 | ``` | ||
| 44 | |||
| 45 | **Benefits:** | ||
| 46 | - ✅ **Queryable**: Can find all audit events via tag filter | ||
| 47 | - ✅ **Isolated**: Each test run has unique run ID | ||
| 48 | - ✅ **Self-cleaning**: Cleanup timestamp indicates when to delete | ||
| 49 | - ✅ **No deletion events**: Direct database cleanup, no KIND 5 events | ||
| 50 | - ✅ **Production safe**: Won't interfere with real events | ||
| 51 | |||
| 52 | **Reference:** See `docs/archive/2025-11-04-tag-migration.md` | ||
| 53 | |||
| 54 | --- | ||
| 55 | |||
| 56 | ### Standard "t" Tags vs Custom Tags | ||
| 57 | |||
| 58 | **Evolution:** | ||
| 59 | 1. **Original**: Custom single-letter tags (`g`, `r`, `c`) | ||
| 60 | 2. **Current**: Standard NIP-01 "t" tags with prefixed values | ||
| 61 | |||
| 62 | **Why we changed:** | ||
| 63 | - ❌ Custom tags could conflict with other systems | ||
| 64 | - ✅ "t" tag is standard for categorization/topics | ||
| 65 | - ✅ Multiple "t" tags are expected and supported | ||
| 66 | - ✅ Self-documenting values (`audit-{run-id}` vs just `{run-id}`) | ||
| 67 | - ✅ Better namespacing with prefixes | ||
| 68 | |||
| 69 | **Migration:** Completed November 4, 2025 | ||
| 70 | |||
| 71 | --- | ||
| 72 | |||
| 73 | ## Code Patterns | ||
| 74 | |||
| 75 | ### Audit Configuration | ||
| 76 | |||
| 77 | ```rust | ||
| 78 | use grasp_audit::audit::AuditConfig; | ||
| 79 | |||
| 80 | // CI mode - isolated test runs | ||
| 81 | let config = AuditConfig::ci(); | ||
| 82 | // Generates UUID run ID: "ci-{uuid}" | ||
| 83 | // Cleanup after 1 hour | ||
| 84 | |||
| 85 | // Production mode - persistent run ID | ||
| 86 | let config = AuditConfig::production("prod-server-1"); | ||
| 87 | // Uses provided run ID | ||
| 88 | // Cleanup after 24 hours | ||
| 89 | ``` | ||
| 90 | |||
| 91 | **When to use:** | ||
| 92 | - **CI mode**: Automated testing, parallel runs, temporary | ||
| 93 | - **Production mode**: Manual audits, monitoring, persistent | ||
| 94 | |||
| 95 | --- | ||
| 96 | |||
| 97 | ### Creating Audit Events | ||
| 98 | |||
| 99 | ```rust | ||
| 100 | use grasp_audit::audit::{AuditConfig, AuditEventBuilder}; | ||
| 101 | use nostr_sdk::prelude::*; | ||
| 102 | |||
| 103 | let config = AuditConfig::ci(); | ||
| 104 | let keys = Keys::generate(); | ||
| 105 | |||
| 106 | // Create audit event | ||
| 107 | let event = AuditEventBuilder::new(&config, Kind::TextNote, "test content") | ||
| 108 | .build(&keys)?; | ||
| 109 | |||
| 110 | // Event automatically includes: | ||
| 111 | // - Audit marker tag | ||
| 112 | // - Run ID tag | ||
| 113 | // - Cleanup timestamp tag | ||
| 114 | ``` | ||
| 115 | |||
| 116 | --- | ||
| 117 | |||
| 118 | ### Querying Audit Events | ||
| 119 | |||
| 120 | ```rust | ||
| 121 | use grasp_audit::client::AuditClient; | ||
| 122 | use grasp_audit::audit::AuditConfig; | ||
| 123 | |||
| 124 | let config = AuditConfig::ci(); | ||
| 125 | let client = AuditClient::new(config, keys); | ||
| 126 | |||
| 127 | // Connect to relay | ||
| 128 | client.add_relay("ws://localhost:7000").await?; | ||
| 129 | client.connect().await; | ||
| 130 | |||
| 131 | // Query audit events for this run | ||
| 132 | let events = client.query().await?; | ||
| 133 | |||
| 134 | // Events are filtered by: | ||
| 135 | // - "grasp-audit-test-event" marker | ||
| 136 | // - Current run ID | ||
| 137 | ``` | ||
| 138 | |||
| 139 | --- | ||
| 140 | |||
| 141 | ### Test Isolation | ||
| 142 | |||
| 143 | **Each test run is isolated by unique run ID:** | ||
| 144 | |||
| 145 | ```rust | ||
| 146 | // CI mode generates unique UUID per run | ||
| 147 | let config1 = AuditConfig::ci(); | ||
| 148 | let config2 = AuditConfig::ci(); | ||
| 149 | |||
| 150 | // config1.run_id != config2.run_id | ||
| 151 | // Tests won't interfere with each other | ||
| 152 | ``` | ||
| 153 | |||
| 154 | **Benefits:** | ||
| 155 | - ✅ Parallel CI/CD runs don't conflict | ||
| 156 | - ✅ Can run multiple test suites simultaneously | ||
| 157 | - ✅ Easy to identify which run created which events | ||
| 158 | - ✅ Cleanup can target specific runs | ||
| 159 | |||
| 160 | --- | ||
| 161 | |||
| 162 | ### Cleanup Strategy | ||
| 163 | |||
| 164 | **Two-phase cleanup:** | ||
| 165 | |||
| 166 | 1. **Automatic expiry** via cleanup timestamp tag | ||
| 167 | 2. **Manual cleanup** by querying and deleting | ||
| 168 | |||
| 169 | ```rust | ||
| 170 | // Events include cleanup timestamp | ||
| 171 | ["t", "audit-cleanup-after-1730707200"] | ||
| 172 | |||
| 173 | // Cleanup process: | ||
| 174 | // 1. Query events with expired cleanup timestamp | ||
| 175 | // 2. Delete from database directly (no KIND 5) | ||
| 176 | // 3. Avoid deletion event pollution | ||
| 177 | ``` | ||
| 178 | |||
| 179 | **Implementation:** To be built in relay (not in audit tool) | ||
| 180 | |||
| 181 | --- | ||
| 182 | |||
| 183 | ## Testing Strategy | ||
| 184 | |||
| 185 | ### Test Organization | ||
| 186 | |||
| 187 | ``` | ||
| 188 | grasp-audit/src/specs/ | ||
| 189 | ├── nip01_smoke.rs # NIP-01 basic functionality | ||
| 190 | ├── grasp_01_relay.rs # GRASP-01 relay requirements (planned) | ||
| 191 | └── mod.rs # Test suite registry | ||
| 192 | ``` | ||
| 193 | |||
| 194 | ### Unit vs Integration Tests | ||
| 195 | |||
| 196 | **Unit Tests** (no relay required): | ||
| 197 | ```rust | ||
| 198 | #[cfg(test)] | ||
| 199 | mod tests { | ||
| 200 | #[test] | ||
| 201 | fn test_audit_config() { | ||
| 202 | let config = AuditConfig::ci(); | ||
| 203 | assert!(config.run_id.starts_with("ci-")); | ||
| 204 | } | ||
| 205 | } | ||
| 206 | ``` | ||
| 207 | |||
| 208 | **Integration Tests** (relay required): | ||
| 209 | ```rust | ||
| 210 | #[cfg(test)] | ||
| 211 | mod tests { | ||
| 212 | #[tokio::test] | ||
| 213 | #[ignore] // Requires relay | ||
| 214 | async fn test_smoke_tests_against_relay() { | ||
| 215 | // Test against real relay | ||
| 216 | } | ||
| 217 | } | ||
| 218 | ``` | ||
| 219 | |||
| 220 | **Running tests:** | ||
| 221 | ```bash | ||
| 222 | # Unit tests (fast, no dependencies) | ||
| 223 | cargo test --lib | ||
| 224 | |||
| 225 | # Integration tests (requires relay) | ||
| 226 | docker run --rm -p 7000:7000 scsibug/nostr-rs-relay | ||
| 227 | cargo test -- --ignored | ||
| 228 | ``` | ||
| 229 | |||
| 230 | --- | ||
| 231 | |||
| 232 | ### Test Result Reporting | ||
| 233 | |||
| 234 | ```rust | ||
| 235 | use grasp_audit::result::AuditResult; | ||
| 236 | |||
| 237 | // Run tests | ||
| 238 | let results = vec![ | ||
| 239 | AuditResult::pass("websocket_connection", "Connected successfully"), | ||
| 240 | AuditResult::fail("invalid_event", "Expected rejection, got acceptance"), | ||
| 241 | ]; | ||
| 242 | |||
| 243 | // Report | ||
| 244 | for result in &results { | ||
| 245 | println!("{}", result); | ||
| 246 | } | ||
| 247 | |||
| 248 | // Summary | ||
| 249 | let passed = results.iter().filter(|r| r.is_pass()).count(); | ||
| 250 | let total = results.len(); | ||
| 251 | println!("Results: {}/{} passed ({:.1}%)", | ||
| 252 | passed, total, (passed as f64 / total as f64) * 100.0); | ||
| 253 | ``` | ||
| 254 | |||
| 255 | --- | ||
| 256 | |||
| 257 | ## CLI Design | ||
| 258 | |||
| 259 | ### Command Structure | ||
| 260 | |||
| 261 | ```bash | ||
| 262 | grasp-audit audit [OPTIONS] | ||
| 263 | |||
| 264 | Options: | ||
| 265 | --relay <URL> Relay to test (required) | ||
| 266 | --mode <MODE> ci or production (default: ci) | ||
| 267 | --run-id <ID> Custom run ID (production mode only) | ||
| 268 | --spec <SPEC> Test spec to run (default: all) | ||
| 269 | --verbose Detailed output | ||
| 270 | ``` | ||
| 271 | |||
| 272 | ### Usage Examples | ||
| 273 | |||
| 274 | ```bash | ||
| 275 | # CI mode - quick smoke test | ||
| 276 | grasp-audit audit \ | ||
| 277 | --relay ws://localhost:7000 \ | ||
| 278 | --mode ci \ | ||
| 279 | --spec nip01-smoke | ||
| 280 | |||
| 281 | # Production mode - full compliance audit | ||
| 282 | grasp-audit audit \ | ||
| 283 | --relay wss://relay.example.com \ | ||
| 284 | --mode production \ | ||
| 285 | --run-id "audit-2025-11-04" \ | ||
| 286 | --verbose | ||
| 287 | |||
| 288 | # Test all specs | ||
| 289 | grasp-audit audit --relay ws://localhost:7000 | ||
| 290 | ``` | ||
| 291 | |||
| 292 | --- | ||
| 293 | |||
| 294 | ## Lessons Learned | ||
| 295 | |||
| 296 | ### 1. Tag Migration is Breaking | ||
| 297 | |||
| 298 | **Lesson:** Changing tag structure breaks event queries. | ||
| 299 | |||
| 300 | **Impact:** Events created with old tags won't be found by new queries. | ||
| 301 | |||
| 302 | **Mitigation:** | ||
| 303 | - ✅ Accept breaking changes in alpha stage | ||
| 304 | - ✅ Document migration clearly | ||
| 305 | - ✅ Old events auto-expire via cleanup | ||
| 306 | - ✅ No production deployments affected | ||
| 307 | |||
| 308 | **Reference:** `docs/archive/2025-11-04-tag-migration.md` | ||
| 309 | |||
| 310 | --- | ||
| 311 | |||
| 312 | ### 2. Test Data Lifecycle Matters | ||
| 313 | |||
| 314 | **Lesson:** Test events accumulate and pollute relay. | ||
| 315 | |||
| 316 | **Solution:** Built-in cleanup strategy from day one. | ||
| 317 | |||
| 318 | **Implementation:** | ||
| 319 | - Every event has cleanup timestamp | ||
| 320 | - Relay can cleanup expired events | ||
| 321 | - No deletion event pollution (direct DB cleanup) | ||
| 322 | |||
| 323 | --- | ||
| 324 | |||
| 325 | ### 3. Isolation Enables Parallel Testing | ||
| 326 | |||
| 327 | **Lesson:** Unique run IDs enable parallel test execution. | ||
| 328 | |||
| 329 | **Benefit:** CI/CD can run multiple test suites simultaneously. | ||
| 330 | |||
| 331 | **Pattern:** | ||
| 332 | ```rust | ||
| 333 | // Each CI run gets unique ID | ||
| 334 | let config = AuditConfig::ci(); | ||
| 335 | // run_id = "ci-{uuid}" | ||
| 336 | |||
| 337 | // Tests isolated by run ID | ||
| 338 | let events = client.query().await?; | ||
| 339 | // Only returns events for this run | ||
| 340 | ``` | ||
| 341 | |||
| 342 | --- | ||
| 343 | |||
| 344 | ### 4. Standards Compliance Reduces Friction | ||
| 345 | |||
| 346 | **Lesson:** Using standard NIP-01 "t" tags instead of custom tags. | ||
| 347 | |||
| 348 | **Benefits:** | ||
| 349 | - ✅ No conflicts with other systems | ||
| 350 | - ✅ Standard relay filtering works | ||
| 351 | - ✅ Better interoperability | ||
| 352 | - ✅ Self-documenting | ||
| 353 | |||
| 354 | --- | ||
| 355 | |||
| 356 | ## Future Enhancements | ||
| 357 | |||
| 358 | ### Planned Features | ||
| 359 | |||
| 360 | - [ ] **GRASP-01 Test Suite**: Repository announcement and state event tests | ||
| 361 | - [ ] **Test Report Generation**: JSON/HTML output for CI/CD | ||
| 362 | - [ ] **Performance Benchmarks**: Measure relay performance | ||
| 363 | - [ ] **Relay Comparison**: Side-by-side compliance comparison | ||
| 364 | - [ ] **Continuous Monitoring**: Periodic production audits | ||
| 365 | |||
| 366 | --- | ||
| 367 | |||
| 368 | ### Possible Improvements | ||
| 369 | |||
| 370 | - [ ] **Parallel Test Execution**: Run specs in parallel | ||
| 371 | - [ ] **Retry Logic**: Handle transient failures | ||
| 372 | - [ ] **Custom Assertions**: Domain-specific test helpers | ||
| 373 | - [ ] **Event Diff Tool**: Compare expected vs actual events | ||
| 374 | - [ ] **Cleanup Automation**: Auto-cleanup after tests | ||
| 375 | |||
| 376 | --- | ||
| 377 | |||
| 378 | ## Common Issues | ||
| 379 | |||
| 380 | ### Issue: Integration Tests Fail | ||
| 381 | |||
| 382 | **Symptoms:** Tests timeout or fail to connect | ||
| 383 | |||
| 384 | **Causes:** | ||
| 385 | 1. No relay running | ||
| 386 | 2. Wrong relay URL | ||
| 387 | 3. Firewall blocking connection | ||
| 388 | |||
| 389 | **Solution:** | ||
| 390 | ```bash | ||
| 391 | # Start relay | ||
| 392 | docker run --rm -p 7000:7000 scsibug/nostr-rs-relay | ||
| 393 | |||
| 394 | # Verify relay is running | ||
| 395 | curl http://localhost:7000 | ||
| 396 | |||
| 397 | # Run tests | ||
| 398 | cargo test -- --ignored | ||
| 399 | ``` | ||
| 400 | |||
| 401 | --- | ||
| 402 | |||
| 403 | ### Issue: Events Not Found in Query | ||
| 404 | |||
| 405 | **Symptoms:** Query returns empty even though events were sent | ||
| 406 | |||
| 407 | **Causes:** | ||
| 408 | 1. Wrong run ID (querying different run) | ||
| 409 | 2. Connection timing (query before event propagated) | ||
| 410 | 3. Tag mismatch (uppercase vs lowercase) | ||
| 411 | |||
| 412 | **Solution:** | ||
| 413 | ```rust | ||
| 414 | // Use same config for send and query | ||
| 415 | let config = AuditConfig::ci(); | ||
| 416 | |||
| 417 | // Wait for event to propagate | ||
| 418 | tokio::time::sleep(Duration::from_millis(500)).await; | ||
| 419 | |||
| 420 | // Verify tags match exactly | ||
| 421 | let t_tag = SingleLetterTag::lowercase(Alphabet::T); // Lowercase! | ||
| 422 | ``` | ||
| 423 | |||
| 424 | --- | ||
| 425 | |||
| 426 | ### Issue: Build Fails in CI | ||
| 427 | |||
| 428 | **Symptoms:** `cargo build` fails with dependency errors | ||
| 429 | |||
| 430 | **Cause:** Not in Nix dev environment | ||
| 431 | |||
| 432 | **Solution:** | ||
| 433 | ```bash | ||
| 434 | # Enter Nix environment first | ||
| 435 | cd grasp-audit | ||
| 436 | nix develop | ||
| 437 | |||
| 438 | # Then build | ||
| 439 | cargo build | ||
| 440 | ``` | ||
| 441 | |||
| 442 | --- | ||
| 443 | |||
| 444 | ## Quick Reference | ||
| 445 | |||
| 446 | ### Configuration | ||
| 447 | |||
| 448 | ```rust | ||
| 449 | // CI mode | ||
| 450 | let config = AuditConfig::ci(); | ||
| 451 | |||
| 452 | // Production mode | ||
| 453 | let config = AuditConfig::production("run-id"); | ||
| 454 | ``` | ||
| 455 | |||
| 456 | ### Event Creation | ||
| 457 | |||
| 458 | ```rust | ||
| 459 | let event = AuditEventBuilder::new(&config, kind, content) | ||
| 460 | .build(&keys)?; | ||
| 461 | ``` | ||
| 462 | |||
| 463 | ### Client Usage | ||
| 464 | |||
| 465 | ```rust | ||
| 466 | let client = AuditClient::new(config, keys); | ||
| 467 | client.add_relay("ws://localhost:7000").await?; | ||
| 468 | client.connect().await; | ||
| 469 | let events = client.query().await?; | ||
| 470 | ``` | ||
| 471 | |||
| 472 | ### Running Tests | ||
| 473 | |||
| 474 | ```bash | ||
| 475 | # Unit tests | ||
| 476 | cargo test --lib | ||
| 477 | |||
| 478 | # Integration tests | ||
| 479 | cargo test -- --ignored | ||
| 480 | |||
| 481 | # CLI | ||
| 482 | cargo run -- audit --relay ws://localhost:7000 | ||
| 483 | ``` | ||
| 484 | |||
| 485 | --- | ||
| 486 | |||
| 487 | ## References | ||
| 488 | |||
| 489 | - **GRASP Protocol**: https://gitworkshop.dev/danconwaydev.com/grasp | ||
| 490 | - **NIP-01**: https://github.com/nostr-protocol/nips/blob/master/01.md | ||
| 491 | - **NIP-34**: https://github.com/nostr-protocol/nips/blob/master/34.md | ||
| 492 | - **grasp-audit README**: `grasp-audit/README.md` | ||
| 493 | - **Tag Migration**: `docs/archive/2025-11-04-tag-migration.md` | ||
| 494 | |||
| 495 | --- | ||
| 496 | |||
| 497 | *Last updated: November 4, 2025* | ||
| 498 | *Status: Living document - update as grasp-audit evolves* | ||