upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/archive/2025-11-03-grasp-audit-plan.md
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2025-12-03 11:19:40 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2025-12-03 11:19:40 +0000
commit2eaff5b79fed364d5eba5eb38e4b7bf76326884d (patch)
treedeacd6294f8860096ee82ee76930204efd65e33c /docs/archive/2025-11-03-grasp-audit-plan.md
parent57bc8cd9c021feaf08e139e8fb62800bc476068e (diff)
remove docs archive
Diffstat (limited to 'docs/archive/2025-11-03-grasp-audit-plan.md')
-rw-r--r--docs/archive/2025-11-03-grasp-audit-plan.md685
1 files changed, 0 insertions, 685 deletions
diff --git a/docs/archive/2025-11-03-grasp-audit-plan.md b/docs/archive/2025-11-03-grasp-audit-plan.md
deleted file mode 100644
index 96097a3..0000000
--- a/docs/archive/2025-11-03-grasp-audit-plan.md
+++ /dev/null
@@ -1,685 +0,0 @@
1# GRASP Audit Tool - Revised Plan
2
3**Decision:** Option B - Parallel development with separate `grasp-audit` crate
4
5## Key Requirements
6
71. ✅ **Separate crate**: `grasp-audit` (not `grasp-compliance-tests`)
82. ✅ **Parallel development**: Build ngit-grasp and tests simultaneously
93. ✅ **Isolated tests**: Can run in parallel for CI/CD
104. ✅ **Production audit**: Can test live production services
115. ✅ **Clean audit events**: Use special tags for easy cleanup (no deletion events)
12
13## Audit Event Strategy
14
15### The Challenge
16
17Tests create events on the relay. We need to:
18- Identify audit events vs. real events
19- Clean them up without leaving deletion trails
20- Support both isolated CI/CD tests and production audits
21
22### Solution: Audit Tags
23
24**Every audit event includes a special tag:**
25
26```json
27{
28 "tags": [
29 ["grasp-audit", "true"],
30 ["audit-run-id", "ci-2025-11-03-12345"],
31 ["audit-cleanup", "2025-11-03T12:00:00Z"]
32 ]
33}
34```
35
36**Tag meanings:**
37- `grasp-audit: true` - Marks this as an audit event
38- `audit-run-id` - Unique ID for this test run (for isolation)
39- `audit-cleanup` - Timestamp after which this can be cleaned up
40
41### Cleanup Script
42
43```bash
44# grasp-audit-cleanup.sh
45# Run this periodically to clean up old audit events
46
47grasp-audit cleanup \
48 --relay ws://localhost:7000 \
49 --older-than 24h \
50 --dry-run # Remove for actual cleanup
51```
52
53The cleanup script:
541. Queries for events with `grasp-audit: true`
552. Checks `audit-cleanup` timestamp
563. Deletes events older than threshold
574. No deletion events - direct database cleanup
58
59### Test Isolation
60
61**CI/CD Mode:**
62```rust
63// Each test run gets unique ID
64let audit_id = format!("ci-{}-{}",
65 env::var("CI_RUN_ID").unwrap_or_default(),
66 Uuid::new_v4()
67);
68
69// Tests only query their own events
70let filter = Filter::new()
71 .custom_tag(SingleLetterTag::lowercase(Alphabet::A), ["true"])
72 .custom_tag(SingleLetterTag::lowercase(Alphabet::B), [&audit_id]);
73```
74
75**Production Audit Mode:**
76```rust
77// Production audits use timestamped IDs
78let audit_id = format!("prod-audit-{}", Utc::now().timestamp());
79
80// Query all events (including real ones) to verify production behavior
81let filter = Filter::new()
82 .kind(Kind::Custom(30617)); // No audit filter - test real state
83```
84
85## Project Structure
86
87```
88grasp-audit/
89├── Cargo.toml
90├── README.md
91├── src/
92│ ├── lib.rs # Public API
93│ ├── client.rs # Test client
94│ ├── audit.rs # Audit event handling
95│ ├── cleanup.rs # Cleanup utilities
96│ ├── isolation.rs # Test isolation helpers
97│ └── specs/
98│ ├── mod.rs
99│ ├── nip01_smoke.rs # 6 smoke tests
100│ └── grasp_01_relay.rs # 12 GRASP tests
101├── fixtures/
102│ ├── repos/
103│ ├── events/
104│ └── keys/
105├── examples/
106│ ├── audit_server.rs # Audit a running server
107│ └── ci_tests.rs # CI/CD isolated tests
108└── bin/
109 └── grasp-audit.rs # CLI tool for cleanup
110```
111
112## Audit Client Design
113
114```rust
115// src/audit.rs
116
117use nostr_sdk::prelude::*;
118use std::time::Duration;
119
120#[derive(Debug, Clone)]
121pub struct AuditConfig {
122 /// Unique ID for this audit run
123 pub run_id: String,
124
125 /// Mode: CI (isolated) or Production (live)
126 pub mode: AuditMode,
127
128 /// Cleanup timestamp (events can be cleaned after this)
129 pub cleanup_after: Timestamp,
130
131 /// Whether to actually create events or just query
132 pub read_only: bool,
133}
134
135#[derive(Debug, Clone, Copy, PartialEq, Eq)]
136pub enum AuditMode {
137 /// Isolated CI/CD tests - only see own events
138 CI,
139
140 /// Production audit - see all events, minimal writes
141 Production,
142}
143
144impl AuditConfig {
145 /// Create config for CI/CD testing
146 pub fn ci() -> Self {
147 let run_id = format!("ci-{}", uuid::Uuid::new_v4());
148 Self {
149 run_id,
150 mode: AuditMode::CI,
151 cleanup_after: Timestamp::now() + Duration::from_secs(3600), // 1 hour
152 read_only: false,
153 }
154 }
155
156 /// Create config for production audit
157 pub fn production() -> Self {
158 let run_id = format!("prod-audit-{}", Timestamp::now().as_u64());
159 Self {
160 run_id,
161 mode: AuditMode::Production,
162 cleanup_after: Timestamp::now() + Duration::from_secs(300), // 5 minutes
163 read_only: true, // Default to read-only for production
164 }
165 }
166}
167
168/// Wrapper that adds audit tags to all events
169pub struct AuditClient {
170 client: Client,
171 config: AuditConfig,
172}
173
174impl AuditClient {
175 pub async fn new(relay_url: &str, config: AuditConfig) -> Result<Self> {
176 let client = Client::new(&Keys::generate());
177 client.add_relay(relay_url).await?;
178 client.connect().await;
179
180 Ok(Self { client, config })
181 }
182
183 /// Send an event with audit tags
184 pub async fn send_event(&self, mut event: Event) -> Result<EventId> {
185 if self.config.read_only {
186 return Err(anyhow!("Client is in read-only mode"));
187 }
188
189 // Add audit tags
190 event = self.add_audit_tags(event)?;
191
192 let event_id = self.client.send_event(event).await?;
193 Ok(event_id)
194 }
195
196 /// Query events, optionally filtered to this audit run
197 pub async fn query(&self, mut filter: Filter) -> Result<Vec<Event>> {
198 if self.config.mode == AuditMode::CI {
199 // In CI mode, only see our own audit events
200 filter = filter
201 .custom_tag(
202 SingleLetterTag::lowercase(Alphabet::A),
203 ["true"]
204 )
205 .custom_tag(
206 SingleLetterTag::lowercase(Alphabet::B),
207 [&self.config.run_id]
208 );
209 }
210 // In Production mode, see all events (no filter modification)
211
212 let events = self.client
213 .get_events_of(vec![filter], Some(Duration::from_secs(10)))
214 .await?;
215
216 Ok(events)
217 }
218
219 fn add_audit_tags(&self, event: Event) -> Result<Event> {
220 // This is tricky - we need to rebuild the event with new tags
221 // For now, we'll require events to be built through our builder
222
223 // TODO: Implement event tag injection
224 // This requires re-signing the event, which needs the private key
225
226 Ok(event)
227 }
228}
229
230/// Builder for audit events
231pub struct AuditEventBuilder {
232 builder: EventBuilder,
233 config: AuditConfig,
234}
235
236impl AuditEventBuilder {
237 pub fn new(kind: Kind, content: impl Into<String>, config: AuditConfig) -> Self {
238 Self {
239 builder: EventBuilder::new(kind, content, []),
240 config,
241 }
242 }
243
244 pub fn tag(mut self, tag: Tag) -> Self {
245 self.builder = self.builder.add_tags(vec![tag]);
246 self
247 }
248
249 pub fn tags(mut self, tags: Vec<Tag>) -> Self {
250 self.builder = self.builder.add_tags(tags);
251 self
252 }
253
254 pub async fn build(mut self, keys: &Keys) -> Result<Event> {
255 // Add audit tags
256 let audit_tags = vec![
257 Tag::custom(
258 TagKind::Custom(std::borrow::Cow::Borrowed("grasp-audit")),
259 vec!["true"]
260 ),
261 Tag::custom(
262 TagKind::Custom(std::borrow::Cow::Borrowed("audit-run-id")),
263 vec![&self.config.run_id]
264 ),
265 Tag::custom(
266 TagKind::Custom(std::borrow::Cow::Borrowed("audit-cleanup")),
267 vec![&self.config.cleanup_after.to_string()]
268 ),
269 ];
270
271 self.builder = self.builder.add_tags(audit_tags);
272
273 Ok(self.builder.to_event(keys).await?)
274 }
275}
276```
277
278## Test Structure with Isolation
279
280```rust
281// src/specs/nip01_smoke.rs
282
283use crate::*;
284
285pub struct Nip01SmokeTests;
286
287impl Nip01SmokeTests {
288 pub async fn run_all(client: &AuditClient) -> AuditResult {
289 let mut results = AuditResult::new("NIP-01 Smoke Tests");
290
291 // All tests run in parallel with isolated audit IDs
292 let tests = vec![
293 Self::test_websocket_connection(client),
294 Self::test_send_receive_event(client),
295 Self::test_create_subscription(client),
296 Self::test_close_subscription(client),
297 Self::test_reject_invalid_event(client),
298 Self::test_reject_invalid_event_id(client),
299 ];
300
301 let test_results = futures::future::join_all(tests).await;
302
303 for result in test_results {
304 results.add(result);
305 }
306
307 results
308 }
309
310 async fn test_websocket_connection(client: &AuditClient) -> TestResult {
311 TestResult::new(
312 "websocket_connection",
313 "NIP-01:basic",
314 "Can establish WebSocket connection to /",
315 )
316 .run(async {
317 // Test connection
318 client.client.connect().await;
319
320 // Verify connected
321 if !client.client.is_connected() {
322 return Err("Failed to connect to relay".into());
323 }
324
325 Ok(())
326 })
327 .await
328 }
329
330 async fn test_send_receive_event(client: &AuditClient) -> TestResult {
331 TestResult::new(
332 "send_receive_event",
333 "NIP-01:event-message",
334 "Can send EVENT and receive OK response",
335 )
336 .run(async {
337 let keys = Keys::generate();
338
339 // Create audit event
340 let event = AuditEventBuilder::new(
341 Kind::TextNote,
342 "Test event for smoke test",
343 client.config.clone(),
344 )
345 .build(&keys)
346 .await?;
347
348 // Send event
349 let event_id = client.send_event(event).await?;
350
351 // Query it back (in CI mode, only sees our events)
352 let filter = Filter::new()
353 .kind(Kind::TextNote)
354 .id(event_id);
355
356 let events = client.query(filter).await?;
357
358 if events.is_empty() {
359 return Err("Event not found after sending".into());
360 }
361
362 Ok(())
363 })
364 .await
365 }
366
367 // ... other tests
368}
369```
370
371## CLI Tool for Cleanup
372
373```rust
374// bin/grasp-audit.rs
375
376use clap::{Parser, Subcommand};
377use grasp_audit::*;
378
379#[derive(Parser)]
380#[command(name = "grasp-audit")]
381#[command(about = "GRASP audit and cleanup tool")]
382struct Cli {
383 #[command(subcommand)]
384 command: Commands,
385}
386
387#[derive(Subcommand)]
388enum Commands {
389 /// Run audit tests against a server
390 Audit {
391 /// Relay URL
392 #[arg(short, long)]
393 relay: String,
394
395 /// Mode: ci or production
396 #[arg(short, long, default_value = "ci")]
397 mode: String,
398
399 /// Spec to test (nip01-smoke, grasp-01-relay, all)
400 #[arg(short, long, default_value = "all")]
401 spec: String,
402 },
403
404 /// Clean up old audit events
405 Cleanup {
406 /// Relay URL
407 #[arg(short, long)]
408 relay: String,
409
410 /// Delete events older than this (e.g., "24h", "7d")
411 #[arg(short, long, default_value = "24h")]
412 older_than: String,
413
414 /// Dry run (don't actually delete)
415 #[arg(short, long)]
416 dry_run: bool,
417 },
418
419 /// List audit events
420 List {
421 /// Relay URL
422 #[arg(short, long)]
423 relay: String,
424
425 /// Filter by run ID
426 #[arg(short = 'i', long)]
427 run_id: Option<String>,
428 },
429}
430
431#[tokio::main]
432async fn main() -> Result<()> {
433 let cli = Cli::parse();
434
435 match cli.command {
436 Commands::Audit { relay, mode, spec } => {
437 let config = match mode.as_str() {
438 "ci" => AuditConfig::ci(),
439 "production" => AuditConfig::production(),
440 _ => return Err(anyhow!("Invalid mode: {}", mode)),
441 };
442
443 let client = AuditClient::new(&relay, config).await?;
444
445 println!("Running audit in {} mode...", mode);
446 println!("Audit run ID: {}", client.config.run_id);
447
448 let results = match spec.as_str() {
449 "nip01-smoke" => Nip01SmokeTests::run_all(&client).await,
450 "grasp-01-relay" => Grasp01RelayTests::run_all(&client).await,
451 "all" => {
452 let mut all = AuditResult::new("All Tests");
453 all.merge(Nip01SmokeTests::run_all(&client).await);
454 all.merge(Grasp01RelayTests::run_all(&client).await);
455 all
456 }
457 _ => return Err(anyhow!("Unknown spec: {}", spec)),
458 };
459
460 results.print_report();
461
462 if !results.all_passed() {
463 std::process::exit(1);
464 }
465 }
466
467 Commands::Cleanup { relay, older_than, dry_run } => {
468 println!("Cleaning up audit events from {}...", relay);
469
470 let duration = parse_duration(&older_than)?;
471 let cutoff = Timestamp::now() - duration;
472
473 let client = Client::new(&Keys::generate());
474 client.add_relay(&relay).await?;
475 client.connect().await;
476
477 // Query audit events
478 let filter = Filter::new()
479 .custom_tag(
480 SingleLetterTag::lowercase(Alphabet::A),
481 ["true"]
482 );
483
484 let events = client
485 .get_events_of(vec![filter], Some(Duration::from_secs(10)))
486 .await?;
487
488 let mut deleted = 0;
489
490 for event in events {
491 // Check cleanup timestamp
492 let cleanup_tag = event.tags.iter()
493 .find(|t| t.kind() == TagKind::Custom("audit-cleanup".into()));
494
495 if let Some(tag) = cleanup_tag {
496 if let Some(timestamp_str) = tag.content() {
497 let cleanup_time = Timestamp::from_str(timestamp_str)?;
498
499 if cleanup_time < cutoff {
500 if dry_run {
501 println!("Would delete: {} ({})",
502 event.id,
503 event.created_at
504 );
505 } else {
506 // TODO: Implement direct database deletion
507 // For now, we can't delete without NIP-09 deletion events
508 println!("Delete: {} ({})",
509 event.id,
510 event.created_at
511 );
512 }
513 deleted += 1;
514 }
515 }
516 }
517 }
518
519 println!("\n{} events cleaned up", deleted);
520 if dry_run {
521 println!("(dry run - no actual deletion)");
522 }
523 }
524
525 Commands::List { relay, run_id } => {
526 let client = Client::new(&Keys::generate());
527 client.add_relay(&relay).await?;
528 client.connect().await;
529
530 let mut filter = Filter::new()
531 .custom_tag(
532 SingleLetterTag::lowercase(Alphabet::A),
533 ["true"]
534 );
535
536 if let Some(id) = run_id {
537 filter = filter.custom_tag(
538 SingleLetterTag::lowercase(Alphabet::B),
539 [id]
540 );
541 }
542
543 let events = client
544 .get_events_of(vec![filter], Some(Duration::from_secs(10)))
545 .await?;
546
547 println!("Found {} audit events:\n", events.len());
548
549 for event in events {
550 let run_id = event.tags.iter()
551 .find(|t| t.kind() == TagKind::Custom("audit-run-id".into()))
552 .and_then(|t| t.content())
553 .unwrap_or("unknown");
554
555 println!("ID: {}", event.id);
556 println!(" Run: {}", run_id);
557 println!(" Kind: {}", event.kind);
558 println!(" Created: {}", event.created_at);
559 println!();
560 }
561 }
562 }
563
564 Ok(())
565}
566
567fn parse_duration(s: &str) -> Result<Duration> {
568 // Simple parser for "24h", "7d", etc.
569 let (num, unit) = s.split_at(s.len() - 1);
570 let num: u64 = num.parse()?;
571
572 let seconds = match unit {
573 "s" => num,
574 "m" => num * 60,
575 "h" => num * 3600,
576 "d" => num * 86400,
577 _ => return Err(anyhow!("Invalid duration unit: {}", unit)),
578 };
579
580 Ok(Duration::from_secs(seconds))
581}
582```
583
584## Usage Examples
585
586### CI/CD Mode (Isolated Tests)
587
588```bash
589# Run in CI - each run is isolated
590grasp-audit audit --relay ws://localhost:7000 --mode ci --spec all
591
592# Output:
593# Running audit in ci mode...
594# Audit run ID: ci-a1b2c3d4-e5f6-7890-abcd-ef1234567890
595#
596# NIP-01 Smoke Tests
597# ══════════════════════════════════════════════════════════
598# ✓ websocket_connection (NIP-01:basic)
599# ✓ send_receive_event (NIP-01:event-message)
600# ...
601# Results: 6/6 passed
602```
603
604### Production Audit Mode
605
606```bash
607# Audit production server (read-only by default)
608grasp-audit audit \
609 --relay wss://relay.example.com \
610 --mode production \
611 --spec grasp-01-relay
612
613# Output:
614# Running audit in production mode...
615# Audit run ID: prod-audit-1699027200
616#
617# GRASP-01: Relay Requirements
618# ══════════════════════════════════════════════════════════
619# ✓ accepts_repository_announcement (GRASP-01:9-10)
620# ✗ rejects_announcement_without_clone_tag (GRASP-01:12-13)
621# Error: Production relay accepted invalid announcement
622# ...
623```
624
625### Cleanup
626
627```bash
628# List all audit events
629grasp-audit list --relay ws://localhost:7000
630
631# Dry run cleanup
632grasp-audit cleanup \
633 --relay ws://localhost:7000 \
634 --older-than 24h \
635 --dry-run
636
637# Actual cleanup
638grasp-audit cleanup \
639 --relay ws://localhost:7000 \
640 --older-than 24h
641```
642
643## Parallel Development Plan
644
645### Week 1: Foundation (Both in Parallel)
646
647**grasp-audit:**
648- Day 1: Create crate structure
649- Day 2: Implement AuditClient with tag injection
650- Day 3: Implement 6 smoke tests
651- Day 4: Implement CLI tool skeleton
652- Day 5: Test isolation and cleanup
653
654**ngit-grasp:**
655- Day 1: Create project structure
656- Day 2: Set up nostr-relay-builder
657- Day 3: Basic relay serving at /
658- Day 4: NIP-11 document
659- Day 5: Event acceptance (no policy yet)
660
661### Week 2: Integration
662
663**grasp-audit:**
664- Day 1-2: Implement GRASP-01 relay tests
665- Day 3: Fixtures and builders
666- Day 4-5: Documentation and examples
667
668**ngit-grasp:**
669- Day 1-2: Implement GRASP policy (clone/relay tags)
670- Day 3: Related event acceptance
671- Day 4-5: Fix failing audit tests
672
673### Week 3-4: Iteration
674
675Run audit tests continuously, fix issues, iterate until all pass.
676
677## Next Steps
678
6791. ✅ Create `grasp-audit/` crate structure
6802. ✅ Implement AuditClient with tag injection
6813. ✅ Implement first smoke test
6824. ✅ Test it against a simple relay
6835. ✅ Report back with results
684
685Let me start with the implementation...
generated by cgit v1.2.3 (git 2.39.1) at 2026-07-19 15:15:12 +0000