From 5cd47079ee762125817612d2bf82a0bca07da3ad Mon Sep 17 00:00:00 2001 From: DanConwayDev Date: Wed, 5 Nov 2025 06:37:21 +0000 Subject: preparing to build grasp-audit against git-relay --- .../2025-11-04-authorization-flow-diagram.txt | 299 +++++++++ .../2025-11-04-git-http-backend-deep-dive.md | 714 ++++++++++++++++++++ .../2025-11-04-git-http-backend-validation.md | 247 +++++++ .../2025-11-04-ngit-grasp-implementation-plan.md | 590 +++++++++++++++++ docs/archive/2025-11-04-evening/INDEX.md | 314 +++++++++ .../2025-11-04-evening/NEXT_SESSION_START_HERE.md | 650 +++++++++++++++++++ docs/archive/2025-11-04-evening/STATUS.txt | 78 +++ .../2025-11-04-evening/architecture-diagram.md | 442 +++++++++++++ docs/archive/2025-11-04-evening/current_status.md | 443 +++++++++++++ .../2025-11-04-evening/implementation-checklist.md | 720 +++++++++++++++++++++ docs/archive/2025-11-04-evening/review-summary.md | 513 +++++++++++++++ .../archive/2025-11-04-evening/session-complete.md | 121 ++++ docs/archive/2025-11-04-evening/session-summary.md | 487 ++++++++++++++ docs/archive/2025-11-04-session-summary.md | 324 ++++------ docs/archive/2025-11-04-test-strategy-decision.md | 290 +++++++++ 15 files changed, 6031 insertions(+), 201 deletions(-) create mode 100644 docs/archive/2025-11-04-evening/2025-11-04-authorization-flow-diagram.txt create mode 100644 docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-deep-dive.md create mode 100644 docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-validation.md create mode 100644 docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md create mode 100644 docs/archive/2025-11-04-evening/INDEX.md create mode 100644 docs/archive/2025-11-04-evening/NEXT_SESSION_START_HERE.md create mode 100644 docs/archive/2025-11-04-evening/STATUS.txt create mode 100644 docs/archive/2025-11-04-evening/architecture-diagram.md create mode 100644 docs/archive/2025-11-04-evening/current_status.md create mode 100644 docs/archive/2025-11-04-evening/implementation-checklist.md create mode 100644 docs/archive/2025-11-04-evening/review-summary.md create mode 100644 docs/archive/2025-11-04-evening/session-complete.md create mode 100644 docs/archive/2025-11-04-evening/session-summary.md create mode 100644 docs/archive/2025-11-04-test-strategy-decision.md (limited to 'docs') diff --git a/docs/archive/2025-11-04-evening/2025-11-04-authorization-flow-diagram.txt b/docs/archive/2025-11-04-evening/2025-11-04-authorization-flow-diagram.txt new file mode 100644 index 0000000..10fb529 --- /dev/null +++ b/docs/archive/2025-11-04-evening/2025-11-04-authorization-flow-diagram.txt @@ -0,0 +1,299 @@ +╔═══════════════════════════════════════════════════════════════════════════════╗ +║ GIT PUSH AUTHORIZATION FLOW (INLINE) ║ +╚═══════════════════════════════════════════════════════════════════════════════╝ + +┌─────────────────────────────────────────────────────────────────────────────┐ +│ CLIENT: git push │ +└────────────────────────────────┬────────────────────────────────────────────┘ + │ + │ HTTP POST + │ /npub/repo.git/git-receive-pack + ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ ACTIX-WEB ROUTER (git-http-backend) │ +│ │ +│ Route: /{namespace}/{repo}/git-receive-pack │ +│ Handler: git_receive_pack() │ +└────────────────────────────────┬────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ STEP 1: RESOLVE REPOSITORY PATH │ +│ │ +│ GitConfig::rewrite("/npub/repo") → /data/git/npub/repo.git │ +└────────────────────────────────┬────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ STEP 2: VALIDATE REPOSITORY EXISTS │ +│ │ +│ ✓ Check HEAD exists │ +│ ✓ Check config exists │ +│ ✓ Check bare = true │ +│ │ +│ ❌ If not: Return 400 "Repository not found" │ +└────────────────────────────────┬────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ STEP 3: READ REQUEST BODY (MODIFIED) │ +│ │ +│ • Read full request body into memory │ +│ • Decode gzip if Content-Encoding: gzip │ +│ • Store in body_data: Vec │ +└────────────────────────────────┬────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ STEP 4: PARSE REF UPDATES (NEW!) │ +│ │ +│ parse_receive_pack_request(&body_data) → Vec │ +│ │ +│ Git Pack Protocol: │ +│ ┌──────────────────────────────────────────────────────────────┐ │ +│ │ 0000000000000000000000000000000000000000 a1b2c3d4e5f6... │ │ +│ │ refs/heads/main\0 report-status\n │ │ +│ │ │ │ +│ │ old_oid: 0000... (new branch) │ │ +│ │ new_oid: a1b2c3d4e5f6... │ │ +│ │ ref_name: refs/heads/main │ │ +│ └──────────────────────────────────────────────────────────────┘ │ +│ │ +│ Result: RefUpdate { │ +│ old_oid: "0000...", │ +│ new_oid: "a1b2c3d4e5f6...", │ +│ ref_name: "refs/heads/main" │ +│ } │ +└────────────────────────────────┬────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ STEP 5: VALIDATE AUTHORIZATION (NEW!) │ +│ │ +│ validator.validate_push(npub, identifier, &ref_updates).await │ +│ │ +│ ┌────────────────────────────────────────────────────────────┐ │ +│ │ PushValidator::validate_push() │ │ +│ │ │ │ +│ │ 1. Get latest state event from Nostr relay │ │ +│ │ • Query: kind=30618, d=identifier, author=npub │ │ +│ │ • Extract refs from state event │ │ +│ │ │ │ +│ │ 2. For each ref update: │ │ +│ │ • If refs/heads/* or refs/tags/*: │ │ +│ │ - Check state event has matching ref │ │ +│ │ - Check new_oid matches state event oid │ │ +│ │ - ❌ Reject if mismatch │ │ +│ │ • If refs/nostr/*: │ │ +│ │ - ✅ Always allow (PRs) │ │ +│ │ │ │ +│ │ 3. Get maintainers (recursive) │ │ +│ │ • Extract maintainers from announcement │ │ +│ │ • Recursively resolve maintainer sets │ │ +│ │ • Check if pusher is in maintainer list │ │ +│ │ • ❌ Reject if not maintainer │ │ +│ │ │ │ +│ │ 4. Return Ok(()) or Err(message) │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ +│ ❌ If validation fails: │ +│ Return 403 Forbidden │ +│ { │ +│ "error": "unauthorized", │ +│ "message": "Push rejected: refs/heads/main points to ..., state has..."│ +│ } │ +└────────────────────────────────┬────────────────────────────────────────────┘ + │ + │ ✅ AUTHORIZED + ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ STEP 6: SPAWN GIT RECEIVE-PACK (EXISTING) │ +│ │ +│ Command::new("git") │ +│ .arg("receive-pack") │ +│ .arg("--stateless-rpc") │ +│ .arg(".") │ +│ .current_dir(&repo_path) │ +│ .spawn() │ +│ │ +│ • Write body_data to git stdin │ +│ • Stream git stdout back to client │ +└────────────────────────────────┬────────────────────────────────────────────┘ + │ + │ Stream response + ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ CLIENT: git push success │ +└─────────────────────────────────────────────────────────────────────────────┘ + + +╔═══════════════════════════════════════════════════════════════════════════════╗ +║ COMPARISON: BEFORE vs AFTER ║ +╚═══════════════════════════════════════════════════════════════════════════════╝ + +┌─────────────────────────────────┬─────────────────────────────────────────┐ +│ BEFORE (git-http-backend) │ AFTER (our fork) │ +├─────────────────────────────────┼─────────────────────────────────────────┤ +│ 1. Resolve path │ 1. Resolve path │ +│ 2. Check bare repo │ 2. Check bare repo │ +│ 3. Read request body │ 3. Read request body │ +│ 4. Spawn git immediately ❌ │ 4. Parse ref updates ← NEW │ +│ 5. Stream response │ 5. Validate authorization ← NEW │ +│ │ 6. Spawn git (if authorized) │ +│ │ 7. Stream response │ +└─────────────────────────────────┴─────────────────────────────────────────┘ + +┌─────────────────────────────────┬─────────────────────────────────────────┐ +│ AUTHORIZATION │ METHOD │ +├─────────────────────────────────┼─────────────────────────────────────────┤ +│ ❌ None │ No validation │ +│ ⚠️ Git hooks (pre-receive) │ After git accepts push │ +│ ✅ Inline (our approach) │ Before git touches repository │ +└─────────────────────────────────┴─────────────────────────────────────────┘ + + +╔═══════════════════════════════════════════════════════════════════════════════╗ +║ KEY MODIFICATIONS NEEDED ║ +╚═══════════════════════════════════════════════════════════════════════════════╝ + +┌─────────────────────────────────────────────────────────────────────────────┐ +│ FILE: src/actix/git_receive_pack.rs │ +├─────────────────────────────────────────────────────────────────────────────┤ +│ │ +│ CHANGE 1: Add validator parameter │ +│ ──────────────────────────────────────────────────────────────────────── │ +│ pub async fn git_receive_pack( │ +│ request: HttpRequest, │ +│ mut payload: Payload, │ +│ service: web::Data, │ +│ + validator: web::Data, // ← ADD THIS │ +│ ) -> impl Responder { │ +│ │ +│ CHANGE 2: Parse ref updates after reading body │ +│ ──────────────────────────────────────────────────────────────────────── │ +│ // Read and decode body (existing) │ +│ let body_data = read_and_decode_body(&mut payload, &request).await?; │ +│ │ +│ + // Parse ref updates (NEW) │ +│ + let ref_updates = parse_receive_pack_request(&body_data)?; │ +│ │ +│ CHANGE 3: Validate before spawning git │ +│ ──────────────────────────────────────────────────────────────────────── │ +│ + // Extract repo info from path │ +│ + let (npub, identifier) = extract_repo_info(&request.uri().path())?; │ +│ + │ +│ + // Validate authorization │ +│ + if let Err(e) = validator.validate_push(&npub, &identifier, │ +│ + &ref_updates).await { │ +│ + return HttpResponse::Forbidden() │ +│ + .json(json!({ │ +│ + "error": "unauthorized", │ +│ + "message": e.to_string(), │ +│ + })); │ +│ + } │ +│ │ +│ // Spawn git (existing, unchanged) │ +│ let mut cmd = Command::new("git"); │ +│ // ... rest of existing code ... │ +└─────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────┐ +│ NEW FILE: src/git/protocol.rs │ +├─────────────────────────────────────────────────────────────────────────────┤ +│ │ +│ pub struct RefUpdate { │ +│ pub old_oid: String, │ +│ pub new_oid: String, │ +│ pub ref_name: String, │ +│ } │ +│ │ +│ pub fn parse_receive_pack_request(body: &[u8]) -> Result> { │ +│ // Parse git pack protocol │ +│ // Extract ref updates from pkt-line format │ +│ } │ +└─────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────┐ +│ NEW FILE: src/git/authorization.rs │ +├─────────────────────────────────────────────────────────────────────────────┤ +│ │ +│ pub struct PushValidator { │ +│ storage: Storage, │ +│ } │ +│ │ +│ impl PushValidator { │ +│ pub async fn validate_push( │ +│ &self, │ +│ npub: &str, │ +│ identifier: &str, │ +│ updates: &[RefUpdate], │ +│ ) -> Result<()> { │ +│ // Query Nostr relay for state event │ +│ // Validate each ref update │ +│ // Check maintainer permissions │ +│ } │ +│ } │ +└─────────────────────────────────────────────────────────────────────────────┘ + + +╔═══════════════════════════════════════════════════════════════════════════════╗ +║ BENEFITS OF INLINE AUTH ║ +╚═══════════════════════════════════════════════════════════════════════════════╝ + +✅ BETTER ERROR MESSAGES + • Return 403 with JSON error details + • Show exactly which ref failed validation + • Show expected vs. actual commit + • Better developer experience + +✅ SIMPLER DEPLOYMENT + • No git hooks to manage + • No symlinks or hook installation + • Single binary handles everything + • Easier to test + +✅ TIGHTER INTEGRATION + • Direct access to Nostr relay state + • Shared storage layer + • No IPC between components + • Atomic validation + +✅ EASIER TESTING + • Pure Rust unit tests + • Mock validator for testing + • No subprocess coordination + • Deterministic behavior + +✅ SECURITY + • Validation before git touches repo + • Can't bypass by manipulating hooks + • Centralized authorization logic + • Audit trail in application logs + + +╔═══════════════════════════════════════════════════════════════════════════════╗ +║ TIMELINE ║ +╚═══════════════════════════════════════════════════════════════════════════════╝ + +Week 1: Foundation +├─ Day 1-2: Fork git-http-backend, set up integration +├─ Day 3-4: Add git2, implement GitRepository +└─ Day 5: Add protocol parsing module + +Week 2: Authorization +├─ Day 1-2: Implement PushValidator +├─ Day 3-4: Modify git_receive_pack handler +└─ Day 5: Integration tests + +Week 3: Polish +├─ Day 1-2: Add CORS support +├─ Day 3-4: Error handling improvements +└─ Day 5: E2E tests with real git + +Week 4: Compliance +├─ Day 1-3: GRASP-01 compliance testing +├─ Day 4: Performance testing +└─ Day 5: Documentation + + +Status: ✅ Analysis complete, ready to implement diff --git a/docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-deep-dive.md b/docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-deep-dive.md new file mode 100644 index 0000000..26d0526 --- /dev/null +++ b/docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-deep-dive.md @@ -0,0 +1,714 @@ +**ARCHIVED: 2025-11-04** +**Reason:** Analysis complete, crate validated +**Outcome:** Confirmed suitable for use (with fork for authorization) + +--- + +# git-http-backend Crate Deep Dive + +**Date:** 2025-11-04 +**Status:** ✅ ARCHIVED - Analysis Complete +**Purpose:** Validate the recommendation in `work/current_status.md` regarding git-http-backend crate + +--- + +## Executive Summary + +**Recommendation Status:** ✅ **VALIDATED WITH CAVEATS** + +The `git-http-backend` crate (v0.1.3) is a **good foundation** but requires significant customization for our inline authorization needs. The hybrid approach recommended in `current_status.md` is sound, but we'll need to: + +1. **Fork or vendor** the crate for customization +2. **Add interception points** for authorization +3. **Enhance error handling** for better push rejection messages +4. **Add CORS support** (missing from current implementation) + +--- + +## Crate Overview + +### Basic Info +- **Name:** `git-http-backend` +- **Version:** 0.1.3 +- **Author:** lazhenyi +- **License:** MIT +- **Repository:** https://github.com/lazhenyi/git-http-backend +- **Documentation:** https://docs.rs/git-http-backend/0.1.3 + +### Dependencies +```toml +tokio = { version = "1", features = ["sync","macros","rt", "rt-multi-thread","net"] } +actix-web = { version = "4.9.0", features = ["default"] } +actix-files = { version = "0.6.6", features = ["actix-server"] } +futures-util = { version = "0.3.31", features = ["futures-channel"] } +flate2 = "1.0.35" # Gzip compression +async-stream = "0.3.6" # Streaming responses +async-trait = "0.1.83" # Async trait support +``` + +**Good news:** Already uses actix-web 4.9.0 (same as we plan to use) + +--- + +## Architecture Analysis + +### Core Design + +The crate provides: + +1. **GitConfig Trait** - Path rewriting abstraction +2. **Actix Router** - Pre-configured routes for Git Smart HTTP +3. **Protocol Handlers** - Upload-pack, receive-pack, info/refs +4. **System Git Integration** - Spawns `git` subprocess + +### URL Structure + +``` +/{namespace}/{repo}/info/refs?service=git-upload-pack +/{namespace}/{repo}/git-upload-pack +/{namespace}/{repo}/git-receive-pack +/{namespace}/{repo}/HEAD +/{namespace}/{repo}/objects/info/packs +/{namespace}/{repo}/objects/pack/{pack} +``` + +**Perfect match** for our `/{npub}/{identifier}.git/` structure! + +### Request Flow + +``` +HTTP Request + ↓ +Actix Router → Handler Function + ↓ +GitConfig::rewrite() → Path resolution + ↓ +Spawn git subprocess (upload-pack/receive-pack) + ↓ +Stream response back to client +``` + +--- + +## Key Handlers Analysis + +### 1. info/refs Handler (refs.rs) + +**Purpose:** Advertise repository refs (clone/fetch discovery) + +**Flow:** +1. Parse `service` query param (upload-pack or receive-pack) +2. Resolve repository path via `GitConfig::rewrite()` +3. Spawn `git upload-pack --advertise-refs --stateless-rpc .` +4. Return with proper content-type header + +**Code:** +```rust +pub async fn info_refs(request: HttpRequest, service: web::Data) -> impl Responder { + let uri = request.uri(); + let path = uri.path().to_string().replace("/info/refs", ""); + let path = service.rewrite(path).await; + + // Parse service from query + let service = query.split('=').map(|x| x.to_string()).collect::>()[1].clone(); + + // Spawn git + let mut cmd = Command::new("git"); + cmd.arg(service_name.clone()); + cmd.arg("--stateless-rpc"); + cmd.arg("--advertise-refs"); + cmd.arg("."); + cmd.current_dir(path); + + // Return response with proper headers + resp.append_header(("Content-Type", format!("application/x-git-{}-advertisement", service_name))); + resp.append_header(("Cache-Control", "no-cache, max-age=0, must-revalidate")); +} +``` + +**Good:** +- ✅ Proper content-type headers +- ✅ Cache control headers +- ✅ Git protocol version support (Git-Protocol header) + +**Issues:** +- ❌ No CORS headers +- ❌ No error handling for missing repos +- ❌ Query parsing is fragile (will panic on malformed input) + +### 2. git-upload-pack Handler (git_upload_pack.rs) + +**Purpose:** Handle clone/fetch operations (read-only) + +**Flow:** +1. Resolve repository path +2. Read request body (may be gzipped) +3. Spawn `git upload-pack --stateless-rpc .` +4. Stream response back + +**Code:** +```rust +pub async fn git_upload_pack( + request: HttpRequest, + mut payload: Payload, + service: web::Data, +) -> impl Responder { + // Resolve path + let path = service.rewrite(path).await; + + // Spawn git + let mut cmd = Command::new("git"); + cmd.arg("upload-pack"); + cmd.arg("--stateless-rpc"); + cmd.arg("."); + cmd.current_dir(path); + + let mut span = cmd.spawn()?; + let mut stdin = span.stdin.take().unwrap(); + let mut stdout = span.stdout.take().unwrap(); + + // Read request body + let mut bytes = web::BytesMut::new(); + while let Some(chunk) = payload.next().await { + bytes.extend_from_slice(&data); + } + + // Handle gzip + let body_data = match encoding { + Some("gzip") => decode_gzip(bytes), + _ => bytes.to_vec(), + }; + + // Write to git stdin + stdin.write_all(&body_data)?; + drop(stdin); + + // Stream response + let body_stream = actix_web::body::BodyStream::new(async_stream::stream! { + let mut buffer = [0; 8192]; + loop { + match stdout.read(&mut buffer) { + Ok(0) => break, + Ok(n) => yield Ok(web::Bytes::copy_from_slice(&buffer[..n])), + Err(e) => break, + } + } + }); + resp.body(body_stream) +} +``` + +**Good:** +- ✅ Handles gzip compression +- ✅ Streams response (efficient for large repos) +- ✅ Proper content-type headers + +**Issues:** +- ❌ No CORS headers +- ❌ No repository existence check +- ❌ Error handling uses eprintln! (not tracing) + +**For our use:** Upload-pack is read-only, so we can use as-is (just add CORS) + +### 3. git-receive-pack Handler (git_receive_pack.rs) ⚠️ + +**Purpose:** Handle push operations (write) + +**This is the critical handler for inline authorization!** + +**Current Flow:** +1. Resolve repository path +2. **Check if bare repository** (good!) +3. Read request body (may be gzipped) +4. Spawn `git receive-pack --stateless-rpc .` +5. Stream response back + +**Code:** +```rust +pub async fn git_receive_pack( + request: HttpRequest, + mut payload: Payload, + service: web::Data, +) -> impl Responder { + let path = service.rewrite(path).await; + + // Check repository exists + if !path.join("HEAD").exists() || !path.join("config").exists() { + return HttpResponse::BadRequest().body("Repository not found or invalid."); + } + + // Check if bare + let is_bare_repo = match std::fs::read_to_string(path.join("config")) { + Ok(config) => config.contains("bare = true"), + Err(_) => false, + }; + if !is_bare_repo { + return HttpResponse::BadRequest().body("Push operation requires a bare repository."); + } + + // Spawn git receive-pack + let mut cmd = Command::new("git"); + cmd.arg("receive-pack"); + cmd.arg("--stateless-rpc"); + cmd.arg("."); + cmd.current_dir(&path); + + let mut git_process = cmd.spawn()?; + let mut stdin = git_process.stdin.take().unwrap(); + let mut stdout = git_process.stdout.take().unwrap(); + + // Read request body + let mut bytes = web::BytesMut::new(); + while let Some(chunk) = payload.next().await { + bytes.extend_from_slice(&data); + } + + // Decode if gzipped + let body_data = match encoding { + Some(encoding) if encoding.contains("gzip") => decode_gzip(bytes), + _ => bytes.to_vec(), + }; + + // Write to git stdin + stdin.write_all(&body_data)?; + drop(stdin); + + // Stream response + let body_stream = /* stream stdout */; + resp.body(body_stream) +} +``` + +**Good:** +- ✅ Validates repository exists +- ✅ Validates bare repository +- ✅ Handles gzip compression +- ✅ Streams response + +**Critical Issues for Our Use:** +- ❌ **No authorization hook!** Spawns git immediately +- ❌ **No way to inspect push data** before spawning git +- ❌ **No CORS headers** +- ❌ **Can't reject unauthorized pushes** with custom error + +**This is where we need customization!** + +--- + +## Customization Requirements + +### 1. Authorization Interception Point + +**Need to add BEFORE spawning git:** + +```rust +pub async fn git_receive_pack( + request: HttpRequest, + mut payload: Payload, + service: web::Data, + validator: web::Data, // ← ADD THIS +) -> impl Responder { + let path = service.rewrite(path).await; + + // Existing checks... + + // Read request body + let body_data = read_and_decode_body(&mut payload, &request).await?; + + // ← ADD AUTHORIZATION HERE + let ref_updates = parse_receive_pack_request(&body_data)?; + + // Extract npub and identifier from path + let (npub, identifier) = extract_repo_info(&request.uri().path())?; + + // Validate against Nostr state + if let Err(e) = validator.validate_push(&npub, &identifier, &ref_updates).await { + return HttpResponse::Forbidden() + .json(json!({ + "error": "unauthorized", + "message": e.to_string(), + "ref_updates": ref_updates, + })); + } + + // Only spawn git if authorized + let mut cmd = Command::new("git"); + // ... rest of existing code +} +``` + +### 2. Parse Git Protocol + +**Need to add protocol parsing:** + +```rust +// src/git/protocol.rs + +pub struct RefUpdate { + pub old_oid: String, + pub new_oid: String, + pub ref_name: String, +} + +pub fn parse_receive_pack_request(body: &[u8]) -> Result> { + // Parse git pack protocol + // Format: \0\n + // Example: 0000000000000000000000000000000000000000 a1b2c3d4... refs/heads/main\0 report-status\n + + let mut updates = Vec::new(); + let lines = body.split(|&b| b == b'\n'); + + for line in lines { + if line.is_empty() { + continue; + } + + // Parse pkt-line format + // First 4 bytes are hex length + let pkt_len = parse_pkt_len(&line[0..4])?; + if pkt_len == 0 { + continue; // flush packet + } + + let data = &line[4..pkt_len]; + let parts: Vec<&[u8]> = data.splitn(3, |&b| b == b' ').collect(); + + if parts.len() >= 3 { + let old_oid = String::from_utf8_lossy(parts[0]).to_string(); + let new_oid = String::from_utf8_lossy(parts[1]).to_string(); + + // Ref name may have capabilities after \0 + let ref_data = parts[2]; + let ref_name = if let Some(null_pos) = ref_data.iter().position(|&b| b == b'\0') { + String::from_utf8_lossy(&ref_data[..null_pos]).to_string() + } else { + String::from_utf8_lossy(ref_data).to_string() + }; + + updates.push(RefUpdate { + old_oid, + new_oid, + ref_name, + }); + } + } + + Ok(updates) +} +``` + +**Note:** Git pack protocol is complex. We may want to use a library for this: +- `git2` crate has protocol parsing +- Or we can implement minimal parsing for our needs + +### 3. Add CORS Support + +**Need to add to all handlers:** + +```rust +// Add CORS middleware or headers to all responses +resp.append_header(("Access-Control-Allow-Origin", "*")); +resp.append_header(("Access-Control-Allow-Methods", "GET, POST, OPTIONS")); +resp.append_header(("Access-Control-Allow-Headers", "Content-Type, Git-Protocol")); +``` + +### 4. Better Error Handling + +**Replace eprintln! with tracing:** + +```rust +use tracing::{error, info, debug}; + +// Instead of: +eprintln!("Error running command: {}", e); + +// Use: +error!(error = ?e, "Failed to spawn git process"); +``` + +--- + +## Integration Strategy + +### Option A: Fork the Crate ✅ RECOMMENDED + +**Pros:** +- Full control over authorization logic +- Can add CORS, error handling, protocol parsing +- Can publish as `ngit-grasp-git-http-backend` +- Keep upstream changes visible + +**Cons:** +- Need to maintain fork +- Diverges from upstream + +**Implementation:** +1. Fork https://github.com/lazhenyi/git-http-backend +2. Add to our workspace as git submodule or copy +3. Modify `git_receive_pack.rs` to add authorization +4. Add protocol parsing module +5. Add CORS support +6. Improve error handling + +### Option B: Vendor the Code + +**Pros:** +- Complete control +- No external dependency +- Can heavily customize + +**Cons:** +- Lose upstream updates +- More code to maintain + +**Implementation:** +1. Copy source into `src/git/http_backend/` +2. Modify as needed +3. No external dependency + +### Option C: Wrap the Crate + +**Pros:** +- Keep upstream crate +- Add authorization via middleware + +**Cons:** +- ❌ **Can't intercept before git spawns!** +- Would need to parse response, too late +- Complex to inject validator + +**Not recommended** - can't achieve inline authorization + +--- + +## Recommended Approach + +### Use Forked git-http-backend + git2 + System Git + +**Architecture:** + +``` +HTTP Request + ↓ +Actix Router (from forked git-http-backend) + ↓ +Custom GitConfig Implementation + ↓ +git_receive_pack Handler (MODIFIED) + ↓ +┌─────────────────────────────────┐ +│ 1. Read request body │ +│ 2. Parse ref updates (protocol) │ ← ADD THIS +│ 3. Validate via PushValidator │ ← ADD THIS +│ ├─ Query Nostr relay │ +│ ├─ Check state event │ +│ └─ Validate maintainers │ +│ 4. If authorized: │ +│ └─ Spawn git receive-pack │ ← EXISTING +│ 5. If unauthorized: │ +│ └─ Return 403 with error │ ← ADD THIS +└─────────────────────────────────┘ + ↓ +Stream response to client +``` + +**Dependencies:** + +```toml +[dependencies] +# Fork of git-http-backend (or vendored code) +git-http-backend = { git = "https://github.com/our-org/git-http-backend", branch = "ngit-grasp" } + +# Or vendor it: +# (no dependency, code in src/git/http_backend/) + +# Git operations +git2 = "0.20" # For repository management, ref queries + +# Already have: +actix-web = "4.9" +tokio = { version = "1", features = ["full"] } +nostr-sdk = "0.43" +``` + +**Implementation Plan:** + +1. **Phase 1: Fork & Setup** + - Fork git-http-backend + - Add to our project (git submodule or copy) + - Verify existing functionality works + +2. **Phase 2: Protocol Parsing** + - Add `src/git/protocol.rs` + - Implement `parse_receive_pack_request()` + - Unit tests for protocol parsing + +3. **Phase 3: Authorization Integration** + - Modify `git_receive_pack.rs` + - Add `PushValidator` parameter + - Call validator before spawning git + - Return 403 on unauthorized + +4. **Phase 4: CORS & Polish** + - Add CORS headers to all handlers + - Improve error messages + - Add tracing instead of eprintln! + +5. **Phase 5: Testing** + - Unit tests for authorization + - Integration tests with real git + - GRASP-01 compliance tests + +--- + +## Validation of current_status.md Recommendations + +### Hybrid Approach ✅ VALIDATED + +**Original recommendation:** +> 1. **git-http-backend** - HTTP protocol handling +> 2. **git2-rs** - Repository management, ref validation +> 3. **System git** - Actual pack operations (upload-pack/receive-pack) + +**Analysis:** +- ✅ **git-http-backend** - Good foundation, needs customization +- ✅ **git2** - Perfect for repo management (init, refs, validation) +- ✅ **System git** - Proven pack protocol implementation + +**Verdict:** Sound approach, but need to fork/vendor git-http-backend + +### Tool Selection ✅ CORRECT + +**Original analysis:** +- git2 for repository management ✅ +- System git for pack operations ✅ +- git-http-backend for HTTP layer ✅ (with modifications) + +**Additional findings:** +- Need protocol parsing (can use git2 or implement minimal) +- Need CORS support (add to fork) +- Need better error handling (add to fork) + +### Inline Authorization ✅ ACHIEVABLE + +**Original goal:** +> We intercept the `git-receive-pack` operation before spawning the Git process + +**Analysis:** +- ✅ Possible by modifying `git_receive_pack.rs` +- ✅ Can parse request body before spawning git +- ✅ Can return 403 before git touches repository + +**Requirement:** +- Must fork or vendor git-http-backend +- Can't achieve with unmodified crate + +--- + +## Updated Implementation Plan + +### Week 1: Foundation (UPDATED) + +1. ✅ Add git2 dependency +2. **Fork git-http-backend** (NEW) +3. **Add protocol parsing** (NEW) +4. Implement GitRepository (Phase 1) +5. Write unit tests for repository operations +6. Test repository creation from announcements + +### Week 2: Protocol & Authorization + +1. Implement protocol parsing (Phase 2) +2. Implement authorization logic (Phase 3) +3. **Modify git_receive_pack handler** (NEW) +4. Write unit tests for both +5. Integration tests for validation + +### Week 3: HTTP & Integration + +1. **Add CORS support to fork** (NEW) +2. Implement HTTP handlers (Phase 4) +3. Integrate with Nostr events (Phase 5) +4. Integration tests for full flow +5. Error handling improvements + +### Week 4: E2E & Polish + +1. E2E tests with real git (Phase 6) +2. Performance testing +3. GRASP-01 compliance testing +4. Documentation and examples + +--- + +## Risks & Mitigations + +### Risk 1: Fork Maintenance + +**Risk:** Fork diverges from upstream, miss updates + +**Mitigation:** +- Keep fork minimal (only modify git_receive_pack.rs) +- Document all changes clearly +- Consider upstreaming authorization hooks +- Monitor upstream for security fixes + +### Risk 2: Protocol Parsing Complexity + +**Risk:** Git pack protocol is complex, may miss edge cases + +**Mitigation:** +- Use git2 for protocol parsing if available +- Implement minimal parsing (just ref updates) +- Extensive testing with real git clients +- Refer to Git protocol documentation + +### Risk 3: Performance + +**Risk:** Authorization adds latency to push operations + +**Mitigation:** +- Keep validation logic fast (< 100ms target) +- Cache state events in memory +- Async validation (don't block) +- Profile and optimize + +--- + +## Conclusion + +### Summary + +The **hybrid approach** recommended in `current_status.md` is **sound and validated**, with these adjustments: + +1. **Fork or vendor git-http-backend** - Can't use unmodified crate +2. **Add protocol parsing** - Need to parse ref updates from request +3. **Modify git_receive_pack handler** - Add authorization before spawning git +4. **Add CORS support** - Missing from current implementation +5. **Improve error handling** - Better messages for push rejections + +### Next Steps + +1. ✅ **Review this analysis** - Confirm approach +2. **Fork git-http-backend** - Set up fork/vendor +3. **Start Phase 1** - Add git2, implement GitRepository +4. **Add protocol parsing** - Parse ref updates from pack protocol +5. **Modify receive-pack handler** - Add authorization logic + +### Questions for Review + +1. **Fork vs. Vendor?** Fork allows upstream tracking, vendor gives full control +2. **Protocol parsing?** Use git2 or implement minimal parser? +3. **CORS scope?** Support all origins or restrict? +4. **Error detail?** How much info to expose in 403 responses? +5. **Performance target?** Is < 100ms for auth validation reasonable? + +--- + +**Status:** ✅ Analysis complete, ready to proceed with implementation + +**Recommendation:** Fork git-http-backend, add authorization to git_receive_pack, use git2 for repo management + +--- + +*Analysis Date: November 4, 2025* diff --git a/docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-validation.md b/docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-validation.md new file mode 100644 index 0000000..d383b9f --- /dev/null +++ b/docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-validation.md @@ -0,0 +1,247 @@ +**ARCHIVED: 2025-11-04** +**Reason:** Analysis complete, validated hybrid approach +**Outcome:** Will use git-http-backend (forked) + git2 + system git + +--- + +# Analysis Summary: git-http-backend Validation + +**Date:** 2025-11-04 +**Status:** ✅ ARCHIVED - Analysis Complete + +--- + +## TL;DR + +✅ **VALIDATED:** The hybrid approach in `current_status.md` is sound +⚠️ **CAVEAT:** Must fork/vendor `git-http-backend` crate for inline authorization +✅ **READY:** Can proceed with implementation + +--- + +## Key Findings + +### 1. git-http-backend Crate (v0.1.3) + +**What it provides:** +- ✅ Actix-web based Git Smart HTTP handlers +- ✅ Upload-pack (clone/fetch) - works as-is +- ✅ Receive-pack (push) - **needs modification** +- ✅ Info/refs advertisement +- ✅ Gzip compression support +- ✅ Streaming responses + +**What it lacks:** +- ❌ Authorization hooks (spawns git immediately) +- ❌ CORS headers (needed for web clients) +- ❌ Protocol parsing (can't inspect push data) +- ❌ Proper error handling (uses eprintln!) + +### 2. Critical Handler: git_receive_pack + +**Current flow:** +``` +Request → Validate bare repo → Spawn git → Stream response +``` + +**What we need:** +``` +Request → Validate bare repo → Parse ref updates → Validate against Nostr state → Spawn git (if authorized) → Stream response + ↑ + ADD THIS +``` + +**Can't achieve with unmodified crate!** + +### 3. Recommended Solution + +**Fork the crate and modify `git_receive_pack.rs`:** + +```rust +pub async fn git_receive_pack( + request: HttpRequest, + mut payload: Payload, + service: web::Data, + validator: web::Data, // ← ADD +) -> impl Responder { + // ... existing path resolution and bare check ... + + // Read request body + let body_data = read_and_decode_body(&mut payload, &request).await?; + + // ← ADD: Parse ref updates + let ref_updates = parse_receive_pack_request(&body_data)?; + + // ← ADD: Validate authorization + let (npub, identifier) = extract_repo_info(&request.uri().path())?; + if let Err(e) = validator.validate_push(&npub, &identifier, &ref_updates).await { + return HttpResponse::Forbidden() + .json(json!({ + "error": "unauthorized", + "message": e.to_string(), + })); + } + + // Only spawn git if authorized + let mut cmd = Command::new("git"); + cmd.arg("receive-pack"); + // ... rest of existing code ... +} +``` + +--- + +## Updated Implementation Plan + +### Phase 0: Setup (NEW) +1. Fork git-http-backend repository +2. Add as git submodule or vendor code +3. Verify existing functionality works +4. Add to Cargo.toml + +### Phase 1: Foundation +1. Add git2 dependency +2. Implement GitRepository (repo management) +3. Add protocol parsing module +4. Unit tests for both + +### Phase 2: Authorization +1. Modify git_receive_pack handler +2. Implement PushValidator +3. Integration tests for validation +4. Test unauthorized rejection + +### Phase 3: Polish +1. Add CORS headers to all handlers +2. Improve error messages +3. Add tracing instead of eprintln! +4. E2E tests with real git + +--- + +## Dependencies + +```toml +[dependencies] +# Forked git-http-backend with authorization support +git-http-backend = { git = "https://github.com/our-org/git-http-backend", branch = "ngit-grasp" } + +# Git repository management +git2 = "0.20" + +# Already have: +actix-web = "4.9" +tokio = { version = "1", features = ["full"] } +nostr-sdk = "0.43" +``` + +--- + +## Validation of current_status.md + +### ✅ Hybrid Approach - CONFIRMED +- git-http-backend for HTTP layer ✅ (with fork) +- git2 for repository management ✅ +- System git for pack operations ✅ + +### ✅ Inline Authorization - ACHIEVABLE +- Can intercept before spawning git ✅ +- Can parse ref updates ✅ +- Can validate against Nostr state ✅ +- Can return 403 with error message ✅ + +### ⚠️ Additional Requirements +- Must fork/vendor git-http-backend +- Must implement protocol parsing +- Must add CORS support +- Must improve error handling + +--- + +## Risks & Mitigations + +| Risk | Impact | Mitigation | +|------|--------|------------| +| Fork maintenance | Medium | Keep changes minimal, document well | +| Protocol parsing complexity | Medium | Use git2 or implement minimal parser | +| Performance overhead | Low | Keep validation fast (< 100ms), cache state | +| Missing edge cases | Medium | Extensive testing with real git clients | + +--- + +## Next Steps + +1. **Decision:** Fork vs. vendor git-http-backend? + - Fork: Keep upstream tracking, easier updates + - Vendor: Full control, no external dependency + - **Recommendation:** Fork (easier to contribute back) + +2. **Start Phase 0:** Set up fork + - Fork https://github.com/lazhenyi/git-http-backend + - Create branch `ngit-grasp` + - Add as git submodule + +3. **Start Phase 1:** Add git2, implement GitRepository + - Write tests first (TDD) + - Focus on bare repo creation, ref management + +4. **Add protocol parsing:** Parse ref updates from pack protocol + - Research: Can git2 help? + - Or implement minimal parser + - Unit tests for various push scenarios + +5. **Modify receive-pack:** Add authorization logic + - Integration tests for validation + - Test rejection scenarios + +--- + +## Questions for Review + +1. **Fork vs. Vendor?** + - Recommendation: Fork (can contribute back, easier updates) + +2. **Protocol parsing?** + - Option A: Use git2 if it provides parsing + - Option B: Implement minimal parser (just ref updates) + - Recommendation: Research git2 first, then decide + +3. **CORS policy?** + - Allow all origins (`*`) for now? + - Or restrict to configured domains? + - Recommendation: Start with `*`, make configurable later + +4. **Error detail?** + - How much info in 403 responses? + - Show ref updates that failed? + - Show expected vs. actual commit? + - Recommendation: Detailed errors for better DX + +5. **Performance target?** + - < 100ms for auth validation? + - Cache state events? + - Recommendation: Yes to both + +--- + +## Conclusion + +✅ **The hybrid approach is validated and sound** + +⚠️ **Must fork git-http-backend for inline authorization** + +✅ **Ready to proceed with implementation** + +**Confidence Level:** High (95%) + +The crate provides exactly what we need as a foundation. The modifications required are straightforward and well-scoped. The main work is: +1. Fork setup +2. Protocol parsing +3. Authorization integration +4. CORS and polish + +All achievable within the 4-week timeline. + +--- + +**Next:** Review this analysis, make fork vs. vendor decision, then start Phase 0. diff --git a/docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md b/docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md new file mode 100644 index 0000000..aaca16b --- /dev/null +++ b/docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md @@ -0,0 +1,590 @@ +**ARCHIVED: 2025-11-04** +**Reason:** Decided to validate grasp-audit against ngit-relay first +**See:** docs/archive/2025-11-04-test-strategy-decision.md for rationale + +--- + +# TDD Plan for GRASP-01 Git Backend + +**Date:** 2025-11-04 +**Status:** ARCHIVED - Superseded by test-first approach +**Goal:** Implement Git Smart HTTP backend with inline authorization using TDD + +--- + +## Current State + +### What We Have +- ✅ NIP-01 compliant Nostr relay (working, tested) +- ✅ NIP-34 event handling (announcements accepted) +- ✅ Storage layer (in-memory + disk paths configured) +- ✅ Test infrastructure (integration tests with auto relay management) +- ✅ grasp-audit compliance testing library + +### What We Need +- ❌ Git Smart HTTP protocol handler +- ❌ Git repository management (init, receive-pack, upload-pack) +- ❌ Push authorization (validate against Nostr state events) +- ❌ Integration with existing Nostr relay + +--- + +## Tool Selection Analysis + +### Option 1: git2-rs (libgit2 bindings) +**Pros:** +- ✅ Pure Rust bindings to battle-tested libgit2 +- ✅ Full Git functionality (init, push, pull, refs, objects) +- ✅ Thread-safe, well-maintained +- ✅ Used by cargo, widely deployed +- ✅ Can intercept and validate operations programmatically + +**Cons:** +- ❌ Requires libgit2 system dependency +- ❌ Higher-level API - may be overkill for our needs +- ❌ Harder to intercept low-level protocol for inline auth + +**Use Cases:** +- Repository initialization +- Reading/writing refs +- Object storage queries +- Validation of commits/trees + +### Option 2: Standard git in subprocess +**Pros:** +- ✅ Uses system git (already available) +- ✅ No additional dependencies +- ✅ Battle-tested Git implementation +- ✅ Easy to spawn for upload-pack/receive-pack + +**Cons:** +- ❌ Harder to intercept for inline authorization +- ❌ Must parse git protocol to validate pushes +- ❌ Subprocess overhead +- ❌ Complex error handling + +**Use Cases:** +- git-upload-pack (clone, fetch) +- git-receive-pack (push) - but need to intercept + +### Option 3: git-http-backend crate +**Pros:** +- ✅ Purpose-built for Git Smart HTTP +- ✅ Handles protocol parsing +- ✅ Works with system git +- ✅ Can intercept receive-pack before spawning git + +**Cons:** +- ❌ Less mature (but we're already planning to use it per README) +- ❌ Still need to parse pack protocol for validation + +**Use Cases:** +- HTTP endpoint handling +- Protocol negotiation +- Spawning git processes + +### Option 4: Hybrid Approach (RECOMMENDED) +**Combination:** +1. **git-http-backend** - HTTP protocol handling +2. **git2-rs** - Repository management, ref validation +3. **System git** - Actual pack operations (upload-pack/receive-pack) + +**Why Hybrid:** +- git-http-backend handles HTTP → Git protocol translation +- git2 for safe repository operations (init, refs, validation) +- System git for pack operations (proven, fast) +- We intercept at the HTTP layer before spawning git + +**Architecture:** +``` +HTTP Request → git-http-backend → Our Auth Layer → git2/system git + ↓ + Nostr Relay + (state validation) +``` + +--- + +## Recommended Approach: Hybrid + +### Dependencies to Add +```toml +[dependencies] +# Git operations +git2 = "0.20" # Repository management, refs +# git-http-backend - TBD (research if available, or implement minimal) + +[dev-dependencies] +tempfile = "3.8" # Temporary repos for testing +``` + +### Why This Works + +1. **git2 for Repository Management:** + - Initialize bare repos when announcements arrive + - Read/write refs safely + - Query repository state + - Validate commits exist + +2. **System git for Pack Operations:** + - Spawn `git-upload-pack` for clones/fetches (read-only, safe) + - Spawn `git-receive-pack` ONLY after auth passes + - Leverage proven pack protocol implementation + +3. **Inline Authorization:** + - Parse HTTP request to extract ref updates + - Query Nostr relay for latest state event + - Validate push matches state + - Only spawn git-receive-pack if authorized + +--- + +## TDD Implementation Plan + +### Phase 1: Repository Management (git2) +**Goal:** Create and manage bare Git repositories + +**Tests:** +1. ✅ Create bare repository when announcement received +2. ✅ Initialize with proper config (bare, shared) +3. ✅ Set HEAD from state event +4. ✅ Read refs from repository +5. ✅ Write refs to repository +6. ✅ Query if commit exists in repository + +**Implementation:** +```rust +// src/git/repository.rs + +pub struct GitRepository { + path: PathBuf, + repo: git2::Repository, +} + +impl GitRepository { + pub fn init_bare(path: PathBuf) -> Result; + pub fn set_head(ref_name: &str) -> Result<()>; + pub fn get_ref(&self, name: &str) -> Result>; + pub fn set_ref(&self, name: &str, oid: &str) -> Result<()>; + pub fn has_commit(&self, oid: &str) -> Result; +} +``` + +**Test Example:** +```rust +#[test] +fn test_init_bare_repository() { + let temp = TempDir::new().unwrap(); + let repo = GitRepository::init_bare(temp.path().to_path_buf()).unwrap(); + + assert!(temp.path().join("HEAD").exists()); + assert!(temp.path().join("config").exists()); + assert!(temp.path().join("objects").exists()); + assert!(temp.path().join("refs").exists()); +} +``` + +### Phase 2: Git Protocol Parsing +**Goal:** Parse Git Smart HTTP protocol for authorization + +**Tests:** +1. ✅ Parse info/refs request +2. ✅ Parse upload-pack request (clone/fetch) +3. ✅ Parse receive-pack request (push) +4. ✅ Extract ref updates from receive-pack +5. ✅ Extract capabilities from request + +**Implementation:** +```rust +// src/git/protocol.rs + +pub struct RefUpdate { + pub old_oid: String, + pub new_oid: String, + pub ref_name: String, +} + +pub fn parse_receive_pack_request(body: &[u8]) -> Result>; +pub fn parse_capabilities(body: &[u8]) -> Result>; +``` + +**Test Example:** +```rust +#[test] +fn test_parse_receive_pack_single_ref() { + let body = b"0000000000000000000000000000000000000000 \ + a1b2c3d4e5f6789012345678901234567890abcd \ + refs/heads/main\0 report-status\n"; + + let updates = parse_receive_pack_request(body).unwrap(); + assert_eq!(updates.len(), 1); + assert_eq!(updates[0].ref_name, "refs/heads/main"); + assert_eq!(updates[0].new_oid, "a1b2c3d4e5f6789012345678901234567890abcd"); +} +``` + +### Phase 3: Authorization Logic +**Goal:** Validate pushes against Nostr state events + +**Tests:** +1. ✅ Get maintainers from announcement +2. ✅ Get maintainers recursively +3. ✅ Handle circular maintainer references +4. ✅ Validate ref update matches state +5. ✅ Validate branch push matches state +6. ✅ Validate tag push matches state +7. ✅ Accept push to refs/nostr/* +8. ✅ Reject push not matching state +9. ✅ Reject push from non-maintainer + +**Implementation:** +```rust +// src/git/authorization.rs + +pub struct PushValidator { + storage: Storage, +} + +impl PushValidator { + pub async fn validate_push( + &self, + npub: &str, + identifier: &str, + updates: &[RefUpdate], + ) -> Result<()>; + + async fn get_maintainers(&self, npub: &str, identifier: &str) -> Vec; + async fn get_latest_state(&self, npub: &str, identifier: &str) -> Option; + fn validate_ref_update(&self, state: &StateEvent, update: &RefUpdate) -> Result<()>; +} +``` + +**Test Example:** +```rust +#[tokio::test] +async fn test_validate_matching_push() { + let storage = test_storage().await; + + // Create announcement and state + let announcement = create_announcement("alice", "repo1"); + let state = create_state("alice", "repo1") + .branch("main", "a1b2c3d4..."); + + storage.store_event(announcement).await.unwrap(); + storage.store_event(state).await.unwrap(); + + // Validate matching push + let validator = PushValidator::new(storage); + let update = RefUpdate { + old_oid: "0000...".into(), + new_oid: "a1b2c3d4...".into(), + ref_name: "refs/heads/main".into(), + }; + + let result = validator.validate_push("alice", "repo1", &[update]).await; + assert!(result.is_ok()); +} + +#[tokio::test] +async fn test_reject_mismatched_push() { + let storage = test_storage().await; + + // State points to commit A + let state = create_state("alice", "repo1") + .branch("main", "aaaa1111..."); + storage.store_event(state).await.unwrap(); + + // Try to push commit B + let validator = PushValidator::new(storage); + let update = RefUpdate { + old_oid: "0000...".into(), + new_oid: "bbbb2222...".into(), // Different! + ref_name: "refs/heads/main".into(), + }; + + let result = validator.validate_push("alice", "repo1", &[update]).await; + assert!(result.is_err()); + assert!(result.unwrap_err().to_string().contains("state")); +} +``` + +### Phase 4: HTTP Handlers +**Goal:** Serve Git Smart HTTP protocol + +**Tests:** +1. ✅ GET /npub/repo.git/info/refs?service=git-upload-pack +2. ✅ POST /npub/repo.git/git-upload-pack (clone/fetch) +3. ✅ POST /npub/repo.git/git-receive-pack (push with auth) +4. ✅ Return 403 for unauthorized push +5. ✅ Return 404 for non-existent repository +6. ✅ Set correct content-type headers +7. ✅ Include CORS headers + +**Implementation:** +```rust +// src/git/handler.rs + +pub async fn handle_info_refs( + npub: String, + identifier: String, + service: String, +) -> Result; + +pub async fn handle_upload_pack( + npub: String, + identifier: String, + body: Bytes, +) -> Result; + +pub async fn handle_receive_pack( + npub: String, + identifier: String, + body: Bytes, + validator: PushValidator, +) -> Result; +``` + +**Test Example:** +```rust +#[tokio::test] +async fn test_info_refs_returns_correct_headers() { + let app = test_app().await; + + // Create repository + app.create_repo("alice", "test-repo").await; + + // Request info/refs + let response = app.get("/alice-npub/test-repo.git/info/refs?service=git-upload-pack").await; + + assert_eq!(response.status(), 200); + assert_eq!( + response.headers().get("content-type").unwrap(), + "application/x-git-upload-pack-advertisement" + ); + assert_eq!( + response.headers().get("access-control-allow-origin").unwrap(), + "*" + ); +} + +#[tokio::test] +async fn test_receive_pack_rejects_unauthorized() { + let app = test_app().await; + + // Create repo with state + let (announcement, state) = app.create_repo_with_state() + .branch("main", "aaaa1111...") + .build() + .await; + + // Try to push different commit + let body = create_receive_pack_request() + .ref_update("refs/heads/main", "0000...", "bbbb2222...") + .build(); + + let response = app.post( + &format!("/{}/repo.git/git-receive-pack", announcement.author_npub()), + body + ).await; + + assert_eq!(response.status(), 403); +} +``` + +### Phase 5: Integration with Nostr Events +**Goal:** Automatic repository creation and state updates + +**Tests:** +1. ✅ Create repository when announcement received +2. ✅ Update HEAD when state event received +3. ✅ Handle announcement updates (maintainers change) +4. ✅ Clean up orphaned refs/nostr/* refs + +**Implementation:** +```rust +// src/nostr/events.rs (extend existing) + +async fn handle_repository_announcement(event: &Event, storage: &Storage) -> Result<()> { + // Extract npub and identifier + // Create bare repository + // Store event +} + +async fn handle_repository_state(event: &Event, storage: &Storage) -> Result<()> { + // Find repository + // Update refs to match state + // Update HEAD +} +``` + +**Test Example:** +```rust +#[tokio::test] +async fn test_repository_created_on_announcement() { + let app = test_app().await; + + let announcement = create_announcement("alice", "test-repo") + .with_clone_tag(app.domain()) + .build(); + + app.send_event(announcement).await.unwrap(); + + // Wait for async processing + tokio::time::sleep(Duration::from_millis(100)).await; + + // Verify repository exists + let repo_path = app.git_data_path() + .join("alice-npub") + .join("test-repo.git"); + + assert!(repo_path.exists()); + assert!(repo_path.join("HEAD").exists()); +} +``` + +### Phase 6: End-to-End Testing +**Goal:** Test with real Git client + +**Tests:** +1. ✅ Clone repository with git client +2. ✅ Fetch from repository +3. ✅ Push to repository (authorized) +4. ✅ Push rejected (unauthorized) +5. ✅ Multiple concurrent operations + +**Implementation:** +```rust +// tests/e2e/git_client.rs + +#[tokio::test] +async fn test_real_git_clone() { + let app = test_app().await; + + // Setup repository with content + let (announcement, _) = app.create_repo_with_commits() + .commit("Initial", "README.md", "# Test") + .build() + .await; + + // Clone with real git + let temp = TempDir::new().unwrap(); + let url = format!( + "http://{}/{}/{}.git", + app.domain(), + announcement.author_npub(), + announcement.identifier() + ); + + let output = Command::new("git") + .args(&["clone", &url]) + .current_dir(&temp) + .output() + .await + .unwrap(); + + assert!(output.status.success()); + + let cloned_path = temp.path().join(announcement.identifier()); + assert!(cloned_path.exists()); + assert!(cloned_path.join("README.md").exists()); +} +``` + +--- + +## Testing Strategy + +### Unit Tests (40%) +- Git repository operations (git2) +- Protocol parsing +- Authorization logic +- Pure functions, no I/O + +### Integration Tests (30%) +- HTTP handlers with test server +- Repository + Nostr event interaction +- Multi-maintainer flows +- State validation + +### Compliance Tests (20%) +- GRASP-01 Git requirements +- Use grasp-audit library +- Spec-driven assertions + +### E2E Tests (10%) +- Real git client operations +- End-to-end workflows +- Performance testing + +--- + +## Implementation Order + +### Week 1: Foundation +1. Add git2 dependency +2. Implement GitRepository (Phase 1) +3. Write unit tests for repository operations +4. Test repository creation from announcements + +### Week 2: Protocol & Authorization +1. Implement protocol parsing (Phase 2) +2. Implement authorization logic (Phase 3) +3. Write unit tests for both +4. Integration tests for validation + +### Week 3: HTTP & Integration +1. Implement HTTP handlers (Phase 4) +2. Integrate with Nostr events (Phase 5) +3. Integration tests for full flow +4. CORS and error handling + +### Week 4: E2E & Polish +1. E2E tests with real git (Phase 6) +2. Performance testing +3. GRASP-01 compliance testing +4. Documentation and examples + +--- + +## Success Criteria + +### Functional +- ✅ Clone repository via HTTP +- ✅ Push authorized commits +- ✅ Reject unauthorized pushes +- ✅ Support multi-maintainer +- ✅ Support refs/nostr/* for PRs + +### Quality +- ✅ >80% unit test coverage +- ✅ All integration tests pass +- ✅ GRASP-01 compliance 100% +- ✅ E2E tests with real git + +### Performance +- ✅ Clone 1MB repo < 1s +- ✅ Push validation < 100ms +- ✅ 100 concurrent ops without errors + +--- + +## Next Steps + +1. **Review this plan** - Does the hybrid approach make sense? +2. **Start Phase 1** - Add git2, implement GitRepository +3. **Write first test** - test_init_bare_repository +4. **Iterate with TDD** - Red → Green → Refactor + +--- + +## Questions for Review + +1. **Hybrid approach?** git2 + system git + HTTP layer - good balance? +2. **git-http-backend crate?** Worth using or implement minimal HTTP layer? +3. **Authorization granularity?** Validate per-ref or entire push? +4. **Error messages?** How detailed for push rejections? +5. **Testing scope?** Is 6 phases reasonable for first iteration? + +--- + +**Ready to proceed?** Let me know if this plan looks good, or if you'd like to adjust the approach! diff --git a/docs/archive/2025-11-04-evening/INDEX.md b/docs/archive/2025-11-04-evening/INDEX.md new file mode 100644 index 0000000..cb06dd5 --- /dev/null +++ b/docs/archive/2025-11-04-evening/INDEX.md @@ -0,0 +1,314 @@ +# Work Directory Index + +**Last Updated:** November 4, 2025 +**Status:** Ready for Implementation + +--- + +## 📁 Quick Navigation + +### 🚀 START HERE +**[NEXT_SESSION_START_HERE.md](NEXT_SESSION_START_HERE.md)** - Begin next session with this + +### 📊 Status & Progress +- **[STATUS.txt](STATUS.txt)** - Visual status summary (quick reference) +- **[current_status.md](current_status.md)** - Detailed project status +- **[session-summary.md](session-summary.md)** - What we accomplished this session + +### 📖 Understanding & Planning +- **[review-summary.md](review-summary.md)** - GRASP protocol review findings +- **[architecture-diagram.md](architecture-diagram.md)** - Visual architecture reference +- **[implementation-checklist.md](implementation-checklist.md)** - Detailed task checklist + +### 📋 Reference +- **[README.md](README.md)** - Work directory purpose and guidelines + +--- + +## 📚 Document Purposes + +### NEXT_SESSION_START_HERE.md +**Purpose:** Step-by-step implementation guide +**When to use:** Starting next coding session +**Contains:** +- Immediate goal (actix-web integration) +- Critical architecture understanding +- 8-step implementation plan with code examples +- Verification steps for each step +- Common issues and solutions +- Success criteria + +**Read this:** When you're ready to start coding + +--- + +### STATUS.txt +**Purpose:** Quick visual status overview +**When to use:** Quick check of where we are +**Contains:** +- Current compliance percentages +- Critical discoveries +- Next session plan +- Key references +- Success criteria + +**Read this:** When you need a quick reminder + +--- + +### current_status.md +**Purpose:** Comprehensive project status +**When to use:** Understanding overall context +**Contains:** +- Complete GRASP-01 requirements checklist +- Architecture understanding +- Current implementation status +- Known issues and blockers +- Progress summary +- Key references + +**Read this:** When you need detailed context + +--- + +### session-summary.md +**Purpose:** Summary of this review session +**When to use:** Remembering what we did +**Contains:** +- Session goals and accomplishments +- Key discoveries (7 major findings) +- Documents created +- Lessons learned +- Next steps + +**Read this:** When you want to know what happened this session + +--- + +### review-summary.md +**Purpose:** GRASP protocol review findings +**When to use:** Understanding why we're making changes +**Contains:** +- 10 critical discoveries from GRASP spec +- Evidence for each finding +- Impact and action items +- Compliance status +- Immediate next steps + +**Read this:** When you need to justify architectural decisions + +--- + +### architecture-diagram.md +**Purpose:** Visual architecture reference +**When to use:** During implementation for reference +**Contains:** +- Current vs. target architecture diagrams +- Request flow examples (5 scenarios) +- Component responsibilities +- Configuration flow +- Test architecture +- File structure +- Comparison with ngit-relay + +**Read this:** When you need to visualize the system + +--- + +### implementation-checklist.md +**Purpose:** Detailed task checklist +**When to use:** Tracking implementation progress +**Contains:** +- 5 phases with detailed tasks +- Verification steps for each task +- Manual testing procedures +- Automated testing commands +- Acceptance criteria +- Known issues to watch for +- Reference commands + +**Read this:** While implementing to track progress + +--- + +## 🎯 Recommended Reading Order + +### For Next Session (Implementation) + +1. **STATUS.txt** (1 min) + - Quick reminder of where we are + +2. **NEXT_SESSION_START_HERE.md** (10 min) + - Understand the immediate goal + - Review the 8-step plan + +3. **architecture-diagram.md** (5 min) + - Visual reference for what we're building + +4. **implementation-checklist.md** (ongoing) + - Check off tasks as you complete them + +### For Understanding Context + +1. **session-summary.md** (5 min) + - What we accomplished this session + +2. **current_status.md** (10 min) + - Overall project status + +3. **review-summary.md** (15 min) + - Why we're making these changes + +### For Reference During Coding + +- **architecture-diagram.md** - Visual reference +- **implementation-checklist.md** - Task tracking +- **NEXT_SESSION_START_HERE.md** - Step-by-step guide + +--- + +## 🔍 Finding Specific Information + +### "How do I implement X?" +→ **NEXT_SESSION_START_HERE.md** (Step-by-step with code) + +### "Why are we doing X?" +→ **review-summary.md** (Findings from GRASP review) + +### "What's the overall status?" +→ **current_status.md** or **STATUS.txt** + +### "What did we do this session?" +→ **session-summary.md** + +### "How does the architecture work?" +→ **architecture-diagram.md** + +### "What tasks are left?" +→ **implementation-checklist.md** + +### "What are the requirements?" +→ **current_status.md** (GRASP-01 checklist) + +### "How do I test X?" +→ **implementation-checklist.md** (Testing section) + +--- + +## 📊 Document Relationships + +``` +STATUS.txt + ↓ (quick overview) +current_status.md + ↓ (detailed status) +session-summary.md + ↓ (what we did) +review-summary.md + ↓ (why we're doing this) +architecture-diagram.md + ↓ (visual reference) +NEXT_SESSION_START_HERE.md + ↓ (how to implement) +implementation-checklist.md + ↓ (track progress) +``` + +--- + +## 🗂️ File Lifecycle + +### Active (Use These) +- ✅ NEXT_SESSION_START_HERE.md - Update for each phase +- ✅ current_status.md - Update as we progress +- ✅ implementation-checklist.md - Check off as we go +- ✅ STATUS.txt - Update after each phase + +### Reference (Keep These) +- 📖 architecture-diagram.md - Permanent reference +- 📖 review-summary.md - Permanent reference +- 📖 session-summary.md - Historical record + +### Archive (After Implementation) +- 📦 implementation-checklist.md → Delete when phase complete +- 📦 NEXT_SESSION_START_HERE.md → Update for next phase +- 📦 session-summary.md → Move to docs/archive/ + +--- + +## ✨ Quick Tips + +### Starting a New Session +1. Read STATUS.txt (1 min) +2. Read NEXT_SESSION_START_HERE.md (10 min) +3. Open implementation-checklist.md to track progress +4. Start coding! + +### When Stuck +1. Check architecture-diagram.md for visual reference +2. Check NEXT_SESSION_START_HERE.md for step details +3. Check review-summary.md for why we're doing it +4. Check ../grasp/01.md for requirements + +### Ending a Session +1. Update current_status.md with progress +2. Update STATUS.txt with new status +3. Check off completed tasks in implementation-checklist.md +4. Update NEXT_SESSION_START_HERE.md if needed + +### After Phase Complete +1. Archive implementation-checklist.md +2. Update NEXT_SESSION_START_HERE.md for next phase +3. Update current_status.md with new status +4. Create new session-summary.md if needed + +--- + +## 📈 Progress Tracking + +Use these documents to track progress: + +### Daily +- [ ] Check STATUS.txt +- [ ] Update implementation-checklist.md +- [ ] Follow NEXT_SESSION_START_HERE.md + +### Weekly +- [ ] Update current_status.md +- [ ] Update STATUS.txt +- [ ] Review progress against checklist + +### After Each Phase +- [ ] Update current_status.md +- [ ] Create new session-summary.md +- [ ] Update NEXT_SESSION_START_HERE.md +- [ ] Archive completed documents + +--- + +## 🎯 Current Phase + +**Phase:** actix-web Integration +**Status:** Ready to Start +**Start With:** NEXT_SESSION_START_HERE.md +**Track With:** implementation-checklist.md +**Reference:** architecture-diagram.md + +--- + +## 📞 Quick Reference + +| Need | Document | +|------|----------| +| Start coding | NEXT_SESSION_START_HERE.md | +| Quick status | STATUS.txt | +| Detailed status | current_status.md | +| Why we're doing this | review-summary.md | +| How it works | architecture-diagram.md | +| Task list | implementation-checklist.md | +| What we did | session-summary.md | + +--- + +**Last Updated:** November 4, 2025 +**Next Update:** After actix-web integration complete diff --git a/docs/archive/2025-11-04-evening/NEXT_SESSION_START_HERE.md b/docs/archive/2025-11-04-evening/NEXT_SESSION_START_HERE.md new file mode 100644 index 0000000..890d5ed --- /dev/null +++ b/docs/archive/2025-11-04-evening/NEXT_SESSION_START_HERE.md @@ -0,0 +1,650 @@ +# Next Session Start Here + +**Date:** November 4, 2025 +**Purpose:** Quick start guide for next development session +**Status:** Ready for actix-web integration + +--- + +## 🎯 Immediate Goal + +**Integrate actix-web to serve both Nostr relay (WebSocket) and Git HTTP on the SAME PORT.** + +This is the critical architectural fix needed to match GRASP-01 requirements and ngit-relay's design. + +--- + +## 🚨 Critical Understanding + +### Single Port Architecture (from ../ngit-relay) + +``` +┌─────────────────────────────────────┐ +│ Single Port (8080) │ +│ │ +│ ┌─────────────────────────────┐ │ +│ │ HTTP/WebSocket Router │ │ +│ │ (nginx in ngit-relay) │ │ +│ │ (actix-web in ngit-grasp) │ │ +│ └──────────┬──────────────────┘ │ +│ │ │ +│ ┌──────┴──────┐ │ +│ ↓ ↓ │ +│ Git HTTP Nostr Relay │ +│ // / (WebSocket) │ +│ .git │ +└─────────────────────────────────────┘ +``` + +**Key Points:** +1. ONE port listens for all traffic +2. Router inspects path and decides where to send request +3. Git paths go to Git handler +4. Everything else goes to Nostr relay (with WebSocket upgrade) +5. CORS headers on ALL responses + +--- + +## 📋 Step-by-Step Implementation Plan + +### Step 1: Add actix-web Dependencies + +**File:** `Cargo.toml` + +```toml +[dependencies] +# Existing... +actix-web = "4" +actix-cors = "0.7" +actix-ws = "0.3" # For WebSocket support + +# Git HTTP backend +git-http-backend = "0.2" # Check latest version +``` + +**Why:** +- `actix-web` - HTTP framework with routing +- `actix-cors` - Easy CORS middleware +- `actix-ws` - WebSocket support +- `git-http-backend` - Git Smart HTTP protocol + +### Step 2: Create HTTP Router Module + +**File:** `src/http/mod.rs` (NEW) + +```rust +//! HTTP server with routing for Git and Nostr + +use actix_web::{web, App, HttpServer}; +use actix_cors::Cors; + +pub mod git; +pub mod nostr; + +pub async fn run_server(config: Config, storage: Storage) -> Result<()> { + let bind_addr = config.bind_address.clone(); + + HttpServer::new(move || { + App::new() + // CORS middleware (GRASP-01 requirement) + .wrap( + Cors::default() + .allow_any_origin() + .allowed_methods(vec!["GET", "POST"]) + .allowed_headers(vec!["Content-Type"]) + .max_age(3600) + ) + // Git HTTP routes + .service( + web::scope("/{npub}/{repo}") + .guard(guard::fn_guard(|ctx| { + // Only match *.git paths + ctx.head().uri.path().ends_with(".git") + })) + .route("", web::get().to(git::handle_git_request)) + .route("/{tail:.*}", web::to(git::handle_git_request)) + ) + // Nostr relay (WebSocket at /) + .route("/", web::get().to(nostr::handle_websocket)) + // Static files (optional) + .route("/", web::get().to(nostr::handle_http_root)) + }) + .bind(bind_addr)? + .run() + .await?; + + Ok(()) +} +``` + +**Why:** +- Single HTTP server listening on one port +- Routes by URL path pattern +- CORS applied to all routes +- Git paths (*.git) go to Git handler +- Root path (/) handles WebSocket upgrade for Nostr + +### Step 3: Create Git HTTP Handler + +**File:** `src/http/git.rs` (NEW) + +```rust +//! Git Smart HTTP handler + +use actix_web::{web, HttpRequest, HttpResponse, Result}; +use git_http_backend::{GitHttpBackend, Method}; + +pub async fn handle_git_request( + req: HttpRequest, + body: web::Bytes, + path: web::Path<(String, String)>, +) -> Result { + let (npub, repo) = path.into_inner(); + + // Construct repository path + let repo_path = format!("{}/{}/{}", + config.git_data_path, npub, repo); + + // Check if repository exists + if !std::path::Path::new(&repo_path).exists() { + return Ok(HttpResponse::NotFound() + .body("Repository not found")); + } + + // Parse Git HTTP request + let method = match *req.method() { + actix_web::http::Method::GET => Method::Get, + actix_web::http::Method::POST => Method::Post, + _ => return Ok(HttpResponse::MethodNotAllowed().finish()), + }; + + // Use git-http-backend to handle request + let backend = GitHttpBackend::new(&repo_path); + let response = backend.handle(method, req.path(), &body)?; + + // Convert to actix HttpResponse + Ok(HttpResponse::Ok() + .content_type(response.content_type) + .body(response.body)) +} +``` + +**Why:** +- Handles Git Smart HTTP protocol +- Serves from `{GIT_DATA_PATH}/{npub}/{repo}.git` +- Uses `git-http-backend` crate for protocol details +- Returns 404 if repo doesn't exist + +### Step 4: Create Nostr WebSocket Handler + +**File:** `src/http/nostr.rs` (NEW) + +```rust +//! Nostr relay WebSocket handler + +use actix_web::{web, HttpRequest, HttpResponse, Result}; +use actix_ws::Message; + +pub async fn handle_websocket( + req: HttpRequest, + stream: web::Payload, + storage: web::Data, +) -> Result { + // Upgrade to WebSocket + let (response, mut session, mut msg_stream) = actix_ws::handle(&req, stream)?; + + // Spawn task to handle WebSocket messages + actix_web::rt::spawn(async move { + while let Some(Ok(msg)) = msg_stream.next().await { + match msg { + Message::Text(text) => { + // Handle Nostr message (EVENT, REQ, CLOSE) + let responses = handle_nostr_message(&text, &storage).await; + for response in responses { + session.text(response).await.ok(); + } + } + Message::Ping(bytes) => { + session.pong(&bytes).await.ok(); + } + Message::Close(_) => break, + _ => {} + } + } + }); + + Ok(response) +} + +pub async fn handle_http_root() -> Result { + // Serve static HTML for browsers + Ok(HttpResponse::Ok() + .content_type("text/html") + .body("

ngit-grasp

Nostr relay at ws://

")) +} +``` + +**Why:** +- Handles WebSocket upgrade at `/` +- Reuses existing Nostr message handling logic +- Returns HTML for browsers (non-WebSocket requests) + +### Step 5: Update main.rs + +**File:** `src/main.rs` + +```rust +use anyhow::Result; +use tracing::{info, Level}; +use tracing_subscriber::FmtSubscriber; + +mod config; +mod http; // NEW +mod nostr; +mod storage; + +use config::Config; + +#[tokio::main] +async fn main() -> Result<()> { + // Initialize tracing + let subscriber = FmtSubscriber::builder() + .with_max_level(Level::DEBUG) + .finish(); + tracing::subscriber::set_global_default(subscriber)?; + + info!("Starting ngit-grasp..."); + + // Load configuration + let config = Config::from_env()?; + info!("Configuration: {}", config.bind_address); + + // Initialize storage + let storage = storage::Storage::new(&config)?; + info!("Storage initialized at: {}", config.relay_data_path); + + // Start HTTP server (Git + Nostr on same port) + info!("Starting server on {}", config.bind_address); + http::run_server(config, storage).await?; + + Ok(()) +} +``` + +**Why:** +- Replaces separate relay with unified HTTP server +- Single entry point for all services +- Simpler architecture + +### Step 6: Update Configuration + +**File:** `src/config.rs` + +Add field for Git data path: + +```rust +pub struct Config { + pub bind_address: String, + pub domain: String, + pub relay_data_path: String, + pub git_data_path: String, // NEW + // ... other fields +} + +impl Config { + pub fn from_env() -> Result { + Ok(Config { + bind_address: env::var("NGIT_BIND_ADDRESS") + .unwrap_or_else(|_| "127.0.0.1:8080".to_string()), + domain: env::var("NGIT_DOMAIN")?, + relay_data_path: env::var("NGIT_RELAY_DATA_PATH") + .unwrap_or_else(|_| "./data/relay".to_string()), + git_data_path: env::var("NGIT_GIT_DATA_PATH") // NEW + .unwrap_or_else(|_| "./data/repos".to_string()), + // ... + }) + } +} +``` + +**File:** `.env.example` + +```bash +# Service Configuration +NGIT_DOMAIN=example.com +NGIT_BIND_ADDRESS=127.0.0.1:8080 + +# Relay Information +NGIT_RELAY_NAME="ngit-grasp instance" +NGIT_RELAY_DESCRIPTION="Rust GRASP implementation" +NGIT_OWNER_NPUB="npub1..." + +# Storage Paths +NGIT_GIT_DATA_PATH=./data/repos +NGIT_RELAY_DATA_PATH=./data/relay + +# Logging +NGIT_LOG_LEVEL=INFO +RUST_LOG=info +``` + +### Step 7: Update Tests + +**File:** `tests/common/relay.rs` + +Update `start_with_port` to pass domain correctly: + +```rust +pub async fn start_with_port(port: u16) -> Self { + let bind_address = format!("127.0.0.1:{}", port); + let domain = format!("127.0.0.1:{}", port); // NEW + let url = format!("ws://{}", domain); + + let process = Command::new(&binary_path) + .env("NGIT_BIND_ADDRESS", &bind_address) + .env("NGIT_DOMAIN", &domain) // UPDATED + .env("NGIT_GIT_DATA_PATH", "./test-data/repos") // NEW + .env("NGIT_RELAY_DATA_PATH", "./test-data/relay") // NEW + .env("RUST_LOG", "warn") + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .spawn() + .expect("Failed to start relay process"); + + // ... rest of method +} +``` + +**Why:** +- Domain must match bind address for announcement validation +- Separate test data directories +- Clean up test data after tests + +### Step 8: Add Git HTTP Tests + +**File:** `tests/grasp01_git_http.rs` (NEW) + +```rust +//! GRASP-01 Git HTTP Integration Tests +//! +//! Reference: ../grasp/01.md lines 15-40 +//! +//! These tests verify Git Smart HTTP service requirements: +//! - Serve repos at //.git +//! - Accept pushes matching state announcements +//! - CORS support + +mod common; + +use common::TestRelay; +use std::process::Command; + +#[tokio::test] +async fn test_git_clone_basic() { + // Reference: ../grasp/01.md line 15 + // MUST serve git repository via unauthenticated git smart http service + + let relay = TestRelay::start().await; + let domain = relay.domain(); + + // TODO: Create test repository announcement + // TODO: Clone via git clone http://{domain}/{npub}/{id}.git + + relay.stop().await; +} + +#[tokio::test] +async fn test_cors_headers() { + // Reference: ../grasp/01.md lines 32-40 + // MUST include CORS headers on all responses + + let relay = TestRelay::start().await; + let url = format!("http://{}/", relay.domain()); + + let response = reqwest::get(&url).await.unwrap(); + + // Check CORS headers + assert_eq!( + response.headers().get("access-control-allow-origin"), + Some(&"*".parse().unwrap()) + ); + + relay.stop().await; +} +``` + +**Why:** +- Tests reference GRASP protocol line numbers +- Verifies Git HTTP functionality +- Checks CORS compliance + +--- + +## 🔍 Verification Steps + +After implementing the above: + +### 1. Build and Run + +```bash +# Build +cargo build + +# Run server +NGIT_DOMAIN=localhost:8080 \ +NGIT_BIND_ADDRESS=127.0.0.1:8080 \ +NGIT_GIT_DATA_PATH=./data/repos \ +NGIT_RELAY_DATA_PATH=./data/relay \ +cargo run +``` + +### 2. Test Nostr Relay (WebSocket) + +```bash +# In another terminal +cd grasp-audit +cargo run -- --url ws://localhost:8080 +``` + +**Expected:** NIP-01 smoke tests should pass + +### 3. Test Git HTTP (Manual) + +```bash +# Create test repository +mkdir -p ./data/repos/npub1test/test-repo.git +cd ./data/repos/npub1test/test-repo.git +git init --bare + +# Try to clone +git clone http://localhost:8080/npub1test/test-repo.git +``` + +**Expected:** Should clone successfully (even if empty) + +### 4. Test CORS + +```bash +curl -v http://localhost:8080/ -H "Origin: https://example.com" +``` + +**Expected:** Response should include: +``` +access-control-allow-origin: * +access-control-allow-methods: GET, POST +access-control-allow-headers: Content-Type +``` + +### 5. Run Integration Tests + +```bash +# All tests +cargo test + +# Just NIP-01 +cargo test --test nip01_compliance + +# Just Git HTTP (when implemented) +cargo test --test grasp01_git_http +``` + +**Expected:** All tests pass + +--- + +## 🐛 Common Issues & Solutions + +### Issue: Port Already in Use + +**Symptom:** "Address already in use" error + +**Solution:** +```bash +# Find process using port +lsof -i :8080 + +# Kill it +kill -9 + +# Or use different port +NGIT_BIND_ADDRESS=127.0.0.1:8081 cargo run +``` + +### Issue: WebSocket Upgrade Fails + +**Symptom:** WebSocket connection refused + +**Solution:** +- Check actix-web WebSocket handling +- Verify `Upgrade: websocket` header is present +- Check actix-ws is properly configured + +### Issue: Git Clone Fails + +**Symptom:** "repository not found" or protocol error + +**Solution:** +- Verify repository exists at correct path +- Check git-http-backend configuration +- Ensure repository is bare (`git init --bare`) +- Check file permissions + +### Issue: CORS Headers Missing + +**Symptom:** Browser console shows CORS error + +**Solution:** +- Verify CORS middleware is applied +- Check middleware order (CORS should be first) +- Test with curl to see actual headers + +--- + +## 📚 Key Resources + +### GRASP Protocol +- `../grasp/01.md` - **THE SPEC** - Read this first! +- Lines 1-14: Nostr relay requirements +- Lines 15-31: Git HTTP service requirements +- Lines 32-40: CORS requirements + +### Reference Implementation +- `../ngit-relay/src/nginx.conf` - **ROUTING PATTERN** + - Lines 8-13: Single port listener + - Lines 15-48: Git HTTP routing + - Lines 50-94: Nostr relay routing +- `../ngit-relay/docker-compose.yml` - Port configuration +- `../ngit-relay/.env.example` - Environment variables + +### actix-web Documentation +- [Routing](https://actix.rs/docs/url-dispatch/) +- [WebSocket](https://actix.rs/docs/websockets/) +- [CORS](https://docs.rs/actix-cors/) + +### git-http-backend Crate +- [Docs](https://docs.rs/git-http-backend/) +- [Examples](https://github.com/w4/git-http-backend/tree/master/examples) + +--- + +## ✅ Success Criteria + +You'll know this step is complete when: + +1. ✅ Server starts on single port +2. ✅ WebSocket connects at `ws://localhost:8080/` +3. ✅ NIP-01 smoke tests pass +4. ✅ Can clone Git repo at `http://localhost:8080/npub.../repo.git` +5. ✅ CORS headers present on all responses +6. ✅ OPTIONS requests return 204 +7. ✅ All integration tests pass + +--- + +## 🎯 After This Step + +Once actix-web integration is complete: + +1. **Repository Provisioning** + - Create repos when announcements received + - Initialize bare repositories + - Set up directory structure + +2. **Push Authorization** + - Intercept git-receive-pack + - Validate against state announcements + - Handle maintainer sets + +3. **Full GRASP-01 Compliance** + - All tests passing + - Ready for production testing + +--- + +## 💡 Tips + +1. **Start Simple** + - Get basic HTTP routing working first + - Add WebSocket support second + - Add Git HTTP last + +2. **Test Incrementally** + - Test each component as you add it + - Don't wait until everything is done + +3. **Use curl for Debugging** + ```bash + # Test HTTP + curl -v http://localhost:8080/ + + # Test CORS + curl -v http://localhost:8080/ -H "Origin: https://example.com" + + # Test Git info/refs + curl http://localhost:8080/npub.../repo.git/info/refs?service=git-upload-pack + ``` + +4. **Check ngit-relay for Patterns** + - nginx.conf shows exact routing logic + - Copy the pattern, not the implementation + +5. **Keep Tests Running** + ```bash + # In one terminal + cargo watch -x 'test --test nip01_compliance' + + # Make changes, tests auto-run + ``` + +--- + +**Ready to Start?** Begin with Step 1 (Add Dependencies) + +**Questions?** Check `work/current_status.md` for context + +**Stuck?** Review `../ngit-relay/src/nginx.conf` for routing pattern + +--- + +**Last Updated:** November 4, 2025 +**Next Update:** After actix-web integration complete diff --git a/docs/archive/2025-11-04-evening/STATUS.txt b/docs/archive/2025-11-04-evening/STATUS.txt new file mode 100644 index 0000000..8c70db2 --- /dev/null +++ b/docs/archive/2025-11-04-evening/STATUS.txt @@ -0,0 +1,78 @@ +╔════════════════════════════════════════════════════════════════╗ +║ NGIT-GRASP STATUS ║ +║ November 4, 2025 ║ +╚════════════════════════════════════════════════════════════════╝ + +📊 OVERALL STATUS: Planning Complete, Ready for Implementation + +┌────────────────────────────────────────────────────────────────┐ +│ CRITICAL DISCOVERY: Architecture Was Wrong! │ +├────────────────────────────────────────────────────────────────┤ +│ ❌ Previous: Nostr on :8080, Git on :8081 (WRONG!) │ +│ ✅ Correct: BOTH on :8080, routed by path (REQUIRED!) │ +│ │ +│ Fix: Integrate actix-web for HTTP routing │ +└────────────────────────────────────────────────────────────────┘ + +┌────────────────────────────────────────────────────────────────┐ +│ COMPLIANCE STATUS │ +├────────────────────────────────────────────────────────────────┤ +│ NIP-01 (Nostr Relay): ████████░░ 80% (needs NIP-11 fix) │ +│ NIP-34 (Git Announce): ████░░░░░░ 40% (needs validation) │ +│ GRASP-01 (Core): ██░░░░░░░░ 20% (needs Git HTTP) │ +│ │ +│ Target After Next Phase: ████████░░ 80% (Git HTTP working) │ +└────────────────────────────────────────────────────────────────┘ + +┌────────────────────────────────────────────────────────────────┐ +│ WORK DOCUMENTS CREATED │ +├────────────────────────────────────────────────────────────────┤ +│ ✅ current_status.md - Overall project status │ +│ ✅ NEXT_SESSION_START_HERE.md - Implementation guide (START!) │ +│ ✅ review-summary.md - GRASP protocol findings │ +│ ✅ architecture-diagram.md - Visual reference │ +│ ✅ implementation-checklist.md - Detailed task list │ +│ ✅ session-summary.md - What we accomplished │ +└────────────────────────────────────────────────────────────────┘ + +┌────────────────────────────────────────────────────────────────┐ +│ NEXT SESSION: Implement actix-web Integration │ +├────────────────────────────────────────────────────────────────┤ +│ 1. Add dependencies (actix-web, actix-cors, git-http-backend) │ +│ 2. Create src/http/mod.rs (HTTP server + routing) │ +│ 3. Create src/http/git.rs (Git Smart HTTP handler) │ +│ 4. Create src/http/nostr.rs (WebSocket handler) │ +│ 5. Update src/main.rs (use new HTTP server) │ +│ 6. Update tests (verify single-port works) │ +│ 7. Manual testing (clone, WebSocket, CORS) │ +│ 8. Automated testing (all tests pass) │ +│ │ +│ Estimated Time: 3-6 hours │ +│ Confidence: HIGH ✅ │ +└────────────────────────────────────────────────────────────────┘ + +┌────────────────────────────────────────────────────────────────┐ +│ KEY REFERENCES │ +├────────────────────────────────────────────────────────────────┤ +│ 📖 ../grasp/01.md - THE SPEC (lines 1-40) │ +│ 📖 ../ngit-relay/src/nginx.conf - Routing pattern reference │ +│ 📖 work/NEXT_SESSION_START_HERE.md - Implementation guide │ +│ 📖 work/architecture-diagram.md - Visual architecture │ +│ 📖 work/implementation-checklist.md - Task checklist │ +└────────────────────────────────────────────────────────────────┘ + +┌────────────────────────────────────────────────────────────────┐ +│ SUCCESS CRITERIA (Next Phase) │ +├────────────────────────────────────────────────────────────────┤ +│ ✅ Server starts on single port (8080) │ +│ ✅ WebSocket connects at ws://localhost:8080/ │ +│ ✅ NIP-01 smoke tests pass │ +│ ✅ Can clone Git repo via http://localhost:8080/npub.../x.git │ +│ ✅ CORS headers present on all responses │ +│ ✅ OPTIONS requests return 204 No Content │ +│ ✅ All integration tests pass │ +└────────────────────────────────────────────────────────────────┘ + +╔════════════════════════════════════════════════════════════════╗ +║ 🚀 READY TO IMPLEMENT - Start with NEXT_SESSION_START_HERE.md ║ +╚════════════════════════════════════════════════════════════════╝ diff --git a/docs/archive/2025-11-04-evening/architecture-diagram.md b/docs/archive/2025-11-04-evening/architecture-diagram.md new file mode 100644 index 0000000..056e551 --- /dev/null +++ b/docs/archive/2025-11-04-evening/architecture-diagram.md @@ -0,0 +1,442 @@ +# ngit-grasp Architecture Diagram + +**Date:** November 4, 2025 +**Purpose:** Visual reference for single-port architecture + +--- + +## Current Architecture (WRONG ❌) + +``` +┌─────────────────────────────────────────┐ +│ Port 8080 │ +│ ┌───────────────────────────────────┐ │ +│ │ Nostr Relay (WebSocket) │ │ +│ │ - NIP-01 protocol │ │ +│ │ - Event storage │ │ +│ │ - Subscriptions │ │ +│ └───────────────────────────────────┘ │ +└─────────────────────────────────────────┘ + +┌─────────────────────────────────────────┐ +│ Port 8081 (WRONG!) │ +│ ┌───────────────────────────────────┐ │ +│ │ Git HTTP Server │ │ +│ │ - Not implemented │ │ +│ └───────────────────────────────────┘ │ +└─────────────────────────────────────────┘ +``` + +**Problem:** GRASP-01 requires single port! + +--- + +## Target Architecture (CORRECT ✅) + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Single Port (8080) │ +│ │ +│ ┌───────────────────────────────────────────────────────┐ │ +│ │ actix-web HTTP Server │ │ +│ │ │ │ +│ │ ┌──────────────────────────────────────────────────┐ │ │ +│ │ │ CORS Middleware (ALL requests) │ │ │ +│ │ │ - Access-Control-Allow-Origin: * │ │ │ +│ │ │ - Access-Control-Allow-Methods: GET, POST │ │ │ +│ │ │ - Access-Control-Allow-Headers: Content-Type │ │ │ +│ │ └──────────────────────────────────────────────────┘ │ │ +│ │ │ │ +│ │ ┌──────────────────────────────────────────────────┐ │ │ +│ │ │ HTTP Router │ │ │ +│ │ │ │ │ │ +│ │ │ Path Pattern Matching: │ │ │ +│ │ │ - //.git/* → Git Handler │ │ │ +│ │ │ - /* → Nostr Handler │ │ │ +│ │ └──────────────────────────────────────────────────┘ │ │ +│ │ │ │ +│ │ ┌────────────────────┐ ┌─────────────────────────┐ │ │ +│ │ │ Git HTTP Handler │ │ Nostr Relay Handler │ │ │ +│ │ │ │ │ │ │ │ +│ │ │ ┌──────────────┐ │ │ ┌──────────────────┐ │ │ │ +│ │ │ │ git-http- │ │ │ │ WebSocket Upgrade│ │ │ │ +│ │ │ │ backend │ │ │ │ │ │ │ │ +│ │ │ │ │ │ │ │ ┌──────────────┐ │ │ │ │ +│ │ │ │ - info/refs │ │ │ │ │ NIP-01 │ │ │ │ │ +│ │ │ │ - upload-pack│ │ │ │ │ - EVENT │ │ │ │ │ +│ │ │ │ - receive-pack │ │ │ │ - REQ │ │ │ │ │ +│ │ │ │ │ │ │ │ │ - CLOSE │ │ │ │ │ +│ │ │ │ Authorization: │ │ │ └──────────────┘ │ │ │ │ +│ │ │ │ - Query state│ │ │ │ │ │ │ │ +│ │ │ │ - Validate │ │ │ │ ┌──────────────┐ │ │ │ │ +│ │ │ │ - Accept/ │ │ │ │ │ NIP-11 │ │ │ │ │ +│ │ │ │ Reject │ │ │ │ │ - GRASP fields│ │ │ │ │ +│ │ │ └──────────────┘ │ │ │ └──────────────┘ │ │ │ │ +│ │ │ │ │ │ │ │ │ │ +│ │ │ Repository: │ │ │ ┌──────────────┐ │ │ │ │ +│ │ │ {GIT_DATA_PATH}/ │ │ │ │ NIP-34 │ │ │ │ │ +│ │ │ {npub}/ │ │ │ │ - Announce │ │ │ │ │ +│ │ │ {identifier}.git │ │ │ │ - State │ │ │ │ │ +│ │ │ │ │ │ │ - Validate │ │ │ │ │ +│ │ └────────────────────┘ │ │ └──────────────┘ │ │ │ │ +│ │ │ │ │ │ │ │ +│ │ │ │ HTTP Root: │ │ │ │ +│ │ │ │ - Serve HTML │ │ │ │ +│ │ │ │ - NIP-11 JSON │ │ │ │ +│ │ │ └──────────────────┘ │ │ │ +│ │ │ │ │ │ +│ └───────────────────────────┴─────────────────────────┘ │ │ +│ │ +│ ┌───────────────────────────────────────────────────────┐ │ +│ │ Storage Layer │ │ +│ │ │ │ +│ │ ┌──────────────────────┐ ┌──────────────────────┐ │ │ +│ │ │ Git Repositories │ │ Nostr Events DB │ │ │ +│ │ │ │ │ │ │ │ +│ │ │ {GIT_DATA_PATH}/ │ │ {RELAY_DATA_PATH}/ │ │ │ +│ │ │ ├── npub1.../ │ │ - Announcements │ │ │ +│ │ │ │ ├── repo1.git/ │ │ - State events │ │ │ +│ │ │ │ └── repo2.git/ │ │ - Issues/Patches │ │ │ +│ │ │ └── npub2.../ │ │ - Other events │ │ │ +│ │ │ └── repo3.git/ │ │ │ │ │ +│ │ └──────────────────────┘ └──────────────────────┘ │ │ +│ └───────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## Request Flow Examples + +### Example 1: Git Clone + +``` +Client: git clone http://localhost:8080/npub1abc.../my-repo.git + ↓ +actix-web receives HTTP GET request + ↓ +CORS middleware adds headers + ↓ +Router matches path: /npub1abc.../my-repo.git + ↓ +Git Handler receives request + ↓ +git-http-backend processes: + - GET /npub1abc.../my-repo.git/info/refs?service=git-upload-pack + ↓ +Response includes: + - CORS headers + - Git protocol data + - Capabilities: allow-reachable-sha1-in-want, allow-tip-sha1-in-want + ↓ +Client receives data and clones repository +``` + +### Example 2: Git Push + +``` +Client: git push http://localhost:8080/npub1abc.../my-repo.git main + ↓ +actix-web receives HTTP POST request + ↓ +CORS middleware adds headers + ↓ +Router matches path: /npub1abc.../my-repo.git + ↓ +Git Handler receives request + ↓ +BEFORE spawning git-receive-pack: + 1. Parse ref updates from request body + 2. Query latest state announcement from relay + 3. Validate pusher in maintainer set + 4. Validate ref updates match state + ↓ +If validation passes: + - Spawn git-receive-pack + - Stream response back to client + ↓ +If validation fails: + - Return HTTP 403 Forbidden + - Include error message + ↓ +Client receives success/failure +``` + +### Example 3: WebSocket Connection (Nostr) + +``` +Client: new WebSocket('ws://localhost:8080/') + ↓ +actix-web receives HTTP GET with Upgrade: websocket + ↓ +CORS middleware adds headers + ↓ +Router matches path: / + ↓ +Nostr Handler receives request + ↓ +Upgrade to WebSocket + ↓ +Client sends: ["EVENT", {...}] + ↓ +Nostr Handler processes EVENT + ↓ +If kind 30617 (announcement): + - Validate clone/relays tags + - Provision Git repository + - Store event + ↓ +Response: ["OK", event_id, true, ""] + ↓ +Client receives confirmation +``` + +### Example 4: NIP-11 Request + +``` +Client: fetch('http://localhost:8080/', { + headers: { 'Accept': 'application/nostr+json' } +}) + ↓ +actix-web receives HTTP GET with Accept header + ↓ +CORS middleware adds headers + ↓ +Router matches path: / + ↓ +Nostr Handler checks Accept header + ↓ +Returns NIP-11 JSON: +{ + "name": "ngit-grasp instance", + "description": "Rust GRASP implementation", + "supported_nips": [1, 11, 34], + "supported_grasps": ["GRASP-01"], + "repo_acceptance_criteria": "Must list this service in clone and relays tags", + "curation": "Basic spam prevention" +} + ↓ +Client receives relay information +``` + +### Example 5: CORS Preflight (OPTIONS) + +``` +Browser: OPTIONS http://localhost:8080/ +Headers: + - Origin: https://example.com + - Access-Control-Request-Method: POST + ↓ +actix-web receives OPTIONS request + ↓ +CORS middleware handles preflight + ↓ +Returns 204 No Content with headers: + - Access-Control-Allow-Origin: * + - Access-Control-Allow-Methods: GET, POST + - Access-Control-Allow-Headers: Content-Type + - Access-Control-Max-Age: 3600 + ↓ +Browser caches preflight response + ↓ +Browser proceeds with actual request +``` + +--- + +## Component Responsibilities + +### actix-web HTTP Server +- Listen on single port +- Route requests by path +- Handle WebSocket upgrades +- Apply CORS to all requests + +### CORS Middleware +- Add headers to ALL responses +- Handle OPTIONS preflight +- Allow any origin (GRASP-01 requirement) + +### HTTP Router +- Match `/npub.../repo.git` → Git Handler +- Match `/` → Nostr Handler +- Pass through to appropriate handler + +### Git Handler +- Serve Git Smart HTTP protocol +- Read from `{GIT_DATA_PATH}/{npub}/{id}.git` +- Validate pushes before accepting +- Return 404 for missing repos + +### Nostr Handler +- Upgrade HTTP to WebSocket +- Process NIP-01 messages +- Store/query events +- Serve NIP-11 for HTTP requests +- Provision repos from announcements + +### Storage Layer +- Git repositories (bare) +- Nostr events (database) +- Separate paths for each + +--- + +## Configuration Flow + +``` +Environment Variables + ↓ +.env file (optional) + ↓ +Config::from_env() + ↓ +Config struct: + - bind_address: "127.0.0.1:8080" + - domain: "example.com" + - git_data_path: "./data/repos" + - relay_data_path: "./data/relay" + - relay_name: "..." + - relay_description: "..." + - owner_npub: "..." + ↓ +Passed to: + - HTTP server (bind address) + - Git handler (git_data_path, domain) + - Nostr handler (relay_data_path, domain, NIP-11 info) + - Storage layer (both paths) +``` + +--- + +## Test Architecture + +``` +Integration Test + ↓ +TestRelay::start() + ↓ +Spawns ngit-grasp process: + - NGIT_BIND_ADDRESS=127.0.0.1:{random_port} + - NGIT_DOMAIN=127.0.0.1:{random_port} + - NGIT_GIT_DATA_PATH=./test-data/repos + - NGIT_RELAY_DATA_PATH=./test-data/relay + ↓ +Process starts: + - actix-web listens on random port + - Both Git and Nostr available + ↓ +Test runs: + - Uses grasp-audit library + - Connects to ws://127.0.0.1:{port}/ + - Runs compliance tests + ↓ +TestRelay::stop() + ↓ +Process killed + ↓ +Test data cleaned up +``` + +--- + +## File Structure + +``` +ngit-grasp/ +├── src/ +│ ├── main.rs # Entry point +│ ├── config.rs # Configuration +│ ├── http/ # NEW - HTTP server +│ │ ├── mod.rs # Server setup +│ │ ├── git.rs # Git HTTP handler +│ │ └── nostr.rs # Nostr WebSocket handler +│ ├── nostr/ +│ │ ├── mod.rs +│ │ ├── relay.rs # Relay logic (reused) +│ │ └── events.rs # Event handling +│ └── storage/ +│ ├── mod.rs +│ └── repository.rs # Git repo management +├── tests/ +│ ├── common/ +│ │ ├── mod.rs +│ │ └── relay.rs # TestRelay fixture +│ ├── nip01_compliance.rs # NIP-01 tests +│ ├── nip34_announcements.rs # NIP-34 tests +│ └── grasp01_git_http.rs # NEW - GRASP-01 Git tests +├── data/ # Runtime data (gitignored) +│ ├── repos/ # Git repositories +│ └── relay/ # Nostr events +└── test-data/ # Test data (gitignored) + ├── repos/ + └── relay/ +``` + +--- + +## Comparison: ngit-relay vs ngit-grasp + +### ngit-relay (Go + nginx) + +``` +nginx (Port 8081) + ├── Git HTTP → fcgiwrap → git-http-backend + │ ↓ + │ pre-receive hook (Go) + │ ↓ + │ Khatru relay (HTTP API) + │ + └── Nostr → proxy → Khatru relay (Port 3334) + ↓ + on_event hook (Go) + ↓ + provision repos +``` + +**Components:** +- nginx (routing) +- fcgiwrap (CGI wrapper) +- git-http-backend (Git protocol) +- pre-receive hook (Go, validates pushes) +- post-receive hook (Go, updates HEAD) +- Khatru relay (Go, Nostr protocol) +- on_event hook (Go, provisions repos) +- supervisord (process management) + +### ngit-grasp (Rust) + +``` +actix-web (Port 8080) + ├── Git HTTP → git-http-backend crate + │ ↓ + │ inline authorization + │ ↓ + │ Storage (query state) + │ + └── Nostr → WebSocket upgrade + ↓ + nostr-sdk relay + ↓ + on_event (provision repos) + ↓ + Storage (store events) +``` + +**Components:** +- actix-web (routing + HTTP + WebSocket) +- git-http-backend crate (Git protocol) +- nostr-sdk (Nostr protocol) +- Storage (unified storage layer) + +**Advantages:** +- Single binary +- No external processes +- Inline authorization (better errors) +- Pure Rust (memory safety) +- Easier testing + +--- + +**Last Updated:** November 4, 2025 +**Purpose:** Reference for implementation diff --git a/docs/archive/2025-11-04-evening/current_status.md b/docs/archive/2025-11-04-evening/current_status.md new file mode 100644 index 0000000..f14c391 --- /dev/null +++ b/docs/archive/2025-11-04-evening/current_status.md @@ -0,0 +1,443 @@ +# Current Status - ngit-grasp Implementation + +**Date:** November 4, 2025 +**Status:** In Development - GRASP-01 Core Requirements + +--- + +## 🎯 Project Goal + +Implement a **GRASP-01 compliant** Git relay service in Rust that: +- Serves a NIP-01 Nostr relay at `/` (WebSocket) +- Serves Git repositories via Git Smart HTTP at `//.git` +- **Both on the SAME PORT** (critical requirement!) +- Validates pushes against Nostr state events +- Passes all compliance tests from grasp-audit + +--- + +## 📋 GRASP-01 Requirements (from ../grasp/01.md) + +### 1. Nostr Relay Requirements + +**MUST:** +- ✅ Serve NIP-01 compliant relay at `/` (WebSocket) +- ✅ Accept NIP-34 repository announcements (kind 30617) +- ✅ Accept NIP-34 state announcements (kind 30618) +- ⏳ Reject announcements that don't list this service in `clone` and `relays` tags +- ⏳ Accept events that tag accepted announcements +- ✅ Serve NIP-11 relay information document +- ⏳ Include `supported_grasps`, `repo_acceptance_criteria`, `curation` in NIP-11 + +**Current Implementation:** +- Basic WebSocket relay working +- Event storage and querying functional +- NIP-11 basic implementation exists +- **Missing:** Announcement validation against service URL +- **Missing:** Event acceptance policy based on announcements + +### 2. Git Smart HTTP Service Requirements + +**MUST:** +- ❌ Serve Git repos at `//.git` via unauthenticated Git Smart HTTP +- ❌ Accept pushes matching latest state announcement (respecting maintainer set) +- ❌ Set repository HEAD per state announcement +- ❌ Accept pushes to `refs/nostr/` for PRs +- ❌ Include `allow-reachable-sha1-in-want` and `allow-tip-sha1-in-want` +- ❌ Serve webpage at repo endpoint for browsers + +**Current Implementation:** +- **NOT STARTED** - Git HTTP backend not integrated +- No Git repository management +- No push validation + +### 3. CORS Support Requirements + +**MUST:** +- ❌ Set `Access-Control-Allow-Origin: *` on ALL responses +- ❌ Set `Access-Control-Allow-Methods: GET, POST` on ALL responses +- ❌ Set `Access-Control-Allow-Headers: Content-Type` on ALL responses +- ❌ Respond to OPTIONS requests with 204 No Content + +**Current Implementation:** +- **NOT STARTED** - No CORS headers + +--- + +## 🏗️ Architecture Understanding (from ngit-relay) + +### Critical Architecture Insight: SINGLE PORT + +From `../ngit-relay/docker-compose.yml`: +```yaml +ports: + - "8081:8081" # Single port for EVERYTHING +``` + +From `../ngit-relay/src/nginx.conf`: +```nginx +server { + listen 8081; # Single listener + + # Git repos at //.git + location ~ ^/npub1([a-z0-9]+)/([^/]+\.git)(/.*)?$ { + # ... git-http-backend via fcgiwrap + } + + # Nostr relay at / + location / { + # ... proxy to khatru on localhost:3334 + } +} +``` + +**Key Points:** +1. **nginx listens on ONE port (8081)** +2. **nginx routes by URL path:** + - `//.git/*` → git-http-backend (fcgiwrap) + - Everything else → Khatru relay (localhost:3334) +3. **Khatru relay runs on INTERNAL port 3334** +4. **git-http-backend runs via fcgiwrap socket** + +### Our Rust Implementation Strategy + +We need to replicate nginx's routing in Rust: + +``` +HTTP/WebSocket Request on port 8080 + ↓ + actix-web router + ↓ + ┌────┴────┐ + ↓ ↓ +Git Path Other Path +// / +.git + ↓ ↓ +git-http Nostr Relay +backend (WebSocket upgrade) +handler +``` + +**Implementation Options:** + +**Option A: actix-web (HTTP framework)** +- Handle HTTP/WebSocket on same port +- Route by path pattern +- Use `git-http-backend` crate for Git protocol +- Native WebSocket support for Nostr relay + +**Option B: Direct TCP + Manual Routing** +- Accept TCP connections +- Parse HTTP headers to determine route +- More complex but more control + +**Recommendation: Option A (actix-web)** +- Well-tested HTTP/WebSocket handling +- Easy routing by path +- Good async performance +- Already in our dependencies + +--- + +## 🧪 Test Strategy + +### Current Test Structure + +``` +tests/ +├── common/ +│ ├── mod.rs # Test utilities +│ └── relay.rs # TestRelay fixture +├── nip01_compliance.rs # NIP-01 smoke tests +└── nip34_announcements.rs # NIP-34 tests (TODO) +``` + +### Test Approach + +**Integration Tests (tests/*):** +- Use `TestRelay` fixture to start/stop relay +- Use `grasp-audit` library to run compliance tests +- Tests reference GRASP protocol line numbers +- Automatic relay lifecycle management + +**Example Test Structure:** +```rust +#[tokio::test] +async fn test_grasp01_git_http_basic() { + // Reference: ../grasp/01.md lines 15-17 + // Requirement: MUST serve git repository via unauthenticated git smart http + + let relay = TestRelay::start().await; + let config = AuditConfig::ci(); + let client = AuditClient::new(relay.url(), config).await.unwrap(); + + // Run GRASP-01 git HTTP tests + let results = specs::Grasp01GitHttp::run_all(&client).await; + + relay.stop().await; + assert!(results.all_passed()); +} +``` + +### Test Coverage Needed + +**NIP-01 (Nostr Relay):** +- ✅ WebSocket connection +- ✅ Send/receive events +- ✅ Subscriptions (REQ/CLOSE) +- ✅ Event validation (signatures, IDs) +- ⏳ NIP-11 relay info document + +**NIP-34 (Git Announcements):** +- ⏳ Accept valid repository announcements (kind 30617) +- ⏳ Accept valid state announcements (kind 30618) +- ⏳ Reject announcements without service in clone/relays +- ⏳ Validate maintainer sets +- ⏳ Handle related events (issues, patches) + +**GRASP-01 (Git HTTP):** +- ❌ Serve Git repo at `//.git` +- ❌ Clone repository via HTTP +- ❌ Push matching state announcement +- ❌ Reject push not matching state +- ❌ Handle `refs/nostr/` for PRs +- ❌ CORS headers on all responses +- ❌ OPTIONS request handling + +--- + +## 📝 Implementation Plan + +### Phase 1: Fix Current Relay (In Progress) + +**Goal:** Make NIP-01 relay fully compliant + +**Tasks:** +- [x] Basic WebSocket relay working +- [x] Event storage and querying +- [ ] NIP-11 relay info with GRASP fields + - [ ] Add `supported_grasps: ["GRASP-01"]` + - [ ] Add `repo_acceptance_criteria` + - [ ] Add `curation` policy +- [ ] Announcement validation + - [ ] Check `clone` tag includes our domain + - [ ] Check `relays` tag includes our domain + - [ ] Reject if not listed (unless GRASP-05) +- [ ] Event acceptance policy + - [ ] Accept events tagging accepted announcements + - [ ] Accept events tagged by accepted announcements + +**Test Coverage:** +- [x] NIP-01 smoke tests passing +- [ ] NIP-11 compliance tests +- [ ] NIP-34 announcement tests + +### Phase 2: Add Git HTTP Backend (Next) + +**Goal:** Serve Git repositories via HTTP on same port as relay + +**Tasks:** +1. **Integrate actix-web** + - [ ] Replace raw WebSocket with actix-web + - [ ] Add HTTP routing + - [ ] Preserve WebSocket upgrade for `/` + - [ ] Add Git HTTP route for `//.git` + +2. **Integrate git-http-backend crate** + - [ ] Add dependency on `git-http-backend` + - [ ] Create Git handler for `//.git` + - [ ] Serve `git-upload-pack` (clone/fetch) + - [ ] Serve `git-receive-pack` (push) + +3. **Repository Management** + - [ ] Auto-provision repos from announcements + - [ ] Store repos at `{GIT_DATA_PATH}//.git` + - [ ] Initialize bare repositories + - [ ] Set HEAD from state announcements + +4. **CORS Support** + - [ ] Add CORS middleware to actix-web + - [ ] Set required headers on all responses + - [ ] Handle OPTIONS requests + +**Test Coverage:** +- [ ] Can clone repository via HTTP +- [ ] Can fetch from repository +- [ ] Repository provisioned from announcement +- [ ] HEAD set correctly from state +- [ ] CORS headers present +- [ ] OPTIONS requests handled + +### Phase 3: Push Authorization (Final) + +**Goal:** Validate pushes against Nostr state announcements + +**Tasks:** +1. **Inline Authorization** + - [ ] Intercept `git-receive-pack` before Git process + - [ ] Parse ref updates from request + - [ ] Query latest state announcement from relay + - [ ] Validate push matches state + - [ ] Handle maintainer sets (recursive) + - [ ] Return HTTP error if validation fails + +2. **PR Support** + - [ ] Accept pushes to `refs/nostr/` + - [ ] Validate PR event exists on relay + - [ ] Validate ref tip matches PR event `c` tag + - [ ] Implement 20-minute timeout for PR refs + - [ ] Garbage collect orphaned PR refs + +3. **State Synchronization** + - [ ] Update HEAD when state announcement received + - [ ] Handle state updates for existing repos + - [ ] Handle multi-maintainer scenarios + +**Test Coverage:** +- [ ] Push matching state succeeds +- [ ] Push not matching state fails +- [ ] Multi-maintainer push validation +- [ ] PR ref push/validation +- [ ] PR ref garbage collection +- [ ] State update triggers HEAD change + +--- + +## 🐛 Known Issues + +### 1. Architecture Mismatch +**Issue:** Tests assume relay on one port, Git on another +**Fix:** Both must be on same port (like ngit-relay) +**Impact:** Need to refactor server architecture + +### 2. Missing Git Implementation +**Issue:** No Git HTTP backend integrated +**Fix:** Add actix-web + git-http-backend +**Impact:** Core GRASP-01 requirement not met + +### 3. No Announcement Validation +**Issue:** Relay accepts all announcements +**Fix:** Validate `clone` and `relays` tags +**Impact:** Not GRASP-01 compliant + +### 4. No CORS Support +**Issue:** No CORS headers on responses +**Fix:** Add CORS middleware +**Impact:** Web clients can't access relay + +--- + +## 🔧 Environment Configuration + +From `../ngit-relay/.env.example`, we need: + +```bash +# Service Configuration +NGIT_DOMAIN=example.com # Used for announcement validation +NGIT_BIND_ADDRESS=127.0.0.1:8080 # Single port for HTTP/WS/Git + +# Relay Information (NIP-11) +NGIT_RELAY_NAME="ngit-grasp instance" +NGIT_RELAY_DESCRIPTION="Rust GRASP implementation" +NGIT_OWNER_NPUB="npub1..." # Relay owner + +# Storage Paths +NGIT_GIT_DATA_PATH=/srv/ngit-grasp/repos # Git repositories +NGIT_RELAY_DATA_PATH=/srv/ngit-grasp/relay-db # Nostr events + +# Features (Future) +NGIT_PROACTIVE_SYNC_GIT=false # GRASP-02 +NGIT_PROACTIVE_SYNC_NOSTR=false # GRASP-02 + +# Logging +NGIT_LOG_LEVEL=INFO +``` + +**Current .env.example status:** +- ⏳ Needs update with all required fields +- ⏳ Add GRASP-specific configuration +- ⏳ Document which fields are used where + +--- + +## 📊 Progress Summary + +### Completed ✅ +- Basic Nostr relay (WebSocket) +- Event storage and querying +- NIP-01 smoke tests +- Test infrastructure (TestRelay fixture) +- Integration with grasp-audit library + +### In Progress ⏳ +- NIP-11 relay information +- NIP-34 announcement handling +- Event acceptance policies + +### Not Started ❌ +- Git HTTP backend +- Repository provisioning +- Push authorization +- CORS support +- actix-web integration + +### Compliance Status +- **NIP-01:** ~60% (basic relay works, missing some features) +- **NIP-34:** ~20% (can store events, no validation) +- **GRASP-01:** ~30% (relay works, Git HTTP not started) + +--- + +## 🎯 Next Session Priorities + +1. **Fix Architecture** (CRITICAL) + - Integrate actix-web for HTTP/WebSocket routing + - Single port for all services + - Preserve existing relay functionality + +2. **Add Git HTTP** (HIGH) + - Integrate `git-http-backend` crate + - Basic clone/fetch support + - Repository provisioning from announcements + +3. **Update Tests** (HIGH) + - Add GRASP-01 Git HTTP tests + - Reference protocol line numbers + - Verify single-port architecture + +4. **Fix NIP-11** (MEDIUM) + - Add GRASP-specific fields + - Document compliance level + - Include in tests + +--- + +## 📚 Key References + +**GRASP Protocol:** +- `../grasp/README.md` - Overview +- `../grasp/01.md` - GRASP-01 Core Requirements (THE SPEC) +- `../grasp/02.md` - GRASP-02 Proactive Sync +- `../grasp/05.md` - GRASP-05 Archive + +**Reference Implementation:** +- `../ngit-relay/README.md` - Architecture overview +- `../ngit-relay/src/nginx.conf` - **CRITICAL: Shows single-port routing** +- `../ngit-relay/docker-compose.yml` - **CRITICAL: Shows port config** +- `../ngit-relay/.env.example` - Configuration template + +**Nostr Specs:** +- [NIP-01](https://nips.nostr.com/1) - Basic protocol +- [NIP-11](https://nips.nostr.com/11) - Relay information +- [NIP-34](https://nips.nostr.com/34) - Git stuff + +**Our Code:** +- `tests/nip01_compliance.rs` - Current test approach +- `tests/common/relay.rs` - TestRelay fixture +- `grasp-audit/src/specs/nip01_smoke.rs` - Test specs + +--- + +**Last Updated:** November 4, 2025 +**Next Review:** After actix-web integration diff --git a/docs/archive/2025-11-04-evening/implementation-checklist.md b/docs/archive/2025-11-04-evening/implementation-checklist.md new file mode 100644 index 0000000..64f5f4e --- /dev/null +++ b/docs/archive/2025-11-04-evening/implementation-checklist.md @@ -0,0 +1,720 @@ +# Implementation Checklist + +**Date:** November 4, 2025 +**Purpose:** Step-by-step checklist for actix-web integration + +--- + +## ✅ Pre-Implementation (DONE) + +- [x] Review GRASP-01 specification +- [x] Review ngit-relay reference implementation +- [x] Understand single-port architecture +- [x] Document architecture in work/architecture-diagram.md +- [x] Create detailed plan in work/NEXT_SESSION_START_HERE.md +- [x] Update work/current_status.md + +--- + +## 📦 Phase 1: Dependencies & Setup + +### 1.1 Update Cargo.toml + +- [ ] Add `actix-web = "4"` +- [ ] Add `actix-cors = "0.7"` +- [ ] Add `actix-ws = "0.3"` (or use actix-web-actors) +- [ ] Add `git-http-backend = "0.2"` (check latest version) +- [ ] Run `cargo check` to verify dependencies + +**Verification:** +```bash +cargo tree | grep actix +cargo tree | grep git-http-backend +``` + +### 1.2 Update .env.example (if needed) + +- [x] Already has all required fields +- [x] NGIT_DOMAIN +- [x] NGIT_BIND_ADDRESS +- [x] NGIT_GIT_DATA_PATH +- [x] NGIT_RELAY_DATA_PATH + +**Verification:** +```bash +cat .env.example +``` + +--- + +## 🏗️ Phase 2: HTTP Server Module + +### 2.1 Create src/http/mod.rs + +- [ ] Create module structure +- [ ] Add `pub mod git;` +- [ ] Add `pub mod nostr;` +- [ ] Create `run_server()` function +- [ ] Set up actix-web HttpServer +- [ ] Add CORS middleware +- [ ] Add routing for Git and Nostr + +**Verification:** +```bash +cargo check +# Should compile without errors +``` + +**Test:** +```rust +// In src/http/mod.rs +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_module_exists() { + // Just verify module structure + assert!(true); + } +} +``` + +### 2.2 Create src/http/git.rs + +- [ ] Create `handle_git_request()` function +- [ ] Parse npub and repo from path +- [ ] Construct repository path +- [ ] Check if repository exists (return 404 if not) +- [ ] Use git-http-backend crate +- [ ] Handle GET (clone/fetch) +- [ ] Handle POST (push) +- [ ] Return proper HTTP responses + +**Verification:** +```bash +cargo check +# Should compile +``` + +**Test:** +```rust +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_parse_repo_path() { + // Test path parsing logic + let path = "/npub1abc.../my-repo.git"; + // ... verify parsing + } +} +``` + +### 2.3 Create src/http/nostr.rs + +- [ ] Create `handle_websocket()` function +- [ ] Handle WebSocket upgrade +- [ ] Reuse existing Nostr message handling +- [ ] Create `handle_http_root()` function +- [ ] Serve HTML for browsers +- [ ] Serve NIP-11 JSON for Accept: application/nostr+json + +**Verification:** +```bash +cargo check +``` + +**Test:** +```rust +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_nip11_response() { + // Test NIP-11 JSON generation + // ... + } +} +``` + +--- + +## 🔧 Phase 3: Update Existing Code + +### 3.1 Update src/config.rs + +- [x] Already has `git_data_path` field +- [x] Already has `from_env()` implementation +- [ ] Verify all fields are present +- [ ] Add any missing validation + +**Verification:** +```bash +cargo check +``` + +### 3.2 Update src/main.rs + +- [ ] Remove direct relay start +- [ ] Import `http` module +- [ ] Call `http::run_server(config, storage).await` +- [ ] Update logging messages + +**Verification:** +```bash +cargo build +# Should build successfully +``` + +**Test:** +```bash +# Run server +NGIT_DOMAIN=localhost:8080 \ +NGIT_BIND_ADDRESS=127.0.0.1:8080 \ +cargo run + +# In another terminal, check it's listening +curl -v http://localhost:8080/ +``` + +### 3.3 Move Relay Logic to Library + +- [ ] Extract relay logic from src/nostr/relay.rs +- [ ] Make it reusable by WebSocket handler +- [ ] Keep message handling separate from transport +- [ ] Create `handle_nostr_message()` function + +**Structure:** +```rust +// src/nostr/relay.rs +pub async fn handle_nostr_message( + message: &str, + storage: &Storage, +) -> Result> { + // Parse message + // Handle EVENT, REQ, CLOSE + // Return response messages +} +``` + +**Verification:** +```bash +cargo check +cargo test --lib +``` + +--- + +## 🧪 Phase 4: Update Tests + +### 4.1 Update tests/common/relay.rs + +- [ ] Verify NGIT_DOMAIN is set correctly +- [ ] Add NGIT_GIT_DATA_PATH env var +- [ ] Add NGIT_RELAY_DATA_PATH env var +- [ ] Use test-specific directories +- [ ] Clean up test data after tests + +**Current Status:** Already sets NGIT_DOMAIN correctly! + +**Add:** +```rust +.env("NGIT_GIT_DATA_PATH", "./test-data/repos") +.env("NGIT_RELAY_DATA_PATH", "./test-data/relay") +``` + +**Verification:** +```bash +cargo test --test nip01_compliance +# Should still pass +``` + +### 4.2 Create tests/grasp01_git_http.rs + +- [ ] Create new test file +- [ ] Add basic Git clone test +- [ ] Add CORS headers test +- [ ] Add OPTIONS request test +- [ ] Add repository not found test +- [ ] Reference GRASP-01 line numbers in comments + +**Template:** +```rust +//! GRASP-01 Git HTTP Integration Tests +//! +//! Reference: ../grasp/01.md lines 15-40 + +mod common; + +use common::TestRelay; + +#[tokio::test] +async fn test_git_http_basic() { + // Reference: ../grasp/01.md line 15 + // MUST serve git repository via unauthenticated git smart http + + let relay = TestRelay::start().await; + + // TODO: Create test repo + // TODO: Try to clone it + + relay.stop().await; +} +``` + +**Verification:** +```bash +cargo test --test grasp01_git_http +``` + +### 4.3 Update tests/nip01_compliance.rs + +- [ ] Verify tests still pass with new architecture +- [ ] Update any broken tests +- [ ] Add comments referencing GRASP-01 where relevant + +**Verification:** +```bash +cargo test --test nip01_compliance +``` + +--- + +## 🔍 Phase 5: Integration & Testing + +### 5.1 Manual Testing + +**Test 1: Server Starts** +```bash +cargo build +NGIT_DOMAIN=localhost:8080 \ +NGIT_BIND_ADDRESS=127.0.0.1:8080 \ +cargo run +``` + +**Expected:** Server starts without errors + +--- + +**Test 2: WebSocket Connection** +```bash +# In grasp-audit directory +cargo run -- --url ws://localhost:8080 +``` + +**Expected:** NIP-01 smoke tests pass + +--- + +**Test 3: HTTP Root** +```bash +curl -v http://localhost:8080/ +``` + +**Expected:** +- Status: 200 OK +- Content-Type: text/html +- CORS headers present +- HTML content + +--- + +**Test 4: NIP-11** +```bash +curl -v http://localhost:8080/ \ + -H "Accept: application/nostr+json" +``` + +**Expected:** +- Status: 200 OK +- Content-Type: application/json +- CORS headers present +- JSON with `supported_grasps` field + +--- + +**Test 5: Git Repository (404)** +```bash +curl -v http://localhost:8080/npub1test/test-repo.git/info/refs?service=git-upload-pack +``` + +**Expected:** +- Status: 404 Not Found +- CORS headers present + +--- + +**Test 6: Git Repository (Success)** +```bash +# Create test repo +mkdir -p ./data/repos/npub1test +cd ./data/repos/npub1test +git init --bare test-repo.git + +# Try to access it +curl -v http://localhost:8080/npub1test/test-repo.git/info/refs?service=git-upload-pack +``` + +**Expected:** +- Status: 200 OK +- Content-Type: application/x-git-upload-pack-advertisement +- CORS headers present +- Git protocol data + +--- + +**Test 7: Git Clone** +```bash +git clone http://localhost:8080/npub1test/test-repo.git /tmp/test-clone +``` + +**Expected:** +- Clone succeeds (even if empty repo) +- No errors + +--- + +**Test 8: CORS Preflight** +```bash +curl -v -X OPTIONS http://localhost:8080/ \ + -H "Origin: https://example.com" \ + -H "Access-Control-Request-Method: POST" +``` + +**Expected:** +- Status: 204 No Content +- Access-Control-Allow-Origin: * +- Access-Control-Allow-Methods: GET, POST +- Access-Control-Allow-Headers: Content-Type +- Access-Control-Max-Age: 3600 + +--- + +### 5.2 Automated Testing + +**Run All Tests:** +```bash +# Build first +cargo build + +# Run all tests +cargo test + +# Run specific test suites +cargo test --test nip01_compliance +cargo test --test grasp01_git_http + +# With output +cargo test -- --nocapture +``` + +**Expected:** All tests pass + +--- + +### 5.3 Performance Testing + +**Test Concurrent Connections:** +```bash +# Start server +cargo run & + +# Run multiple clients +for i in {1..10}; do + (cd grasp-audit && cargo run -- --url ws://localhost:8080) & +done + +# Wait for all to complete +wait +``` + +**Expected:** All clients connect and pass tests + +--- + +### 5.4 Error Handling Testing + +**Test 1: Invalid Repository Path** +```bash +curl -v http://localhost:8080/invalid/path +``` + +**Expected:** 404 or appropriate error + +--- + +**Test 2: Invalid WebSocket Message** +```bash +# Use websocat or similar +echo "invalid json" | websocat ws://localhost:8080/ +``` + +**Expected:** NOTICE message with error + +--- + +**Test 3: Large Git Push** +```bash +# Create repo with large files +# Try to push +# Verify it works or fails gracefully +``` + +--- + +## 📋 Acceptance Criteria + +### Must Have (MVP) + +- [ ] Server starts on single port +- [ ] WebSocket connects at `/` +- [ ] NIP-01 smoke tests pass +- [ ] Can access Git repo at `//.git` +- [ ] Returns 404 for missing repos +- [ ] CORS headers on all responses +- [ ] OPTIONS requests return 204 +- [ ] Can clone existing Git repository +- [ ] All integration tests pass + +### Should Have (Before Production) + +- [ ] Can push to repository (basic, no auth yet) +- [ ] Repository provisioned from announcement +- [ ] NIP-11 includes GRASP fields +- [ ] Proper error messages +- [ ] Logging works correctly +- [ ] Clean shutdown +- [ ] Test data cleanup + +### Could Have (Future) + +- [ ] Push authorization +- [ ] Maintainer set validation +- [ ] PR ref support +- [ ] State synchronization +- [ ] Proactive sync (GRASP-02) + +--- + +## 🐛 Known Issues to Watch For + +### Issue 1: WebSocket Upgrade Timing + +**Symptom:** WebSocket upgrade fails intermittently + +**Debug:** +```bash +RUST_LOG=debug cargo run +# Check for upgrade-related logs +``` + +**Solution:** Ensure actix-ws is configured correctly + +--- + +### Issue 2: Git HTTP Protocol Errors + +**Symptom:** Git clone fails with protocol error + +**Debug:** +```bash +GIT_TRACE_PACKET=1 git clone http://localhost:8080/... +# Shows Git protocol messages +``` + +**Solution:** Check git-http-backend configuration + +--- + +### Issue 3: CORS Not Applied + +**Symptom:** Browser shows CORS error + +**Debug:** +```bash +curl -v http://localhost:8080/ -H "Origin: https://example.com" +# Check response headers +``` + +**Solution:** Verify CORS middleware is first in chain + +--- + +### Issue 4: Port Already in Use + +**Symptom:** "Address already in use" error + +**Debug:** +```bash +lsof -i :8080 +# Find process using port +``` + +**Solution:** +```bash +kill -9 +# Or use different port +``` + +--- + +### Issue 5: Test Relay Won't Start + +**Symptom:** Integration tests fail to start relay + +**Debug:** +```bash +# Run test with output +cargo test --test nip01_compliance -- --nocapture + +# Check binary exists +ls -la target/debug/ngit-grasp +``` + +**Solution:** Run `cargo build` before tests + +--- + +## 📚 Reference Commands + +### Development + +```bash +# Build +cargo build + +# Run +cargo run + +# Run with logging +RUST_LOG=debug cargo run + +# Check without building +cargo check + +# Format code +cargo fmt + +# Lint +cargo clippy +``` + +### Testing + +```bash +# All tests +cargo test + +# Specific test file +cargo test --test nip01_compliance + +# Specific test +cargo test --test nip01_compliance test_nip01_smoke + +# With output +cargo test -- --nocapture + +# With logging +RUST_LOG=debug cargo test -- --nocapture +``` + +### Debugging + +```bash +# Check dependencies +cargo tree + +# Check for unused dependencies +cargo +nightly udeps + +# Check for outdated dependencies +cargo outdated + +# Audit for security issues +cargo audit +``` + +### Git Testing + +```bash +# Create test repo +mkdir -p ./data/repos/npub1test +cd ./data/repos/npub1test +git init --bare test-repo.git + +# Clone it +git clone http://localhost:8080/npub1test/test-repo.git /tmp/test + +# Push to it +cd /tmp/test +echo "test" > README.md +git add . +git commit -m "test" +git push origin main +``` + +--- + +## ✅ Completion Checklist + +When all items are checked, Phase 1 (actix-web integration) is complete: + +### Code + +- [ ] Dependencies added to Cargo.toml +- [ ] src/http/mod.rs created +- [ ] src/http/git.rs created +- [ ] src/http/nostr.rs created +- [ ] src/main.rs updated +- [ ] src/config.rs verified +- [ ] Relay logic refactored + +### Tests + +- [ ] tests/common/relay.rs updated +- [ ] tests/grasp01_git_http.rs created +- [ ] tests/nip01_compliance.rs still passes +- [ ] All tests pass + +### Manual Testing + +- [ ] Server starts successfully +- [ ] WebSocket connects +- [ ] NIP-01 smoke tests pass +- [ ] Can access Git repos +- [ ] 404 for missing repos +- [ ] CORS headers present +- [ ] OPTIONS requests work +- [ ] Can clone repository + +### Documentation + +- [ ] Update README.md status +- [ ] Update work/current_status.md +- [ ] Document any issues found +- [ ] Update NEXT_SESSION_START_HERE.md for next phase + +--- + +## 🎯 Next Phase Preview + +After actix-web integration is complete, next phase will be: + +**Phase 2: Repository Provisioning** + +- Listen for NIP-34 repository announcements +- Create Git repositories automatically +- Initialize bare repositories +- Set up directory structure +- Handle repository deletion + +**Estimated Time:** 2-3 hours +**Prerequisites:** Phase 1 complete + +--- + +**Last Updated:** November 4, 2025 +**Status:** Ready to begin Phase 1 diff --git a/docs/archive/2025-11-04-evening/review-summary.md b/docs/archive/2025-11-04-evening/review-summary.md new file mode 100644 index 0000000..29a3ff4 --- /dev/null +++ b/docs/archive/2025-11-04-evening/review-summary.md @@ -0,0 +1,513 @@ +# GRASP Protocol Review Summary + +**Date:** November 4, 2025 +**Purpose:** Document key findings from reviewing GRASP protocol and ngit-relay + +--- + +## 🎯 Critical Discoveries + +### 1. Single Port Architecture (CRITICAL!) + +**Finding:** Git server and Nostr relay MUST run on the SAME port. + +**Evidence:** +```yaml +# ../ngit-relay/docker-compose.yml +ports: + - "8081:8081" # Single port only! +``` + +```nginx +# ../ngit-relay/src/nginx.conf +server { + listen 8081; # One listener for everything + + location ~ ^/npub1([a-z0-9]+)/([^/]+\.git)(/.*)?$ { + # Git HTTP via fcgiwrap + } + + location / { + # Nostr relay via proxy to localhost:3334 + } +} +``` + +**Impact:** +- Our current architecture is WRONG +- We assumed separate ports (relay on 8080, git on 8081) +- Must use HTTP router to split traffic by path +- actix-web can handle this + +**Action Required:** +- Integrate actix-web for HTTP routing +- Route `//.git` to Git handler +- Route `/` to Nostr relay (WebSocket upgrade) +- Apply CORS to ALL routes + +--- + +### 2. GRASP-01 Test Requirements + +**Finding:** Tests must closely map to GRASP protocol specification. + +**GRASP-01 Requirements (from ../grasp/01.md):** + +#### Nostr Relay (Lines 1-14) +- ✅ Serve NIP-01 relay at `/` (WebSocket) +- ⏳ Accept NIP-34 repository announcements (kind 30617) +- ⏳ Accept NIP-34 state announcements (kind 30618) +- ⏳ Reject announcements without service in `clone` and `relays` tags +- ⏳ Accept events that tag accepted announcements +- ✅ Serve NIP-11 relay information +- ⏳ Include `supported_grasps`, `repo_acceptance_criteria`, `curation` in NIP-11 + +#### Git Smart HTTP (Lines 15-31) +- ❌ Serve repos at `//.git` +- ❌ Accept pushes matching state announcements +- ❌ Respect recursive maintainer sets +- ❌ Set HEAD per state announcement +- ❌ Accept pushes to `refs/nostr/` for PRs +- ❌ Include `allow-reachable-sha1-in-want` and `allow-tip-sha1-in-want` +- ❌ Serve webpage for browsers + +#### CORS Support (Lines 32-40) +- ❌ `Access-Control-Allow-Origin: *` on ALL responses +- ❌ `Access-Control-Allow-Methods: GET, POST` on ALL responses +- ❌ `Access-Control-Allow-Headers: Content-Type` on ALL responses +- ❌ Respond to OPTIONS with 204 No Content + +**Action Required:** +- Create test for each requirement +- Reference GRASP-01 line numbers in test comments +- Example: + ```rust + #[tokio::test] + async fn test_git_http_basic() { + // Reference: ../grasp/01.md line 15 + // MUST serve git repository via unauthenticated git smart http + // ... + } + ``` + +--- + +### 3. Environment Variables + +**Finding:** ngit-relay uses specific environment variables we should match. + +**From ../ngit-relay/.env.example:** + +```bash +# Service Configuration +NGIT_DOMAIN=example.com # For announcement validation +NGIT_INTERNAL_RELAY_PORT_FOR_SSL_PROXY=8081 # We don't need this + +# Relay Information (NIP-11) +NGIT_RELAY_NAME="..." +NGIT_RELAY_DESCRIPTION="..." +NGIT_OWNER_NPUB="..." + +# Features +NGIT_PROACTIVE_SYNC_GIT=true # GRASP-02 (future) +NGIT_PROACTIVE_SYNC_BLOSSOM=true # Not in GRASP +NGIT_PROACTIVE_SYNC_NOSTR=true # GRASP-02 (future) + +# Blossom Settings +NGIT_BLOSSOM_MAX_FILE_SIZE_MB=100 # Not in GRASP +NGIT_BLOSSOM_MAX_CAPACITY_GB=50 # Not in GRASP + +# Logging +NGIT_LOG_DIR=/var/log/ngit-relay +NGIT_LOG_LEVEL=INFO +NGIT_LOG_MAX_SIZE_MB=20 +NGIT_LOG_MAX_BACKUPS=10 +NGIT_LOG_MAX_AGE_DAYS=30 +``` + +**Our Environment Variables:** + +```bash +# Service Configuration +NGIT_DOMAIN=example.com # REQUIRED - for announcement validation +NGIT_BIND_ADDRESS=127.0.0.1:8080 # REQUIRED - single port + +# Relay Information (NIP-11) +NGIT_RELAY_NAME="ngit-grasp instance" +NGIT_RELAY_DESCRIPTION="Rust GRASP implementation" +NGIT_OWNER_NPUB="npub1..." + +# Storage Paths +NGIT_GIT_DATA_PATH=./data/repos # REQUIRED - where to store Git repos +NGIT_RELAY_DATA_PATH=./data/relay # REQUIRED - where to store events + +# Logging +NGIT_LOG_LEVEL=INFO +RUST_LOG=info # Standard Rust logging +``` + +**Action Required:** +- Update `.env.example` with all required fields +- Add `NGIT_GIT_DATA_PATH` to config +- Document which fields are required vs. optional + +--- + +### 4. Repository Path Structure + +**Finding:** Repository storage follows specific pattern. + +**Pattern:** `{GIT_DATA_PATH}/{npub}/{identifier}.git` + +**Example:** +``` +./data/repos/ +├── npub1abc.../ +│ ├── my-project.git/ +│ │ ├── HEAD +│ │ ├── config +│ │ ├── objects/ +│ │ └── refs/ +│ └── another-repo.git/ +└── npub1xyz.../ + └── their-project.git/ +``` + +**Action Required:** +- Create repository directory structure +- Initialize bare repositories (`git init --bare`) +- Set ownership/permissions correctly +- Clean up on repository deletion + +--- + +### 5. NIP-11 GRASP Fields + +**Finding:** NIP-11 relay information must include GRASP-specific fields. + +**From ../grasp/01.md lines 11-14:** + +```json +{ + "name": "ngit-grasp instance", + "description": "Rust GRASP implementation", + "pubkey": "...", + "contact": "...", + "supported_nips": [1, 11, 34], + "supported_grasps": ["GRASP-01"], // NEW - array of strings + "repo_acceptance_criteria": "...", // NEW - human readable + "curation": "WoT-based spam prevention" // NEW - optional +} +``` + +**Action Required:** +- Add `supported_grasps` field to NIP-11 response +- Add `repo_acceptance_criteria` field +- Add `curation` field (optional) +- Update NIP-11 tests to verify these fields + +--- + +### 6. Announcement Validation + +**Finding:** Relay must validate announcements list this service. + +**From ../grasp/01.md lines 3-5:** + +> MUST reject [git repository announcements] that do not list the service +> in both `clone` and `relays` tags unless implementing `GRASP-05`. + +**NIP-34 Repository Announcement (kind 30617):** +```json +{ + "kind": 30617, + "tags": [ + ["d", "my-project"], // identifier + ["name", "My Project"], + ["clone", "https://example.com/npub.../my-project.git"], + ["clone", "https://github.com/user/my-project"], + ["relays", "wss://example.com"], + ["relays", "wss://relay.nostr.band"] + ] +} +``` + +**Validation Logic:** +```rust +fn validate_announcement(event: &Event, our_domain: &str) -> Result<()> { + // Check for clone tag with our domain + let has_clone = event.tags.iter().any(|tag| { + tag.kind() == TagKind::Custom("clone".into()) && + tag.content().map(|c| c.contains(our_domain)).unwrap_or(false) + }); + + // Check for relays tag with our domain + let has_relay = event.tags.iter().any(|tag| { + tag.kind() == TagKind::Custom("relays".into()) && + tag.content().map(|c| c.contains(our_domain)).unwrap_or(false) + }); + + if !has_clone || !has_relay { + return Err(anyhow!("Announcement must list this service in both clone and relays tags")); + } + + Ok(()) +} +``` + +**Action Required:** +- Implement announcement validation +- Check both `clone` and `relays` tags +- Reject if service not listed +- Add tests for validation + +--- + +### 7. State Announcement Handling + +**Finding:** State announcements control repository state. + +**NIP-34 Repository State (kind 30618):** +```json +{ + "kind": 30618, + "tags": [ + ["d", "my-project"], // identifier + ["refs/heads/main", "abc123..."], // branch → commit + ["refs/heads/develop", "def456..."], + ["HEAD", "ref: refs/heads/main"], // symbolic ref + ["maintainers", "npub1...", "npub2..."] // maintainer set + ] +} +``` + +**State Handling:** +1. When state announcement received: + - Update repository HEAD if needed + - Store state for push validation + - Handle maintainer set + +2. When push received: + - Query latest state announcement + - Validate pusher is in maintainer set (recursive) + - Validate ref updates match state + - Accept or reject push + +**Action Required:** +- Parse state announcements +- Update repository HEAD +- Implement push validation +- Handle recursive maintainer sets + +--- + +### 8. PR Ref Handling + +**Finding:** Special handling for PR refs. + +**From ../grasp/01.md lines 22-23:** + +> MUST accept pushes via this service to `refs/nostr/` but SHOULD +> reject if event exists on relay listing a different tip and MAY reject based +> on criteria such as size, SPAM prevention, etc. SHOULD delete and MAY garbage +> collect these refs if no corresponding [git PR event] or [git PR update event], +> with a `c` tag that matches the ref tip, is accepted by relay with 20 minutes. + +**PR Ref Lifecycle:** +1. Push to `refs/nostr/` +2. Verify PR event exists on relay +3. Verify ref tip matches PR event `c` tag +4. Accept push +5. After 20 minutes, check if PR event still exists +6. If not, delete ref and garbage collect + +**Action Required:** +- Accept pushes to `refs/nostr/` +- Validate against PR events +- Implement 20-minute timeout +- Implement garbage collection + +--- + +### 9. Git HTTP Protocol Details + +**Finding:** Must support specific Git protocol features. + +**From ../grasp/01.md lines 25-26:** + +> MUST include `allow-reachable-sha1-in-want` and `allow-tip-sha1-in-want` +> in advertisement and serve available oids. + +**Git Capabilities:** +``` +# info/refs response must include: +allow-reachable-sha1-in-want +allow-tip-sha1-in-want +``` + +**Action Required:** +- Configure git-http-backend to advertise these capabilities +- Ensure Git process is configured correctly +- Test with actual Git client + +--- + +### 10. CORS Requirements + +**Finding:** CORS must be on ALL responses, not just some. + +**From ../grasp/01.md lines 32-40:** + +``` +1. Set `Access-Control-Allow-Origin: *` on ALL responses +2. Set `Access-Control-Allow-Methods: GET, POST` on ALL responses +3. Set `Access-Control-Allow-Headers: Content-Type` on ALL responses +4. Respond to OPTIONS requests with 204 No Content +``` + +**Implementation:** +```rust +// In actix-web +App::new() + .wrap( + Cors::default() + .allow_any_origin() + .allowed_methods(vec!["GET", "POST"]) + .allowed_headers(vec!["Content-Type"]) + .max_age(3600) + ) + // ... routes +``` + +**Action Required:** +- Add CORS middleware to actix-web +- Verify headers on all responses +- Handle OPTIONS requests +- Test with browser + +--- + +## 📊 Compliance Status + +### NIP-01 (Nostr Relay) +- ✅ WebSocket connection +- ✅ EVENT message handling +- ✅ REQ subscription +- ✅ CLOSE subscription +- ✅ Event validation +- ⏳ NIP-11 with GRASP fields + +**Status:** ~80% complete + +### NIP-34 (Git Announcements) +- ✅ Store announcements (kind 30617) +- ✅ Store state events (kind 30618) +- ⏳ Validate announcements list this service +- ⏳ Handle maintainer sets +- ⏳ Accept related events + +**Status:** ~40% complete + +### GRASP-01 (Core Requirements) +- ✅ Nostr relay at `/` +- ❌ Git HTTP at `//.git` +- ❌ Push validation +- ❌ Repository provisioning +- ❌ CORS support + +**Status:** ~20% complete + +--- + +## 🎯 Immediate Next Steps + +### 1. Fix Architecture (CRITICAL) +- [ ] Add actix-web dependencies +- [ ] Create HTTP router module +- [ ] Route Git paths to Git handler +- [ ] Route `/` to Nostr relay (WebSocket) +- [ ] Apply CORS to all routes + +**Estimated Time:** 2-4 hours +**Priority:** CRITICAL +**Blocker:** Nothing else can proceed without this + +### 2. Add Git HTTP Backend +- [ ] Integrate git-http-backend crate +- [ ] Create Git request handler +- [ ] Serve from `{GIT_DATA_PATH}/{npub}/{id}.git` +- [ ] Return 404 for missing repos +- [ ] Test with `git clone` + +**Estimated Time:** 2-3 hours +**Priority:** HIGH +**Blocker:** Requires architecture fix + +### 3. Repository Provisioning +- [ ] Create repos when announcements received +- [ ] Initialize bare repositories +- [ ] Set up directory structure +- [ ] Handle repository deletion + +**Estimated Time:** 1-2 hours +**Priority:** HIGH +**Blocker:** Requires Git HTTP backend + +### 4. Update Tests +- [ ] Add GRASP-01 line number references +- [ ] Create Git HTTP tests +- [ ] Create CORS tests +- [ ] Update NIP-11 tests + +**Estimated Time:** 2-3 hours +**Priority:** MEDIUM +**Blocker:** None (can start now) + +--- + +## 📚 Key Files to Reference + +### GRASP Protocol +- `../grasp/01.md` - **THE SPEC** - Lines 1-40 +- `../grasp/README.md` - Overview +- `../grasp/02.md` - GRASP-02 (future) +- `../grasp/05.md` - GRASP-05 (future) + +### Reference Implementation +- `../ngit-relay/src/nginx.conf` - **ROUTING PATTERN** - Lines 8-94 +- `../ngit-relay/docker-compose.yml` - Port configuration +- `../ngit-relay/.env.example` - Environment variables +- `../ngit-relay/README.md` - Architecture overview + +### Our Code +- `tests/nip01_compliance.rs` - Current test approach +- `tests/common/relay.rs` - TestRelay fixture (already correct!) +- `src/nostr/relay.rs` - Current relay implementation +- `src/config.rs` - Configuration (needs Git path) + +--- + +## ✅ Checklist for Next Session + +Before starting implementation: +- [x] Read GRASP-01 specification (../grasp/01.md) +- [x] Review ngit-relay nginx.conf routing +- [x] Understand single-port architecture +- [x] Review environment variables +- [x] Understand repository path structure + +Ready to implement: +- [ ] Add actix-web dependencies to Cargo.toml +- [ ] Create src/http/mod.rs module +- [ ] Create src/http/git.rs handler +- [ ] Create src/http/nostr.rs handler +- [ ] Update src/main.rs +- [ ] Update src/config.rs +- [ ] Update .env.example +- [ ] Update tests/common/relay.rs +- [ ] Create tests/grasp01_git_http.rs + +--- + +**Last Updated:** November 4, 2025 +**Next Review:** After actix-web integration diff --git a/docs/archive/2025-11-04-evening/session-complete.md b/docs/archive/2025-11-04-evening/session-complete.md new file mode 100644 index 0000000..56cb711 --- /dev/null +++ b/docs/archive/2025-11-04-evening/session-complete.md @@ -0,0 +1,121 @@ +# Session Complete: Ready for Test Validation Phase + +**Date:** 2025-11-04 +**Status:** ✅ READY TO BEGIN + +--- + +## ✅ What We Did + +### 1. Strategic Planning +- Analyzed test-first vs TDD parallel approaches +- Decided to validate grasp-audit against ngit-relay first +- Validated hybrid architecture (git2 + git-http-backend + system git) + +### 2. Documentation +- Archived all planning documents to `docs/archive/2025-11-04-*` +- Created fresh `work/current_status.md` for test validation phase +- Documented strategic decision and rationale + +### 3. Preparation +- Identified ngit-relay location: `../ngit-relay/` +- Confirmed Docker image: `ghcr.io/danconwaydev/ngit-relay:latest` +- Outlined complete test validation plan + +--- + +## 📋 Current State + +``` +work/ +├── README.md ✅ (gitignored, explains work/) +└── current_status.md ✅ (test validation plan) + +docs/archive/ +├── 2025-11-04-session-summary.md ✅ (this session) +├── 2025-11-04-ngit-grasp-implementation-plan.md ✅ (for later) +├── 2025-11-04-git-http-backend-validation.md ✅ (architecture) +├── 2025-11-04-test-strategy-decision.md ✅ (rationale) +├── 2025-11-04-git-http-backend-deep-dive.md ✅ (crate analysis) +└── 2025-11-04-authorization-flow-diagram.txt ✅ (visual ref) +``` + +--- + +## 🎯 Next Session: Start Here + +### Quick Start Command +```bash +# 1. Read the plan +cat work/current_status.md + +# 2. Start ngit-relay +cd ../ngit-relay +docker-compose up -d + +# 3. Verify it's working +curl http://localhost:8080 # Nostr relay +curl http://localhost:3000 # Git server + +# 4. Begin building tests +cd ../ngit-grasp/grasp-audit +nix develop +# Create src/specs/grasp01_git.rs +``` + +### Timeline +- **Phase 1:** Setup ngit-relay (30 min) +- **Phase 2:** Build GRASP-01 Git tests (1 day) +- **Phase 3:** Validate against ngit-relay (1 day) +- **Phase 4:** Document findings (2 hours) +- **Total:** ~2 days + +--- + +## 📚 Key Documents + +### For This Phase (Test Validation) +- **Plan:** `work/current_status.md` ← START HERE +- **Rationale:** `docs/archive/2025-11-04-test-strategy-decision.md` +- **Reference:** `../ngit-relay/README.md` + +### For Later (Implementation) +- **Implementation Plan:** `docs/archive/2025-11-04-ngit-grasp-implementation-plan.md` +- **Architecture:** `docs/archive/2025-11-04-git-http-backend-validation.md` +- **Flow Diagram:** `docs/archive/2025-11-04-authorization-flow-diagram.txt` + +--- + +## 🚀 The Goal + +**By end of next session:** +- ✅ grasp-audit has complete GRASP-01 Git test suite +- ✅ All tests pass against ngit-relay reference implementation +- ✅ Reference behavior documented +- ✅ Confident test suite ready for ngit-grasp implementation + +**Then we can implement ngit-grasp knowing our tests are correct!** + +--- + +## 💡 Why This Approach? + +**Question:** Why not just start implementing ngit-grasp? + +**Answer:** +- Testing against reference validates our test suite first +- Eliminates "is it the test or the code?" debugging +- Only 1-2 day investment for weeks of confidence +- Same total timeline but much lower risk + +**See:** `docs/archive/2025-11-04-test-strategy-decision.md` for full analysis + +--- + +## ✅ Ready! + +**Status:** All planning complete, ready to begin test validation +**First Step:** `cd ../ngit-relay && docker-compose up -d` +**Reference:** `work/current_status.md` + +Let's build a rock-solid test suite! 🚀 diff --git a/docs/archive/2025-11-04-evening/session-summary.md b/docs/archive/2025-11-04-evening/session-summary.md new file mode 100644 index 0000000..398f4d0 --- /dev/null +++ b/docs/archive/2025-11-04-evening/session-summary.md @@ -0,0 +1,487 @@ +# Session Summary - GRASP Protocol Review + +**Date:** November 4, 2025 +**Duration:** ~2 hours +**Status:** ✅ Complete - Ready for implementation + +--- + +## 🎯 Session Goals + +1. ✅ Review GRASP protocol specification +2. ✅ Review ngit-relay reference implementation +3. ✅ Understand architecture requirements +4. ✅ Update work documents with accurate plan +5. ✅ Fix mistakes in previous understanding + +--- + +## 🔍 Key Discoveries + +### 1. Single Port Architecture (CRITICAL!) + +**Previous Understanding (WRONG):** +- Nostr relay on port 8080 +- Git server on port 8081 +- Separate services + +**Correct Understanding:** +- **BOTH services on SAME port** (e.g., 8080) +- HTTP router splits traffic by path: + - `//.git` → Git handler + - `/` → Nostr relay (WebSocket) +- This is a GRASP-01 requirement! + +**Evidence:** +- `../ngit-relay/docker-compose.yml` - Single port (8081) +- `../ngit-relay/src/nginx.conf` - nginx routes by path on one listener + +**Impact:** +- Complete architecture redesign needed +- Must use actix-web for HTTP routing +- All previous assumptions about ports were wrong + +--- + +### 2. Test Requirements Must Map to Protocol + +**Discovery:** Tests must reference GRASP protocol line numbers. + +**Example:** +```rust +#[tokio::test] +async fn test_git_http_basic() { + // Reference: ../grasp/01.md line 15 + // MUST serve git repository via unauthenticated git smart http service + // at //.git + + // Test implementation... +} +``` + +**Why:** +- Makes tests traceable to requirements +- Easy to verify compliance +- Documents what we're testing +- Helps reviewers understand intent + +**Action:** +- Update all test files with protocol references +- Create new tests for missing requirements +- Organize tests by GRASP-01 sections + +--- + +### 3. Environment Variables + +**Discovery:** ngit-relay uses specific env var naming we should match. + +**Critical Variables:** +- `NGIT_DOMAIN` - Used for announcement validation (REQUIRED) +- `NGIT_BIND_ADDRESS` - Single port for all services (REQUIRED) +- `NGIT_GIT_DATA_PATH` - Where to store Git repos (REQUIRED) +- `NGIT_RELAY_DATA_PATH` - Where to store events (REQUIRED) + +**Our .env.example:** +- ✅ Already has all required fields +- ✅ Follows ngit-relay naming convention +- ✅ No changes needed + +--- + +### 4. NIP-11 GRASP Fields + +**Discovery:** NIP-11 must include GRASP-specific fields. + +**Required Fields:** +```json +{ + "supported_grasps": ["GRASP-01"], + "repo_acceptance_criteria": "Must list service in clone and relays tags", + "curation": "Basic spam prevention" // optional +} +``` + +**Action:** +- Add fields to NIP-11 response +- Update tests to verify fields +- Document in code + +--- + +### 5. Repository Path Structure + +**Discovery:** Repos follow specific path pattern. + +**Pattern:** `{GIT_DATA_PATH}/{npub}/{identifier}.git` + +**Example:** +``` +./data/repos/ +├── npub1abc.../ +│ └── my-project.git/ +└── npub1xyz.../ + └── their-repo.git/ +``` + +**Action:** +- Create directory structure on repo provision +- Initialize bare repositories +- Handle cleanup on deletion + +--- + +### 6. CORS Requirements + +**Discovery:** CORS must be on ALL responses, not optional. + +**Requirements (GRASP-01 lines 32-40):** +1. `Access-Control-Allow-Origin: *` on ALL responses +2. `Access-Control-Allow-Methods: GET, POST` on ALL responses +3. `Access-Control-Allow-Headers: Content-Type` on ALL responses +4. Respond to OPTIONS with 204 No Content + +**Implementation:** +- Use actix-cors middleware +- Apply to all routes +- Test with browser + +--- + +### 7. Announcement Validation + +**Discovery:** Must validate announcements list this service. + +**Rule (GRASP-01 lines 3-5):** +> MUST reject announcements that do not list the service in both +> `clone` and `relays` tags unless implementing GRASP-05. + +**Validation Logic:** +```rust +fn validate_announcement(event: &Event, domain: &str) -> Result<()> { + let has_clone = event.tags.iter().any(|tag| + tag.is_clone() && tag.content().contains(domain) + ); + let has_relay = event.tags.iter().any(|tag| + tag.is_relay() && tag.content().contains(domain) + ); + + if !has_clone || !has_relay { + return Err("Must list service in clone and relays"); + } + Ok(()) +} +``` + +**Action:** +- Implement validation in event handler +- Reject invalid announcements +- Add tests + +--- + +## 📄 Documents Created + +### 1. work/current_status.md +**Purpose:** Comprehensive status of implementation + +**Contents:** +- GRASP-01 requirements checklist +- Architecture understanding +- Current implementation status +- Known issues +- Next priorities +- Key references + +**Use:** Reference for overall project status + +--- + +### 2. work/NEXT_SESSION_START_HERE.md +**Purpose:** Step-by-step implementation guide + +**Contents:** +- Immediate goal (actix-web integration) +- Critical architecture understanding +- 8-step implementation plan with code examples +- Verification steps +- Common issues & solutions +- Success criteria + +**Use:** Start here next session for implementation + +--- + +### 3. work/review-summary.md +**Purpose:** Document findings from GRASP/ngit-relay review + +**Contents:** +- 10 critical discoveries +- Evidence for each +- Action items +- Compliance status +- Next steps + +**Use:** Reference for why we're making changes + +--- + +### 4. work/architecture-diagram.md +**Purpose:** Visual reference for architecture + +**Contents:** +- Current vs. target architecture diagrams +- Request flow examples +- Component responsibilities +- File structure +- Comparison with ngit-relay + +**Use:** Visual reference during implementation + +--- + +### 5. work/implementation-checklist.md +**Purpose:** Detailed checklist for implementation + +**Contents:** +- 5 phases with detailed tasks +- Verification steps for each task +- Manual testing procedures +- Automated testing commands +- Acceptance criteria +- Known issues to watch for +- Reference commands + +**Use:** Track progress during implementation + +--- + +### 6. work/session-summary.md (this file) +**Purpose:** Summary of this review session + +**Contents:** +- What we accomplished +- Key discoveries +- Documents created +- Next steps + +**Use:** Remember what we did this session + +--- + +## 📊 Compliance Status + +### Before This Session +- **Understanding:** Incorrect (separate ports) +- **NIP-01:** ~60% (relay works) +- **NIP-34:** ~20% (basic storage) +- **GRASP-01:** ~10% (wrong architecture) + +### After This Session +- **Understanding:** ✅ Correct (single port, routing) +- **NIP-01:** ~60% (no change, but plan to improve) +- **NIP-34:** ~20% (no change, but plan ready) +- **GRASP-01:** ~20% (plan ready, architecture understood) + +### Target After Next Session +- **Understanding:** ✅ Complete +- **NIP-01:** ~80% (with actix-web) +- **NIP-34:** ~40% (with announcement validation) +- **GRASP-01:** ~60% (with Git HTTP working) + +--- + +## 🎯 Next Session Plan + +### Immediate Goal +**Integrate actix-web for single-port HTTP/WebSocket/Git routing** + +### Steps (from NEXT_SESSION_START_HERE.md) +1. Add dependencies (actix-web, actix-cors, actix-ws, git-http-backend) +2. Create src/http/mod.rs (HTTP server) +3. Create src/http/git.rs (Git handler) +4. Create src/http/nostr.rs (WebSocket handler) +5. Update src/main.rs +6. Update tests +7. Manual testing +8. Automated testing + +### Success Criteria +- ✅ Server starts on single port +- ✅ WebSocket connects at `/` +- ✅ NIP-01 smoke tests pass +- ✅ Can clone Git repo +- ✅ CORS headers present +- ✅ All tests pass + +### Estimated Time +2-4 hours for core implementation +1-2 hours for testing and debugging +**Total: 3-6 hours** + +--- + +## 📚 Key References for Next Session + +### Must Read Before Starting +1. `work/NEXT_SESSION_START_HERE.md` - Implementation guide +2. `work/architecture-diagram.md` - Visual reference +3. `../grasp/01.md` - THE SPEC (lines 1-40) +4. `../ngit-relay/src/nginx.conf` - Routing pattern + +### Reference During Implementation +1. `work/implementation-checklist.md` - Track progress +2. `work/current_status.md` - Overall context +3. [actix-web docs](https://actix.rs/docs/) - Framework reference +4. [git-http-backend docs](https://docs.rs/git-http-backend/) - Git protocol + +### Reference for Testing +1. `tests/common/relay.rs` - TestRelay fixture +2. `grasp-audit/src/specs/nip01_smoke.rs` - Test specs +3. `work/implementation-checklist.md` - Testing procedures + +--- + +## ✅ Accomplishments + +### Understanding +- ✅ Fully understand GRASP-01 requirements +- ✅ Understand ngit-relay architecture +- ✅ Identified critical mistake (separate ports) +- ✅ Understand how to fix it (actix-web routing) + +### Documentation +- ✅ Created comprehensive status document +- ✅ Created step-by-step implementation guide +- ✅ Created architecture diagrams +- ✅ Created detailed checklist +- ✅ Created review summary +- ✅ Created session summary + +### Planning +- ✅ Detailed 8-step implementation plan +- ✅ Identified all required changes +- ✅ Created acceptance criteria +- ✅ Prepared verification steps +- ✅ Listed common issues to watch for + +### Preparation +- ✅ Verified .env.example is correct +- ✅ Verified TestRelay is correct +- ✅ Identified which files need changes +- ✅ Created code templates for new files + +--- + +## 🚀 Ready for Implementation + +### What's Ready +- ✅ Complete understanding of requirements +- ✅ Detailed implementation plan +- ✅ Code templates prepared +- ✅ Test strategy defined +- ✅ Verification procedures documented + +### What's Needed +- [ ] Time to implement (~4 hours) +- [ ] Focus for coding +- [ ] Testing as we go +- [ ] Patience for debugging + +### Confidence Level +**HIGH** - We have: +- Clear understanding of problem +- Detailed solution plan +- Reference implementation to follow +- Good test coverage strategy +- Comprehensive documentation + +--- + +## 💡 Key Insights + +### 1. Architecture Matters +The single-port architecture is not just a detail - it's fundamental to GRASP-01 compliance. Getting this wrong means the whole implementation is wrong. + +### 2. Reference Implementation is Gold +ngit-relay's nginx.conf showed us EXACTLY how to route traffic. We don't need to guess - we can copy the pattern. + +### 3. Tests Must Map to Spec +Having tests that reference protocol line numbers makes verification trivial. We can see exactly which requirements we've met. + +### 4. Documentation Saves Time +Taking time to document our understanding and plan saves hours of confused implementation. We know exactly what to do. + +### 5. Incremental Progress +We can implement in phases: +1. HTTP routing (this phase) +2. Repository provisioning +3. Push authorization +4. Full compliance + +Each phase is testable and valuable on its own. + +--- + +## 🎓 Lessons Learned + +### What Went Well +- Thorough review of GRASP protocol +- Found critical architecture issue early +- Created comprehensive documentation +- Have clear path forward + +### What Could Be Better +- Could have reviewed GRASP spec earlier +- Could have checked ngit-relay architecture first +- Could have validated assumptions sooner + +### For Next Time +- Always check reference implementation first +- Read the spec thoroughly before coding +- Validate architecture assumptions early +- Document understanding before implementing + +--- + +## 📝 Notes for Future + +### When to Revisit This +- Before starting implementation (read NEXT_SESSION_START_HERE.md) +- When confused about architecture (read architecture-diagram.md) +- When stuck on a requirement (read review-summary.md) +- When tracking progress (read implementation-checklist.md) + +### What to Archive Later +- This session-summary.md → docs/archive/2025-11-04-grasp-review.md +- implementation-checklist.md → Delete after implementation complete +- NEXT_SESSION_START_HERE.md → Update for next phase + +### What to Keep +- current_status.md → Update as we progress +- architecture-diagram.md → Reference documentation +- review-summary.md → Reference for decisions + +--- + +## ✨ Final Thoughts + +This session was highly productive. We: +1. Identified a critical architectural flaw +2. Fully understood the correct architecture +3. Created a detailed implementation plan +4. Prepared everything needed for next session + +**We're ready to build this right.** + +The next session will be focused implementation - we have everything we need to succeed. + +--- + +**Session End:** November 4, 2025 +**Next Session:** Implementation of actix-web integration +**Confidence:** HIGH ✅ + +--- + +**Remember:** Start with `work/NEXT_SESSION_START_HERE.md` diff --git a/docs/archive/2025-11-04-session-summary.md b/docs/archive/2025-11-04-session-summary.md index 4cc53b0..150dd84 100644 --- a/docs/archive/2025-11-04-session-summary.md +++ b/docs/archive/2025-11-04-session-summary.md @@ -1,254 +1,176 @@ -# Session Summary - November 4, 2025 - -## Objective -Fix compilation errors in the `grasp-audit` crate and upgrade to latest nostr-sdk. - -## Status: ✅ COMPLETE - Upgraded to nostr-sdk 0.43 +**ARCHIVED: 2025-11-04** +**Session:** Strategic Planning & Test Validation Prep +**Outcome:** Decided to validate grasp-audit against ngit-relay first --- -## What We Did - -### 1. Identified Compilation Errors (nostr-sdk 0.35) -Started by attempting to build the project and discovered 9 compilation errors caused by API changes in `nostr-sdk` v0.35. +# Session Summary: Strategic Planning -### 2. Fixed Errors for 0.35 -Systematically fixed each error for nostr-sdk 0.35: +**Date:** 2025-11-04 +**Duration:** ~3 hours +**Status:** ✅ Complete - Ready for implementation -### 3. Discovered Version Gap -Realized the project was using nostr-sdk **0.35** when the latest is **0.43** - **8 minor versions behind**! - -### 4. Upgraded to nostr-sdk 0.43 -Completely upgraded to the latest version, fixing all new breaking changes: - -1. **EventBuilder::new()** - Removed tags parameter, use `.tags()` method instead -2. **EventBuilder::to_event()** → **sign_with_keys()** - Renamed method -3. **Client::new()** - Takes ownership of keys (clone instead of reference) -4. **Relay::is_connected()** - No longer async (remove `.await`) -5. **Client::get_events_of()** → **fetch_events()** - Complete API redesign -6. **EventSource** - Removed entirely -7. **Filter::custom_tag()** - Takes single value instead of array -8. **Client::send_event()** - Takes reference instead of ownership -9. **Multiple filters** - Loop and combine instead of vec parameter -10. **Events type** - New return type, convert to `Vec` with `.into_iter().collect()` +--- -### 5. Verified Build Success -- ✅ Clean build with no errors -- ✅ All 12 unit tests passing -- ✅ CLI binary builds successfully -- ✅ Example builds successfully +## What We Accomplished + +### 1. Strategic Analysis +- ✅ Analyzed two approaches: TDD parallel vs. test-first +- ✅ Evaluated git-http-backend crate for inline authorization +- ✅ Validated hybrid architecture (git2 + git-http-backend + system git) +- ✅ Decided to test ngit-relay first (1-2 day investment) + +### 2. Documentation Created +- ✅ `current_status.md` - TDD implementation plan for ngit-grasp +- ✅ `analysis-summary.md` - git-http-backend validation +- ✅ `strategic-recommendation.md` - Test strategy decision +- ✅ `git-http-backend-analysis.md` - Deep dive into crate +- ✅ `authorization-flow.txt` - Visual flow diagram + +### 3. Documentation Archived +All planning docs moved to `docs/archive/2025-11-04-*`: +- `ngit-grasp-implementation-plan.md` - Full TDD plan (for later) +- `git-http-backend-validation.md` - Crate analysis +- `test-strategy-decision.md` - Why test-first approach +- `git-http-backend-deep-dive.md` - Detailed crate analysis +- `authorization-flow-diagram.txt` - Visual reference + +### 4. New Current Status +Created fresh `work/current_status.md` for Phase 1: +- **Goal:** Validate grasp-audit against ngit-relay +- **Timeline:** 2 days +- **Phases:** Setup → Build tests → Validate → Document +- **Ready to begin immediately** --- -## Results +## Key Decisions -### Build Output -``` -Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.65s -``` +### ✅ Test ngit-relay First +**Decision:** Build and validate grasp-audit test suite against reference implementation before implementing ngit-grasp -### Test Results -``` -running 13 tests -test audit::tests::test_production_config ... ok -test audit::tests::test_ci_config ... ok -test audit::tests::test_audit_tags ... ok -test isolation::tests::test_generate_prod_run_id ... ok -test isolation::tests::test_generate_ci_run_id ... ok -test result::tests::test_audit_result ... ok -test specs::nip01_smoke::tests::test_smoke_tests_against_relay ... ignored -test isolation::tests::test_generate_test_id ... ok -test result::tests::test_result_fail ... ok -test result::tests::test_result_pass ... ok -test client::tests::test_event_builder ... ok -test audit::tests::test_audit_event_builder ... ok -test client::tests::test_client_creation ... ok - -test result: ok. 12 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out -``` +**Rationale:** +- Only 1-2 day investment +- Eliminates "is it the test or the code?" debugging +- Provides reference behavior documentation +- Same total timeline but higher confidence +- Lower risk of wasted implementation effort -### CLI Verification -```bash -$ ./target/debug/grasp-audit --help -GRASP audit and compliance testing tool +**Alternative Rejected:** TDD parallel development (higher risk, same timeline) -Usage: grasp-audit +### ✅ Hybrid Architecture Validated +**Decision:** Use git-http-backend (forked) + git2 + system git -Commands: - audit Run audit tests against a server - help Print this message or the help of the given subcommand(s) +**Components:** +- `git-http-backend` - HTTP protocol handling (will fork for inline auth) +- `git2` - Repository management, ref operations +- System git - Pack operations (upload-pack, receive-pack) -Options: - -h, --help Print help -``` +**Why:** Best balance of control, reliability, and implementation effort --- -## Files Modified - -1. **Cargo.toml** - - Updated `nostr-sdk = "0.35"` → `nostr-sdk = "0.43"` - -2. **src/audit.rs** - - Changed `EventBuilder::new()` to not take tags parameter - - Changed `.to_event(keys)` → `.tags(tags).sign_with_keys(keys)` - -3. **src/client.rs** - - Changed `Client::new(&keys)` → `Client::new(keys.clone())` - - Changed `is_connected()` to not await (no longer async) - - Changed `get_events_of()` → `fetch_events()` - - Removed `EventSource::relays()` usage - - Changed `Filter::custom_tag()` to use single values - - Changed `send_event(event)` → `send_event(&event)` - - Updated `subscribe()` to loop over filters - -4. **src/specs/nip01_smoke.rs** - - Changed `EventBuilder::new()` to not take tags parameter - - Changed `.to_event(keys)` → `.tags(tags).sign_with_keys(keys)` +## Resources Available ---- +### Reference Implementation +- **Location:** `../ngit-relay/` +- **Docker:** `ghcr.io/danconwaydev/ngit-relay:latest` +- **Endpoints:** + - Nostr: `ws://localhost:8080` + - Git: `http://localhost:3000` -## Documentation Created +### Test Suite +- **Location:** `grasp-audit/` +- **Status:** Basic structure, NIP-01 smoke test working +- **Next:** Add GRASP-01 Git compliance tests -1. **NOSTR_SDK_0.43_UPGRADE.md** - Comprehensive upgrade guide -2. **COMPILATION_FIXES.md** - Original 0.35 fixes (now obsolete) -3. **SESSION_2025_11_04_SUMMARY.md** - This file -4. Updated **NEXT_SESSION_QUICKSTART.md** - Marked completed items +### Documentation +- **GRASP Spec:** https://gitworkshop.dev/danconwaydev.com/grasp +- **NIP-34:** https://nips.nostr.com/34 +- **Archived Plans:** `docs/archive/2025-11-04-*` --- -## Next Steps - -### Ready for Integration Testing - -The code is now ready for integration testing. To proceed: +## Next Session Goals -#### Option 1: Run Integration Tests +### Phase 1: Setup (30 min) ```bash -# Terminal 1: Start test relay -docker run -p 7000:7000 scsibug/nostr-rs-relay - -# Terminal 2: Run tests -cd grasp-audit -nix develop --command cargo test --ignored +cd ../ngit-relay +docker-compose up -d +# Verify services running ``` -#### Option 2: Run CLI Audit -```bash -# Terminal 1: Start test relay -docker run -p 7000:7000 scsibug/nostr-rs-relay +### Phase 2: Build Tests (1 day) +- Create `grasp-audit/src/specs/grasp01_git.rs` +- Create `grasp-audit/src/git.rs` (test helpers) +- Add git2 dependency +- Implement all GRASP-01 Git tests -# Terminal 2: Run audit -cd grasp-audit -nix develop --command cargo run -- audit --relay ws://localhost:7000 --mode ci --spec nip01-smoke -``` - -#### Option 3: Continue Development -- Implement GRASP-01 compliance tests -- Start building the ngit-grasp relay -- Add more test specifications - ---- - -## Time Spent +### Phase 3: Validate (1 day) +- Run tests against ngit-relay +- Fix test bugs (not ngit-relay) +- Document reference behavior +- Iterate until all pass -- **Problem Identification (0.35):** 5 minutes -- **Fixing 0.35 Errors:** 25 minutes -- **Discovering Version Gap:** 5 minutes -- **Upgrading to 0.43:** 30 minutes -- **Testing & Verification:** 10 minutes -- **Documentation:** 15 minutes -- **Total:** ~90 minutes +### Phase 4: Document (2 hours) +- Test suite documentation +- Reference behavior guide +- Prepare for ngit-grasp implementation --- -## Key Learnings +## Files to Reference -### nostr-sdk v0.43 Breaking Changes +### For Implementation (Later) +- `docs/archive/2025-11-04-ngit-grasp-implementation-plan.md` - Full TDD plan +- `docs/archive/2025-11-04-git-http-backend-validation.md` - Crate details +- `docs/archive/2025-11-04-authorization-flow-diagram.txt` - Visual reference -The main API changes from 0.35 → 0.43: - -1. **EventBuilder Redesign** - Builder pattern for tags, explicit signing with `sign_with_keys()` -2. **Client Ownership** - Client takes ownership of signer (use `.clone()`) -3. **Sync Relay Status** - `is_connected()` is no longer async -4. **Query API Redesign** - `fetch_events()` instead of `get_events_of()`, single filter -5. **Events Type** - New collection type instead of `Vec` -6. **Simplified Filters** - `custom_tag()` takes single value -7. **Reference Passing** - `send_event()` takes reference for efficiency -8. **Removed EventSource** - Simpler API without source parameter - -### Best Practices Applied - -1. **Incremental Fixing** - Fixed one error at a time, testing after each fix -2. **Understanding Root Causes** - Identified API changes rather than just patching symptoms -3. **Proper Testing** - Verified unit tests after all fixes -4. **Documentation** - Created comprehensive documentation of all changes +### For Current Phase +- `work/current_status.md` - Test validation plan +- `docs/archive/2025-11-04-test-strategy-decision.md` - Why this approach +- `../ngit-relay/README.md` - Reference implementation docs --- -## Project Health - -| Metric | Status | Notes | -|--------|--------|-------| -| Build | ✅ Success | Clean build, no warnings | -| Unit Tests | ✅ 12/12 Pass | All tests passing | -| Integration Tests | ⏳ Pending | Need relay to run | -| Documentation | ✅ Complete | All changes documented | -| Code Quality | ✅ Good | No clippy warnings | - ---- +## Metrics -## Commands for Next Session +### Time Investment +- Planning & Analysis: ~3 hours +- Next Phase (Test Validation): ~2 days +- Future Phase (Implementation): ~3 weeks -### Quick Start -```bash -# Enter dev environment and build -cd grasp-audit -nix develop --command cargo build +### Confidence Level +- Test-first approach: 95% confident this is right path +- Architecture decisions: 90% confident (validated) +- Timeline estimates: 80% confident (reasonable) -# Run unit tests -cargo test --lib +--- -# Build CLI -cargo build --bin grasp-audit +## Lessons Learned -# Show help -./target/debug/grasp-audit --help -``` +### 1. Test Validation is Critical +Having a reference implementation to test against is a huge advantage. Use it! -### Integration Testing -```bash -# In one terminal, start relay: -docker run -p 7000:7000 scsibug/nostr-rs-relay +### 2. Upfront Planning Pays Off +The 3 hours of analysis and planning will save weeks of implementation time. -# In another terminal, run tests: -cd grasp-audit -nix develop --command cargo test --ignored +### 3. Documentation Structure Matters +Archiving session work keeps things clean and makes it easy to reference later. -# Or run CLI: -nix develop --command cargo run -- audit --relay ws://localhost:7000 -``` +### 4. Strategic Thinking > Speed +Taking 2 days to validate tests is smarter than rushing into implementation. --- -## Success Metrics - -✅ **All compilation errors fixed** -✅ **Clean build with no warnings** -✅ **All unit tests passing (12/12)** -✅ **CLI builds and shows help correctly** -✅ **Example builds successfully** -✅ **Comprehensive documentation created** +## Ready for Next Session ---- +**Status:** ✅ Ready to begin Phase 1 +**First Command:** `cd ../ngit-relay && docker-compose up -d` +**Reference:** `work/current_status.md` -## Conclusion +**Goal:** By end of next session (2 days), have a validated GRASP-01 Git test suite that we can confidently use to implement ngit-grasp. -The grasp-audit crate has been successfully upgraded to **nostr-sdk 0.43** (latest stable). All compilation errors have been resolved, the code builds cleanly with the modern API, and all unit tests pass. The upgrade brings: - -- **Better APIs** - Cleaner, more intuitive interfaces -- **Performance improvements** - Reference passing, sync operations where appropriate -- **Future compatibility** - On latest stable, ready for new features -- **8 versions of bug fixes** - All improvements from 0.35 → 0.43 +--- -**Status:** Ready for integration testing with latest nostr-sdk. +*Session complete. All work archived. Ready to proceed with test validation phase.* diff --git a/docs/archive/2025-11-04-test-strategy-decision.md b/docs/archive/2025-11-04-test-strategy-decision.md new file mode 100644 index 0000000..63a6961 --- /dev/null +++ b/docs/archive/2025-11-04-test-strategy-decision.md @@ -0,0 +1,290 @@ +**ARCHIVED: 2025-11-04** +**Decision:** Test ngit-relay first (Option 1) +**Rationale:** Validate test suite before implementation (1-2 day investment) + +--- + +# Strategic Recommendation: Test-First vs TDD Approach + +**Date:** 2025-11-04 +**Status:** ✅ ARCHIVED - Decision Made +**Context:** We have ngit-relay reference implementation available with Docker + +--- + +## The Question + +Should we: +1. **Test ngit-relay first** - Build grasp-audit against working reference, then apply to ngit-grasp +2. **TDD approach** - Build grasp-audit and ngit-grasp in parallel, test-driven + +--- + +## Option 1: Test ngit-relay First (RECOMMENDED) + +### Approach +``` +Phase 1: Validate Test Suite (1-2 days) +├── Run ngit-relay Docker image +├── Build grasp-audit GRASP-01 tests +├── Test against ngit-relay +└── Fix grasp-audit until all tests pass + +Phase 2: Apply to ngit-grasp (2-3 weeks) +├── Implement ngit-grasp features +├── Run same grasp-audit tests +├── Fix ngit-grasp until tests pass +└── Know tests are reliable (validated against reference) +``` + +### Pros +✅ **Validates test suite first** - Know tests work before implementing +✅ **Clear success criteria** - Tests pass against reference = tests are correct +✅ **Faster feedback** - Catch test bugs early, not during implementation +✅ **Reference behavior** - See how ngit-relay handles edge cases +✅ **Confidence** - When ngit-grasp passes, we know it's compliant +✅ **Documentation** - Tests become living spec examples +✅ **Lower risk** - Don't waste time implementing against broken tests + +### Cons +❌ **Sequential** - Can't start ngit-grasp until tests validated (but only 1-2 days) +❌ **Docker dependency** - Need Docker to run ngit-relay (already have) +❌ **Different tech stack** - ngit-relay is Go, might have quirks + +### Timeline +- **Phase 1:** 1-2 days (build + validate grasp-audit) +- **Phase 2:** 2-3 weeks (implement ngit-grasp) +- **Total:** ~3 weeks + +### Risk Level +🟢 **LOW** - Tests validated before implementation + +--- + +## Option 2: TDD Parallel Development + +### Approach +``` +Parallel Development +├── Write grasp-audit test +├── Run against ngit-grasp (fails - not implemented) +├── Implement ngit-grasp feature +├── Run test again (should pass) +└── Repeat for each feature +``` + +### Pros +✅ **True TDD** - Red → Green → Refactor cycle +✅ **Parallel work** - No waiting for test validation +✅ **Faster start** - Begin implementation immediately +✅ **Integrated learning** - Discover test issues during implementation + +### Cons +❌ **Test uncertainty** - Don't know if test failures are test bugs or implementation bugs +❌ **Debugging complexity** - Two moving targets (tests + implementation) +❌ **Wasted effort** - Might implement wrong thing if test is wrong +❌ **No reference** - Can't verify expected behavior +❌ **Higher risk** - Could build to wrong spec + +### Timeline +- **Parallel:** 2-3 weeks (but with more debugging) +- **Total:** ~3 weeks (but less confidence) + +### Risk Level +🟡 **MEDIUM** - Could implement to wrong spec + +--- + +## Comparison + +| Aspect | Test ngit-relay First | TDD Parallel | +|--------|----------------------|--------------| +| **Confidence** | High (tests validated) | Medium (tests unproven) | +| **Speed to start** | 1-2 day delay | Immediate | +| **Debugging complexity** | Low (one target) | High (two targets) | +| **Risk of rework** | Low | Medium-High | +| **Learning** | See reference behavior | Discover as you go | +| **Total time** | ~3 weeks | ~3 weeks | +| **Quality** | Higher | Lower | + +--- + +## Real-World Analogy + +**Option 1 (Test First):** +- Like calibrating a measuring tape against a known standard before measuring +- Build the test rig, validate it, then use it +- Science lab approach: calibrate instruments first + +**Option 2 (TDD Parallel):** +- Like building a measuring tape and the thing you're measuring at the same time +- Hope the tape is accurate while measuring +- Risky if tape is wrong + +--- + +## Recommendation: TEST NGIT-RELAY FIRST + +### Why? + +1. **We already have the reference** - ngit-relay Docker image is available +2. **Low time cost** - Only 1-2 days to validate tests +3. **High confidence gain** - Know tests are correct before implementing +4. **Better debugging** - One variable at a time (test bugs, then implementation bugs) +5. **Living documentation** - Tests show how reference implementation behaves +6. **Risk mitigation** - Don't waste weeks implementing to broken tests + +### Concrete Plan + +#### Step 1: Setup ngit-relay (30 minutes) +```bash +# Pull and run ngit-relay +docker pull ngitrelay/ngit-relay:latest +docker run -d -p 8080:8080 -p 3000:3000 ngitrelay/ngit-relay + +# Verify it's running +curl http://localhost:8080 # Nostr relay +curl http://localhost:3000 # Git HTTP backend +``` + +#### Step 2: Build grasp-audit GRASP-01 tests (1 day) +```bash +cd grasp-audit + +# Add GRASP-01 Git tests +# - Repository creation on announcement +# - Clone via HTTP +# - Push with valid state (should succeed) +# - Push without state (should fail) +# - Push with wrong state (should fail) +# - Multi-maintainer validation +# - refs/nostr/* support + +nix develop -c cargo test +``` + +#### Step 3: Test against ngit-relay (1 day) +```bash +# Run compliance tests +cd grasp-audit +nix develop -c cargo run -- --url ws://localhost:8080 --git-url http://localhost:3000 + +# Fix test bugs until all pass +# Document any ngit-relay quirks +# Create test fixtures +``` + +#### Step 4: Apply to ngit-grasp (2-3 weeks) +```bash +# Now implement ngit-grasp with confidence +cd ../ +# Implement features +# Run grasp-audit tests +# Fix ngit-grasp until tests pass +``` + +--- + +## What We Learn from ngit-relay + +By testing against the reference, we learn: + +1. **Expected behavior** - How should authorization work exactly? +2. **Error messages** - What does a proper rejection look like? +3. **Edge cases** - How does it handle: + - Empty repositories + - Multiple refs in one push + - Tag vs branch pushes + - refs/nostr/* special handling + - Concurrent pushes + - Invalid state events + - Circular maintainer references + +4. **Protocol details** - Git Smart HTTP quirks +5. **Performance** - What's reasonable for validation time? + +--- + +## Migration Path + +### Phase 1: Validate Tests (Days 1-2) +- [ ] Setup ngit-relay Docker +- [ ] Build grasp-audit Git tests +- [ ] Test against ngit-relay +- [ ] Fix test bugs +- [ ] Document reference behavior + +### Phase 2: Implement ngit-grasp (Weeks 1-3) +- [ ] Follow current_status.md plan +- [ ] Run grasp-audit after each phase +- [ ] Fix implementation bugs +- [ ] Achieve parity with ngit-relay + +### Phase 3: Exceed Reference (Week 4+) +- [ ] Add Rust-specific optimizations +- [ ] Better error messages +- [ ] Inline authorization benefits +- [ ] Performance improvements + +--- + +## Decision Criteria + +Choose **Test ngit-relay First** if: +- ✅ We value confidence over speed to start +- ✅ We want to minimize rework risk +- ✅ We can spare 1-2 days upfront +- ✅ We want tests as living documentation + +Choose **TDD Parallel** if: +- ❌ We can't run ngit-relay (Docker issues, etc.) +- ❌ We need to start implementation TODAY +- ❌ We're comfortable with higher debugging complexity +- ❌ We're okay with potential rework + +--- + +## My Recommendation + +**🎯 Test ngit-relay first** + +**Reasoning:** +1. Only 1-2 days upfront investment +2. Massively reduces risk of wasted effort +3. Provides living documentation +4. Gives confidence in test suite +5. We already have Docker and ngit-relay available +6. Total timeline is same (~3 weeks) but with higher quality + +**The 1-2 day investment in test validation will save us days or weeks of debugging "is it the test or the implementation?"** + +--- + +## Next Steps + +If you agree with this recommendation: + +1. **Today:** Setup ngit-relay Docker +2. **Tomorrow:** Build GRASP-01 Git tests in grasp-audit +3. **Day 3:** Validate tests against ngit-relay +4. **Week 2-4:** Implement ngit-grasp with confidence + +If you prefer TDD parallel: +1. **Today:** Start implementing ngit-grasp Git backend +2. **Ongoing:** Write tests alongside implementation +3. **Risk:** Accept higher debugging complexity + +--- + +## Questions? + +- Is Docker available for ngit-relay? +- Any blockers to testing against reference? +- Time constraints that require immediate implementation? +- Other considerations I'm missing? + +--- + +**Recommendation:** 🎯 **Test ngit-relay first** (1-2 day investment, weeks of confidence) + +**Confidence Level:** 95% - This is the right approach -- cgit v1.2.3