From 2eaff5b79fed364d5eba5eb38e4b7bf76326884d Mon Sep 17 00:00:00 2001 From: DanConwayDev Date: Wed, 3 Dec 2025 11:19:40 +0000 Subject: remove docs archive --- .../2025-11-04-authorization-flow-diagram.txt | 299 --------- .../2025-11-04-git-http-backend-deep-dive.md | 714 -------------------- .../2025-11-04-git-http-backend-validation.md | 247 ------- .../2025-11-04-ngit-grasp-implementation-plan.md | 590 ----------------- docs/archive/2025-11-04-evening/INDEX.md | 314 --------- .../2025-11-04-evening/NEXT_SESSION_START_HERE.md | 650 ------------------- docs/archive/2025-11-04-evening/STATUS.txt | 78 --- .../2025-11-04-evening/architecture-diagram.md | 442 ------------- docs/archive/2025-11-04-evening/current_status.md | 443 ------------- .../2025-11-04-evening/implementation-checklist.md | 720 --------------------- docs/archive/2025-11-04-evening/review-summary.md | 513 --------------- .../archive/2025-11-04-evening/session-complete.md | 121 ---- docs/archive/2025-11-04-evening/session-summary.md | 487 -------------- 13 files changed, 5618 deletions(-) delete mode 100644 docs/archive/2025-11-04-evening/2025-11-04-authorization-flow-diagram.txt delete mode 100644 docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-deep-dive.md delete mode 100644 docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-validation.md delete mode 100644 docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md delete mode 100644 docs/archive/2025-11-04-evening/INDEX.md delete mode 100644 docs/archive/2025-11-04-evening/NEXT_SESSION_START_HERE.md delete mode 100644 docs/archive/2025-11-04-evening/STATUS.txt delete mode 100644 docs/archive/2025-11-04-evening/architecture-diagram.md delete mode 100644 docs/archive/2025-11-04-evening/current_status.md delete mode 100644 docs/archive/2025-11-04-evening/implementation-checklist.md delete mode 100644 docs/archive/2025-11-04-evening/review-summary.md delete mode 100644 docs/archive/2025-11-04-evening/session-complete.md delete mode 100644 docs/archive/2025-11-04-evening/session-summary.md (limited to 'docs/archive/2025-11-04-evening') diff --git a/docs/archive/2025-11-04-evening/2025-11-04-authorization-flow-diagram.txt b/docs/archive/2025-11-04-evening/2025-11-04-authorization-flow-diagram.txt deleted file mode 100644 index 10fb529..0000000 --- a/docs/archive/2025-11-04-evening/2025-11-04-authorization-flow-diagram.txt +++ /dev/null @@ -1,299 +0,0 @@ -╔═══════════════════════════════════════════════════════════════════════════════╗ -║ GIT PUSH AUTHORIZATION FLOW (INLINE) ║ -╚═══════════════════════════════════════════════════════════════════════════════╝ - -┌─────────────────────────────────────────────────────────────────────────────┐ -│ CLIENT: git push │ -└────────────────────────────────┬────────────────────────────────────────────┘ - │ - │ HTTP POST - │ /npub/repo.git/git-receive-pack - ▼ -┌─────────────────────────────────────────────────────────────────────────────┐ -│ ACTIX-WEB ROUTER (git-http-backend) │ -│ │ -│ Route: /{namespace}/{repo}/git-receive-pack │ -│ Handler: git_receive_pack() │ -└────────────────────────────────┬────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────────────────────┐ -│ STEP 1: RESOLVE REPOSITORY PATH │ -│ │ -│ GitConfig::rewrite("/npub/repo") → /data/git/npub/repo.git │ -└────────────────────────────────┬────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────────────────────┐ -│ STEP 2: VALIDATE REPOSITORY EXISTS │ -│ │ -│ ✓ Check HEAD exists │ -│ ✓ Check config exists │ -│ ✓ Check bare = true │ -│ │ -│ ❌ If not: Return 400 "Repository not found" │ -└────────────────────────────────┬────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────────────────────┐ -│ STEP 3: READ REQUEST BODY (MODIFIED) │ -│ │ -│ • Read full request body into memory │ -│ • Decode gzip if Content-Encoding: gzip │ -│ • Store in body_data: Vec │ -└────────────────────────────────┬────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────────────────────┐ -│ STEP 4: PARSE REF UPDATES (NEW!) │ -│ │ -│ parse_receive_pack_request(&body_data) → Vec │ -│ │ -│ Git Pack Protocol: │ -│ ┌──────────────────────────────────────────────────────────────┐ │ -│ │ 0000000000000000000000000000000000000000 a1b2c3d4e5f6... │ │ -│ │ refs/heads/main\0 report-status\n │ │ -│ │ │ │ -│ │ old_oid: 0000... (new branch) │ │ -│ │ new_oid: a1b2c3d4e5f6... │ │ -│ │ ref_name: refs/heads/main │ │ -│ └──────────────────────────────────────────────────────────────┘ │ -│ │ -│ Result: RefUpdate { │ -│ old_oid: "0000...", │ -│ new_oid: "a1b2c3d4e5f6...", │ -│ ref_name: "refs/heads/main" │ -│ } │ -└────────────────────────────────┬────────────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────────────────────┐ -│ STEP 5: VALIDATE AUTHORIZATION (NEW!) │ -│ │ -│ validator.validate_push(npub, identifier, &ref_updates).await │ -│ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ PushValidator::validate_push() │ │ -│ │ │ │ -│ │ 1. Get latest state event from Nostr relay │ │ -│ │ • Query: kind=30618, d=identifier, author=npub │ │ -│ │ • Extract refs from state event │ │ -│ │ │ │ -│ │ 2. For each ref update: │ │ -│ │ • If refs/heads/* or refs/tags/*: │ │ -│ │ - Check state event has matching ref │ │ -│ │ - Check new_oid matches state event oid │ │ -│ │ - ❌ Reject if mismatch │ │ -│ │ • If refs/nostr/*: │ │ -│ │ - ✅ Always allow (PRs) │ │ -│ │ │ │ -│ │ 3. Get maintainers (recursive) │ │ -│ │ • Extract maintainers from announcement │ │ -│ │ • Recursively resolve maintainer sets │ │ -│ │ • Check if pusher is in maintainer list │ │ -│ │ • ❌ Reject if not maintainer │ │ -│ │ │ │ -│ │ 4. Return Ok(()) or Err(message) │ │ -│ └────────────────────────────────────────────────────────────┘ │ -│ │ -│ ❌ If validation fails: │ -│ Return 403 Forbidden │ -│ { │ -│ "error": "unauthorized", │ -│ "message": "Push rejected: refs/heads/main points to ..., state has..."│ -│ } │ -└────────────────────────────────┬────────────────────────────────────────────┘ - │ - │ ✅ AUTHORIZED - ▼ -┌─────────────────────────────────────────────────────────────────────────────┐ -│ STEP 6: SPAWN GIT RECEIVE-PACK (EXISTING) │ -│ │ -│ Command::new("git") │ -│ .arg("receive-pack") │ -│ .arg("--stateless-rpc") │ -│ .arg(".") │ -│ .current_dir(&repo_path) │ -│ .spawn() │ -│ │ -│ • Write body_data to git stdin │ -│ • Stream git stdout back to client │ -└────────────────────────────────┬────────────────────────────────────────────┘ - │ - │ Stream response - ▼ -┌─────────────────────────────────────────────────────────────────────────────┐ -│ CLIENT: git push success │ -└─────────────────────────────────────────────────────────────────────────────┘ - - -╔═══════════════════════════════════════════════════════════════════════════════╗ -║ COMPARISON: BEFORE vs AFTER ║ -╚═══════════════════════════════════════════════════════════════════════════════╝ - -┌─────────────────────────────────┬─────────────────────────────────────────┐ -│ BEFORE (git-http-backend) │ AFTER (our fork) │ -├─────────────────────────────────┼─────────────────────────────────────────┤ -│ 1. Resolve path │ 1. Resolve path │ -│ 2. Check bare repo │ 2. Check bare repo │ -│ 3. Read request body │ 3. Read request body │ -│ 4. Spawn git immediately ❌ │ 4. Parse ref updates ← NEW │ -│ 5. Stream response │ 5. Validate authorization ← NEW │ -│ │ 6. Spawn git (if authorized) │ -│ │ 7. Stream response │ -└─────────────────────────────────┴─────────────────────────────────────────┘ - -┌─────────────────────────────────┬─────────────────────────────────────────┐ -│ AUTHORIZATION │ METHOD │ -├─────────────────────────────────┼─────────────────────────────────────────┤ -│ ❌ None │ No validation │ -│ ⚠️ Git hooks (pre-receive) │ After git accepts push │ -│ ✅ Inline (our approach) │ Before git touches repository │ -└─────────────────────────────────┴─────────────────────────────────────────┘ - - -╔═══════════════════════════════════════════════════════════════════════════════╗ -║ KEY MODIFICATIONS NEEDED ║ -╚═══════════════════════════════════════════════════════════════════════════════╝ - -┌─────────────────────────────────────────────────────────────────────────────┐ -│ FILE: src/actix/git_receive_pack.rs │ -├─────────────────────────────────────────────────────────────────────────────┤ -│ │ -│ CHANGE 1: Add validator parameter │ -│ ──────────────────────────────────────────────────────────────────────── │ -│ pub async fn git_receive_pack( │ -│ request: HttpRequest, │ -│ mut payload: Payload, │ -│ service: web::Data, │ -│ + validator: web::Data, // ← ADD THIS │ -│ ) -> impl Responder { │ -│ │ -│ CHANGE 2: Parse ref updates after reading body │ -│ ──────────────────────────────────────────────────────────────────────── │ -│ // Read and decode body (existing) │ -│ let body_data = read_and_decode_body(&mut payload, &request).await?; │ -│ │ -│ + // Parse ref updates (NEW) │ -│ + let ref_updates = parse_receive_pack_request(&body_data)?; │ -│ │ -│ CHANGE 3: Validate before spawning git │ -│ ──────────────────────────────────────────────────────────────────────── │ -│ + // Extract repo info from path │ -│ + let (npub, identifier) = extract_repo_info(&request.uri().path())?; │ -│ + │ -│ + // Validate authorization │ -│ + if let Err(e) = validator.validate_push(&npub, &identifier, │ -│ + &ref_updates).await { │ -│ + return HttpResponse::Forbidden() │ -│ + .json(json!({ │ -│ + "error": "unauthorized", │ -│ + "message": e.to_string(), │ -│ + })); │ -│ + } │ -│ │ -│ // Spawn git (existing, unchanged) │ -│ let mut cmd = Command::new("git"); │ -│ // ... rest of existing code ... │ -└─────────────────────────────────────────────────────────────────────────────┘ - -┌─────────────────────────────────────────────────────────────────────────────┐ -│ NEW FILE: src/git/protocol.rs │ -├─────────────────────────────────────────────────────────────────────────────┤ -│ │ -│ pub struct RefUpdate { │ -│ pub old_oid: String, │ -│ pub new_oid: String, │ -│ pub ref_name: String, │ -│ } │ -│ │ -│ pub fn parse_receive_pack_request(body: &[u8]) -> Result> { │ -│ // Parse git pack protocol │ -│ // Extract ref updates from pkt-line format │ -│ } │ -└─────────────────────────────────────────────────────────────────────────────┘ - -┌─────────────────────────────────────────────────────────────────────────────┐ -│ NEW FILE: src/git/authorization.rs │ -├─────────────────────────────────────────────────────────────────────────────┤ -│ │ -│ pub struct PushValidator { │ -│ storage: Storage, │ -│ } │ -│ │ -│ impl PushValidator { │ -│ pub async fn validate_push( │ -│ &self, │ -│ npub: &str, │ -│ identifier: &str, │ -│ updates: &[RefUpdate], │ -│ ) -> Result<()> { │ -│ // Query Nostr relay for state event │ -│ // Validate each ref update │ -│ // Check maintainer permissions │ -│ } │ -│ } │ -└─────────────────────────────────────────────────────────────────────────────┘ - - -╔═══════════════════════════════════════════════════════════════════════════════╗ -║ BENEFITS OF INLINE AUTH ║ -╚═══════════════════════════════════════════════════════════════════════════════╝ - -✅ BETTER ERROR MESSAGES - • Return 403 with JSON error details - • Show exactly which ref failed validation - • Show expected vs. actual commit - • Better developer experience - -✅ SIMPLER DEPLOYMENT - • No git hooks to manage - • No symlinks or hook installation - • Single binary handles everything - • Easier to test - -✅ TIGHTER INTEGRATION - • Direct access to Nostr relay state - • Shared storage layer - • No IPC between components - • Atomic validation - -✅ EASIER TESTING - • Pure Rust unit tests - • Mock validator for testing - • No subprocess coordination - • Deterministic behavior - -✅ SECURITY - • Validation before git touches repo - • Can't bypass by manipulating hooks - • Centralized authorization logic - • Audit trail in application logs - - -╔═══════════════════════════════════════════════════════════════════════════════╗ -║ TIMELINE ║ -╚═══════════════════════════════════════════════════════════════════════════════╝ - -Week 1: Foundation -├─ Day 1-2: Fork git-http-backend, set up integration -├─ Day 3-4: Add git2, implement GitRepository -└─ Day 5: Add protocol parsing module - -Week 2: Authorization -├─ Day 1-2: Implement PushValidator -├─ Day 3-4: Modify git_receive_pack handler -└─ Day 5: Integration tests - -Week 3: Polish -├─ Day 1-2: Add CORS support -├─ Day 3-4: Error handling improvements -└─ Day 5: E2E tests with real git - -Week 4: Compliance -├─ Day 1-3: GRASP-01 compliance testing -├─ Day 4: Performance testing -└─ Day 5: Documentation - - -Status: ✅ Analysis complete, ready to implement diff --git a/docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-deep-dive.md b/docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-deep-dive.md deleted file mode 100644 index 26d0526..0000000 --- a/docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-deep-dive.md +++ /dev/null @@ -1,714 +0,0 @@ -**ARCHIVED: 2025-11-04** -**Reason:** Analysis complete, crate validated -**Outcome:** Confirmed suitable for use (with fork for authorization) - ---- - -# git-http-backend Crate Deep Dive - -**Date:** 2025-11-04 -**Status:** ✅ ARCHIVED - Analysis Complete -**Purpose:** Validate the recommendation in `work/current_status.md` regarding git-http-backend crate - ---- - -## Executive Summary - -**Recommendation Status:** ✅ **VALIDATED WITH CAVEATS** - -The `git-http-backend` crate (v0.1.3) is a **good foundation** but requires significant customization for our inline authorization needs. The hybrid approach recommended in `current_status.md` is sound, but we'll need to: - -1. **Fork or vendor** the crate for customization -2. **Add interception points** for authorization -3. **Enhance error handling** for better push rejection messages -4. **Add CORS support** (missing from current implementation) - ---- - -## Crate Overview - -### Basic Info -- **Name:** `git-http-backend` -- **Version:** 0.1.3 -- **Author:** lazhenyi -- **License:** MIT -- **Repository:** https://github.com/lazhenyi/git-http-backend -- **Documentation:** https://docs.rs/git-http-backend/0.1.3 - -### Dependencies -```toml -tokio = { version = "1", features = ["sync","macros","rt", "rt-multi-thread","net"] } -actix-web = { version = "4.9.0", features = ["default"] } -actix-files = { version = "0.6.6", features = ["actix-server"] } -futures-util = { version = "0.3.31", features = ["futures-channel"] } -flate2 = "1.0.35" # Gzip compression -async-stream = "0.3.6" # Streaming responses -async-trait = "0.1.83" # Async trait support -``` - -**Good news:** Already uses actix-web 4.9.0 (same as we plan to use) - ---- - -## Architecture Analysis - -### Core Design - -The crate provides: - -1. **GitConfig Trait** - Path rewriting abstraction -2. **Actix Router** - Pre-configured routes for Git Smart HTTP -3. **Protocol Handlers** - Upload-pack, receive-pack, info/refs -4. **System Git Integration** - Spawns `git` subprocess - -### URL Structure - -``` -/{namespace}/{repo}/info/refs?service=git-upload-pack -/{namespace}/{repo}/git-upload-pack -/{namespace}/{repo}/git-receive-pack -/{namespace}/{repo}/HEAD -/{namespace}/{repo}/objects/info/packs -/{namespace}/{repo}/objects/pack/{pack} -``` - -**Perfect match** for our `/{npub}/{identifier}.git/` structure! - -### Request Flow - -``` -HTTP Request - ↓ -Actix Router → Handler Function - ↓ -GitConfig::rewrite() → Path resolution - ↓ -Spawn git subprocess (upload-pack/receive-pack) - ↓ -Stream response back to client -``` - ---- - -## Key Handlers Analysis - -### 1. info/refs Handler (refs.rs) - -**Purpose:** Advertise repository refs (clone/fetch discovery) - -**Flow:** -1. Parse `service` query param (upload-pack or receive-pack) -2. Resolve repository path via `GitConfig::rewrite()` -3. Spawn `git upload-pack --advertise-refs --stateless-rpc .` -4. Return with proper content-type header - -**Code:** -```rust -pub async fn info_refs(request: HttpRequest, service: web::Data) -> impl Responder { - let uri = request.uri(); - let path = uri.path().to_string().replace("/info/refs", ""); - let path = service.rewrite(path).await; - - // Parse service from query - let service = query.split('=').map(|x| x.to_string()).collect::>()[1].clone(); - - // Spawn git - let mut cmd = Command::new("git"); - cmd.arg(service_name.clone()); - cmd.arg("--stateless-rpc"); - cmd.arg("--advertise-refs"); - cmd.arg("."); - cmd.current_dir(path); - - // Return response with proper headers - resp.append_header(("Content-Type", format!("application/x-git-{}-advertisement", service_name))); - resp.append_header(("Cache-Control", "no-cache, max-age=0, must-revalidate")); -} -``` - -**Good:** -- ✅ Proper content-type headers -- ✅ Cache control headers -- ✅ Git protocol version support (Git-Protocol header) - -**Issues:** -- ❌ No CORS headers -- ❌ No error handling for missing repos -- ❌ Query parsing is fragile (will panic on malformed input) - -### 2. git-upload-pack Handler (git_upload_pack.rs) - -**Purpose:** Handle clone/fetch operations (read-only) - -**Flow:** -1. Resolve repository path -2. Read request body (may be gzipped) -3. Spawn `git upload-pack --stateless-rpc .` -4. Stream response back - -**Code:** -```rust -pub async fn git_upload_pack( - request: HttpRequest, - mut payload: Payload, - service: web::Data, -) -> impl Responder { - // Resolve path - let path = service.rewrite(path).await; - - // Spawn git - let mut cmd = Command::new("git"); - cmd.arg("upload-pack"); - cmd.arg("--stateless-rpc"); - cmd.arg("."); - cmd.current_dir(path); - - let mut span = cmd.spawn()?; - let mut stdin = span.stdin.take().unwrap(); - let mut stdout = span.stdout.take().unwrap(); - - // Read request body - let mut bytes = web::BytesMut::new(); - while let Some(chunk) = payload.next().await { - bytes.extend_from_slice(&data); - } - - // Handle gzip - let body_data = match encoding { - Some("gzip") => decode_gzip(bytes), - _ => bytes.to_vec(), - }; - - // Write to git stdin - stdin.write_all(&body_data)?; - drop(stdin); - - // Stream response - let body_stream = actix_web::body::BodyStream::new(async_stream::stream! { - let mut buffer = [0; 8192]; - loop { - match stdout.read(&mut buffer) { - Ok(0) => break, - Ok(n) => yield Ok(web::Bytes::copy_from_slice(&buffer[..n])), - Err(e) => break, - } - } - }); - resp.body(body_stream) -} -``` - -**Good:** -- ✅ Handles gzip compression -- ✅ Streams response (efficient for large repos) -- ✅ Proper content-type headers - -**Issues:** -- ❌ No CORS headers -- ❌ No repository existence check -- ❌ Error handling uses eprintln! (not tracing) - -**For our use:** Upload-pack is read-only, so we can use as-is (just add CORS) - -### 3. git-receive-pack Handler (git_receive_pack.rs) ⚠️ - -**Purpose:** Handle push operations (write) - -**This is the critical handler for inline authorization!** - -**Current Flow:** -1. Resolve repository path -2. **Check if bare repository** (good!) -3. Read request body (may be gzipped) -4. Spawn `git receive-pack --stateless-rpc .` -5. Stream response back - -**Code:** -```rust -pub async fn git_receive_pack( - request: HttpRequest, - mut payload: Payload, - service: web::Data, -) -> impl Responder { - let path = service.rewrite(path).await; - - // Check repository exists - if !path.join("HEAD").exists() || !path.join("config").exists() { - return HttpResponse::BadRequest().body("Repository not found or invalid."); - } - - // Check if bare - let is_bare_repo = match std::fs::read_to_string(path.join("config")) { - Ok(config) => config.contains("bare = true"), - Err(_) => false, - }; - if !is_bare_repo { - return HttpResponse::BadRequest().body("Push operation requires a bare repository."); - } - - // Spawn git receive-pack - let mut cmd = Command::new("git"); - cmd.arg("receive-pack"); - cmd.arg("--stateless-rpc"); - cmd.arg("."); - cmd.current_dir(&path); - - let mut git_process = cmd.spawn()?; - let mut stdin = git_process.stdin.take().unwrap(); - let mut stdout = git_process.stdout.take().unwrap(); - - // Read request body - let mut bytes = web::BytesMut::new(); - while let Some(chunk) = payload.next().await { - bytes.extend_from_slice(&data); - } - - // Decode if gzipped - let body_data = match encoding { - Some(encoding) if encoding.contains("gzip") => decode_gzip(bytes), - _ => bytes.to_vec(), - }; - - // Write to git stdin - stdin.write_all(&body_data)?; - drop(stdin); - - // Stream response - let body_stream = /* stream stdout */; - resp.body(body_stream) -} -``` - -**Good:** -- ✅ Validates repository exists -- ✅ Validates bare repository -- ✅ Handles gzip compression -- ✅ Streams response - -**Critical Issues for Our Use:** -- ❌ **No authorization hook!** Spawns git immediately -- ❌ **No way to inspect push data** before spawning git -- ❌ **No CORS headers** -- ❌ **Can't reject unauthorized pushes** with custom error - -**This is where we need customization!** - ---- - -## Customization Requirements - -### 1. Authorization Interception Point - -**Need to add BEFORE spawning git:** - -```rust -pub async fn git_receive_pack( - request: HttpRequest, - mut payload: Payload, - service: web::Data, - validator: web::Data, // ← ADD THIS -) -> impl Responder { - let path = service.rewrite(path).await; - - // Existing checks... - - // Read request body - let body_data = read_and_decode_body(&mut payload, &request).await?; - - // ← ADD AUTHORIZATION HERE - let ref_updates = parse_receive_pack_request(&body_data)?; - - // Extract npub and identifier from path - let (npub, identifier) = extract_repo_info(&request.uri().path())?; - - // Validate against Nostr state - if let Err(e) = validator.validate_push(&npub, &identifier, &ref_updates).await { - return HttpResponse::Forbidden() - .json(json!({ - "error": "unauthorized", - "message": e.to_string(), - "ref_updates": ref_updates, - })); - } - - // Only spawn git if authorized - let mut cmd = Command::new("git"); - // ... rest of existing code -} -``` - -### 2. Parse Git Protocol - -**Need to add protocol parsing:** - -```rust -// src/git/protocol.rs - -pub struct RefUpdate { - pub old_oid: String, - pub new_oid: String, - pub ref_name: String, -} - -pub fn parse_receive_pack_request(body: &[u8]) -> Result> { - // Parse git pack protocol - // Format: \0\n - // Example: 0000000000000000000000000000000000000000 a1b2c3d4... refs/heads/main\0 report-status\n - - let mut updates = Vec::new(); - let lines = body.split(|&b| b == b'\n'); - - for line in lines { - if line.is_empty() { - continue; - } - - // Parse pkt-line format - // First 4 bytes are hex length - let pkt_len = parse_pkt_len(&line[0..4])?; - if pkt_len == 0 { - continue; // flush packet - } - - let data = &line[4..pkt_len]; - let parts: Vec<&[u8]> = data.splitn(3, |&b| b == b' ').collect(); - - if parts.len() >= 3 { - let old_oid = String::from_utf8_lossy(parts[0]).to_string(); - let new_oid = String::from_utf8_lossy(parts[1]).to_string(); - - // Ref name may have capabilities after \0 - let ref_data = parts[2]; - let ref_name = if let Some(null_pos) = ref_data.iter().position(|&b| b == b'\0') { - String::from_utf8_lossy(&ref_data[..null_pos]).to_string() - } else { - String::from_utf8_lossy(ref_data).to_string() - }; - - updates.push(RefUpdate { - old_oid, - new_oid, - ref_name, - }); - } - } - - Ok(updates) -} -``` - -**Note:** Git pack protocol is complex. We may want to use a library for this: -- `git2` crate has protocol parsing -- Or we can implement minimal parsing for our needs - -### 3. Add CORS Support - -**Need to add to all handlers:** - -```rust -// Add CORS middleware or headers to all responses -resp.append_header(("Access-Control-Allow-Origin", "*")); -resp.append_header(("Access-Control-Allow-Methods", "GET, POST, OPTIONS")); -resp.append_header(("Access-Control-Allow-Headers", "Content-Type, Git-Protocol")); -``` - -### 4. Better Error Handling - -**Replace eprintln! with tracing:** - -```rust -use tracing::{error, info, debug}; - -// Instead of: -eprintln!("Error running command: {}", e); - -// Use: -error!(error = ?e, "Failed to spawn git process"); -``` - ---- - -## Integration Strategy - -### Option A: Fork the Crate ✅ RECOMMENDED - -**Pros:** -- Full control over authorization logic -- Can add CORS, error handling, protocol parsing -- Can publish as `ngit-grasp-git-http-backend` -- Keep upstream changes visible - -**Cons:** -- Need to maintain fork -- Diverges from upstream - -**Implementation:** -1. Fork https://github.com/lazhenyi/git-http-backend -2. Add to our workspace as git submodule or copy -3. Modify `git_receive_pack.rs` to add authorization -4. Add protocol parsing module -5. Add CORS support -6. Improve error handling - -### Option B: Vendor the Code - -**Pros:** -- Complete control -- No external dependency -- Can heavily customize - -**Cons:** -- Lose upstream updates -- More code to maintain - -**Implementation:** -1. Copy source into `src/git/http_backend/` -2. Modify as needed -3. No external dependency - -### Option C: Wrap the Crate - -**Pros:** -- Keep upstream crate -- Add authorization via middleware - -**Cons:** -- ❌ **Can't intercept before git spawns!** -- Would need to parse response, too late -- Complex to inject validator - -**Not recommended** - can't achieve inline authorization - ---- - -## Recommended Approach - -### Use Forked git-http-backend + git2 + System Git - -**Architecture:** - -``` -HTTP Request - ↓ -Actix Router (from forked git-http-backend) - ↓ -Custom GitConfig Implementation - ↓ -git_receive_pack Handler (MODIFIED) - ↓ -┌─────────────────────────────────┐ -│ 1. Read request body │ -│ 2. Parse ref updates (protocol) │ ← ADD THIS -│ 3. Validate via PushValidator │ ← ADD THIS -│ ├─ Query Nostr relay │ -│ ├─ Check state event │ -│ └─ Validate maintainers │ -│ 4. If authorized: │ -│ └─ Spawn git receive-pack │ ← EXISTING -│ 5. If unauthorized: │ -│ └─ Return 403 with error │ ← ADD THIS -└─────────────────────────────────┘ - ↓ -Stream response to client -``` - -**Dependencies:** - -```toml -[dependencies] -# Fork of git-http-backend (or vendored code) -git-http-backend = { git = "https://github.com/our-org/git-http-backend", branch = "ngit-grasp" } - -# Or vendor it: -# (no dependency, code in src/git/http_backend/) - -# Git operations -git2 = "0.20" # For repository management, ref queries - -# Already have: -actix-web = "4.9" -tokio = { version = "1", features = ["full"] } -nostr-sdk = "0.43" -``` - -**Implementation Plan:** - -1. **Phase 1: Fork & Setup** - - Fork git-http-backend - - Add to our project (git submodule or copy) - - Verify existing functionality works - -2. **Phase 2: Protocol Parsing** - - Add `src/git/protocol.rs` - - Implement `parse_receive_pack_request()` - - Unit tests for protocol parsing - -3. **Phase 3: Authorization Integration** - - Modify `git_receive_pack.rs` - - Add `PushValidator` parameter - - Call validator before spawning git - - Return 403 on unauthorized - -4. **Phase 4: CORS & Polish** - - Add CORS headers to all handlers - - Improve error messages - - Add tracing instead of eprintln! - -5. **Phase 5: Testing** - - Unit tests for authorization - - Integration tests with real git - - GRASP-01 compliance tests - ---- - -## Validation of current_status.md Recommendations - -### Hybrid Approach ✅ VALIDATED - -**Original recommendation:** -> 1. **git-http-backend** - HTTP protocol handling -> 2. **git2-rs** - Repository management, ref validation -> 3. **System git** - Actual pack operations (upload-pack/receive-pack) - -**Analysis:** -- ✅ **git-http-backend** - Good foundation, needs customization -- ✅ **git2** - Perfect for repo management (init, refs, validation) -- ✅ **System git** - Proven pack protocol implementation - -**Verdict:** Sound approach, but need to fork/vendor git-http-backend - -### Tool Selection ✅ CORRECT - -**Original analysis:** -- git2 for repository management ✅ -- System git for pack operations ✅ -- git-http-backend for HTTP layer ✅ (with modifications) - -**Additional findings:** -- Need protocol parsing (can use git2 or implement minimal) -- Need CORS support (add to fork) -- Need better error handling (add to fork) - -### Inline Authorization ✅ ACHIEVABLE - -**Original goal:** -> We intercept the `git-receive-pack` operation before spawning the Git process - -**Analysis:** -- ✅ Possible by modifying `git_receive_pack.rs` -- ✅ Can parse request body before spawning git -- ✅ Can return 403 before git touches repository - -**Requirement:** -- Must fork or vendor git-http-backend -- Can't achieve with unmodified crate - ---- - -## Updated Implementation Plan - -### Week 1: Foundation (UPDATED) - -1. ✅ Add git2 dependency -2. **Fork git-http-backend** (NEW) -3. **Add protocol parsing** (NEW) -4. Implement GitRepository (Phase 1) -5. Write unit tests for repository operations -6. Test repository creation from announcements - -### Week 2: Protocol & Authorization - -1. Implement protocol parsing (Phase 2) -2. Implement authorization logic (Phase 3) -3. **Modify git_receive_pack handler** (NEW) -4. Write unit tests for both -5. Integration tests for validation - -### Week 3: HTTP & Integration - -1. **Add CORS support to fork** (NEW) -2. Implement HTTP handlers (Phase 4) -3. Integrate with Nostr events (Phase 5) -4. Integration tests for full flow -5. Error handling improvements - -### Week 4: E2E & Polish - -1. E2E tests with real git (Phase 6) -2. Performance testing -3. GRASP-01 compliance testing -4. Documentation and examples - ---- - -## Risks & Mitigations - -### Risk 1: Fork Maintenance - -**Risk:** Fork diverges from upstream, miss updates - -**Mitigation:** -- Keep fork minimal (only modify git_receive_pack.rs) -- Document all changes clearly -- Consider upstreaming authorization hooks -- Monitor upstream for security fixes - -### Risk 2: Protocol Parsing Complexity - -**Risk:** Git pack protocol is complex, may miss edge cases - -**Mitigation:** -- Use git2 for protocol parsing if available -- Implement minimal parsing (just ref updates) -- Extensive testing with real git clients -- Refer to Git protocol documentation - -### Risk 3: Performance - -**Risk:** Authorization adds latency to push operations - -**Mitigation:** -- Keep validation logic fast (< 100ms target) -- Cache state events in memory -- Async validation (don't block) -- Profile and optimize - ---- - -## Conclusion - -### Summary - -The **hybrid approach** recommended in `current_status.md` is **sound and validated**, with these adjustments: - -1. **Fork or vendor git-http-backend** - Can't use unmodified crate -2. **Add protocol parsing** - Need to parse ref updates from request -3. **Modify git_receive_pack handler** - Add authorization before spawning git -4. **Add CORS support** - Missing from current implementation -5. **Improve error handling** - Better messages for push rejections - -### Next Steps - -1. ✅ **Review this analysis** - Confirm approach -2. **Fork git-http-backend** - Set up fork/vendor -3. **Start Phase 1** - Add git2, implement GitRepository -4. **Add protocol parsing** - Parse ref updates from pack protocol -5. **Modify receive-pack handler** - Add authorization logic - -### Questions for Review - -1. **Fork vs. Vendor?** Fork allows upstream tracking, vendor gives full control -2. **Protocol parsing?** Use git2 or implement minimal parser? -3. **CORS scope?** Support all origins or restrict? -4. **Error detail?** How much info to expose in 403 responses? -5. **Performance target?** Is < 100ms for auth validation reasonable? - ---- - -**Status:** ✅ Analysis complete, ready to proceed with implementation - -**Recommendation:** Fork git-http-backend, add authorization to git_receive_pack, use git2 for repo management - ---- - -*Analysis Date: November 4, 2025* diff --git a/docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-validation.md b/docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-validation.md deleted file mode 100644 index d383b9f..0000000 --- a/docs/archive/2025-11-04-evening/2025-11-04-git-http-backend-validation.md +++ /dev/null @@ -1,247 +0,0 @@ -**ARCHIVED: 2025-11-04** -**Reason:** Analysis complete, validated hybrid approach -**Outcome:** Will use git-http-backend (forked) + git2 + system git - ---- - -# Analysis Summary: git-http-backend Validation - -**Date:** 2025-11-04 -**Status:** ✅ ARCHIVED - Analysis Complete - ---- - -## TL;DR - -✅ **VALIDATED:** The hybrid approach in `current_status.md` is sound -⚠️ **CAVEAT:** Must fork/vendor `git-http-backend` crate for inline authorization -✅ **READY:** Can proceed with implementation - ---- - -## Key Findings - -### 1. git-http-backend Crate (v0.1.3) - -**What it provides:** -- ✅ Actix-web based Git Smart HTTP handlers -- ✅ Upload-pack (clone/fetch) - works as-is -- ✅ Receive-pack (push) - **needs modification** -- ✅ Info/refs advertisement -- ✅ Gzip compression support -- ✅ Streaming responses - -**What it lacks:** -- ❌ Authorization hooks (spawns git immediately) -- ❌ CORS headers (needed for web clients) -- ❌ Protocol parsing (can't inspect push data) -- ❌ Proper error handling (uses eprintln!) - -### 2. Critical Handler: git_receive_pack - -**Current flow:** -``` -Request → Validate bare repo → Spawn git → Stream response -``` - -**What we need:** -``` -Request → Validate bare repo → Parse ref updates → Validate against Nostr state → Spawn git (if authorized) → Stream response - ↑ - ADD THIS -``` - -**Can't achieve with unmodified crate!** - -### 3. Recommended Solution - -**Fork the crate and modify `git_receive_pack.rs`:** - -```rust -pub async fn git_receive_pack( - request: HttpRequest, - mut payload: Payload, - service: web::Data, - validator: web::Data, // ← ADD -) -> impl Responder { - // ... existing path resolution and bare check ... - - // Read request body - let body_data = read_and_decode_body(&mut payload, &request).await?; - - // ← ADD: Parse ref updates - let ref_updates = parse_receive_pack_request(&body_data)?; - - // ← ADD: Validate authorization - let (npub, identifier) = extract_repo_info(&request.uri().path())?; - if let Err(e) = validator.validate_push(&npub, &identifier, &ref_updates).await { - return HttpResponse::Forbidden() - .json(json!({ - "error": "unauthorized", - "message": e.to_string(), - })); - } - - // Only spawn git if authorized - let mut cmd = Command::new("git"); - cmd.arg("receive-pack"); - // ... rest of existing code ... -} -``` - ---- - -## Updated Implementation Plan - -### Phase 0: Setup (NEW) -1. Fork git-http-backend repository -2. Add as git submodule or vendor code -3. Verify existing functionality works -4. Add to Cargo.toml - -### Phase 1: Foundation -1. Add git2 dependency -2. Implement GitRepository (repo management) -3. Add protocol parsing module -4. Unit tests for both - -### Phase 2: Authorization -1. Modify git_receive_pack handler -2. Implement PushValidator -3. Integration tests for validation -4. Test unauthorized rejection - -### Phase 3: Polish -1. Add CORS headers to all handlers -2. Improve error messages -3. Add tracing instead of eprintln! -4. E2E tests with real git - ---- - -## Dependencies - -```toml -[dependencies] -# Forked git-http-backend with authorization support -git-http-backend = { git = "https://github.com/our-org/git-http-backend", branch = "ngit-grasp" } - -# Git repository management -git2 = "0.20" - -# Already have: -actix-web = "4.9" -tokio = { version = "1", features = ["full"] } -nostr-sdk = "0.43" -``` - ---- - -## Validation of current_status.md - -### ✅ Hybrid Approach - CONFIRMED -- git-http-backend for HTTP layer ✅ (with fork) -- git2 for repository management ✅ -- System git for pack operations ✅ - -### ✅ Inline Authorization - ACHIEVABLE -- Can intercept before spawning git ✅ -- Can parse ref updates ✅ -- Can validate against Nostr state ✅ -- Can return 403 with error message ✅ - -### ⚠️ Additional Requirements -- Must fork/vendor git-http-backend -- Must implement protocol parsing -- Must add CORS support -- Must improve error handling - ---- - -## Risks & Mitigations - -| Risk | Impact | Mitigation | -|------|--------|------------| -| Fork maintenance | Medium | Keep changes minimal, document well | -| Protocol parsing complexity | Medium | Use git2 or implement minimal parser | -| Performance overhead | Low | Keep validation fast (< 100ms), cache state | -| Missing edge cases | Medium | Extensive testing with real git clients | - ---- - -## Next Steps - -1. **Decision:** Fork vs. vendor git-http-backend? - - Fork: Keep upstream tracking, easier updates - - Vendor: Full control, no external dependency - - **Recommendation:** Fork (easier to contribute back) - -2. **Start Phase 0:** Set up fork - - Fork https://github.com/lazhenyi/git-http-backend - - Create branch `ngit-grasp` - - Add as git submodule - -3. **Start Phase 1:** Add git2, implement GitRepository - - Write tests first (TDD) - - Focus on bare repo creation, ref management - -4. **Add protocol parsing:** Parse ref updates from pack protocol - - Research: Can git2 help? - - Or implement minimal parser - - Unit tests for various push scenarios - -5. **Modify receive-pack:** Add authorization logic - - Integration tests for validation - - Test rejection scenarios - ---- - -## Questions for Review - -1. **Fork vs. Vendor?** - - Recommendation: Fork (can contribute back, easier updates) - -2. **Protocol parsing?** - - Option A: Use git2 if it provides parsing - - Option B: Implement minimal parser (just ref updates) - - Recommendation: Research git2 first, then decide - -3. **CORS policy?** - - Allow all origins (`*`) for now? - - Or restrict to configured domains? - - Recommendation: Start with `*`, make configurable later - -4. **Error detail?** - - How much info in 403 responses? - - Show ref updates that failed? - - Show expected vs. actual commit? - - Recommendation: Detailed errors for better DX - -5. **Performance target?** - - < 100ms for auth validation? - - Cache state events? - - Recommendation: Yes to both - ---- - -## Conclusion - -✅ **The hybrid approach is validated and sound** - -⚠️ **Must fork git-http-backend for inline authorization** - -✅ **Ready to proceed with implementation** - -**Confidence Level:** High (95%) - -The crate provides exactly what we need as a foundation. The modifications required are straightforward and well-scoped. The main work is: -1. Fork setup -2. Protocol parsing -3. Authorization integration -4. CORS and polish - -All achievable within the 4-week timeline. - ---- - -**Next:** Review this analysis, make fork vs. vendor decision, then start Phase 0. diff --git a/docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md b/docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md deleted file mode 100644 index aaca16b..0000000 --- a/docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md +++ /dev/null @@ -1,590 +0,0 @@ -**ARCHIVED: 2025-11-04** -**Reason:** Decided to validate grasp-audit against ngit-relay first -**See:** docs/archive/2025-11-04-test-strategy-decision.md for rationale - ---- - -# TDD Plan for GRASP-01 Git Backend - -**Date:** 2025-11-04 -**Status:** ARCHIVED - Superseded by test-first approach -**Goal:** Implement Git Smart HTTP backend with inline authorization using TDD - ---- - -## Current State - -### What We Have -- ✅ NIP-01 compliant Nostr relay (working, tested) -- ✅ NIP-34 event handling (announcements accepted) -- ✅ Storage layer (in-memory + disk paths configured) -- ✅ Test infrastructure (integration tests with auto relay management) -- ✅ grasp-audit compliance testing library - -### What We Need -- ❌ Git Smart HTTP protocol handler -- ❌ Git repository management (init, receive-pack, upload-pack) -- ❌ Push authorization (validate against Nostr state events) -- ❌ Integration with existing Nostr relay - ---- - -## Tool Selection Analysis - -### Option 1: git2-rs (libgit2 bindings) -**Pros:** -- ✅ Pure Rust bindings to battle-tested libgit2 -- ✅ Full Git functionality (init, push, pull, refs, objects) -- ✅ Thread-safe, well-maintained -- ✅ Used by cargo, widely deployed -- ✅ Can intercept and validate operations programmatically - -**Cons:** -- ❌ Requires libgit2 system dependency -- ❌ Higher-level API - may be overkill for our needs -- ❌ Harder to intercept low-level protocol for inline auth - -**Use Cases:** -- Repository initialization -- Reading/writing refs -- Object storage queries -- Validation of commits/trees - -### Option 2: Standard git in subprocess -**Pros:** -- ✅ Uses system git (already available) -- ✅ No additional dependencies -- ✅ Battle-tested Git implementation -- ✅ Easy to spawn for upload-pack/receive-pack - -**Cons:** -- ❌ Harder to intercept for inline authorization -- ❌ Must parse git protocol to validate pushes -- ❌ Subprocess overhead -- ❌ Complex error handling - -**Use Cases:** -- git-upload-pack (clone, fetch) -- git-receive-pack (push) - but need to intercept - -### Option 3: git-http-backend crate -**Pros:** -- ✅ Purpose-built for Git Smart HTTP -- ✅ Handles protocol parsing -- ✅ Works with system git -- ✅ Can intercept receive-pack before spawning git - -**Cons:** -- ❌ Less mature (but we're already planning to use it per README) -- ❌ Still need to parse pack protocol for validation - -**Use Cases:** -- HTTP endpoint handling -- Protocol negotiation -- Spawning git processes - -### Option 4: Hybrid Approach (RECOMMENDED) -**Combination:** -1. **git-http-backend** - HTTP protocol handling -2. **git2-rs** - Repository management, ref validation -3. **System git** - Actual pack operations (upload-pack/receive-pack) - -**Why Hybrid:** -- git-http-backend handles HTTP → Git protocol translation -- git2 for safe repository operations (init, refs, validation) -- System git for pack operations (proven, fast) -- We intercept at the HTTP layer before spawning git - -**Architecture:** -``` -HTTP Request → git-http-backend → Our Auth Layer → git2/system git - ↓ - Nostr Relay - (state validation) -``` - ---- - -## Recommended Approach: Hybrid - -### Dependencies to Add -```toml -[dependencies] -# Git operations -git2 = "0.20" # Repository management, refs -# git-http-backend - TBD (research if available, or implement minimal) - -[dev-dependencies] -tempfile = "3.8" # Temporary repos for testing -``` - -### Why This Works - -1. **git2 for Repository Management:** - - Initialize bare repos when announcements arrive - - Read/write refs safely - - Query repository state - - Validate commits exist - -2. **System git for Pack Operations:** - - Spawn `git-upload-pack` for clones/fetches (read-only, safe) - - Spawn `git-receive-pack` ONLY after auth passes - - Leverage proven pack protocol implementation - -3. **Inline Authorization:** - - Parse HTTP request to extract ref updates - - Query Nostr relay for latest state event - - Validate push matches state - - Only spawn git-receive-pack if authorized - ---- - -## TDD Implementation Plan - -### Phase 1: Repository Management (git2) -**Goal:** Create and manage bare Git repositories - -**Tests:** -1. ✅ Create bare repository when announcement received -2. ✅ Initialize with proper config (bare, shared) -3. ✅ Set HEAD from state event -4. ✅ Read refs from repository -5. ✅ Write refs to repository -6. ✅ Query if commit exists in repository - -**Implementation:** -```rust -// src/git/repository.rs - -pub struct GitRepository { - path: PathBuf, - repo: git2::Repository, -} - -impl GitRepository { - pub fn init_bare(path: PathBuf) -> Result; - pub fn set_head(ref_name: &str) -> Result<()>; - pub fn get_ref(&self, name: &str) -> Result>; - pub fn set_ref(&self, name: &str, oid: &str) -> Result<()>; - pub fn has_commit(&self, oid: &str) -> Result; -} -``` - -**Test Example:** -```rust -#[test] -fn test_init_bare_repository() { - let temp = TempDir::new().unwrap(); - let repo = GitRepository::init_bare(temp.path().to_path_buf()).unwrap(); - - assert!(temp.path().join("HEAD").exists()); - assert!(temp.path().join("config").exists()); - assert!(temp.path().join("objects").exists()); - assert!(temp.path().join("refs").exists()); -} -``` - -### Phase 2: Git Protocol Parsing -**Goal:** Parse Git Smart HTTP protocol for authorization - -**Tests:** -1. ✅ Parse info/refs request -2. ✅ Parse upload-pack request (clone/fetch) -3. ✅ Parse receive-pack request (push) -4. ✅ Extract ref updates from receive-pack -5. ✅ Extract capabilities from request - -**Implementation:** -```rust -// src/git/protocol.rs - -pub struct RefUpdate { - pub old_oid: String, - pub new_oid: String, - pub ref_name: String, -} - -pub fn parse_receive_pack_request(body: &[u8]) -> Result>; -pub fn parse_capabilities(body: &[u8]) -> Result>; -``` - -**Test Example:** -```rust -#[test] -fn test_parse_receive_pack_single_ref() { - let body = b"0000000000000000000000000000000000000000 \ - a1b2c3d4e5f6789012345678901234567890abcd \ - refs/heads/main\0 report-status\n"; - - let updates = parse_receive_pack_request(body).unwrap(); - assert_eq!(updates.len(), 1); - assert_eq!(updates[0].ref_name, "refs/heads/main"); - assert_eq!(updates[0].new_oid, "a1b2c3d4e5f6789012345678901234567890abcd"); -} -``` - -### Phase 3: Authorization Logic -**Goal:** Validate pushes against Nostr state events - -**Tests:** -1. ✅ Get maintainers from announcement -2. ✅ Get maintainers recursively -3. ✅ Handle circular maintainer references -4. ✅ Validate ref update matches state -5. ✅ Validate branch push matches state -6. ✅ Validate tag push matches state -7. ✅ Accept push to refs/nostr/* -8. ✅ Reject push not matching state -9. ✅ Reject push from non-maintainer - -**Implementation:** -```rust -// src/git/authorization.rs - -pub struct PushValidator { - storage: Storage, -} - -impl PushValidator { - pub async fn validate_push( - &self, - npub: &str, - identifier: &str, - updates: &[RefUpdate], - ) -> Result<()>; - - async fn get_maintainers(&self, npub: &str, identifier: &str) -> Vec; - async fn get_latest_state(&self, npub: &str, identifier: &str) -> Option; - fn validate_ref_update(&self, state: &StateEvent, update: &RefUpdate) -> Result<()>; -} -``` - -**Test Example:** -```rust -#[tokio::test] -async fn test_validate_matching_push() { - let storage = test_storage().await; - - // Create announcement and state - let announcement = create_announcement("alice", "repo1"); - let state = create_state("alice", "repo1") - .branch("main", "a1b2c3d4..."); - - storage.store_event(announcement).await.unwrap(); - storage.store_event(state).await.unwrap(); - - // Validate matching push - let validator = PushValidator::new(storage); - let update = RefUpdate { - old_oid: "0000...".into(), - new_oid: "a1b2c3d4...".into(), - ref_name: "refs/heads/main".into(), - }; - - let result = validator.validate_push("alice", "repo1", &[update]).await; - assert!(result.is_ok()); -} - -#[tokio::test] -async fn test_reject_mismatched_push() { - let storage = test_storage().await; - - // State points to commit A - let state = create_state("alice", "repo1") - .branch("main", "aaaa1111..."); - storage.store_event(state).await.unwrap(); - - // Try to push commit B - let validator = PushValidator::new(storage); - let update = RefUpdate { - old_oid: "0000...".into(), - new_oid: "bbbb2222...".into(), // Different! - ref_name: "refs/heads/main".into(), - }; - - let result = validator.validate_push("alice", "repo1", &[update]).await; - assert!(result.is_err()); - assert!(result.unwrap_err().to_string().contains("state")); -} -``` - -### Phase 4: HTTP Handlers -**Goal:** Serve Git Smart HTTP protocol - -**Tests:** -1. ✅ GET /npub/repo.git/info/refs?service=git-upload-pack -2. ✅ POST /npub/repo.git/git-upload-pack (clone/fetch) -3. ✅ POST /npub/repo.git/git-receive-pack (push with auth) -4. ✅ Return 403 for unauthorized push -5. ✅ Return 404 for non-existent repository -6. ✅ Set correct content-type headers -7. ✅ Include CORS headers - -**Implementation:** -```rust -// src/git/handler.rs - -pub async fn handle_info_refs( - npub: String, - identifier: String, - service: String, -) -> Result; - -pub async fn handle_upload_pack( - npub: String, - identifier: String, - body: Bytes, -) -> Result; - -pub async fn handle_receive_pack( - npub: String, - identifier: String, - body: Bytes, - validator: PushValidator, -) -> Result; -``` - -**Test Example:** -```rust -#[tokio::test] -async fn test_info_refs_returns_correct_headers() { - let app = test_app().await; - - // Create repository - app.create_repo("alice", "test-repo").await; - - // Request info/refs - let response = app.get("/alice-npub/test-repo.git/info/refs?service=git-upload-pack").await; - - assert_eq!(response.status(), 200); - assert_eq!( - response.headers().get("content-type").unwrap(), - "application/x-git-upload-pack-advertisement" - ); - assert_eq!( - response.headers().get("access-control-allow-origin").unwrap(), - "*" - ); -} - -#[tokio::test] -async fn test_receive_pack_rejects_unauthorized() { - let app = test_app().await; - - // Create repo with state - let (announcement, state) = app.create_repo_with_state() - .branch("main", "aaaa1111...") - .build() - .await; - - // Try to push different commit - let body = create_receive_pack_request() - .ref_update("refs/heads/main", "0000...", "bbbb2222...") - .build(); - - let response = app.post( - &format!("/{}/repo.git/git-receive-pack", announcement.author_npub()), - body - ).await; - - assert_eq!(response.status(), 403); -} -``` - -### Phase 5: Integration with Nostr Events -**Goal:** Automatic repository creation and state updates - -**Tests:** -1. ✅ Create repository when announcement received -2. ✅ Update HEAD when state event received -3. ✅ Handle announcement updates (maintainers change) -4. ✅ Clean up orphaned refs/nostr/* refs - -**Implementation:** -```rust -// src/nostr/events.rs (extend existing) - -async fn handle_repository_announcement(event: &Event, storage: &Storage) -> Result<()> { - // Extract npub and identifier - // Create bare repository - // Store event -} - -async fn handle_repository_state(event: &Event, storage: &Storage) -> Result<()> { - // Find repository - // Update refs to match state - // Update HEAD -} -``` - -**Test Example:** -```rust -#[tokio::test] -async fn test_repository_created_on_announcement() { - let app = test_app().await; - - let announcement = create_announcement("alice", "test-repo") - .with_clone_tag(app.domain()) - .build(); - - app.send_event(announcement).await.unwrap(); - - // Wait for async processing - tokio::time::sleep(Duration::from_millis(100)).await; - - // Verify repository exists - let repo_path = app.git_data_path() - .join("alice-npub") - .join("test-repo.git"); - - assert!(repo_path.exists()); - assert!(repo_path.join("HEAD").exists()); -} -``` - -### Phase 6: End-to-End Testing -**Goal:** Test with real Git client - -**Tests:** -1. ✅ Clone repository with git client -2. ✅ Fetch from repository -3. ✅ Push to repository (authorized) -4. ✅ Push rejected (unauthorized) -5. ✅ Multiple concurrent operations - -**Implementation:** -```rust -// tests/e2e/git_client.rs - -#[tokio::test] -async fn test_real_git_clone() { - let app = test_app().await; - - // Setup repository with content - let (announcement, _) = app.create_repo_with_commits() - .commit("Initial", "README.md", "# Test") - .build() - .await; - - // Clone with real git - let temp = TempDir::new().unwrap(); - let url = format!( - "http://{}/{}/{}.git", - app.domain(), - announcement.author_npub(), - announcement.identifier() - ); - - let output = Command::new("git") - .args(&["clone", &url]) - .current_dir(&temp) - .output() - .await - .unwrap(); - - assert!(output.status.success()); - - let cloned_path = temp.path().join(announcement.identifier()); - assert!(cloned_path.exists()); - assert!(cloned_path.join("README.md").exists()); -} -``` - ---- - -## Testing Strategy - -### Unit Tests (40%) -- Git repository operations (git2) -- Protocol parsing -- Authorization logic -- Pure functions, no I/O - -### Integration Tests (30%) -- HTTP handlers with test server -- Repository + Nostr event interaction -- Multi-maintainer flows -- State validation - -### Compliance Tests (20%) -- GRASP-01 Git requirements -- Use grasp-audit library -- Spec-driven assertions - -### E2E Tests (10%) -- Real git client operations -- End-to-end workflows -- Performance testing - ---- - -## Implementation Order - -### Week 1: Foundation -1. Add git2 dependency -2. Implement GitRepository (Phase 1) -3. Write unit tests for repository operations -4. Test repository creation from announcements - -### Week 2: Protocol & Authorization -1. Implement protocol parsing (Phase 2) -2. Implement authorization logic (Phase 3) -3. Write unit tests for both -4. Integration tests for validation - -### Week 3: HTTP & Integration -1. Implement HTTP handlers (Phase 4) -2. Integrate with Nostr events (Phase 5) -3. Integration tests for full flow -4. CORS and error handling - -### Week 4: E2E & Polish -1. E2E tests with real git (Phase 6) -2. Performance testing -3. GRASP-01 compliance testing -4. Documentation and examples - ---- - -## Success Criteria - -### Functional -- ✅ Clone repository via HTTP -- ✅ Push authorized commits -- ✅ Reject unauthorized pushes -- ✅ Support multi-maintainer -- ✅ Support refs/nostr/* for PRs - -### Quality -- ✅ >80% unit test coverage -- ✅ All integration tests pass -- ✅ GRASP-01 compliance 100% -- ✅ E2E tests with real git - -### Performance -- ✅ Clone 1MB repo < 1s -- ✅ Push validation < 100ms -- ✅ 100 concurrent ops without errors - ---- - -## Next Steps - -1. **Review this plan** - Does the hybrid approach make sense? -2. **Start Phase 1** - Add git2, implement GitRepository -3. **Write first test** - test_init_bare_repository -4. **Iterate with TDD** - Red → Green → Refactor - ---- - -## Questions for Review - -1. **Hybrid approach?** git2 + system git + HTTP layer - good balance? -2. **git-http-backend crate?** Worth using or implement minimal HTTP layer? -3. **Authorization granularity?** Validate per-ref or entire push? -4. **Error messages?** How detailed for push rejections? -5. **Testing scope?** Is 6 phases reasonable for first iteration? - ---- - -**Ready to proceed?** Let me know if this plan looks good, or if you'd like to adjust the approach! diff --git a/docs/archive/2025-11-04-evening/INDEX.md b/docs/archive/2025-11-04-evening/INDEX.md deleted file mode 100644 index cb06dd5..0000000 --- a/docs/archive/2025-11-04-evening/INDEX.md +++ /dev/null @@ -1,314 +0,0 @@ -# Work Directory Index - -**Last Updated:** November 4, 2025 -**Status:** Ready for Implementation - ---- - -## 📁 Quick Navigation - -### 🚀 START HERE -**[NEXT_SESSION_START_HERE.md](NEXT_SESSION_START_HERE.md)** - Begin next session with this - -### 📊 Status & Progress -- **[STATUS.txt](STATUS.txt)** - Visual status summary (quick reference) -- **[current_status.md](current_status.md)** - Detailed project status -- **[session-summary.md](session-summary.md)** - What we accomplished this session - -### 📖 Understanding & Planning -- **[review-summary.md](review-summary.md)** - GRASP protocol review findings -- **[architecture-diagram.md](architecture-diagram.md)** - Visual architecture reference -- **[implementation-checklist.md](implementation-checklist.md)** - Detailed task checklist - -### 📋 Reference -- **[README.md](README.md)** - Work directory purpose and guidelines - ---- - -## 📚 Document Purposes - -### NEXT_SESSION_START_HERE.md -**Purpose:** Step-by-step implementation guide -**When to use:** Starting next coding session -**Contains:** -- Immediate goal (actix-web integration) -- Critical architecture understanding -- 8-step implementation plan with code examples -- Verification steps for each step -- Common issues and solutions -- Success criteria - -**Read this:** When you're ready to start coding - ---- - -### STATUS.txt -**Purpose:** Quick visual status overview -**When to use:** Quick check of where we are -**Contains:** -- Current compliance percentages -- Critical discoveries -- Next session plan -- Key references -- Success criteria - -**Read this:** When you need a quick reminder - ---- - -### current_status.md -**Purpose:** Comprehensive project status -**When to use:** Understanding overall context -**Contains:** -- Complete GRASP-01 requirements checklist -- Architecture understanding -- Current implementation status -- Known issues and blockers -- Progress summary -- Key references - -**Read this:** When you need detailed context - ---- - -### session-summary.md -**Purpose:** Summary of this review session -**When to use:** Remembering what we did -**Contains:** -- Session goals and accomplishments -- Key discoveries (7 major findings) -- Documents created -- Lessons learned -- Next steps - -**Read this:** When you want to know what happened this session - ---- - -### review-summary.md -**Purpose:** GRASP protocol review findings -**When to use:** Understanding why we're making changes -**Contains:** -- 10 critical discoveries from GRASP spec -- Evidence for each finding -- Impact and action items -- Compliance status -- Immediate next steps - -**Read this:** When you need to justify architectural decisions - ---- - -### architecture-diagram.md -**Purpose:** Visual architecture reference -**When to use:** During implementation for reference -**Contains:** -- Current vs. target architecture diagrams -- Request flow examples (5 scenarios) -- Component responsibilities -- Configuration flow -- Test architecture -- File structure -- Comparison with ngit-relay - -**Read this:** When you need to visualize the system - ---- - -### implementation-checklist.md -**Purpose:** Detailed task checklist -**When to use:** Tracking implementation progress -**Contains:** -- 5 phases with detailed tasks -- Verification steps for each task -- Manual testing procedures -- Automated testing commands -- Acceptance criteria -- Known issues to watch for -- Reference commands - -**Read this:** While implementing to track progress - ---- - -## 🎯 Recommended Reading Order - -### For Next Session (Implementation) - -1. **STATUS.txt** (1 min) - - Quick reminder of where we are - -2. **NEXT_SESSION_START_HERE.md** (10 min) - - Understand the immediate goal - - Review the 8-step plan - -3. **architecture-diagram.md** (5 min) - - Visual reference for what we're building - -4. **implementation-checklist.md** (ongoing) - - Check off tasks as you complete them - -### For Understanding Context - -1. **session-summary.md** (5 min) - - What we accomplished this session - -2. **current_status.md** (10 min) - - Overall project status - -3. **review-summary.md** (15 min) - - Why we're making these changes - -### For Reference During Coding - -- **architecture-diagram.md** - Visual reference -- **implementation-checklist.md** - Task tracking -- **NEXT_SESSION_START_HERE.md** - Step-by-step guide - ---- - -## 🔍 Finding Specific Information - -### "How do I implement X?" -→ **NEXT_SESSION_START_HERE.md** (Step-by-step with code) - -### "Why are we doing X?" -→ **review-summary.md** (Findings from GRASP review) - -### "What's the overall status?" -→ **current_status.md** or **STATUS.txt** - -### "What did we do this session?" -→ **session-summary.md** - -### "How does the architecture work?" -→ **architecture-diagram.md** - -### "What tasks are left?" -→ **implementation-checklist.md** - -### "What are the requirements?" -→ **current_status.md** (GRASP-01 checklist) - -### "How do I test X?" -→ **implementation-checklist.md** (Testing section) - ---- - -## 📊 Document Relationships - -``` -STATUS.txt - ↓ (quick overview) -current_status.md - ↓ (detailed status) -session-summary.md - ↓ (what we did) -review-summary.md - ↓ (why we're doing this) -architecture-diagram.md - ↓ (visual reference) -NEXT_SESSION_START_HERE.md - ↓ (how to implement) -implementation-checklist.md - ↓ (track progress) -``` - ---- - -## 🗂️ File Lifecycle - -### Active (Use These) -- ✅ NEXT_SESSION_START_HERE.md - Update for each phase -- ✅ current_status.md - Update as we progress -- ✅ implementation-checklist.md - Check off as we go -- ✅ STATUS.txt - Update after each phase - -### Reference (Keep These) -- 📖 architecture-diagram.md - Permanent reference -- 📖 review-summary.md - Permanent reference -- 📖 session-summary.md - Historical record - -### Archive (After Implementation) -- 📦 implementation-checklist.md → Delete when phase complete -- 📦 NEXT_SESSION_START_HERE.md → Update for next phase -- 📦 session-summary.md → Move to docs/archive/ - ---- - -## ✨ Quick Tips - -### Starting a New Session -1. Read STATUS.txt (1 min) -2. Read NEXT_SESSION_START_HERE.md (10 min) -3. Open implementation-checklist.md to track progress -4. Start coding! - -### When Stuck -1. Check architecture-diagram.md for visual reference -2. Check NEXT_SESSION_START_HERE.md for step details -3. Check review-summary.md for why we're doing it -4. Check ../grasp/01.md for requirements - -### Ending a Session -1. Update current_status.md with progress -2. Update STATUS.txt with new status -3. Check off completed tasks in implementation-checklist.md -4. Update NEXT_SESSION_START_HERE.md if needed - -### After Phase Complete -1. Archive implementation-checklist.md -2. Update NEXT_SESSION_START_HERE.md for next phase -3. Update current_status.md with new status -4. Create new session-summary.md if needed - ---- - -## 📈 Progress Tracking - -Use these documents to track progress: - -### Daily -- [ ] Check STATUS.txt -- [ ] Update implementation-checklist.md -- [ ] Follow NEXT_SESSION_START_HERE.md - -### Weekly -- [ ] Update current_status.md -- [ ] Update STATUS.txt -- [ ] Review progress against checklist - -### After Each Phase -- [ ] Update current_status.md -- [ ] Create new session-summary.md -- [ ] Update NEXT_SESSION_START_HERE.md -- [ ] Archive completed documents - ---- - -## 🎯 Current Phase - -**Phase:** actix-web Integration -**Status:** Ready to Start -**Start With:** NEXT_SESSION_START_HERE.md -**Track With:** implementation-checklist.md -**Reference:** architecture-diagram.md - ---- - -## 📞 Quick Reference - -| Need | Document | -|------|----------| -| Start coding | NEXT_SESSION_START_HERE.md | -| Quick status | STATUS.txt | -| Detailed status | current_status.md | -| Why we're doing this | review-summary.md | -| How it works | architecture-diagram.md | -| Task list | implementation-checklist.md | -| What we did | session-summary.md | - ---- - -**Last Updated:** November 4, 2025 -**Next Update:** After actix-web integration complete diff --git a/docs/archive/2025-11-04-evening/NEXT_SESSION_START_HERE.md b/docs/archive/2025-11-04-evening/NEXT_SESSION_START_HERE.md deleted file mode 100644 index 890d5ed..0000000 --- a/docs/archive/2025-11-04-evening/NEXT_SESSION_START_HERE.md +++ /dev/null @@ -1,650 +0,0 @@ -# Next Session Start Here - -**Date:** November 4, 2025 -**Purpose:** Quick start guide for next development session -**Status:** Ready for actix-web integration - ---- - -## 🎯 Immediate Goal - -**Integrate actix-web to serve both Nostr relay (WebSocket) and Git HTTP on the SAME PORT.** - -This is the critical architectural fix needed to match GRASP-01 requirements and ngit-relay's design. - ---- - -## 🚨 Critical Understanding - -### Single Port Architecture (from ../ngit-relay) - -``` -┌─────────────────────────────────────┐ -│ Single Port (8080) │ -│ │ -│ ┌─────────────────────────────┐ │ -│ │ HTTP/WebSocket Router │ │ -│ │ (nginx in ngit-relay) │ │ -│ │ (actix-web in ngit-grasp) │ │ -│ └──────────┬──────────────────┘ │ -│ │ │ -│ ┌──────┴──────┐ │ -│ ↓ ↓ │ -│ Git HTTP Nostr Relay │ -│ // / (WebSocket) │ -│ .git │ -└─────────────────────────────────────┘ -``` - -**Key Points:** -1. ONE port listens for all traffic -2. Router inspects path and decides where to send request -3. Git paths go to Git handler -4. Everything else goes to Nostr relay (with WebSocket upgrade) -5. CORS headers on ALL responses - ---- - -## 📋 Step-by-Step Implementation Plan - -### Step 1: Add actix-web Dependencies - -**File:** `Cargo.toml` - -```toml -[dependencies] -# Existing... -actix-web = "4" -actix-cors = "0.7" -actix-ws = "0.3" # For WebSocket support - -# Git HTTP backend -git-http-backend = "0.2" # Check latest version -``` - -**Why:** -- `actix-web` - HTTP framework with routing -- `actix-cors` - Easy CORS middleware -- `actix-ws` - WebSocket support -- `git-http-backend` - Git Smart HTTP protocol - -### Step 2: Create HTTP Router Module - -**File:** `src/http/mod.rs` (NEW) - -```rust -//! HTTP server with routing for Git and Nostr - -use actix_web::{web, App, HttpServer}; -use actix_cors::Cors; - -pub mod git; -pub mod nostr; - -pub async fn run_server(config: Config, storage: Storage) -> Result<()> { - let bind_addr = config.bind_address.clone(); - - HttpServer::new(move || { - App::new() - // CORS middleware (GRASP-01 requirement) - .wrap( - Cors::default() - .allow_any_origin() - .allowed_methods(vec!["GET", "POST"]) - .allowed_headers(vec!["Content-Type"]) - .max_age(3600) - ) - // Git HTTP routes - .service( - web::scope("/{npub}/{repo}") - .guard(guard::fn_guard(|ctx| { - // Only match *.git paths - ctx.head().uri.path().ends_with(".git") - })) - .route("", web::get().to(git::handle_git_request)) - .route("/{tail:.*}", web::to(git::handle_git_request)) - ) - // Nostr relay (WebSocket at /) - .route("/", web::get().to(nostr::handle_websocket)) - // Static files (optional) - .route("/", web::get().to(nostr::handle_http_root)) - }) - .bind(bind_addr)? - .run() - .await?; - - Ok(()) -} -``` - -**Why:** -- Single HTTP server listening on one port -- Routes by URL path pattern -- CORS applied to all routes -- Git paths (*.git) go to Git handler -- Root path (/) handles WebSocket upgrade for Nostr - -### Step 3: Create Git HTTP Handler - -**File:** `src/http/git.rs` (NEW) - -```rust -//! Git Smart HTTP handler - -use actix_web::{web, HttpRequest, HttpResponse, Result}; -use git_http_backend::{GitHttpBackend, Method}; - -pub async fn handle_git_request( - req: HttpRequest, - body: web::Bytes, - path: web::Path<(String, String)>, -) -> Result { - let (npub, repo) = path.into_inner(); - - // Construct repository path - let repo_path = format!("{}/{}/{}", - config.git_data_path, npub, repo); - - // Check if repository exists - if !std::path::Path::new(&repo_path).exists() { - return Ok(HttpResponse::NotFound() - .body("Repository not found")); - } - - // Parse Git HTTP request - let method = match *req.method() { - actix_web::http::Method::GET => Method::Get, - actix_web::http::Method::POST => Method::Post, - _ => return Ok(HttpResponse::MethodNotAllowed().finish()), - }; - - // Use git-http-backend to handle request - let backend = GitHttpBackend::new(&repo_path); - let response = backend.handle(method, req.path(), &body)?; - - // Convert to actix HttpResponse - Ok(HttpResponse::Ok() - .content_type(response.content_type) - .body(response.body)) -} -``` - -**Why:** -- Handles Git Smart HTTP protocol -- Serves from `{GIT_DATA_PATH}/{npub}/{repo}.git` -- Uses `git-http-backend` crate for protocol details -- Returns 404 if repo doesn't exist - -### Step 4: Create Nostr WebSocket Handler - -**File:** `src/http/nostr.rs` (NEW) - -```rust -//! Nostr relay WebSocket handler - -use actix_web::{web, HttpRequest, HttpResponse, Result}; -use actix_ws::Message; - -pub async fn handle_websocket( - req: HttpRequest, - stream: web::Payload, - storage: web::Data, -) -> Result { - // Upgrade to WebSocket - let (response, mut session, mut msg_stream) = actix_ws::handle(&req, stream)?; - - // Spawn task to handle WebSocket messages - actix_web::rt::spawn(async move { - while let Some(Ok(msg)) = msg_stream.next().await { - match msg { - Message::Text(text) => { - // Handle Nostr message (EVENT, REQ, CLOSE) - let responses = handle_nostr_message(&text, &storage).await; - for response in responses { - session.text(response).await.ok(); - } - } - Message::Ping(bytes) => { - session.pong(&bytes).await.ok(); - } - Message::Close(_) => break, - _ => {} - } - } - }); - - Ok(response) -} - -pub async fn handle_http_root() -> Result { - // Serve static HTML for browsers - Ok(HttpResponse::Ok() - .content_type("text/html") - .body("

ngit-grasp

Nostr relay at ws://

")) -} -``` - -**Why:** -- Handles WebSocket upgrade at `/` -- Reuses existing Nostr message handling logic -- Returns HTML for browsers (non-WebSocket requests) - -### Step 5: Update main.rs - -**File:** `src/main.rs` - -```rust -use anyhow::Result; -use tracing::{info, Level}; -use tracing_subscriber::FmtSubscriber; - -mod config; -mod http; // NEW -mod nostr; -mod storage; - -use config::Config; - -#[tokio::main] -async fn main() -> Result<()> { - // Initialize tracing - let subscriber = FmtSubscriber::builder() - .with_max_level(Level::DEBUG) - .finish(); - tracing::subscriber::set_global_default(subscriber)?; - - info!("Starting ngit-grasp..."); - - // Load configuration - let config = Config::from_env()?; - info!("Configuration: {}", config.bind_address); - - // Initialize storage - let storage = storage::Storage::new(&config)?; - info!("Storage initialized at: {}", config.relay_data_path); - - // Start HTTP server (Git + Nostr on same port) - info!("Starting server on {}", config.bind_address); - http::run_server(config, storage).await?; - - Ok(()) -} -``` - -**Why:** -- Replaces separate relay with unified HTTP server -- Single entry point for all services -- Simpler architecture - -### Step 6: Update Configuration - -**File:** `src/config.rs` - -Add field for Git data path: - -```rust -pub struct Config { - pub bind_address: String, - pub domain: String, - pub relay_data_path: String, - pub git_data_path: String, // NEW - // ... other fields -} - -impl Config { - pub fn from_env() -> Result { - Ok(Config { - bind_address: env::var("NGIT_BIND_ADDRESS") - .unwrap_or_else(|_| "127.0.0.1:8080".to_string()), - domain: env::var("NGIT_DOMAIN")?, - relay_data_path: env::var("NGIT_RELAY_DATA_PATH") - .unwrap_or_else(|_| "./data/relay".to_string()), - git_data_path: env::var("NGIT_GIT_DATA_PATH") // NEW - .unwrap_or_else(|_| "./data/repos".to_string()), - // ... - }) - } -} -``` - -**File:** `.env.example` - -```bash -# Service Configuration -NGIT_DOMAIN=example.com -NGIT_BIND_ADDRESS=127.0.0.1:8080 - -# Relay Information -NGIT_RELAY_NAME="ngit-grasp instance" -NGIT_RELAY_DESCRIPTION="Rust GRASP implementation" -NGIT_OWNER_NPUB="npub1..." - -# Storage Paths -NGIT_GIT_DATA_PATH=./data/repos -NGIT_RELAY_DATA_PATH=./data/relay - -# Logging -NGIT_LOG_LEVEL=INFO -RUST_LOG=info -``` - -### Step 7: Update Tests - -**File:** `tests/common/relay.rs` - -Update `start_with_port` to pass domain correctly: - -```rust -pub async fn start_with_port(port: u16) -> Self { - let bind_address = format!("127.0.0.1:{}", port); - let domain = format!("127.0.0.1:{}", port); // NEW - let url = format!("ws://{}", domain); - - let process = Command::new(&binary_path) - .env("NGIT_BIND_ADDRESS", &bind_address) - .env("NGIT_DOMAIN", &domain) // UPDATED - .env("NGIT_GIT_DATA_PATH", "./test-data/repos") // NEW - .env("NGIT_RELAY_DATA_PATH", "./test-data/relay") // NEW - .env("RUST_LOG", "warn") - .stdout(Stdio::null()) - .stderr(Stdio::null()) - .spawn() - .expect("Failed to start relay process"); - - // ... rest of method -} -``` - -**Why:** -- Domain must match bind address for announcement validation -- Separate test data directories -- Clean up test data after tests - -### Step 8: Add Git HTTP Tests - -**File:** `tests/grasp01_git_http.rs` (NEW) - -```rust -//! GRASP-01 Git HTTP Integration Tests -//! -//! Reference: ../grasp/01.md lines 15-40 -//! -//! These tests verify Git Smart HTTP service requirements: -//! - Serve repos at //.git -//! - Accept pushes matching state announcements -//! - CORS support - -mod common; - -use common::TestRelay; -use std::process::Command; - -#[tokio::test] -async fn test_git_clone_basic() { - // Reference: ../grasp/01.md line 15 - // MUST serve git repository via unauthenticated git smart http service - - let relay = TestRelay::start().await; - let domain = relay.domain(); - - // TODO: Create test repository announcement - // TODO: Clone via git clone http://{domain}/{npub}/{id}.git - - relay.stop().await; -} - -#[tokio::test] -async fn test_cors_headers() { - // Reference: ../grasp/01.md lines 32-40 - // MUST include CORS headers on all responses - - let relay = TestRelay::start().await; - let url = format!("http://{}/", relay.domain()); - - let response = reqwest::get(&url).await.unwrap(); - - // Check CORS headers - assert_eq!( - response.headers().get("access-control-allow-origin"), - Some(&"*".parse().unwrap()) - ); - - relay.stop().await; -} -``` - -**Why:** -- Tests reference GRASP protocol line numbers -- Verifies Git HTTP functionality -- Checks CORS compliance - ---- - -## 🔍 Verification Steps - -After implementing the above: - -### 1. Build and Run - -```bash -# Build -cargo build - -# Run server -NGIT_DOMAIN=localhost:8080 \ -NGIT_BIND_ADDRESS=127.0.0.1:8080 \ -NGIT_GIT_DATA_PATH=./data/repos \ -NGIT_RELAY_DATA_PATH=./data/relay \ -cargo run -``` - -### 2. Test Nostr Relay (WebSocket) - -```bash -# In another terminal -cd grasp-audit -cargo run -- --url ws://localhost:8080 -``` - -**Expected:** NIP-01 smoke tests should pass - -### 3. Test Git HTTP (Manual) - -```bash -# Create test repository -mkdir -p ./data/repos/npub1test/test-repo.git -cd ./data/repos/npub1test/test-repo.git -git init --bare - -# Try to clone -git clone http://localhost:8080/npub1test/test-repo.git -``` - -**Expected:** Should clone successfully (even if empty) - -### 4. Test CORS - -```bash -curl -v http://localhost:8080/ -H "Origin: https://example.com" -``` - -**Expected:** Response should include: -``` -access-control-allow-origin: * -access-control-allow-methods: GET, POST -access-control-allow-headers: Content-Type -``` - -### 5. Run Integration Tests - -```bash -# All tests -cargo test - -# Just NIP-01 -cargo test --test nip01_compliance - -# Just Git HTTP (when implemented) -cargo test --test grasp01_git_http -``` - -**Expected:** All tests pass - ---- - -## 🐛 Common Issues & Solutions - -### Issue: Port Already in Use - -**Symptom:** "Address already in use" error - -**Solution:** -```bash -# Find process using port -lsof -i :8080 - -# Kill it -kill -9 - -# Or use different port -NGIT_BIND_ADDRESS=127.0.0.1:8081 cargo run -``` - -### Issue: WebSocket Upgrade Fails - -**Symptom:** WebSocket connection refused - -**Solution:** -- Check actix-web WebSocket handling -- Verify `Upgrade: websocket` header is present -- Check actix-ws is properly configured - -### Issue: Git Clone Fails - -**Symptom:** "repository not found" or protocol error - -**Solution:** -- Verify repository exists at correct path -- Check git-http-backend configuration -- Ensure repository is bare (`git init --bare`) -- Check file permissions - -### Issue: CORS Headers Missing - -**Symptom:** Browser console shows CORS error - -**Solution:** -- Verify CORS middleware is applied -- Check middleware order (CORS should be first) -- Test with curl to see actual headers - ---- - -## 📚 Key Resources - -### GRASP Protocol -- `../grasp/01.md` - **THE SPEC** - Read this first! -- Lines 1-14: Nostr relay requirements -- Lines 15-31: Git HTTP service requirements -- Lines 32-40: CORS requirements - -### Reference Implementation -- `../ngit-relay/src/nginx.conf` - **ROUTING PATTERN** - - Lines 8-13: Single port listener - - Lines 15-48: Git HTTP routing - - Lines 50-94: Nostr relay routing -- `../ngit-relay/docker-compose.yml` - Port configuration -- `../ngit-relay/.env.example` - Environment variables - -### actix-web Documentation -- [Routing](https://actix.rs/docs/url-dispatch/) -- [WebSocket](https://actix.rs/docs/websockets/) -- [CORS](https://docs.rs/actix-cors/) - -### git-http-backend Crate -- [Docs](https://docs.rs/git-http-backend/) -- [Examples](https://github.com/w4/git-http-backend/tree/master/examples) - ---- - -## ✅ Success Criteria - -You'll know this step is complete when: - -1. ✅ Server starts on single port -2. ✅ WebSocket connects at `ws://localhost:8080/` -3. ✅ NIP-01 smoke tests pass -4. ✅ Can clone Git repo at `http://localhost:8080/npub.../repo.git` -5. ✅ CORS headers present on all responses -6. ✅ OPTIONS requests return 204 -7. ✅ All integration tests pass - ---- - -## 🎯 After This Step - -Once actix-web integration is complete: - -1. **Repository Provisioning** - - Create repos when announcements received - - Initialize bare repositories - - Set up directory structure - -2. **Push Authorization** - - Intercept git-receive-pack - - Validate against state announcements - - Handle maintainer sets - -3. **Full GRASP-01 Compliance** - - All tests passing - - Ready for production testing - ---- - -## 💡 Tips - -1. **Start Simple** - - Get basic HTTP routing working first - - Add WebSocket support second - - Add Git HTTP last - -2. **Test Incrementally** - - Test each component as you add it - - Don't wait until everything is done - -3. **Use curl for Debugging** - ```bash - # Test HTTP - curl -v http://localhost:8080/ - - # Test CORS - curl -v http://localhost:8080/ -H "Origin: https://example.com" - - # Test Git info/refs - curl http://localhost:8080/npub.../repo.git/info/refs?service=git-upload-pack - ``` - -4. **Check ngit-relay for Patterns** - - nginx.conf shows exact routing logic - - Copy the pattern, not the implementation - -5. **Keep Tests Running** - ```bash - # In one terminal - cargo watch -x 'test --test nip01_compliance' - - # Make changes, tests auto-run - ``` - ---- - -**Ready to Start?** Begin with Step 1 (Add Dependencies) - -**Questions?** Check `work/current_status.md` for context - -**Stuck?** Review `../ngit-relay/src/nginx.conf` for routing pattern - ---- - -**Last Updated:** November 4, 2025 -**Next Update:** After actix-web integration complete diff --git a/docs/archive/2025-11-04-evening/STATUS.txt b/docs/archive/2025-11-04-evening/STATUS.txt deleted file mode 100644 index 8c70db2..0000000 --- a/docs/archive/2025-11-04-evening/STATUS.txt +++ /dev/null @@ -1,78 +0,0 @@ -╔════════════════════════════════════════════════════════════════╗ -║ NGIT-GRASP STATUS ║ -║ November 4, 2025 ║ -╚════════════════════════════════════════════════════════════════╝ - -📊 OVERALL STATUS: Planning Complete, Ready for Implementation - -┌────────────────────────────────────────────────────────────────┐ -│ CRITICAL DISCOVERY: Architecture Was Wrong! │ -├────────────────────────────────────────────────────────────────┤ -│ ❌ Previous: Nostr on :8080, Git on :8081 (WRONG!) │ -│ ✅ Correct: BOTH on :8080, routed by path (REQUIRED!) │ -│ │ -│ Fix: Integrate actix-web for HTTP routing │ -└────────────────────────────────────────────────────────────────┘ - -┌────────────────────────────────────────────────────────────────┐ -│ COMPLIANCE STATUS │ -├────────────────────────────────────────────────────────────────┤ -│ NIP-01 (Nostr Relay): ████████░░ 80% (needs NIP-11 fix) │ -│ NIP-34 (Git Announce): ████░░░░░░ 40% (needs validation) │ -│ GRASP-01 (Core): ██░░░░░░░░ 20% (needs Git HTTP) │ -│ │ -│ Target After Next Phase: ████████░░ 80% (Git HTTP working) │ -└────────────────────────────────────────────────────────────────┘ - -┌────────────────────────────────────────────────────────────────┐ -│ WORK DOCUMENTS CREATED │ -├────────────────────────────────────────────────────────────────┤ -│ ✅ current_status.md - Overall project status │ -│ ✅ NEXT_SESSION_START_HERE.md - Implementation guide (START!) │ -│ ✅ review-summary.md - GRASP protocol findings │ -│ ✅ architecture-diagram.md - Visual reference │ -│ ✅ implementation-checklist.md - Detailed task list │ -│ ✅ session-summary.md - What we accomplished │ -└────────────────────────────────────────────────────────────────┘ - -┌────────────────────────────────────────────────────────────────┐ -│ NEXT SESSION: Implement actix-web Integration │ -├────────────────────────────────────────────────────────────────┤ -│ 1. Add dependencies (actix-web, actix-cors, git-http-backend) │ -│ 2. Create src/http/mod.rs (HTTP server + routing) │ -│ 3. Create src/http/git.rs (Git Smart HTTP handler) │ -│ 4. Create src/http/nostr.rs (WebSocket handler) │ -│ 5. Update src/main.rs (use new HTTP server) │ -│ 6. Update tests (verify single-port works) │ -│ 7. Manual testing (clone, WebSocket, CORS) │ -│ 8. Automated testing (all tests pass) │ -│ │ -│ Estimated Time: 3-6 hours │ -│ Confidence: HIGH ✅ │ -└────────────────────────────────────────────────────────────────┘ - -┌────────────────────────────────────────────────────────────────┐ -│ KEY REFERENCES │ -├────────────────────────────────────────────────────────────────┤ -│ 📖 ../grasp/01.md - THE SPEC (lines 1-40) │ -│ 📖 ../ngit-relay/src/nginx.conf - Routing pattern reference │ -│ 📖 work/NEXT_SESSION_START_HERE.md - Implementation guide │ -│ 📖 work/architecture-diagram.md - Visual architecture │ -│ 📖 work/implementation-checklist.md - Task checklist │ -└────────────────────────────────────────────────────────────────┘ - -┌────────────────────────────────────────────────────────────────┐ -│ SUCCESS CRITERIA (Next Phase) │ -├────────────────────────────────────────────────────────────────┤ -│ ✅ Server starts on single port (8080) │ -│ ✅ WebSocket connects at ws://localhost:8080/ │ -│ ✅ NIP-01 smoke tests pass │ -│ ✅ Can clone Git repo via http://localhost:8080/npub.../x.git │ -│ ✅ CORS headers present on all responses │ -│ ✅ OPTIONS requests return 204 No Content │ -│ ✅ All integration tests pass │ -└────────────────────────────────────────────────────────────────┘ - -╔════════════════════════════════════════════════════════════════╗ -║ 🚀 READY TO IMPLEMENT - Start with NEXT_SESSION_START_HERE.md ║ -╚════════════════════════════════════════════════════════════════╝ diff --git a/docs/archive/2025-11-04-evening/architecture-diagram.md b/docs/archive/2025-11-04-evening/architecture-diagram.md deleted file mode 100644 index 056e551..0000000 --- a/docs/archive/2025-11-04-evening/architecture-diagram.md +++ /dev/null @@ -1,442 +0,0 @@ -# ngit-grasp Architecture Diagram - -**Date:** November 4, 2025 -**Purpose:** Visual reference for single-port architecture - ---- - -## Current Architecture (WRONG ❌) - -``` -┌─────────────────────────────────────────┐ -│ Port 8080 │ -│ ┌───────────────────────────────────┐ │ -│ │ Nostr Relay (WebSocket) │ │ -│ │ - NIP-01 protocol │ │ -│ │ - Event storage │ │ -│ │ - Subscriptions │ │ -│ └───────────────────────────────────┘ │ -└─────────────────────────────────────────┘ - -┌─────────────────────────────────────────┐ -│ Port 8081 (WRONG!) │ -│ ┌───────────────────────────────────┐ │ -│ │ Git HTTP Server │ │ -│ │ - Not implemented │ │ -│ └───────────────────────────────────┘ │ -└─────────────────────────────────────────┘ -``` - -**Problem:** GRASP-01 requires single port! - ---- - -## Target Architecture (CORRECT ✅) - -``` -┌─────────────────────────────────────────────────────────────┐ -│ Single Port (8080) │ -│ │ -│ ┌───────────────────────────────────────────────────────┐ │ -│ │ actix-web HTTP Server │ │ -│ │ │ │ -│ │ ┌──────────────────────────────────────────────────┐ │ │ -│ │ │ CORS Middleware (ALL requests) │ │ │ -│ │ │ - Access-Control-Allow-Origin: * │ │ │ -│ │ │ - Access-Control-Allow-Methods: GET, POST │ │ │ -│ │ │ - Access-Control-Allow-Headers: Content-Type │ │ │ -│ │ └──────────────────────────────────────────────────┘ │ │ -│ │ │ │ -│ │ ┌──────────────────────────────────────────────────┐ │ │ -│ │ │ HTTP Router │ │ │ -│ │ │ │ │ │ -│ │ │ Path Pattern Matching: │ │ │ -│ │ │ - //.git/* → Git Handler │ │ │ -│ │ │ - /* → Nostr Handler │ │ │ -│ │ └──────────────────────────────────────────────────┘ │ │ -│ │ │ │ -│ │ ┌────────────────────┐ ┌─────────────────────────┐ │ │ -│ │ │ Git HTTP Handler │ │ Nostr Relay Handler │ │ │ -│ │ │ │ │ │ │ │ -│ │ │ ┌──────────────┐ │ │ ┌──────────────────┐ │ │ │ -│ │ │ │ git-http- │ │ │ │ WebSocket Upgrade│ │ │ │ -│ │ │ │ backend │ │ │ │ │ │ │ │ -│ │ │ │ │ │ │ │ ┌──────────────┐ │ │ │ │ -│ │ │ │ - info/refs │ │ │ │ │ NIP-01 │ │ │ │ │ -│ │ │ │ - upload-pack│ │ │ │ │ - EVENT │ │ │ │ │ -│ │ │ │ - receive-pack │ │ │ │ - REQ │ │ │ │ │ -│ │ │ │ │ │ │ │ │ - CLOSE │ │ │ │ │ -│ │ │ │ Authorization: │ │ │ └──────────────┘ │ │ │ │ -│ │ │ │ - Query state│ │ │ │ │ │ │ │ -│ │ │ │ - Validate │ │ │ │ ┌──────────────┐ │ │ │ │ -│ │ │ │ - Accept/ │ │ │ │ │ NIP-11 │ │ │ │ │ -│ │ │ │ Reject │ │ │ │ │ - GRASP fields│ │ │ │ │ -│ │ │ └──────────────┘ │ │ │ └──────────────┘ │ │ │ │ -│ │ │ │ │ │ │ │ │ │ -│ │ │ Repository: │ │ │ ┌──────────────┐ │ │ │ │ -│ │ │ {GIT_DATA_PATH}/ │ │ │ │ NIP-34 │ │ │ │ │ -│ │ │ {npub}/ │ │ │ │ - Announce │ │ │ │ │ -│ │ │ {identifier}.git │ │ │ │ - State │ │ │ │ │ -│ │ │ │ │ │ │ - Validate │ │ │ │ │ -│ │ └────────────────────┘ │ │ └──────────────┘ │ │ │ │ -│ │ │ │ │ │ │ │ -│ │ │ │ HTTP Root: │ │ │ │ -│ │ │ │ - Serve HTML │ │ │ │ -│ │ │ │ - NIP-11 JSON │ │ │ │ -│ │ │ └──────────────────┘ │ │ │ -│ │ │ │ │ │ -│ └───────────────────────────┴─────────────────────────┘ │ │ -│ │ -│ ┌───────────────────────────────────────────────────────┐ │ -│ │ Storage Layer │ │ -│ │ │ │ -│ │ ┌──────────────────────┐ ┌──────────────────────┐ │ │ -│ │ │ Git Repositories │ │ Nostr Events DB │ │ │ -│ │ │ │ │ │ │ │ -│ │ │ {GIT_DATA_PATH}/ │ │ {RELAY_DATA_PATH}/ │ │ │ -│ │ │ ├── npub1.../ │ │ - Announcements │ │ │ -│ │ │ │ ├── repo1.git/ │ │ - State events │ │ │ -│ │ │ │ └── repo2.git/ │ │ - Issues/Patches │ │ │ -│ │ │ └── npub2.../ │ │ - Other events │ │ │ -│ │ │ └── repo3.git/ │ │ │ │ │ -│ │ └──────────────────────┘ └──────────────────────┘ │ │ -│ └───────────────────────────────────────────────────────┘ │ -└─────────────────────────────────────────────────────────────┘ -``` - ---- - -## Request Flow Examples - -### Example 1: Git Clone - -``` -Client: git clone http://localhost:8080/npub1abc.../my-repo.git - ↓ -actix-web receives HTTP GET request - ↓ -CORS middleware adds headers - ↓ -Router matches path: /npub1abc.../my-repo.git - ↓ -Git Handler receives request - ↓ -git-http-backend processes: - - GET /npub1abc.../my-repo.git/info/refs?service=git-upload-pack - ↓ -Response includes: - - CORS headers - - Git protocol data - - Capabilities: allow-reachable-sha1-in-want, allow-tip-sha1-in-want - ↓ -Client receives data and clones repository -``` - -### Example 2: Git Push - -``` -Client: git push http://localhost:8080/npub1abc.../my-repo.git main - ↓ -actix-web receives HTTP POST request - ↓ -CORS middleware adds headers - ↓ -Router matches path: /npub1abc.../my-repo.git - ↓ -Git Handler receives request - ↓ -BEFORE spawning git-receive-pack: - 1. Parse ref updates from request body - 2. Query latest state announcement from relay - 3. Validate pusher in maintainer set - 4. Validate ref updates match state - ↓ -If validation passes: - - Spawn git-receive-pack - - Stream response back to client - ↓ -If validation fails: - - Return HTTP 403 Forbidden - - Include error message - ↓ -Client receives success/failure -``` - -### Example 3: WebSocket Connection (Nostr) - -``` -Client: new WebSocket('ws://localhost:8080/') - ↓ -actix-web receives HTTP GET with Upgrade: websocket - ↓ -CORS middleware adds headers - ↓ -Router matches path: / - ↓ -Nostr Handler receives request - ↓ -Upgrade to WebSocket - ↓ -Client sends: ["EVENT", {...}] - ↓ -Nostr Handler processes EVENT - ↓ -If kind 30617 (announcement): - - Validate clone/relays tags - - Provision Git repository - - Store event - ↓ -Response: ["OK", event_id, true, ""] - ↓ -Client receives confirmation -``` - -### Example 4: NIP-11 Request - -``` -Client: fetch('http://localhost:8080/', { - headers: { 'Accept': 'application/nostr+json' } -}) - ↓ -actix-web receives HTTP GET with Accept header - ↓ -CORS middleware adds headers - ↓ -Router matches path: / - ↓ -Nostr Handler checks Accept header - ↓ -Returns NIP-11 JSON: -{ - "name": "ngit-grasp instance", - "description": "Rust GRASP implementation", - "supported_nips": [1, 11, 34], - "supported_grasps": ["GRASP-01"], - "repo_acceptance_criteria": "Must list this service in clone and relays tags", - "curation": "Basic spam prevention" -} - ↓ -Client receives relay information -``` - -### Example 5: CORS Preflight (OPTIONS) - -``` -Browser: OPTIONS http://localhost:8080/ -Headers: - - Origin: https://example.com - - Access-Control-Request-Method: POST - ↓ -actix-web receives OPTIONS request - ↓ -CORS middleware handles preflight - ↓ -Returns 204 No Content with headers: - - Access-Control-Allow-Origin: * - - Access-Control-Allow-Methods: GET, POST - - Access-Control-Allow-Headers: Content-Type - - Access-Control-Max-Age: 3600 - ↓ -Browser caches preflight response - ↓ -Browser proceeds with actual request -``` - ---- - -## Component Responsibilities - -### actix-web HTTP Server -- Listen on single port -- Route requests by path -- Handle WebSocket upgrades -- Apply CORS to all requests - -### CORS Middleware -- Add headers to ALL responses -- Handle OPTIONS preflight -- Allow any origin (GRASP-01 requirement) - -### HTTP Router -- Match `/npub.../repo.git` → Git Handler -- Match `/` → Nostr Handler -- Pass through to appropriate handler - -### Git Handler -- Serve Git Smart HTTP protocol -- Read from `{GIT_DATA_PATH}/{npub}/{id}.git` -- Validate pushes before accepting -- Return 404 for missing repos - -### Nostr Handler -- Upgrade HTTP to WebSocket -- Process NIP-01 messages -- Store/query events -- Serve NIP-11 for HTTP requests -- Provision repos from announcements - -### Storage Layer -- Git repositories (bare) -- Nostr events (database) -- Separate paths for each - ---- - -## Configuration Flow - -``` -Environment Variables - ↓ -.env file (optional) - ↓ -Config::from_env() - ↓ -Config struct: - - bind_address: "127.0.0.1:8080" - - domain: "example.com" - - git_data_path: "./data/repos" - - relay_data_path: "./data/relay" - - relay_name: "..." - - relay_description: "..." - - owner_npub: "..." - ↓ -Passed to: - - HTTP server (bind address) - - Git handler (git_data_path, domain) - - Nostr handler (relay_data_path, domain, NIP-11 info) - - Storage layer (both paths) -``` - ---- - -## Test Architecture - -``` -Integration Test - ↓ -TestRelay::start() - ↓ -Spawns ngit-grasp process: - - NGIT_BIND_ADDRESS=127.0.0.1:{random_port} - - NGIT_DOMAIN=127.0.0.1:{random_port} - - NGIT_GIT_DATA_PATH=./test-data/repos - - NGIT_RELAY_DATA_PATH=./test-data/relay - ↓ -Process starts: - - actix-web listens on random port - - Both Git and Nostr available - ↓ -Test runs: - - Uses grasp-audit library - - Connects to ws://127.0.0.1:{port}/ - - Runs compliance tests - ↓ -TestRelay::stop() - ↓ -Process killed - ↓ -Test data cleaned up -``` - ---- - -## File Structure - -``` -ngit-grasp/ -├── src/ -│ ├── main.rs # Entry point -│ ├── config.rs # Configuration -│ ├── http/ # NEW - HTTP server -│ │ ├── mod.rs # Server setup -│ │ ├── git.rs # Git HTTP handler -│ │ └── nostr.rs # Nostr WebSocket handler -│ ├── nostr/ -│ │ ├── mod.rs -│ │ ├── relay.rs # Relay logic (reused) -│ │ └── events.rs # Event handling -│ └── storage/ -│ ├── mod.rs -│ └── repository.rs # Git repo management -├── tests/ -│ ├── common/ -│ │ ├── mod.rs -│ │ └── relay.rs # TestRelay fixture -│ ├── nip01_compliance.rs # NIP-01 tests -│ ├── nip34_announcements.rs # NIP-34 tests -│ └── grasp01_git_http.rs # NEW - GRASP-01 Git tests -├── data/ # Runtime data (gitignored) -│ ├── repos/ # Git repositories -│ └── relay/ # Nostr events -└── test-data/ # Test data (gitignored) - ├── repos/ - └── relay/ -``` - ---- - -## Comparison: ngit-relay vs ngit-grasp - -### ngit-relay (Go + nginx) - -``` -nginx (Port 8081) - ├── Git HTTP → fcgiwrap → git-http-backend - │ ↓ - │ pre-receive hook (Go) - │ ↓ - │ Khatru relay (HTTP API) - │ - └── Nostr → proxy → Khatru relay (Port 3334) - ↓ - on_event hook (Go) - ↓ - provision repos -``` - -**Components:** -- nginx (routing) -- fcgiwrap (CGI wrapper) -- git-http-backend (Git protocol) -- pre-receive hook (Go, validates pushes) -- post-receive hook (Go, updates HEAD) -- Khatru relay (Go, Nostr protocol) -- on_event hook (Go, provisions repos) -- supervisord (process management) - -### ngit-grasp (Rust) - -``` -actix-web (Port 8080) - ├── Git HTTP → git-http-backend crate - │ ↓ - │ inline authorization - │ ↓ - │ Storage (query state) - │ - └── Nostr → WebSocket upgrade - ↓ - nostr-sdk relay - ↓ - on_event (provision repos) - ↓ - Storage (store events) -``` - -**Components:** -- actix-web (routing + HTTP + WebSocket) -- git-http-backend crate (Git protocol) -- nostr-sdk (Nostr protocol) -- Storage (unified storage layer) - -**Advantages:** -- Single binary -- No external processes -- Inline authorization (better errors) -- Pure Rust (memory safety) -- Easier testing - ---- - -**Last Updated:** November 4, 2025 -**Purpose:** Reference for implementation diff --git a/docs/archive/2025-11-04-evening/current_status.md b/docs/archive/2025-11-04-evening/current_status.md deleted file mode 100644 index f14c391..0000000 --- a/docs/archive/2025-11-04-evening/current_status.md +++ /dev/null @@ -1,443 +0,0 @@ -# Current Status - ngit-grasp Implementation - -**Date:** November 4, 2025 -**Status:** In Development - GRASP-01 Core Requirements - ---- - -## 🎯 Project Goal - -Implement a **GRASP-01 compliant** Git relay service in Rust that: -- Serves a NIP-01 Nostr relay at `/` (WebSocket) -- Serves Git repositories via Git Smart HTTP at `//.git` -- **Both on the SAME PORT** (critical requirement!) -- Validates pushes against Nostr state events -- Passes all compliance tests from grasp-audit - ---- - -## 📋 GRASP-01 Requirements (from ../grasp/01.md) - -### 1. Nostr Relay Requirements - -**MUST:** -- ✅ Serve NIP-01 compliant relay at `/` (WebSocket) -- ✅ Accept NIP-34 repository announcements (kind 30617) -- ✅ Accept NIP-34 state announcements (kind 30618) -- ⏳ Reject announcements that don't list this service in `clone` and `relays` tags -- ⏳ Accept events that tag accepted announcements -- ✅ Serve NIP-11 relay information document -- ⏳ Include `supported_grasps`, `repo_acceptance_criteria`, `curation` in NIP-11 - -**Current Implementation:** -- Basic WebSocket relay working -- Event storage and querying functional -- NIP-11 basic implementation exists -- **Missing:** Announcement validation against service URL -- **Missing:** Event acceptance policy based on announcements - -### 2. Git Smart HTTP Service Requirements - -**MUST:** -- ❌ Serve Git repos at `//.git` via unauthenticated Git Smart HTTP -- ❌ Accept pushes matching latest state announcement (respecting maintainer set) -- ❌ Set repository HEAD per state announcement -- ❌ Accept pushes to `refs/nostr/` for PRs -- ❌ Include `allow-reachable-sha1-in-want` and `allow-tip-sha1-in-want` -- ❌ Serve webpage at repo endpoint for browsers - -**Current Implementation:** -- **NOT STARTED** - Git HTTP backend not integrated -- No Git repository management -- No push validation - -### 3. CORS Support Requirements - -**MUST:** -- ❌ Set `Access-Control-Allow-Origin: *` on ALL responses -- ❌ Set `Access-Control-Allow-Methods: GET, POST` on ALL responses -- ❌ Set `Access-Control-Allow-Headers: Content-Type` on ALL responses -- ❌ Respond to OPTIONS requests with 204 No Content - -**Current Implementation:** -- **NOT STARTED** - No CORS headers - ---- - -## 🏗️ Architecture Understanding (from ngit-relay) - -### Critical Architecture Insight: SINGLE PORT - -From `../ngit-relay/docker-compose.yml`: -```yaml -ports: - - "8081:8081" # Single port for EVERYTHING -``` - -From `../ngit-relay/src/nginx.conf`: -```nginx -server { - listen 8081; # Single listener - - # Git repos at //.git - location ~ ^/npub1([a-z0-9]+)/([^/]+\.git)(/.*)?$ { - # ... git-http-backend via fcgiwrap - } - - # Nostr relay at / - location / { - # ... proxy to khatru on localhost:3334 - } -} -``` - -**Key Points:** -1. **nginx listens on ONE port (8081)** -2. **nginx routes by URL path:** - - `//.git/*` → git-http-backend (fcgiwrap) - - Everything else → Khatru relay (localhost:3334) -3. **Khatru relay runs on INTERNAL port 3334** -4. **git-http-backend runs via fcgiwrap socket** - -### Our Rust Implementation Strategy - -We need to replicate nginx's routing in Rust: - -``` -HTTP/WebSocket Request on port 8080 - ↓ - actix-web router - ↓ - ┌────┴────┐ - ↓ ↓ -Git Path Other Path -// / -.git - ↓ ↓ -git-http Nostr Relay -backend (WebSocket upgrade) -handler -``` - -**Implementation Options:** - -**Option A: actix-web (HTTP framework)** -- Handle HTTP/WebSocket on same port -- Route by path pattern -- Use `git-http-backend` crate for Git protocol -- Native WebSocket support for Nostr relay - -**Option B: Direct TCP + Manual Routing** -- Accept TCP connections -- Parse HTTP headers to determine route -- More complex but more control - -**Recommendation: Option A (actix-web)** -- Well-tested HTTP/WebSocket handling -- Easy routing by path -- Good async performance -- Already in our dependencies - ---- - -## 🧪 Test Strategy - -### Current Test Structure - -``` -tests/ -├── common/ -│ ├── mod.rs # Test utilities -│ └── relay.rs # TestRelay fixture -├── nip01_compliance.rs # NIP-01 smoke tests -└── nip34_announcements.rs # NIP-34 tests (TODO) -``` - -### Test Approach - -**Integration Tests (tests/*):** -- Use `TestRelay` fixture to start/stop relay -- Use `grasp-audit` library to run compliance tests -- Tests reference GRASP protocol line numbers -- Automatic relay lifecycle management - -**Example Test Structure:** -```rust -#[tokio::test] -async fn test_grasp01_git_http_basic() { - // Reference: ../grasp/01.md lines 15-17 - // Requirement: MUST serve git repository via unauthenticated git smart http - - let relay = TestRelay::start().await; - let config = AuditConfig::ci(); - let client = AuditClient::new(relay.url(), config).await.unwrap(); - - // Run GRASP-01 git HTTP tests - let results = specs::Grasp01GitHttp::run_all(&client).await; - - relay.stop().await; - assert!(results.all_passed()); -} -``` - -### Test Coverage Needed - -**NIP-01 (Nostr Relay):** -- ✅ WebSocket connection -- ✅ Send/receive events -- ✅ Subscriptions (REQ/CLOSE) -- ✅ Event validation (signatures, IDs) -- ⏳ NIP-11 relay info document - -**NIP-34 (Git Announcements):** -- ⏳ Accept valid repository announcements (kind 30617) -- ⏳ Accept valid state announcements (kind 30618) -- ⏳ Reject announcements without service in clone/relays -- ⏳ Validate maintainer sets -- ⏳ Handle related events (issues, patches) - -**GRASP-01 (Git HTTP):** -- ❌ Serve Git repo at `//.git` -- ❌ Clone repository via HTTP -- ❌ Push matching state announcement -- ❌ Reject push not matching state -- ❌ Handle `refs/nostr/` for PRs -- ❌ CORS headers on all responses -- ❌ OPTIONS request handling - ---- - -## 📝 Implementation Plan - -### Phase 1: Fix Current Relay (In Progress) - -**Goal:** Make NIP-01 relay fully compliant - -**Tasks:** -- [x] Basic WebSocket relay working -- [x] Event storage and querying -- [ ] NIP-11 relay info with GRASP fields - - [ ] Add `supported_grasps: ["GRASP-01"]` - - [ ] Add `repo_acceptance_criteria` - - [ ] Add `curation` policy -- [ ] Announcement validation - - [ ] Check `clone` tag includes our domain - - [ ] Check `relays` tag includes our domain - - [ ] Reject if not listed (unless GRASP-05) -- [ ] Event acceptance policy - - [ ] Accept events tagging accepted announcements - - [ ] Accept events tagged by accepted announcements - -**Test Coverage:** -- [x] NIP-01 smoke tests passing -- [ ] NIP-11 compliance tests -- [ ] NIP-34 announcement tests - -### Phase 2: Add Git HTTP Backend (Next) - -**Goal:** Serve Git repositories via HTTP on same port as relay - -**Tasks:** -1. **Integrate actix-web** - - [ ] Replace raw WebSocket with actix-web - - [ ] Add HTTP routing - - [ ] Preserve WebSocket upgrade for `/` - - [ ] Add Git HTTP route for `//.git` - -2. **Integrate git-http-backend crate** - - [ ] Add dependency on `git-http-backend` - - [ ] Create Git handler for `//.git` - - [ ] Serve `git-upload-pack` (clone/fetch) - - [ ] Serve `git-receive-pack` (push) - -3. **Repository Management** - - [ ] Auto-provision repos from announcements - - [ ] Store repos at `{GIT_DATA_PATH}//.git` - - [ ] Initialize bare repositories - - [ ] Set HEAD from state announcements - -4. **CORS Support** - - [ ] Add CORS middleware to actix-web - - [ ] Set required headers on all responses - - [ ] Handle OPTIONS requests - -**Test Coverage:** -- [ ] Can clone repository via HTTP -- [ ] Can fetch from repository -- [ ] Repository provisioned from announcement -- [ ] HEAD set correctly from state -- [ ] CORS headers present -- [ ] OPTIONS requests handled - -### Phase 3: Push Authorization (Final) - -**Goal:** Validate pushes against Nostr state announcements - -**Tasks:** -1. **Inline Authorization** - - [ ] Intercept `git-receive-pack` before Git process - - [ ] Parse ref updates from request - - [ ] Query latest state announcement from relay - - [ ] Validate push matches state - - [ ] Handle maintainer sets (recursive) - - [ ] Return HTTP error if validation fails - -2. **PR Support** - - [ ] Accept pushes to `refs/nostr/` - - [ ] Validate PR event exists on relay - - [ ] Validate ref tip matches PR event `c` tag - - [ ] Implement 20-minute timeout for PR refs - - [ ] Garbage collect orphaned PR refs - -3. **State Synchronization** - - [ ] Update HEAD when state announcement received - - [ ] Handle state updates for existing repos - - [ ] Handle multi-maintainer scenarios - -**Test Coverage:** -- [ ] Push matching state succeeds -- [ ] Push not matching state fails -- [ ] Multi-maintainer push validation -- [ ] PR ref push/validation -- [ ] PR ref garbage collection -- [ ] State update triggers HEAD change - ---- - -## 🐛 Known Issues - -### 1. Architecture Mismatch -**Issue:** Tests assume relay on one port, Git on another -**Fix:** Both must be on same port (like ngit-relay) -**Impact:** Need to refactor server architecture - -### 2. Missing Git Implementation -**Issue:** No Git HTTP backend integrated -**Fix:** Add actix-web + git-http-backend -**Impact:** Core GRASP-01 requirement not met - -### 3. No Announcement Validation -**Issue:** Relay accepts all announcements -**Fix:** Validate `clone` and `relays` tags -**Impact:** Not GRASP-01 compliant - -### 4. No CORS Support -**Issue:** No CORS headers on responses -**Fix:** Add CORS middleware -**Impact:** Web clients can't access relay - ---- - -## 🔧 Environment Configuration - -From `../ngit-relay/.env.example`, we need: - -```bash -# Service Configuration -NGIT_DOMAIN=example.com # Used for announcement validation -NGIT_BIND_ADDRESS=127.0.0.1:8080 # Single port for HTTP/WS/Git - -# Relay Information (NIP-11) -NGIT_RELAY_NAME="ngit-grasp instance" -NGIT_RELAY_DESCRIPTION="Rust GRASP implementation" -NGIT_OWNER_NPUB="npub1..." # Relay owner - -# Storage Paths -NGIT_GIT_DATA_PATH=/srv/ngit-grasp/repos # Git repositories -NGIT_RELAY_DATA_PATH=/srv/ngit-grasp/relay-db # Nostr events - -# Features (Future) -NGIT_PROACTIVE_SYNC_GIT=false # GRASP-02 -NGIT_PROACTIVE_SYNC_NOSTR=false # GRASP-02 - -# Logging -NGIT_LOG_LEVEL=INFO -``` - -**Current .env.example status:** -- ⏳ Needs update with all required fields -- ⏳ Add GRASP-specific configuration -- ⏳ Document which fields are used where - ---- - -## 📊 Progress Summary - -### Completed ✅ -- Basic Nostr relay (WebSocket) -- Event storage and querying -- NIP-01 smoke tests -- Test infrastructure (TestRelay fixture) -- Integration with grasp-audit library - -### In Progress ⏳ -- NIP-11 relay information -- NIP-34 announcement handling -- Event acceptance policies - -### Not Started ❌ -- Git HTTP backend -- Repository provisioning -- Push authorization -- CORS support -- actix-web integration - -### Compliance Status -- **NIP-01:** ~60% (basic relay works, missing some features) -- **NIP-34:** ~20% (can store events, no validation) -- **GRASP-01:** ~30% (relay works, Git HTTP not started) - ---- - -## 🎯 Next Session Priorities - -1. **Fix Architecture** (CRITICAL) - - Integrate actix-web for HTTP/WebSocket routing - - Single port for all services - - Preserve existing relay functionality - -2. **Add Git HTTP** (HIGH) - - Integrate `git-http-backend` crate - - Basic clone/fetch support - - Repository provisioning from announcements - -3. **Update Tests** (HIGH) - - Add GRASP-01 Git HTTP tests - - Reference protocol line numbers - - Verify single-port architecture - -4. **Fix NIP-11** (MEDIUM) - - Add GRASP-specific fields - - Document compliance level - - Include in tests - ---- - -## 📚 Key References - -**GRASP Protocol:** -- `../grasp/README.md` - Overview -- `../grasp/01.md` - GRASP-01 Core Requirements (THE SPEC) -- `../grasp/02.md` - GRASP-02 Proactive Sync -- `../grasp/05.md` - GRASP-05 Archive - -**Reference Implementation:** -- `../ngit-relay/README.md` - Architecture overview -- `../ngit-relay/src/nginx.conf` - **CRITICAL: Shows single-port routing** -- `../ngit-relay/docker-compose.yml` - **CRITICAL: Shows port config** -- `../ngit-relay/.env.example` - Configuration template - -**Nostr Specs:** -- [NIP-01](https://nips.nostr.com/1) - Basic protocol -- [NIP-11](https://nips.nostr.com/11) - Relay information -- [NIP-34](https://nips.nostr.com/34) - Git stuff - -**Our Code:** -- `tests/nip01_compliance.rs` - Current test approach -- `tests/common/relay.rs` - TestRelay fixture -- `grasp-audit/src/specs/nip01_smoke.rs` - Test specs - ---- - -**Last Updated:** November 4, 2025 -**Next Review:** After actix-web integration diff --git a/docs/archive/2025-11-04-evening/implementation-checklist.md b/docs/archive/2025-11-04-evening/implementation-checklist.md deleted file mode 100644 index 64f5f4e..0000000 --- a/docs/archive/2025-11-04-evening/implementation-checklist.md +++ /dev/null @@ -1,720 +0,0 @@ -# Implementation Checklist - -**Date:** November 4, 2025 -**Purpose:** Step-by-step checklist for actix-web integration - ---- - -## ✅ Pre-Implementation (DONE) - -- [x] Review GRASP-01 specification -- [x] Review ngit-relay reference implementation -- [x] Understand single-port architecture -- [x] Document architecture in work/architecture-diagram.md -- [x] Create detailed plan in work/NEXT_SESSION_START_HERE.md -- [x] Update work/current_status.md - ---- - -## 📦 Phase 1: Dependencies & Setup - -### 1.1 Update Cargo.toml - -- [ ] Add `actix-web = "4"` -- [ ] Add `actix-cors = "0.7"` -- [ ] Add `actix-ws = "0.3"` (or use actix-web-actors) -- [ ] Add `git-http-backend = "0.2"` (check latest version) -- [ ] Run `cargo check` to verify dependencies - -**Verification:** -```bash -cargo tree | grep actix -cargo tree | grep git-http-backend -``` - -### 1.2 Update .env.example (if needed) - -- [x] Already has all required fields -- [x] NGIT_DOMAIN -- [x] NGIT_BIND_ADDRESS -- [x] NGIT_GIT_DATA_PATH -- [x] NGIT_RELAY_DATA_PATH - -**Verification:** -```bash -cat .env.example -``` - ---- - -## 🏗️ Phase 2: HTTP Server Module - -### 2.1 Create src/http/mod.rs - -- [ ] Create module structure -- [ ] Add `pub mod git;` -- [ ] Add `pub mod nostr;` -- [ ] Create `run_server()` function -- [ ] Set up actix-web HttpServer -- [ ] Add CORS middleware -- [ ] Add routing for Git and Nostr - -**Verification:** -```bash -cargo check -# Should compile without errors -``` - -**Test:** -```rust -// In src/http/mod.rs -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn test_module_exists() { - // Just verify module structure - assert!(true); - } -} -``` - -### 2.2 Create src/http/git.rs - -- [ ] Create `handle_git_request()` function -- [ ] Parse npub and repo from path -- [ ] Construct repository path -- [ ] Check if repository exists (return 404 if not) -- [ ] Use git-http-backend crate -- [ ] Handle GET (clone/fetch) -- [ ] Handle POST (push) -- [ ] Return proper HTTP responses - -**Verification:** -```bash -cargo check -# Should compile -``` - -**Test:** -```rust -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn test_parse_repo_path() { - // Test path parsing logic - let path = "/npub1abc.../my-repo.git"; - // ... verify parsing - } -} -``` - -### 2.3 Create src/http/nostr.rs - -- [ ] Create `handle_websocket()` function -- [ ] Handle WebSocket upgrade -- [ ] Reuse existing Nostr message handling -- [ ] Create `handle_http_root()` function -- [ ] Serve HTML for browsers -- [ ] Serve NIP-11 JSON for Accept: application/nostr+json - -**Verification:** -```bash -cargo check -``` - -**Test:** -```rust -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn test_nip11_response() { - // Test NIP-11 JSON generation - // ... - } -} -``` - ---- - -## 🔧 Phase 3: Update Existing Code - -### 3.1 Update src/config.rs - -- [x] Already has `git_data_path` field -- [x] Already has `from_env()` implementation -- [ ] Verify all fields are present -- [ ] Add any missing validation - -**Verification:** -```bash -cargo check -``` - -### 3.2 Update src/main.rs - -- [ ] Remove direct relay start -- [ ] Import `http` module -- [ ] Call `http::run_server(config, storage).await` -- [ ] Update logging messages - -**Verification:** -```bash -cargo build -# Should build successfully -``` - -**Test:** -```bash -# Run server -NGIT_DOMAIN=localhost:8080 \ -NGIT_BIND_ADDRESS=127.0.0.1:8080 \ -cargo run - -# In another terminal, check it's listening -curl -v http://localhost:8080/ -``` - -### 3.3 Move Relay Logic to Library - -- [ ] Extract relay logic from src/nostr/relay.rs -- [ ] Make it reusable by WebSocket handler -- [ ] Keep message handling separate from transport -- [ ] Create `handle_nostr_message()` function - -**Structure:** -```rust -// src/nostr/relay.rs -pub async fn handle_nostr_message( - message: &str, - storage: &Storage, -) -> Result> { - // Parse message - // Handle EVENT, REQ, CLOSE - // Return response messages -} -``` - -**Verification:** -```bash -cargo check -cargo test --lib -``` - ---- - -## 🧪 Phase 4: Update Tests - -### 4.1 Update tests/common/relay.rs - -- [ ] Verify NGIT_DOMAIN is set correctly -- [ ] Add NGIT_GIT_DATA_PATH env var -- [ ] Add NGIT_RELAY_DATA_PATH env var -- [ ] Use test-specific directories -- [ ] Clean up test data after tests - -**Current Status:** Already sets NGIT_DOMAIN correctly! - -**Add:** -```rust -.env("NGIT_GIT_DATA_PATH", "./test-data/repos") -.env("NGIT_RELAY_DATA_PATH", "./test-data/relay") -``` - -**Verification:** -```bash -cargo test --test nip01_compliance -# Should still pass -``` - -### 4.2 Create tests/grasp01_git_http.rs - -- [ ] Create new test file -- [ ] Add basic Git clone test -- [ ] Add CORS headers test -- [ ] Add OPTIONS request test -- [ ] Add repository not found test -- [ ] Reference GRASP-01 line numbers in comments - -**Template:** -```rust -//! GRASP-01 Git HTTP Integration Tests -//! -//! Reference: ../grasp/01.md lines 15-40 - -mod common; - -use common::TestRelay; - -#[tokio::test] -async fn test_git_http_basic() { - // Reference: ../grasp/01.md line 15 - // MUST serve git repository via unauthenticated git smart http - - let relay = TestRelay::start().await; - - // TODO: Create test repo - // TODO: Try to clone it - - relay.stop().await; -} -``` - -**Verification:** -```bash -cargo test --test grasp01_git_http -``` - -### 4.3 Update tests/nip01_compliance.rs - -- [ ] Verify tests still pass with new architecture -- [ ] Update any broken tests -- [ ] Add comments referencing GRASP-01 where relevant - -**Verification:** -```bash -cargo test --test nip01_compliance -``` - ---- - -## 🔍 Phase 5: Integration & Testing - -### 5.1 Manual Testing - -**Test 1: Server Starts** -```bash -cargo build -NGIT_DOMAIN=localhost:8080 \ -NGIT_BIND_ADDRESS=127.0.0.1:8080 \ -cargo run -``` - -**Expected:** Server starts without errors - ---- - -**Test 2: WebSocket Connection** -```bash -# In grasp-audit directory -cargo run -- --url ws://localhost:8080 -``` - -**Expected:** NIP-01 smoke tests pass - ---- - -**Test 3: HTTP Root** -```bash -curl -v http://localhost:8080/ -``` - -**Expected:** -- Status: 200 OK -- Content-Type: text/html -- CORS headers present -- HTML content - ---- - -**Test 4: NIP-11** -```bash -curl -v http://localhost:8080/ \ - -H "Accept: application/nostr+json" -``` - -**Expected:** -- Status: 200 OK -- Content-Type: application/json -- CORS headers present -- JSON with `supported_grasps` field - ---- - -**Test 5: Git Repository (404)** -```bash -curl -v http://localhost:8080/npub1test/test-repo.git/info/refs?service=git-upload-pack -``` - -**Expected:** -- Status: 404 Not Found -- CORS headers present - ---- - -**Test 6: Git Repository (Success)** -```bash -# Create test repo -mkdir -p ./data/repos/npub1test -cd ./data/repos/npub1test -git init --bare test-repo.git - -# Try to access it -curl -v http://localhost:8080/npub1test/test-repo.git/info/refs?service=git-upload-pack -``` - -**Expected:** -- Status: 200 OK -- Content-Type: application/x-git-upload-pack-advertisement -- CORS headers present -- Git protocol data - ---- - -**Test 7: Git Clone** -```bash -git clone http://localhost:8080/npub1test/test-repo.git /tmp/test-clone -``` - -**Expected:** -- Clone succeeds (even if empty repo) -- No errors - ---- - -**Test 8: CORS Preflight** -```bash -curl -v -X OPTIONS http://localhost:8080/ \ - -H "Origin: https://example.com" \ - -H "Access-Control-Request-Method: POST" -``` - -**Expected:** -- Status: 204 No Content -- Access-Control-Allow-Origin: * -- Access-Control-Allow-Methods: GET, POST -- Access-Control-Allow-Headers: Content-Type -- Access-Control-Max-Age: 3600 - ---- - -### 5.2 Automated Testing - -**Run All Tests:** -```bash -# Build first -cargo build - -# Run all tests -cargo test - -# Run specific test suites -cargo test --test nip01_compliance -cargo test --test grasp01_git_http - -# With output -cargo test -- --nocapture -``` - -**Expected:** All tests pass - ---- - -### 5.3 Performance Testing - -**Test Concurrent Connections:** -```bash -# Start server -cargo run & - -# Run multiple clients -for i in {1..10}; do - (cd grasp-audit && cargo run -- --url ws://localhost:8080) & -done - -# Wait for all to complete -wait -``` - -**Expected:** All clients connect and pass tests - ---- - -### 5.4 Error Handling Testing - -**Test 1: Invalid Repository Path** -```bash -curl -v http://localhost:8080/invalid/path -``` - -**Expected:** 404 or appropriate error - ---- - -**Test 2: Invalid WebSocket Message** -```bash -# Use websocat or similar -echo "invalid json" | websocat ws://localhost:8080/ -``` - -**Expected:** NOTICE message with error - ---- - -**Test 3: Large Git Push** -```bash -# Create repo with large files -# Try to push -# Verify it works or fails gracefully -``` - ---- - -## 📋 Acceptance Criteria - -### Must Have (MVP) - -- [ ] Server starts on single port -- [ ] WebSocket connects at `/` -- [ ] NIP-01 smoke tests pass -- [ ] Can access Git repo at `//.git` -- [ ] Returns 404 for missing repos -- [ ] CORS headers on all responses -- [ ] OPTIONS requests return 204 -- [ ] Can clone existing Git repository -- [ ] All integration tests pass - -### Should Have (Before Production) - -- [ ] Can push to repository (basic, no auth yet) -- [ ] Repository provisioned from announcement -- [ ] NIP-11 includes GRASP fields -- [ ] Proper error messages -- [ ] Logging works correctly -- [ ] Clean shutdown -- [ ] Test data cleanup - -### Could Have (Future) - -- [ ] Push authorization -- [ ] Maintainer set validation -- [ ] PR ref support -- [ ] State synchronization -- [ ] Proactive sync (GRASP-02) - ---- - -## 🐛 Known Issues to Watch For - -### Issue 1: WebSocket Upgrade Timing - -**Symptom:** WebSocket upgrade fails intermittently - -**Debug:** -```bash -RUST_LOG=debug cargo run -# Check for upgrade-related logs -``` - -**Solution:** Ensure actix-ws is configured correctly - ---- - -### Issue 2: Git HTTP Protocol Errors - -**Symptom:** Git clone fails with protocol error - -**Debug:** -```bash -GIT_TRACE_PACKET=1 git clone http://localhost:8080/... -# Shows Git protocol messages -``` - -**Solution:** Check git-http-backend configuration - ---- - -### Issue 3: CORS Not Applied - -**Symptom:** Browser shows CORS error - -**Debug:** -```bash -curl -v http://localhost:8080/ -H "Origin: https://example.com" -# Check response headers -``` - -**Solution:** Verify CORS middleware is first in chain - ---- - -### Issue 4: Port Already in Use - -**Symptom:** "Address already in use" error - -**Debug:** -```bash -lsof -i :8080 -# Find process using port -``` - -**Solution:** -```bash -kill -9 -# Or use different port -``` - ---- - -### Issue 5: Test Relay Won't Start - -**Symptom:** Integration tests fail to start relay - -**Debug:** -```bash -# Run test with output -cargo test --test nip01_compliance -- --nocapture - -# Check binary exists -ls -la target/debug/ngit-grasp -``` - -**Solution:** Run `cargo build` before tests - ---- - -## 📚 Reference Commands - -### Development - -```bash -# Build -cargo build - -# Run -cargo run - -# Run with logging -RUST_LOG=debug cargo run - -# Check without building -cargo check - -# Format code -cargo fmt - -# Lint -cargo clippy -``` - -### Testing - -```bash -# All tests -cargo test - -# Specific test file -cargo test --test nip01_compliance - -# Specific test -cargo test --test nip01_compliance test_nip01_smoke - -# With output -cargo test -- --nocapture - -# With logging -RUST_LOG=debug cargo test -- --nocapture -``` - -### Debugging - -```bash -# Check dependencies -cargo tree - -# Check for unused dependencies -cargo +nightly udeps - -# Check for outdated dependencies -cargo outdated - -# Audit for security issues -cargo audit -``` - -### Git Testing - -```bash -# Create test repo -mkdir -p ./data/repos/npub1test -cd ./data/repos/npub1test -git init --bare test-repo.git - -# Clone it -git clone http://localhost:8080/npub1test/test-repo.git /tmp/test - -# Push to it -cd /tmp/test -echo "test" > README.md -git add . -git commit -m "test" -git push origin main -``` - ---- - -## ✅ Completion Checklist - -When all items are checked, Phase 1 (actix-web integration) is complete: - -### Code - -- [ ] Dependencies added to Cargo.toml -- [ ] src/http/mod.rs created -- [ ] src/http/git.rs created -- [ ] src/http/nostr.rs created -- [ ] src/main.rs updated -- [ ] src/config.rs verified -- [ ] Relay logic refactored - -### Tests - -- [ ] tests/common/relay.rs updated -- [ ] tests/grasp01_git_http.rs created -- [ ] tests/nip01_compliance.rs still passes -- [ ] All tests pass - -### Manual Testing - -- [ ] Server starts successfully -- [ ] WebSocket connects -- [ ] NIP-01 smoke tests pass -- [ ] Can access Git repos -- [ ] 404 for missing repos -- [ ] CORS headers present -- [ ] OPTIONS requests work -- [ ] Can clone repository - -### Documentation - -- [ ] Update README.md status -- [ ] Update work/current_status.md -- [ ] Document any issues found -- [ ] Update NEXT_SESSION_START_HERE.md for next phase - ---- - -## 🎯 Next Phase Preview - -After actix-web integration is complete, next phase will be: - -**Phase 2: Repository Provisioning** - -- Listen for NIP-34 repository announcements -- Create Git repositories automatically -- Initialize bare repositories -- Set up directory structure -- Handle repository deletion - -**Estimated Time:** 2-3 hours -**Prerequisites:** Phase 1 complete - ---- - -**Last Updated:** November 4, 2025 -**Status:** Ready to begin Phase 1 diff --git a/docs/archive/2025-11-04-evening/review-summary.md b/docs/archive/2025-11-04-evening/review-summary.md deleted file mode 100644 index 29a3ff4..0000000 --- a/docs/archive/2025-11-04-evening/review-summary.md +++ /dev/null @@ -1,513 +0,0 @@ -# GRASP Protocol Review Summary - -**Date:** November 4, 2025 -**Purpose:** Document key findings from reviewing GRASP protocol and ngit-relay - ---- - -## 🎯 Critical Discoveries - -### 1. Single Port Architecture (CRITICAL!) - -**Finding:** Git server and Nostr relay MUST run on the SAME port. - -**Evidence:** -```yaml -# ../ngit-relay/docker-compose.yml -ports: - - "8081:8081" # Single port only! -``` - -```nginx -# ../ngit-relay/src/nginx.conf -server { - listen 8081; # One listener for everything - - location ~ ^/npub1([a-z0-9]+)/([^/]+\.git)(/.*)?$ { - # Git HTTP via fcgiwrap - } - - location / { - # Nostr relay via proxy to localhost:3334 - } -} -``` - -**Impact:** -- Our current architecture is WRONG -- We assumed separate ports (relay on 8080, git on 8081) -- Must use HTTP router to split traffic by path -- actix-web can handle this - -**Action Required:** -- Integrate actix-web for HTTP routing -- Route `//.git` to Git handler -- Route `/` to Nostr relay (WebSocket upgrade) -- Apply CORS to ALL routes - ---- - -### 2. GRASP-01 Test Requirements - -**Finding:** Tests must closely map to GRASP protocol specification. - -**GRASP-01 Requirements (from ../grasp/01.md):** - -#### Nostr Relay (Lines 1-14) -- ✅ Serve NIP-01 relay at `/` (WebSocket) -- ⏳ Accept NIP-34 repository announcements (kind 30617) -- ⏳ Accept NIP-34 state announcements (kind 30618) -- ⏳ Reject announcements without service in `clone` and `relays` tags -- ⏳ Accept events that tag accepted announcements -- ✅ Serve NIP-11 relay information -- ⏳ Include `supported_grasps`, `repo_acceptance_criteria`, `curation` in NIP-11 - -#### Git Smart HTTP (Lines 15-31) -- ❌ Serve repos at `//.git` -- ❌ Accept pushes matching state announcements -- ❌ Respect recursive maintainer sets -- ❌ Set HEAD per state announcement -- ❌ Accept pushes to `refs/nostr/` for PRs -- ❌ Include `allow-reachable-sha1-in-want` and `allow-tip-sha1-in-want` -- ❌ Serve webpage for browsers - -#### CORS Support (Lines 32-40) -- ❌ `Access-Control-Allow-Origin: *` on ALL responses -- ❌ `Access-Control-Allow-Methods: GET, POST` on ALL responses -- ❌ `Access-Control-Allow-Headers: Content-Type` on ALL responses -- ❌ Respond to OPTIONS with 204 No Content - -**Action Required:** -- Create test for each requirement -- Reference GRASP-01 line numbers in test comments -- Example: - ```rust - #[tokio::test] - async fn test_git_http_basic() { - // Reference: ../grasp/01.md line 15 - // MUST serve git repository via unauthenticated git smart http - // ... - } - ``` - ---- - -### 3. Environment Variables - -**Finding:** ngit-relay uses specific environment variables we should match. - -**From ../ngit-relay/.env.example:** - -```bash -# Service Configuration -NGIT_DOMAIN=example.com # For announcement validation -NGIT_INTERNAL_RELAY_PORT_FOR_SSL_PROXY=8081 # We don't need this - -# Relay Information (NIP-11) -NGIT_RELAY_NAME="..." -NGIT_RELAY_DESCRIPTION="..." -NGIT_OWNER_NPUB="..." - -# Features -NGIT_PROACTIVE_SYNC_GIT=true # GRASP-02 (future) -NGIT_PROACTIVE_SYNC_BLOSSOM=true # Not in GRASP -NGIT_PROACTIVE_SYNC_NOSTR=true # GRASP-02 (future) - -# Blossom Settings -NGIT_BLOSSOM_MAX_FILE_SIZE_MB=100 # Not in GRASP -NGIT_BLOSSOM_MAX_CAPACITY_GB=50 # Not in GRASP - -# Logging -NGIT_LOG_DIR=/var/log/ngit-relay -NGIT_LOG_LEVEL=INFO -NGIT_LOG_MAX_SIZE_MB=20 -NGIT_LOG_MAX_BACKUPS=10 -NGIT_LOG_MAX_AGE_DAYS=30 -``` - -**Our Environment Variables:** - -```bash -# Service Configuration -NGIT_DOMAIN=example.com # REQUIRED - for announcement validation -NGIT_BIND_ADDRESS=127.0.0.1:8080 # REQUIRED - single port - -# Relay Information (NIP-11) -NGIT_RELAY_NAME="ngit-grasp instance" -NGIT_RELAY_DESCRIPTION="Rust GRASP implementation" -NGIT_OWNER_NPUB="npub1..." - -# Storage Paths -NGIT_GIT_DATA_PATH=./data/repos # REQUIRED - where to store Git repos -NGIT_RELAY_DATA_PATH=./data/relay # REQUIRED - where to store events - -# Logging -NGIT_LOG_LEVEL=INFO -RUST_LOG=info # Standard Rust logging -``` - -**Action Required:** -- Update `.env.example` with all required fields -- Add `NGIT_GIT_DATA_PATH` to config -- Document which fields are required vs. optional - ---- - -### 4. Repository Path Structure - -**Finding:** Repository storage follows specific pattern. - -**Pattern:** `{GIT_DATA_PATH}/{npub}/{identifier}.git` - -**Example:** -``` -./data/repos/ -├── npub1abc.../ -│ ├── my-project.git/ -│ │ ├── HEAD -│ │ ├── config -│ │ ├── objects/ -│ │ └── refs/ -│ └── another-repo.git/ -└── npub1xyz.../ - └── their-project.git/ -``` - -**Action Required:** -- Create repository directory structure -- Initialize bare repositories (`git init --bare`) -- Set ownership/permissions correctly -- Clean up on repository deletion - ---- - -### 5. NIP-11 GRASP Fields - -**Finding:** NIP-11 relay information must include GRASP-specific fields. - -**From ../grasp/01.md lines 11-14:** - -```json -{ - "name": "ngit-grasp instance", - "description": "Rust GRASP implementation", - "pubkey": "...", - "contact": "...", - "supported_nips": [1, 11, 34], - "supported_grasps": ["GRASP-01"], // NEW - array of strings - "repo_acceptance_criteria": "...", // NEW - human readable - "curation": "WoT-based spam prevention" // NEW - optional -} -``` - -**Action Required:** -- Add `supported_grasps` field to NIP-11 response -- Add `repo_acceptance_criteria` field -- Add `curation` field (optional) -- Update NIP-11 tests to verify these fields - ---- - -### 6. Announcement Validation - -**Finding:** Relay must validate announcements list this service. - -**From ../grasp/01.md lines 3-5:** - -> MUST reject [git repository announcements] that do not list the service -> in both `clone` and `relays` tags unless implementing `GRASP-05`. - -**NIP-34 Repository Announcement (kind 30617):** -```json -{ - "kind": 30617, - "tags": [ - ["d", "my-project"], // identifier - ["name", "My Project"], - ["clone", "https://example.com/npub.../my-project.git"], - ["clone", "https://github.com/user/my-project"], - ["relays", "wss://example.com"], - ["relays", "wss://relay.nostr.band"] - ] -} -``` - -**Validation Logic:** -```rust -fn validate_announcement(event: &Event, our_domain: &str) -> Result<()> { - // Check for clone tag with our domain - let has_clone = event.tags.iter().any(|tag| { - tag.kind() == TagKind::Custom("clone".into()) && - tag.content().map(|c| c.contains(our_domain)).unwrap_or(false) - }); - - // Check for relays tag with our domain - let has_relay = event.tags.iter().any(|tag| { - tag.kind() == TagKind::Custom("relays".into()) && - tag.content().map(|c| c.contains(our_domain)).unwrap_or(false) - }); - - if !has_clone || !has_relay { - return Err(anyhow!("Announcement must list this service in both clone and relays tags")); - } - - Ok(()) -} -``` - -**Action Required:** -- Implement announcement validation -- Check both `clone` and `relays` tags -- Reject if service not listed -- Add tests for validation - ---- - -### 7. State Announcement Handling - -**Finding:** State announcements control repository state. - -**NIP-34 Repository State (kind 30618):** -```json -{ - "kind": 30618, - "tags": [ - ["d", "my-project"], // identifier - ["refs/heads/main", "abc123..."], // branch → commit - ["refs/heads/develop", "def456..."], - ["HEAD", "ref: refs/heads/main"], // symbolic ref - ["maintainers", "npub1...", "npub2..."] // maintainer set - ] -} -``` - -**State Handling:** -1. When state announcement received: - - Update repository HEAD if needed - - Store state for push validation - - Handle maintainer set - -2. When push received: - - Query latest state announcement - - Validate pusher is in maintainer set (recursive) - - Validate ref updates match state - - Accept or reject push - -**Action Required:** -- Parse state announcements -- Update repository HEAD -- Implement push validation -- Handle recursive maintainer sets - ---- - -### 8. PR Ref Handling - -**Finding:** Special handling for PR refs. - -**From ../grasp/01.md lines 22-23:** - -> MUST accept pushes via this service to `refs/nostr/` but SHOULD -> reject if event exists on relay listing a different tip and MAY reject based -> on criteria such as size, SPAM prevention, etc. SHOULD delete and MAY garbage -> collect these refs if no corresponding [git PR event] or [git PR update event], -> with a `c` tag that matches the ref tip, is accepted by relay with 20 minutes. - -**PR Ref Lifecycle:** -1. Push to `refs/nostr/` -2. Verify PR event exists on relay -3. Verify ref tip matches PR event `c` tag -4. Accept push -5. After 20 minutes, check if PR event still exists -6. If not, delete ref and garbage collect - -**Action Required:** -- Accept pushes to `refs/nostr/` -- Validate against PR events -- Implement 20-minute timeout -- Implement garbage collection - ---- - -### 9. Git HTTP Protocol Details - -**Finding:** Must support specific Git protocol features. - -**From ../grasp/01.md lines 25-26:** - -> MUST include `allow-reachable-sha1-in-want` and `allow-tip-sha1-in-want` -> in advertisement and serve available oids. - -**Git Capabilities:** -``` -# info/refs response must include: -allow-reachable-sha1-in-want -allow-tip-sha1-in-want -``` - -**Action Required:** -- Configure git-http-backend to advertise these capabilities -- Ensure Git process is configured correctly -- Test with actual Git client - ---- - -### 10. CORS Requirements - -**Finding:** CORS must be on ALL responses, not just some. - -**From ../grasp/01.md lines 32-40:** - -``` -1. Set `Access-Control-Allow-Origin: *` on ALL responses -2. Set `Access-Control-Allow-Methods: GET, POST` on ALL responses -3. Set `Access-Control-Allow-Headers: Content-Type` on ALL responses -4. Respond to OPTIONS requests with 204 No Content -``` - -**Implementation:** -```rust -// In actix-web -App::new() - .wrap( - Cors::default() - .allow_any_origin() - .allowed_methods(vec!["GET", "POST"]) - .allowed_headers(vec!["Content-Type"]) - .max_age(3600) - ) - // ... routes -``` - -**Action Required:** -- Add CORS middleware to actix-web -- Verify headers on all responses -- Handle OPTIONS requests -- Test with browser - ---- - -## 📊 Compliance Status - -### NIP-01 (Nostr Relay) -- ✅ WebSocket connection -- ✅ EVENT message handling -- ✅ REQ subscription -- ✅ CLOSE subscription -- ✅ Event validation -- ⏳ NIP-11 with GRASP fields - -**Status:** ~80% complete - -### NIP-34 (Git Announcements) -- ✅ Store announcements (kind 30617) -- ✅ Store state events (kind 30618) -- ⏳ Validate announcements list this service -- ⏳ Handle maintainer sets -- ⏳ Accept related events - -**Status:** ~40% complete - -### GRASP-01 (Core Requirements) -- ✅ Nostr relay at `/` -- ❌ Git HTTP at `//.git` -- ❌ Push validation -- ❌ Repository provisioning -- ❌ CORS support - -**Status:** ~20% complete - ---- - -## 🎯 Immediate Next Steps - -### 1. Fix Architecture (CRITICAL) -- [ ] Add actix-web dependencies -- [ ] Create HTTP router module -- [ ] Route Git paths to Git handler -- [ ] Route `/` to Nostr relay (WebSocket) -- [ ] Apply CORS to all routes - -**Estimated Time:** 2-4 hours -**Priority:** CRITICAL -**Blocker:** Nothing else can proceed without this - -### 2. Add Git HTTP Backend -- [ ] Integrate git-http-backend crate -- [ ] Create Git request handler -- [ ] Serve from `{GIT_DATA_PATH}/{npub}/{id}.git` -- [ ] Return 404 for missing repos -- [ ] Test with `git clone` - -**Estimated Time:** 2-3 hours -**Priority:** HIGH -**Blocker:** Requires architecture fix - -### 3. Repository Provisioning -- [ ] Create repos when announcements received -- [ ] Initialize bare repositories -- [ ] Set up directory structure -- [ ] Handle repository deletion - -**Estimated Time:** 1-2 hours -**Priority:** HIGH -**Blocker:** Requires Git HTTP backend - -### 4. Update Tests -- [ ] Add GRASP-01 line number references -- [ ] Create Git HTTP tests -- [ ] Create CORS tests -- [ ] Update NIP-11 tests - -**Estimated Time:** 2-3 hours -**Priority:** MEDIUM -**Blocker:** None (can start now) - ---- - -## 📚 Key Files to Reference - -### GRASP Protocol -- `../grasp/01.md` - **THE SPEC** - Lines 1-40 -- `../grasp/README.md` - Overview -- `../grasp/02.md` - GRASP-02 (future) -- `../grasp/05.md` - GRASP-05 (future) - -### Reference Implementation -- `../ngit-relay/src/nginx.conf` - **ROUTING PATTERN** - Lines 8-94 -- `../ngit-relay/docker-compose.yml` - Port configuration -- `../ngit-relay/.env.example` - Environment variables -- `../ngit-relay/README.md` - Architecture overview - -### Our Code -- `tests/nip01_compliance.rs` - Current test approach -- `tests/common/relay.rs` - TestRelay fixture (already correct!) -- `src/nostr/relay.rs` - Current relay implementation -- `src/config.rs` - Configuration (needs Git path) - ---- - -## ✅ Checklist for Next Session - -Before starting implementation: -- [x] Read GRASP-01 specification (../grasp/01.md) -- [x] Review ngit-relay nginx.conf routing -- [x] Understand single-port architecture -- [x] Review environment variables -- [x] Understand repository path structure - -Ready to implement: -- [ ] Add actix-web dependencies to Cargo.toml -- [ ] Create src/http/mod.rs module -- [ ] Create src/http/git.rs handler -- [ ] Create src/http/nostr.rs handler -- [ ] Update src/main.rs -- [ ] Update src/config.rs -- [ ] Update .env.example -- [ ] Update tests/common/relay.rs -- [ ] Create tests/grasp01_git_http.rs - ---- - -**Last Updated:** November 4, 2025 -**Next Review:** After actix-web integration diff --git a/docs/archive/2025-11-04-evening/session-complete.md b/docs/archive/2025-11-04-evening/session-complete.md deleted file mode 100644 index 56cb711..0000000 --- a/docs/archive/2025-11-04-evening/session-complete.md +++ /dev/null @@ -1,121 +0,0 @@ -# Session Complete: Ready for Test Validation Phase - -**Date:** 2025-11-04 -**Status:** ✅ READY TO BEGIN - ---- - -## ✅ What We Did - -### 1. Strategic Planning -- Analyzed test-first vs TDD parallel approaches -- Decided to validate grasp-audit against ngit-relay first -- Validated hybrid architecture (git2 + git-http-backend + system git) - -### 2. Documentation -- Archived all planning documents to `docs/archive/2025-11-04-*` -- Created fresh `work/current_status.md` for test validation phase -- Documented strategic decision and rationale - -### 3. Preparation -- Identified ngit-relay location: `../ngit-relay/` -- Confirmed Docker image: `ghcr.io/danconwaydev/ngit-relay:latest` -- Outlined complete test validation plan - ---- - -## 📋 Current State - -``` -work/ -├── README.md ✅ (gitignored, explains work/) -└── current_status.md ✅ (test validation plan) - -docs/archive/ -├── 2025-11-04-session-summary.md ✅ (this session) -├── 2025-11-04-ngit-grasp-implementation-plan.md ✅ (for later) -├── 2025-11-04-git-http-backend-validation.md ✅ (architecture) -├── 2025-11-04-test-strategy-decision.md ✅ (rationale) -├── 2025-11-04-git-http-backend-deep-dive.md ✅ (crate analysis) -└── 2025-11-04-authorization-flow-diagram.txt ✅ (visual ref) -``` - ---- - -## 🎯 Next Session: Start Here - -### Quick Start Command -```bash -# 1. Read the plan -cat work/current_status.md - -# 2. Start ngit-relay -cd ../ngit-relay -docker-compose up -d - -# 3. Verify it's working -curl http://localhost:8080 # Nostr relay -curl http://localhost:3000 # Git server - -# 4. Begin building tests -cd ../ngit-grasp/grasp-audit -nix develop -# Create src/specs/grasp01_git.rs -``` - -### Timeline -- **Phase 1:** Setup ngit-relay (30 min) -- **Phase 2:** Build GRASP-01 Git tests (1 day) -- **Phase 3:** Validate against ngit-relay (1 day) -- **Phase 4:** Document findings (2 hours) -- **Total:** ~2 days - ---- - -## 📚 Key Documents - -### For This Phase (Test Validation) -- **Plan:** `work/current_status.md` ← START HERE -- **Rationale:** `docs/archive/2025-11-04-test-strategy-decision.md` -- **Reference:** `../ngit-relay/README.md` - -### For Later (Implementation) -- **Implementation Plan:** `docs/archive/2025-11-04-ngit-grasp-implementation-plan.md` -- **Architecture:** `docs/archive/2025-11-04-git-http-backend-validation.md` -- **Flow Diagram:** `docs/archive/2025-11-04-authorization-flow-diagram.txt` - ---- - -## 🚀 The Goal - -**By end of next session:** -- ✅ grasp-audit has complete GRASP-01 Git test suite -- ✅ All tests pass against ngit-relay reference implementation -- ✅ Reference behavior documented -- ✅ Confident test suite ready for ngit-grasp implementation - -**Then we can implement ngit-grasp knowing our tests are correct!** - ---- - -## 💡 Why This Approach? - -**Question:** Why not just start implementing ngit-grasp? - -**Answer:** -- Testing against reference validates our test suite first -- Eliminates "is it the test or the code?" debugging -- Only 1-2 day investment for weeks of confidence -- Same total timeline but much lower risk - -**See:** `docs/archive/2025-11-04-test-strategy-decision.md` for full analysis - ---- - -## ✅ Ready! - -**Status:** All planning complete, ready to begin test validation -**First Step:** `cd ../ngit-relay && docker-compose up -d` -**Reference:** `work/current_status.md` - -Let's build a rock-solid test suite! 🚀 diff --git a/docs/archive/2025-11-04-evening/session-summary.md b/docs/archive/2025-11-04-evening/session-summary.md deleted file mode 100644 index 398f4d0..0000000 --- a/docs/archive/2025-11-04-evening/session-summary.md +++ /dev/null @@ -1,487 +0,0 @@ -# Session Summary - GRASP Protocol Review - -**Date:** November 4, 2025 -**Duration:** ~2 hours -**Status:** ✅ Complete - Ready for implementation - ---- - -## 🎯 Session Goals - -1. ✅ Review GRASP protocol specification -2. ✅ Review ngit-relay reference implementation -3. ✅ Understand architecture requirements -4. ✅ Update work documents with accurate plan -5. ✅ Fix mistakes in previous understanding - ---- - -## 🔍 Key Discoveries - -### 1. Single Port Architecture (CRITICAL!) - -**Previous Understanding (WRONG):** -- Nostr relay on port 8080 -- Git server on port 8081 -- Separate services - -**Correct Understanding:** -- **BOTH services on SAME port** (e.g., 8080) -- HTTP router splits traffic by path: - - `//.git` → Git handler - - `/` → Nostr relay (WebSocket) -- This is a GRASP-01 requirement! - -**Evidence:** -- `../ngit-relay/docker-compose.yml` - Single port (8081) -- `../ngit-relay/src/nginx.conf` - nginx routes by path on one listener - -**Impact:** -- Complete architecture redesign needed -- Must use actix-web for HTTP routing -- All previous assumptions about ports were wrong - ---- - -### 2. Test Requirements Must Map to Protocol - -**Discovery:** Tests must reference GRASP protocol line numbers. - -**Example:** -```rust -#[tokio::test] -async fn test_git_http_basic() { - // Reference: ../grasp/01.md line 15 - // MUST serve git repository via unauthenticated git smart http service - // at //.git - - // Test implementation... -} -``` - -**Why:** -- Makes tests traceable to requirements -- Easy to verify compliance -- Documents what we're testing -- Helps reviewers understand intent - -**Action:** -- Update all test files with protocol references -- Create new tests for missing requirements -- Organize tests by GRASP-01 sections - ---- - -### 3. Environment Variables - -**Discovery:** ngit-relay uses specific env var naming we should match. - -**Critical Variables:** -- `NGIT_DOMAIN` - Used for announcement validation (REQUIRED) -- `NGIT_BIND_ADDRESS` - Single port for all services (REQUIRED) -- `NGIT_GIT_DATA_PATH` - Where to store Git repos (REQUIRED) -- `NGIT_RELAY_DATA_PATH` - Where to store events (REQUIRED) - -**Our .env.example:** -- ✅ Already has all required fields -- ✅ Follows ngit-relay naming convention -- ✅ No changes needed - ---- - -### 4. NIP-11 GRASP Fields - -**Discovery:** NIP-11 must include GRASP-specific fields. - -**Required Fields:** -```json -{ - "supported_grasps": ["GRASP-01"], - "repo_acceptance_criteria": "Must list service in clone and relays tags", - "curation": "Basic spam prevention" // optional -} -``` - -**Action:** -- Add fields to NIP-11 response -- Update tests to verify fields -- Document in code - ---- - -### 5. Repository Path Structure - -**Discovery:** Repos follow specific path pattern. - -**Pattern:** `{GIT_DATA_PATH}/{npub}/{identifier}.git` - -**Example:** -``` -./data/repos/ -├── npub1abc.../ -│ └── my-project.git/ -└── npub1xyz.../ - └── their-repo.git/ -``` - -**Action:** -- Create directory structure on repo provision -- Initialize bare repositories -- Handle cleanup on deletion - ---- - -### 6. CORS Requirements - -**Discovery:** CORS must be on ALL responses, not optional. - -**Requirements (GRASP-01 lines 32-40):** -1. `Access-Control-Allow-Origin: *` on ALL responses -2. `Access-Control-Allow-Methods: GET, POST` on ALL responses -3. `Access-Control-Allow-Headers: Content-Type` on ALL responses -4. Respond to OPTIONS with 204 No Content - -**Implementation:** -- Use actix-cors middleware -- Apply to all routes -- Test with browser - ---- - -### 7. Announcement Validation - -**Discovery:** Must validate announcements list this service. - -**Rule (GRASP-01 lines 3-5):** -> MUST reject announcements that do not list the service in both -> `clone` and `relays` tags unless implementing GRASP-05. - -**Validation Logic:** -```rust -fn validate_announcement(event: &Event, domain: &str) -> Result<()> { - let has_clone = event.tags.iter().any(|tag| - tag.is_clone() && tag.content().contains(domain) - ); - let has_relay = event.tags.iter().any(|tag| - tag.is_relay() && tag.content().contains(domain) - ); - - if !has_clone || !has_relay { - return Err("Must list service in clone and relays"); - } - Ok(()) -} -``` - -**Action:** -- Implement validation in event handler -- Reject invalid announcements -- Add tests - ---- - -## 📄 Documents Created - -### 1. work/current_status.md -**Purpose:** Comprehensive status of implementation - -**Contents:** -- GRASP-01 requirements checklist -- Architecture understanding -- Current implementation status -- Known issues -- Next priorities -- Key references - -**Use:** Reference for overall project status - ---- - -### 2. work/NEXT_SESSION_START_HERE.md -**Purpose:** Step-by-step implementation guide - -**Contents:** -- Immediate goal (actix-web integration) -- Critical architecture understanding -- 8-step implementation plan with code examples -- Verification steps -- Common issues & solutions -- Success criteria - -**Use:** Start here next session for implementation - ---- - -### 3. work/review-summary.md -**Purpose:** Document findings from GRASP/ngit-relay review - -**Contents:** -- 10 critical discoveries -- Evidence for each -- Action items -- Compliance status -- Next steps - -**Use:** Reference for why we're making changes - ---- - -### 4. work/architecture-diagram.md -**Purpose:** Visual reference for architecture - -**Contents:** -- Current vs. target architecture diagrams -- Request flow examples -- Component responsibilities -- File structure -- Comparison with ngit-relay - -**Use:** Visual reference during implementation - ---- - -### 5. work/implementation-checklist.md -**Purpose:** Detailed checklist for implementation - -**Contents:** -- 5 phases with detailed tasks -- Verification steps for each task -- Manual testing procedures -- Automated testing commands -- Acceptance criteria -- Known issues to watch for -- Reference commands - -**Use:** Track progress during implementation - ---- - -### 6. work/session-summary.md (this file) -**Purpose:** Summary of this review session - -**Contents:** -- What we accomplished -- Key discoveries -- Documents created -- Next steps - -**Use:** Remember what we did this session - ---- - -## 📊 Compliance Status - -### Before This Session -- **Understanding:** Incorrect (separate ports) -- **NIP-01:** ~60% (relay works) -- **NIP-34:** ~20% (basic storage) -- **GRASP-01:** ~10% (wrong architecture) - -### After This Session -- **Understanding:** ✅ Correct (single port, routing) -- **NIP-01:** ~60% (no change, but plan to improve) -- **NIP-34:** ~20% (no change, but plan ready) -- **GRASP-01:** ~20% (plan ready, architecture understood) - -### Target After Next Session -- **Understanding:** ✅ Complete -- **NIP-01:** ~80% (with actix-web) -- **NIP-34:** ~40% (with announcement validation) -- **GRASP-01:** ~60% (with Git HTTP working) - ---- - -## 🎯 Next Session Plan - -### Immediate Goal -**Integrate actix-web for single-port HTTP/WebSocket/Git routing** - -### Steps (from NEXT_SESSION_START_HERE.md) -1. Add dependencies (actix-web, actix-cors, actix-ws, git-http-backend) -2. Create src/http/mod.rs (HTTP server) -3. Create src/http/git.rs (Git handler) -4. Create src/http/nostr.rs (WebSocket handler) -5. Update src/main.rs -6. Update tests -7. Manual testing -8. Automated testing - -### Success Criteria -- ✅ Server starts on single port -- ✅ WebSocket connects at `/` -- ✅ NIP-01 smoke tests pass -- ✅ Can clone Git repo -- ✅ CORS headers present -- ✅ All tests pass - -### Estimated Time -2-4 hours for core implementation -1-2 hours for testing and debugging -**Total: 3-6 hours** - ---- - -## 📚 Key References for Next Session - -### Must Read Before Starting -1. `work/NEXT_SESSION_START_HERE.md` - Implementation guide -2. `work/architecture-diagram.md` - Visual reference -3. `../grasp/01.md` - THE SPEC (lines 1-40) -4. `../ngit-relay/src/nginx.conf` - Routing pattern - -### Reference During Implementation -1. `work/implementation-checklist.md` - Track progress -2. `work/current_status.md` - Overall context -3. [actix-web docs](https://actix.rs/docs/) - Framework reference -4. [git-http-backend docs](https://docs.rs/git-http-backend/) - Git protocol - -### Reference for Testing -1. `tests/common/relay.rs` - TestRelay fixture -2. `grasp-audit/src/specs/nip01_smoke.rs` - Test specs -3. `work/implementation-checklist.md` - Testing procedures - ---- - -## ✅ Accomplishments - -### Understanding -- ✅ Fully understand GRASP-01 requirements -- ✅ Understand ngit-relay architecture -- ✅ Identified critical mistake (separate ports) -- ✅ Understand how to fix it (actix-web routing) - -### Documentation -- ✅ Created comprehensive status document -- ✅ Created step-by-step implementation guide -- ✅ Created architecture diagrams -- ✅ Created detailed checklist -- ✅ Created review summary -- ✅ Created session summary - -### Planning -- ✅ Detailed 8-step implementation plan -- ✅ Identified all required changes -- ✅ Created acceptance criteria -- ✅ Prepared verification steps -- ✅ Listed common issues to watch for - -### Preparation -- ✅ Verified .env.example is correct -- ✅ Verified TestRelay is correct -- ✅ Identified which files need changes -- ✅ Created code templates for new files - ---- - -## 🚀 Ready for Implementation - -### What's Ready -- ✅ Complete understanding of requirements -- ✅ Detailed implementation plan -- ✅ Code templates prepared -- ✅ Test strategy defined -- ✅ Verification procedures documented - -### What's Needed -- [ ] Time to implement (~4 hours) -- [ ] Focus for coding -- [ ] Testing as we go -- [ ] Patience for debugging - -### Confidence Level -**HIGH** - We have: -- Clear understanding of problem -- Detailed solution plan -- Reference implementation to follow -- Good test coverage strategy -- Comprehensive documentation - ---- - -## 💡 Key Insights - -### 1. Architecture Matters -The single-port architecture is not just a detail - it's fundamental to GRASP-01 compliance. Getting this wrong means the whole implementation is wrong. - -### 2. Reference Implementation is Gold -ngit-relay's nginx.conf showed us EXACTLY how to route traffic. We don't need to guess - we can copy the pattern. - -### 3. Tests Must Map to Spec -Having tests that reference protocol line numbers makes verification trivial. We can see exactly which requirements we've met. - -### 4. Documentation Saves Time -Taking time to document our understanding and plan saves hours of confused implementation. We know exactly what to do. - -### 5. Incremental Progress -We can implement in phases: -1. HTTP routing (this phase) -2. Repository provisioning -3. Push authorization -4. Full compliance - -Each phase is testable and valuable on its own. - ---- - -## 🎓 Lessons Learned - -### What Went Well -- Thorough review of GRASP protocol -- Found critical architecture issue early -- Created comprehensive documentation -- Have clear path forward - -### What Could Be Better -- Could have reviewed GRASP spec earlier -- Could have checked ngit-relay architecture first -- Could have validated assumptions sooner - -### For Next Time -- Always check reference implementation first -- Read the spec thoroughly before coding -- Validate architecture assumptions early -- Document understanding before implementing - ---- - -## 📝 Notes for Future - -### When to Revisit This -- Before starting implementation (read NEXT_SESSION_START_HERE.md) -- When confused about architecture (read architecture-diagram.md) -- When stuck on a requirement (read review-summary.md) -- When tracking progress (read implementation-checklist.md) - -### What to Archive Later -- This session-summary.md → docs/archive/2025-11-04-grasp-review.md -- implementation-checklist.md → Delete after implementation complete -- NEXT_SESSION_START_HERE.md → Update for next phase - -### What to Keep -- current_status.md → Update as we progress -- architecture-diagram.md → Reference documentation -- review-summary.md → Reference for decisions - ---- - -## ✨ Final Thoughts - -This session was highly productive. We: -1. Identified a critical architectural flaw -2. Fully understood the correct architecture -3. Created a detailed implementation plan -4. Prepared everything needed for next session - -**We're ready to build this right.** - -The next session will be focused implementation - we have everything we need to succeed. - ---- - -**Session End:** November 4, 2025 -**Next Session:** Implementation of actix-web integration -**Confidence:** HIGH ✅ - ---- - -**Remember:** Start with `work/NEXT_SESSION_START_HERE.md` -- cgit v1.2.3