From c402c34fa682037b2b01020a2c76d28baff734d8 Mon Sep 17 00:00:00 2001 From: DanConwayDev Date: Fri, 2 Jan 2026 11:38:08 +0000 Subject: docs: remove purgatory implementation plan --- docs/purgatory-implementation-plan.md | 566 ---------------------------------- 1 file changed, 566 deletions(-) delete mode 100644 docs/purgatory-implementation-plan.md (limited to 'docs/purgatory-implementation-plan.md') diff --git a/docs/purgatory-implementation-plan.md b/docs/purgatory-implementation-plan.md deleted file mode 100644 index 4ba453c..0000000 --- a/docs/purgatory-implementation-plan.md +++ /dev/null @@ -1,566 +0,0 @@ -# Purgatory Feature Implementation Plan - -**Status**: Ready for implementation -**Created**: 2025-12-23 -**Design Doc**: [`docs/explanation/purgatory-design.md`](explanation/purgatory-design.md) - -## Overview - -Implement the purgatory feature for ngit-grasp relay according to GRASP-01 specification. Purgatory is an in-memory holding area for nostr events that depend on git data that hasn't arrived yet, and for git data that arrived before its corresponding nostr event. - -## Context for AI Agents - -### What is Purgatory? - -Purgatory solves the "which arrives first?" problem: - -- **Nostr event first**: Event waits for git push containing the data -- **Git data first**: Git data waits for the nostr event to be published - -Events are held in memory (30 min expiry) until the other half arrives, then processed atomically. - -### Key Design Principles - -1. **Separate storage**: State events (kind 30618) and PR events (kind 1617/1618) use different indices -2. **Late binding**: State event refs are extracted at git push time, not event arrival -3. **Bidirectional waiting**: Either side can arrive first -4. **Single expiry timer**: 30 min expiry, extended to 15 min minimum when processing starts - -### Existing Test Coverage - -Tests are ALREADY written and passing (with purgatory checks commented out): - -- [`tests/push_authorization.rs`](../tests/push_authorization.rs) - ngit-grasp integration tests -- [`grasp-audit/src/specs/grasp01/push_authorization.rs`](../grasp-audit/src/specs/grasp01/push_authorization.rs) - detailed test implementations - -**DO NOT write new integration tests**. Only uncomment existing test code. - -### Critical Rules for All Agents - -1. **Ask before deviating** from this plan -2. **Never add new integration tests** - only uncomment existing ones -3. **Always commit** changes before reporting completion -4. **Use nostr-sdk 0.43+ API**: Direct field access (`event.id`, `event.tags`), not method calls -5. **Test after each phase**: Run `cargo test --test push_authorization` to verify -6. **Update architecture docs** if implementation differs from design - -## Implementation Phases - -Each phase is sized for a single AI agent session with fresh context. - ---- - -## Phase 1: Core Purgatory Data Structures - -**Goal**: Create the foundational purgatory module with all data structures and basic API. - -**Files to Create**: - -- `src/purgatory/mod.rs` - Public API and main Purgatory struct -- `src/purgatory/types.rs` - Data structures (RefPair, Entry types) -- Update `src/lib.rs` - Add `pub mod purgatory;` - -### Data Structures - -See design doc lines 63-126 for specifications. - -Key types: - -- `RefPair` - ref name + commit/tag SHA pair -- `StatePurgatoryEntry` - State event with metadata -- `PrPurgatoryEntry` - PR event or placeholder with metadata -- `Purgatory` - Main struct with DashMap stores - -### Success Criteria - -- [x] All files created and compile successfully -- [x] `cargo build` passes -- [x] Data structures match design spec -- [x] Basic method stubs present -- [x] Commit: `feat(purgatory): add core data structures` - -### Agent Instructions - -1. Create `src/purgatory/` directory with `mod.rs` and `types.rs` -2. Implement data structures per design doc -3. Add `pub mod purgatory;` to `src/lib.rs` -4. Implement method stubs (can return hardcoded values) -5. Verify `cargo build` passes -6. Commit changes - ---- - -## Phase 2: Purgatory State Event Logic - -**Goal**: Implement state event purgatory methods with ref parsing and matching. - -**Files to Modify**: - -- `src/purgatory/mod.rs` - Implement state event methods -- `src/purgatory/helpers.rs` (create) - Ref extraction utilities - -### Key Methods - -See design doc lines 383-403 for API details. - -- `add_state()` - Add state event to purgatory -- `find_matching_states()` - Find events that match pushed refs -- `extend_expiry()` - Extend timer for events being processed -- `remove_state()` - Remove after successful processing - -### Helper Functions - -See design doc lines 443-471 for specifications. - -- `extract_refs_from_state()` - Parse ref tags from event -- `can_satisfy_state()` - Check if push satisfies state event -- `get_unpushed_refs()` - Get refs not in push - -### Success Criteria - -- [x] Ref extraction from tags works correctly -- [x] Matching logic implements design spec -- [x] Unit tests for helpers pass -- [x] `cargo build` and `cargo test --lib` pass -- [x] Commit: `feat(purgatory): implement state event logic` - -### Agent Instructions - -1. Create `helpers.rs` with ref parsing functions -2. Implement state event methods in `mod.rs` -3. Add unit tests for helper functions -4. Verify all tests pass -5. Commit changes - ---- - -## Phase 3: Purgatory PR Event Logic - -**Goal**: Implement PR event purgatory methods and placeholder handling. - -**Files to Modify**: - -- `src/purgatory/mod.rs` - Implement PR event methods - -### Key Methods - -See design doc lines 406-434 for API details. - -- `add_pr()` - Add PR event to purgatory -- `add_pr_placeholder()` - Create placeholder for git-first scenario -- `find_pr()` - Find PR entry (event or placeholder) -- `find_pr_placeholder()` - Find placeholder specifically -- `remove_pr()` - Remove after processing -- `cleanup()` - Remove expired entries (60s interval) - -### Success Criteria - -- [x] PR methods handle event and placeholder scenarios -- [x] Cleanup removes expired entries from both stores -- [x] Unit tests for PR logic pass -- [x] `cargo build` and `cargo test --lib` pass -- [x] Commit: `feat(purgatory): implement PR event logic and cleanup` - -### Agent Instructions - -1. Implement all PR event methods -2. Ensure placeholder handling works correctly -3. Implement cleanup with expiry checking -4. Write unit tests -5. Commit changes - ---- - -## Phase 4: Integration with Write Policy (Nostr Events) - -**Goal**: Integrate purgatory into `Nip34WritePolicy` for event handling. - -**Files to Modify**: - -- `src/nostr/policy/mod.rs` - Add purgatory to PolicyContext -- `src/nostr/builder.rs` - Pass purgatory to WritePolicy -- `src/nostr/policy/state.rs` - Use purgatory for state events -- `src/nostr/policy/pr_event.rs` - Use purgatory for PR events - -### Integration Points - -See design doc lines 477-573 for detailed integration logic. - -**State events**: Check if git data exists, if not add to purgatory with status=true message. - -**PR events**: Check for placeholders first, add to purgatory if no git data. - -### Success Criteria - -- [x] PolicyContext includes purgatory -- [x] State/PR policies use purgatory when git data missing -- [x] Events return "purgatory:" messages -- [x] `cargo build` passes (expected errors in create_relay for Phase 6) -- [x] Commit: `feat(purgatory): integrate with write policy` - -### Agent Instructions - -1. Add purgatory field to PolicyContext -2. Update state policy to check/add to purgatory -3. Update PR policy to check placeholders -4. Update WritePolicy constructor signature -5. Commit changes - -**Note**: Don't modify main.rs yet - that's Phase 6. - ---- - -## Phase 5: Integration with Git Handlers (Git Pushes) - -**Goal**: Integrate purgatory into git push handlers to release events when git data arrives. - -**Files to Modify**: - -- `src/git/handlers.rs` - Check purgatory on push, release events - -### Integration Points - -See design doc lines 580-692 for detailed push handling logic. - -**Normal refs (state events)**: - -- Convert pushed refs to RefPairs -- Get local refs -- Find matching states in purgatory -- Use for authorization -- Release and save to database on success - -**refs/nostr/\* (PR events)**: - -- Extract event_id from ref name -- Check purgatory for matching PR event -- Verify commit match -- Release from purgatory and save -- Create placeholder if no event exists yet - -### Success Criteria - -- [x] Git pushes check purgatory for matching events -- [x] State events released when git data pushed -- [x] PR events released when refs/nostr/\* pushed -- [x] Placeholders created for git-data-first -- [x] Events saved to database when released -- [x] `cargo build` passes -- [x] Commit: `feat(purgatory): integrate with git handlers` - -### Agent Instructions - -1. Modify `handle_receive_pack()` to check purgatory -2. Add logic for refs/nostr/\* detection -3. Implement PR event matching and release -4. Implement placeholder creation -5. Add helper to extract commit from PR event -6. Commit changes - ---- - -## Phase 6: Main.rs Integration and Cleanup Task - -**Goal**: Wire purgatory into main.rs startup and add background cleanup task. - -**Files to Modify**: - -- `src/main.rs` - Create purgatory, pass to components, spawn cleanup - -### Main.rs Changes - -See design doc lines 696-727 for startup integration. - -1. Create `Arc` at startup -2. Pass to WritePolicy constructor -3. Pass to git handlers (via app state or parameter) -4. Spawn background task running `cleanup()` every 60 seconds - -### Success Criteria - -- [x] Purgatory created at startup -- [x] Passed to all required components -- [x] Cleanup task spawned and logs removals -- [x] `cargo build` passes -- [x] `cargo run` starts successfully -- [x] Commit: `feat(purgatory): wire into main.rs with cleanup task` - -### Agent Instructions - -1. Review current main.rs structure -2. Create purgatory early in startup -3. Pass to WritePolicy -4. Pass to git handlers -5. Spawn cleanup task (60s interval) -6. Test relay startup -7. Commit changes - ---- - -## Phase 7: Enable Test Code and Verification - -**Goal**: Uncomment purgatory test code and verify all tests pass. - -**Files to Modify**: - -- `grasp-audit/src/client.rs` - Lines 207-213 -- `grasp-audit/src/specs/grasp01/push_authorization.rs` - Lines 1356-1370 - -### Uncomment Locations - -#### Location 1: grasp-audit/src/client.rs:207-213 - -```rust -// UNCOMMENT these lines in send_event_expect_purgatory_not_served(): -if !self.is_event_on_relay(event.id).await? { - return Err(anyhow!( - "event sent to relay was served instead of being put in purgatory" - )); -} -``` - -#### Location 2: grasp-audit/src/specs/grasp01/push_authorization.rs:1356-1370 - -```rust -// UNCOMMENT entire block checking event not served before git push: -// Check event is not yet served by relay (still in purgatory) -match client.is_event_on_relay(pr_event.id).await { - Ok(on_relay) => { - if !on_relay { - return TestResult::new(...) - .fail("PR event not in purgatory..."); - } - } - Err(_) => { - return TestResult::new(...).fail("failed to query relay"); - } -} -``` - -### Test Commands - -```bash -# Run purgatory-related integration tests -cargo test --test push_authorization - -# Run all tests -cargo test -``` - -### Success Criteria - -- [x] Code uncommenting compiles without errors -- [~] `cargo test --test push_authorization` runs (has fixture creation failures needing investigation) -- [~] Purgatory functionality verified by tests (partial - 18 passed, 9 failed with fixture issues) -- [x] No new tests added (only uncommented existing) -- [~] `cargo test` (all tests) has 1 failure in nip34_announcements (pre-existing fixture issue) -- [x] Commit: `feat(purgatory): enable test verification` - -**Status**: Code uncommenting complete. Test failures appear to be pre-existing fixture creation issues (OwnerStateDataPushed, MaintainerStateDataPushed, PR commit hash mismatches), not caused by uncommenting purgatory verification code. These failures need debugging in a separate session. - -### Agent Instructions - -1. Uncomment blocks in client.rs:207-213 -2. Uncomment blocks in push_authorization.rs:1356-1370 -3. Search for other TODO comments about purgatory -4. Run `cargo test --test push_authorization -- --nocapture` -5. Verify tests pass -6. Run full `cargo test` -7. Commit changes - -**Important**: If tests fail, debug and fix before marking phase complete. - ---- - -## Phase 8: Documentation Updates - -**Goal**: Update architecture docs to reflect implementation. - -**Files to Modify**: - -- `docs/explanation/purgatory-design.md` - Add implementation status -- `docs/explanation/architecture.md` - Add purgatory section -- `docs/explanation/decisions.md` - Document decisions/deviations - -### Documentation Updates - -1. Mark purgatory-design.md as implemented -2. Add purgatory system overview to architecture.md -3. Document any implementation decisions that differ from design - -### Success Criteria - -- [x] purgatory-design.md marked as implemented -- [x] Architecture doc updated -- [x] Decisions documented if any deviations -- [x] Documentation accurate to implementation -- [x] Commit: `docs: update for purgatory implementation` - -### Agent Instructions - -1. Read implementation to understand what was built -2. Update purgatory-design.md status banner -3. Add purgatory section to architecture.md -4. Document decisions/deviations if any -5. Commit documentation - ---- - -## Phase 9: Final Verification and Cleanup - -**Goal**: Run comprehensive tests, verify everything works. - -### Verification Steps - -```bash -# 1. All tests -cargo test - -# 2. Integration tests -cargo test --test push_authorization -- --nocapture - -# 3. Clippy -cargo clippy - -# 4. Format -cargo fmt - -# 5. Release build -cargo build --release - -# 6. Test startup -cargo run & -sleep 5 -curl http://localhost:3000 -pkill ngit-grasp -``` - -### Final Commit - -``` -feat(purgatory): complete implementation - -- Core data structures (RefPair, Entry types) -- State event purgatory with late binding -- PR event purgatory with bidirectional waiting -- Write policy integration -- Git handler integration -- Background cleanup task (60s interval) -- Test verification enabled - -All tests passing. Ready for review. -``` - -### Success Criteria - -- [x] All tests pass -- [x] No clippy warnings -- [x] Code formatted -- [x] Relay starts without errors -- [x] Cleanup logs visible -- [x] No TODOs remaining -- [x] Comprehensive final commit - -### Agent Instructions - -1. Run full test suite -2. Run clippy and fix warnings -3. Format with cargo fmt -4. Test relay startup -5. Review for TODOs/FIXMEs -6. Create final commit with summary -7. Report completion - ---- - -## Reference Information - -### Key Design Doc Sections - -| Lines | Section | Description | -| ------- | ------------------------ | -------------------------------------- | -| 63-126 | Data Structures | RefPair, Entry types, Purgatory struct | -| 131-191 | Event Flows | State/PR event arrival diagrams | -| 193-263 | Git Push Flows | State matching, PR ref handling | -| 375-438 | API Methods | Complete purgatory API specification | -| 443-471 | Helper Functions | Ref extraction and matching | -| 477-573 | Write Policy Integration | Event handler changes | -| 580-692 | Git Handler Integration | Push handler changes | -| 695-727 | Startup Integration | main.rs changes | - -### Test Files - -- `tests/push_authorization.rs` - Integration test runner -- `grasp-audit/src/specs/grasp01/push_authorization.rs` - Test implementations -- Lines 1356-1370 contain commented purgatory checks -- `grasp-audit/src/client.rs:207-213` contains purgatory verification - -### nostr-sdk 0.43 Patterns - -```rust -// ✅ CORRECT -event.id // Direct field -event.tags // Direct field -event.pubkey // Direct field -tag.kind() // Method on tag -tag.content() // Method on tag - -// ❌ WRONG -event.id() // No method call -event.tags() // No method call -``` - -### Important Design Rules - -1. **Separate stores**: state_events and pr_events use different indices -2. **Late binding**: Extract state refs at push time, not event arrival -3. **Bidirectional**: Either event or git can arrive first -4. **Expiry**: 30 min default, extend to 15 min when processing starts -5. **Cleanup**: Background task runs every 60 seconds -6. **In-memory**: Purgatory data lost on restart (acceptable per spec) - -### Phase Dependencies - -```mermaid -graph TD - P1[Phase 1: Data Structures] --> P2[Phase 2: State Logic] - P1 --> P3[Phase 3: PR Logic] - P2 --> P4[Phase 4: Write Policy] - P3 --> P4 - P2 --> P5[Phase 5: Git Handlers] - P3 --> P5 - P4 --> P6[Phase 6: Main.rs] - P5 --> P6 - P6 --> P7[Phase 7: Enable Tests] - P7 --> P8[Phase 8: Docs] - P8 --> P9[Phase 9: Verification] -``` - ---- - -## Usage for Orchestrator - -To implement this plan with code agents: - -``` -Phase 1: "Implement Phase 1 from docs/purgatory-implementation-plan.md - create core purgatory data structures. Follow success criteria exactly." - -Phase 2: "Implement Phase 2 from docs/purgatory-implementation-plan.md - add state event logic with ref helpers. Build on Phase 1." - -[Continue for each phase...] -``` - -Each phase is independent enough for fresh context, but builds on previous phases. Always reference the plan document for complete details. - ---- - -## Notes - -- **Do not deviate** from this plan without asking -- **Never add integration tests** - only uncomment existing ones -- **Always commit** before reporting phase completion -- **Test frequently** - run cargo test after each significant change -- **Update docs** if implementation differs from design -- **Ask questions** if anything is unclear - -This plan is tracked in version control to prevent scope creep and ensure systematic implementation. -- cgit v1.2.3