diff options
| author | DanConwayDev <DanConwayDev@protonmail.com> | 2025-12-04 12:18:11 +0000 |
|---|---|---|
| committer | DanConwayDev <DanConwayDev@protonmail.com> | 2025-12-04 12:18:11 +0000 |
| commit | 40831a9025d05fa354b7d8386eeebd902092ea86 (patch) | |
| tree | bafdfd60bb96fb90bfe3e6e95955a925d7b47e30 /docs/learnings/grasp-01-implementation.md | |
| parent | a2bb8ff62366d805ddb8ee08ac70ea71250a1c2d (diff) | |
docs: add llm review of implemenation vs plan
Diffstat (limited to 'docs/learnings/grasp-01-implementation.md')
| -rw-r--r-- | docs/learnings/grasp-01-implementation.md | 238 |
1 files changed, 238 insertions, 0 deletions
diff --git a/docs/learnings/grasp-01-implementation.md b/docs/learnings/grasp-01-implementation.md new file mode 100644 index 0000000..dea6389 --- /dev/null +++ b/docs/learnings/grasp-01-implementation.md | |||
| @@ -0,0 +1,238 @@ | |||
| 1 | # GRASP-01 Implementation Learnings | ||
| 2 | |||
| 3 | **Purpose:** Document learnings from the GRASP-01 implementation | ||
| 4 | **Date:** December 4, 2025 | ||
| 5 | |||
| 6 | --- | ||
| 7 | |||
| 8 | ## What Was Built | ||
| 9 | |||
| 10 | ### Core Relay Implementation (`src/`) | ||
| 11 | |||
| 12 | **Single Rust Binary** serving: | ||
| 13 | - **Nostr Relay** at `/` (WebSocket + HTTP for NIP-11) | ||
| 14 | - **Git Smart HTTP** at `/<npub>/<identifier>.git/*` | ||
| 15 | - **Landing Pages** at `/<npub>/<identifier>.git` and `/` | ||
| 16 | |||
| 17 | **Key Technical Choices:** | ||
| 18 | |||
| 19 | | Component | Choice | Rationale | | ||
| 20 | |-----------|--------|-----------| | ||
| 21 | | HTTP Server | Hyper (not actix-web) | Better control over WebSocket upgrade handling | | ||
| 22 | | Nostr Relay | nostr-relay-builder | Mature, well-tested, supports custom policies | | ||
| 23 | | Database | LMDB (default), NostrDB, Memory | LMDB for production, Memory for testing | | ||
| 24 | | Configuration | clap + dotenvy | CLI flags > env vars > .env > defaults | | ||
| 25 | |||
| 26 | --- | ||
| 27 | |||
| 28 | ### GRASP-01 Compliance Features | ||
| 29 | |||
| 30 | **Nostr Relay Requirements:** | ||
| 31 | - ✅ NIP-01 compliant relay at `/` | ||
| 32 | - ✅ Accepts repository announcements listing this service | ||
| 33 | - ✅ Accepts repository state announcements | ||
| 34 | - ✅ Accepts events tagging/tagged by accepted repos | ||
| 35 | - ✅ Recursive maintainer announcement support | ||
| 36 | - ✅ NIP-11 document with `supported_grasps` field | ||
| 37 | - ✅ CORS headers on all endpoints | ||
| 38 | |||
| 39 | **Git HTTP Service:** | ||
| 40 | - ✅ Serves git at `/<npub>/<identifier>.git` | ||
| 41 | - ✅ Push validation against state events | ||
| 42 | - ✅ Recursive maintainer chain support | ||
| 43 | - ✅ HEAD set from state events | ||
| 44 | - ✅ `refs/nostr/<event-id>` support for PRs | ||
| 45 | - ✅ `allow-tip-sha1-in-want` and `allow-reachable-sha1-in-want` | ||
| 46 | |||
| 47 | --- | ||
| 48 | |||
| 49 | ### Audit Tool (`grasp-audit/`) | ||
| 50 | |||
| 51 | **Reusable Compliance Testing:** | ||
| 52 | - Separate crate with own `Cargo.toml` and `flake.nix` | ||
| 53 | - Spec-mirrored test structure matching GRASP-01 sections | ||
| 54 | - Automatic cleanup tags for production-safe testing | ||
| 55 | - Can audit any GRASP implementation (not just ngit-grasp) | ||
| 56 | |||
| 57 | --- | ||
| 58 | |||
| 59 | ## Key Implementation Patterns | ||
| 60 | |||
| 61 | ### 1. Event Acceptance Flow | ||
| 62 | |||
| 63 | ```mermaid | ||
| 64 | flowchart TD | ||
| 65 | E[Incoming Event] --> K{Kind?} | ||
| 66 | K -->|30617| A[Repository Announcement] | ||
| 67 | K -->|30618| S[Repository State] | ||
| 68 | K -->|Other| R[Related Event Check] | ||
| 69 | |||
| 70 | A --> AC{Lists our service in clone AND relays?} | ||
| 71 | AC -->|Yes| ACCEPT1[Accept + Create Repo] | ||
| 72 | AC -->|No| RM{Is recursive maintainer?} | ||
| 73 | RM -->|Yes| ACCEPT2[Accept - for discovery] | ||
| 74 | RM -->|No| REJECT1[Reject] | ||
| 75 | |||
| 76 | S --> SA{Author authorized?} | ||
| 77 | SA -->|Yes| SL{Is latest state?} | ||
| 78 | SL -->|Yes| ACCEPT3[Accept + Align Refs] | ||
| 79 | SL -->|No| ACCEPT4[Accept - not latest] | ||
| 80 | SA -->|No| REJECT2[Reject] | ||
| 81 | |||
| 82 | R --> RF{References accepted event OR referenced by accepted?} | ||
| 83 | RF -->|Yes| ACCEPT5[Accept] | ||
| 84 | RF -->|No| REJECT3[Reject] | ||
| 85 | ``` | ||
| 86 | |||
| 87 | ### 2. State Event to Git Alignment | ||
| 88 | |||
| 89 | When a valid state event arrives: | ||
| 90 | 1. Find all announcements where author is authorized | ||
| 91 | 2. For each matching announcement, check if this is the latest state | ||
| 92 | 3. If latest AND git data exists, align refs: | ||
| 93 | - Create refs that exist in state but not in repo | ||
| 94 | - Update refs pointing to wrong commits | ||
| 95 | - Delete refs not in state (for refs/heads/ and refs/tags/) | ||
| 96 | - Set HEAD if HEAD branch commit is available | ||
| 97 | |||
| 98 | ### 3. Push Authorization | ||
| 99 | |||
| 100 | ```mermaid | ||
| 101 | flowchart LR | ||
| 102 | P[Push Request] --> PA[Parse Ref Updates] | ||
| 103 | PA --> QS[Query State Event] | ||
| 104 | QS --> GM[Build Maintainer Set] | ||
| 105 | GM --> VR[Validate Each Ref] | ||
| 106 | VR -->|Valid| FWD[Forward to Git] | ||
| 107 | VR -->|Invalid| ERR[HTTP 403] | ||
| 108 | ``` | ||
| 109 | |||
| 110 | --- | ||
| 111 | |||
| 112 | ## What Worked Well | ||
| 113 | |||
| 114 | ### 1. Inline Authorization | ||
| 115 | |||
| 116 | The decision to validate pushes **before** spawning git-receive-pack worked extremely well: | ||
| 117 | - Better error messages (HTTP responses vs git hook stderr) | ||
| 118 | - No hook management complexity | ||
| 119 | - Shared state between Nostr and Git components | ||
| 120 | - Easy Rust-native testing | ||
| 121 | |||
| 122 | ### 2. nostr-relay-builder Integration | ||
| 123 | |||
| 124 | Using rust-nostr's relay builder was the right call: | ||
| 125 | - Handles NIP-01 protocol correctly | ||
| 126 | - Custom `WritePolicy` trait for our validation | ||
| 127 | - Database abstraction (LMDB, NostrDB, Memory) | ||
| 128 | - Active maintenance and updates | ||
| 129 | |||
| 130 | ### 3. Separate Audit Tool | ||
| 131 | |||
| 132 | Building `grasp-audit` as a separate crate enabled: | ||
| 133 | - Test development in parallel with implementation | ||
| 134 | - Reusable by other GRASP implementations | ||
| 135 | - Production-safe auditing with cleanup tags | ||
| 136 | - Spec-mirrored test organization | ||
| 137 | |||
| 138 | ### 4. Test Isolation Strategy | ||
| 139 | |||
| 140 | Automatic cleanup tags on audit events: | ||
| 141 | ```rust | ||
| 142 | ["t", "grasp-audit-test-event"] // Marker | ||
| 143 | ["t", "audit-{run_id}"] // Run isolation | ||
| 144 | ["t", "audit-cleanup-after-{unix_timestamp}"] // Cleanup time | ||
| 145 | ``` | ||
| 146 | |||
| 147 | This enables parallel CI runs without interference. | ||
| 148 | |||
| 149 | --- | ||
| 150 | |||
| 151 | ## What I'd Do Differently | ||
| 152 | |||
| 153 | ### 1. Keep Architecture Docs Updated | ||
| 154 | |||
| 155 | **What happened:** Architecture design docs were essential to start - they guided the implementation. But as we made decisions (e.g., hyper instead of actix-web), the docs weren't updated to reflect reality. | ||
| 156 | |||
| 157 | **Better approach:** Treat architecture docs as living documents. When implementation diverges from the plan, update the doc immediately. The initial design document was valuable and should remain, but it should reflect what was built. | ||
| 158 | |||
| 159 | ### 2. Smaller Nip34WritePolicy | ||
| 160 | |||
| 161 | **What happened:** The [`Nip34WritePolicy`](src/nostr/builder.rs:51) grew to ~900 lines handling all event types. | ||
| 162 | |||
| 163 | **Better approach:** Split into: | ||
| 164 | - `AnnouncementPolicy` - Repository announcement validation | ||
| 165 | - `StatePolicy` - State event validation + ref alignment | ||
| 166 | - `RelatedEventPolicy` - Forward/backward reference checking | ||
| 167 | - `PrEventPolicy` - PR/PR Update validation | ||
| 168 | |||
| 169 | This would improve testability and readability. | ||
| 170 | |||
| 171 | ### 3. Git Operations Module Organization | ||
| 172 | |||
| 173 | **What happened:** Many git operations are in [`src/git/mod.rs`](src/git/mod.rs) (500+ lines). | ||
| 174 | |||
| 175 | **Better approach:** Split into: | ||
| 176 | - `refs.rs` - Ref manipulation (create, update, delete, list) | ||
| 177 | - `head.rs` - HEAD management | ||
| 178 | - `objects.rs` - Object verification (commit_exists) | ||
| 179 | - `repository.rs` - Repository lifecycle (create, configure) | ||
| 180 | |||
| 181 | ### 4. Unit Tests for Policy Logic | ||
| 182 | |||
| 183 | **What happened:** Event acceptance policy testing relies on integration tests in `grasp-audit` and the main project's test suite. | ||
| 184 | |||
| 185 | **Better approach:** Also have unit tests for [`Nip34WritePolicy`](src/nostr/builder.rs:51) that mock the database queries. This provides faster feedback during development without needing a running relay. | ||
| 186 | |||
| 187 | --- | ||
| 188 | |||
| 189 | ## Technical Debt to Address | ||
| 190 | |||
| 191 | ### High Priority | ||
| 192 | |||
| 193 | 1. **Split `Nip34WritePolicy`** - Too large, hard to test/maintain | ||
| 194 | 2. **Add unit tests for policy logic** - Currently relies on integration tests | ||
| 195 | 3. **Document actual architecture** - Docs describe plans, not implementation | ||
| 196 | |||
| 197 | ### Medium Priority | ||
| 198 | |||
| 199 | 1. **Error types cleanup** - Some places use `String` errors, should use proper error types | ||
| 200 | 2. **Tracing improvements** - Add structured logging for better debugging | ||
| 201 | 3. **Configuration validation** - Validate domain format, paths, etc. | ||
| 202 | |||
| 203 | ### Low Priority | ||
| 204 | |||
| 205 | 1. **Git protocol parsing** - Could be more efficient with zero-copy | ||
| 206 | 2. **Connection pooling** - For future GRASP-02 relay connections | ||
| 207 | 3. **Metrics/observability** - Prometheus integration | ||
| 208 | |||
| 209 | --- | ||
| 210 | |||
| 211 | ## Lessons for GRASP-02 | ||
| 212 | |||
| 213 | Based on GRASP-01 experience: | ||
| 214 | |||
| 215 | 1. **Define sync policies early** - Which events to sync, from where | ||
| 216 | 2. **Plan connection lifecycle** - Backoff, reconnection, health tracking | ||
| 217 | 3. **Consider filter efficiency** - Negentropy vs regular subscriptions | ||
| 218 | 4. **Test sync scenarios** - Network partitions, out-of-order events | ||
| 219 | 5. **Measure sync gaps** - Track when catchup finds events that live sync missed | ||
| 220 | |||
| 221 | --- | ||
| 222 | |||
| 223 | ## Quick Reference: What's Where | ||
| 224 | |||
| 225 | | Feature | Location | | ||
| 226 | |---------|----------| | ||
| 227 | | Event acceptance policy | [`src/nostr/builder.rs:51`](src/nostr/builder.rs:51) - `Nip34WritePolicy` | | ||
| 228 | | Repo/State parsing | [`src/nostr/events.rs`](src/nostr/events.rs) | | ||
| 229 | | Git HTTP handlers | [`src/git/handlers.rs`](src/git/handlers.rs) | | ||
| 230 | | Push authorization | [`src/git/authorization.rs`](src/git/authorization.rs) | | ||
| 231 | | HTTP server | [`src/http/mod.rs`](src/http/mod.rs) | | ||
| 232 | | NIP-11 document | [`src/http/nip11.rs`](src/http/nip11.rs) | | ||
| 233 | | Audit test specs | [`grasp-audit/src/specs/grasp01/`](grasp-audit/src/specs/grasp01/) | | ||
| 234 | | Integration tests | [`tests/`](tests/) | | ||
| 235 | |||
| 236 | --- | ||
| 237 | |||
| 238 | *Created: December 4, 2025* \ No newline at end of file | ||