upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/explanation/architecture.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/explanation/architecture.md')
-rw-r--r--docs/explanation/architecture.md131
1 files changed, 62 insertions, 69 deletions
diff --git a/docs/explanation/architecture.md b/docs/explanation/architecture.md
index e0a57e5..09737df 100644
--- a/docs/explanation/architecture.md
+++ b/docs/explanation/architecture.md
@@ -2,53 +2,7 @@
2 2
3## Executive Summary 3## Executive Summary
4 4
5`ngit-grasp` implements the GRASP protocol in Rust with **inline authorization** rather than Git hooks. The key architectural insight is that we can intercept and validate Git push operations at the HTTP handler level before reaching the Git repository, eliminating the need for pre-receive hooks. 5`ngit-grasp` implements the GRASP protocol in Rust with **inline authorization** rather than Git hooks. Git push operations are intercepted and validated at the HTTP handler level before reaching the Git repository, eliminating the need for pre-receive hooks.
6
7## Architectural Decision: Inline vs. Hook-Based Authorization
8
9### Investigation Summary
10
11After examining both the reference implementation and HTTP server options, we have two options:
12
13#### Option 1: Hook-Based (Reference Implementation Approach)
14
15- Use standard Git HTTP backend
16- Create pre-receive and post-receive hooks
17- Hooks query the Nostr relay and validate pushes
18- **Pros**: Follows reference implementation closely
19- **Cons**: Requires hook management, harder to test, less Rust-native
20
21#### Option 2: Inline Authorization (Recommended)
22
23- Intercept Git receive-pack requests in the HTTP handler
24- Validate against Nostr state before spawning Git process
25- Only forward valid pushes to Git
26- **Pros**: Better error handling, easier testing, pure Rust, simpler deployment
27- **Cons**: Requires custom Git protocol handling
28
29### Decision: Inline Authorization (Option 2)
30
31**Rationale:**
32
331. **Full control over HTTP layer**: Using Hyper directly gives us complete control over request handling, WebSocket upgrades, and CORS headers.
34
352. **Better Developer Experience**:
36
37 - Validation errors can be returned as proper HTTP responses
38 - No need to parse hook stderr output
39 - Shared state between Git and Nostr components
40 - Pure Rust testing without shell scripts
41
423. **Simpler Deployment**:
43
44 - Single binary
45 - No hook symlinks or permissions to manage
46 - No multi-process coordination
47
484. **Performance**:
49 - Can parse incoming pack data once
50 - Avoid process spawn overhead for invalid pushes
51 - Better async integration
52 6
53## System Architecture 7## System Architecture
54 8
@@ -257,57 +211,73 @@ State events undergo authorization checks at multiple points:
257 211
258### 5. Purgatory System ([`src/purgatory/`](../../src/purgatory/)) 212### 5. Purgatory System ([`src/purgatory/`](../../src/purgatory/))
259 213
260The purgatory system solves the "which arrives first?" problem where either nostr events or git pushes can arrive in any order. It provides an in-memory holding area for events and git data awaiting their counterparts. 214The purgatory system solves two related problems:
215
2161. **"Which arrives first?"** — Either nostr events or git pushes can arrive in any order. Purgatory holds events awaiting their git data counterparts.
2172. **Misleading empty repository announcements** — New announcements are held in purgatory until git data arrives, ensuring clients are never served announcements for repos with no content.
261 218
262**Design Document**: See [`purgatory-design.md`](purgatory-design.md) for complete design specifications. 219**Design Document**: See [`purgatory-design.md`](purgatory-design.md) for complete design specifications.
263 220
264#### Architecture 221#### Architecture
265 222
266```rust 223```rust
267/// Main purgatory structure with two separate stores 224/// Main purgatory structure with separate stores per event type
268pub struct Purgatory { 225pub struct Purgatory {
226 /// Announcement events (kind 30617) indexed by (owner, identifier)
227 /// Held until git data proves content exists
228 announcement_purgatory: DashMap<(PublicKey, String), AnnouncementPurgatoryEntry>,
229
269 /// State events (kind 30618) indexed by repository identifier 230 /// State events (kind 30618) indexed by repository identifier
270 state_events: Arc<DashMap<String, Vec<StatePurgatoryEntry>>>, 231 state_events: DashMap<String, Vec<StatePurgatoryEntry>>,
271 232
272 /// PR events (kind 1617/1618) or placeholders indexed by event ID 233 /// PR events (kind 1617/1618) or placeholders indexed by event ID
273 pr_events: Arc<DashMap<String, PrPurgatoryEntry>>, 234 pr_events: DashMap<String, PrPurgatoryEntry>,
274} 235}
275``` 236```
276 237
277**Key Design Principles:** 238**Key Design Principles:**
278 239
2791. **Separate Storage**: State events and PR events use different indexing strategies 2401. **Separate Storage**: Each event type uses a different indexing strategy
241 - Announcements: Indexed by `(pubkey, identifier)` (unique per owner)
280 - State events: Indexed by `identifier` (multiple events can wait for same repo) 242 - State events: Indexed by `identifier` (multiple events can wait for same repo)
281 - PR events: Indexed by `event_id` (one-to-one mapping) 243 - PR events: Indexed by `event_id` (one-to-one mapping)
282 244
2832. **Late Binding**: State event refs are extracted at git push time, not event arrival 2452. **Announcement Purgatory**: New announcements are held until git data arrives
246 - Bare repo created immediately so pushes can succeed
247 - Announcement promoted to database only when git data proves content exists
248 - Two-phase soft expiry: bare repo deleted at 30 min, event retained 24h for revival
249
2503. **Late Binding**: State event refs are extracted at git push time, not event arrival
284 - Enables flexible matching when pushes arrive out-of-order 251 - Enables flexible matching when pushes arrive out-of-order
285 - Helper functions in [`helpers.rs`](../../src/purgatory/helpers.rs) handle ref extraction 252 - Helper functions in [`helpers.rs`](../../src/purgatory/helpers.rs) handle ref extraction
286 253
2873. **Bidirectional Waiting**: Either side can arrive first 2544. **Bidirectional Waiting**: Either side can arrive first
288 - **Event-first**: Event waits for git push 255 - **Event-first**: Event waits for git push
289 - **Git-first**: Placeholder created, waits for event 256 - **Git-first**: Placeholder created, waits for event
290 257
2914. **Automatic Expiry**: 30-minute default expiry, extensible during processing 2585. **Automatic Expiry**: 30-minute default expiry, extensible during processing
292 - Background cleanup task runs every 60 seconds 259 - Background cleanup task runs every 60 seconds
293 - Removes expired entries from both stores 260 - Removes expired entries from all stores
294 261
295#### Data Types 262#### Data Types
296 263
297See [`types.rs`](../../src/purgatory/types.rs) for complete definitions: 264See [`types.rs`](../../src/purgatory/types.rs) for complete definitions:
298 265
299- **[`RefPair`](../../src/purgatory/types.rs:16)**: Ref name + object SHA pair 266- **[`RefPair`](../../src/purgatory/types.rs:16)**: Ref name + object SHA pair
267- **[`AnnouncementPurgatoryEntry`](../../src/purgatory/types.rs)**: Announcement with bare repo path, relays, and expiry
300- **[`StatePurgatoryEntry`](../../src/purgatory/types.rs:29)**: State event with metadata 268- **[`StatePurgatoryEntry`](../../src/purgatory/types.rs:29)**: State event with metadata
301- **[`PrPurgatoryEntry`](../../src/purgatory/types.rs:52)**: PR event or placeholder with metadata 269- **[`PrPurgatoryEntry`](../../src/purgatory/types.rs:52)**: PR event or placeholder with metadata
302 270
303#### Integration Points 271#### Integration Points
304 272
305**Write Policy** ([`src/nostr/policy/`](../../src/nostr/policy/)): 273**Write Policy** ([`src/nostr/policy/`](../../src/nostr/policy/)):
306- State policy checks git data existence before adding to purgatory 274- Announcement policy routes new announcements to purgatory; replacements accepted immediately
275- State policy checks git data existence before adding to purgatory; checks purgatory announcements for authorization
307- PR policy checks for placeholders before adding to purgatory 276- PR policy checks for placeholders before adding to purgatory
308- Events return "purgatory: will not be served until git data arrives" message 277- Events return "purgatory: will not be served until git data arrives" message
309 278
310**Git Handlers** ([`src/git/handlers.rs`](../../src/git/handlers.rs)): 279**Git Handlers** ([`src/git/handlers.rs`](../../src/git/handlers.rs)):
280- On git push: Promote announcement from purgatory to database if present
311- On git push: Check purgatory for matching state events 281- On git push: Check purgatory for matching state events
312- On refs/nostr/* push: Check purgatory for PR events or create placeholders 282- On refs/nostr/* push: Check purgatory for PR events or create placeholders
313- Release events from purgatory when git data arrives 283- Release events from purgatory when git data arrives
@@ -392,11 +362,25 @@ Configuration is loaded via **clap CLI > environment variables > .env > defaults
392 └─ Accept or reject 362 └─ Accept or reject
393 363
3944. If ACCEPTED: 3644. If ACCEPTED:
395 ├─ Event saved to database 365 ├─ Is there an active announcement for (pubkey, identifier) in DB?
396 └─ ensure_bare_repository() called 366 │ ├─ YES → Accept immediately (replacement, repo already proven)
367 │ └─ NO → Route to announcement purgatory
397 368
3985. Bare Git repository created at 3695. Announcement Purgatory path:
399 <git_data_path>/<npub>/<identifier>.git 370 ├─ Bare Git repository created immediately at
371 │ <git_data_path>/<npub>/<identifier>.git
372 ├─ Announcement held in purgatory (not served to clients)
373 └─ Awaiting git data to prove content exists
374
3756. When git data arrives (push or background sync):
376 ├─ Announcement promoted from purgatory to database
377 ├─ Event now served to clients
378 └─ SyncManager upgrades to Full sync level
379
3807. If no git data within 30 minutes:
381 ├─ Bare repo deleted (soft expiry)
382 ├─ Event retained 24h for potential revival
383 └─ Eventually discarded if no git data arrives
400``` 384```
401 385
402### State Event Flow 386### State Event Flow
@@ -407,14 +391,25 @@ Configuration is loaded via **clap CLI > environment variables > .env > defaults
4072. Nostr relay receives event 3912. Nostr relay receives event
408 392
4093. Nip34WritePolicy::admit_event() 3933. Nip34WritePolicy::admit_event()
410 ├─ Check author is in maintainer set 394 ├─ Check author is in maintainer set (DB + purgatory announcements)
411 ├─ Validate state structure 395 ├─ Validate state structure
412 └─ Accept or reject 396 └─ Accept or reject
413 397
4144. If ACCEPTED and is latest state: 3984. If ACCEPTED:
415 ├─ Align repository refs to match state 399 ├─ Does git data already exist for this state?
416 ├─ Create/update/delete refs as needed 400 │ ├─ YES → Save to database immediately
417 └─ Set HEAD if commit available 401 │ └─ NO → Add to state purgatory
402
4035. State Purgatory path:
404 ├─ Event held in purgatory (not served to clients)
405 ├─ Enqueued for background git data sync (3 min delay)
406 └─ Awaiting git push or background sync
407
4086. When git push arrives:
409 ├─ Authorization checks both database AND purgatory
410 ├─ If authorized via purgatory state: push proceeds
411 ├─ After successful push: state event saved to database
412 └─ Removed from purgatory
418``` 413```
419 414
420## Testing Strategy 415## Testing Strategy
@@ -613,9 +608,7 @@ WantedBy=multi-user.target
613 608
614## Conclusion 609## Conclusion
615 610
616The inline authorization approach provides a cleaner, more maintainable architecture than hook-based authorization while maintaining full GRASP-01 compliance. Using Hyper for the HTTP layer gives us complete control over request handling, WebSocket upgrades, and CORS headers. 611ngit-grasp uses inline authorization at the HTTP handler level, giving full control over request handling, WebSocket upgrades, and CORS headers while maintaining full GRASP-01 compliance. The purgatory system ensures that only repositories with actual git content are served to clients, and that events and git data are always consistent when released to the database.
617
618The key insight is that we don't need to rely on Git's hook mechanism when we have full control over the HTTP layer that Git operates through. By intercepting at the HTTP handler level, we gain better error handling, easier testing, and tighter integration between the Git and Nostr components.
619 612
620## Related Documentation 613## Related Documentation
621 614