upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/explanation/architecture.md
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2025-12-10 13:08:41 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2025-12-10 13:08:41 +0000
commit67f7ebdaeff7b17af077c6ae7d2ecdf786ddf2ef (patch)
tree7f8952c9648b8563ee7e345e761013944acb0d01 /docs/explanation/architecture.md
parent8cfe8546e5ed1118adae6bfa041611e94d15c6dd (diff)
docs: update sync docs post implementation
Diffstat (limited to 'docs/explanation/architecture.md')
-rw-r--r--docs/explanation/architecture.md60
1 files changed, 55 insertions, 5 deletions
diff --git a/docs/explanation/architecture.md b/docs/explanation/architecture.md
index 3fb895c..efa3423 100644
--- a/docs/explanation/architecture.md
+++ b/docs/explanation/architecture.md
@@ -11,6 +11,7 @@
11After examining both the reference implementation and HTTP server options, we have two options: 11After examining both the reference implementation and HTTP server options, we have two options:
12 12
13#### Option 1: Hook-Based (Reference Implementation Approach) 13#### Option 1: Hook-Based (Reference Implementation Approach)
14
14- Use standard Git HTTP backend 15- Use standard Git HTTP backend
15- Create pre-receive and post-receive hooks 16- Create pre-receive and post-receive hooks
16- Hooks query the Nostr relay and validate pushes 17- Hooks query the Nostr relay and validate pushes
@@ -18,6 +19,7 @@ After examining both the reference implementation and HTTP server options, we ha
18- **Cons**: Requires hook management, harder to test, less Rust-native 19- **Cons**: Requires hook management, harder to test, less Rust-native
19 20
20#### Option 2: Inline Authorization (Recommended) 21#### Option 2: Inline Authorization (Recommended)
22
21- Intercept Git receive-pack requests in the HTTP handler 23- Intercept Git receive-pack requests in the HTTP handler
22- Validate against Nostr state before spawning Git process 24- Validate against Nostr state before spawning Git process
23- Only forward valid pushes to Git 25- Only forward valid pushes to Git
@@ -30,13 +32,15 @@ After examining both the reference implementation and HTTP server options, we ha
30 32
311. **Full control over HTTP layer**: Using Hyper directly gives us complete control over request handling, WebSocket upgrades, and CORS headers. 331. **Full control over HTTP layer**: Using Hyper directly gives us complete control over request handling, WebSocket upgrades, and CORS headers.
32 34
332. **Better Developer Experience**: 352. **Better Developer Experience**:
36
34 - Validation errors can be returned as proper HTTP responses 37 - Validation errors can be returned as proper HTTP responses
35 - No need to parse hook stderr output 38 - No need to parse hook stderr output
36 - Shared state between Git and Nostr components 39 - Shared state between Git and Nostr components
37 - Pure Rust testing without shell scripts 40 - Pure Rust testing without shell scripts
38 41
393. **Simpler Deployment**: 423. **Simpler Deployment**:
43
40 - Single binary 44 - Single binary
41 - No hook symlinks or permissions to manage 45 - No hook symlinks or permissions to manage
42 - No multi-process coordination 46 - No multi-process coordination
@@ -93,6 +97,7 @@ After examining both the reference implementation and HTTP server options, we ha
93### 1. Main Server ([`src/main.rs`](src/main.rs)) 97### 1. Main Server ([`src/main.rs`](src/main.rs))
94 98
95**Responsibilities:** 99**Responsibilities:**
100
96- Initialize configuration from environment (clap + dotenvy) 101- Initialize configuration from environment (clap + dotenvy)
97- Set up Hyper HTTP server with request routing 102- Set up Hyper HTTP server with request routing
98- Initialize Nostr relay builder with custom [`Nip34WritePolicy`](src/nostr/builder.rs:51) 103- Initialize Nostr relay builder with custom [`Nip34WritePolicy`](src/nostr/builder.rs:51)
@@ -101,6 +106,7 @@ After examining both the reference implementation and HTTP server options, we ha
101- Handle graceful shutdown 106- Handle graceful shutdown
102 107
103**Key Dependencies:** 108**Key Dependencies:**
109
104```rust 110```rust
105hyper = "1" 111hyper = "1"
106tokio = { version = "1", features = ["full"] } 112tokio = { version = "1", features = ["full"] }
@@ -112,6 +118,7 @@ nostr-lmdb = "0.43"
112### 2. HTTP Module ([`src/http/mod.rs`](src/http/mod.rs)) 118### 2. HTTP Module ([`src/http/mod.rs`](src/http/mod.rs))
113 119
114**Responsibilities:** 120**Responsibilities:**
121
115- Route HTTP requests to appropriate handlers 122- Route HTTP requests to appropriate handlers
116- WebSocket upgrade for Nostr relay at `/` 123- WebSocket upgrade for Nostr relay at `/`
117- Git Smart HTTP endpoints at `/<npub>/<identifier>.git/*` 124- Git Smart HTTP endpoints at `/<npub>/<identifier>.git/*`
@@ -228,7 +235,7 @@ Provides structures for parsing NIP-34 events:
228/// Parsed repository announcement (Kind 30617) 235/// Parsed repository announcement (Kind 30617)
229pub struct RepositoryAnnouncement { ... } 236pub struct RepositoryAnnouncement { ... }
230 237
231/// Parsed repository state (Kind 30618) 238/// Parsed repository state (Kind 30618)
232pub struct RepositoryState { ... } 239pub struct RepositoryState { ... }
233``` 240```
234 241
@@ -332,10 +339,12 @@ See [test-strategy.md](../reference/test-strategy.md) for comprehensive testing
332### Quick Overview 339### Quick Overview
333 340
334**Integration Tests** ([`tests/`](tests/)): 341**Integration Tests** ([`tests/`](tests/)):
342
335- Use [`TestRelay`](tests/common/relay.rs:14) fixture for automatic relay lifecycle 343- Use [`TestRelay`](tests/common/relay.rs:14) fixture for automatic relay lifecycle
336- Each test file in [`tests/`](tests/) covers a GRASP-01 requirement 344- Each test file in [`tests/`](tests/) covers a GRASP-01 requirement
337 345
338**Audit Tests** ([`grasp-audit/`](grasp-audit/)): 346**Audit Tests** ([`grasp-audit/`](grasp-audit/)):
347
339- Reusable compliance testing for any GRASP implementation 348- Reusable compliance testing for any GRASP implementation
340- Spec-mirrored structure in [`grasp-audit/src/specs/grasp01/`](grasp-audit/src/specs/grasp01/) 349- Spec-mirrored structure in [`grasp-audit/src/specs/grasp01/`](grasp-audit/src/specs/grasp01/)
341 350
@@ -367,11 +376,52 @@ async fn test_nip01_websocket_connection() {
367- Maintainer sets computed once per event validation 376- Maintainer sets computed once per event validation
368- State lookups use database indexes 377- State lookups use database indexes
369 378
379## Proactive Sync (GRASP-02)
380
381The ngit-grasp relay implements **Proactive Sync of Nostr Eevents**, which synchronizes repository data from external relays listed in 30617 repository announcements. This enables the relay to maintain complete repository graphs even when events are published to other listed relays.
382
383**Key Features:**
384
385- **Self-subscription** discovery - monitors own relay for announcements and root events to follow
386- **Three-way diff** (`compute_actions`) determines new subscriptions needed
387- **Smart reconnection** - uses `since` filter for quick reconnects (<15 min), fresh sync otherwise
388- **Health tracking** with exponential backoff for failing relays
389- **Daily sync** with random 23-25h timer to detect state drift
390- **Filter consolidation** when count exceeds 70 to prevent subscription explosion
391
392**Architecture:**
393
394```
395┌─────────────────────────────────────────────────────────────┐
396│ SyncManager │
397├─────────────────────────────────────────────────────────────┤
398│ │
399│ ┌──────────────────┐ ┌──────────────────┐ │
400│ │ SelfSubscriber │──actions──▶ │ Main Event Loop │ │
401│ │ (own relay) │ │ (Arc<Mutex>) │ │
402│ └──────────────────┘ └────────┬─────────┘ │
403│ │ │
404│ ┌──────────────────┐ ┌────────▼─────────┐ │
405│ │ Daily Timer │──────────────▶ RelayConnection │ │
406│ │ (23-25h random) │ │ per external │ │
407│ └──────────────────┘ │ relay │ │
408│ └──────────────────┘ │
409│ ┌──────────────────┐ │
410│ │ Health Tracker │ Exponential backoff, dead detection │
411│ │ (DashMap) │ │
412│ └──────────────────┘ │
413└─────────────────────────────────────────────────────────────┘
414```
415
416**Source Code:** [`src/sync/`](src/sync/)
417
418For full design details, see [grasp-02-proactive-sync-v4.md](grasp-02-proactive-sync-v4.md).
419
370## Future Extensions 420## Future Extensions
371 421
372### GRASP-02: Proactive Sync 422### GRASP-02: Proactive Sync
373 423
374See [grasp-02-proactive-sync.md](grasp-02-proactive-sync.md) for detailed design. 424GRASP-02 is only partially implemented. still outstanding is the proactive sync of git data for 1. state event and 2. PRs / PR Update refs.
375 425
376### GRASP-05: Archive 426### GRASP-05: Archive
377 427
@@ -437,6 +487,6 @@ The key insight is that we don't need to rely on Git's hook mechanism when we ha
437## Related Documentation 487## Related Documentation
438 488
439- [Inline Authorization Explanation](inline-authorization.md) - Why we chose this approach 489- [Inline Authorization Explanation](inline-authorization.md) - Why we chose this approach
440- [GRASP-02 Proactive Sync](grasp-02-proactive-sync.md) - Future work design 490- [GRASP-02 Proactive Sync v4 Design](grasp-02-proactive-sync-v4.md) - Current production sync implementation
441- [Test Strategy](../reference/test-strategy.md) - Comprehensive testing documentation 491- [Test Strategy](../reference/test-strategy.md) - Comprehensive testing documentation
442- [GRASP-01 Implementation Learnings](../learnings/grasp-01-implementation.md) - Patterns and lessons learned \ No newline at end of file 492- [GRASP-01 Implementation Learnings](../learnings/grasp-01-implementation.md) - Patterns and lessons learned