diff options
Diffstat (limited to 'docs/explanation')
| -rw-r--r-- | docs/explanation/grasp-02-proactive-sync.md | 91 |
1 files changed, 71 insertions, 20 deletions
diff --git a/docs/explanation/grasp-02-proactive-sync.md b/docs/explanation/grasp-02-proactive-sync.md index 666b048..c07f07c 100644 --- a/docs/explanation/grasp-02-proactive-sync.md +++ b/docs/explanation/grasp-02-proactive-sync.md | |||
| @@ -18,7 +18,11 @@ This document covers **event syncing only**. Git data syncing is out of scope fo | |||
| 18 | ```mermaid | 18 | ```mermaid |
| 19 | flowchart TB | 19 | flowchart TB |
| 20 | subgraph ngit-grasp | 20 | subgraph ngit-grasp |
| 21 | SM[SyncManager] | 21 | subgraph SyncManager |
| 22 | SS[Self-Subscriber] | ||
| 23 | RC[Remote Connections] | ||
| 24 | end | ||
| 25 | WS[WebSocket Server] | ||
| 22 | FS[FilterService] | 26 | FS[FilterService] |
| 23 | RH[RelayHealthTracker] | 27 | RH[RelayHealthTracker] |
| 24 | DB[(Database)] | 28 | DB[(Database)] |
| @@ -32,41 +36,88 @@ flowchart TB | |||
| 32 | R3[nostr.land] | 36 | R3[nostr.land] |
| 33 | end | 37 | end |
| 34 | 38 | ||
| 35 | SM -->|builds filters| FS | 39 | WS -->|broadcasts events| SS |
| 36 | SM -->|tracks health| RH | 40 | SS -->|discovers relays| RC |
| 37 | SM -->|stores events| DB | 41 | RC -->|builds filters| FS |
| 38 | SM -->|validates| AP | 42 | RC -->|tracks health| RH |
| 43 | RC -->|stores events| DB | ||
| 44 | RC -->|validates| AP | ||
| 39 | 45 | ||
| 40 | SM <-->|WebSocket + NEG| R1 | 46 | RC <-->|WebSocket + NEG| R1 |
| 41 | SM <-->|WebSocket + NEG| R2 | 47 | RC <-->|WebSocket + NEG| R2 |
| 42 | SM <-->|WebSocket + NEG| R3 | 48 | RC <-->|WebSocket + NEG| R3 |
| 43 | 49 | ||
| 44 | RH -->|exposes state| MET | 50 | RH -->|exposes state| MET |
| 45 | ``` | 51 | ``` |
| 46 | 52 | ||
| 53 | **Key Insight: Self-Subscribe Architecture** | ||
| 54 | |||
| 55 | The SyncManager uses a "self-subscribe" pattern for relay discovery. Rather than polling the database periodically, it connects to its own WebSocket server as a client and subscribes to kind 30617 events. When new announcements are saved (from any source), the self-subscriber receives them instantly and can spawn connections to newly discovered relays. | ||
| 56 | |||
| 47 | ## Connection Management | 57 | ## Connection Management |
| 48 | 58 | ||
| 49 | ### Relay Discovery | 59 | ### Relay Discovery |
| 50 | 60 | ||
| 51 | Relays to connect to are discovered from **all stored repository announcements**: | 61 | Relays to connect to are discovered using a **self-subscribe architecture** rather than periodic polling. The SyncManager connects to its own relay as a client and subscribes to kind 30617 (repository announcement) events. When a new announcement is saved to the database (from direct submission or sync), the self-subscriber receives it immediately and discovers new relays to connect to. |
| 62 | |||
| 63 | ```mermaid | ||
| 64 | flowchart LR | ||
| 65 | subgraph Relay | ||
| 66 | WS[WebSocket Server] | ||
| 67 | DB[(Database)] | ||
| 68 | end | ||
| 69 | |||
| 70 | subgraph SyncManager | ||
| 71 | SS[Self-Subscribe Client] | ||
| 72 | RC[Remote Connections] | ||
| 73 | end | ||
| 74 | |||
| 75 | WS -->|broadcast| SS | ||
| 76 | SS -->|extract relay URLs| RC | ||
| 77 | RC -->|sync events| WS | ||
| 78 | ``` | ||
| 79 | |||
| 80 | **Why Self-Subscribe vs Polling?** | ||
| 81 | |||
| 82 | | Approach | Latency | Complexity | Resource Use | | ||
| 83 | |----------|---------|------------|--------------| | ||
| 84 | | Self-Subscribe | Instant | Low | Minimal (1 WS connection) | | ||
| 85 | | Periodic Polling | 30s+ delay | Higher | DB queries every N seconds | | ||
| 86 | |||
| 87 | The self-subscribe approach provides: | ||
| 88 | - **Immediate discovery**: New relays discovered instantly when announcement saved | ||
| 89 | - **No polling overhead**: No periodic database queries | ||
| 90 | - **Simple architecture**: Reuses existing WebSocket infrastructure | ||
| 91 | |||
| 92 | **Implementation Pattern:** | ||
| 52 | 93 | ||
| 53 | ```rust | 94 | ```rust |
| 54 | // Pseudocode for relay discovery | 95 | // In SyncManager::run() |
| 55 | fn discover_relays(database: &Database) -> HashSet<RelayUrl> { | 96 | let self_client = Client::default(); |
| 56 | let announcements = database.query(Filter::new().kind(30617)); | 97 | self_client.add_relay(&own_relay_url).await?; |
| 57 | let mut relays = HashSet::new(); | 98 | self_client.connect().await; |
| 58 | 99 | ||
| 59 | for announcement in announcements { | 100 | let filter = Filter::new().kind(Kind::Custom(30617)); |
| 60 | for relay_url in announcement.relays_tags() { | 101 | self_client.subscribe(filter, None).await?; |
| 61 | if relay_url != our_domain { // Exclude ourselves | 102 | |
| 62 | relays.insert(relay_url); | 103 | // Handle notifications - when announcement arrives, extract relay URLs |
| 104 | client.handle_notifications(|notification| async { | ||
| 105 | if let RelayPoolNotification::Event { event, .. } = notification { | ||
| 106 | let new_urls = filter_service.extract_relay_urls_from_event(&event); | ||
| 107 | for url in new_urls { | ||
| 108 | if !active_relays.contains(&url) && !is_own_relay(&url) { | ||
| 109 | spawn_connection(url, tx.clone(), filter_service.clone()); | ||
| 63 | } | 110 | } |
| 64 | } | 111 | } |
| 65 | } | 112 | } |
| 66 | relays | 113 | Ok(false) // Continue processing |
| 67 | } | 114 | }); |
| 68 | ``` | 115 | ``` |
| 69 | 116 | ||
| 117 | **Startup Discovery:** At startup, existing announcements in the database are queried once to discover initial relays. After startup, all discovery is event-driven via self-subscribe. | ||
| 118 | |||
| 119 | **Reconnection:** The self-subscriber has built-in exponential backoff reconnection (1s → 60s max) to handle temporary disconnections from our own relay. | ||
| 120 | |||
| 70 | ### Connection Lifecycle | 121 | ### Connection Lifecycle |
| 71 | 122 | ||
| 72 | ```mermaid | 123 | ```mermaid |