upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2025-12-11 16:20:23 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2025-12-11 16:20:23 +0000
commit6d0447f31eb9f9282e60ac3c90c665a8b3781331 (patch)
tree52a15001bda47c1096f82eb0598c8320df0b637c /docs
parent497df415749039236126140193af0ea612358cc7 (diff)
feat: implement NIP-77 negentropy sync for historical data
Replace EOSE-based sync completion with negentropy reconciliation for: - Initial connect (fresh sync) - Daily sync (Layer 1 announcements) - Stale reconnect (>15 min) Key changes: - Add NegentropySyncResult struct with remote_only, local_only, received fields - Add supports_negentropy() using try-and-fallback approach - Add negentropy_sync_filter() using nostr-sdk client.sync() API - Modify handle_connect_or_reconnect() to use negentropy for fresh/stale sync - Modify daily_sync() to use negentropy for Layer 1 - Single-warning logging per relay when negentropy fails Quick reconnects (<15 min) unchanged - still use REQ with since filter. If negentropy unsupported, gracefully falls back to REQ+EOSE flow.
Diffstat (limited to 'docs')
-rw-r--r--docs/explanation/grasp-02-proactive-sync.md74
1 files changed, 74 insertions, 0 deletions
diff --git a/docs/explanation/grasp-02-proactive-sync.md b/docs/explanation/grasp-02-proactive-sync.md
index d6665cc..f11148e 100644
--- a/docs/explanation/grasp-02-proactive-sync.md
+++ b/docs/explanation/grasp-02-proactive-sync.md
@@ -9,6 +9,7 @@ This document explains the proactive sync system that synchronizes repository da
93. **Two subscription paths on reconnect** - Catch-up (retained, with since) vs new items (via compute_actions) 93. **Two subscription paths on reconnect** - Catch-up (retained, with since) vs new items (via compute_actions)
104. **Blank state = fresh sync** - Empty confirmed state triggers full historical fetch 104. **Blank state = fresh sync** - Empty confirmed state triggers full historical fetch
115. **Clear on disconnect, not reconnect** - PendingSyncIndex cleared at event boundary 115. **Clear on disconnect, not reconnect** - PendingSyncIndex cleared at event boundary
126. **NIP-77 negentropy for historical sync** - Efficient set reconciliation, fallback to REQ if unsupported
12 13
13--- 14---
14 15
@@ -487,6 +488,79 @@ flowchart TB
487| Clear on disconnect | Clear PSI on disconnect | Cleanup at event boundary, simpler than on reconnect | 488| Clear on disconnect | Clear PSI on disconnect | Cleanup at event boundary, simpler than on reconnect |
488| 15-minute rule | Clear confirmed if disconnected >15min | Matches since filter buffer, prevents stale subscriptions | 489| 15-minute rule | Clear confirmed if disconnected >15min | Matches since filter buffer, prevents stale subscriptions |
489| Daily timer | Fresh sync (clears state) | Ensures consistency, detects drift | 490| Daily timer | Fresh sync (clears state) | Ensures consistency, detects drift |
491| NIP-77 negentropy | Try first, fallback to REQ | Efficient set reconciliation when supported |
492
493---
494
495## NIP-77 Negentropy Sync
496
497The sync system supports NIP-77 negentropy for efficient set reconciliation when syncing with external relays.
498
499### What is Negentropy?
500
501NIP-77 defines the negentropy protocol for efficient event set comparison. Instead of requesting all events matching a filter (REQ+EOSE), negentropy allows relays to compare fingerprints of their event sets and only transfer the differences.
502
503### When Negentropy is Used
504
505Negentropy sync is attempted for:
506- **Initial connect** - Fresh sync without `last_connected`
507- **Daily sync** - Periodic full refresh (23-25 hour timer)
508- **Stale reconnect** - Disconnected for more than 15 minutes
509
510Negentropy is NOT used for:
511- **Quick reconnect** - Less than 15 minutes disconnected (uses REQ with `since`)
512- **Live subscriptions** - Ongoing event streams always use REQ
513
514### Implementation
515
516The [`RelayConnection`](../../src/sync/relay_connection.rs:71) now includes NIP-77 methods:
517
518```rust
519/// Check if negentropy sync should be attempted
520pub async fn supports_negentropy(&self) -> bool {
521 // Always returns true - we try negentropy and handle failure gracefully
522 true
523}
524
525/// Perform negentropy synchronization for a filter
526pub async fn negentropy_sync_filter(&self, filter: Filter)
527 -> Result<NegentropySyncResult, String> {
528 // Uses nostr-sdk's client.sync() method
529}
530```
531
532### Sync Flow with Negentropy
533
534```mermaid
535flowchart TB
536 CONNECT[Connect to relay] --> NEG{Try negentropy}
537 NEG --> |success| L1[Layer 1 synced via negentropy]
538 NEG --> |failure| FALLBACK[Fall back to REQ+EOSE]
539
540 L1 --> SINCE[Record timestamp = now]
541 FALLBACK --> EOSE[Wait for EOSE]
542 EOSE --> SINCE
543
544 SINCE --> LIVE[Open live REQ with since=now]
545```
546
547### Fallback Behavior
548
549If negentropy fails (relay doesn't support NIP-77, network error, etc.):
5501. A warning is logged (once per relay to avoid spam)
5512. The sync falls back to traditional REQ+EOSE
5523. No error is raised - fallback is automatic
553
554**Implementation:** [`negentropy_sync_and_process()`](../../src/sync/mod.rs:1549)
555
556### Key Design Decisions for Negentropy
557
558| Decision | Choice | Rationale |
559|----------|--------|-----------|
560| Detection approach | Try and fallback | More reliable than NIP-11 document detection |
561| When to use | Fresh/daily/stale sync only | Quick reconnect with `since` is already efficient |
562| Error handling | Log once, fallback silently | Avoid log spam while maintaining visibility |
563| Layer application | Layer 1 first | Announcements are highest priority |
490 564
491--- 565---
492 566