docs: simplify grasp-02 doc

1 files changed, 124 insertions, 924 deletions

diff --git a/docs/explanation/grasp-02-proactive-sync.md b/docs/explanation/grasp-02-proactive-sync.md
index dd508b3..d6665cc 100644
--- a/docs/explanation/grasp-02-proactive-sync.md
+++ b/docs/explanation/grasp-02-proactive-sync.md
@@ -1,8 +1,8 @@
-# GRASP-02: Proactive Sync v4 - Health & Reconnection Design
+# GRASP-02: Proactive Sync - Design & Implementation
 
 ## Overview
 
-This document presents v4 of the proactive sync design, refining the connection lifecycle and reconnection patterns. Key principles:
+This document explains the proactive sync system that synchronizes repository data from external relays based on relay URLs listed in 30617 repository announcements. Key principles:
 
 1. **Self-subscription as the only mechanism** - No database initialization at startup
 2. **compute_actions as single decision point** - Determines what NEW subscriptions to create
@@ -72,7 +72,7 @@ impl RelayState {
         match self.disconnected_at {
             Some(disconnected) => {
                 let now = Timestamp::now();
-                now.as_u64().saturating_sub(disconnected.as_u64()) > 900 // 15 minutes
+                now.as_secs().saturating_sub(disconnected.as_secs()) > 900 // 15 minutes
             }
             None => false, // Still connected or never connected
         }
@@ -132,7 +132,7 @@ stateDiagram-v2
 
 ## Flow Scenarios
 
-### Scenario 1: Initial Connect via handle_connect_or_reconnect
+### Scenario 1: Initial Connect
 
 ```mermaid
 flowchart TB
@@ -166,7 +166,7 @@ flowchart TB
 - Layer 1: `build_announcement_filter(None)` - subscribed immediately without since
 - Layer 2+3: handled via `recompute_actions_for_relay` → `compute_actions` with PendingBatch tracking
 
-### Scenario 2: Quick Reconnect via handle_connect_or_reconnect - less than 15 minutes
+### Scenario 2: Quick Reconnect (less than 15 minutes)
 
 ```mermaid
 flowchart TB
@@ -178,7 +178,7 @@ flowchart TB
 
     subgraph handle_connect_or_reconnect - Quick Reconnect
         HC --> CHECK{is_fresh_sync?}
-        CHECK --> |no - last_connected exists AND <15min| SINCE[since = last_connected - 15min]
+        CHECK --> |no - last_connected exists AND less than 15min| SINCE[since = last_connected - 15min]
         SINCE --> L1[build_announcement_filter - with since]
         L1 --> L23[rebuild_layer2_and_layer3 - with since]
         L23 --> RCA[recompute_actions_for_relay]
@@ -200,7 +200,7 @@ flowchart TB
   3. `recompute_actions_for_relay` - check for new items
 - since = last_connected - 15min ensures we catch events during disconnection
 
-### Scenario 3: Stale Reconnect via handle_connect_or_reconnect - greater than 15 minutes
+### Scenario 3: Stale Reconnect (greater than 15 minutes)
 
 ```mermaid
 flowchart TB
@@ -208,7 +208,7 @@ flowchart TB
 
     subgraph handle_connect_or_reconnect - Stale Reconnect
         HC --> CHECK{is_fresh_sync?}
-        CHECK --> |yes - disconnected >15min| CLEAR[clear_sync_state]
+        CHECK --> |yes - disconnected greater than 15min| CLEAR[clear_sync_state]
         CLEAR --> L1[build_announcement_filter - no since]
         L1 --> RCA[recompute_actions_for_relay]
     end
@@ -228,7 +228,7 @@ flowchart TB
 - Layer 1: `build_announcement_filter(None)` - full history
 - Layer 2+3: handled via empty confirmed state → compute_actions generates AddFilters for everything
 
-### Scenario 4: Consolidation - Triggered on Filter Add
+### Scenario 4: Consolidation (Triggered on Filter Add)
 
 ```mermaid
 flowchart TB
@@ -252,7 +252,7 @@ flowchart TB
 - `since = now - 15min` prevents re-fetching old events
 - Keeps confirmed state, just reduces filter count
 
-### Scenario 5: Daily Timer - 23 to 25h Random
+### Scenario 5: Daily Timer (23-25h Random)
 
 ```mermaid
 flowchart TB
@@ -281,635 +281,127 @@ flowchart TB
 
 ## Core Algorithms
 
-### 1. derive_relay_targets
+### derive_relay_targets
 
-Transform RepoSyncIndex into per-relay sync targets:
+Transforms the repo-centric `RepoSyncIndex` into a relay-centric view. For each relay URL mentioned in any repo's announcements, collects all the repos and root events that should be synced from that relay.
 
-```rust
-/// Inverts RepoSyncIndex to get per-relay view
-fn derive_relay_targets(
-    repo_index: &HashMap<String, RepoSyncNeeds>
-) -> HashMap<String, RelaySyncNeeds> {
-    let mut targets: HashMap<String, RelaySyncNeeds> = HashMap::new();
-
-    for (repo_ref, needs) in repo_index {
-        for relay_url in &needs.relays {
-            let target = targets.entry(relay_url.clone()).or_default();
-            target.repos.insert(repo_ref.clone());
-            target.root_events.extend(needs.root_events.iter().cloned());
-        }
-    }
-
-    targets
-}
-```
-
-### 2. compute_actions (Three-Way Diff)
-
-**This is the ONLY decision point for what NEW subscriptions to create.**
-
-```rust
-/// Computes AddFilters for items that are:
-/// - In targets (what we want)
-/// - NOT in pending (already in-flight)
-/// - NOT in confirmed (already confirmed)
-fn compute_actions(
-    targets: &HashMap<String, RelaySyncNeeds>,
-    pending: &HashMap<String, Vec<PendingBatch>>,
-    confirmed: &HashMap<String, RelayState>,
-) -> Vec<AddFilters> {
-    let mut actions = Vec::new();
-
-    for (relay_url, target) in targets {
-        // Skip disconnected relays - they will get AddFilters on reconnect
-        if let Some(state) = confirmed.get(relay_url) {
-            if state.connection_status != ConnectionStatus::Connected {
-                continue;
-            }
-        }
-
-        // Collect all pending items for this relay
-        let pending_repos: HashSet<_> = pending.get(relay_url)
-            .map(|batches| batches.iter()
-                .flat_map(|b| b.items.repos.iter().cloned())
-                .collect())
-            .unwrap_or_default();
-        let pending_events: HashSet<_> = pending.get(relay_url)
-            .map(|batches| batches.iter()
-                .flat_map(|b| b.items.root_events.iter().cloned())
-                .collect())
-            .unwrap_or_default();
-
-        // Collect confirmed items for this relay
-        let confirmed_repos = confirmed.get(relay_url)
-            .map(|c| &c.repos)
-            .unwrap_or(&HashSet::new());
-        let confirmed_events = confirmed.get(relay_url)
-            .map(|c| &c.root_events)
-            .unwrap_or(&HashSet::new());
-
-        // New = target - pending - confirmed
-        let new_repos: HashSet<_> = target.repos.iter()
-            .filter(|r| !pending_repos.contains(*r) && !confirmed_repos.contains(*r))
-            .cloned()
-            .collect();
-        let new_events: HashSet<_> = target.root_events.iter()
-            .filter(|e| !pending_events.contains(*e) && !confirmed_events.contains(*e))
-            .cloned()
-            .collect();
-
-        if !new_repos.is_empty() || !new_events.is_empty() {
-            let filters = build_filters(&new_repos, &new_events);
-            actions.push(AddFilters {
-                relay_url: relay_url.clone(),
-                repos: new_repos,
-                root_events: new_events,
-                filters,
-            });
-        }
-    }
-
-    actions
-}
-```
-
-### 3. Filter Building Functions (Three-Layer Strategy)
-
-The filter strategy uses three layers:
-
-- **Layer 1**: Announcements (30617/30618) - subscribed ONCE on connect, NOT rebuilt during consolidation
-- **Layer 2**: Events tagging our repos
-- **Layer 3**: Events tagging our root events
-
-**Key insight**: Layer 1 is connection-level (subscribe once), Layer 2+3 are item-level (managed by compute_actions and PendingBatch).
-
-```rust
-/// Layer 1: Announcements filter (kinds 30617 + 30618)
-/// Subscribed ONCE on connect - NOT included in consolidation rebuilds.
-/// Note: 30618 is ONLY synced from remote relays, not self-subscribed.
-fn build_announcement_filter(since: Option<Timestamp>) -> Filter {
-    let filter = Filter::new().kinds([
-        Kind::Custom(30617),  // Repository announcements
-        Kind::Custom(30618),  // Maintainer lists
-    ]);
-
-    match since {
-        Some(ts) => filter.since(ts),
-        None => filter,
-    }
-}
-
-/// Layer 2: Events tagging one of our repos
-/// Uses lowercase a, uppercase A, and q tags for comprehensive coverage.
-/// Batched per 100 repo refs.
-fn tagged_one_of_our_repo_event_filters(
-    repos: &HashSet<String>,
-    since: Option<Timestamp>,
-) -> Vec<Filter> {
-    let mut filters = Vec::new();
-    let repo_refs: Vec<_> = repos.iter().collect();
-
-    for chunk in repo_refs.chunks(100) {
-        let chunk_vec: Vec<&str> = chunk.iter().map(|s| s.as_str()).collect();
-
-        // Lowercase 'a' tag - standard addressable reference
-        let mut f1 = Filter::new()
-            .custom_tag(SingleLetterTag::lowercase(Alphabet::A), chunk_vec.clone());
-        // Uppercase 'A' tag - some clients use this
-        let mut f2 = Filter::new()
-            .custom_tag(SingleLetterTag::uppercase(Alphabet::A), chunk_vec.clone());
-        // Quote 'q' tag - NIP-10 quote references to addressable events
-        let mut f3 = Filter::new()
-            .custom_tag(SingleLetterTag::lowercase(Alphabet::Q), chunk_vec);
-
-        if let Some(ts) = since {
-            f1 = f1.since(ts);
-            f2 = f2.since(ts);
-            f3 = f3.since(ts);
-        }
-
-        filters.push(f1);
-        filters.push(f2);
-        filters.push(f3);
-    }
-
-    filters
-}
-
-/// Layer 3: Events tagging one of our root events
-/// Uses lowercase e, uppercase E, and q tags for comprehensive coverage.
-/// Batched per 100 event IDs.
-fn tagged_one_of_our_root_event_filters(
-    root_events: &HashSet<EventId>,
-    since: Option<Timestamp>,
-) -> Vec<Filter> {
-    let mut filters = Vec::new();
-    let event_ids: Vec<String> = root_events.iter().map(|id| id.to_hex()).collect();
-
-    for chunk in event_ids.chunks(100) {
-        let chunk_vec: Vec<&str> = chunk.iter().map(|s| s.as_str()).collect();
-
-        // Lowercase 'e' tag - standard event reference
-        let mut f1 = Filter::new()
-            .custom_tag(SingleLetterTag::lowercase(Alphabet::E), chunk_vec.clone());
-        // Uppercase 'E' tag - some clients use this
-        let mut f2 = Filter::new()
-            .custom_tag(SingleLetterTag::uppercase(Alphabet::E), chunk_vec.clone());
-        // Quote 'q' tag - NIP-10 quote references to events
-        let mut f3 = Filter::new()
-            .custom_tag(SingleLetterTag::lowercase(Alphabet::Q), chunk_vec);
-
-        if let Some(ts) = since {
-            f1 = f1.since(ts);
-            f2 = f2.since(ts);
-            f3 = f3.since(ts);
-        }
-
-        filters.push(f1);
-        filters.push(f2);
-        filters.push(f3);
-    }
-
-    filters
-}
-
-/// Builds Layer 2 + Layer 3 filters only (NOT Layer 1)
-/// Used by:
-/// - compute_actions for incremental subscriptions
-/// - consolidation rebuilds (Layer 1 remains active)
-fn build_layer2_and_layer3_filters(
-    repos: &HashSet<String>,
-    root_events: &HashSet<EventId>,
-    since: Option<Timestamp>,
-) -> Vec<Filter> {
-    let mut filters = Vec::new();
-    filters.extend(tagged_one_of_our_repo_event_filters(repos, since));
-    filters.extend(tagged_one_of_our_root_event_filters(root_events, since));
-    filters
-}
-```
-
-**Note**: There is no `build_all_filters` function. Layer 1 is subscribed separately on connect, and Layer 2+3 are managed independently.
-
-### 4. handle_add_filters (SyncManager)
-
-```rust
-impl SyncManager {
-    async fn handle_add_filters(&mut self, action: AddFilters) {
-        let AddFilters { relay_url, repos, root_events, filters } = action;
-
-        // Auto-spawn connection if needed
-        let state = self.relay_sync_index.read().await.get(&relay_url).cloned();
-        match state {
-            None => {
-                // New relay discovered - create entry and spawn connection
-                self.relay_sync_index.write().await.insert(
-                    relay_url.clone(),
-                    RelayState {
-                        repos: HashSet::new(),
-                        root_events: HashSet::new(),
-                        is_bootstrap: false,
-                        connection_status: ConnectionStatus::Connecting,
-                        last_connected: None,
-                        disconnected_at: None,
-                        connection: None,
-                    }
-                );
-                self.spawn_connection(&relay_url).await;
-                return; // Subscriptions will happen on connection success
-            }
-            Some(state) if state.connection_status != ConnectionStatus::Connected => {
-                // Not connected - subscriptions will happen on connection success
-                return;
-            }
-            Some(_) => {
-                // Already connected - proceed with subscription
-            }
-        }
-
-        // Subscribe and collect subscription IDs
-        let conn = self.connections.get(&relay_url).unwrap();
-        let mut sub_ids = HashSet::new();
-
-        for filter in filters {
-            match conn.client.subscribe(filter, None).await {
-                Ok(output) => {
-                    for sub_id in output.val {
-                        sub_ids.insert(sub_id);
-                    }
-                }
-                Err(e) => {
-                    tracing::warn!("Failed to subscribe: {}", e);
-                }
-            }
-        }
-
-        // Create pending batch
-        let batch = PendingBatch {
-            batch_id: self.next_batch_id(),
-            items: PendingItems { repos, root_events },
-            outstanding_subs: sub_ids,
-        };
-
-        // Add to pending index
-        self.pending_sync_index.write().await
-            .entry(relay_url)
-            .or_default()
-            .push(batch);
-    }
-}
-```
-
-### 5. handle_disconnect
+**Implementation:** [`derive_relay_targets()`](../../src/sync/algorithms.rs:61)
 
 ```rust
-impl SyncManager {
-    /// Called when connection to a relay is lost
-    async fn handle_disconnect(&mut self, relay_url: &str) {
-        let mut index = self.relay_sync_index.write().await;
-
-        if let Some(state) = index.get_mut(relay_url) {
-            state.connection_status = ConnectionStatus::Disconnected;
-            state.disconnected_at = Some(Timestamp::now());
-            state.connection = None;
-        }
-
-        // Clear pending batches - these items were not confirmed
-        self.pending_sync_index.write().await.remove(relay_url);
-
-        // Remove from active connections map
-        self.connections.remove(relay_url);
-
-        // Health tracker records failure for backoff
-        self.health_tracker.record_failure(relay_url);
-    }
-}
+// Conceptual: inverts repo → relays to relay → repos
+fn derive_relay_targets(repo_index: &HashMap<String, RepoSyncNeeds>) 
+    -> HashMap<String, RelaySyncNeeds>
 ```
 
-### 6. handle_connect_or_reconnect (Unified)
-
-This method handles BOTH initial connection AND reconnection with unified logic:
-
-```rust
-impl SyncManager {
-    /// Called when connection to a relay succeeds - handles both initial connect and reconnect.
-    ///
-    /// Decision tree:
-    /// - Fresh sync (no last_connected OR disconnected >15min): No since filter, full history
-    /// - Quick reconnect (<15min): since = last_connected - 15min
-    async fn handle_connect_or_reconnect(&mut self, relay_url: &str) {
-        let mut index = self.relay_sync_index.write().await;
-        let state = match index.get_mut(relay_url) {
-            Some(s) => s,
-            None => return, // Relay was removed while disconnected
-        };
-
-        // Determine if this is a fresh sync or quick reconnect
-        let is_fresh_sync = state.last_connected.is_none() || state.should_clear_state();
-        let last_connected = state.last_connected;
-
-        if is_fresh_sync && state.last_connected.is_some() {
-            // Stale reconnect (>15min) - clear state
-            tracing::info!("Reconnect after >15min for {}, clearing state for fresh sync", relay_url);
-            state.clear_sync_state();
-        }
-
-        // Update connection state
-        state.connection_status = ConnectionStatus::Connected;
-        state.last_connected = Some(Timestamp::now());
-        state.disconnected_at = None;
-
-        // Record success in health tracker
-        self.health_tracker.record_success(relay_url);
+### compute_actions (Three-Way Diff)
 
-        drop(index); // Release lock
-
-        let conn = match self.connections.get(relay_url) {
-            Some(c) => c,
-            None => return,
-        };
-
-        if is_fresh_sync {
-            // Fresh sync: Layer 1 without since, Layer 2+3 handled by compute_actions
-
-            // Step 1: Subscribe Layer 1 (announcements) without since
-            let layer1 = build_announcement_filter(None);
-            let _ = conn.client.subscribe(layer1, None).await;
+**This is the ONLY decision point for what NEW subscriptions to create.**
 
-            // Step 2: compute_actions will handle Layer 2+3 (with since=None in build)
-            self.recompute_actions_for_relay(relay_url).await;
-        } else {
-            // Quick reconnect: Layer 1 with since, Layer 2+3 with since
-            let since = last_connected
-                .map(|ts| Timestamp::from(ts.as_u64().saturating_sub(900)))
-                .unwrap_or(Timestamp::from(0));
+Performs a three-way diff: `target - pending - confirmed = new`
 
-            // Step 1: Subscribe Layer 1 (announcements) with since
-            let layer1 = build_announcement_filter(Some(since));
-            let _ = conn.client.subscribe(layer1, None).await;
+- **targets**: What we want (from derive_relay_targets)
+- **pending**: What's already in-flight awaiting EOSE
+- **confirmed**: What's already confirmed syncing
 
-            // Step 2: Rebuild Layer 2+3 for confirmed items with since
-            self.rebuild_layer2_and_layer3(relay_url, Some(since)).await;
+Only creates `AddFilters` actions for items not already pending or confirmed. Skips disconnected relays (they will get AddFilters on reconnect).
 
-            // Step 3: Check for NEW items via compute_actions
-            self.recompute_actions_for_relay(relay_url).await;
-        }
-    }
+**Implementation:** [`compute_actions()`](../../src/sync/algorithms.rs:96)
 
-    /// Rebuild Layer 2+3 subscriptions only (NOT Layer 1).
-    /// Used by:
-    /// - Quick reconnect: rebuild confirmed items with since filter
-    /// - Consolidation: close and rebuild with since filter
-    async fn rebuild_layer2_and_layer3(&mut self, relay_url: &str, since: Option<Timestamp>) {
-        let confirmed = self.relay_sync_index.read().await;
-        let state = match confirmed.get(relay_url) {
-            Some(s) => s,
-            None => return,
-        };
-
-        // Build Layer 2+3 filters WITH since
-        let filters = build_layer2_and_layer3_filters(&state.repos, &state.root_events, since);
-        drop(confirmed);
-
-        // Subscribe directly - no PendingBatch for catch-up (items already confirmed)
-        let conn = match self.connections.get(relay_url) {
-            Some(c) => c,
-            None => return,
-        };
-
-        for filter in filters {
-            let _ = conn.client.subscribe(filter, None).await;
-        }
-    }
+---
 
-    /// Rerun compute_actions for a specific relay and process resulting AddFilters.
-    /// compute_actions builds Layer 2+3 filters for NEW items not yet in confirmed state.
-    async fn recompute_actions_for_relay(&mut self, relay_url: &str) {
-        let repo_index = self.repo_sync_index.read().await;
-        let targets = derive_relay_targets(&repo_index);
-        drop(repo_index);
+## Filter Building (Three-Layer Strategy)
 
-        // Filter to just this relay
-        let target = match targets.get(relay_url) {
-            Some(t) => t.clone(),
-            None => return, // No repos reference this relay anymore
-        };
+The filter strategy uses three layers to ensure comprehensive event coverage:
 
-        let pending = self.pending_sync_index.read().await;
-        let confirmed = self.relay_sync_index.read().await;
+### Layer 1: Announcements
 
-        let mut single_relay_targets = HashMap::new();
-        single_relay_targets.insert(relay_url.to_string(), target);
+- **Kinds**: 30617 (Repository Announcements), 30618 (Maintainer Lists)
+- **When subscribed**: ONCE on connect, NOT rebuilt during consolidation
+- **Function**: [`build_announcement_filter()`](../../src/sync/filters.rs:20)
+- 30618 is ONLY synced from remote relays, not self-subscribed
 
-        let actions = compute_actions(&single_relay_targets, &pending, &confirmed);
+### Layer 2: Events Tagging Our Repos
 
-        drop(pending);
-        drop(confirmed);
+- **Tags**: lowercase `a`, uppercase `A`, and `q` tags for comprehensive coverage
+- **Batching**: Per 100 repo refs
+- **Function**: [`tagged_one_of_our_repo_event_filters()`](../../src/sync/filters.rs:43)
 
-        // Process AddFilters
-        for action in actions {
-            self.handle_add_filters(action).await;
-        }
-    }
-}
-```
+### Layer 3: Events Tagging Our Root Events
 
-### 7. Daily Timer
+- **Tags**: lowercase `e`, uppercase `E`, and `q` tags for comprehensive coverage
+- **Batching**: Per 100 event IDs
+- **Function**: [`tagged_one_of_our_root_event_filters()`](../../src/sync/filters.rs:98)
 
-```rust
-impl SyncManager {
-    async fn run_daily_timer(&self) {
-        loop {
-            // Random 23-25 hours
-            let hours = 23.0 + rand::random::<f64>() * 2.0;
-            tokio::time::sleep(Duration::from_secs_f64(hours * 3600.0)).await;
-
-            let relay_urls: Vec<_> = self.relay_sync_index.read().await
-                .keys()
-                .cloned()
-                .collect();
-
-            for relay_url in relay_urls {
-                self.daily_sync(&relay_url).await;
-            }
-        }
-    }
-
-    /// Perform daily fresh sync for a relay
-    async fn daily_sync(&mut self, relay_url: &str) {
-        tracing::info!("Daily sync triggered for {}", relay_url);
+### Combined Layer 2+3
 
-        // Close all subscriptions
-        if let Some(conn) = self.connections.get(relay_url) {
-            conn.client.unsubscribe_all().await;
-        }
+The [`build_layer2_and_layer3_filters()`](../../src/sync/filters.rs:152) function combines both layers. Used by:
+- `compute_actions` for incremental subscriptions
+- `rebuild_layer2_and_layer3` during reconnection
+- Consolidation rebuilds (Layer 1 remains active separately)
 
-        // Clear PendingSyncIndex
-        self.pending_sync_index.write().await.remove(relay_url);
-
-        // Clear confirmed state - triggers fresh sync
-        {
-            let mut index = self.relay_sync_index.write().await;
-            if let Some(state) = index.get_mut(relay_url) {
-                state.clear_sync_state();
-            }
-        }
+**Key insight**: Layer 1 is connection-level (subscribe once), Layer 2+3 are item-level (managed by compute_actions and PendingBatch).
 
-        // Recompute actions - will generate AddFilters for everything
-        self.recompute_actions_for_relay(relay_url).await;
-    }
-}
-```
+---
 
-### 8. Consolidation (Threshold-Based, Triggered on Add)
+## SyncManager Key Methods
 
-Consolidation is checked when adding new subscriptions, not periodically. **Key insight**: Consolidation only closes and rebuilds Layer 2+3 - Layer 1 remains active.
+The [`SyncManager`](../../src/sync/mod.rs:308) orchestrates all sync operations. Key methods:
 
-```rust
-impl SyncManager {
-    /// Check filter count and consolidate if needed.
-    /// Called from handle_add_filters BEFORE adding new filters.
-    async fn maybe_consolidate(&mut self, relay_url: &str, new_filter_count: usize) {
-        let current_count = self.get_filter_count(relay_url).await;
-
-        if current_count + new_filter_count > 70 {
-            self.consolidate(relay_url).await;
-        }
-    }
+### Connection Lifecycle
 
-    /// Consolidate filters - only rebuilds Layer 2+3, Layer 1 stays active.
-    /// Does NOT clear state - just reduces filter count.
-    async fn consolidate(&mut self, relay_url: &str) {
-        tracing::info!("Consolidating filters for {} (count > 70)", relay_url);
+| Method | Purpose |
+|--------|---------|
+| `handle_connect_or_reconnect()` | Unified handler for initial connect and reconnect. Determines fresh vs quick reconnect based on `last_connected` and 15-minute rule |
+| `handle_disconnect()` | Updates RelayState to Disconnected, sets disconnected_at, clears pending batches, records failure in health tracker |
+| `spawn_relay_connection()` | Creates RelayConnection, subscribes to Layer 1, spawns event loop task |
 
-        // Wait for all pending batches to complete first
-        self.wait_pending_complete(relay_url).await;
+### Sync Operations
 
-        // Close Layer 2+3 subscriptions only - Layer 1 remains active
-        // NOTE: In practice, we close all then re-add Layer 1, or track sub IDs separately
-        // For simplicity, we close all and re-add Layer 1
-        if let Some(conn) = self.connections.get(relay_url) {
-            conn.client.unsubscribe_all().await;
-        }
+| Method | Purpose |
+|--------|---------|
+| `handle_add_filters()` | Auto-spawns connection if needed, checks consolidation threshold (>70 filters), subscribes and creates PendingBatch |
+| `handle_eose()` | Processes EOSE for subscription, moves items from pending to confirmed when batch completes |
+| `recompute_actions_for_relay()` | Runs derive_relay_targets → compute_actions for a specific relay to find new items |
+| `rebuild_layer2_and_layer3()` | Rebuilds subscriptions from confirmed state with optional since filter |
 
-        // Re-subscribe Layer 1 with since (maintains announcements stream)
-        let since = Timestamp::from(Timestamp::now().as_u64().saturating_sub(900));
-        let conn = self.connections.get(relay_url).unwrap();
-        let layer1 = build_announcement_filter(Some(since));
-        let _ = conn.client.subscribe(layer1, None).await;
+### Maintenance
 
-        // Rebuild Layer 2+3 only
-        self.rebuild_layer2_and_layer3(relay_url, Some(since)).await;
-    }
-}
-```
+| Method | Purpose |
+|--------|---------|
+| `daily_sync()` | Full fresh sync - unsubscribes all, clears state, recomputes actions |
+| `consolidate()` | Reduces filter count by unsubscribing and rebuilding with combined filters |
+| `check_disconnects()` | Periodic check for empty relays (no repos) to disconnect |
+| `check_reconnects()` | Attempts reconnection for disconnected relays with pending work |
 
-**Updated handle_add_filters to check consolidation:**
+---
 
-```rust
-impl SyncManager {
-    async fn handle_add_filters(&mut self, action: AddFilters) {
-        let AddFilters { relay_url, repos, root_events, filters } = action;
-
-        // Auto-spawn connection if needed (unchanged)
-        let state = self.relay_sync_index.read().await.get(&relay_url).cloned();
-        match state {
-            None => {
-                // New relay discovered - create entry and spawn connection
-                self.relay_sync_index.write().await.insert(
-                    relay_url.clone(),
-                    RelayState {
-                        repos: HashSet::new(),
-                        root_events: HashSet::new(),
-                        is_bootstrap: false,
-                        connection_status: ConnectionStatus::Connecting,
-                        last_connected: None,
-                        disconnected_at: None,
-                        connection: None,
-                    }
-                );
-                self.spawn_connection(&relay_url).await;
-                return; // Subscriptions will happen on connection success
-            }
-            Some(state) if state.connection_status != ConnectionStatus::Connected => {
-                return; // Not connected - subscriptions will happen on connection success
-            }
-            Some(_) => {
-                // Already connected - proceed
-            }
-        }
+## Self-Subscriber
 
-        // CHECK CONSOLIDATION BEFORE ADDING
-        self.maybe_consolidate(&relay_url, filters.len()).await;
-
-        // Subscribe and collect subscription IDs
-        let conn = self.connections.get(&relay_url).unwrap();
-        let mut sub_ids = HashSet::new();
-
-        for filter in filters {
-            match conn.client.subscribe(filter, None).await {
-                Ok(output) => {
-                    for sub_id in output.val {
-                        sub_ids.insert(sub_id);
-                    }
-                }
-                Err(e) => {
-                    tracing::warn!("Failed to subscribe: {}", e);
-                }
-            }
-        }
+The [`SelfSubscriber`](../../src/sync/self_subscriber.rs:86) monitors our own relay for repository announcements and root events, updating the `RepoSyncIndex`.
 
-        // Create pending batch (unchanged)
-        let batch = PendingBatch {
-            batch_id: self.next_batch_id(),
-            items: PendingItems { repos, root_events },
-            outstanding_subs: sub_ids,
-        };
-
-        self.pending_sync_index.write().await
-            .entry(relay_url)
-            .or_default()
-            .push(batch);
-    }
-}
-```
+### Event Kinds Monitored
 
----
+- **30617** - Repository Announcements (triggers discovery of repos listing our relay)
+- **1617** - Patches (root events referencing repos)
+- **1618** - Issues
+- **1619** - Replies/Status
+- **1621** - Pull Requests
 
-## Disconnect (Relay Removal) Handling
+Note: 30618 (Maintainer Lists) is NOT self-subscribed - only synced from remote relays.
 
-```rust
-impl SyncManager {
-    /// Periodically check for relays that should be disconnected
-    async fn check_disconnects(&mut self) {
-        let confirmed = self.relay_sync_index.read().await;
-        let relays_to_disconnect: Vec<_> = confirmed.iter()
-            .filter(|(_, state)| {
-                !state.is_bootstrap &&
-                state.repos.is_empty() &&
-                state.root_events.is_empty()
-            })
-            .map(|(url, _)| url.clone())
-            .collect();
-        drop(confirmed);
-
-        for relay_url in relays_to_disconnect {
-            self.disconnect_relay(&relay_url).await;
-        }
-    }
+### Batching Flow
 
-    async fn disconnect_relay(&mut self, relay_url: &str) {
-        tracing::info!("Disconnecting relay {} (no repos)", relay_url);
+1. **Receive events** from own relay subscription
+2. **Queue to pending** - announcements get repo ID + relay URLs; root events get repo ref + event ID
+3. **Timer fires** (configurable window, default 5 seconds) - does NOT reset on new events
+4. **Process batch**:
+   - Update `RepoSyncIndex` with discovered repos and root events
+   - Call `derive_relay_targets()` → `compute_actions()`
+   - Send `AddFilters` actions to SyncManager
 
-        self.relay_sync_index.write().await.remove(relay_url);
-        self.pending_sync_index.write().await.remove(relay_url);
+### Reconnection
 
-        if let Some(conn) = self.connections.remove(relay_url) {
-            let _ = conn.client.disconnect().await;
-        }
-    }
-}
-```
+Uses `last_connected` timestamp to apply since filter on reconnect (15-minute buffer), similar to external relay reconnection logic.
 
 ---
 
@@ -981,25 +473,20 @@ flowchart TB
 
 ## Key Design Decisions
 
-| Decision                   | Choice                                                                     | Rationale                                                                   |
-| -------------------------- | -------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
-| Startup mechanism          | Self-subscription only                                                     | Single code path, fresh DB behaves same as reconnect                        |
-| Connect/reconnect handling | Unified handle_connect_or_reconnect                                        | Single entry point for both initial and reconnect                           |
-| Layer 1 handling           | Separate build_announcement_filter                                         | Connection-level: subscribe ONCE on connect, NOT rebuilt in consolidation   |
-| Layer 2+3 handling         | Separate rebuild_layer2_and_layer3                                         | Item-level: managed by compute_actions, consolidated when filter count > 70 |
-| Filter functions           | since as Option parameter                                                  | Allows same functions for fresh sync and catch-up                           |
-| Layer 2+3 tags             | tagged_one_of_our_repo_event_filters, tagged_one_of_our_root_event_filters | Descriptive names, uses a/A/q for repos, e/E/q for events                   |
-| Since filter               | Only on catch-up paths                                                     | Initial/stale gets full history, quick reconnect catches up                 |
-| compute_actions role       | ONLY for new Layer 2+3 items                                               | Does NOT handle Layer 1 or catch-up                                         |
-| Catch-up pending tracking  | No PendingBatch                                                            | Items already confirmed, don't need re-confirmation                         |
-| Consolidation trigger      | On filter add, not periodic                                                | Check in handle_add_filters before adding new filters                       |
-| Consolidation Layer 1      | Re-subscribe with since after unsubscribe_all                              | Maintains announcement stream                                               |
-| Consolidation Layer 2+3    | rebuild_layer2_and_layer3 with since                                       | Shared logic with quick_reconnect                                           |
-| Clear on disconnect        | Clear PSI on disconnect                                                    | Cleanup at event boundary, simpler than on reconnect                        |
-| 15-minute rule             | Clear confirmed if disconnected >15min                                     | Matches since filter buffer, prevents stale subscriptions                   |
-| Daily timer                | Fresh sync (clears state)                                                  | Ensures consistency, detects drift                                          |
-| Connection spawning        | Via AddFilters handler                                                     | Single path for new relay discovery                                         |
-| Self-subscriber reconnect  | Use since-15min filter                                                     | Simpler than immediate RepoSyncIndex updates                                |
+| Decision | Choice | Rationale |
+|----------|--------|-----------|
+| Startup mechanism | Self-subscription only | Single code path, fresh DB behaves same as reconnect |
+| Connect/reconnect handling | Unified handle_connect_or_reconnect | Single entry point for both initial and reconnect |
+| Layer 1 handling | Separate build_announcement_filter | Connection-level: subscribe ONCE on connect, NOT rebuilt in consolidation |
+| Layer 2+3 handling | Separate rebuild_layer2_and_layer3 | Item-level: managed by compute_actions, consolidated when filter count > 70 |
+| Filter functions | since as Option parameter | Allows same functions for fresh sync and catch-up |
+| Since filter | Only on catch-up paths | Initial/stale gets full history, quick reconnect catches up |
+| compute_actions role | ONLY for new Layer 2+3 items | Does NOT handle Layer 1 or catch-up |
+| Catch-up pending tracking | No PendingBatch | Items already confirmed, don't need re-confirmation |
+| Consolidation trigger | On filter add, not periodic | Check in handle_add_filters before adding new filters |
+| Clear on disconnect | Clear PSI on disconnect | Cleanup at event boundary, simpler than on reconnect |
+| 15-minute rule | Clear confirmed if disconnected >15min | Matches since filter buffer, prevents stale subscriptions |
+| Daily timer | Fresh sync (clears state) | Ensures consistency, detects drift |
 
 ---
 
@@ -1007,324 +494,37 @@ flowchart TB
 
 ```
 src/sync/
-├── mod.rs              # SyncManager, main loop
-├── state.rs            # RepoSyncIndex, RelaySyncIndex, PendingSyncIndex types
-├── actions.rs          # AddFilters struct, compute_actions, build_filters
-├── self_subscriber.rs  # SelfSubscriber, batching logic
-├── relay_connection.rs # Per-relay WebSocket connection
-├── consolidation.rs    # Consolidation logic, daily timer
-├── health.rs           # Health tracking (reuse from v2)
-└── metrics.rs          # Prometheus metrics (reuse from v2)
+├── mod.rs              # SyncManager, main loop, data structures (RepoSyncNeeds, RelayState, etc.)
+├── algorithms.rs       # derive_relay_targets(), compute_actions(), AddFilters
+├── filters.rs          # build_announcement_filter(), build_layer2_and_layer3_filters()
+├── health.rs           # RelayHealthTracker with exponential backoff
+├── relay_connection.rs # RelayConnection, RelayEvent handling
+├── self_subscriber.rs  # SelfSubscriber with batching
+└── metrics.rs          # SyncMetrics for Prometheus
 ```
 
 ---
 
-## Comparison: v3 vs v4
-
-| Aspect                   | v3                                        | v4                                            |
-| ------------------------ | ----------------------------------------- | --------------------------------------------- |
-| Connect handling         | Separate initial vs reconnect             | Unified handle_connect_or_reconnect           |
-| Layer 1 handling         | Mixed with other layers                   | Separate build_layer1_filter, always included |
-| Layer 2+3 tags           | Basic a/e tags                            | Comprehensive a/A/q and e/E/q per v2          |
-| Rebuild logic            | Duplicated in reconnect and consolidation | Shared rebuild_all_subscriptions method       |
-| Consolidation trigger    | Maybe periodic                            | On filter add in handle_add_filters           |
-| Since filter application | Applied in handle_reconnect               | build_all_filters with optional since         |
-| PSI clearing             | On disconnect                             | On disconnect (confirmed)                     |
-| Daily timer              | Consolidation-style                       | Fresh sync (different from consolidation)     |
-
----
-
-## Self-Subscriber Flow
-
-The SelfSubscriber connects to the own relay and maintains a subscription to discover repos and events. It batches incoming events and triggers compute_actions.
-
-### State Tracking
-
-```rust
-pub struct SelfSubscriber {
-    own_relay_url: String,
-    relay_domain: String,
-    repo_sync_index: RepoSyncIndex,
-    pending_sync_index: PendingSyncIndex,
-    relay_sync_index: RelaySyncIndex,
-    action_tx: mpsc::Sender<AddFilters>,
-    /// Timestamp of last successful connection - used for since filter on reconnection
-    last_connected: Option<Timestamp>,
-    /// The active client connection
-    client: Option<Client>,
-}
-```
-
-### On Startup / Reconnect (Unified)
-
-Both initial startup and reconnection use the same `connect_and_subscribe` method:
-
-```rust
-impl SelfSubscriber {
-    async fn run(mut self) {
-        loop {
-            // Connect or reconnect
-            if let Err(e) = self.connect_and_subscribe().await {
-                tracing::warn!("Connection failed: {}, will retry", e);
-                tokio::time::sleep(Duration::from_secs(5)).await;
-                continue;
-            }
-
-            // Run event loop until disconnection
-            self.event_loop().await;
-
-            // Loop will retry connection
-        }
-    }
-
-    async fn connect_and_subscribe(&mut self) -> Result<(), Error> {
-        let client = Client::new(Keys::generate());
-        client.add_relay(&self.own_relay_url).await?;
-        client.connect().await;
-
-        // Build filter - add since only on reconnect
-        let filter = Filter::new().kinds([
-            Kind::Custom(30617),  // Repository announcements
-            Kind::GitPatch,       // 1617
-            Kind::Custom(1618),   // PRs
-            Kind::Custom(1619),   // PR updates
-            Kind::GitIssue,       // 1621
-        ]);
-
-        let filter = if let Some(ts) = self.last_connected {
-            // Reconnection: use since filter
-            let since = Timestamp::from(ts.as_u64().saturating_sub(900)); // -15 min buffer
-            filter.since(since)
-        } else {
-            // Initial connect: no since filter - get full history
-            filter
-        };
-
-        // Update last_connected AFTER computing since
-        self.last_connected = Some(Timestamp::now());
-
-        client.subscribe(filter, None).await?;
-        self.client = Some(client);
-        Ok(())
-    }
-}
-```
-
-### Event Loop with Batching
-
-```rust
-impl SelfSubscriber {
-    async fn event_loop(&mut self) {
-        let client = self.client.as_ref().unwrap();
-        let mut pending_events: Vec<Event> = Vec::new();
-        let mut batch_timer: Option<Instant> = None;
-        let batch_window = Duration::from_secs(5);
-
-        loop {
-            let timeout = batch_timer
-                .map(|t| batch_window.saturating_sub(t.elapsed()))
-                .unwrap_or(Duration::from_secs(60));
-
-            tokio::select! {
-                notification = client.notifications().recv() => {
-                    match notification {
-                        Ok(RelayPoolNotification::Event { event, .. }) => {
-                            pending_events.push(*event);
-
-                            // Start timer on first event - does NOT reset
-                            if batch_timer.is_none() {
-                                batch_timer = Some(Instant::now());
-                            }
-                        }
-                        Ok(RelayPoolNotification::Shutdown) => {
-                            // Connection lost
-                            break;
-                        }
-                        _ => {}
-                    }
-                }
-                _ = tokio::time::sleep(timeout), if batch_timer.is_some() => {
-                    // Batch window elapsed
-                    self.process_batch(pending_events.drain(..).collect()).await;
-                    batch_timer = None;
-                }
-            }
-        }
-    }
-
-    async fn process_batch(&self, events: Vec<Event>) {
-        // 1. Update RepoSyncIndex
-        for event in events {
-            match event.kind.as_u16() {
-                30617 => self.handle_announcement(&event).await,
-                1617 | 1618 | 1619 | 1621 => self.handle_root_event(&event).await,
-                _ => {}
-            }
-        }
-
-        // 2. Derive targets and compute actions
-        let repo_index = self.repo_sync_index.read().await;
-        let targets = derive_relay_targets(&repo_index);
+## Health Tracking
 
-        let pending = self.pending_sync_index.read().await;
-        let confirmed = self.relay_sync_index.read().await;
+The [`RelayHealthTracker`](../../src/sync/health.rs:93) manages connection health with exponential backoff:
 
-        let actions = compute_actions(&targets, &pending, &confirmed);
+- **States**: Healthy, Degraded, Dead
+- **Backoff**: `base * 2^(failures-1)`, capped at max_backoff
+- **Dead threshold**: 24 hours of continuous failures
+- **Dead relay retry**: Once per 24 hours
 
-        drop(repo_index);
-        drop(pending);
-        drop(confirmed);
-
-        // 3. Send actions to SyncManager
-        for action in actions {
-            let _ = self.action_tx.send(action).await;
-        }
-    }
-
-    async fn handle_announcement(&self, event: &Event) {
-        // Extract repo_ref from event - 30617:pubkey:identifier
-        let d_tag = event.tags.iter()
-            .find_map(|tag| {
-                if tag.kind() == TagKind::D {
-                    tag.content().map(|s| s.to_string())
-                } else {
-                    None
-                }
-            })
-            .unwrap_or_default();
-
-        let repo_ref = format!("30617:{}:{}", event.pubkey, d_tag);
-
-        // Extract relay URLs from 'r' tags
-        let relays: HashSet<String> = event.tags.iter()
-            .filter_map(|tag| {
-                if tag.kind() == TagKind::Relay {
-                    tag.content().map(|s| s.to_string())
-                } else {
-                    None
-                }
-            })
-            .collect();
-
-        // Update RepoSyncIndex
-        let mut index = self.repo_sync_index.write().await;
-        let needs = index.entry(repo_ref).or_default();
-        needs.relays = relays;
-    }
-
-    async fn handle_root_event(&self, event: &Event) {
-        // Extract repo_ref from 'a' tag
-        let repo_ref = event.tags.iter()
-            .find_map(|tag| {
-                if tag.kind() == TagKind::A {
-                    tag.content().map(|s| s.to_string())
-                } else {
-                    None
-                }
-            });
-
-        if let Some(repo_ref) = repo_ref {
-            let mut index = self.repo_sync_index.write().await;
-            let needs = index.entry(repo_ref).or_default();
-            needs.root_events.insert(event.id);
-        }
-    }
-}
-```
+Bootstrap relays are never disconnected by the cleanup system, even if empty.
 
 ---
 
-## Implementation Notes
-
-This section documents the actual implementation details as of December 2024 (Phases 1-10 complete).
-
-### Architectural Decisions During Implementation
-
-**Phase 7 Refactoring**: The `SyncManager::run()` method required refactoring to use `Arc<Mutex<SyncManager>>` for shared access. The daily timer and disconnect checker tasks need to access the manager, so `self` is wrapped after initial setup:
-
-```rust
-// 7. Wrap self in Arc<Mutex> for sharing with timer task
-let sync_manager = Arc::new(Mutex::new(self));
-```
-
-This allows background tasks (daily timer, disconnect checker) to acquire the lock when needed while the main event loop handles actions from the self-subscriber.
-
-**Health Module**: The health tracking module was adapted from the v3 implementation at `work/sync-v3/health.rs`. The implementation uses:
-
-- `DashMap` for thread-safe concurrent access without external locking
-- Three states: `Healthy`, `Degraded`, `Dead`
-- Exponential backoff: `base * 2^(failures-1)`, capped at max_backoff
-- Dead threshold: 24 hours of continuous failures
-- Dead relay retry: Once per 24 hours
-
-### Implementation Constants
-
-| Constant                          | Value      | Purpose                                          |
-| --------------------------------- | ---------- | ------------------------------------------------ |
-| `CONSOLIDATION_THRESHOLD`         | 70 filters | Maximum filters before triggering consolidation  |
-| `CONSOLIDATION_WAIT_TIMEOUT_SECS` | 30 seconds | Timeout for pending batches during consolidation |
-| `QUICK_RECONNECT_WINDOW_SECS`     | 15 minutes | Window for quick reconnect vs fresh sync         |
-| `DISCONNECT_CHECK_INTERVAL_SECS`  | 60 seconds | Interval for checking empty relays to disconnect |
-| `DEAD_THRESHOLD_HOURS`            | 24 hours   | Time before relay marked as dead                 |
-| `BASE_BACKOFF_SECS`               | 5 seconds  | Base duration for exponential backoff            |
-
-### Daily Timer Randomization
-
-The daily timer uses randomization between 23-25 hours to prevent thundering herd effects when multiple ngit-grasp instances are running:
-
-```rust
-let hours = 23.0 + rand::thread_rng().gen::<f64>() * 2.0;
-```
-
-### Bootstrap Relay Protection
-
-Bootstrap relays are never disconnected by the cleanup system. The `check_disconnects()` method explicitly filters them out:
-
-```rust
-.filter(|(_, state)| {
-    !state.is_bootstrap &&
-    state.repos.is_empty() &&
-    state.root_events.is_empty()
-})
-```
-
-### Graceful Shutdown
-
-Shutdown uses a tokio broadcast channel for coordinated termination:
-
-```rust
-let (shutdown_tx, _shutdown_rx) = broadcast::channel(1);
-```
-
-Each background task (self-subscriber, daily timer, disconnect checker) receives its own `broadcast::Receiver` subscription and monitors for the shutdown signal in its main loop.
-
-### Actual Module Structure
-
-The implemented module structure differs from the original spec:
-
-```
-src/sync/
-├── mod.rs              # SyncManager, main loop, index types, metrics
-├── algorithms.rs       # derive_relay_targets, compute_actions, AddFilters
-├── filters.rs          # build_announcement_filter, build_layer2_and_layer3_filters
-├── health.rs           # RelayHealthTracker, HealthState, exponential backoff
-├── relay_connection.rs # RelayConnection, RelayEvent, WebSocket handling
-└── self_subscriber.rs  # SelfSubscriber, RelayAction, batching logic
-```
-
-Key differences from spec:
-
-- No separate `state.rs` - types are defined in `mod.rs`
-- No separate `actions.rs` - moved to `algorithms.rs`
-- No separate `consolidation.rs` - consolidation logic in `mod.rs`
-- No separate `metrics.rs` - `SyncMetrics` defined in `mod.rs`
-
-### Deviations from Original v4 Spec
-
-1. **RelayState lacks `connection` field**: The spec showed `connection: Option<RelayConnection>` in `RelayState`, but the implementation stores connections in a separate `HashMap<String, RelayConnection>` in `SyncManager`.
+## Disconnect Handling
 
-2. **SelfSubscriber simplified**: The actual implementation uses `RelayAction` enum (SpawnRelay/AddFilters) rather than directly using `AddFilters` struct.
+The disconnect checker runs periodically (default: 60 seconds) to clean up empty relays:
 
-3. **Consolidation wait_pending_complete**: The spec described a `wait_pending_complete()` method, but the implementation uses a simpler timeout-based approach checking pending batches.
+- Finds relays with `repos.is_empty() && root_events.is_empty()`
+- Skips bootstrap relays (`is_bootstrap == true`)
+- Removes from relay_sync_index, pending_sync_index, and connections
+- Disconnects the WebSocket connection
 
-4. **Timestamp API**: Uses `Timestamp::now().as_secs()` instead of `.as_u64()` due to nostr-sdk 0.43 API.
+Also triggers reconnection attempts for disconnected relays that have pending work.