diff options
Diffstat (limited to 'docs/explanation/grasp-02-proactive-sync-v4.md')
| -rw-r--r-- | docs/explanation/grasp-02-proactive-sync-v4.md | 97 |
1 files changed, 96 insertions, 1 deletions
diff --git a/docs/explanation/grasp-02-proactive-sync-v4.md b/docs/explanation/grasp-02-proactive-sync-v4.md index aba88a5..5ac92cd 100644 --- a/docs/explanation/grasp-02-proactive-sync-v4.md +++ b/docs/explanation/grasp-02-proactive-sync-v4.md | |||
| @@ -1226,4 +1226,99 @@ impl SelfSubscriber { | |||
| 1226 | } | 1226 | } |
| 1227 | } | 1227 | } |
| 1228 | } | 1228 | } |
| 1229 | ``` \ No newline at end of file | 1229 | ``` |
| 1230 | |||
| 1231 | --- | ||
| 1232 | |||
| 1233 | ## Implementation Notes | ||
| 1234 | |||
| 1235 | This section documents the actual implementation details as of December 2024 (Phases 1-10 complete). | ||
| 1236 | |||
| 1237 | ### Architectural Decisions During Implementation | ||
| 1238 | |||
| 1239 | **Phase 7 Refactoring**: The `SyncManager::run()` method required refactoring to use `Arc<Mutex<SyncManager>>` for shared access. The daily timer and disconnect checker tasks need to access the manager, so `self` is wrapped after initial setup: | ||
| 1240 | |||
| 1241 | ```rust | ||
| 1242 | // 7. Wrap self in Arc<Mutex> for sharing with timer task | ||
| 1243 | let sync_manager = Arc::new(Mutex::new(self)); | ||
| 1244 | ``` | ||
| 1245 | |||
| 1246 | This allows background tasks (daily timer, disconnect checker) to acquire the lock when needed while the main event loop handles actions from the self-subscriber. | ||
| 1247 | |||
| 1248 | **Health Module**: The health tracking module was adapted from the v3 implementation at `work/sync-v3/health.rs`. The implementation uses: | ||
| 1249 | - `DashMap` for thread-safe concurrent access without external locking | ||
| 1250 | - Three states: `Healthy`, `Degraded`, `Dead` | ||
| 1251 | - Exponential backoff: `base * 2^(failures-1)`, capped at max_backoff | ||
| 1252 | - Dead threshold: 24 hours of continuous failures | ||
| 1253 | - Dead relay retry: Once per 24 hours | ||
| 1254 | |||
| 1255 | ### Implementation Constants | ||
| 1256 | |||
| 1257 | | Constant | Value | Purpose | | ||
| 1258 | |----------|-------|---------| | ||
| 1259 | | `CONSOLIDATION_THRESHOLD` | 70 filters | Maximum filters before triggering consolidation | | ||
| 1260 | | `CONSOLIDATION_WAIT_TIMEOUT_SECS` | 30 seconds | Timeout for pending batches during consolidation | | ||
| 1261 | | `QUICK_RECONNECT_WINDOW_SECS` | 15 minutes | Window for quick reconnect vs fresh sync | | ||
| 1262 | | `DISCONNECT_CHECK_INTERVAL_SECS` | 60 seconds | Interval for checking empty relays to disconnect | | ||
| 1263 | | `DEAD_THRESHOLD_HOURS` | 24 hours | Time before relay marked as dead | | ||
| 1264 | | `BASE_BACKOFF_SECS` | 5 seconds | Base duration for exponential backoff | | ||
| 1265 | |||
| 1266 | ### Daily Timer Randomization | ||
| 1267 | |||
| 1268 | The daily timer uses randomization between 23-25 hours to prevent thundering herd effects when multiple ngit-grasp instances are running: | ||
| 1269 | |||
| 1270 | ```rust | ||
| 1271 | let hours = 23.0 + rand::thread_rng().gen::<f64>() * 2.0; | ||
| 1272 | ``` | ||
| 1273 | |||
| 1274 | ### Bootstrap Relay Protection | ||
| 1275 | |||
| 1276 | Bootstrap relays are never disconnected by the cleanup system. The `check_disconnects()` method explicitly filters them out: | ||
| 1277 | |||
| 1278 | ```rust | ||
| 1279 | .filter(|(_, state)| { | ||
| 1280 | !state.is_bootstrap && | ||
| 1281 | state.repos.is_empty() && | ||
| 1282 | state.root_events.is_empty() | ||
| 1283 | }) | ||
| 1284 | ``` | ||
| 1285 | |||
| 1286 | ### Graceful Shutdown | ||
| 1287 | |||
| 1288 | Shutdown uses a tokio broadcast channel for coordinated termination: | ||
| 1289 | |||
| 1290 | ```rust | ||
| 1291 | let (shutdown_tx, _shutdown_rx) = broadcast::channel(1); | ||
| 1292 | ``` | ||
| 1293 | |||
| 1294 | Each background task (self-subscriber, daily timer, disconnect checker) receives its own `broadcast::Receiver` subscription and monitors for the shutdown signal in its main loop. | ||
| 1295 | |||
| 1296 | ### Actual Module Structure | ||
| 1297 | |||
| 1298 | The implemented module structure differs from the original spec: | ||
| 1299 | |||
| 1300 | ``` | ||
| 1301 | src/sync/ | ||
| 1302 | ├── mod.rs # SyncManager, main loop, index types, metrics | ||
| 1303 | ├── algorithms.rs # derive_relay_targets, compute_actions, AddFilters | ||
| 1304 | ├── filters.rs # build_announcement_filter, build_layer2_and_layer3_filters | ||
| 1305 | ├── health.rs # RelayHealthTracker, HealthState, exponential backoff | ||
| 1306 | ├── relay_connection.rs # RelayConnection, RelayEvent, WebSocket handling | ||
| 1307 | └── self_subscriber.rs # SelfSubscriber, RelayAction, batching logic | ||
| 1308 | ``` | ||
| 1309 | |||
| 1310 | Key differences from spec: | ||
| 1311 | - No separate `state.rs` - types are defined in `mod.rs` | ||
| 1312 | - No separate `actions.rs` - moved to `algorithms.rs` | ||
| 1313 | - No separate `consolidation.rs` - consolidation logic in `mod.rs` | ||
| 1314 | - No separate `metrics.rs` - `SyncMetrics` defined in `mod.rs` | ||
| 1315 | |||
| 1316 | ### Deviations from Original v4 Spec | ||
| 1317 | |||
| 1318 | 1. **RelayState lacks `connection` field**: The spec showed `connection: Option<RelayConnection>` in `RelayState`, but the implementation stores connections in a separate `HashMap<String, RelayConnection>` in `SyncManager`. | ||
| 1319 | |||
| 1320 | 2. **SelfSubscriber simplified**: The actual implementation uses `RelayAction` enum (SpawnRelay/AddFilters) rather than directly using `AddFilters` struct. | ||
| 1321 | |||
| 1322 | 3. **Consolidation wait_pending_complete**: The spec described a `wait_pending_complete()` method, but the implementation uses a simpler timeout-based approach checking pending batches. | ||
| 1323 | |||
| 1324 | 4. **Timestamp API**: Uses `Timestamp::now().as_secs()` instead of `.as_u64()` due to nostr-sdk 0.43 API. \ No newline at end of file | ||