From 18bfb246029a848a0b307e7c8a8e4df57addabb2 Mon Sep 17 00:00:00 2001 From: DanConwayDev Date: Wed, 7 Jan 2026 12:12:49 +0000 Subject: Add background sync loop for purgatory identifier processing Implement the main sync loop that runs in the background and processes identifiers that are ready for git data synchronization: - Runs every 1 second (hardcoded interval, not configurable) - Finds all ready identifiers where !in_progress && next_attempt <= now - Spawns parallel tasks for each ready identifier - Each task calls sync_identifier to try fetching git data from remotes - Applies backoff when sync completes but events remain in purgatory - Removes identifiers from queue when sync completes or no events remain The loop integrates with the existing sync infrastructure: - Uses SyncContext trait for testability - Uses ThrottleManager for domain-based rate limiting - Uses sync_identifier for the actual fetch orchestration This enables automatic background fetching of git data for events in purgatory, complementing the existing push-triggered sync path. --- src/purgatory/sync/mod.rs | 2 ++ 1 file changed, 2 insertions(+) (limited to 'src/purgatory/sync/mod.rs') diff --git a/src/purgatory/sync/mod.rs b/src/purgatory/sync/mod.rs index 8ac9216..be89130 100644 --- a/src/purgatory/sync/mod.rs +++ b/src/purgatory/sync/mod.rs @@ -5,9 +5,11 @@ //! - Domain-based throttling (configurable requests/minute per domain) //! - Exponential backoff per identifier (20s → 2m, then 2m intervals) //! - Debouncing for burst event arrivals +//! - Background sync loop processing ready identifiers every 1 second mod context; mod functions; +mod r#loop; mod queue; mod throttle; -- cgit v1.2.3