diff options
| author | DanConwayDev <DanConwayDev@protonmail.com> | 2026-01-07 12:12:49 +0000 |
|---|---|---|
| committer | DanConwayDev <DanConwayDev@protonmail.com> | 2026-01-07 12:12:49 +0000 |
| commit | 18bfb246029a848a0b307e7c8a8e4df57addabb2 (patch) | |
| tree | 1ca8977edecdab10973415b222d2d7e1a2d12e7c /src/purgatory/sync/mod.rs | |
| parent | fc9f1e282b16bc373c3913973879b43d3f254eb2 (diff) | |
Add background sync loop for purgatory identifier processing
Implement the main sync loop that runs in the background and processes
identifiers that are ready for git data synchronization:
- Runs every 1 second (hardcoded interval, not configurable)
- Finds all ready identifiers where !in_progress && next_attempt <= now
- Spawns parallel tasks for each ready identifier
- Each task calls sync_identifier to try fetching git data from remotes
- Applies backoff when sync completes but events remain in purgatory
- Removes identifiers from queue when sync completes or no events remain
The loop integrates with the existing sync infrastructure:
- Uses SyncContext trait for testability
- Uses ThrottleManager for domain-based rate limiting
- Uses sync_identifier for the actual fetch orchestration
This enables automatic background fetching of git data for events in
purgatory, complementing the existing push-triggered sync path.
Diffstat (limited to 'src/purgatory/sync/mod.rs')
| -rw-r--r-- | src/purgatory/sync/mod.rs | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/src/purgatory/sync/mod.rs b/src/purgatory/sync/mod.rs index 8ac9216..be89130 100644 --- a/src/purgatory/sync/mod.rs +++ b/src/purgatory/sync/mod.rs | |||
| @@ -5,9 +5,11 @@ | |||
| 5 | //! - Domain-based throttling (configurable requests/minute per domain) | 5 | //! - Domain-based throttling (configurable requests/minute per domain) |
| 6 | //! - Exponential backoff per identifier (20s → 2m, then 2m intervals) | 6 | //! - Exponential backoff per identifier (20s → 2m, then 2m intervals) |
| 7 | //! - Debouncing for burst event arrivals | 7 | //! - Debouncing for burst event arrivals |
| 8 | //! - Background sync loop processing ready identifiers every 1 second | ||
| 8 | 9 | ||
| 9 | mod context; | 10 | mod context; |
| 10 | mod functions; | 11 | mod functions; |
| 12 | mod r#loop; | ||
| 11 | mod queue; | 13 | mod queue; |
| 12 | mod throttle; | 14 | mod throttle; |
| 13 | 15 | ||