diff options
| author | DanConwayDev <DanConwayDev@protonmail.com> | 2026-01-07 11:26:48 +0000 |
|---|---|---|
| committer | DanConwayDev <DanConwayDev@protonmail.com> | 2026-01-07 11:26:48 +0000 |
| commit | 08eba888fe510896f1863be6f2b5a72a91616975 (patch) | |
| tree | c5c947246dd34fbffbc1cb15b48102bb472d3d4f /src/purgatory/sync/mod.rs | |
| parent | 2665811f54f62f147b7d773c76bd26d032b8f9cb (diff) | |
Add DomainThrottle for per-domain rate limiting
Implement per-domain throttling for purgatory sync operations:
- Concurrent request limit (max in-flight requests per domain)
- Rate limit (max requests per minute via sliding window)
- Fair round-robin queue processing across identifiers
- In-progress tracking to prevent duplicate fetches
- Tried URL tracking per identifier
Add indexmap dependency for ordered iteration in round-robin queue.
Includes 6 unit tests covering:
- Concurrent limit enforcement
- Rate limit enforcement (sliding window)
- Round-robin fair processing
- In-progress identifier skipping
- Round-robin index adjustment on removal
- Tried URL merging on re-enqueue
Diffstat (limited to 'src/purgatory/sync/mod.rs')
| -rw-r--r-- | src/purgatory/sync/mod.rs | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/src/purgatory/sync/mod.rs b/src/purgatory/sync/mod.rs index 7b6d64a..3c0d875 100644 --- a/src/purgatory/sync/mod.rs +++ b/src/purgatory/sync/mod.rs | |||
| @@ -7,5 +7,7 @@ | |||
| 7 | //! - Debouncing for burst event arrivals | 7 | //! - Debouncing for burst event arrivals |
| 8 | 8 | ||
| 9 | mod queue; | 9 | mod queue; |
| 10 | mod throttle; | ||
| 10 | 11 | ||
| 11 | pub use queue::SyncQueueEntry; | 12 | pub use queue::SyncQueueEntry; |
| 13 | pub use throttle::DomainThrottle; | ||