upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/src/purgatory/sync/mod.rs
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2026-01-07 11:44:27 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2026-01-07 11:44:27 +0000
commit1c5dac680b5a446e26b161208a17030f5fbd8a88 (patch)
tree76a7113edcdd7436872665473727c1f535d7ab83 /src/purgatory/sync/mod.rs
parent5bd6b9b93cd52da2075bc00a08cf7feca4b85d5c (diff)
Add core sync functions for identifier-based purgatory synchronization
Implement sync_identifier_next_url and sync_identifier_from_url functions that provide the core URL selection and fetch logic for purgatory sync. sync_identifier_next_url: - Pure URL selection logic with no side effects - Filters out our own domain and already-tried URLs - Respects domain throttling when domain parameter is None - Can target a specific domain when domain parameter is Some sync_identifier_from_url: - Fetches OIDs from a specific URL via the SyncContext - Tracks request start/completion with ThrottleManager for rate limiting - Calls process_newly_available_git_data on successful fetch Also adds get_throttled_domains_with_untried_urls helper for the main sync loop to know which DomainThrottle queues to enqueue identifiers to. These functions are designed to be called by both: - Main sync loop (tries non-throttled URLs immediately) - DomainThrottle queue processing (when capacity frees up) Includes 10 unit tests covering: - Throttled domain skipping - Tried URL skipping - Our domain filtering - Specific domain targeting - Fetch success/failure handling - Throttle request tracking
Diffstat (limited to 'src/purgatory/sync/mod.rs')
-rw-r--r--src/purgatory/sync/mod.rs5
1 files changed, 5 insertions, 0 deletions
diff --git a/src/purgatory/sync/mod.rs b/src/purgatory/sync/mod.rs
index 1ac0cb1..d26c1f0 100644
--- a/src/purgatory/sync/mod.rs
+++ b/src/purgatory/sync/mod.rs
@@ -7,10 +7,15 @@
7//! - Debouncing for burst event arrivals 7//! - Debouncing for burst event arrivals
8 8
9mod context; 9mod context;
10mod functions;
10mod queue; 11mod queue;
11mod throttle; 12mod throttle;
12 13
13pub use context::{ProcessResult, SyncContext}; 14pub use context::{ProcessResult, SyncContext};
15pub use functions::{
16 get_throttled_domains_with_untried_urls, sync_identifier_from_url, sync_identifier_next_url,
17 ThrottledDomainInfo,
18};
14pub use queue::SyncQueueEntry; 19pub use queue::SyncQueueEntry;
15pub use throttle::{DomainThrottle, ThrottleManager}; 20pub use throttle::{DomainThrottle, ThrottleManager};
16 21