diff options
| author | DanConwayDev <DanConwayDev@protonmail.com> | 2026-01-07 11:44:27 +0000 |
|---|---|---|
| committer | DanConwayDev <DanConwayDev@protonmail.com> | 2026-01-07 11:44:27 +0000 |
| commit | 1c5dac680b5a446e26b161208a17030f5fbd8a88 (patch) | |
| tree | 76a7113edcdd7436872665473727c1f535d7ab83 /src/git | |
| parent | 5bd6b9b93cd52da2075bc00a08cf7feca4b85d5c (diff) | |
Add core sync functions for identifier-based purgatory synchronization
Implement sync_identifier_next_url and sync_identifier_from_url functions
that provide the core URL selection and fetch logic for purgatory sync.
sync_identifier_next_url:
- Pure URL selection logic with no side effects
- Filters out our own domain and already-tried URLs
- Respects domain throttling when domain parameter is None
- Can target a specific domain when domain parameter is Some
sync_identifier_from_url:
- Fetches OIDs from a specific URL via the SyncContext
- Tracks request start/completion with ThrottleManager for rate limiting
- Calls process_newly_available_git_data on successful fetch
Also adds get_throttled_domains_with_untried_urls helper for the main
sync loop to know which DomainThrottle queues to enqueue identifiers to.
These functions are designed to be called by both:
- Main sync loop (tries non-throttled URLs immediately)
- DomainThrottle queue processing (when capacity frees up)
Includes 10 unit tests covering:
- Throttled domain skipping
- Tried URL skipping
- Our domain filtering
- Specific domain targeting
- Fetch success/failure handling
- Throttle request tracking
Diffstat (limited to 'src/git')
0 files changed, 0 insertions, 0 deletions