npub1tkq8unhsd5jqx6ueex5lcpsgknrpquxuk44ftpjlpm3ulaake7xs76txrw/ngit-grasp-mirror/src/purgatory/sync/throttle.rs, branch master

Add naughty list for git remotes with persistent SSL/DNS errors

2026-01-10T02:15:22+00:00

Implement domain-level naughty list tracking for git remotes, reusing the existing NaughtyListTracker from relay sync. This prevents repeated attempts to fetch from git domains with persistent infrastructure issues (SSL/TLS certificate errors, DNS failures). Changes: - Updated NaughtyListTracker to track both relay URLs and git domains - Added git_naughty_list field to RealSyncContext for error classification - Modified fetch_oids() to classify git fetch errors and record naughty domains - Updated sync_identifier_next_url() to filter out naughty domains during URL selection - Added git_naughty_list parameter to ThrottleManager for domain queue processing - Threaded naughty list through start_sync_loop and all sync functions - Updated all tests to pass naughty list parameter The naughty list uses 12-hour expiration (configurable) to allow domains to recover from infrastructure issues. First occurrence logs WARN, repeats log DEBUG.

chore: cargo fmt

2026-01-08T00:50:54+00:00

fix: resolve clippy warnings

2026-01-07T17:16:37+00:00

- Prefix unused variable auth_result with underscore - Prefix unused field git_data_path with underscore in Purgatory struct - Add #[allow(clippy::too_many_arguments)] to handle_receive_pack - Replace len() >= 1 with !is_empty() - Replace .last() with .next_back() on DoubleEndedIterator - Fix doc list item overindentation - Replace map_or(true, ...) with is_none_or(...) - Replace map_or(false, ...) with is_some_and(...)

Refactor handle_receive_pack to use unified process_newly_available_git_data

2026-01-07T12:34:59+00:00

Replace ~100 lines of duplicated post-push processing in handle_receive_pack with a single call to the unified process_newly_available_git_data function. The unified function handles all post-git-data-available processing: - Discovering satisfiable events from purgatory (state and PR events) - Syncing OIDs to authorized owner repos - Aligning refs (+ setting HEAD) in all owner repos - Saving events to database - Notifying WebSocket subscribers - Removing from purgatory This ensures consistent behavior regardless of how git data arrives (git push vs purgatory sync fetching from remote servers). Also mark test-only internal methods with #[cfg(test)] to silence dead code warnings.

Add sync_identifier orchestration and ThrottleManager queue processing

2026-01-07T11:57:55+00:00

Implement the main sync orchestration function and trigger-based queue processing for throttled domains: sync_identifier function: - Orchestrates syncing git data for a single identifier - Tries all non-throttled URLs in sequence - Checks completion after each fetch (no pending events or all OIDs fetched) - Enqueues with throttled domains when non-throttled URLs are exhausted - Returns true if complete, false if events remain (for backoff) ThrottleManager enhancements: - Add set_context() to provide SyncContext for queue processing - Add try_process_next() to spawn tasks when capacity frees - Add process_queued_identifier() to handle queued work - Update complete_request() to trigger processing on completion - Update enqueue_identifier() to trigger processing when capacity available - Add internal methods for non-Arc testing compatibility Generic function updates: - Add ?Sized bound to sync_identifier_next_url, sync_identifier_from_url, sync_identifier, and get_throttled_domains_with_untried_urls for dynamic dispatch support (Arc) Tests: - sync_identifier_tries_multiple_urls_until_complete: verifies sequential URL fetching until all OIDs are available - sync_identifier_enqueues_throttled_domains_when_incomplete: verifies throttled domains get the identifier enqueued for later processing - has_queued_work_reflects_queue_state: verifies queue state tracking

Add ThrottleManager for cross-domain rate limiting

2026-01-07T11:33:03+00:00

Implements ThrottleManager which manages all per-domain DomainThrottle instances and provides: - Throttle status checking via is_throttled() for sync URL selection - Request tracking via start_request()/complete_request() - Identifier queue management via enqueue_identifier() - Automatic domain throttle creation on first access - Thread-safe access via DashMap with Mutex-wrapped throttles The manager uses the configured max_concurrent and max_per_minute limits for all domains. Trigger-based queue processing (set_context, process_queued_identifier) will be added after SyncContext is available. Tests verify: - is_throttled reflects domain capacity correctly - enqueue_identifier creates domain throttle if needed - start_request creates domain throttle if needed

Add DomainThrottle for per-domain rate limiting

2026-01-07T11:26:48+00:00

Implement per-domain throttling for purgatory sync operations: - Concurrent request limit (max in-flight requests per domain) - Rate limit (max requests per minute via sliding window) - Fair round-robin queue processing across identifiers - In-progress tracking to prevent duplicate fetches - Tried URL tracking per identifier Add indexmap dependency for ordered iteration in round-robin queue. Includes 6 unit tests covering: - Concurrent limit enforcement - Rate limit enforcement (sliding window) - Round-robin fair processing - In-progress identifier skipping - Round-robin index adjustment on removal - Tried URL merging on re-enqueue