| Age | Commit message (Collapse) | Author |
|---|
|
|
|
Remove the redundant inline kind-30617 registration block from the sync
event loop and the three is_generic/recompute_new_sync_filters_for_relay
calls from confirm_batch error paths. The purgatory announcement sync
timer (run_purgatory_announcement_sync) is now the sole registration path.
Consolidate NGIT_SYNC_BATCH_WINDOW_MS and NGIT_PURGATORY_SYNC_INTERVAL_MS
into a single NGIT_TEST=1 flag that sets both timers to 200ms, replacing
two ad-hoc env vars with one reusable test-mode flag.
|
|
negentropy fallback
Three targeted fixes for purgatory announcement sync:
1. SelfSubscriber sync_level upgrade: After or_insert_with in process_batch,
always set entry.sync_level = SyncLevel::Full so that when a promoted
announcement is broadcast via notify_event and SelfSubscriber receives it,
an existing StateOnly entry gets upgraded to Full and PR event subscriptions
are triggered immediately (not delayed up to 24h).
2. Negentropy fallback filter split: In handle_eose, when falling back from
negentropy to REQ+EOSE, split batch_repos by SyncLevel and call
build_sync_level_aware_filters instead of build_layer2_and_layer3_filters.
Prevents StateOnly (purgatory) repos from getting Layer 2 #a/#A/#q filters
prematurely, which caused nostr-sdk client deduplication to permanently
drop PR events after orphan rejection.
3. Recompute sync filters after announcement batch EOSE: Add
recompute_new_sync_filters_for_relay calls at all three batch-completion
paths in handle_eose for generic filter (announcement) batches. This
triggers state-only subscriptions for any purgatory repos registered during
that batch, fixing the 24h delay before state event sync starts.
4. User-submitted purgatory announcements: Add repo_sync_index field to
PolicyContext with setter/getter, wire in main.rs after SyncManager
creation, and register in AcceptPurgatory handler so user-submitted
announcements get StateOnly sync started immediately.
5. Update archive tests: test_archive_without_state_events_does_not_sync_git
updated to reflect that StateOnly subscription now proactively fetches
state events from source relays. test_archive_read_only_creates_bare_repo
un-ignored as it now works end-to-end.
|
|
Purgatory announcements need state events (kind 30618) synced from
external relays, but not full L2/L3 events (patches, issues, PRs)
which would be rejected anyway. This implements the SyncLevel concept
from the design doc (decision #6):
- Add SyncLevel enum (Full vs StateOnly) to RepoSyncNeeds
- When announcement enters purgatory during sync, register in
RepoSyncIndex with SyncLevel::StateOnly
- Add build_sync_level_aware_filters() that partitions repos by level:
StateOnly repos only get state event filters (kind 30618)
- Update derive_relay_targets to track state_only_repos separately
- Update compute_actions to handle both repo sets
- SelfSubscriber always uses SyncLevel::Full (promoted repos)
|
|
load_existing_events()
Root cause: `last_connected` was set to Timestamp::now() BEFORE
load_existing_events() was called (line 425), causing the database
query to filter out all existing events with .since(current_time).
The query became: SELECT * FROM events WHERE created_at >= <now>
Result: 0 events returned (nothing has created_at in the future)
Solution: Remove .since() filter from database queries entirely.
The `last_connected` field is now only used for WebSocket subscription
filters to avoid re-fetching events from remote relays on reconnect.
Rationale for this approach over reordering operations:
- Database queries are fast (indexed by kind and created_at)
- Loading all events on startup ensures consistency
- Eliminates subtle ordering dependency that could break in refactoring
- Cleaner mental model: database = full load, WebSocket = incremental
This fixes the issue where ~190 state events weren't being fetched
after deploying the database query fix (commit 4162c90).
Evidence: Production logs showed "Loaded announcements from database
count=0" when there should have been hundreds of announcements.
|
|
Previously, SelfSubscriber only saw events returned by the WebSocket
subscription to the local relay, which has limits on the number of
events returned. This caused repos with announcements in the database
to never get Layer 2/3 filters created, resulting in missing state events.
Now, on startup, we query the database directly with two separate queries:
1. Query announcements (30617) to populate repo_sync_index
2. Query root events (1617/1618/1621) to create Layer 3 filters
Both queries use .since(last_connected) if available for incremental
loading on reconnect.
Filters are created inline and made mutable to support the .since()
clause, rather than using a shared create_event_filter() method.
Fixes the issue where state events were missing for repos like cashbird
and creative-space that had announcements in the database but weren't
returned by the WebSocket subscription.
|
|
The bug: SelfSubscriber filtered announcements with lists_our_relay() check,
preventing archive_all mode from discovering relays in announcements that
don't list our relay domain.
The insight: SelfSubscriber only receives events that ALREADY passed
write policy validation (archive_all, archive_whitelist, blacklist, etc.)
via admit_event() before being saved to the database. The event flow:
External relay → process_event_static() → write_policy.admit_event()
→ (validation happens here) → save to DB → notify_event()
→ SelfSubscriber receives via WebSocket
So the lists_our_relay() check was redundant double-validation that
broke archive_all mode by filtering events that had already been
accepted by the write policy.
The fix: Simply remove the lists_our_relay() filtering. Events reaching
SelfSubscriber are pre-validated and should all be processed for relay
discovery according to the configured archive policy.
Changes:
- Removed lists_our_relay() check from process_notification() (4 lines)
- Removed unused lists_our_relay() helper function (9 lines)
- Added comment explaining events are pre-validated (3 lines)
- Total: 13 lines removed, 3 lines added
Fixes #194d
|
|
- Replace KIND_REPOSITORY_ANNOUNCEMENT with Kind::GitRepoAnnouncement
- Replace KIND_REPOSITORY_STATE with Kind::RepoState
- Replace KIND_PR with Kind::GitPullRequest
- Replace KIND_PR_UPDATE with Kind::GitPullRequestUpdate
- Replace KIND_USER_GRASP_LIST with Kind::GitUserGraspList
- Replace KIND_PATCH with Kind::GitPatch
- Replace KIND_ISSUE with Kind::GitIssue
- Replace KIND_COMMENT with Kind::Comment
- Replace all Kind::Custom(30617|30618|1617|1618|1619|1621|1111|10317) patterns
- Remove all hardcoded KIND_* constants from events.rs
- Update all match statements to use Kind enum directly
- Update all filter builders to use Kind variants
- Update all test helpers and assertions
Benefits:
- Type safety: compiler prevents wrong kind numbers
- Readability: Kind::GitRepoAnnouncement is self-documenting
- Maintainability: single source of truth (rust-nostr)
- IDE support: full autocompletion and refactoring
- Standards: aligns with rust-nostr best practices
Files modified: 21
Constants removed: 9
Patterns replaced: 100+
Tests passing: 222/222
|
|
|
|
|
|
they are legacy and not root events
|
|
Main lib (src/):
- Add #[allow(dead_code)] for build_info field (stored to prevent Prometheus unregistration)
- Add #[allow(dead_code)] for first_seen field (reserved for future rate limiting)
- Replace .or_insert_with(RelaySyncNeeds::default) with .or_default()
- Replace manual div_ceil implementations with .div_ceil(100)
Test code (tests/):
- Replace .expect(&format!(...)) with .unwrap_or_else(|_| panic!(...))
- Remove needless borrows in fetch_metrics() calls
- Add #[allow(dead_code)] and #[allow(unused_imports)] to test helpers module
grasp-audit:
- Apply cargo fmt to fix formatting
|
|
When root events (issues/patches) are received via self-subscription,
handle_root_event() was only updating the repo_sync_index directly.
This caused process_batch() to early-return when pending.is_empty(),
so Layer 3 filters for comments/replies were never created.
The fix adds root events to both:
1. repo_sync_index (for immediate availability)
2. pending queue (to trigger Layer 3 filter creation in next batch)
Critical: The pending entry must include relays from repo_sync_index
so derive_relay_targets() knows where to send Layer 3 subscriptions.
The Layer 3 test now verifies that events sent BEFORE the subscription
is established are still synced - proving subscriptions without 'since'
correctly fetch historical events.
Enabled 4 previously ignored Layer 3 tests:
- test_live_sync_layer3_events
- test_layer3_sync_with_lowercase_e_tag
- test_layer3_sync_with_uppercase_e_tag
- test_layer3_sync_with_q_tag
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|