From 0a1908bd6ee19f7079bb2914c0009bea1fc1db37 Mon Sep 17 00:00:00 2001 From: DanConwayDev Date: Thu, 12 Feb 2026 09:11:38 +0000 Subject: docs: annocunment purgatory clarify soft expiry rationale now we have added announcement purgatory to the protocol spec --- docs/explanation/announcements-purgatory-design.md | 72 ++++++++++++---------- .../announcements-purgatory-implementation.md | 13 ++-- 2 files changed, 47 insertions(+), 38 deletions(-) (limited to 'docs') diff --git a/docs/explanation/announcements-purgatory-design.md b/docs/explanation/announcements-purgatory-design.md index a06a8b2..009547b 100644 --- a/docs/explanation/announcements-purgatory-design.md +++ b/docs/explanation/announcements-purgatory-design.md @@ -48,14 +48,14 @@ This ensures we only serve announcements for repos that actually have content. ### 4. Expiry Extension (Two Places) -**Decision:** Extend purgatory announcement expiry in two scenarios: +**Decision:** Extend purgatory announcement expiry (reset the 30-minute protocol timer) in two scenarios: | Trigger | Location | Why | | ---------------------------- | ------------------------------------ | ----------------------------------- | | State event arrives | `StatePolicy::process_state_event()` | Repo is actively receiving metadata | | Git auth extends state event | `src/git/auth.rs` | Repo is actively receiving git data | -**Why:** Prevents premature expiry during slow sync operations or multi-step pushes. +**Why:** Prevents premature expiry during slow sync operations or multi-step pushes. The protocol's 30-minute expiry is intended for abandoned repositories, not active ones receiving data. ### 5. Authorization Must Check Purgatory Announcements @@ -75,20 +75,26 @@ This ensures we only serve announcements for repos that actually have content. ### 7. Soft Expiry Preserves Event Without Bare Repo -**Decision:** When a purgatory announcement expires, delete the bare repo but retain the announcement event for an extended period (e.g., 24h). +**Decision:** When a purgatory announcement expires (30 minutes per protocol spec), delete the bare repo but retain the announcement event for an extended period (e.g., 24h). -**Why:** This handles the case where a state event arrives after initial expiry. Without soft expiry, we'd either: -- Add to `failed_events` and reject the state event (losing potential revival) -- Re-fetch the announcement repeatedly (wasting sync bandwidth) +**Why the protocol specifies 30 minutes:** The grasp protocol defines a 30-minute expiry for announcement events to ensure clients don't indefinitely cache stale repository information. + +**Why we implement soft expiry:** The protocol's 30-minute expiry creates a sync/storage problem. Without soft expiry, we'd either: + +- Add expired announcements to `failed_events` and permanently reject future state events (losing potential revival when state events arrive late) +- Re-fetch the announcement event repeatedly on every sync cycle (wasting bandwidth and creating unnecessary sync traffic) **Behavior during soft expiry:** -- Bare repo is deleted (saves disk space) + +- Bare repo is deleted (saves disk space, respects protocol expiry) - Announcement event retained in purgatory with `soft_expired` flag - Sync continues requesting state events (same as active purgatory) - If state event arrives: recreate bare repo, clear `soft_expired`, extend expiry - If announcement republished directly to us: treat as fresh arrival - After extended expiry: fully remove from purgatory +**In summary:** Soft expiry is an implementation optimization that prevents us from constantly re-syncing announcement events or permanently blocking repositories that receive delayed state events. + ## Data Structure ```rust @@ -198,29 +204,29 @@ Announcement ──> ACTIVE ───────────────── └── Extended expiry ──> REMOVED (exit) ``` -| Exit | Trigger | Action | -| -------------- | ---------------------------------------------- | -------------------------------------------- | -| **Promotion** | Git data arrives | Move to database, upgrade sync to Full | -| **Soft expiry**| Initial timeout | Delete bare repo, retain event, continue sync| -| **Full expiry**| Extended timeout (soft-expired) | Remove from purgatory entirely | -| **Deletion** | Kind 5 event | Delete bare repo, remove from purgatory | -| **Replacement**| Newer announcement (same pubkey, identifier) | Replace entry | -| **Service change** | Newer announcement removes our service | Remove from purgatory | +| Exit | Trigger | Action | +| ------------------ | -------------------------------------------- | --------------------------------------------- | +| **Promotion** | Git data arrives | Move to database, upgrade sync to Full | +| **Soft expiry** | Initial timeout | Delete bare repo, retain event, continue sync | +| **Full expiry** | Extended timeout (soft-expired) | Remove from purgatory entirely | +| **Deletion** | Kind 5 event | Delete bare repo, remove from purgatory | +| **Replacement** | Newer announcement (same pubkey, identifier) | Replace entry | +| **Service change** | Newer announcement removes our service | Remove from purgatory | ## Integration Points -| File | Change | -| ---------------------------------- | --------------------------------------------------------- | -| `src/purgatory/mod.rs` | Add `announcement_purgatory` store | -| `src/purgatory/types.rs` | Add `AnnouncementPurgatoryEntry` | -| `src/nostr/policy/announcement.rs` | Route new announcements to purgatory | -| `src/git/receive.rs` | Promote on git data arrival | -| `src/git/auth.rs` | Extend purgatory expiry when extending state event expiry | -| `src/git/authorization.rs` | Check purgatory announcements for maintainer authorization| -| `src/nostr/policy/state.rs` | Check purgatory for authorization | -| `src/sync/mod.rs` | Add `SyncLevel` to `RepoSyncNeeds` | -| `src/sync/filters.rs` | Respect sync level when building filters | -| `src/sync/self_subscriber.rs` | Register purgatory announcements with `StateOnly` level | +| File | Change | +| ---------------------------------- | ---------------------------------------------------------- | +| `src/purgatory/mod.rs` | Add `announcement_purgatory` store | +| `src/purgatory/types.rs` | Add `AnnouncementPurgatoryEntry` | +| `src/nostr/policy/announcement.rs` | Route new announcements to purgatory | +| `src/git/receive.rs` | Promote on git data arrival | +| `src/git/auth.rs` | Extend purgatory expiry when extending state event expiry | +| `src/git/authorization.rs` | Check purgatory announcements for maintainer authorization | +| `src/nostr/policy/state.rs` | Check purgatory for authorization | +| `src/sync/mod.rs` | Add `SyncLevel` to `RepoSyncNeeds` | +| `src/sync/filters.rs` | Respect sync level when building filters | +| `src/sync/self_subscriber.rs` | Register purgatory announcements with `StateOnly` level | See [announcements-purgatory-implementation.md](./announcements-purgatory-implementation.md) for detailed implementation notes. @@ -240,9 +246,9 @@ See [announcements-purgatory-implementation.md](./announcements-purgatory-implem ## Risks -| Risk | Mitigation | -| ------------------------------------ | ------------------------------------------------------- | -| Disk exhaustion from purgatory repos | Short expiry, soft expiry deletes repo early | -| Race between promotion and expiry | Atomic operations | -| Sync re-fetching expired events | Soft expiry retains event; no need for `failed_events` | -| Filter explosion from many purgatory | Existing consolidation handles this (threshold at 70) | +| Risk | Mitigation | +| ------------------------------------ | ------------------------------------------------------ | +| Disk exhaustion from purgatory repos | Short expiry, soft expiry deletes repo early | +| Race between promotion and expiry | Atomic operations | +| Sync re-fetching expired events | Soft expiry retains event; no need for `failed_events` | +| Filter explosion from many purgatory | Existing consolidation handles this (threshold at 70) | diff --git a/docs/explanation/announcements-purgatory-implementation.md b/docs/explanation/announcements-purgatory-implementation.md index d5b8698..263c253 100644 --- a/docs/explanation/announcements-purgatory-implementation.md +++ b/docs/explanation/announcements-purgatory-implementation.md @@ -204,21 +204,22 @@ impl Purgatory { .collect() } - /// Transition to soft-expired state + /// Transition to soft-expired state (protocol's 30min expiry reached) pub fn soft_expire_announcement( &self, key: &(PublicKey, String), ) -> Option { if let Some(mut entry) = self.announcement_purgatory.get_mut(key) { entry.soft_expired = true; - entry.expires_at = Instant::now() + SOFT_EXPIRY_DURATION; // e.g., 24h + entry.expires_at = Instant::now() + SOFT_EXPIRY_DURATION; // e.g., 24h extended retention Some(entry.repo_path.clone()) // Return path for bare repo deletion } else { None } } - /// Revive soft-expired announcement (caller must recreate bare repo) + /// Revive soft-expired announcement when state event arrives + /// (caller must recreate bare repo) pub fn revive_announcement( &self, key: &(PublicKey, String), @@ -226,7 +227,7 @@ impl Purgatory { if let Some(mut entry) = self.announcement_purgatory.get_mut(key) { if entry.soft_expired { entry.soft_expired = false; - entry.expires_at = Instant::now() + ACTIVE_EXPIRY_DURATION; + entry.expires_at = Instant::now() + ACTIVE_EXPIRY_DURATION; // Reset 30min protocol timer return Some(entry.repo_path.clone()); // Caller recreates bare repo } } @@ -270,9 +271,11 @@ When a state event arrives for a soft-expired announcement, the state policy mus 1. Check purgatory for a matching announcement (in addition to DB) 2. Validate authorization against the purgatory announcement 3. If soft-expired, call `revive_announcement()` and recreate the bare repo -4. Extend the announcement's expiry +4. Extend the announcement's expiry (reset the 30-minute protocol timer) 5. Route the state event to state purgatory +**Why revival is necessary:** Without soft expiry + revival, late-arriving state events would either be permanently rejected (if we added the announcement to `failed_events`) or cause constant re-syncing of the announcement event. Revival allows us to respect the protocol's 30-minute expiry while still handling delayed state events gracefully. + The exact integration will depend on the current structure of `StatePolicy::process_state_event()` - see implementation phase for details. ## File Change Summary -- cgit v1.2.3