upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
Diffstat (limited to 'docs')
-rw-r--r--docs/explanation/announcements-purgatory-design.md72
-rw-r--r--docs/explanation/announcements-purgatory-implementation.md13
2 files changed, 47 insertions, 38 deletions
diff --git a/docs/explanation/announcements-purgatory-design.md b/docs/explanation/announcements-purgatory-design.md
index a06a8b2..009547b 100644
--- a/docs/explanation/announcements-purgatory-design.md
+++ b/docs/explanation/announcements-purgatory-design.md
@@ -48,14 +48,14 @@ This ensures we only serve announcements for repos that actually have content.
48 48
49### 4. Expiry Extension (Two Places) 49### 4. Expiry Extension (Two Places)
50 50
51**Decision:** Extend purgatory announcement expiry in two scenarios: 51**Decision:** Extend purgatory announcement expiry (reset the 30-minute protocol timer) in two scenarios:
52 52
53| Trigger | Location | Why | 53| Trigger | Location | Why |
54| ---------------------------- | ------------------------------------ | ----------------------------------- | 54| ---------------------------- | ------------------------------------ | ----------------------------------- |
55| State event arrives | `StatePolicy::process_state_event()` | Repo is actively receiving metadata | 55| State event arrives | `StatePolicy::process_state_event()` | Repo is actively receiving metadata |
56| Git auth extends state event | `src/git/auth.rs` | Repo is actively receiving git data | 56| Git auth extends state event | `src/git/auth.rs` | Repo is actively receiving git data |
57 57
58**Why:** Prevents premature expiry during slow sync operations or multi-step pushes. 58**Why:** Prevents premature expiry during slow sync operations or multi-step pushes. The protocol's 30-minute expiry is intended for abandoned repositories, not active ones receiving data.
59 59
60### 5. Authorization Must Check Purgatory Announcements 60### 5. Authorization Must Check Purgatory Announcements
61 61
@@ -75,20 +75,26 @@ This ensures we only serve announcements for repos that actually have content.
75 75
76### 7. Soft Expiry Preserves Event Without Bare Repo 76### 7. Soft Expiry Preserves Event Without Bare Repo
77 77
78**Decision:** When a purgatory announcement expires, delete the bare repo but retain the announcement event for an extended period (e.g., 24h). 78**Decision:** When a purgatory announcement expires (30 minutes per protocol spec), delete the bare repo but retain the announcement event for an extended period (e.g., 24h).
79 79
80**Why:** This handles the case where a state event arrives after initial expiry. Without soft expiry, we'd either: 80**Why the protocol specifies 30 minutes:** The grasp protocol defines a 30-minute expiry for announcement events to ensure clients don't indefinitely cache stale repository information.
81- Add to `failed_events` and reject the state event (losing potential revival) 81
82- Re-fetch the announcement repeatedly (wasting sync bandwidth) 82**Why we implement soft expiry:** The protocol's 30-minute expiry creates a sync/storage problem. Without soft expiry, we'd either:
83
84- Add expired announcements to `failed_events` and permanently reject future state events (losing potential revival when state events arrive late)
85- Re-fetch the announcement event repeatedly on every sync cycle (wasting bandwidth and creating unnecessary sync traffic)
83 86
84**Behavior during soft expiry:** 87**Behavior during soft expiry:**
85- Bare repo is deleted (saves disk space) 88
89- Bare repo is deleted (saves disk space, respects protocol expiry)
86- Announcement event retained in purgatory with `soft_expired` flag 90- Announcement event retained in purgatory with `soft_expired` flag
87- Sync continues requesting state events (same as active purgatory) 91- Sync continues requesting state events (same as active purgatory)
88- If state event arrives: recreate bare repo, clear `soft_expired`, extend expiry 92- If state event arrives: recreate bare repo, clear `soft_expired`, extend expiry
89- If announcement republished directly to us: treat as fresh arrival 93- If announcement republished directly to us: treat as fresh arrival
90- After extended expiry: fully remove from purgatory 94- After extended expiry: fully remove from purgatory
91 95
96**In summary:** Soft expiry is an implementation optimization that prevents us from constantly re-syncing announcement events or permanently blocking repositories that receive delayed state events.
97
92## Data Structure 98## Data Structure
93 99
94```rust 100```rust
@@ -198,29 +204,29 @@ Announcement ──> ACTIVE ─────────────────â
198 └── Extended expiry ──> REMOVED (exit) 204 └── Extended expiry ──> REMOVED (exit)
199``` 205```
200 206
201| Exit | Trigger | Action | 207| Exit | Trigger | Action |
202| -------------- | ---------------------------------------------- | -------------------------------------------- | 208| ------------------ | -------------------------------------------- | --------------------------------------------- |
203| **Promotion** | Git data arrives | Move to database, upgrade sync to Full | 209| **Promotion** | Git data arrives | Move to database, upgrade sync to Full |
204| **Soft expiry**| Initial timeout | Delete bare repo, retain event, continue sync| 210| **Soft expiry** | Initial timeout | Delete bare repo, retain event, continue sync |
205| **Full expiry**| Extended timeout (soft-expired) | Remove from purgatory entirely | 211| **Full expiry** | Extended timeout (soft-expired) | Remove from purgatory entirely |
206| **Deletion** | Kind 5 event | Delete bare repo, remove from purgatory | 212| **Deletion** | Kind 5 event | Delete bare repo, remove from purgatory |
207| **Replacement**| Newer announcement (same pubkey, identifier) | Replace entry | 213| **Replacement** | Newer announcement (same pubkey, identifier) | Replace entry |
208| **Service change** | Newer announcement removes our service | Remove from purgatory | 214| **Service change** | Newer announcement removes our service | Remove from purgatory |
209 215
210## Integration Points 216## Integration Points
211 217
212| File | Change | 218| File | Change |
213| ---------------------------------- | --------------------------------------------------------- | 219| ---------------------------------- | ---------------------------------------------------------- |
214| `src/purgatory/mod.rs` | Add `announcement_purgatory` store | 220| `src/purgatory/mod.rs` | Add `announcement_purgatory` store |
215| `src/purgatory/types.rs` | Add `AnnouncementPurgatoryEntry` | 221| `src/purgatory/types.rs` | Add `AnnouncementPurgatoryEntry` |
216| `src/nostr/policy/announcement.rs` | Route new announcements to purgatory | 222| `src/nostr/policy/announcement.rs` | Route new announcements to purgatory |
217| `src/git/receive.rs` | Promote on git data arrival | 223| `src/git/receive.rs` | Promote on git data arrival |
218| `src/git/auth.rs` | Extend purgatory expiry when extending state event expiry | 224| `src/git/auth.rs` | Extend purgatory expiry when extending state event expiry |
219| `src/git/authorization.rs` | Check purgatory announcements for maintainer authorization| 225| `src/git/authorization.rs` | Check purgatory announcements for maintainer authorization |
220| `src/nostr/policy/state.rs` | Check purgatory for authorization | 226| `src/nostr/policy/state.rs` | Check purgatory for authorization |
221| `src/sync/mod.rs` | Add `SyncLevel` to `RepoSyncNeeds` | 227| `src/sync/mod.rs` | Add `SyncLevel` to `RepoSyncNeeds` |
222| `src/sync/filters.rs` | Respect sync level when building filters | 228| `src/sync/filters.rs` | Respect sync level when building filters |
223| `src/sync/self_subscriber.rs` | Register purgatory announcements with `StateOnly` level | 229| `src/sync/self_subscriber.rs` | Register purgatory announcements with `StateOnly` level |
224 230
225See [announcements-purgatory-implementation.md](./announcements-purgatory-implementation.md) for detailed implementation notes. 231See [announcements-purgatory-implementation.md](./announcements-purgatory-implementation.md) for detailed implementation notes.
226 232
@@ -240,9 +246,9 @@ See [announcements-purgatory-implementation.md](./announcements-purgatory-implem
240 246
241## Risks 247## Risks
242 248
243| Risk | Mitigation | 249| Risk | Mitigation |
244| ------------------------------------ | ------------------------------------------------------- | 250| ------------------------------------ | ------------------------------------------------------ |
245| Disk exhaustion from purgatory repos | Short expiry, soft expiry deletes repo early | 251| Disk exhaustion from purgatory repos | Short expiry, soft expiry deletes repo early |
246| Race between promotion and expiry | Atomic operations | 252| Race between promotion and expiry | Atomic operations |
247| Sync re-fetching expired events | Soft expiry retains event; no need for `failed_events` | 253| Sync re-fetching expired events | Soft expiry retains event; no need for `failed_events` |
248| Filter explosion from many purgatory | Existing consolidation handles this (threshold at 70) | 254| Filter explosion from many purgatory | Existing consolidation handles this (threshold at 70) |
diff --git a/docs/explanation/announcements-purgatory-implementation.md b/docs/explanation/announcements-purgatory-implementation.md
index d5b8698..263c253 100644
--- a/docs/explanation/announcements-purgatory-implementation.md
+++ b/docs/explanation/announcements-purgatory-implementation.md
@@ -204,21 +204,22 @@ impl Purgatory {
204 .collect() 204 .collect()
205 } 205 }
206 206
207 /// Transition to soft-expired state 207 /// Transition to soft-expired state (protocol's 30min expiry reached)
208 pub fn soft_expire_announcement( 208 pub fn soft_expire_announcement(
209 &self, 209 &self,
210 key: &(PublicKey, String), 210 key: &(PublicKey, String),
211 ) -> Option<PathBuf> { 211 ) -> Option<PathBuf> {
212 if let Some(mut entry) = self.announcement_purgatory.get_mut(key) { 212 if let Some(mut entry) = self.announcement_purgatory.get_mut(key) {
213 entry.soft_expired = true; 213 entry.soft_expired = true;
214 entry.expires_at = Instant::now() + SOFT_EXPIRY_DURATION; // e.g., 24h 214 entry.expires_at = Instant::now() + SOFT_EXPIRY_DURATION; // e.g., 24h extended retention
215 Some(entry.repo_path.clone()) // Return path for bare repo deletion 215 Some(entry.repo_path.clone()) // Return path for bare repo deletion
216 } else { 216 } else {
217 None 217 None
218 } 218 }
219 } 219 }
220 220
221 /// Revive soft-expired announcement (caller must recreate bare repo) 221 /// Revive soft-expired announcement when state event arrives
222 /// (caller must recreate bare repo)
222 pub fn revive_announcement( 223 pub fn revive_announcement(
223 &self, 224 &self,
224 key: &(PublicKey, String), 225 key: &(PublicKey, String),
@@ -226,7 +227,7 @@ impl Purgatory {
226 if let Some(mut entry) = self.announcement_purgatory.get_mut(key) { 227 if let Some(mut entry) = self.announcement_purgatory.get_mut(key) {
227 if entry.soft_expired { 228 if entry.soft_expired {
228 entry.soft_expired = false; 229 entry.soft_expired = false;
229 entry.expires_at = Instant::now() + ACTIVE_EXPIRY_DURATION; 230 entry.expires_at = Instant::now() + ACTIVE_EXPIRY_DURATION; // Reset 30min protocol timer
230 return Some(entry.repo_path.clone()); // Caller recreates bare repo 231 return Some(entry.repo_path.clone()); // Caller recreates bare repo
231 } 232 }
232 } 233 }
@@ -270,9 +271,11 @@ When a state event arrives for a soft-expired announcement, the state policy mus
2701. Check purgatory for a matching announcement (in addition to DB) 2711. Check purgatory for a matching announcement (in addition to DB)
2712. Validate authorization against the purgatory announcement 2722. Validate authorization against the purgatory announcement
2723. If soft-expired, call `revive_announcement()` and recreate the bare repo 2733. If soft-expired, call `revive_announcement()` and recreate the bare repo
2734. Extend the announcement's expiry 2744. Extend the announcement's expiry (reset the 30-minute protocol timer)
2745. Route the state event to state purgatory 2755. Route the state event to state purgatory
275 276
277**Why revival is necessary:** Without soft expiry + revival, late-arriving state events would either be permanently rejected (if we added the announcement to `failed_events`) or cause constant re-syncing of the announcement event. Revival allows us to respect the protocol's 30-minute expiry while still handling delayed state events gracefully.
278
276The exact integration will depend on the current structure of `StatePolicy::process_state_event()` - see implementation phase for details. 279The exact integration will depend on the current structure of `StatePolicy::process_state_event()` - see implementation phase for details.
277 280
278## File Change Summary 281## File Change Summary