diff options
| author | DanConwayDev <DanConwayDev@protonmail.com> | 2026-02-05 15:25:14 +0000 |
|---|---|---|
| committer | DanConwayDev <DanConwayDev@protonmail.com> | 2026-02-05 15:25:14 +0000 |
| commit | 854484813dfe45f882fe66ff866621f9a21186fe (patch) | |
| tree | 4e5cb15e547bf25216de5bbc8362043cf8532955 /docs/explanation | |
| parent | 2be44c604062c7579e08c0d37b2f32ea8b6c4fcf (diff) | |
add notes to announcment purgatory design
Diffstat (limited to 'docs/explanation')
| -rw-r--r-- | docs/explanation/announcements-purgatory-design.md | 73 |
1 files changed, 40 insertions, 33 deletions
diff --git a/docs/explanation/announcements-purgatory-design.md b/docs/explanation/announcements-purgatory-design.md index c9077d9..4d0cc6d 100644 --- a/docs/explanation/announcements-purgatory-design.md +++ b/docs/explanation/announcements-purgatory-design.md | |||
| @@ -2,7 +2,7 @@ | |||
| 2 | 2 | ||
| 3 | ## Problem Statement | 3 | ## Problem Statement |
| 4 | 4 | ||
| 5 | **Primary problem:** Empty bare git repos mislead clients into thinking we host content. | 5 | **Primary problem:** serving an announcement event and also an empty bare git repos mislead clients into thinking we host content. |
| 6 | 6 | ||
| 7 | When an announcement arrives, we must create the bare repo immediately (so git pushes can succeed). But if no git data ever arrives, we serve an empty repo and its announcement indefinitely. Clients see the announcement, try to clone, and get nothing. This is misleading. | 7 | When an announcement arrives, we must create the bare repo immediately (so git pushes can succeed). But if no git data ever arrives, we serve an empty repo and its announcement indefinitely. Clients see the announcement, try to clone, and get nothing. This is misleading. |
| 8 | 8 | ||
| @@ -50,10 +50,10 @@ This ensures we only serve announcements for repos that actually have content. | |||
| 50 | 50 | ||
| 51 | **Decision:** Extend purgatory announcement expiry in two scenarios: | 51 | **Decision:** Extend purgatory announcement expiry in two scenarios: |
| 52 | 52 | ||
| 53 | | Trigger | Location | Why | | 53 | | Trigger | Location | Why | |
| 54 | |---------|----------|-----| | 54 | | ---------------------------- | ------------------------------------ | ----------------------------------- | |
| 55 | | State event arrives | `StatePolicy::process_state_event()` | Repo is actively receiving metadata | | 55 | | State event arrives | `StatePolicy::process_state_event()` | Repo is actively receiving metadata | |
| 56 | | Git auth extends state event | `src/git/auth.rs` | Repo is actively receiving git data | | 56 | | Git auth extends state event | `src/git/auth.rs` | Repo is actively receiving git data | |
| 57 | 57 | ||
| 58 | **Why:** Prevents premature expiry during slow sync operations or multi-step pushes. | 58 | **Why:** Prevents premature expiry during slow sync operations or multi-step pushes. |
| 59 | 59 | ||
| @@ -63,6 +63,10 @@ This ensures we only serve announcements for repos that actually have content. | |||
| 63 | 63 | ||
| 64 | **Why:** State events may arrive before git data promotes the announcement. They still need authorization from the announcement's maintainer set. | 64 | **Why:** State events may arrive before git data promotes the announcement. They still need authorization from the announcement's maintainer set. |
| 65 | 65 | ||
| 66 | ### 6. We need to request State Events in sysc for announcement in purgatory but not other l2 or l3 events because they will be rejected. | ||
| 67 | |||
| 68 | ### 7. When creating the authorised maintainers for a repositoriy we need to also get relivant announcement events from purgatory as well as db. | ||
| 69 | |||
| 66 | ## Data Structure | 70 | ## Data Structure |
| 67 | 71 | ||
| 68 | ```rust | 72 | ```rust |
| @@ -80,6 +84,7 @@ pub struct AnnouncementPurgatoryEntry { | |||
| 80 | ``` | 84 | ``` |
| 81 | 85 | ||
| 82 | **Indexed by `(pubkey, identifier)`** because identifier is not unique across different owners. | 86 | **Indexed by `(pubkey, identifier)`** because identifier is not unique across different owners. |
| 87 | question: would it be more efficent to index by repo_path? this contains the pubkey and identifier data? | ||
| 83 | 88 | ||
| 84 | ## Flows | 89 | ## Flows |
| 85 | 90 | ||
| @@ -133,38 +138,38 @@ Is there an active announcement? | |||
| 133 | 138 | ||
| 134 | ## Edge Cases | 139 | ## Edge Cases |
| 135 | 140 | ||
| 136 | | Scenario | Behavior | | 141 | | Scenario | Behavior | |
| 137 | |----------|----------| | 142 | | ------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- | |
| 138 | | Git data before announcement | Push fails (no repo exists) | | 143 | | Git data before announcement | Push fails (no repo exists) | |
| 139 | | Announcement expires, no git data | Delete bare repo, discard announcement | | 144 | | Announcement expires, no git data | Delete bare repo, discard announcement | |
| 140 | | State expires, announcement in purgatory | Announcement keeps its own expiry | | 145 | | State expires, announcement in purgatory | Announcement keeps its own expiry | |
| 141 | | Multiple owners, same identifier | Each tracked separately by `(pubkey, identifier)` | | 146 | | Multiple owners, same identifier | Each tracked separately by `(pubkey, identifier)` | |
| 142 | | **Newer announcement replaces older (same pubkey)** | Replace purgatory entry, extend expiry | | 147 | | **Newer announcement replaces older (same pubkey)** | Replace purgatory entry, extend expiry, and state event expiry | |
| 143 | | **Newer announcement changes services (unacceptable)** | Clear older announcement from purgatory for that `(pubkey, identifier)` | | 148 | | **Newer announcement changes services (unacceptable)** | Clear older announcement from purgatory for that `(pubkey, identifier)`, delete bare repo, remove state event for puragatory if exists | |
| 144 | | Deletion event for purgatory announcement | Remove from purgatory, delete bare repo | | 149 | | Deletion event for purgatory announcement | Remove from purgatory, delete bare repo | |
| 145 | 150 | ||
| 146 | ## Purgatory Exit Conditions | 151 | ## Purgatory Exit Conditions |
| 147 | 152 | ||
| 148 | An announcement leaves purgatory via: | 153 | An announcement leaves purgatory via: |
| 149 | 154 | ||
| 150 | | Exit | Trigger | Action | | 155 | | Exit | Trigger | Action | |
| 151 | |------|---------|--------| | 156 | | ------------------ | ---------------------------------------------- | ---------------------------------- | |
| 152 | | **Promotion** | Git data arrives | Move to database, serve to clients | | 157 | | **Promotion** | Git data arrives | Move to database, serve to clients | |
| 153 | | **Expiry** | Timeout | Delete bare repo, discard | | 158 | | **Expiry** | Timeout | Delete bare repo, discard | |
| 154 | | **Deletion** | Kind 5 event | Delete bare repo, discard | | 159 | | **Deletion** | Kind 5 event | Delete bare repo, discard | |
| 155 | | **Replacement** | Newer announcement (same pubkey, identifier) | Replace entry | | 160 | | **Replacement** | Newer announcement (same pubkey, identifier) | Replace entry | |
| 156 | | **Service change** | Newer announcement no longer lists our service | Discard old entry | | 161 | | **Service change** | Newer announcement no longer lists our service | Discard old entry | |
| 157 | 162 | ||
| 158 | ## Integration Points | 163 | ## Integration Points |
| 159 | 164 | ||
| 160 | | File | Change | | 165 | | File | Change | |
| 161 | |------|--------| | 166 | | ---------------------------------- | --------------------------------------------------------- | |
| 162 | | `src/purgatory/mod.rs` | Add `announcement_purgatory` store | | 167 | | `src/purgatory/mod.rs` | Add `announcement_purgatory` store | |
| 163 | | `src/purgatory/types.rs` | Add `AnnouncementPurgatoryEntry` | | 168 | | `src/purgatory/types.rs` | Add `AnnouncementPurgatoryEntry` | |
| 164 | | `src/nostr/policy/announcement.rs` | Route new announcements to purgatory | | 169 | | `src/nostr/policy/announcement.rs` | Route new announcements to purgatory | |
| 165 | | `src/git/receive.rs` | Promote on git data arrival | | 170 | | `src/git/receive.rs` | Promote on git data arrival | |
| 166 | | `src/git/auth.rs` | Extend purgatory expiry when extending state event expiry | | 171 | | `src/git/auth.rs` | Extend purgatory expiry when extending state event expiry | |
| 167 | | `src/nostr/policy/state.rs` | Check purgatory for authorization | | 172 | | `src/nostr/policy/state.rs` | Check purgatory for authorization | |
| 168 | 173 | ||
| 169 | ## Testing | 174 | ## Testing |
| 170 | 175 | ||
| @@ -178,8 +183,10 @@ An announcement leaves purgatory via: | |||
| 178 | 183 | ||
| 179 | ## Risks | 184 | ## Risks |
| 180 | 185 | ||
| 181 | | Risk | Mitigation | | 186 | | Risk | Mitigation | |
| 182 | |------|------------| | 187 | | ------------------------------------ | ------------------------------------ | |
| 183 | | Disk exhaustion from purgatory repos | Short expiry, monitor purgatory size | | 188 | | Disk exhaustion from purgatory repos | Short expiry, monitor purgatory size | |
| 184 | | Race between promotion and expiry | Atomic operations | | 189 | | Race between promotion and expiry | Atomic operations | |
| 185 | | Sync re-fetching expired events | Track expired event IDs | | 190 | | Sync re-fetching expired events | Track expired event IDs | |
| 191 | |||
| 192 | question: do expired annoucements go on failed_events list? what if a new state event comes in for it? surely then we want it again? but if not we dont want to keep donwloading it and havea a repo made for it. Should we have a longer period were we keep the event just in case, but delete the bare repo and only remake it when the state event arrives? | ||