diff options
Diffstat (limited to 'docs/explanation/purgatory-design.md')
| -rw-r--r-- | docs/explanation/purgatory-design.md | 901 |
1 files changed, 901 insertions, 0 deletions
diff --git a/docs/explanation/purgatory-design.md b/docs/explanation/purgatory-design.md new file mode 100644 index 0000000..bf867ad --- /dev/null +++ b/docs/explanation/purgatory-design.md | |||
| @@ -0,0 +1,901 @@ | |||
| 1 | # Purgatory Implementation Design | ||
| 2 | |||
| 3 | ## Overview | ||
| 4 | |||
| 5 | Purgatory is an in-memory holding area for nostr events that depend on git data that hasn't arrived yet, **and** for git data that arrived before its corresponding nostr event. Events/placeholders are held until the other half arrives, at which point they are processed and saved to the database. | ||
| 6 | |||
| 7 | **Spec Reference**: [GRASP-01 Purgatory Section](../grasp/01.md:20-22) | ||
| 8 | |||
| 9 | > Accepted repo state announcements, PRs and PR Updates SHOULD be accepted with message "purgatory: won't be served until git data arrives" and kept in purgatory (not served) until the related git data arrives and otherwise discarded after 30 minutes. | ||
| 10 | |||
| 11 | ## Key Design Principles | ||
| 12 | |||
| 13 | ### 1. Separate Storage for State vs PR Events | ||
| 14 | |||
| 15 | State events (kind 30618) and PR events (kind 1617/1618) have fundamentally different matching and authorization patterns. They are stored in **separate purgatory stores** with different indices: | ||
| 16 | |||
| 17 | - **State Events**: Indexed by `identifier` (d tag), matched via ref comparison | ||
| 18 | - **PR Events**: Indexed by `event_id`, matched directly via `refs/nostr/<event-id>` | ||
| 19 | |||
| 20 | ### 2. Late Binding of Refs (State Events) | ||
| 21 | |||
| 22 | **Do NOT extract refs at event arrival time.** Extract and match refs at git push time. | ||
| 23 | |||
| 24 | **Why?** Multiple state events might be in purgatory with different target states. An older state event's git data might arrive after a newer one is received. By waiting until push time, we: | ||
| 25 | |||
| 26 | - Compare the pushed refs against each purgatory state event's expected state | ||
| 27 | - Handle out-of-order git data arrival correctly | ||
| 28 | - Only release events when their specific target state is achieved | ||
| 29 | |||
| 30 | ### 3. Bidirectional Waiting (PR Events) | ||
| 31 | |||
| 32 | For PR events, **either side can arrive first**: | ||
| 33 | |||
| 34 | - **Event first**: PR event waits in purgatory for git push to `refs/nostr/<event-id>` | ||
| 35 | - **Git first**: Push creates placeholder, waits for PR event to arrive | ||
| 36 | |||
| 37 | ### 4. Ref Pairs, Not Just Commits | ||
| 38 | |||
| 39 | State events define a target state: specific refs pointing to specific objects (commits or annotated tags). We match **ref name + object SHA** pairs, not just raw commits. | ||
| 40 | |||
| 41 | ### 5. No Separate PurgatoryState Tracking | ||
| 42 | |||
| 43 | All entries use a single 30-minute expiry timer. When git push activity begins, we simply ensure at least 15 minutes remain on the timer (extend if needed). This eliminates complexity of tracking Secured vs Pending states. | ||
| 44 | |||
| 45 | ## Event Lifecycle | ||
| 46 | |||
| 47 | ```mermaid | ||
| 48 | stateDiagram-v2 | ||
| 49 | [*] --> Waiting: Event or git data arrives | ||
| 50 | Waiting --> Processing: Other half arrives | ||
| 51 | Waiting --> Discarded: 30 min expiry | ||
| 52 | Processing --> Released: Match verified + authorized | ||
| 53 | Processing --> Rejected: Mismatch or unauthorized | ||
| 54 | Released --> [*]: Event saved to database | ||
| 55 | Rejected --> Waiting: Return to waiting - different match expected | ||
| 56 | Discarded --> [*]: Entry dropped | ||
| 57 | ``` | ||
| 58 | |||
| 59 | ## Data Structures | ||
| 60 | |||
| 61 | ### RefPair - A single ref target | ||
| 62 | |||
| 63 | ```rust | ||
| 64 | /// A reference name and its target object | ||
| 65 | #[derive(Debug, Clone, Hash, Eq, PartialEq)] | ||
| 66 | pub struct RefPair { | ||
| 67 | /// Full ref name, e.g., "refs/heads/main" or "refs/tags/v1.0" | ||
| 68 | pub ref_name: String, | ||
| 69 | /// Target object SHA (commit or annotated tag) | ||
| 70 | pub object_sha: String, | ||
| 71 | } | ||
| 72 | ``` | ||
| 73 | |||
| 74 | ### StatePurgatoryEntry | ||
| 75 | |||
| 76 | ```rust | ||
| 77 | pub struct StatePurgatoryEntry { | ||
| 78 | /// The nostr state event (kind 30618) awaiting git data | ||
| 79 | pub event: Event, | ||
| 80 | |||
| 81 | /// The repository identifier from the event's 'd' tag | ||
| 82 | pub identifier: String, | ||
| 83 | |||
| 84 | /// Event author pubkey | ||
| 85 | pub author: PublicKey, | ||
| 86 | |||
| 87 | /// When this entry was added to purgatory | ||
| 88 | pub created_at: Instant, | ||
| 89 | |||
| 90 | /// Expiry deadline (30 min from creation, may be extended) | ||
| 91 | pub expires_at: Instant, | ||
| 92 | } | ||
| 93 | ``` | ||
| 94 | |||
| 95 | ### PrPurgatoryEntry | ||
| 96 | |||
| 97 | ```rust | ||
| 98 | pub struct PrPurgatoryEntry { | ||
| 99 | /// The nostr PR event, if received (None = git data arrived first) | ||
| 100 | pub event: Option<Event>, | ||
| 101 | |||
| 102 | /// The expected commit SHA from 'c' tag (if event exists) | ||
| 103 | /// or the actual commit pushed (if git arrived first) | ||
| 104 | pub commit: String, | ||
| 105 | |||
| 106 | /// When this entry was added to purgatory | ||
| 107 | pub created_at: Instant, | ||
| 108 | |||
| 109 | /// Expiry deadline (30 min from creation, may be extended) | ||
| 110 | pub expires_at: Instant, | ||
| 111 | } | ||
| 112 | ``` | ||
| 113 | |||
| 114 | **Note:** `PrPurgatoryEntry.event` being `None` indicates the "git data first" scenario - we have a placeholder waiting for the PR event. | ||
| 115 | |||
| 116 | ### Purgatory Stores | ||
| 117 | |||
| 118 | ```rust | ||
| 119 | pub struct Purgatory { | ||
| 120 | /// State events indexed by identifier | ||
| 121 | state_events: DashMap<String, Vec<StatePurgatoryEntry>>, | ||
| 122 | |||
| 123 | /// PR events indexed by event_id (hex string) | ||
| 124 | pr_events: DashMap<String, PrPurgatoryEntry>, | ||
| 125 | } | ||
| 126 | ``` | ||
| 127 | |||
| 128 | ## Event Flows | ||
| 129 | |||
| 130 | ### State Event (Kind 30618) Arrival | ||
| 131 | |||
| 132 | ```mermaid | ||
| 133 | sequenceDiagram | ||
| 134 | participant Client | ||
| 135 | participant WritePolicy | ||
| 136 | participant Purgatory | ||
| 137 | participant GitRepos | ||
| 138 | |||
| 139 | Client->>WritePolicy: EVENT kind:30618 | ||
| 140 | WritePolicy->>WritePolicy: Validate structure | ||
| 141 | WritePolicy->>WritePolicy: Parse identifier and author | ||
| 142 | |||
| 143 | Note right of WritePolicy: Check if we already have git data | ||
| 144 | WritePolicy->>GitRepos: Check if any authorized repo has matching refs | ||
| 145 | |||
| 146 | alt Git data already exists | ||
| 147 | GitRepos-->>WritePolicy: Refs match in repo X | ||
| 148 | WritePolicy->>WritePolicy: Process immediately | ||
| 149 | WritePolicy->>Client: OK true - event saved | ||
| 150 | else Git data not available yet | ||
| 151 | WritePolicy->>Purgatory: add_state - event, identifier | ||
| 152 | Purgatory->>Purgatory: Add entry indexed by identifier | ||
| 153 | WritePolicy->>Client: OK true purgatory: will not be served until git data arrives | ||
| 154 | end | ||
| 155 | ``` | ||
| 156 | |||
| 157 | ### PR Event (Kind 1617/1618) Arrival | ||
| 158 | |||
| 159 | ```mermaid | ||
| 160 | sequenceDiagram | ||
| 161 | participant Client | ||
| 162 | participant WritePolicy | ||
| 163 | participant Purgatory | ||
| 164 | participant GitRepos | ||
| 165 | |||
| 166 | Client->>WritePolicy: EVENT kind:1617 | ||
| 167 | WritePolicy->>WritePolicy: Validate structure | ||
| 168 | WritePolicy->>WritePolicy: Extract event_id and commit from c tag | ||
| 169 | |||
| 170 | Note right of WritePolicy: Check if git data already exists | ||
| 171 | WritePolicy->>GitRepos: Check refs/nostr/event-id in authorized repos | ||
| 172 | |||
| 173 | alt Git data already exists | ||
| 174 | GitRepos-->>WritePolicy: Found matching commit | ||
| 175 | WritePolicy->>WritePolicy: Process immediately | ||
| 176 | WritePolicy->>Client: OK true - event saved | ||
| 177 | else Git data arrived first - placeholder exists | ||
| 178 | WritePolicy->>Purgatory: find_pr_placeholder - event_id | ||
| 179 | alt Placeholder found with matching commit | ||
| 180 | Purgatory-->>WritePolicy: Placeholder entry | ||
| 181 | WritePolicy->>WritePolicy: Process - save event | ||
| 182 | WritePolicy->>Purgatory: remove - event_id | ||
| 183 | WritePolicy->>Client: OK true - event saved | ||
| 184 | else Placeholder found with different commit | ||
| 185 | WritePolicy->>Client: OK false - commit mismatch | ||
| 186 | else No placeholder | ||
| 187 | WritePolicy->>Purgatory: add_pr - event, commit | ||
| 188 | WritePolicy->>Client: OK true purgatory: will not be served until git data arrives | ||
| 189 | end | ||
| 190 | end | ||
| 191 | ``` | ||
| 192 | |||
| 193 | ### Git Push - State Event Matching | ||
| 194 | |||
| 195 | When a push arrives to normal refs (branches/tags), we check for matching state events: | ||
| 196 | |||
| 197 | **Key Rule**: For a given `pubkey/identifier` repo, there can only be **one authoritative state event** - the one with the largest `created_at` among all authorized maintainers. State events in purgatory are only processed if they aren't superseded by existing state events on the relay. | ||
| 198 | |||
| 199 | ```mermaid | ||
| 200 | sequenceDiagram | ||
| 201 | participant GitClient | ||
| 202 | participant GitHandler | ||
| 203 | participant Purgatory | ||
| 204 | participant Database | ||
| 205 | participant GitRepo | ||
| 206 | |||
| 207 | GitClient->>GitHandler: POST /npub/identifier/git-receive-pack | ||
| 208 | GitHandler->>GitHandler: Parse pushed refs into RefPairs | ||
| 209 | |||
| 210 | Note over GitHandler: Look up state events by identifier | ||
| 211 | GitHandler->>Purgatory: find_matching_states - identifier, pushed_refs | ||
| 212 | |||
| 213 | loop For each state event in purgatory[identifier] | ||
| 214 | Purgatory->>Purgatory: Parse state event refs | ||
| 215 | Purgatory->>Purgatory: Check: pushed_refs covers event refs that differ from local | ||
| 216 | Purgatory->>Purgatory: Check: event refs not in push already exist locally | ||
| 217 | alt All event refs can be satisfied | ||
| 218 | Purgatory->>Purgatory: Add to candidates | ||
| 219 | end | ||
| 220 | end | ||
| 221 | |||
| 222 | Purgatory-->>GitHandler: Vec of candidate state events | ||
| 223 | |||
| 224 | Note over GitHandler: Get all state events and announcements for identifier | ||
| 225 | GitHandler->>Database: Get announcements for identifier | ||
| 226 | GitHandler->>Database: Get all state events for identifier from relay | ||
| 227 | GitHandler->>GitHandler: collect_authorized_maintainers from announcements | ||
| 228 | |||
| 229 | Note over GitHandler: Find authoritative state event | ||
| 230 | loop For each candidate from purgatory | ||
| 231 | GitHandler->>GitHandler: Check if event.author is authorized for target repo | ||
| 232 | alt Author authorized | ||
| 233 | GitHandler->>GitHandler: Check relay state events for this identifier | ||
| 234 | GitHandler->>GitHandler: Compare created_at with all authorized state events | ||
| 235 | alt This is the largest created_at among authorized | ||
| 236 | GitHandler->>GitHandler: Verify OIDs for unpushed refs exist locally | ||
| 237 | alt All OIDs available | ||
| 238 | GitHandler->>GitHandler: Set as approved state event | ||
| 239 | end | ||
| 240 | else Superseded by existing state on relay | ||
| 241 | GitHandler->>Purgatory: Remove superseded event | ||
| 242 | end | ||
| 243 | end | ||
| 244 | end | ||
| 245 | |||
| 246 | Note over GitHandler: Only ONE approved state event per repo | ||
| 247 | alt Approved state event exists | ||
| 248 | GitHandler->>Purgatory: extend_expiry - event_id - ensure 15 min | ||
| 249 | GitHandler->>GitRepo: Execute git-receive-pack | ||
| 250 | |||
| 251 | alt Push successful | ||
| 252 | GitHandler->>Database: Save state event | ||
| 253 | GitHandler->>GitRepo: Align ALL refs to state - including unpushed | ||
| 254 | GitHandler->>GitHandler: Sync to other authorized maintainer repos | ||
| 255 | GitHandler->>Purgatory: remove - event_id | ||
| 256 | GitHandler->>GitClient: Push accepted | ||
| 257 | else Push failed | ||
| 258 | GitHandler->>GitClient: Push rejected | ||
| 259 | end | ||
| 260 | else No approved state event | ||
| 261 | GitHandler->>GitClient: Push rejected - no authorized state | ||
| 262 | end | ||
| 263 | ``` | ||
| 264 | |||
| 265 | ### Git Push - PR Event (refs/nostr/event-id) | ||
| 266 | |||
| 267 | ```mermaid | ||
| 268 | sequenceDiagram | ||
| 269 | participant GitClient | ||
| 270 | participant GitHandler | ||
| 271 | participant Database | ||
| 272 | participant Purgatory | ||
| 273 | participant GitRepo | ||
| 274 | |||
| 275 | GitClient->>GitHandler: POST /npub/identifier/git-receive-pack refs/nostr/abc123 | ||
| 276 | GitHandler->>GitHandler: Extract event_id from ref | ||
| 277 | GitHandler->>GitHandler: Extract pushed commit SHA | ||
| 278 | |||
| 279 | Note over GitHandler: First check relay database | ||
| 280 | GitHandler->>Database: Query for PR event with event_id | ||
| 281 | |||
| 282 | alt Event exists in database | ||
| 283 | Database-->>GitHandler: PR event found | ||
| 284 | GitHandler->>GitHandler: Compare commit tags | ||
| 285 | alt Commit matches | ||
| 286 | GitHandler->>GitRepo: Execute push | ||
| 287 | GitHandler->>GitClient: Push accepted | ||
| 288 | else Commit mismatch | ||
| 289 | GitHandler->>GitClient: Push rejected - commit mismatch with existing event | ||
| 290 | end | ||
| 291 | else Event not in database | ||
| 292 | GitHandler->>Purgatory: find_pr - event_id | ||
| 293 | |||
| 294 | alt PR event in purgatory | ||
| 295 | Purgatory-->>GitHandler: PR entry with event | ||
| 296 | GitHandler->>GitHandler: Compare commit tags | ||
| 297 | alt Commit matches | ||
| 298 | GitHandler->>GitRepo: Execute push | ||
| 299 | GitHandler->>Database: Save PR event | ||
| 300 | GitHandler->>GitHandler: Sync to other authorized repos | ||
| 301 | GitHandler->>Purgatory: remove - event_id | ||
| 302 | GitHandler->>GitClient: Push accepted | ||
| 303 | else Commit mismatch | ||
| 304 | GitHandler->>GitClient: Push rejected - commit mismatch | ||
| 305 | end | ||
| 306 | else No PR event anywhere | ||
| 307 | Note over GitHandler: Git data first scenario | ||
| 308 | GitHandler->>GitRepo: Execute push - accept any data | ||
| 309 | GitHandler->>Purgatory: add_pr_placeholder - event_id, commit | ||
| 310 | GitHandler->>GitClient: Push accepted - awaiting PR event | ||
| 311 | end | ||
| 312 | end | ||
| 313 | ``` | ||
| 314 | |||
| 315 | ### Sync to Other Maintainer Repos | ||
| 316 | |||
| 317 | After successfully processing a state event or PR, we sync to other authorized repos: | ||
| 318 | |||
| 319 | ```mermaid | ||
| 320 | sequenceDiagram | ||
| 321 | participant Handler | ||
| 322 | participant Database | ||
| 323 | participant Authorization | ||
| 324 | participant GitRepos | ||
| 325 | |||
| 326 | Handler->>Database: Get all announcements for identifier | ||
| 327 | Handler->>Authorization: collect_authorized_maintainers - announcements | ||
| 328 | Authorization-->>Handler: Map of owner -> authorized maintainers | ||
| 329 | |||
| 330 | loop For each owner in map | ||
| 331 | alt Event author in owner's authorized set | ||
| 332 | Note right of Handler: This owner's repo should have this state/PR | ||
| 333 | |||
| 334 | alt State event | ||
| 335 | Handler->>GitRepos: Align owner's repo to state event refs | ||
| 336 | else PR event | ||
| 337 | Handler->>GitRepos: Ensure refs/nostr/event-id exists | ||
| 338 | end | ||
| 339 | end | ||
| 340 | end | ||
| 341 | ``` | ||
| 342 | |||
| 343 | ### Background Cleanup | ||
| 344 | |||
| 345 | ```mermaid | ||
| 346 | sequenceDiagram | ||
| 347 | participant Timer | ||
| 348 | participant Purgatory | ||
| 349 | |||
| 350 | loop Every 60 seconds | ||
| 351 | Timer->>Purgatory: cleanup | ||
| 352 | |||
| 353 | Note over Purgatory: Clean state events | ||
| 354 | loop For each identifier in state_events | ||
| 355 | loop For each entry | ||
| 356 | alt now > expires_at | ||
| 357 | Purgatory->>Purgatory: Remove entry | ||
| 358 | end | ||
| 359 | end | ||
| 360 | end | ||
| 361 | |||
| 362 | Note over Purgatory: Clean PR events | ||
| 363 | loop For each event_id in pr_events | ||
| 364 | alt now > entry.expires_at | ||
| 365 | Purgatory->>Purgatory: Remove entry | ||
| 366 | end | ||
| 367 | end | ||
| 368 | end | ||
| 369 | ``` | ||
| 370 | |||
| 371 | ## API Methods | ||
| 372 | |||
| 373 | ### Purgatory | ||
| 374 | |||
| 375 | ```rust | ||
| 376 | impl Purgatory { | ||
| 377 | /// Create a new empty purgatory | ||
| 378 | pub fn new() -> Self; | ||
| 379 | |||
| 380 | // ==================== State Events ==================== | ||
| 381 | |||
| 382 | /// Add a state event (kind 30618) to purgatory | ||
| 383 | /// Returns purgatory message for client response | ||
| 384 | pub fn add_state(&self, event: Event, identifier: String) -> String; | ||
| 385 | |||
| 386 | /// Find state events that could be satisfied by pushed refs | ||
| 387 | /// Returns events where: | ||
| 388 | /// - All refs in event are either in pushed_refs OR already exist locally | ||
| 389 | /// - At least one ref in event is in pushed_refs (something to update) | ||
| 390 | pub fn find_matching_states( | ||
| 391 | &self, | ||
| 392 | identifier: &str, | ||
| 393 | pushed_refs: &[RefPair], | ||
| 394 | local_refs: &HashMap<String, String>, | ||
| 395 | ) -> Vec<Event>; | ||
| 396 | |||
| 397 | /// Extend expiry for entries about to be processed | ||
| 398 | /// Ensures at least `duration` remaining on timer | ||
| 399 | pub fn extend_expiry(&self, event_ids: &[EventId], duration: Duration); | ||
| 400 | |||
| 401 | /// Remove state event after successful processing | ||
| 402 | pub fn remove_state(&self, event_id: &EventId); | ||
| 403 | |||
| 404 | // ==================== PR Events ==================== | ||
| 405 | |||
| 406 | /// Add a PR event (kind 1617/1618) to purgatory | ||
| 407 | /// Returns purgatory message for client response | ||
| 408 | pub fn add_pr(&self, event: Event, commit: String) -> String; | ||
| 409 | |||
| 410 | /// Add a placeholder for git-data-first scenario | ||
| 411 | /// Called when push to refs/nostr/<event-id> arrives before the PR event | ||
| 412 | pub fn add_pr_placeholder(&self, event_id: EventId, commit: String); | ||
| 413 | |||
| 414 | /// Find PR entry by event ID | ||
| 415 | /// Returns the entry if found (may or may not have event) | ||
| 416 | pub fn find_pr(&self, event_id: &EventId) -> Option<PrPurgatoryEntry>; | ||
| 417 | |||
| 418 | /// Find PR placeholder (git-data-first entry without event) | ||
| 419 | pub fn find_pr_placeholder(&self, event_id: &EventId) -> Option<PrPurgatoryEntry>; | ||
| 420 | |||
| 421 | /// Remove PR entry after successful processing | ||
| 422 | pub fn remove_pr(&self, event_id: &EventId); | ||
| 423 | |||
| 424 | // ==================== Maintenance ==================== | ||
| 425 | |||
| 426 | /// Remove expired entries (30 min) | ||
| 427 | /// Returns count of removed entries | ||
| 428 | pub fn cleanup(&self) -> usize; | ||
| 429 | |||
| 430 | /// Get counts for metrics/debugging | ||
| 431 | pub fn state_event_count(&self) -> usize; | ||
| 432 | pub fn pr_event_count(&self) -> usize; | ||
| 433 | pub fn pr_placeholder_count(&self) -> usize; | ||
| 434 | |||
| 435 | /// Check if an event is in purgatory (either store) | ||
| 436 | pub fn contains(&self, event_id: &EventId) -> bool; | ||
| 437 | } | ||
| 438 | ``` | ||
| 439 | |||
| 440 | ### Helper: Extract and Match Refs | ||
| 441 | |||
| 442 | ```rust | ||
| 443 | /// Extract ref pairs from a state event | ||
| 444 | pub fn extract_refs_from_state(event: &Event) -> Vec<RefPair> { | ||
| 445 | // Parse refs/heads/* and refs/tags/* tags | ||
| 446 | // Return vec of RefPair { ref_name, object_sha } | ||
| 447 | } | ||
| 448 | |||
| 449 | /// Check if a state event can be satisfied by a push | ||
| 450 | /// | ||
| 451 | /// Returns true if: | ||
| 452 | /// - Every ref in state_refs is either in pushed_refs (matching SHA) OR in local_refs (matching SHA) | ||
| 453 | /// - At least one ref in state_refs is actually being changed by the push | ||
| 454 | pub fn can_satisfy_state( | ||
| 455 | state_refs: &[RefPair], | ||
| 456 | pushed_refs: &[RefPair], | ||
| 457 | local_refs: &HashMap<String, String>, | ||
| 458 | ) -> bool; | ||
| 459 | |||
| 460 | /// Get refs from state event that aren't being pushed but need updating | ||
| 461 | pub fn get_unpushed_refs( | ||
| 462 | state_refs: &[RefPair], | ||
| 463 | pushed_refs: &[RefPair], | ||
| 464 | ) -> Vec<RefPair>; | ||
| 465 | |||
| 466 | /// Verify all OIDs from refs exist in the local git repo | ||
| 467 | pub fn verify_oids_exist( | ||
| 468 | repo_path: &Path, | ||
| 469 | refs: &[RefPair], | ||
| 470 | ) -> Result<bool, git::Error>; | ||
| 471 | ``` | ||
| 472 | |||
| 473 | ## Integration Points | ||
| 474 | |||
| 475 | ### 1. Nip34WritePolicy Changes | ||
| 476 | |||
| 477 | ```rust | ||
| 478 | pub struct Nip34WritePolicy { | ||
| 479 | ctx: PolicyContext, | ||
| 480 | purgatory: Arc<Purgatory>, // Shared with git handlers | ||
| 481 | // ... existing fields | ||
| 482 | } | ||
| 483 | ``` | ||
| 484 | |||
| 485 | ### 2. handle_state Changes | ||
| 486 | |||
| 487 | On state event arrival: | ||
| 488 | |||
| 489 | **Key Rules**: | ||
| 490 | |||
| 491 | 1. Reject if we already have a state event from this author for this identifier with a larger `created_at` date (outdated event) | ||
| 492 | 2. If accepted, check if we need to sync repos with the same identifier | ||
| 493 | |||
| 494 | ```rust | ||
| 495 | async fn handle_state(&self, event: &Event) -> WritePolicyResult { | ||
| 496 | let identifier = extract_identifier(&event)?; | ||
| 497 | let author = event.pubkey; | ||
| 498 | let state_refs = extract_refs_from_state(&event); | ||
| 499 | |||
| 500 | // Check for existing state event from this author with larger created_at | ||
| 501 | let existing_states = self.database.get_state_events_by_author_identifier( | ||
| 502 | &author, | ||
| 503 | &identifier | ||
| 504 | ).await?; | ||
| 505 | |||
| 506 | for existing in existing_states { | ||
| 507 | if existing.created_at > event.created_at { | ||
| 508 | // Reject - we have a newer state from this author | ||
| 509 | return WritePolicyResult::Reject { | ||
| 510 | status: false, | ||
| 511 | message: "rejected: newer state event exists for this author/identifier".into() | ||
| 512 | }; | ||
| 513 | } | ||
| 514 | } | ||
| 515 | |||
| 516 | // Check if we already have matching git data | ||
| 517 | let repos = self.find_repos_for_identifier(&identifier).await?; | ||
| 518 | for repo in repos { | ||
| 519 | if self.refs_match_state(&repo, &state_refs).await? { | ||
| 520 | // Git data exists - process immediately | ||
| 521 | // Also trigger sync check for other repos with same identifier | ||
| 522 | // Pass the repo that has the git data so it can be used as source | ||
| 523 | self.check_and_sync_repos_for_identifier(&identifier, &event, &repo).await?; | ||
| 524 | return WritePolicyResult::Accept; | ||
| 525 | } | ||
| 526 | } | ||
| 527 | |||
| 528 | // Add to purgatory | ||
| 529 | let msg = self.purgatory.add_state(event.clone(), identifier); | ||
| 530 | WritePolicyResult::Reject { | ||
| 531 | status: true, // Client sees OK | ||
| 532 | message: msg.into() | ||
| 533 | } | ||
| 534 | } | ||
| 535 | ``` | ||
| 536 | |||
| 537 | ### 3. handle_pr_event Changes | ||
| 538 | |||
| 539 | On PR event arrival: | ||
| 540 | |||
| 541 | **Key Rule**: Incoming PR events supersede existing refs. If the existing `refs/nostr/<event-id>` ref has a different commit_id, the ref should be removed and the event stored in purgatory to await new git data. | ||
| 542 | |||
| 543 | ```rust | ||
| 544 | async fn handle_pr_event(&self, event: &Event) -> WritePolicyResult { | ||
| 545 | let commit = extract_c_tag_commit(&event)?; | ||
| 546 | let event_id = event.id.to_hex(); | ||
| 547 | |||
| 548 | // Check if placeholder exists (git-data-first) | ||
| 549 | if let Some(placeholder) = self.purgatory.find_pr_placeholder(&event.id) { | ||
| 550 | if placeholder.commit == commit { | ||
| 551 | self.purgatory.remove_pr(&event.id); // Note this shouldnt remove the git data | ||
| 552 | return WritePolicyResult::Accept; | ||
| 553 | } else { | ||
| 554 | // Placeholder has different commit - incoming event supersedes | ||
| 555 | // Update placeholder with new expected commit | ||
| 556 | self.purgatory.remove_pr(&event.id); | ||
| 557 | // TODO also remove git data | ||
| 558 | let msg = self.purgatory.add_pr(event.clone(), commit); | ||
| 559 | return WritePolicyResult::Reject { | ||
| 560 | status: true, // Client sees OK - in purgatory awaiting correct git data | ||
| 561 | message: msg.into() | ||
| 562 | }; | ||
| 563 | } | ||
| 564 | } | ||
| 565 | |||
| 566 | // Add to purgatory | ||
| 567 | let msg = self.purgatory.add_pr(event.clone(), commit); | ||
| 568 | WritePolicyResult::Reject { | ||
| 569 | status: true, | ||
| 570 | message: msg.into() | ||
| 571 | } | ||
| 572 | } | ||
| 573 | ``` | ||
| 574 | |||
| 575 | ### 4. Git Handler Changes | ||
| 576 | |||
| 577 | In `handle_receive_pack` for normal refs: | ||
| 578 | |||
| 579 | **Key Rule**: Refs are sent to a specific repo (`pubkey/identifier`), and there can only be **one authorized state event** for that repo. Therefore, this function returns a single `Option<Event>` rather than `Vec<Event>`. | ||
| 580 | |||
| 581 | ```rust | ||
| 582 | async fn handle_state_refs_push( | ||
| 583 | &self, | ||
| 584 | identifier: &str, | ||
| 585 | repo_owner: &str, | ||
| 586 | pushed_refs: &[RefPair], | ||
| 587 | ) -> Result<Option<Event>, GitError> { | ||
| 588 | let local_refs = git::list_refs(&self.repo_path)?; | ||
| 589 | |||
| 590 | // Find matching state events | ||
| 591 | let candidates = self.purgatory.find_matching_states( | ||
| 592 | identifier, | ||
| 593 | pushed_refs, | ||
| 594 | &local_refs, | ||
| 595 | ); | ||
| 596 | |||
| 597 | // Get all state events from relay for this identifier | ||
| 598 | let relay_states = self.database.get_state_events_for_identifier(identifier).await?; | ||
| 599 | |||
| 600 | // Get announcements to determine authorization | ||
| 601 | let announcements = self.database.get_announcements(identifier).await?; | ||
| 602 | let auth_map = collect_authorized_maintainers(&announcements); | ||
| 603 | |||
| 604 | // Find the authoritative state event (largest created_at among all authorized) | ||
| 605 | let mut best_candidate: Option<(Event, Timestamp)> = None; | ||
| 606 | |||
| 607 | // Check purgatory candidates | ||
| 608 | for event in candidates { | ||
| 609 | if let Some(maintainers) = auth_map.get(repo_owner) { | ||
| 610 | if maintainers.contains(&event.pubkey.to_hex()) { | ||
| 611 | // Check if this event is superseded by any relay state | ||
| 612 | let superseded = relay_states.iter().any(|relay_state| { | ||
| 613 | let relay_authorized = auth_map.values() | ||
| 614 | .any(|m| m.contains(&relay_state.pubkey.to_hex())); | ||
| 615 | relay_authorized && relay_state.created_at > event.created_at | ||
| 616 | }); | ||
| 617 | |||
| 618 | if superseded { | ||
| 619 | // Don't use this as best_candidate for THIS repo | ||
| 620 | // Note: Do NOT remove from purgatory - it may still be | ||
| 621 | // authoritative for a DIFFERENT repo with a different maintainer set | ||
| 622 | continue; | ||
| 623 | } | ||
| 624 | |||
| 625 | // Verify OIDs for unpushed refs exist | ||
| 626 | let state_refs = extract_refs_from_state(&event); | ||
| 627 | let unpushed = get_unpushed_refs(&state_refs, pushed_refs); | ||
| 628 | if verify_oids_exist(&self.repo_path, &unpushed)? { | ||
| 629 | // Track best candidate by created_at | ||
| 630 | if best_candidate.is_none() || event.created_at > best_candidate.as_ref().unwrap().1 { | ||
| 631 | best_candidate = Some((event, event.created_at)); | ||
| 632 | } | ||
| 633 | } | ||
| 634 | } | ||
| 635 | } | ||
| 636 | } | ||
| 637 | |||
| 638 | // If we found an approved event, extend its expiry | ||
| 639 | if let Some((ref event, _)) = best_candidate { | ||
| 640 | self.purgatory.extend_expiry(&[event.id], Duration::from_secs(900)); | ||
| 641 | } | ||
| 642 | |||
| 643 | Ok(best_candidate.map(|(e, _)| e)) | ||
| 644 | } | ||
| 645 | ``` | ||
| 646 | |||
| 647 | For `refs/nostr/<event-id>` pushes: | ||
| 648 | |||
| 649 | **Key Rule**: If there is no event (neither in database nor in purgatory), a push of a different commit_id should be **accepted**. This is the "git-data-first" scenario where we're waiting for the PR event to arrive. | ||
| 650 | |||
| 651 | ```rust | ||
| 652 | async fn handle_nostr_ref_push( | ||
| 653 | &self, | ||
| 654 | event_id: &str, | ||
| 655 | pushed_commit: &str, | ||
| 656 | ) -> Result<PushDecision, GitError> { | ||
| 657 | // Check database first | ||
| 658 | if let Some(event) = self.database.get_event_by_id(event_id).await? { | ||
| 659 | let expected_commit = extract_c_tag_commit(&event)?; | ||
| 660 | if expected_commit == pushed_commit { | ||
| 661 | return Ok(PushDecision::Accept); | ||
| 662 | } else { | ||
| 663 | return Ok(PushDecision::Reject("commit mismatch with existing event")); | ||
| 664 | } | ||
| 665 | } | ||
| 666 | |||
| 667 | // Check purgatory for PR event | ||
| 668 | if let Some(entry) = self.purgatory.find_pr(&EventId::from_hex(event_id)?) { | ||
| 669 | if let Some(event) = entry.event { | ||
| 670 | // Event exists in purgatory - must match commit | ||
| 671 | let expected_commit = extract_c_tag_commit(&event)?; | ||
| 672 | if expected_commit == pushed_commit { | ||
| 673 | // Remove from purgatory before returning | ||
| 674 | self.purgatory.remove_pr(&EventId::from_hex(event_id)?); // note this shouldnt delete the git data | ||
| 675 | return Ok(PushDecision::AcceptAndRelease(event)); | ||
| 676 | } else { | ||
| 677 | return Ok(PushDecision::Reject("commit mismatch with purgatory event")); | ||
| 678 | } | ||
| 679 | } else { | ||
| 680 | // Placeholder exists (previous push, no event yet) | ||
| 681 | // Accept and update placeholder with new commit | ||
| 682 | // This allows re-pushing with corrected data before event arrives | ||
| 683 | return Ok(PushDecision::AcceptAndUpdatePlaceholder(pushed_commit.to_string())); | ||
| 684 | } | ||
| 685 | } | ||
| 686 | |||
| 687 | // No event anywhere - git-data-first scenario | ||
| 688 | // Accept ANY commit and create placeholder awaiting the PR event | ||
| 689 | Ok(PushDecision::AcceptAndCreatePlaceholder(pushed_commit.to_string())) | ||
| 690 | } | ||
| 691 | // TODO when AcceptAndUpdatePlaceholder gets called purgatory must get udpated with a new / updated entry either here of where AcceptAndCreatePlaceholder is handled | ||
| 692 | ``` | ||
| 693 | |||
| 694 | ### 5. Main.rs Changes | ||
| 695 | |||
| 696 | ```rust | ||
| 697 | // During startup | ||
| 698 | let purgatory = Arc::new(Purgatory::new()); | ||
| 699 | |||
| 700 | // Pass to WritePolicy | ||
| 701 | let write_policy = Nip34WritePolicy::new( | ||
| 702 | &config.domain, | ||
| 703 | database.clone(), | ||
| 704 | &git_data_path, | ||
| 705 | purgatory.clone(), | ||
| 706 | ); | ||
| 707 | |||
| 708 | // Pass to git handlers (via shared state) | ||
| 709 | let git_state = GitState { | ||
| 710 | purgatory: purgatory.clone(), | ||
| 711 | database: database.clone(), | ||
| 712 | // ... | ||
| 713 | }; | ||
| 714 | |||
| 715 | // Spawn cleanup task | ||
| 716 | let purgatory_cleanup = purgatory.clone(); | ||
| 717 | tokio::spawn(async move { | ||
| 718 | let mut interval = tokio::time::interval(Duration::from_secs(60)); | ||
| 719 | loop { | ||
| 720 | interval.tick().await; | ||
| 721 | let removed = purgatory_cleanup.cleanup(); | ||
| 722 | if removed > 0 { | ||
| 723 | tracing::debug!("Purgatory cleanup removed {} expired entries", removed); | ||
| 724 | } | ||
| 725 | } | ||
| 726 | }); | ||
| 727 | ``` | ||
| 728 | |||
| 729 | ## Post-Push Sync Flow | ||
| 730 | |||
| 731 | After successfully processing a state or PR event, sync to other maintainer repos: | ||
| 732 | |||
| 733 | **Key Rules for State Events (30618)**: | ||
| 734 | |||
| 735 | 1. Fetch all state events matching the identifier from the database | ||
| 736 | 2. Check if another existing state event supersedes this one (the maintainer set may be different, so another maintainer may have a more recent state event) | ||
| 737 | 3. Only sync if this event is not superseded | ||
| 738 | |||
| 739 | ```rust | ||
| 740 | async fn sync_to_other_repos( | ||
| 741 | &self, | ||
| 742 | identifier: &str, | ||
| 743 | event: &Event, | ||
| 744 | processed_repo_owner: &str, | ||
| 745 | ) -> Result<usize, SyncError> { | ||
| 746 | let announcements = self.database.get_announcements(identifier).await?; | ||
| 747 | let auth_map = collect_authorized_maintainers(&announcements); | ||
| 748 | |||
| 749 | // Fetch all state events for this identifier to check for superseding | ||
| 750 | let all_state_events = self.database.get_state_events_for_identifier(identifier).await?; | ||
| 751 | |||
| 752 | let mut synced = 0; | ||
| 753 | for (owner, maintainers) in auth_map { | ||
| 754 | // Skip the repo we just processed | ||
| 755 | if owner == processed_repo_owner { | ||
| 756 | continue; | ||
| 757 | } | ||
| 758 | |||
| 759 | // Check if event author is authorized for this owner's repo | ||
| 760 | if maintainers.contains(&event.pubkey.to_hex()) { | ||
| 761 | let repo_path = self.repo_path_for_owner(&owner); | ||
| 762 | |||
| 763 | match event.kind.as_u64() { | ||
| 764 | 30618 => { | ||
| 765 | // State event - check if another state event supersedes this one | ||
| 766 | // for THIS owner's repo (maintainer sets may differ between owners) | ||
| 767 | let owner_maintainers = auth_map.get(&owner).cloned().unwrap_or_default(); | ||
| 768 | |||
| 769 | // Find the authoritative state for this owner's repo | ||
| 770 | let superseding_state = all_state_events.iter() | ||
| 771 | .filter(|s| owner_maintainers.contains(&s.pubkey.to_hex())) | ||
| 772 | .filter(|s| s.created_at > event.created_at) | ||
| 773 | .max_by_key(|s| s.created_at); | ||
| 774 | |||
| 775 | if let Some(newer_state) = superseding_state { | ||
| 776 | // This event is superseded by a more recent state | ||
| 777 | // for this owner's repo - skip syncing this event | ||
| 778 | tracing::debug!( | ||
| 779 | "Skipping sync to {}: event {} superseded by {}", | ||
| 780 | owner, | ||
| 781 | event.id, | ||
| 782 | newer_state.id | ||
| 783 | ); | ||
| 784 | continue; | ||
| 785 | } | ||
| 786 | |||
| 787 | // No superseding state - align refs | ||
| 788 | let state_refs = extract_refs_from_state(&event); | ||
| 789 | self.align_refs(&repo_path, &state_refs)?; | ||
| 790 | } | ||
| 791 | 1617 | 1618 => { | ||
| 792 | // PR event - ensure nostr ref exists | ||
| 793 | let commit = extract_c_tag_commit(&event)?; | ||
| 794 | self.ensure_nostr_ref(&repo_path, &event.id, &commit)?; | ||
| 795 | } | ||
| 796 | _ => {} | ||
| 797 | } | ||
| 798 | synced += 1; | ||
| 799 | } | ||
| 800 | } | ||
| 801 | |||
| 802 | Ok(synced) | ||
| 803 | } | ||
| 804 | ``` | ||
| 805 | |||
| 806 | ## Implementation File Structure | ||
| 807 | |||
| 808 | ``` | ||
| 809 | src/ | ||
| 810 | ├── purgatory/ | ||
| 811 | │ ├── mod.rs # Purgatory struct, public API | ||
| 812 | │ ├── types.rs # RefPair, StatePurgatoryEntry, PrPurgatoryEntry | ||
| 813 | │ ├── state_events.rs # State event purgatory logic | ||
| 814 | │ ├── pr_events.rs # PR event purgatory logic | ||
| 815 | │ └── helpers.rs # extract_refs_from_state, can_satisfy_state, etc. | ||
| 816 | ├── nostr/ | ||
| 817 | │ ├── builder.rs # Modified: Nip34WritePolicy accepts Arc<Purgatory> | ||
| 818 | │ └── policy/ | ||
| 819 | │ ├── state.rs # Modified: handle_state uses purgatory | ||
| 820 | │ └── pr_event.rs # Modified: handle_pr_event uses purgatory | ||
| 821 | ├── git/ | ||
| 822 | │ └── handlers.rs # Modified: handle_receive_pack integrates purgatory | ||
| 823 | └── main.rs # Modified: creates Purgatory, spawns cleanup task | ||
| 824 | ``` | ||
| 825 | |||
| 826 | ## Additional Type Definitions | ||
| 827 | |||
| 828 | ### PushDecision Enum | ||
| 829 | |||
| 830 | Used by `handle_nostr_ref_push` to communicate different outcomes to the caller: | ||
| 831 | |||
| 832 | ```rust | ||
| 833 | /// Result of evaluating a push to refs/nostr/<event-id> | ||
| 834 | pub enum PushDecision { | ||
| 835 | /// Push is valid - event exists in database and commit matches | ||
| 836 | Accept, | ||
| 837 | |||
| 838 | /// Push valid and event should be released from purgatory to database | ||
| 839 | AcceptAndRelease(Event), | ||
| 840 | |||
| 841 | /// Push valid - create new placeholder awaiting PR event | ||
| 842 | AcceptAndCreatePlaceholder(String), // commit SHA | ||
| 843 | |||
| 844 | /// Push valid - update existing placeholder with new commit | ||
| 845 | AcceptAndUpdatePlaceholder(String), // new commit SHA | ||
| 846 | |||
| 847 | /// Push rejected with reason | ||
| 848 | Reject(&'static str), | ||
| 849 | } | ||
| 850 | ``` | ||
| 851 | |||
| 852 | ### WritePolicyResult Clarification | ||
| 853 | |||
| 854 | The design uses a pattern where purgatory events return `status: true` but the event is NOT saved: | ||
| 855 | |||
| 856 | ```rust | ||
| 857 | // Event goes to purgatory - client sees OK but event not served until git data arrives | ||
| 858 | WritePolicyResult::Reject { | ||
| 859 | status: true, // Nostr OK message to client | ||
| 860 | message: "purgatory: won't be served until git data arrives".into() | ||
| 861 | } | ||
| 862 | |||
| 863 | // Event rejected - client sees error | ||
| 864 | WritePolicyResult::Reject { | ||
| 865 | status: false, // Nostr error message to client | ||
| 866 | message: "rejected: reason...".into() | ||
| 867 | } | ||
| 868 | |||
| 869 | // Event accepted and saved to database | ||
| 870 | WritePolicyResult::Accept | ||
| 871 | ``` | ||
| 872 | |||
| 873 | **Note:** This is a quirk of using the `WritePolicyResult::Reject` variant for purgatory - the `status: true` ensures the client receives an OK response, but since we're in the Reject variant, the event is not automatically saved to the database. The purgatory mechanism holds it until git data arrives. | ||
| 874 | |||
| 875 | ## Test Scenarios | ||
| 876 | |||
| 877 | ### State Event Tests | ||
| 878 | |||
| 879 | 1. **Event arrives, git data exists** - Event processed immediately, saved to DB | ||
| 880 | 2. **Event arrives, git data doesn't exist** - Event goes to purgatory, client sees OK | ||
| 881 | 3. **Git push arrives, matching event in purgatory** - Event released from purgatory, saved to DB | ||
| 882 | 4. **Git push arrives, no matching event** - Push rejected (no authorized state) | ||
| 883 | 5. **Event expires in purgatory** - Entry removed after 30 minutes (dont implement test due to 30m wait) | ||
| 884 | 6. **Multiple state events for same identifier** - Late binding at push time selects correct one | ||
| 885 | |||
| 886 | ### PR Event Tests | ||
| 887 | |||
| 888 | 1. **PR event arrives, git data exists** - Event processed immediately, saved to DB | ||
| 889 | 2. **PR event arrives, no git data** - Event goes to purgatory awaiting git push | ||
| 890 | 3. **PR event arrives, placeholder exists with matching commit** - Event released, saved to DB | ||
| 891 | 4. **PR event arrives, placeholder exists with different commit** - Ref deleted, event to purgatory | ||
| 892 | 5. **Git push to refs/nostr/ arrives, PR event exists in purgatory** - Event released, ref created | ||
| 893 | 6. **Git push to refs/nostr/ arrives, no PR event** - Placeholder created, awaiting event | ||
| 894 | 7. **Second git push updates placeholder** - Placeholder commit updated | ||
| 895 | |||
| 896 | ### Edge Cases (NOT TESTED) | ||
| 897 | |||
| 898 | 1. **Relay restart** - All purgatory entries lost (acceptable per design) | ||
| 899 | 2. **Same event submitted twice** - Deduplicated by event ID | ||
| 900 | 3. **Push timeout during processing** - Entry expiry extended to 15 min minimum | ||
| 901 | 4. **Race between event and git push** - Whichever completes the pair triggers release | ||