upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2026-02-23 15:08:37 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2026-02-23 15:08:37 +0000
commit26f608e5011b9d1ad6036da75b89272835e69695 (patch)
tree8b5dfe29f65abe80e59bddbcd3ee09c0a369dba8
parent4848c4029fc58f6f310a2babeae1ee82a7e41656 (diff)
persist and restore announcement events across graceful restarts
Extends purgatory persistence to include announcement purgatory entries. On graceful shutdown, non-soft-expired announcements are serialised to purgatory-state.json alongside state/PR/expired events; on startup they are restored, skipping any entry whose bare repo path no longer exists. Updates purgatory-design.md to reflect that purgatory persists through graceful shutdown and documents the new PurgatoryState disk format. Adds create_announcement_event helper to purgatory_helpers and three new integration tests in purgatory_persistence covering the full save/restore cycle, missing-repo skip, and the combined roundtrip with all entry types.
-rw-r--r--docs/explanation/purgatory-design.md66
-rw-r--r--src/purgatory/mod.rs264
-rw-r--r--tests/common/purgatory_helpers.rs38
-rw-r--r--tests/purgatory_persistence.rs135
4 files changed, 493 insertions, 10 deletions
diff --git a/docs/explanation/purgatory-design.md b/docs/explanation/purgatory-design.md
index bd792d4..8e7d75c 100644
--- a/docs/explanation/purgatory-design.md
+++ b/docs/explanation/purgatory-design.md
@@ -39,14 +39,36 @@ This ensures we only serve announcements for repos that actually have content.
39 39
40## Key Design Principles 40## Key Design Principles
41 41
42### 1. In-Memory Only 42### 1. Graceful-Shutdown Persistence
43 43
44Purgatory data is **not persisted** to disk. On restart, all purgatory entries are lost. This is acceptable because: 44Purgatory state is **saved to disk on graceful shutdown** and **restored on startup**. This preserves in-flight work across planned restarts (deployments, reboots).
45
46On `SIGINT` / Ctrl-C, `main.rs` calls `purgatory.save_to_disk()` before exiting. On startup, if the state file exists, `purgatory.restore_from_disk()` is called before the server begins accepting connections.
47
48**What is persisted:**
49
50| Store | Persisted? | Notes |
51|-------|-----------|-------|
52| `announcement_purgatory` | ✅ Yes | Non-soft-expired entries only (bare repo must exist) |
53| `state_events` | ✅ Yes | All active entries |
54| `pr_events` | ✅ Yes | Both events and placeholders |
55| `expired_events` | ✅ Yes | Prevents re-sync loops after restart |
56| `sync_queue` | ❌ No | Rebuilt automatically after restore |
57
58**What is NOT persisted (unclean shutdown):**
59
60On a crash or `SIGKILL`, the state file is not written. In that case:
45 61
46- Events are still on other relays (can be re-submitted) 62- Events are still on other relays (can be re-submitted)
47- Git data can be re-pushed 63- Git data can be re-pushed
48- 30-minute expiry means data is transient anyway 64- 30-minute expiry means data is transient anyway
49 65
66**State file location:** `<git_data_path>/purgatory-state.json`
67
68**Downtime accounting:** Expiry deadlines are stored as duration offsets from the save timestamp. On restore, elapsed downtime is subtracted from each deadline. Entries that expired during downtime are immediately swept by the next cleanup tick.
69
70**Soft-expired announcements are excluded:** Their bare repos have already been deleted, so they cannot be meaningfully restored. They will be re-fetched via background sync if needed.
71
50### 2. Separate Storage for Each Event Type 72### 2. Separate Storage for Each Event Type
51 73
52| Store | Index | Purpose | 74| Store | Index | Purpose |
@@ -233,6 +255,31 @@ pub struct Purgatory {
233} 255}
234``` 256```
235 257
258### Persistence State (Disk Format)
259
260`Instant` fields cannot be serialized directly. Each entry type has a corresponding `Serializable*` wrapper that stores time fields as `u64` second offsets from a `saved_at: SystemTime` reference point. On restore, elapsed downtime is subtracted to produce the correct remaining TTL.
261
262```rust
263struct PurgatoryState {
264 version: u32, // currently 1
265 saved_at: SystemTime, // reference for offset math
266
267 /// Non-soft-expired announcements indexed by "owner_hex:identifier"
268 announcement_purgatory: HashMap<String, SerializableAnnouncementPurgatoryEntry>,
269
270 /// State events indexed by repository identifier
271 state_events: HashMap<String, Vec<SerializableStatePurgatoryEntry>>,
272
273 /// PR events (and placeholders) indexed by event ID hex
274 pr_events: HashMap<String, SerializablePrPurgatoryEntry>,
275
276 /// Expired event IDs → approximate expiry SystemTime
277 expired_events: HashMap<String, SystemTime>,
278}
279```
280
281The `announcement_purgatory` field uses `#[serde(default)]` so that state files written before announcement persistence was added (version 1 without the field) still deserialize correctly.
282
236--- 283---
237 284
238## Announcement Purgatory Flows 285## Announcement Purgatory Flows
@@ -806,8 +853,9 @@ A background timer (`run_purgatory_announcement_sync`, every 5 seconds) ensures
806``` 853```
807src/ 854src/
808├── purgatory/ 855├── purgatory/
809│ ├── mod.rs # Main Purgatory struct and API 856│ ├── mod.rs # Main Purgatory struct, API, save_to_disk, restore_from_disk
810│ ├── types.rs # RefPair, AnnouncementPurgatoryEntry, StatePurgatoryEntry, PrPurgatoryEntry 857│ ├── types.rs # RefPair, AnnouncementPurgatoryEntry, StatePurgatoryEntry, PrPurgatoryEntry
858│ ├── persistence.rs # instant_to_offset / offset_to_instant time conversion utilities
811│ ├── helpers.rs # Ref extraction and matching functions 859│ ├── helpers.rs # Ref extraction and matching functions
812│ └── sync/ 860│ └── sync/
813│ ├── mod.rs # Sync module exports 861│ ├── mod.rs # Sync module exports
@@ -835,7 +883,8 @@ src/
835 883
836Located in each module: 884Located in each module:
837 885
838- **[`src/purgatory/mod.rs`](../../src/purgatory/mod.rs)** - Core purgatory operations including announcement purgatory 886- **[`src/purgatory/mod.rs`](../../src/purgatory/mod.rs)** - Core purgatory operations including announcement purgatory; persistence round-trip tests for all entry types (state, PR, announcement, expired events, downtime calculation, soft-expired exclusion, missing-repo skip)
887- **[`src/purgatory/persistence.rs`](../../src/purgatory/persistence.rs)** - `instant_to_offset` / `offset_to_instant` round-trip tests
839- **[`src/purgatory/helpers.rs`](../../src/purgatory/helpers.rs)** - Ref matching logic 888- **[`src/purgatory/helpers.rs`](../../src/purgatory/helpers.rs)** - Ref matching logic
840- **[`src/purgatory/sync/functions.rs`](../../src/purgatory/sync/functions.rs)** - Sync functions with MockSyncContext 889- **[`src/purgatory/sync/functions.rs`](../../src/purgatory/sync/functions.rs)** - Sync functions with MockSyncContext
841- **[`src/purgatory/sync/throttle.rs`](../../src/purgatory/sync/throttle.rs)** - Throttle manager 890- **[`src/purgatory/sync/throttle.rs`](../../src/purgatory/sync/throttle.rs)** - Throttle manager
@@ -852,6 +901,7 @@ Located in [`tests/`](../../tests/):
852- **Git-data-first flow** - Git push creates placeholder, event completes it 901- **Git-data-first flow** - Git push creates placeholder, event completes it
853- **Authorization with purgatory** - Push authorized by purgatory state 902- **Authorization with purgatory** - Push authorized by purgatory state
854- **Background sync** - Sync fetches git data and releases events 903- **Background sync** - Sync fetches git data and releases events
904- **Persistence across restart** - Save/restore cycle preserves all entry types including announcements
855 905
856--- 906---
857 907
@@ -894,6 +944,14 @@ PR events can arrive before or after git data:
894 944
895**Solution:** `PrPurgatoryEntry.event: Option<Event>` with `None` = placeholder. 945**Solution:** `PrPurgatoryEntry.event: Option<Event>` with `None` = placeholder.
896 946
947### 6. Persistence Requires Instant → Duration Conversion
948
949`std::time::Instant` is not serializable and is not meaningful across process boundaries. Expiry deadlines must be converted to a portable form.
950
951**Solution:** Store each deadline as a `u64` second offset from a `saved_at: SystemTime` reference. On restore, subtract elapsed downtime from each offset to compute the new `Instant`. Entries whose deadline already passed during downtime get `expires_at = now` and are swept by the next cleanup tick.
952
953**Soft-expired announcements are excluded from persistence** because their bare repos have been deleted. Restoring them would leave purgatory entries pointing at non-existent repos. They are simply dropped; background sync will re-fetch the announcement event if needed.
954
897--- 955---
898 956
899## Related Documentation 957## Related Documentation
diff --git a/src/purgatory/mod.rs b/src/purgatory/mod.rs
index f5f8b31..9a63bf6 100644
--- a/src/purgatory/mod.rs
+++ b/src/purgatory/mod.rs
@@ -83,9 +83,35 @@ struct SerializablePrPurgatoryEntry {
83 expires_at_offset_secs: u64, 83 expires_at_offset_secs: u64,
84} 84}
85 85
86/// Serializable wrapper for `AnnouncementPurgatoryEntry` with time offsets.
87///
88/// Stores `Instant` fields as `Duration` offsets from the `saved_at` timestamp
89/// in `PurgatoryState`, allowing state to be persisted and restored across restarts.
90///
91/// Note: soft-expired entries (bare repo deleted) are NOT persisted — they have
92/// no git repo on disk and would be immediately cleaned up on restore anyway.
93#[derive(Debug, Clone, Serialize, Deserialize)]
94struct SerializableAnnouncementPurgatoryEntry {
95 /// The nostr announcement event (kind 30617)
96 event: Event,
97 /// The repository identifier from the event's 'd' tag
98 identifier: String,
99 /// The owner pubkey (event author)
100 owner: PublicKey,
101 /// Path to the bare git repository (must exist on disk)
102 repo_path: PathBuf,
103 /// Relay URLs from the announcement (for sync registration)
104 relays: HashSet<String>,
105 /// Duration offset from saved_at for created_at
106 created_at_offset_secs: u64,
107 /// Duration offset from saved_at for expires_at
108 expires_at_offset_secs: u64,
109}
110
86/// Serializable purgatory state for disk persistence. 111/// Serializable purgatory state for disk persistence.
87/// 112///
88/// Contains all purgatory data needed to restore state across restarts: 113/// Contains all purgatory data needed to restore state across restarts:
114/// - Announcement events (indexed by (owner, identifier)) — non-soft-expired only
89/// - State events (indexed by identifier) 115/// - State events (indexed by identifier)
90/// - PR events (indexed by event ID) 116/// - PR events (indexed by event ID)
91/// - Expired events (to prevent re-sync loops) 117/// - Expired events (to prevent re-sync loops)
@@ -97,6 +123,10 @@ struct PurgatoryState {
97 version: u32, 123 version: u32,
98 /// When this state was saved to disk 124 /// When this state was saved to disk
99 saved_at: SystemTime, 125 saved_at: SystemTime,
126 /// Announcement events indexed by "owner_hex:identifier"
127 /// Only non-soft-expired entries are persisted (bare repo must exist).
128 #[serde(default)]
129 announcement_purgatory: HashMap<String, SerializableAnnouncementPurgatoryEntry>,
100 /// State events indexed by repository identifier 130 /// State events indexed by repository identifier
101 state_events: HashMap<String, Vec<SerializableStatePurgatoryEntry>>, 131 state_events: HashMap<String, Vec<SerializableStatePurgatoryEntry>>,
102 /// PR events indexed by event ID (hex string) 132 /// PR events indexed by event ID (hex string)
@@ -1114,6 +1144,34 @@ impl Purgatory {
1114 let saved_at = SystemTime::now(); 1144 let saved_at = SystemTime::now();
1115 let now_instant = Instant::now(); 1145 let now_instant = Instant::now();
1116 1146
1147 // Convert announcement_purgatory to serializable format.
1148 // Skip soft-expired entries: their bare repos have been deleted, so they
1149 // cannot be meaningfully restored (the repo path no longer exists on disk).
1150 let mut announcement_purgatory = HashMap::new();
1151 for entry in self.announcement_purgatory.iter() {
1152 let e = entry.value();
1153 if e.soft_expired {
1154 continue;
1155 }
1156 let created_offset =
1157 persistence::instant_to_offset(e.created_at, saved_at, now_instant);
1158 let expires_offset =
1159 persistence::instant_to_offset(e.expires_at, saved_at, now_instant);
1160 let key = format!("{}:{}", e.owner.to_hex(), e.identifier);
1161 announcement_purgatory.insert(
1162 key,
1163 SerializableAnnouncementPurgatoryEntry {
1164 event: e.event.clone(),
1165 identifier: e.identifier.clone(),
1166 owner: e.owner,
1167 repo_path: e.repo_path.clone(),
1168 relays: e.relays.clone(),
1169 created_at_offset_secs: created_offset.as_secs(),
1170 expires_at_offset_secs: expires_offset.as_secs(),
1171 },
1172 );
1173 }
1174
1117 // Convert state_events to serializable format 1175 // Convert state_events to serializable format
1118 let mut state_events = HashMap::new(); 1176 let mut state_events = HashMap::new();
1119 for entry in self.state_events.iter() { 1177 for entry in self.state_events.iter() {
@@ -1176,6 +1234,7 @@ impl Purgatory {
1176 let state = PurgatoryState { 1234 let state = PurgatoryState {
1177 version: 1, 1235 version: 1,
1178 saved_at, 1236 saved_at,
1237 announcement_purgatory,
1179 state_events, 1238 state_events,
1180 pr_events, 1239 pr_events,
1181 expired_events, 1240 expired_events,
@@ -1187,6 +1246,7 @@ impl Purgatory {
1187 1246
1188 tracing::info!( 1247 tracing::info!(
1189 path = %path.display(), 1248 path = %path.display(),
1249 announcements = state.announcement_purgatory.len(),
1190 state_events = state.state_events.len(), 1250 state_events = state.state_events.len(),
1191 pr_events = state.pr_events.len(), 1251 pr_events = state.pr_events.len(),
1192 expired_events = state.expired_events.len(), 1252 expired_events = state.expired_events.len(),
@@ -1234,6 +1294,45 @@ impl Purgatory {
1234 1294
1235 let now_instant = Instant::now(); 1295 let now_instant = Instant::now();
1236 1296
1297 // Restore announcement_purgatory.
1298 // Skip entries whose bare repo no longer exists on disk — this can happen
1299 // if the repo was deleted externally between save and restore.
1300 for (_key, e) in state.announcement_purgatory {
1301 if !e.repo_path.exists() {
1302 tracing::warn!(
1303 owner = %e.owner,
1304 identifier = %e.identifier,
1305 repo_path = %e.repo_path.display(),
1306 "Skipping announcement restore: bare repo no longer exists"
1307 );
1308 continue;
1309 }
1310 let created_at = persistence::offset_to_instant(
1311 Duration::from_secs(e.created_at_offset_secs),
1312 state.saved_at,
1313 now_instant,
1314 );
1315 let expires_at = persistence::offset_to_instant(
1316 Duration::from_secs(e.expires_at_offset_secs),
1317 state.saved_at,
1318 now_instant,
1319 );
1320 let key = (e.owner, e.identifier.clone());
1321 self.announcement_purgatory.insert(
1322 key,
1323 AnnouncementPurgatoryEntry {
1324 event: e.event,
1325 identifier: e.identifier,
1326 owner: e.owner,
1327 repo_path: e.repo_path,
1328 relays: e.relays,
1329 created_at,
1330 expires_at,
1331 soft_expired: false,
1332 },
1333 );
1334 }
1335
1237 // Restore state_events 1336 // Restore state_events
1238 for (identifier, entries) in state.state_events { 1337 for (identifier, entries) in state.state_events {
1239 let restored_entries: Vec<StatePurgatoryEntry> = entries 1338 let restored_entries: Vec<StatePurgatoryEntry> = entries
@@ -1301,6 +1400,7 @@ impl Purgatory {
1301 1400
1302 tracing::info!( 1401 tracing::info!(
1303 path = %path.display(), 1402 path = %path.display(),
1403 announcements = self.announcement_purgatory.len(),
1304 state_events = self.state_events.len(), 1404 state_events = self.state_events.len(),
1305 pr_events = self.pr_events.len(), 1405 pr_events = self.pr_events.len(),
1306 expired_events = self.expired_events.len(), 1406 expired_events = self.expired_events.len(),
@@ -2426,6 +2526,141 @@ async fn test_file_cleanup_after_successful_restore() {
2426} 2526}
2427 2527
2428#[tokio::test] 2528#[tokio::test]
2529async fn test_save_and_restore_announcement_events() {
2530 use tempfile::tempdir;
2531
2532 let temp_dir = tempdir().unwrap();
2533 let state_file = temp_dir.path().join("purgatory_state.json");
2534
2535 // Create a real bare repo directory so the restore path-existence check passes
2536 let repo_dir = temp_dir.path().join("owner.git");
2537 std::fs::create_dir_all(&repo_dir).unwrap();
2538
2539 let purgatory = Purgatory::new(PathBuf::new());
2540 let keys = Keys::generate();
2541
2542 let ann_event = EventBuilder::text_note("announcement event")
2543 .sign_with_keys(&keys)
2544 .unwrap();
2545 let ann_event_id = ann_event.id;
2546
2547 let mut relays = HashSet::new();
2548 relays.insert("wss://relay.example.com".to_string());
2549
2550 purgatory.add_announcement(
2551 ann_event.clone(),
2552 "my-repo".to_string(),
2553 keys.public_key(),
2554 repo_dir.clone(),
2555 relays.clone(),
2556 );
2557
2558 // Save to disk
2559 purgatory.save_to_disk(&state_file).unwrap();
2560 assert!(state_file.exists());
2561
2562 // Create new purgatory and restore
2563 let purgatory2 = Purgatory::new(PathBuf::new());
2564 purgatory2.restore_from_disk(&state_file).unwrap();
2565
2566 // File should be deleted after restore
2567 assert!(!state_file.exists());
2568
2569 // Verify announcement was restored
2570 let (ann_count, _, _) = purgatory2.count();
2571 assert_eq!(ann_count, 1);
2572
2573 let restored = purgatory2
2574 .find_announcement(&keys.public_key(), "my-repo")
2575 .unwrap();
2576 assert_eq!(restored.event.id, ann_event_id);
2577 assert_eq!(restored.identifier, "my-repo");
2578 assert_eq!(restored.owner, keys.public_key());
2579 assert_eq!(restored.repo_path, repo_dir);
2580 assert_eq!(restored.relays, relays);
2581 assert!(!restored.soft_expired);
2582}
2583
2584#[tokio::test]
2585async fn test_soft_expired_announcements_not_persisted() {
2586 use tempfile::tempdir;
2587
2588 let temp_dir = tempdir().unwrap();
2589 let state_file = temp_dir.path().join("purgatory_state.json");
2590
2591 let repo_dir = temp_dir.path().join("owner.git");
2592 std::fs::create_dir_all(&repo_dir).unwrap();
2593
2594 let purgatory = Purgatory::new(PathBuf::new());
2595 let keys = Keys::generate();
2596
2597 let ann_event = EventBuilder::text_note("announcement event")
2598 .sign_with_keys(&keys)
2599 .unwrap();
2600
2601 purgatory.add_announcement(
2602 ann_event.clone(),
2603 "my-repo".to_string(),
2604 keys.public_key(),
2605 repo_dir.clone(),
2606 HashSet::new(),
2607 );
2608
2609 // Manually mark as soft-expired (bare repo deleted)
2610 let key = (keys.public_key(), "my-repo".to_string());
2611 if let Some(mut entry) = purgatory.announcement_purgatory.get_mut(&key) {
2612 entry.soft_expired = true;
2613 }
2614
2615 // Save to disk — soft-expired entry should be excluded
2616 purgatory.save_to_disk(&state_file).unwrap();
2617
2618 // Create new purgatory and restore
2619 let purgatory2 = Purgatory::new(PathBuf::new());
2620 purgatory2.restore_from_disk(&state_file).unwrap();
2621
2622 // Soft-expired announcement should NOT be restored
2623 let (ann_count, _, _) = purgatory2.count();
2624 assert_eq!(ann_count, 0);
2625}
2626
2627#[tokio::test]
2628async fn test_announcement_with_missing_repo_skipped_on_restore() {
2629 use tempfile::tempdir;
2630
2631 let temp_dir = tempdir().unwrap();
2632 let state_file = temp_dir.path().join("purgatory_state.json");
2633
2634 // Point to a repo path that does NOT exist
2635 let missing_repo = temp_dir.path().join("nonexistent.git");
2636
2637 let purgatory = Purgatory::new(PathBuf::new());
2638 let keys = Keys::generate();
2639
2640 let ann_event = EventBuilder::text_note("announcement event")
2641 .sign_with_keys(&keys)
2642 .unwrap();
2643
2644 purgatory.add_announcement(
2645 ann_event.clone(),
2646 "my-repo".to_string(),
2647 keys.public_key(),
2648 missing_repo.clone(),
2649 HashSet::new(),
2650 );
2651
2652 // Save to disk (repo path is serialized even though it doesn't exist)
2653 purgatory.save_to_disk(&state_file).unwrap();
2654
2655 // Create new purgatory and restore — entry should be skipped
2656 let purgatory2 = Purgatory::new(PathBuf::new());
2657 purgatory2.restore_from_disk(&state_file).unwrap();
2658
2659 let (ann_count, _, _) = purgatory2.count();
2660 assert_eq!(ann_count, 0);
2661}
2662
2663#[tokio::test]
2429async fn test_comprehensive_roundtrip() { 2664async fn test_comprehensive_roundtrip() {
2430 use nostr_sdk::{Kind, Tag, TagKind}; 2665 use nostr_sdk::{Kind, Tag, TagKind};
2431 use tempfile::tempdir; 2666 use tempfile::tempdir;
@@ -2433,10 +2668,27 @@ async fn test_comprehensive_roundtrip() {
2433 let temp_dir = tempdir().unwrap(); 2668 let temp_dir = tempdir().unwrap();
2434 let state_file = temp_dir.path().join("purgatory_state.json"); 2669 let state_file = temp_dir.path().join("purgatory_state.json");
2435 2670
2671 // Create a real bare repo directory for the announcement
2672 let repo_dir = temp_dir.path().join("owner.git");
2673 std::fs::create_dir_all(&repo_dir).unwrap();
2674
2436 let purgatory = Purgatory::new(PathBuf::new()); 2675 let purgatory = Purgatory::new(PathBuf::new());
2437 let keys1 = Keys::generate(); 2676 let keys1 = Keys::generate();
2438 let keys2 = Keys::generate(); 2677 let keys2 = Keys::generate();
2439 2678
2679 // Add announcement
2680 let ann_event = EventBuilder::text_note("announcement")
2681 .sign_with_keys(&keys1)
2682 .unwrap();
2683 let ann_event_id = ann_event.id;
2684 purgatory.add_announcement(
2685 ann_event,
2686 "repo1".to_string(),
2687 keys1.public_key(),
2688 repo_dir.clone(),
2689 HashSet::new(),
2690 );
2691
2440 // Add multiple state events 2692 // Add multiple state events
2441 let state1 = EventBuilder::text_note("state 1") 2693 let state1 = EventBuilder::text_note("state 1")
2442 .sign_with_keys(&keys1) 2694 .sign_with_keys(&keys1)
@@ -2476,7 +2728,8 @@ async fn test_comprehensive_roundtrip() {
2476 purgatory.cleanup(); 2728 purgatory.cleanup();
2477 2729
2478 // Verify initial state 2730 // Verify initial state
2479 let (_, state_count, pr_count) = purgatory.count(); 2731 let (ann_count, state_count, pr_count) = purgatory.count();
2732 assert_eq!(ann_count, 1); // announcement
2480 assert_eq!(state_count, 2); // state1, state2 (expired_event was cleaned up) 2733 assert_eq!(state_count, 2); // state1, state2 (expired_event was cleaned up)
2481 assert_eq!(pr_count, 2); // pr-1, pr-2 2734 assert_eq!(pr_count, 2); // pr-1, pr-2
2482 assert_eq!(purgatory.expired_count(), 1); // expired_event 2735 assert_eq!(purgatory.expired_count(), 1); // expired_event
@@ -2489,11 +2742,18 @@ async fn test_comprehensive_roundtrip() {
2489 purgatory2.restore_from_disk(&state_file).unwrap(); 2742 purgatory2.restore_from_disk(&state_file).unwrap();
2490 2743
2491 // Verify all data was restored correctly 2744 // Verify all data was restored correctly
2492 let (_, state_count2, pr_count2) = purgatory2.count(); 2745 let (ann_count2, state_count2, pr_count2) = purgatory2.count();
2746 assert_eq!(ann_count2, 1);
2493 assert_eq!(state_count2, 2); 2747 assert_eq!(state_count2, 2);
2494 assert_eq!(pr_count2, 2); 2748 assert_eq!(pr_count2, 2);
2495 assert_eq!(purgatory2.expired_count(), 1); 2749 assert_eq!(purgatory2.expired_count(), 1);
2496 2750
2751 // Verify announcement
2752 let restored_ann = purgatory2
2753 .find_announcement(&keys1.public_key(), "repo1")
2754 .unwrap();
2755 assert_eq!(restored_ann.event.id, ann_event_id);
2756
2497 // Verify state events 2757 // Verify state events
2498 assert_eq!(purgatory2.find_state("repo1").len(), 1); 2758 assert_eq!(purgatory2.find_state("repo1").len(), 1);
2499 assert_eq!(purgatory2.find_state("repo2").len(), 1); 2759 assert_eq!(purgatory2.find_state("repo2").len(), 1);
diff --git a/tests/common/purgatory_helpers.rs b/tests/common/purgatory_helpers.rs
index 1d06f22..cfcea1c 100644
--- a/tests/common/purgatory_helpers.rs
+++ b/tests/common/purgatory_helpers.rs
@@ -338,6 +338,44 @@ pub fn build_repo_coord(keys: &Keys, identifier: &str) -> String {
338 format!("30617:{}:{}", keys.public_key().to_hex(), identifier) 338 format!("30617:{}:{}", keys.public_key().to_hex(), identifier)
339} 339}
340 340
341/// Create a repository announcement event (kind 30617) for purgatory tests.
342///
343/// Creates a minimal but valid NIP-34 repository announcement with a `d` tag,
344/// optional `clone` URLs, and optional `relays` URLs.
345///
346/// # Arguments
347/// * `keys` - Keys for signing
348/// * `identifier` - Repository identifier (d-tag)
349/// * `clone_urls` - Clone URLs to include (may be empty)
350/// * `relay_urls` - Relay URLs to include (may be empty)
351///
352/// # Returns
353/// * `Ok(Event)` - Signed announcement event
354/// * `Err(String)` - If signing fails
355pub fn create_announcement_event(
356 keys: &Keys,
357 identifier: &str,
358 clone_urls: &[&str],
359 relay_urls: &[&str],
360) -> Result<Event, String> {
361 let mut tags = vec![Tag::identifier(identifier)];
362
363 if !clone_urls.is_empty() {
364 let urls: Vec<String> = clone_urls.iter().map(|s| s.to_string()).collect();
365 tags.push(Tag::custom(TagKind::custom("clone"), urls));
366 }
367
368 if !relay_urls.is_empty() {
369 let urls: Vec<String> = relay_urls.iter().map(|s| s.to_string()).collect();
370 tags.push(Tag::custom(TagKind::custom("relays"), urls));
371 }
372
373 EventBuilder::new(Kind::GitRepoAnnouncement, "")
374 .tags(tags)
375 .sign_with_keys(keys)
376 .map_err(|e| format!("Failed to sign announcement event: {}", e))
377}
378
341/// Wait for an event to be served by a relay (not in purgatory). 379/// Wait for an event to be served by a relay (not in purgatory).
342/// 380///
343/// Polls the relay until the event is queryable, indicating it has 381/// Polls the relay until the event is queryable, indicating it has
diff --git a/tests/purgatory_persistence.rs b/tests/purgatory_persistence.rs
index 5abbf15..05cb44b 100644
--- a/tests/purgatory_persistence.rs
+++ b/tests/purgatory_persistence.rs
@@ -31,9 +31,11 @@
31 31
32mod common; 32mod common;
33 33
34use common::purgatory_helpers::create_announcement_event;
34use ngit_grasp::purgatory::Purgatory; 35use ngit_grasp::purgatory::Purgatory;
35use ngit_grasp::sync::rejected_index::{EventType, RejectedEventsIndex, RejectionReason}; 36use ngit_grasp::sync::rejected_index::{EventType, RejectedEventsIndex, RejectionReason};
36use nostr_sdk::prelude::*; 37use nostr_sdk::prelude::*;
38use std::collections::HashSet;
37use std::time::Duration; 39use std::time::Duration;
38 40
39/// Helper to create a test event 41/// Helper to create a test event
@@ -116,12 +118,31 @@ async fn test_full_purgatory_save_restore_cycle() {
116 // Add a PR placeholder (git-data-first scenario) 118 // Add a PR placeholder (git-data-first scenario)
117 purgatory.add_pr_placeholder("placeholder-id".to_string(), "commit-xyz".to_string()); 119 purgatory.add_pr_placeholder("placeholder-id".to_string(), "commit-xyz".to_string());
118 120
119 // Note: We can't directly test expired events without accessing private fields, 121 // Add an announcement to purgatory (requires a real directory for the repo path)
120 // so we'll focus on testing state and PR events persistence 122 let repo_dir = temp_dir.path().join("repo.git");
123 std::fs::create_dir_all(&repo_dir).unwrap();
124 let ann_keys = Keys::generate();
125 let ann_event = create_announcement_event(
126 &ann_keys,
127 "my-repo",
128 &["http://example.com/my-repo.git"],
129 &["wss://relay.example.com"],
130 )
131 .unwrap();
132 let ann_event_id = ann_event.id;
133 let mut ann_relays = HashSet::new();
134 ann_relays.insert("wss://relay.example.com".to_string());
135 purgatory.add_announcement(
136 ann_event,
137 "my-repo".to_string(),
138 ann_keys.public_key(),
139 repo_dir.clone(),
140 ann_relays,
141 );
121 142
122 // Verify initial counts 143 // Verify initial counts
123 let (announcement_count, state_count, pr_count) = purgatory.count(); 144 let (announcement_count, state_count, pr_count) = purgatory.count();
124 assert_eq!(announcement_count, 0, "Should have 0 announcements"); 145 assert_eq!(announcement_count, 1, "Should have 1 announcement");
125 assert_eq!(state_count, 2, "Should have 2 state events"); 146 assert_eq!(state_count, 2, "Should have 2 state events");
126 assert_eq!( 147 assert_eq!(
127 pr_count, 3, 148 pr_count, 3,
@@ -144,13 +165,22 @@ async fn test_full_purgatory_save_restore_cycle() {
144 165
145 // Verify all data was restored 166 // Verify all data was restored
146 let (announcement_count2, state_count2, pr_count2) = purgatory2.count(); 167 let (announcement_count2, state_count2, pr_count2) = purgatory2.count();
147 assert_eq!(announcement_count2, 0, "Should have 0 announcements after restore"); 168 assert_eq!(announcement_count2, 1, "Should have 1 announcement after restore");
148 assert_eq!(state_count2, 2, "Should have 2 state events after restore"); 169 assert_eq!(state_count2, 2, "Should have 2 state events after restore");
149 assert_eq!( 170 assert_eq!(
150 pr_count2, 3, 171 pr_count2, 3,
151 "Should have 3 PR events after restore (2 events + 1 placeholder)" 172 "Should have 3 PR events after restore (2 events + 1 placeholder)"
152 ); 173 );
153 174
175 // Verify announcement was restored correctly
176 let restored_ann = purgatory2
177 .find_announcement(&ann_keys.public_key(), "my-repo")
178 .expect("Announcement should be restored");
179 assert_eq!(restored_ann.event.id, ann_event_id);
180 assert_eq!(restored_ann.identifier, "my-repo");
181 assert_eq!(restored_ann.repo_path, repo_dir);
182 assert!(!restored_ann.soft_expired);
183
154 // Verify specific state events 184 // Verify specific state events
155 let repo1_states = purgatory2.find_state("repo1"); 185 let repo1_states = purgatory2.find_state("repo1");
156 assert_eq!(repo1_states.len(), 1); 186 assert_eq!(repo1_states.len(), 1);
@@ -748,3 +778,100 @@ async fn test_rejected_cache_entries_expired_during_downtime() {
748 assert_eq!(index2.hot_cache_len(), 0); 778 assert_eq!(index2.hot_cache_len(), 0);
749 assert_eq!(index2.cold_index_len(), 1); 779 assert_eq!(index2.cold_index_len(), 1);
750} 780}
781
782/// Test 18: Announcement events are saved and restored across restarts
783#[tokio::test]
784async fn test_announcement_save_restore_cycle() {
785 let temp_dir = tempfile::tempdir().unwrap();
786 let git_data_path = temp_dir.path().join("git");
787 let state_path = temp_dir.path().join("purgatory.json");
788
789 // Create a real bare repo directory (restore skips entries whose path is missing)
790 let repo_dir = temp_dir.path().join("owner.git");
791 std::fs::create_dir_all(&repo_dir).unwrap();
792
793 let purgatory = Purgatory::new(&git_data_path);
794 let keys = Keys::generate();
795
796 let ann_event = create_announcement_event(
797 &keys,
798 "my-repo",
799 &["http://example.com/my-repo.git"],
800 &["wss://relay.example.com"],
801 )
802 .unwrap();
803 let ann_event_id = ann_event.id;
804
805 let mut relays = HashSet::new();
806 relays.insert("wss://relay.example.com".to_string());
807
808 purgatory.add_announcement(
809 ann_event,
810 "my-repo".to_string(),
811 keys.public_key(),
812 repo_dir.clone(),
813 relays.clone(),
814 );
815
816 let (ann_count, _, _) = purgatory.count();
817 assert_eq!(ann_count, 1);
818
819 // Save to disk
820 purgatory.save_to_disk(&state_path).unwrap();
821 assert!(state_path.exists());
822
823 // Restore into a fresh purgatory
824 let purgatory2 = Purgatory::new(&git_data_path);
825 purgatory2.restore_from_disk(&state_path).unwrap();
826
827 assert!(!state_path.exists(), "State file should be deleted after restore");
828
829 let (ann_count2, _, _) = purgatory2.count();
830 assert_eq!(ann_count2, 1, "Announcement should be restored");
831
832 let restored = purgatory2
833 .find_announcement(&keys.public_key(), "my-repo")
834 .expect("Announcement should be findable after restore");
835
836 assert_eq!(restored.event.id, ann_event_id);
837 assert_eq!(restored.identifier, "my-repo");
838 assert_eq!(restored.owner, keys.public_key());
839 assert_eq!(restored.repo_path, repo_dir);
840 assert_eq!(restored.relays, relays);
841 assert!(!restored.soft_expired);
842}
843
844/// Test 19: Announcement with missing repo path is skipped on restore
845#[tokio::test]
846async fn test_announcement_missing_repo_skipped_on_restore() {
847 let temp_dir = tempfile::tempdir().unwrap();
848 let git_data_path = temp_dir.path().join("git");
849 let state_path = temp_dir.path().join("purgatory.json");
850
851 // Point to a path that does NOT exist on disk
852 let missing_repo = temp_dir.path().join("nonexistent.git");
853
854 let purgatory = Purgatory::new(&git_data_path);
855 let keys = Keys::generate();
856
857 let ann_event = create_announcement_event(&keys, "my-repo", &[], &[]).unwrap();
858
859 purgatory.add_announcement(
860 ann_event,
861 "my-repo".to_string(),
862 keys.public_key(),
863 missing_repo,
864 HashSet::new(),
865 );
866
867 purgatory.save_to_disk(&state_path).unwrap();
868
869 let purgatory2 = Purgatory::new(&git_data_path);
870 purgatory2.restore_from_disk(&state_path).unwrap();
871
872 let (ann_count, _, _) = purgatory2.count();
873 assert_eq!(
874 ann_count, 0,
875 "Announcement with missing repo path must be skipped"
876 );
877}