2 files changed, 385 insertions, 36 deletions

diff --git a/docs/explanation/announcements-purgatory-design.md b/docs/explanation/announcements-purgatory-design.md
index 4d0cc6d..a06a8b2 100644
--- a/docs/explanation/announcements-purgatory-design.md
+++ b/docs/explanation/announcements-purgatory-design.md
@@ -2,13 +2,13 @@
 
 ## Problem Statement
 
-**Primary problem:** serving an announcement event and also an empty bare git repos mislead clients into thinking we host content.
+**Primary problem:** Serving announcement events alongside empty bare git repos misleads clients into thinking we host content.
 
 When an announcement arrives, we must create the bare repo immediately (so git pushes can succeed). But if no git data ever arrives, we serve an empty repo and its announcement indefinitely. Clients see the announcement, try to clone, and get nothing. This is misleading.
 
-**Secondary problem:** Sync downloads refs to deleted repos.
+**Secondary problem:** Sync downloads events for repos that may never have content.
 
-When a repo expires or is cleaned up, sync may still try to download state event refs to it. We need announcements to remain in a holding state until git data proves the repo has content worth serving.
+Without purgatory, sync would fetch all L2/L3 events (patches, issues, etc.) for announcements that may never receive git data. This wastes bandwidth and creates orphaned events.
 
 ## Solution Overview
 
@@ -57,15 +57,37 @@ This ensures we only serve announcements for repos that actually have content.
 
 **Why:** Prevents premature expiry during slow sync operations or multi-step pushes.
 
-### 5. State Events Consider Purgatory Announcements
+### 5. Authorization Must Check Purgatory Announcements
 
-**Decision:** When validating state events, check purgatory announcements for authorization.
+**Decision:** When validating state events or git operations, check purgatory announcements in addition to the database.
 
-**Why:** State events may arrive before git data promotes the announcement. They still need authorization from the announcement's maintainer set.
+**Why:** State events and git pushes may arrive before git data promotes the announcement. They still need authorization from the announcement's maintainer set.
 
-### 6. We need to request State Events in sysc for announcement in purgatory but not other l2 or l3 events because they will be rejected.
+**Where:** `fetch_repository_data()` and related authorization functions must query both DB and purgatory.
 
-### 7. When creating the authorised maintainers for a repositoriy we need to also get relivant announcement events from purgatory as well as db.
+### 6. Sync Only State Events for Purgatory Announcements
+
+**Decision:** Purgatory announcements trigger sync for state events only, not other L2/L3 events (patches, issues, PRs, etc.).
+
+**Why:** Other L2/L3 events would be rejected anyway (no promoted announcement in DB). Syncing them wastes bandwidth and creates work for announcements that may never promote.
+
+**How:** Sync uses a `SyncLevel` concept - `Full` for promoted repos, `StateOnly` for purgatory. On promotion, upgrade to `Full`.
+
+### 7. Soft Expiry Preserves Event Without Bare Repo
+
+**Decision:** When a purgatory announcement expires, delete the bare repo but retain the announcement event for an extended period (e.g., 24h).
+
+**Why:** This handles the case where a state event arrives after initial expiry. Without soft expiry, we'd either:
+- Add to `failed_events` and reject the state event (losing potential revival)
+- Re-fetch the announcement repeatedly (wasting sync bandwidth)
+
+**Behavior during soft expiry:**
+- Bare repo is deleted (saves disk space)
+- Announcement event retained in purgatory with `soft_expired` flag
+- Sync continues requesting state events (same as active purgatory)
+- If state event arrives: recreate bare repo, clear `soft_expired`, extend expiry
+- If announcement republished directly to us: treat as fresh arrival
+- After extended expiry: fully remove from purgatory
 
 ## Data Structure
 
@@ -78,13 +100,14 @@ pub struct AnnouncementPurgatoryEntry {
     pub identifier: String,
     pub owner: PublicKey,
     pub repo_path: PathBuf,
+    pub relays: HashSet<String>,  // For sync registration
     pub created_at: Instant,
     pub expires_at: Instant,
+    pub soft_expired: bool,       // Bare repo deleted, event retained
 }
 ```
 
-**Indexed by `(pubkey, identifier)`** because identifier is not unique across different owners.
-question: would it be more efficent to index by repo_path? this contains the pubkey and identifier data?
+**Indexed by `(pubkey, identifier)`** because identifier is not unique across different owners. Lookups are primarily from nostr events which have pubkey and identifier readily available.
 
 ## Flows
 
@@ -138,27 +161,51 @@ Is there an active announcement?
 
 ## Edge Cases
 
-| Scenario                                               | Behavior                                                                                                                               |
-| ------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- |
-| Git data before announcement                           | Push fails (no repo exists)                                                                                                            |
-| Announcement expires, no git data                      | Delete bare repo, discard announcement                                                                                                 |
-| State expires, announcement in purgatory               | Announcement keeps its own expiry                                                                                                      |
-| Multiple owners, same identifier                       | Each tracked separately by `(pubkey, identifier)`                                                                                      |
-| **Newer announcement replaces older (same pubkey)**    | Replace purgatory entry, extend expiry, and state event expiry                                                                         |
-| **Newer announcement changes services (unacceptable)** | Clear older announcement from purgatory for that `(pubkey, identifier)`, delete bare repo, remove state event for puragatory if exists |
-| Deletion event for purgatory announcement              | Remove from purgatory, delete bare repo                                                                                                |
+| Scenario                                               | Behavior                                                                                                |
+| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------- |
+| Git data before announcement                           | Push fails (no repo exists)                                                                             |
+| Announcement expires, no git data                      | Delete bare repo, set `soft_expired` flag, retain event for extended period                             |
+| Soft-expired announcement fully expires                | Remove from purgatory entirely                                                                          |
+| State event arrives for soft-expired announcement      | Recreate bare repo, clear `soft_expired`, extend expiry                                                 |
+| State expires, announcement in purgatory               | Announcement keeps its own expiry                                                                       |
+| Multiple owners, same identifier                       | Each tracked separately by `(pubkey, identifier)`                                                       |
+| **Newer announcement replaces older (same pubkey)**    | Replace purgatory entry, extend expiry, and state event expiry                                          |
+| **Newer announcement changes services (unacceptable)** | Clear older announcement from purgatory, delete bare repo, remove state events from purgatory if exists |
+| Deletion event for purgatory announcement              | Remove from purgatory, delete bare repo                                                                 |
+
+## Purgatory Lifecycle
 
-## Purgatory Exit Conditions
+An announcement progresses through purgatory states:
 
-An announcement leaves purgatory via:
+```
+                    ┌─────────────────────────────────────┐
+                    │                                     │
+                    v                                     │
+Announcement ──> ACTIVE ──────────────────────────────────┤
+  arrives        (bare repo exists)                       │
+                    │                                     │
+                    ├── Git data ──> PROMOTED (exit)      │
+                    │                                     │
+                    ├── Deletion ──> REMOVED (exit)       │
+                    │                                     │
+                    v                                     │
+               SOFT_EXPIRED ──────────────────────────────┘
+               (bare repo deleted,        ^
+                event retained)           │
+                    │                     │
+                    ├── State event arrives (revival)
+                    │
+                    └── Extended expiry ──> REMOVED (exit)
+```
 
-| Exit               | Trigger                                        | Action                             |
-| ------------------ | ---------------------------------------------- | ---------------------------------- |
-| **Promotion**      | Git data arrives                               | Move to database, serve to clients |
-| **Expiry**         | Timeout                                        | Delete bare repo, discard          |
-| **Deletion**       | Kind 5 event                                   | Delete bare repo, discard          |
-| **Replacement**    | Newer announcement (same pubkey, identifier)   | Replace entry                      |
-| **Service change** | Newer announcement no longer lists our service | Discard old entry                  |
+| Exit           | Trigger                                        | Action                                       |
+| -------------- | ---------------------------------------------- | -------------------------------------------- |
+| **Promotion**  | Git data arrives                               | Move to database, upgrade sync to Full       |
+| **Soft expiry**| Initial timeout                                | Delete bare repo, retain event, continue sync|
+| **Full expiry**| Extended timeout (soft-expired)                | Remove from purgatory entirely               |
+| **Deletion**   | Kind 5 event                                   | Delete bare repo, remove from purgatory      |
+| **Replacement**| Newer announcement (same pubkey, identifier)   | Replace entry                                |
+| **Service change** | Newer announcement removes our service     | Remove from purgatory                        |
 
 ## Integration Points
 
@@ -169,24 +216,33 @@ An announcement leaves purgatory via:
 | `src/nostr/policy/announcement.rs` | Route new announcements to purgatory                      |
 | `src/git/receive.rs`               | Promote on git data arrival                               |
 | `src/git/auth.rs`                  | Extend purgatory expiry when extending state event expiry |
+| `src/git/authorization.rs`         | Check purgatory announcements for maintainer authorization|
 | `src/nostr/policy/state.rs`        | Check purgatory for authorization                         |
+| `src/sync/mod.rs`                  | Add `SyncLevel` to `RepoSyncNeeds`                        |
+| `src/sync/filters.rs`              | Respect sync level when building filters                  |
+| `src/sync/self_subscriber.rs`      | Register purgatory announcements with `StateOnly` level   |
+
+See [announcements-purgatory-implementation.md](./announcements-purgatory-implementation.md) for detailed implementation notes.
 
 ## Testing
 
 - Announcement to purgatory, git data promotes it
-- Announcement expires without git data (repo deleted)
+- Announcement soft-expires without git data (repo deleted, event retained)
+- State event revives soft-expired announcement (repo recreated)
+- Soft-expired announcement fully expires after extended period
 - State event extends purgatory expiry
 - Git auth extends purgatory expiry
 - Newer announcement replaces older in purgatory
 - Service change clears purgatory entry
 - `(pubkey, identifier)` indexing with multiple owners
+- Sync requests only state events for purgatory announcements
+- Sync upgrades to full on promotion
 
 ## Risks
 
-| Risk                                 | Mitigation                           |
-| ------------------------------------ | ------------------------------------ |
-| Disk exhaustion from purgatory repos | Short expiry, monitor purgatory size |
-| Race between promotion and expiry    | Atomic operations                    |
-| Sync re-fetching expired events      | Track expired event IDs              |
-
-question: do expired annoucements go on failed_events list? what if a new state event comes in for it? surely then we want it again? but if not we dont want to keep donwloading it and havea a repo made for it. Should we have a longer period were we keep the event just in case, but delete the bare repo and only remake it when the state event arrives?
+| Risk                                 | Mitigation                                              |
+| ------------------------------------ | ------------------------------------------------------- |
+| Disk exhaustion from purgatory repos | Short expiry, soft expiry deletes repo early            |
+| Race between promotion and expiry    | Atomic operations                                       |
+| Sync re-fetching expired events      | Soft expiry retains event; no need for `failed_events`  |
+| Filter explosion from many purgatory | Existing consolidation handles this (threshold at 70)   |
diff --git a/docs/explanation/announcements-purgatory-implementation.md b/docs/explanation/announcements-purgatory-implementation.md
new file mode 100644
index 0000000..d5b8698
--- /dev/null
+++ b/docs/explanation/announcements-purgatory-implementation.md
@@ -0,0 +1,293 @@
+# Announcements Purgatory Implementation Details
+
+This document provides detailed implementation notes for the [Announcements Purgatory Design](./announcements-purgatory-design.md).
+
+## Sync Integration
+
+### Current Sync Architecture
+
+The sync system uses a two-index approach:
+
+```rust
+// What we WANT to sync - source of truth from self-subscription
+// Key: repo addressable ref (30617:pubkey:identifier)
+pub type RepoSyncIndex = Arc<RwLock<HashMap<String, RepoSyncNeeds>>>;
+
+pub struct RepoSyncNeeds {
+    pub relays: HashSet<String>,       // Relay URLs from announcement
+    pub root_events: HashSet<EventId>, // 1617/1618/1621 event IDs
+}
+
+// What we have CONFIRMED syncing + connection state
+// Key: relay URL
+pub type RelaySyncIndex = Arc<RwLock<HashMap<String, RelayState>>>;
+```
+
+**Three-Layer Sync Strategy:**
+1. **Layer 1:** Announcements (kinds 30617, 10317)
+2. **Layer 2:** Repo-tagging events (events with `a`/`A`/`q` tags + kind 30618 by identifier)
+3. **Layer 3:** Root-event-tagging events (events with `e`/`E`/`q` tags)
+
+### Adding SyncLevel
+
+Add a `sync_level` field to distinguish purgatory from promoted repos:
+
+```rust
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
+pub enum SyncLevel {
+    #[default]
+    Full,       // L2 + L3 (promoted repos)
+    StateOnly,  // Only state events (purgatory announcements)
+}
+
+pub struct RepoSyncNeeds {
+    pub relays: HashSet<String>,
+    pub root_events: HashSet<EventId>,
+    pub sync_level: SyncLevel,  // NEW
+}
+```
+
+### Filter Building Changes
+
+In `src/sync/filters.rs`, modify filter building to respect sync level:
+
+```rust
+// For StateOnly repos, only build state event filters
+pub fn build_layer2_and_layer3_filters(
+    repos: &HashMap<String, RepoSyncNeeds>,
+    // ...
+) -> Vec<Filter> {
+    let (full_repos, state_only_repos): (Vec<_>, Vec<_>) = repos
+        .iter()
+        .partition(|(_, needs)| needs.sync_level == SyncLevel::Full);
+    
+    let mut filters = Vec::new();
+    
+    // Full repos get all L2/L3 filters
+    if !full_repos.is_empty() {
+        filters.extend(tagged_one_of_our_repo_event_filters(&full_repos));
+        filters.extend(state_event_filters_for_our_repos(&full_repos));
+        filters.extend(tagged_one_of_our_root_event_filters(&full_repos));
+    }
+    
+    // StateOnly repos get only state event filters
+    if !state_only_repos.is_empty() {
+        filters.extend(state_event_filters_for_our_repos(&state_only_repos));
+    }
+    
+    filters
+}
+```
+
+The existing `state_event_filters_for_our_repos()` function already builds kind 30618 filters with `#d` tags, which is exactly what we need.
+
+### Self-Subscriber Changes
+
+In `src/sync/self_subscriber.rs`, add purgatory announcements to the sync index:
+
+```rust
+// When announcement enters purgatory
+fn on_announcement_to_purgatory(
+    &self,
+    event: &Event,
+    identifier: &str,
+    relays: HashSet<String>,
+) {
+    let key = format!("30617:{}:{}", event.pubkey, identifier);
+    let mut index = self.repo_sync_index.write().unwrap();
+    index.insert(key, RepoSyncNeeds {
+        relays,
+        root_events: HashSet::new(),
+        sync_level: SyncLevel::StateOnly,
+    });
+}
+
+// When announcement promotes to database
+fn on_announcement_promoted(
+    &self,
+    event: &Event,
+    identifier: &str,
+) {
+    let key = format!("30617:{}:{}", event.pubkey, identifier);
+    let mut index = self.repo_sync_index.write().unwrap();
+    if let Some(needs) = index.get_mut(&key) {
+        needs.sync_level = SyncLevel::Full;
+    }
+}
+```
+
+### Algorithm Changes
+
+In `src/sync/algorithms.rs`, preserve sync level when inverting repo->relay:
+
+```rust
+pub fn derive_relay_targets(
+    repo_index: &RepoSyncIndex,
+) -> HashMap<String, RelaySyncNeeds> {
+    // ... existing inversion logic ...
+    // Ensure sync_level is preserved/aggregated per relay
+    // A relay gets Full if ANY of its repos are Full
+}
+```
+
+## Authorization Integration
+
+### Current Authorization Flow
+
+Authorization lookups happen in `src/git/authorization.rs`:
+
+| Function | Purpose | Currently Queries |
+|----------|---------|-------------------|
+| `fetch_repository_data()` | Get announcements + states by identifier | DB only |
+| `collect_authorized_maintainers()` | Build maintainer set from announcements | DB only |
+| `pubkey_authorised_for_repo_owners()` | Check if pubkey authorized | DB only |
+
+### Required Changes
+
+Modify `fetch_repository_data()` to also query purgatory:
+
+```rust
+pub async fn fetch_repository_data(
+    db: &Database,
+    purgatory: &Purgatory,  // NEW parameter
+    identifier: &str,
+) -> Result<RepositoryData> {
+    // Existing DB query
+    let db_events = db.query(/* kind 30617, 30618 by identifier */).await?;
+    
+    // NEW: Also check purgatory for announcements
+    let purgatory_announcements = purgatory
+        .get_announcements_by_identifier(identifier);
+    
+    // Merge results
+    let mut announcements = parse_announcements(db_events);
+    announcements.extend(purgatory_announcements);
+    
+    // ... rest of function
+}
+```
+
+This affects:
+- `StatePolicy::process_state_event()` - state event validation
+- `get_state_authorization_for_specific_owner_repo()` - git push authorization
+- `AnnouncementPolicy::is_maintainer_in_any_announcement()` - maintainer exception
+
+## Purgatory Store Changes
+
+### New Fields
+
+```rust
+pub struct AnnouncementPurgatoryEntry {
+    pub event: Event,
+    pub identifier: String,
+    pub owner: PublicKey,
+    pub repo_path: PathBuf,
+    pub relays: HashSet<String>,  // For sync registration
+    pub created_at: Instant,
+    pub expires_at: Instant,
+    pub soft_expired: bool,       // Bare repo deleted, event retained
+}
+```
+
+### New Methods
+
+```rust
+impl Purgatory {
+    /// Get announcements by identifier (for authorization)
+    pub fn get_announcements_by_identifier(
+        &self,
+        identifier: &str,
+    ) -> Vec<&AnnouncementPurgatoryEntry> {
+        self.announcement_purgatory
+            .iter()
+            .filter(|entry| entry.identifier == identifier)
+            .collect()
+    }
+    
+    /// Transition to soft-expired state
+    pub fn soft_expire_announcement(
+        &self,
+        key: &(PublicKey, String),
+    ) -> Option<PathBuf> {
+        if let Some(mut entry) = self.announcement_purgatory.get_mut(key) {
+            entry.soft_expired = true;
+            entry.expires_at = Instant::now() + SOFT_EXPIRY_DURATION; // e.g., 24h
+            Some(entry.repo_path.clone()) // Return path for bare repo deletion
+        } else {
+            None
+        }
+    }
+    
+    /// Revive soft-expired announcement (caller must recreate bare repo)
+    pub fn revive_announcement(
+        &self,
+        key: &(PublicKey, String),
+    ) -> Option<PathBuf> {
+        if let Some(mut entry) = self.announcement_purgatory.get_mut(key) {
+            if entry.soft_expired {
+                entry.soft_expired = false;
+                entry.expires_at = Instant::now() + ACTIVE_EXPIRY_DURATION;
+                return Some(entry.repo_path.clone()); // Caller recreates bare repo
+            }
+        }
+        None
+    }
+}
+```
+
+## Expiry Cleanup Task
+
+The existing cleanup task needs to handle the two-phase expiry:
+
+```rust
+async fn cleanup_expired_announcements(&self) {
+    let now = Instant::now();
+    
+    for entry in self.announcement_purgatory.iter() {
+        if entry.expires_at <= now {
+            let key = (entry.owner.clone(), entry.identifier.clone());
+            
+            if entry.soft_expired {
+                // Fully expired - remove entirely
+                self.announcement_purgatory.remove(&key);
+                self.unregister_from_sync(&key);
+            } else {
+                // First expiry - transition to soft-expired
+                if let Some(repo_path) = self.soft_expire_announcement(&key) {
+                    delete_bare_repo(&repo_path).await;
+                }
+                // Note: stays in sync index with StateOnly level
+            }
+        }
+    }
+}
+```
+
+## State Event Revival Flow
+
+When a state event arrives for a soft-expired announcement, the state policy must:
+
+1. Check purgatory for a matching announcement (in addition to DB)
+2. Validate authorization against the purgatory announcement
+3. If soft-expired, call `revive_announcement()` and recreate the bare repo
+4. Extend the announcement's expiry
+5. Route the state event to state purgatory
+
+The exact integration will depend on the current structure of `StatePolicy::process_state_event()` - see implementation phase for details.
+
+## File Change Summary
+
+| File | Estimated Lines | Changes |
+|------|-----------------|---------|
+| `src/sync/mod.rs` | ~10 | Add `SyncLevel` enum, field to `RepoSyncNeeds` |
+| `src/sync/filters.rs` | ~20 | Partition repos by sync level, build appropriate filters |
+| `src/sync/algorithms.rs` | ~15 | Preserve sync level in relay target derivation |
+| `src/sync/self_subscriber.rs` | ~40 | Register purgatory announcements, handle promotion |
+| `src/purgatory/mod.rs` | ~80 | Add announcement store, soft expiry methods |
+| `src/purgatory/types.rs` | ~20 | Add `AnnouncementPurgatoryEntry` |
+| `src/git/authorization.rs` | ~30 | Query purgatory in `fetch_repository_data()` |
+| `src/nostr/policy/state.rs` | ~40 | Handle soft-expired revival |
+| `src/nostr/policy/announcement.rs` | ~30 | Route to purgatory, check for replacements |
+| `src/git/receive.rs` | ~20 | Trigger promotion on git data |
+
+**Total: ~305 lines of changes**