upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--docs/explanation/purgatory-sync-redesign.md171
1 files changed, 160 insertions, 11 deletions
diff --git a/docs/explanation/purgatory-sync-redesign.md b/docs/explanation/purgatory-sync-redesign.md
index c1e6af6..dd59870 100644
--- a/docs/explanation/purgatory-sync-redesign.md
+++ b/docs/explanation/purgatory-sync-redesign.md
@@ -1272,7 +1272,8 @@ impl MockSyncContext {
1272| ThrottleManager | `src/purgatory/sync/throttle.rs` | Phase 3 | 1272| ThrottleManager | `src/purgatory/sync/throttle.rs` | Phase 3 |
1273| Core sync functions | `src/purgatory/sync/functions.rs` | Phase 5-6 | 1273| Core sync functions | `src/purgatory/sync/functions.rs` | Phase 5-6 |
1274| Queue integration | `src/purgatory/mod.rs` | Phase 7 | 1274| Queue integration | `src/purgatory/mod.rs` | Phase 7 |
1275| Integration tests | `tests/purgatory_sync.rs` | Phase 10 | 1275| Unified function helpers | `src/git/sync.rs` | Phase 9 |
1276| Integration tests | `tests/purgatory_sync.rs` | Phase 12 |
1276 1277
1277## Implementation Phases 1278## Implementation Phases
1278 1279
@@ -1637,7 +1638,146 @@ impl Purgatory {
1637 1638
1638--- 1639---
1639 1640
1640### Phase 9: RealSyncContext Implementation 1641### Phase 9: Unified `process_newly_available_git_data` Function
1642
1643**Goal**: Implement the unified function that handles all post-git-data-available processing.
1644
1645This is the core function described in [Unified Git Data Sync](unify-git-data-sync.md). It will be called by both:
1646- `handle_receive_pack` after a successful git push
1647- `RealSyncContext::process_newly_available_git_data` after purgatory sync fetches OIDs
1648
1649**Files**:
1650- `src/git/sync.rs` (new)
1651- `src/purgatory/mod.rs` (extend - add secondary index for PR events by identifier)
1652
1653**Deliverables**:
1654```rust
1655/// Result of processing newly available git data
1656#[derive(Debug, Default)]
1657pub struct ProcessResult {
1658 pub states_released: usize,
1659 pub prs_released: usize,
1660 pub repos_synced: usize,
1661 pub refs_created: usize,
1662 pub refs_updated: usize,
1663 pub refs_deleted: usize,
1664 pub errors: Vec<String>,
1665}
1666
1667/// Unified processing of newly available git data.
1668///
1669/// Called whenever git data becomes available, whether from:
1670/// - A successful `git push` (handle_receive_pack)
1671/// - Purgatory sync fetching OIDs from remote servers
1672pub async fn process_newly_available_git_data(
1673 source_repo_path: &Path,
1674 new_oids: &HashSet<String>,
1675 database: &SharedDatabase,
1676 local_relay: Option<&nostr_relay_builder::LocalRelay>,
1677 purgatory: &Purgatory,
1678 git_data_path: &Path,
1679) -> Result<ProcessResult>;
1680
1681// Helper functions
1682fn extract_identifier_from_repo_path(repo_path: &Path, git_data_path: &Path) -> Option<String>;
1683fn extract_identifier_from_pr_event(event: &Event) -> Option<String>;
1684
1685// Purgatory additions
1686impl Purgatory {
1687 /// Find all PR events for an identifier (uses secondary index)
1688 pub fn find_prs_for_identifier(&self, identifier: &str) -> Vec<PrPurgatoryEntry>;
1689}
1690```
1691
1692**Unit Tests** (3 tests):
1693```rust
1694#[cfg(test)]
1695mod tests {
1696 #[test]
1697 fn extract_identifier_from_repo_path_valid() {
1698 // {git_data_path}/{npub}/{identifier}.git → identifier
1699 }
1700
1701 #[test]
1702 fn extract_identifier_from_pr_event_valid() {
1703 // Event with "a" tag "30617:<pubkey>:<identifier>" → identifier
1704 }
1705
1706 #[test]
1707 fn extract_identifier_from_pr_event_missing_tag() {
1708 // Event without "a" tag → None
1709 }
1710}
1711```
1712
1713**Success Criteria**:
1714- [ ] `process_newly_available_git_data` discovers satisfiable events from purgatory
1715- [ ] State events: syncs OIDs to owner repos, aligns refs, sets HEAD, saves to DB, notifies WS, removes from purgatory
1716- [ ] PR events: syncs commit to owner repos, creates refs/nostr/<event-id>, saves to DB, notifies WS, removes from purgatory
1717- [ ] Secondary index `pr_events_by_identifier` maintained correctly
1718- [ ] All 3 unit tests pass
1719
1720---
1721
1722### Phase 10: Update `handle_receive_pack` to Use Unified Function
1723
1724**Goal**: Refactor the push authorization handler to use `process_newly_available_git_data`.
1725
1726This replaces ~100 lines of duplicated post-push processing with a single call to the unified function.
1727
1728**Files**:
1729- `src/git/handlers.rs` (modify)
1730
1731**Before** (current code):
1732```rust
1733// After git receive-pack succeeds:
1734// - try_set_head_if_available()
1735// - database.save_event()
1736// - remove_state_event() / remove_pr()
1737// - relay.notify_event()
1738// - sync_to_owner_repos()
1739// - sync_pr_refs_to_tagged_owner_repos()
1740// ... ~100 lines of processing
1741```
1742
1743**After** (simplified):
1744```rust
1745// After git receive-pack succeeds:
1746let new_oids: HashSet<String> = pushed_refs
1747 .iter()
1748 .filter(|(_, new_oid, _)| new_oid != "0000000000000000000000000000000000000000")
1749 .map(|(_, new_oid, _)| new_oid.clone())
1750 .collect();
1751
1752let result = process_newly_available_git_data(
1753 &repo_path,
1754 &new_oids,
1755 &database,
1756 Some(&relay),
1757 &purgatory,
1758 Path::new(git_data_path),
1759).await;
1760
1761info!(
1762 "Processed push: {} states, {} PRs released, {} repos synced",
1763 result.states_released,
1764 result.prs_released,
1765 result.repos_synced
1766);
1767```
1768
1769**Unit Tests** (0 tests):
1770- Behavior tested via existing integration tests which should continue to pass
1771
1772**Success Criteria**:
1773- [ ] `handle_receive_pack` uses `process_newly_available_git_data`
1774- [ ] Duplicate code removed (~100 lines)
1775- [ ] All existing push-related tests still pass
1776- [ ] Push behavior unchanged (same events saved, same refs created)
1777
1778---
1779
1780### Phase 11: RealSyncContext Implementation
1641 1781
1642**Goal**: Implement the production `SyncContext` that connects to real systems. 1782**Goal**: Implement the production `SyncContext` that connects to real systems.
1643 1783
@@ -1663,18 +1803,18 @@ impl SyncContext for RealSyncContext { ... }
1663**Success Criteria**: 1803**Success Criteria**:
1664- [ ] All `SyncContext` methods implemented 1804- [ ] All `SyncContext` methods implemented
1665- [ ] Connects to real database, git, and relay 1805- [ ] Connects to real database, git, and relay
1666- [ ] `process_newly_available_git_data` releases events from purgatory 1806- [ ] `process_newly_available_git_data` method delegates to unified function from Phase 9
1667 1807
1668--- 1808---
1669 1809
1670### Phase 10: Integration Tests 1810### Phase 12: Integration Tests
1671 1811
1672**Goal**: Verify end-to-end sync behavior with real relay instances. 1812**Goal**: Verify end-to-end sync behavior with real relay instances.
1673 1813
1674**Files**: 1814**Files**:
1675- `tests/purgatory_sync.rs` (new) 1815- `tests/purgatory_sync.rs` (new)
1676 1816
1677**Integration Tests** (4 tests): 1817**Integration Tests** (5 tests):
1678```rust 1818```rust
1679#[tokio::test] 1819#[tokio::test]
1680async fn state_event_syncs_from_remote() { 1820async fn state_event_syncs_from_remote() {
@@ -1695,17 +1835,24 @@ async fn concurrent_state_and_pr_sync() {
1695async fn partial_oid_aggregation_from_multiple_servers() { 1835async fn partial_oid_aggregation_from_multiple_servers() {
1696 // OIDs aggregated when no single server has all 1836 // OIDs aggregated when no single server has all
1697} 1837}
1838
1839#[tokio::test]
1840async fn push_triggers_unified_processing() {
1841 // Git push triggers process_newly_available_git_data
1842 // Verifies Phase 10 integration
1843}
1698``` 1844```
1699 1845
1700**Success Criteria**: 1846**Success Criteria**:
1701- [ ] All 4 integration tests pass 1847- [ ] All 5 integration tests pass
1702- [ ] State events release after git sync 1848- [ ] State events release after git sync
1703- [ ] PR events release after commit sync 1849- [ ] PR events release after commit sync
1704- [ ] Partial OID scenarios handled correctly 1850- [ ] Partial OID scenarios handled correctly
1851- [ ] Push path uses unified function (same behavior as purgatory sync)
1705 1852
1706--- 1853---
1707 1854
1708### Phase 11: Cleanup 1855### Phase 13: Cleanup
1709 1856
1710**Goal**: Remove old `start_state_sync` code and wire up new system. 1857**Goal**: Remove old `start_state_sync` code and wire up new system.
1711 1858
@@ -1738,10 +1885,12 @@ async fn partial_oid_aggregation_from_multiple_servers() {
1738| 6. sync_identifier | 2 | - | 2 | 1885| 6. sync_identifier | 2 | - | 2 |
1739| 7. Queue Integration | 1 | - | 1 | 1886| 7. Queue Integration | 1 | - | 1 |
1740| 8. Sync Loop | 0 | - | 0 | 1887| 8. Sync Loop | 0 | - | 0 |
1741| 9. RealSyncContext | 0 | - | 0 | 1888| 9. Unified Function | 3 | - | 3 |
1742| 10. Integration | - | 4 | 4 | 1889| 10. Push Handler Update | 0 | - | 0 |
1743| 11. Cleanup | 0 | - | 0 | 1890| 11. RealSyncContext | 0 | - | 0 |
1744| **Total** | **13** | **4** | **17** | 1891| 12. Integration | - | 5 | 5 |
1892| 13. Cleanup | 0 | - | 0 |
1893| **Total** | **16** | **5** | **21** |
1745 1894
1746## Configuration 1895## Configuration
1747 1896