From c29191b1e1239e931c575a926ec9480e594476d6 Mon Sep 17 00:00:00 2001 From: DanConwayDev Date: Mon, 12 Jan 2026 17:40:25 +0000 Subject: feat(grasp-05): implement archive mode for backup/mirror operation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implements GRASP-05 specification for accepting repository announcements that don't list this relay, enabling archive, mirror, and backup use cases. Core Features: - Three whitelist formats: , /, - Archive-all mode for complete ecosystem mirrors - Fail-fast npub validation at startup - Read-only enforcement (archived repos reject pushes) - Full GRASP-02 sync (git data + Nostr events) - Dynamic archive status (no flags/metadata) Implementation: - Add ArchiveWhitelistEntry enum with Pubkey/Repository/Identifier variants - Add ArchiveConfig with validation and matching logic - Update AnnouncementResult to include AcceptArchive variant - Refactor validate_announcement() to return AnnouncementResult with archive check - Update AnnouncementPolicy with catch-all pattern for cleaner code - Wire archive config through builder and policy layers Configuration: - NGIT_ARCHIVE_ALL: Accept all announcements (⚠️ storage risk) - NGIT_ARCHIVE_WHITELIST: Comma-separated whitelist entries - Updated docs, .env.example, and nix/module.nix Testing: - 28 unit tests for config parsing and whitelist matching - 7 integration tests for archive mode validation - All 296 tests passing Validation Priority: 1. Lists our service → Accept (GRASP-01, read/write) 2. Is maintainer → AcceptMaintainer (multi-maintainer, read/write) 3. Matches archive config → AcceptArchive (GRASP-05, read-only) 4. None of above → Reject Security Considerations: - Archive-all mode has storage/bandwidth DoS risk - Identifier-only format matches any pubkey (use npub/identifier for high-value) - Invalid npubs cause startup failure (fail-fast) Documentation: - Concise explanation focused on rationale - Reference docs updated with all config options - README updated to reflect completed feature - Removed from roadmap, added to compliance section See docs/explanation/grasp-05-archive.md for details. --- docs/explanation/grasp-05-archive.md | 148 +++++++++++++++++++++++++++++++++++ docs/reference/configuration.md | 76 ++++++++++++++++++ 2 files changed, 224 insertions(+) create mode 100644 docs/explanation/grasp-05-archive.md (limited to 'docs') diff --git a/docs/explanation/grasp-05-archive.md b/docs/explanation/grasp-05-archive.md new file mode 100644 index 0000000..e43a87e --- /dev/null +++ b/docs/explanation/grasp-05-archive.md @@ -0,0 +1,148 @@ +# GRASP-05 Archive Mode + +**Purpose:** Understand archive/mirror/backup functionality +**Audience:** Operators and developers + +--- + +## What It Does + +GRASP-05 enables ngit-grasp to accept repository announcements that **don't list your relay**, allowing you to run an archive, mirror, or backup service. + +**Standard GRASP-01:** Announcement must list your service → You host it (read/write) +**GRASP-05 Extension:** Announcement matches your whitelist → You archive it (read-only) + +## Why It Exists + +### Problem +In GRASP-01 strict mode, you can only host repositories whose maintainers explicitly list your relay. This prevents: +- Creating backup archives of critical projects without maintainer cooperation +- Building comprehensive mirrors of the Nostr Git ecosystem +- Providing disaster recovery for projects that might disappear + +### Solution +Archive mode relaxes the "must list service" requirement for whitelisted repositories, enabling passive mirroring while maintaining read-only guarantees. + +## How It Works + +### Three Whitelist Formats + +| Format | Example | Archives | +|--------|---------|----------| +| `` | `npub1alice...` | All repos from Alice | +| `/` | `npub1bob.../linux` | Only Bob's linux repo | +| `` | `bitcoin-core` | Any bitcoin-core repo (⚠️ any pubkey) | + +**Configuration:** +```bash +# Specific repos (safest) +NGIT_ARCHIVE_WHITELIST=npub1torvalds.../linux,npub1satoshi.../bitcoin + +# All repos from trusted maintainers +NGIT_ARCHIVE_WHITELIST=npub1alice...,npub1bob... + +# Archive everything (⚠️ storage risk) +NGIT_ARCHIVE_ALL=true +``` + +### Validation Priority + +Announcements are checked in this order: + +1. **Lists your service?** → `Accept` (GRASP-01, read/write) +2. **Is author a maintainer?** → `AcceptMaintainer` (multi-maintainer, read/write) +3. **Matches archive config?** → `AcceptArchive` (GRASP-05, read-only) +4. **None of the above** → `Reject` + +This ensures GRASP-01 compliant repos are always writable, even if they match the archive whitelist. + +### Storage Model + +Archived repos use the same directory structure as hosted repos: +``` +/ + npub1alice.../ + hosted-repo.git/ # Lists your service (writable) + archived-repo.git/ # Whitelisted (read-only) +``` + +**No flags or metadata** - archive status determined dynamically from config + announcement contents. + +### Full Sync + +Archived repositories trigger complete GRASP-02 sync: +- ✅ Nostr events (PRs, issues, patches) +- ✅ Git data via purgatory +- ✅ Same validation as hosted repos + +Archive mode is a **complete mirror**, not just git-only backup. + +## Security Considerations + +### 1. Archive-All Mode (Dangerous) + +**Don't use `NGIT_ARCHIVE_ALL=true` unless:** +- You have unlimited storage/bandwidth +- You trust the relay network +- You've implemented monitoring + +**Attack vector:** Anyone can publish announcements → unlimited storage consumption. + +### 2. Identifier-Only Format (Risky) + +```bash +NGIT_ARCHIVE_WHITELIST=bitcoin-core # Matches ANY pubkey! +``` + +Malicious users can publish fake repos with popular identifiers. Use `/` for high-value archives. + +### 3. Npub Validation + +Invalid npubs → server fails to start (fail-fast). Identifiers aren't validated (any string allowed). + +## Operational Guide + +### Start Small + +```bash +# Day 1: One critical repo +NGIT_ARCHIVE_WHITELIST=npub1torvalds.../linux + +# Week 1: Add trusted maintainers +NGIT_ARCHIVE_WHITELIST=npub1alice...,npub1bob... + +# Month 1: Consider popular identifiers (with monitoring) +NGIT_ARCHIVE_WHITELIST=npub1alice...,bitcoin-core +``` + +### Monitor Growth + +Watch for: +- Storage consumption rate +- Purgatory git fetch failures +- Bandwidth usage spikes + +### Whitelist Changes + +**Current:** Static config - edit `.env`, restart server +**Future:** REST API for dynamic management (no restart) + +## Comparison: Hosted vs Archived + +| Aspect | Hosted (GRASP-01) | Archived (GRASP-05) | +|--------|-------------------|---------------------| +| Announcement must list you | ✅ Required | ❌ Whitelisted instead | +| Git pushes | ✅ Accepted | ❌ Rejected (read-only) | +| GRASP-02 sync | ✅ Full sync | ✅ Full sync | +| Relay discovery | ✅ Listed | ❌ Not listed | +| Use case | Hosting workspace | Backup/mirror | + +## Related Documentation + +- [Configuration Reference](../reference/configuration.md) - `NGIT_ARCHIVE_*` options +- [GRASP-05 Spec](https://gitworkshop.dev/danconwaydev.com/grasp/05.md) - Protocol specification +- [GRASP-02 Sync](./grasp-02-proactive-sync.md) - How sync works + +--- + +_Part of the [ngit-grasp explanation documentation](./)_ diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index bdd832f..52418ad 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -498,6 +498,82 @@ NGIT_REJECTED_COLD_INDEX_EXPIRY_SECS=1209600 --- +### GRASP-05 Archive Configuration + +These options enable archive/mirror/backup mode per the GRASP-05 specification. + +#### `NGIT_ARCHIVE_ALL` + +**Description:** Accept all repository announcements regardless of whether they list this instance +**Type:** Boolean +**Default:** `false` +**Required:** No + +**Examples:** + +```bash +# Enable archive-all mode (⚠️ WARNING: Storage risk) +NGIT_ARCHIVE_ALL=true + +# Disable (default - GRASP-01 strict mode) +NGIT_ARCHIVE_ALL=false +``` + +**Security Warning:** When enabled, any repository can be mirrored to this relay, potentially causing storage and bandwidth exhaustion. Only enable if you have unlimited resources and trust the relay network. + +**Notes:** + +- Archived repositories are read-only (pushes rejected) +- Full sync enabled (both git data and Nostr events) +- Takes precedence over whitelist (accepts everything) + +--- + +#### `NGIT_ARCHIVE_WHITELIST` + +**Description:** Comma-separated list of repositories/pubkeys/identifiers to archive +**Type:** String (comma-separated) +**Default:** (empty) +**Required:** No + +**Formats:** + +- `` - Archive all repos from this pubkey +- `/` - Archive specific repo from specific pubkey +- `` - Archive repos with this identifier from any pubkey + +**Examples:** + +```bash +# Archive all repos from Alice +NGIT_ARCHIVE_WHITELIST=npub1alice23 + +# Archive specific repos +NGIT_ARCHIVE_WHITELIST=npub1alice23/linux,npub1bob23/bitcoin-core + +# Archive by identifier (any pubkey) +NGIT_ARCHIVE_WHITELIST=bitcoin-core,linux,rust + +# Mixed formats +NGIT_ARCHIVE_WHITELIST=npub1alice23...,npub1bob23.../linux,bitcoin-core +``` + +**Validation:** + +- Npub entries are validated at startup (invalid npub = server fails to start) +- Identifier entries accept any string +- Whitespace is trimmed +- Empty entries are ignored + +**Security Notes:** + +- Identifier-only format (`bitcoin-core`) matches ANY pubkey +- Use `npub/identifier` format for high-value archives +- Whitelist is static (restart required to change) +- Future: Dynamic management via API + +--- + ### Logging Configuration #### `RUST_LOG` -- cgit v1.2.3