1 files changed, 412 insertions, 0 deletions

diff --git a/docs/how-to/migrate-to-ngit-grasp.md b/docs/how-to/migrate-to-ngit-grasp.md
new file mode 100644
index 0000000..f4dff86
--- /dev/null
+++ b/docs/how-to/migrate-to-ngit-grasp.md
@@ -0,0 +1,412 @@
+# Migrate to ngit-grasp from another GRASP implementation
+
+This guide walks you through migrating a production GRASP relay to ngit-grasp. The process involves analyzing your existing data to identify repositories that need attention before switching over.
+
+## Compatibility
+
+This migration process works with any GRASP implementation that:
+
+- Stores git data in the `<npub>/<identifier>.git` directory structure
+- Uses standard GRASP events (kind 30617 announcements, kind 30618 state, kind 5 deletions)
+- Exposes a Nostr relay WebSocket endpoint
+
+**Known compatible implementations:**
+- ngit-relay (reference implementation)
+- ngit-grasp (when migrating between instances or from archive mode)
+- Other GRASP-compliant relays following the specification
+
+The migration scripts analyze Nostr events and git data directly, making them implementation-agnostic.
+
+## Quick Start
+
+Run the migration analysis with a single command:
+
+```bash
+# Basic analysis (fetches events, compares relays)
+./docs/how-to/migration-scripts/run-migration-analysis.sh \
+  --prod-relay wss://source-relay.example.com \
+  --archive-relay wss://target-relay.example.com
+
+# Full analysis (includes git sync check - run on VPS)
+./docs/how-to/migration-scripts/run-migration-analysis.sh \
+  --prod-relay wss://source-relay.example.com \
+  --archive-relay wss://target-relay.example.com \
+  --prod-git /var/lib/grasp-relay/git \
+  --archive-git /var/lib/ngit-grasp/git \
+  --service ngit-grasp.service
+```
+
+The script produces three output files:
+- `results/no-action-required.txt` - Repos ready for migration
+- `results/action-required.txt` - Repos needing intervention
+- `results/manual-investigation.txt` - Repos needing human review
+
+See [Running the Analysis](#running-the-analysis) for detailed options.
+
+## Prerequisites
+
+### Required Tools
+
+- **nak** - Nostr Army Knife for fetching events ([install](https://github.com/fiatjaf/nak))
+- **jq** - JSON processing (install via package manager)
+
+### For Full Analysis (VPS)
+
+- SSH access to the VPS running your source relay
+- Read access to git data directories
+- Access to systemd journal (for log extraction)
+
+### Verify Installation
+
+```bash
+# Check required tools
+nak --version
+jq --version
+
+# Check optional tools (for VPS phases)
+journalctl --version
+```
+
+## Migration Overview
+
+The migration process has three stages:
+
+### Stage 1: Deploy Archive Instance
+
+Deploy ngit-grasp alongside your production relay:
+
+1. Configure ngit-grasp with:
+   - `domain` set to `<prod-domain>.internal` (temporary)
+   - `archiveService` set to your production domain
+   - Running on a different port
+
+2. Let it sync for ~1 hour to gather all events and git data
+
+### Stage 2: Analyze Data
+
+Run the migration analysis to identify:
+- Repositories successfully migrated (no action needed)
+- Repositories with incomplete data (need investigation)
+- Repositories with parse failures (may need re-announcement)
+
+### Stage 3: Switch Over
+
+Once all issues are resolved:
+1. Set `domain` to your production URL
+2. Disable archive mode
+3. Update your reverse proxy to point to ngit-grasp
+
+## Running the Analysis
+
+### Basic Usage
+
+```bash
+# Preview what will happen (dry run)
+./run-migration-analysis.sh \
+  --prod-relay wss://source-relay.example.com \
+  --archive-relay wss://target-relay.example.com \
+  --dry-run
+
+# Run the analysis
+./run-migration-analysis.sh \
+  --prod-relay wss://source-relay.example.com \
+  --archive-relay wss://target-relay.example.com
+```
+
+### Full Analysis on VPS
+
+```bash
+./run-migration-analysis.sh \
+  --prod-relay wss://source-relay.example.com \
+  --archive-relay wss://target-relay.example.com \
+  --prod-git /var/lib/grasp-relay/git \
+  --archive-git /var/lib/ngit-grasp/git \
+  --service ngit-grasp.service
+```
+
+### Phase Control
+
+Skip or run specific phases:
+
+```bash
+# Skip Phase 2 (use cached git sync data)
+./run-migration-analysis.sh ... --skip-phase-2
+
+# Run only Phase 1 (fetch events)
+./run-migration-analysis.sh ... --only-phase-1
+
+# Resume from Phase 3 (using existing data)
+./run-migration-analysis.sh ... --from-phase-3 --output work/migration-analysis-20260122-1430
+```
+
+### All Options
+
+| Option | Description |
+|--------|-------------|
+| `--prod-relay <url>` | Source relay WebSocket URL (required) |
+| `--archive-relay <url>` | Target relay WebSocket URL (required) |
+| `--prod-git <path>` | Git base directory for prod (enables Phase 2) |
+| `--archive-git <path>` | Git base directory for archive (enables Phase 2) |
+| `--service <name>` | Systemd service name (enables Phase 4) |
+| `--output <dir>` | Output directory (default: auto-generated) |
+| `--skip-phase-N` | Skip phase N (1-5) |
+| `--only-phase-N` | Run only phase N |
+| `--from-phase-N` | Start from phase N |
+| `--dry-run` | Show what would be executed |
+| `--continue-on-error` | Continue even if a phase fails |
+
+## Understanding Results
+
+### Summary File
+
+The `results/summary.txt` file provides an overview:
+
+```
+## Overview
+
+| Category | Count | Percentage |
+|----------|-------|------------|
+| No Action Required | 450 | 85.7% |
+| Action Required | 52 | 9.9% |
+| Manual Investigation | 23 | 4.4% |
+```
+
+### No Action Required
+
+Repositories in `no-action-required.txt` are ready for migration:
+
+```
+myrepo | npub1abc... | complete in both prod and archive
+oldrepo | npub1def... | deleted by user
+testrepo | npub1ghi... | empty/blank in both (user never pushed)
+```
+
+**Common reasons:**
+- `complete in both prod and archive` - Successfully migrated
+- `deleted by user` - User requested deletion (kind 5 event)
+- `empty/blank in both` - No git data was ever pushed
+- `purgatory expired` - System already handled the timeout
+
+### Action Required
+
+Repositories in `action-required.txt` need intervention:
+
+```
+myrepo | npub1abc... | complete in prod, missing from archive | trigger re-sync or investigate
+otherrepo | npub1def... | incomplete in both (prod=cat3, archive=cat2) | investigate git data source
+```
+
+**Common actions:**
+- **Re-sync needed**: Trigger the archive to re-fetch from the source
+- **Wait for sync**: Archive sync may still be in progress
+- **Investigate git source**: Original git data may be incomplete
+- **Fix parse failure**: Event format issue, may need re-announcement
+
+### Manual Investigation
+
+Repositories in `manual-investigation.txt` have unusual states:
+
+```
+weirdrepo | npub1abc... | in archive (cat1) but not in prod | may be new announcement or deleted from prod
+conflictrepo | npub1def... | complete in prod, missing from archive, parse failure logged | investigate parse failure
+```
+
+These require human judgment to determine the correct action.
+
+## Troubleshooting
+
+### "nak not found"
+
+Install nak from https://github.com/fiatjaf/nak:
+
+```bash
+# Using Go
+go install github.com/fiatjaf/nak@latest
+
+# Or download binary from releases
+```
+
+### "Permission denied" on git directories
+
+Run with sudo or ensure your user has read access:
+
+```bash
+# Check permissions
+ls -la /var/lib/grasp-relay/git
+
+# Run with sudo if needed
+sudo ./run-migration-analysis.sh ...
+```
+
+### Phase 2 takes too long
+
+The git sync check processes each repository individually (~20 minutes total). To speed up iteration:
+
+1. Run Phase 2 once and save the output
+2. Use `--skip-phase-2` for subsequent runs
+3. Use `--from-phase-3` to re-run classification with existing data
+
+### No parse failures found
+
+This is expected if:
+- ngit-grasp logging improvements aren't deployed yet
+- No events actually failed to parse
+
+The analysis will continue without log data.
+
+### Event counts are multiples of 250
+
+This suggests pagination may have failed. The scripts use `--paginate` by default, but if you see exactly 250, 500, 750 events, verify the relay is responding correctly.
+
+## Architecture
+
+### Analysis Phases
+
+The analysis is split into 5 modular phases:
+
+| Phase | Name | Time | Location | Description |
+|-------|------|------|----------|-------------|
+| 1 | Fetch Events | ~30s each | Local | Fetch events from both relays |
+| 2 | Git Sync Check | ~20 min each | VPS | Compare state events to git data |
+| 3 | Categorize & Compare | <1s | Local | Categorize and compare results |
+| 4 | Extract Logs | <30s | VPS | Extract parse failures and purgatory expiry |
+| 5 | Final Classification | <5s | Local | Combine all data into actionable results |
+
+### Phase Flow Diagram
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│ PHASE 1: Fetch Events (~30s, local)                             │
+│ Fetches kind 30618 (state), 30617 (announcements), 5 (deletion) │
+│ Run twice: once for prod, once for archive                      │
+└─────────────────────────────────────────────────────────────────┘
+                              ↓
+┌─────────────────────────────────────────────────────────────────┐
+│ PHASE 2: Git Sync Check (~20 mins, VPS required)                │
+│ Compares state event refs to actual git data on disk            │
+│ Categorizes into: complete, empty, partial, no-match            │
+└─────────────────────────────────────────────────────────────────┘
+                              ↓
+┌─────────────────────────────────────────────────────────────────┐
+│ PHASE 3: Categorize & Compare (fast, local)                     │
+│ Compares prod vs archive categories                             │
+│ Identifies gaps and sync issues                                 │
+└─────────────────────────────────────────────────────────────────┘
+                              ↓
+┌─────────────────────────────────────────────────────────────────┐
+│ PHASE 4: Log-Based Categories (VPS required)                    │
+│ Extracts [PARSE_FAIL] and [PURGATORY_EXPIRED] from logs         │
+│ Provides context for why repos failed to sync                   │
+└─────────────────────────────────────────────────────────────────┘
+                              ↓
+┌─────────────────────────────────────────────────────────────────┐
+│ PHASE 5: Final Classification (fast, local)                     │
+│ Combines all data sources                                       │
+│ Outputs: no-action, action-required, manual-investigation       │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+### Git Sync Categories
+
+Phase 2 categorizes repositories into 4 categories:
+
+| Category | Description | Meaning |
+|----------|-------------|---------|
+| 1 | Complete Match | All refs in state event match git data |
+| 2 | Empty/Blank | No git data available |
+| 3 | Partial Match | Some refs match, some don't |
+| 4 | No Match | Git data exists but refs don't match |
+
+### Output Directory Structure
+
+```
+work/migration-analysis-YYYYMMDD-HHMM/
+├── prod/
+│   ├── raw/
+│   │   ├── state-events.json       # Phase 1
+│   │   ├── announcements.json      # Phase 1
+│   │   └── deletions.json          # Phase 1
+│   ├── git-sync-status.tsv         # Phase 2
+│   └── category*.txt               # Phase 2/3
+├── archive/
+│   └── (same structure as prod)
+├── comparison/
+│   ├── complete-in-both.txt        # Phase 3
+│   ├── complete-prod-missing-archive.txt
+│   ├── complete-prod-incomplete-archive.txt
+│   ├── incomplete-in-both.txt
+│   ├── in-archive-not-prod.txt
+│   └── summary.txt
+├── logs/
+│   ├── parse-failures.txt          # Phase 4
+│   └── purgatory-expired.txt       # Phase 4
+└── results/
+    ├── no-action-required.txt      # Phase 5
+    ├── action-required.txt         # Phase 5
+    ├── manual-investigation.txt    # Phase 5
+    └── summary.txt                 # Phase 5
+```
+
+## Why Migration May Require Attention
+
+Different GRASP implementations may handle edge cases differently. ngit-grasp has stricter validation and better observability, which can surface issues that were previously hidden:
+
+| Aspect | Typical Source Relay | ngit-grasp |
+|--------|---------------------|------------|
+| Git data validation | May accept partial data | Requires all git data to reproduce state |
+| PR refs cleanup | May not clear `refs/nostr/<event-id>` | Properly manages PR refs |
+| Parse failures | May silently ignore | Logs structured `[PARSE_FAIL]` entries |
+| Sync timeout | May have no timeout | Purgatory expires after configurable period |
+
+These differences explain why some repositories may need attention during migration - ngit-grasp's stricter validation catches issues that other implementations may have silently accepted.
+
+## Next Steps
+
+After running the analysis:
+
+1. **Review the summary** - Check `results/summary.txt` for the overview
+2. **Address action items** - Work through `results/action-required.txt`
+3. **Investigate edge cases** - Review `results/manual-investigation.txt`
+4. **Re-run analysis** - After fixing issues, re-run to verify
+5. **Plan cutover** - Schedule the switch when all issues are resolved
+
+### When to Re-run
+
+Re-run the analysis when:
+- Archive sync has had time to complete
+- You've fixed parse failures or re-announced events
+- You want to verify fixes before cutover
+
+```bash
+# Re-run with existing Phase 2 data (faster)
+./run-migration-analysis.sh ... --skip-phase-2 --output work/migration-analysis-20260122-1430
+```
+
+## Individual Scripts
+
+For advanced usage, you can run individual phase scripts:
+
+```bash
+# Phase 1: Fetch events
+./migration-scripts/01-fetch-events.sh wss://source-relay.example.com output/prod
+
+# Phase 2: Git sync check
+./migration-scripts/10-check-git-sync.sh output/prod/raw/state-events.json /var/lib/grasp-relay/git output/prod --categorize
+
+# Phase 3a: Categorize
+./migration-scripts/20-categorize.sh output/prod/git-sync-status.tsv output/prod
+
+# Phase 3b: Compare relays
+./migration-scripts/21-compare-relays.sh output/prod output/archive output/comparison
+
+# Phase 4a: Extract parse failures
+./migration-scripts/30-extract-parse-failures.sh ngit-grasp.service output/logs
+
+# Phase 4b: Extract purgatory expiry
+./migration-scripts/31-extract-purgatory-expiry.sh ngit-grasp.service output/logs
+
+# Phase 5: Final classification
+./migration-scripts/40-classify-actions.sh work/migration-analysis-20260122-1430
+```
+
+Each script has detailed help available with `--help` or by reading the script header.