diff options
| author | DanConwayDev <DanConwayDev@protonmail.com> | 2026-01-23 14:58:31 +0000 |
|---|---|---|
| committer | DanConwayDev <DanConwayDev@protonmail.com> | 2026-01-27 20:38:03 +0000 |
| commit | 6a6c8cf8b70bc387ea7241b5c9ec457cb525eb40 (patch) | |
| tree | a810ca522872e82e6fc053eb53aa3a3914b310b3 /docs | |
| parent | b61d388162438d12df37aa3fcd40bc9d3344d5bd (diff) | |
Update migration guide with lessons learned from relay.ngit.dev analysis
- Add Gotchas section with common issues: git installation, localhost-only
archive relays, non-standard git paths, service name variations, and
permission requirements
- Add relay.ngit.dev-specific migration notes with actual paths, service
names, and analysis results (315 repos need re-sync, 382 purgatory expired)
- Enhance Running the Analysis section with path discovery guidance
- Expand Troubleshooting section with solutions for git not found, archive
connection failures, and wrong git paths
- Add git --version to prerequisite checks
- Update examples to use realistic localhost archive URLs
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/how-to/migrate-to-ngit-grasp.md | 257 |
1 files changed, 252 insertions, 5 deletions
diff --git a/docs/how-to/migrate-to-ngit-grasp.md b/docs/how-to/migrate-to-ngit-grasp.md index f4dff86..9b812a5 100644 --- a/docs/how-to/migrate-to-ngit-grasp.md +++ b/docs/how-to/migrate-to-ngit-grasp.md | |||
| @@ -62,11 +62,112 @@ See [Running the Analysis](#running-the-analysis) for detailed options. | |||
| 62 | # Check required tools | 62 | # Check required tools |
| 63 | nak --version | 63 | nak --version |
| 64 | jq --version | 64 | jq --version |
| 65 | git --version | ||
| 65 | 66 | ||
| 66 | # Check optional tools (for VPS phases) | 67 | # Check optional tools (for VPS phases) |
| 67 | journalctl --version | 68 | journalctl --version |
| 68 | ``` | 69 | ``` |
| 69 | 70 | ||
| 71 | ## Gotchas and Common Issues | ||
| 72 | |||
| 73 | Before running the analysis, be aware of these common issues discovered during real migrations: | ||
| 74 | |||
| 75 | ### Git Must Be Installed | ||
| 76 | |||
| 77 | The analysis scripts require `git` to be installed and in PATH. This may not be present on minimal VPS installations. | ||
| 78 | |||
| 79 | ```bash | ||
| 80 | # Check if git is available | ||
| 81 | which git || echo "Git not found - install it first" | ||
| 82 | |||
| 83 | # Install on Debian/Ubuntu | ||
| 84 | apt install git | ||
| 85 | |||
| 86 | # Install on NixOS (add to configuration.nix) | ||
| 87 | environment.systemPackages = [ pkgs.git ]; | ||
| 88 | ``` | ||
| 89 | |||
| 90 | ### Archive Relay May Only Be Accessible Locally | ||
| 91 | |||
| 92 | If your archive relay is configured to listen only on localhost (e.g., `ws://localhost:7443`), you must run the analysis **on the VPS itself**, not from a remote machine. | ||
| 93 | |||
| 94 | ```bash | ||
| 95 | # Check if archive relay is accessible | ||
| 96 | # This will fail if run remotely against a localhost-only relay | ||
| 97 | nak req -k 30618 --limit 1 ws://localhost:7443 | ||
| 98 | |||
| 99 | # Solution: SSH into the VPS and run analysis there | ||
| 100 | ssh user@your-vps | ||
| 101 | cd /path/to/scripts | ||
| 102 | ./run-migration-analysis.sh --archive-relay ws://localhost:7443 ... | ||
| 103 | ``` | ||
| 104 | |||
| 105 | ### Git Data Paths May Differ from Defaults | ||
| 106 | |||
| 107 | Different deployments store git data in different locations. **Always verify paths before running the analysis.** | ||
| 108 | |||
| 109 | ```bash | ||
| 110 | # Find actual git data paths from service configuration | ||
| 111 | systemctl cat ngit-relay.service | grep -E 'ExecStart|WorkingDirectory|Environment' | ||
| 112 | systemctl cat ngit-grasp-*.service | grep -E 'ExecStart|WorkingDirectory|Environment' | ||
| 113 | |||
| 114 | # Common locations: | ||
| 115 | # - /var/lib/ngit-relay/git (default) | ||
| 116 | # - /var/lib/ngit-grasp/git (default) | ||
| 117 | # - /persistent/*/data/repos (custom deployments) | ||
| 118 | |||
| 119 | # Verify the path exists and contains expected structure | ||
| 120 | ls /path/to/git/npub1*/ # Should show *.git directories | ||
| 121 | ``` | ||
| 122 | |||
| 123 | ### Phase 4 Needs the Correct Service Name | ||
| 124 | |||
| 125 | Phase 4 extracts structured logs (`[PARSE_FAIL]`, `[PURGATORY_EXPIRED]`) from journald. You must specify the service that has these logs - typically the **archive** service (ngit-grasp), not the production service (ngit-relay). | ||
| 126 | |||
| 127 | ```bash | ||
| 128 | # Find all ngit-related services | ||
| 129 | systemctl list-units 'ngit-*' --all | ||
| 130 | |||
| 131 | # Check which service has structured logging | ||
| 132 | journalctl -u ngit-grasp-*.service | grep -E '\[PARSE_FAIL\]|\[PURGATORY_EXPIRED\]' | head -5 | ||
| 133 | |||
| 134 | # Use the archive service name for Phase 4 | ||
| 135 | ./run-migration-analysis.sh ... --service ngit-grasp-relay-ngit-dev.service | ||
| 136 | ``` | ||
| 137 | |||
| 138 | ### Permission Issues with Service-Owned Directories | ||
| 139 | |||
| 140 | Git data directories are typically owned by the service user and may require elevated permissions to read. | ||
| 141 | |||
| 142 | ```bash | ||
| 143 | # Check directory permissions | ||
| 144 | ls -la /var/lib/ngit-grasp/git | ||
| 145 | |||
| 146 | # Options: | ||
| 147 | # 1. Run as root/sudo | ||
| 148 | sudo ./run-migration-analysis.sh ... | ||
| 149 | |||
| 150 | # 2. Run as the service user | ||
| 151 | sudo -u ngit-grasp ./run-migration-analysis.sh ... | ||
| 152 | |||
| 153 | # 3. Add your user to the service group | ||
| 154 | sudo usermod -aG ngit-grasp $USER | ||
| 155 | # (logout/login required) | ||
| 156 | ``` | ||
| 157 | |||
| 158 | ### Service Names Vary by Deployment | ||
| 159 | |||
| 160 | NixOS multi-instance deployments use service names like `ngit-grasp-<instance>.service`. Always check actual service names. | ||
| 161 | |||
| 162 | ```bash | ||
| 163 | # List all ngit services | ||
| 164 | systemctl list-units 'ngit-*' --all --no-pager | ||
| 165 | |||
| 166 | # Example output: | ||
| 167 | # ngit-relay.service loaded active running ngit-relay | ||
| 168 | # ngit-grasp-relay-ngit-dev.service loaded active running ngit-grasp (relay-ngit-dev) | ||
| 169 | ``` | ||
| 170 | |||
| 70 | ## Migration Overview | 171 | ## Migration Overview |
| 71 | 172 | ||
| 72 | The migration process has three stages: | 173 | The migration process has three stages: |
| @@ -98,6 +199,26 @@ Once all issues are resolved: | |||
| 98 | 199 | ||
| 99 | ## Running the Analysis | 200 | ## Running the Analysis |
| 100 | 201 | ||
| 202 | ### Before You Start | ||
| 203 | |||
| 204 | **Verify paths and service names** before running the analysis. Incorrect paths are the most common source of errors. | ||
| 205 | |||
| 206 | ```bash | ||
| 207 | # 1. Find actual git data paths | ||
| 208 | systemctl cat ngit-relay.service | grep -E 'ExecStart|data|git' | ||
| 209 | systemctl cat ngit-grasp-*.service | grep -E 'ExecStart|data|git' | ||
| 210 | |||
| 211 | # 2. Find service names | ||
| 212 | systemctl list-units 'ngit-*' --all --no-pager | ||
| 213 | |||
| 214 | # 3. Verify git data exists at the paths | ||
| 215 | ls /path/to/prod/git/npub1*/ | head -5 | ||
| 216 | ls /path/to/archive/git/npub1*/ | head -5 | ||
| 217 | |||
| 218 | # 4. Check if archive relay is accessible | ||
| 219 | nak req -k 30618 --limit 1 ws://localhost:7443 # or your archive URL | ||
| 220 | ``` | ||
| 221 | |||
| 101 | ### Basic Usage | 222 | ### Basic Usage |
| 102 | 223 | ||
| 103 | ```bash | 224 | ```bash |
| @@ -115,13 +236,18 @@ Once all issues are resolved: | |||
| 115 | 236 | ||
| 116 | ### Full Analysis on VPS | 237 | ### Full Analysis on VPS |
| 117 | 238 | ||
| 239 | **Important:** If your archive relay is localhost-only, you must run this on the VPS. | ||
| 240 | |||
| 118 | ```bash | 241 | ```bash |
| 242 | # First, discover your actual paths (see "Before You Start" above) | ||
| 243 | # Then run with the correct values: | ||
| 244 | |||
| 119 | ./run-migration-analysis.sh \ | 245 | ./run-migration-analysis.sh \ |
| 120 | --prod-relay wss://source-relay.example.com \ | 246 | --prod-relay wss://source-relay.example.com \ |
| 121 | --archive-relay wss://target-relay.example.com \ | 247 | --archive-relay ws://localhost:7443 \ |
| 122 | --prod-git /var/lib/grasp-relay/git \ | 248 | --prod-git /path/to/prod/git \ |
| 123 | --archive-git /var/lib/ngit-grasp/git \ | 249 | --archive-git /path/to/archive/git \ |
| 124 | --service ngit-grasp.service | 250 | --service ngit-grasp-your-instance.service |
| 125 | ``` | 251 | ``` |
| 126 | 252 | ||
| 127 | ### Phase Control | 253 | ### Phase Control |
| @@ -226,6 +352,21 @@ go install github.com/fiatjaf/nak@latest | |||
| 226 | # Or download binary from releases | 352 | # Or download binary from releases |
| 227 | ``` | 353 | ``` |
| 228 | 354 | ||
| 355 | ### "git not found" | ||
| 356 | |||
| 357 | Git must be installed and in PATH: | ||
| 358 | |||
| 359 | ```bash | ||
| 360 | # Check if git is available | ||
| 361 | which git | ||
| 362 | |||
| 363 | # Install on Debian/Ubuntu | ||
| 364 | sudo apt install git | ||
| 365 | |||
| 366 | # Install on NixOS (add to configuration.nix) | ||
| 367 | environment.systemPackages = [ pkgs.git ]; | ||
| 368 | ``` | ||
| 369 | |||
| 229 | ### "Permission denied" on git directories | 370 | ### "Permission denied" on git directories |
| 230 | 371 | ||
| 231 | Run with sudo or ensure your user has read access: | 372 | Run with sudo or ensure your user has read access: |
| @@ -234,8 +375,38 @@ Run with sudo or ensure your user has read access: | |||
| 234 | # Check permissions | 375 | # Check permissions |
| 235 | ls -la /var/lib/grasp-relay/git | 376 | ls -la /var/lib/grasp-relay/git |
| 236 | 377 | ||
| 237 | # Run with sudo if needed | 378 | # Option 1: Run with sudo |
| 238 | sudo ./run-migration-analysis.sh ... | 379 | sudo ./run-migration-analysis.sh ... |
| 380 | |||
| 381 | # Option 2: Run as service user | ||
| 382 | sudo -u ngit-grasp ./run-migration-analysis.sh ... | ||
| 383 | ``` | ||
| 384 | |||
| 385 | ### Archive relay connection failed | ||
| 386 | |||
| 387 | If you get connection errors to the archive relay: | ||
| 388 | |||
| 389 | ```bash | ||
| 390 | # Check if relay is running | ||
| 391 | systemctl status ngit-grasp-*.service | ||
| 392 | |||
| 393 | # Check if it's localhost-only | ||
| 394 | # If archive is ws://localhost:7443, you MUST run on the VPS | ||
| 395 | ssh user@your-vps | ||
| 396 | ./run-migration-analysis.sh --archive-relay ws://localhost:7443 ... | ||
| 397 | ``` | ||
| 398 | |||
| 399 | ### Wrong git paths / "No such file or directory" | ||
| 400 | |||
| 401 | Git data paths vary by deployment. Discover the actual paths: | ||
| 402 | |||
| 403 | ```bash | ||
| 404 | # Find paths from service configuration | ||
| 405 | systemctl cat ngit-relay.service | grep -E 'ExecStart|WorkingDirectory|Environment' | ||
| 406 | systemctl cat ngit-grasp-*.service | grep -E 'ExecStart|WorkingDirectory|Environment' | ||
| 407 | |||
| 408 | # Verify the path contains git repos | ||
| 409 | ls /discovered/path/npub1*/ | ||
| 239 | ``` | 410 | ``` |
| 240 | 411 | ||
| 241 | ### Phase 2 takes too long | 412 | ### Phase 2 takes too long |
| @@ -254,6 +425,18 @@ This is expected if: | |||
| 254 | 425 | ||
| 255 | The analysis will continue without log data. | 426 | The analysis will continue without log data. |
| 256 | 427 | ||
| 428 | ### Phase 4 finds no structured logs | ||
| 429 | |||
| 430 | Structured logging (`[PARSE_FAIL]`, `[PURGATORY_EXPIRED]`) is only available in ngit-grasp. If checking an ngit-relay service, no structured logs will be found. | ||
| 431 | |||
| 432 | ```bash | ||
| 433 | # Verify you're checking the right service (should be ngit-grasp) | ||
| 434 | journalctl -u ngit-grasp-*.service | grep -E '\[PARSE_FAIL\]|\[PURGATORY_EXPIRED\]' | head -5 | ||
| 435 | |||
| 436 | # If checking ngit-relay, structured logs won't exist | ||
| 437 | # Use --service with the ngit-grasp archive service name instead | ||
| 438 | ``` | ||
| 439 | |||
| 257 | ### Event counts are multiples of 250 | 440 | ### Event counts are multiples of 250 |
| 258 | 441 | ||
| 259 | This suggests pagination may have failed. The scripts use `--paginate` by default, but if you see exactly 250, 500, 750 events, verify the relay is responding correctly. | 442 | This suggests pagination may have failed. The scripts use `--paginate` by default, but if you see exactly 250, 500, 750 events, verify the relay is responding correctly. |
| @@ -410,3 +593,67 @@ For advanced usage, you can run individual phase scripts: | |||
| 410 | ``` | 593 | ``` |
| 411 | 594 | ||
| 412 | Each script has detailed help available with `--help` or by reading the script header. | 595 | Each script has detailed help available with `--help` or by reading the script header. |
| 596 | |||
| 597 | ## relay.ngit.dev Migration Notes | ||
| 598 | |||
| 599 | This section documents the specific configuration and lessons learned from migrating relay.ngit.dev from ngit-relay to ngit-grasp. Use this as a reference for similar deployments. | ||
| 600 | |||
| 601 | ### Deployment Configuration | ||
| 602 | |||
| 603 | | Component | Value | | ||
| 604 | |-----------|-------| | ||
| 605 | | **Production relay** | `wss://relay.ngit.dev` | | ||
| 606 | | **Production service** | `ngit-relay.service` | | ||
| 607 | | **Production git path** | `/persistent/relay-ngit-dev-ngit-relay/data/repos` | | ||
| 608 | | **Archive relay** | `ws://localhost:7443` (localhost only) | | ||
| 609 | | **Archive service** | `ngit-grasp-relay-ngit-dev.service` | | ||
| 610 | | **Archive git path** | `/persistent/grasp/relay-ngit-dev/git` | | ||
| 611 | |||
| 612 | ### Key Differences from Defaults | ||
| 613 | |||
| 614 | 1. **Git paths are non-standard**: The production relay uses `/persistent/relay-ngit-dev-ngit-relay/data/repos` instead of `/var/lib/ngit-relay/git` | ||
| 615 | |||
| 616 | 2. **Archive is localhost-only**: The archive relay listens on `ws://localhost:7443`, not a public URL. All analysis must run on the VPS. | ||
| 617 | |||
| 618 | 3. **Service names include instance**: NixOS multi-instance deployment uses `ngit-grasp-relay-ngit-dev.service`, not `ngit-grasp.service` | ||
| 619 | |||
| 620 | ### Analysis Command | ||
| 621 | |||
| 622 | ```bash | ||
| 623 | # Run on VPS (archive is localhost-only) | ||
| 624 | ./docs/how-to/migration-scripts/run-migration-analysis.sh \ | ||
| 625 | --prod-relay wss://relay.ngit.dev \ | ||
| 626 | --archive-relay ws://localhost:7443 \ | ||
| 627 | --prod-git /persistent/relay-ngit-dev-ngit-relay/data/repos \ | ||
| 628 | --archive-git /persistent/grasp/relay-ngit-dev/git \ | ||
| 629 | --service ngit-grasp-relay-ngit-dev.service | ||
| 630 | ``` | ||
| 631 | |||
| 632 | ### Analysis Results (January 2026) | ||
| 633 | |||
| 634 | | Category | Count | Notes | | ||
| 635 | |----------|-------|-------| | ||
| 636 | | Complete in both | ~400 | Ready for migration | | ||
| 637 | | Complete in prod, missing from archive | 315 | Need re-sync | | ||
| 638 | | Empty in both | 100 | Users never pushed git data | | ||
| 639 | | Manual investigation | 5 | Unusual states | | ||
| 640 | | Purgatory expired | 382 | Structured logging working | | ||
| 641 | |||
| 642 | ### Lessons Learned | ||
| 643 | |||
| 644 | 1. **Always verify paths first**: The default paths in examples didn't match the actual deployment. Use `systemctl cat <service>` to find real paths. | ||
| 645 | |||
| 646 | 2. **Check archive accessibility**: We initially tried to run analysis remotely, but the archive relay was localhost-only. Had to SSH to VPS. | ||
| 647 | |||
| 648 | 3. **Use archive service for Phase 4**: Structured logging (`[PARSE_FAIL]`, `[PURGATORY_EXPIRED]`) is in the ngit-grasp archive service, not the ngit-relay production service. | ||
| 649 | |||
| 650 | 4. **Install git on VPS**: Git wasn't installed on the minimal VPS. The scripts now check for this in prerequisites. | ||
| 651 | |||
| 652 | 5. **Permissions matter**: Some directories required `sudo` to access. Running as root or the service user resolved this. | ||
| 653 | |||
| 654 | ### Next Steps for relay.ngit.dev | ||
| 655 | |||
| 656 | 1. **Re-sync 315 repos**: Trigger archive to re-fetch from production | ||
| 657 | 2. **Investigate 5 edge cases**: Manual review of unusual states | ||
| 658 | 3. **Monitor purgatory**: 382 expired entries indicate sync issues to investigate | ||
| 659 | 4. **Plan cutover**: Once re-sync complete, switch DNS/proxy to ngit-grasp | ||