upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/how-to/migrate-to-ngit-grasp.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/how-to/migrate-to-ngit-grasp.md')
-rw-r--r--docs/how-to/migrate-to-ngit-grasp.md69
1 files changed, 60 insertions, 9 deletions
diff --git a/docs/how-to/migrate-to-ngit-grasp.md b/docs/how-to/migrate-to-ngit-grasp.md
index 9b812a5..00af6c8 100644
--- a/docs/how-to/migrate-to-ngit-grasp.md
+++ b/docs/how-to/migrate-to-ngit-grasp.md
@@ -122,19 +122,36 @@ ls /path/to/git/npub1*/ # Should show *.git directories
122 122
123### Phase 4 Needs the Correct Service Name 123### Phase 4 Needs the Correct Service Name
124 124
125Phase 4 extracts structured logs (`[PARSE_FAIL]`, `[PURGATORY_EXPIRED]`) from journald. You must specify the service that has these logs - typically the **archive** service (ngit-grasp), not the production service (ngit-relay). 125> **CRITICAL:** Phase 4 extracts structured logs (`[PARSE_FAIL]`, `[PURGATORY_EXPIRED]`) from journald. These logs **ONLY exist in ngit-grasp services**, NOT in ngit-relay services.
126
127If you specify an ngit-relay service (like `ngit-relay.service`), Phase 4 will find **zero logs** and produce empty results. This is a common mistake that wastes time and produces misleading analysis.
128
129**Correct service names (ngit-grasp):**
130- `ngit-grasp.service`
131- `ngit-grasp-relay-ngit-dev.service` (NixOS multi-instance)
132- `ngit-grasp-archive.service`
133
134**Incorrect service names (ngit-relay - NO structured logging):**
135- `ngit-relay.service`
136- `relay-ngit-dev.service`
126 137
127```bash 138```bash
128# Find all ngit-related services 139# Find all ngit-related services
129systemctl list-units 'ngit-*' --all 140systemctl list-units 'ngit-*' --all
130 141
131# Check which service has structured logging 142# Check which service has structured logging (should be ngit-grasp)
132journalctl -u ngit-grasp-*.service | grep -E '\[PARSE_FAIL\]|\[PURGATORY_EXPIRED\]' | head -5 143journalctl -u ngit-grasp-*.service | grep -E '\[PARSE_FAIL\]|\[PURGATORY_EXPIRED\]' | head -5
133 144
145# Verify ngit-relay does NOT have structured logging
146journalctl -u ngit-relay.service | grep -E '\[PARSE_FAIL\]|\[PURGATORY_EXPIRED\]' | head -5
147# ^ This should return nothing
148
134# Use the archive service name for Phase 4 149# Use the archive service name for Phase 4
135./run-migration-analysis.sh ... --service ngit-grasp-relay-ngit-dev.service 150./run-migration-analysis.sh ... --service ngit-grasp-relay-ngit-dev.service
136``` 151```
137 152
153The migration scripts now validate the service name and will **error** if you specify an ngit-relay service, preventing this common mistake.
154
138### Permission Issues with Service-Owned Directories 155### Permission Issues with Service-Owned Directories
139 156
140Git data directories are typically owned by the service user and may require elevated permissions to read. 157Git data directories are typically owned by the service user and may require elevated permissions to read.
@@ -273,7 +290,7 @@ Skip or run specific phases:
273| `--archive-relay <url>` | Target relay WebSocket URL (required) | 290| `--archive-relay <url>` | Target relay WebSocket URL (required) |
274| `--prod-git <path>` | Git base directory for prod (enables Phase 2) | 291| `--prod-git <path>` | Git base directory for prod (enables Phase 2) |
275| `--archive-git <path>` | Git base directory for archive (enables Phase 2) | 292| `--archive-git <path>` | Git base directory for archive (enables Phase 2) |
276| `--service <name>` | Systemd service name (enables Phase 4) | 293| `--service <name>` | Systemd service name for Phase 4 log extraction. **MUST be an ngit-grasp service** (not ngit-relay). Structured logging only exists in ngit-grasp. |
277| `--output <dir>` | Output directory (default: auto-generated) | 294| `--output <dir>` | Output directory (default: auto-generated) |
278| `--skip-phase-N` | Skip phase N (1-5) | 295| `--skip-phase-N` | Skip phase N (1-5) |
279| `--only-phase-N` | Run only phase N | 296| `--only-phase-N` | Run only phase N |
@@ -427,16 +444,50 @@ The analysis will continue without log data.
427 444
428### Phase 4 finds no structured logs 445### Phase 4 finds no structured logs
429 446
430Structured logging (`[PARSE_FAIL]`, `[PURGATORY_EXPIRED]`) is only available in ngit-grasp. If checking an ngit-relay service, no structured logs will be found. 447**Symptom:** Phase 4 completes but `parse-failures.txt` and `purgatory-expired.txt` are empty or contain only header comments.
448
449**Most common cause:** You're querying the wrong service (ngit-relay instead of ngit-grasp).
450
451Structured logging (`[PARSE_FAIL]`, `[PURGATORY_EXPIRED]`) **only exists in ngit-grasp services**. If you specify an ngit-relay service, Phase 4 will find zero logs.
452
453**How to diagnose:**
431 454
432```bash 455```bash
433# Verify you're checking the right service (should be ngit-grasp) 456# 1. Check what service you configured
434journalctl -u ngit-grasp-*.service | grep -E '\[PARSE_FAIL\]|\[PURGATORY_EXPIRED\]' | head -5 457cat /path/to/output/config.txt | grep SERVICE_NAME
435 458
436# If checking ngit-relay, structured logs won't exist 459# 2. If it contains "ngit-relay", that's the problem!
437# Use --service with the ngit-grasp archive service name instead 460# ngit-relay does NOT have structured logging
461
462# 3. Find the correct ngit-grasp service
463systemctl list-units 'ngit-grasp*' --all
464
465# 4. Verify the ngit-grasp service has structured logs
466journalctl -u ngit-grasp-relay-ngit-dev.service --since "7 days ago" | \
467 grep -E '\[PARSE_FAIL\]|\[PURGATORY_EXPIRED\]' | head -5
438``` 468```
439 469
470**How to fix:**
471
472```bash
473# Update SERVICE_NAME to the ngit-grasp archive service and re-run
474./run-migration-analysis.sh \
475 --prod-relay wss://relay.ngit.dev \
476 --archive-relay ws://localhost:7443 \
477 --service ngit-grasp-relay-ngit-dev.service \
478 --from-phase-4 # Skip phases 1-3, just re-run phase 4
479```
480
481**Other possible causes:**
482
4831. **Structured logging not deployed:** If the ngit-grasp instance doesn't have the logging improvements deployed, no structured logs will exist. Check the ngit-grasp version.
484
4852. **No events in time window:** If there genuinely were no parse failures or purgatory expiry events, the files will be empty. This is valid - it means everything parsed successfully.
486
4873. **Wrong time range:** The default is 30 days. If your archive has been running longer, you may need `--since` to extend the range.
488
489**Prevention:** The migration scripts now validate the service name and will error if you specify an ngit-relay service.
490
440### Event counts are multiples of 250 491### Event counts are multiples of 250
441 492
442This suggests pagination may have failed. The scripts use `--paginate` by default, but if you see exactly 250, 500, 750 events, verify the relay is responding correctly. 493This suggests pagination may have failed. The scripts use `--paginate` by default, but if you see exactly 250, 500, 750 events, verify the relay is responding correctly.
@@ -645,7 +696,7 @@ This section documents the specific configuration and lessons learned from migra
645 696
6462. **Check archive accessibility**: We initially tried to run analysis remotely, but the archive relay was localhost-only. Had to SSH to VPS. 6972. **Check archive accessibility**: We initially tried to run analysis remotely, but the archive relay was localhost-only. Had to SSH to VPS.
647 698
6483. **Use archive service for Phase 4**: Structured logging (`[PARSE_FAIL]`, `[PURGATORY_EXPIRED]`) is in the ngit-grasp archive service, not the ngit-relay production service. 6993. **Use archive service for Phase 4 (CRITICAL)**: Structured logging (`[PARSE_FAIL]`, `[PURGATORY_EXPIRED]`) is **ONLY** in the ngit-grasp archive service, NOT the ngit-relay production service. Running Phase 4 against `ngit-relay.service` produces zero results because ngit-relay doesn't emit structured logs. The scripts now validate this and error if you specify an ngit-relay service.
649 700
6504. **Install git on VPS**: Git wasn't installed on the minimal VPS. The scripts now check for this in prerequisites. 7014. **Install git on VPS**: Git wasn't installed on the minimal VPS. The scripts now check for this in prerequisites.
651 702