diff options
Diffstat (limited to 'docs/how-to/migrate-to-ngit-grasp.md')
| -rw-r--r-- | docs/how-to/migrate-to-ngit-grasp.md | 20 |
1 files changed, 13 insertions, 7 deletions
diff --git a/docs/how-to/migrate-to-ngit-grasp.md b/docs/how-to/migrate-to-ngit-grasp.md index 00af6c8..62cad87 100644 --- a/docs/how-to/migrate-to-ngit-grasp.md +++ b/docs/how-to/migrate-to-ngit-grasp.md | |||
| @@ -122,7 +122,7 @@ ls /path/to/git/npub1*/ # Should show *.git directories | |||
| 122 | 122 | ||
| 123 | ### Phase 4 Needs the Correct Service Name | 123 | ### Phase 4 Needs the Correct Service Name |
| 124 | 124 | ||
| 125 | > **CRITICAL:** Phase 4 extracts structured logs (`[PARSE_FAIL]`, `[PURGATORY_EXPIRED]`) from journald. These logs **ONLY exist in ngit-grasp services**, NOT in ngit-relay services. | 125 | > **CRITICAL:** Phase 4 extracts structured logs (`[PARSE_FAIL]`, `[PURGATORY_EXPIRED]`, `Invalid announcement` rejections) from journald. These logs **ONLY exist in ngit-grasp services**, NOT in ngit-relay services. |
| 126 | 126 | ||
| 127 | If you specify an ngit-relay service (like `ngit-relay.service`), Phase 4 will find **zero logs** and produce empty results. This is a common mistake that wastes time and produces misleading analysis. | 127 | If you specify an ngit-relay service (like `ngit-relay.service`), Phase 4 will find **zero logs** and produce empty results. This is a common mistake that wastes time and produces misleading analysis. |
| 128 | 128 | ||
| @@ -140,10 +140,10 @@ If you specify an ngit-relay service (like `ngit-relay.service`), Phase 4 will f | |||
| 140 | systemctl list-units 'ngit-*' --all | 140 | systemctl list-units 'ngit-*' --all |
| 141 | 141 | ||
| 142 | # Check which service has structured logging (should be ngit-grasp) | 142 | # Check which service has structured logging (should be ngit-grasp) |
| 143 | journalctl -u ngit-grasp-*.service | grep -E '\[PARSE_FAIL\]|\[PURGATORY_EXPIRED\]' | head -5 | 143 | journalctl -u ngit-grasp-*.service | grep -E '\[PARSE_FAIL\]|\[PURGATORY_EXPIRED\]|Invalid announcement' | head -5 |
| 144 | 144 | ||
| 145 | # Verify ngit-relay does NOT have structured logging | 145 | # Verify ngit-relay does NOT have structured logging |
| 146 | journalctl -u ngit-relay.service | grep -E '\[PARSE_FAIL\]|\[PURGATORY_EXPIRED\]' | head -5 | 146 | journalctl -u ngit-relay.service | grep -E '\[PARSE_FAIL\]|\[PURGATORY_EXPIRED\]|Invalid announcement' | head -5 |
| 147 | # ^ This should return nothing | 147 | # ^ This should return nothing |
| 148 | 148 | ||
| 149 | # Use the archive service name for Phase 4 | 149 | # Use the archive service name for Phase 4 |
| @@ -448,7 +448,7 @@ The analysis will continue without log data. | |||
| 448 | 448 | ||
| 449 | **Most common cause:** You're querying the wrong service (ngit-relay instead of ngit-grasp). | 449 | **Most common cause:** You're querying the wrong service (ngit-relay instead of ngit-grasp). |
| 450 | 450 | ||
| 451 | Structured logging (`[PARSE_FAIL]`, `[PURGATORY_EXPIRED]`) **only exists in ngit-grasp services**. If you specify an ngit-relay service, Phase 4 will find zero logs. | 451 | Structured logging (`[PARSE_FAIL]`, `[PURGATORY_EXPIRED]`, `Invalid announcement` rejections) **only exists in ngit-grasp services**. If you specify an ngit-relay service, Phase 4 will find zero logs. |
| 452 | 452 | ||
| 453 | **How to diagnose:** | 453 | **How to diagnose:** |
| 454 | 454 | ||
| @@ -464,7 +464,7 @@ systemctl list-units 'ngit-grasp*' --all | |||
| 464 | 464 | ||
| 465 | # 4. Verify the ngit-grasp service has structured logs | 465 | # 4. Verify the ngit-grasp service has structured logs |
| 466 | journalctl -u ngit-grasp-relay-ngit-dev.service --since "7 days ago" | \ | 466 | journalctl -u ngit-grasp-relay-ngit-dev.service --since "7 days ago" | \ |
| 467 | grep -E '\[PARSE_FAIL\]|\[PURGATORY_EXPIRED\]' | head -5 | 467 | grep -E '\[PARSE_FAIL\]|\[PURGATORY_EXPIRED\]|Invalid announcement' | head -5 |
| 468 | ``` | 468 | ``` |
| 469 | 469 | ||
| 470 | **How to fix:** | 470 | **How to fix:** |
| @@ -482,12 +482,14 @@ journalctl -u ngit-grasp-relay-ngit-dev.service --since "7 days ago" | \ | |||
| 482 | 482 | ||
| 483 | 1. **Structured logging not deployed:** If the ngit-grasp instance doesn't have the logging improvements deployed, no structured logs will exist. Check the ngit-grasp version. | 483 | 1. **Structured logging not deployed:** If the ngit-grasp instance doesn't have the logging improvements deployed, no structured logs will exist. Check the ngit-grasp version. |
| 484 | 484 | ||
| 485 | 2. **No events in time window:** If there genuinely were no parse failures or purgatory expiry events, the files will be empty. This is valid - it means everything parsed successfully. | 485 | 2. **No events in time window:** If there genuinely were no parse failures, purgatory expiry events, or invalid announcement rejections, the files will be empty. This is valid - it means everything parsed successfully. |
| 486 | 486 | ||
| 487 | 3. **Wrong time range:** The default is 30 days. If your archive has been running longer, you may need `--since` to extend the range. | 487 | 3. **Wrong time range:** The default is 30 days. If your archive has been running longer, you may need `--since` to extend the range. |
| 488 | 488 | ||
| 489 | **Prevention:** The migration scripts now validate the service name and will error if you specify an ngit-relay service. | 489 | **Prevention:** The migration scripts now validate the service name and will error if you specify an ngit-relay service. |
| 490 | 490 | ||
| 491 | **Note on "Invalid announcement" rejections:** These are announcements (kind 30617) that were rejected by the write policy due to format violations. The most common reason is "multiple clone tags found" - the NIP-34 spec requires a single clone tag with multiple values, not multiple clone tags. These rejections are logged as `Event rejected by write policy ... reason=Invalid announcement: ...`. | ||
| 492 | |||
| 491 | ### Event counts are multiples of 250 | 493 | ### Event counts are multiples of 250 |
| 492 | 494 | ||
| 493 | This suggests pagination may have failed. The scripts use `--paginate` by default, but if you see exactly 250, 500, 750 events, verify the relay is responding correctly. | 495 | This suggests pagination may have failed. The scripts use `--paginate` by default, but if you see exactly 250, 500, 750 events, verify the relay is responding correctly. |
| @@ -529,7 +531,11 @@ The analysis is split into 5 modular phases: | |||
| 529 | ↓ | 531 | ↓ |
| 530 | ┌─────────────────────────────────────────────────────────────────┐ | 532 | ┌─────────────────────────────────────────────────────────────────┐ |
| 531 | │ PHASE 4: Log-Based Categories (VPS required) │ | 533 | │ PHASE 4: Log-Based Categories (VPS required) │ |
| 532 | │ Extracts [PARSE_FAIL] and [PURGATORY_EXPIRED] from logs │ | 534 | │ Extracts structured logs from the archive service: │ |
| 535 | │ - [PARSE_FAIL] - Events that failed to parse │ | ||
| 536 | │ - [PURGATORY_EXPIRED] - Repos where git data never arrived │ | ||
| 537 | │ - "Invalid announcement" - Announcements rejected for format │ | ||
| 538 | │ violations (e.g., multiple clone tags) │ | ||
| 533 | │ Provides context for why repos failed to sync │ | 539 | │ Provides context for why repos failed to sync │ |
| 534 | └─────────────────────────────────────────────────────────────────┘ | 540 | └─────────────────────────────────────────────────────────────────┘ |
| 535 | ↓ | 541 | ↓ |