npub1tkq8unhsd5jqx6ueex5lcpsgknrpquxuk44ftpjlpm3ulaake7xs76txrw/ngit-grasp-mirror/docs/how-to/production-sync-testing.md, branch masterUnnamed repository; edit this file 'description' to name the repository.
https://upleb.uk/npub1tkq8unhsd5jqx6ueex5lcpsgknrpquxuk44ftpjlpm3ulaake7xs76txrw/ngit-grasp-mirror/atom?h=master2026-02-03T15:18:23+00:00Merge relay.ngit.dev migration: bug fixes and migration tooling2026-02-03T15:18:23+00:00DanConwayDevDanConwayDev@protonmail.com2026-02-03T14:50:22+00:00urn:sha1:874a8abe1d076cfafd9baf919ec23d7d58200698
This merge includes critical bug fixes and comprehensive migration tooling
developed during the relay.ngit.dev migration effort.
Bug Fixes:
- Fix git protocol error handling to return HTTP 200 with ERR pkt-line
- Fix naughty list false positives and DNS failure identification
- Fix database query filters in load_existing_events (remove .since())
- Fix OID fetch tracking to distinguish 0 OIDs from successful fetches
- Fix purgatory event source tracking for filtered expiry logging
- Implement OID retry logic for 'not our ref' errors
Migration Tools & Documentation:
- Complete 5-phase migration analysis pipeline with orchestration script
- Phase 1: Event fetching from source relay
- Phase 2: Git sync verification
- Phase 3: Categorization and relay comparison
- Phase 4: Log extraction (parse failures, purgatory expiry)
- Phase 5: Action classification for migration decisions
- Comprehensive migration guide with lessons learned
- Troubleshooting guide for permission and corruption issues
Configuration:
- Add NGIT_LOG_LEVEL configuration option
- Update git throttle limits to 60/minute
- Improve logging throughout for better observability
docs: update production sync testing to require 60 seconds2026-01-10T03:24:45+00:00DanConwayDevDanConwayDev@protonmail.com2026-01-10T03:24:45+00:00urn:sha1:0bae1738ace1af196272a333b5d835a7e497861b
The sync system uses a 5-second batch window for discovered repos.
Repos discovered late in a 30-second test don't have enough time for
the full Layer 2→3→4 cascade:
- Layer 1: Discover repo announcements (0-5s)
- Layer 2: Send #a, #A, #q filters for repos (5-30s)
- Layer 3: Receive issues, patches, PRs (30-60s)
- Layer 4: Receive comments on root events (40-60s)
Testing confirmed that 60 seconds allows late-discovered repos
(gitworkshop, ngit) to complete all layers, while 30 seconds only
allows 1 second after Layer 2 filters are sent.
Updated all references from 30s to 60s throughout the guide and added
explanation of why this duration is necessary.
docs: save both raw and sanitized logs for flexible analysis2026-01-10T01:49:33+00:00DanConwayDevDanConwayDev@protonmail.com2026-01-10T01:49:33+00:00urn:sha1:730f430c906c6c2d43ea8f2e5fc3b408a3de128b
- Update production sync testing workflow to save both sync-raw.log and sync.log
- Raw log contains complete untruncated messages (rejection reasons, event data, etc.)
- Sanitized log remains for quick scanning and pattern recognition
- Add guidance on when to use each log and how to retrieve full details from raw log
- Resolves truncated rejection warning issue by making full details accessible
docs: add permission step to fix mode workflow2026-01-09T23:20:01+00:00DanConwayDevDanConwayDev@protonmail.com2026-01-09T23:20:01+00:00urn:sha1:5c68d55b59cc7e59f2a618926059ac3f405876d0
Mode 1 (Fix Existing Issues) now requires reviewing the proposed fix
and asking for user permission before implementing changes. This ensures
users have visibility and control over what code changes are made.
Changes:
- Added Step 3: Review Proposed Fix and Get Permission
- Renumbered subsequent steps (4-7)
- Updated both overview and detailed workflow sections
- Updated workflow diagram to show review/permission steps
docs: update production sync testing workflow to two-mode process2026-01-09T22:30:41+00:00DanConwayDevDanConwayDev@protonmail.com2026-01-09T22:23:18+00:00urn:sha1:f1579d1c099869de67b1741b7775cbf651b34ef0
- Mode 1: Fix one existing issue, test, commit, report
- Mode 2: Discover new issues with minimal documentation
- Emphasize stopping after each cycle
- Remove detailed investigation requirements
- Simplify issue documentation format
docs: track production sync issues in work/active-issues/2026-01-09T22:07:33+00:00DanConwayDevDanConwayDev@protonmail.com2026-01-09T22:07:33+00:00urn:sha1:ab43f7264a05f694f4a7b1f4a09f8add972381ab
- Update production-sync-testing.md to document issues as individual markdown files in work/active-issues/ instead of polluting the tracked how-to guide
- Add issue template and workflow for creating, viewing, and resolving issues
- Document active-issues/ purpose in work/README.md
- Prevents accidental commits of transient testing issues
- Makes issue management cleaner and more focused
Fix production-sync-testing.md guide and sanitize-logs.sh script2026-01-09T21:51:52+00:00DanConwayDevDanConwayDev@protonmail.com2026-01-09T21:51:52+00:00urn:sha1:5570b9631e03c077faa95fbf1f068a4360d25bd7
- Fix shebang in sanitize-logs.sh from #!/bin/bash to #!/usr/bin/env bash for NixOS compatibility
- Update sanitizer defaults from 100/20 to 200/100 chars for better log readability
- Fix CLI argument names in guide: --sync-bootstrap-relay -> --sync-bootstrap-relay-url
- Fix CLI argument names in guide: --git-path -> --git-data-path
These issues were discovered during first-time testing of the production sync testing guide.
docs: add production sync testing guide and log sanitizer script2026-01-09T21:41:45+00:00DanConwayDevDanConwayDev@protonmail.com2026-01-09T21:41:45+00:00urn:sha1:6a8346441ea605248e09858d0e6b9096a1c87259
Add infrastructure for iterative debugging of sync against production data:
- scripts/sanitize-logs.sh: Truncates verbose log lines for LLM analysis
- docs/how-to/production-sync-testing.md: Step-by-step guide for testing
sync against real relays, identifying issues, and improving logging