| Age | Commit message (Collapse) | Author |
|---|
|
|
|
Add mandatory uploadpack.allowFilter capability to support partial clones
and fetches as required by GRASP-01 specification. This enables efficient
git operations for bandwidth-constrained clients (e.g., browser-based git
clients like git-natural-api).
Changes:
- Add uploadpack.allowFilter=true to git subprocess configuration
- Update SmartGitServer test helper with filter support
- Add integration tests for filter capability advertisement and functionality
- Update documentation to reflect filter as required capability
Tests verify:
- Filter capability is advertised in info/refs
- Filtered clones with blob:none work correctly
- Filtered fetches with tree:0 work correctly
|
|
- Add new how-to guide covering hash updates for git dependencies
- Applies to any git dependency (e.g., nostr-sdk fork)
- Add critical note in AGENTS.md linking to this guide
- Emphasize that hash updates in both flake.nix and nix/module.nix are MANDATORY
|
|
- Complete guide for deploying ngit-grasp to NixOS servers
- Step-by-step deployment instructions
- Configuration options reference
- Troubleshooting section
- Security hardening recommendations
- Multiple instance examples
- References nix/example-configuration.nix which has clear examples
|
|
- Correct git protocol: ngit-grasp implements HTTP layer, not full git implementation
- Correct nostr relay: both use libraries (Khatru vs nostr-relay-builder)
- Highlight key difference: ngit-relay has NO nostr event sync (only git sync)
- Explain code size difference: mainly due to event sync (~5k lines) that ngit-relay lacks
- Update when-to-choose: ngit-grasp required for event discovery from relay network
|
|
The sync system uses a 5-second batch window for discovered repos.
Repos discovered late in a 30-second test don't have enough time for
the full Layer 2→3→4 cascade:
- Layer 1: Discover repo announcements (0-5s)
- Layer 2: Send #a, #A, #q filters for repos (5-30s)
- Layer 3: Receive issues, patches, PRs (30-60s)
- Layer 4: Receive comments on root events (40-60s)
Testing confirmed that 60 seconds allows late-discovered repos
(gitworkshop, ngit) to complete all layers, while 30 seconds only
allows 1 second after Layer 2 filters are sent.
Updated all references from 30s to 60s throughout the guide and added
explanation of why this duration is necessary.
|
|
- Update production sync testing workflow to save both sync-raw.log and sync.log
- Raw log contains complete untruncated messages (rejection reasons, event data, etc.)
- Sanitized log remains for quick scanning and pattern recognition
- Add guidance on when to use each log and how to retrieve full details from raw log
- Resolves truncated rejection warning issue by making full details accessible
|
|
Mode 1 (Fix Existing Issues) now requires reviewing the proposed fix
and asking for user permission before implementing changes. This ensures
users have visibility and control over what code changes are made.
Changes:
- Added Step 3: Review Proposed Fix and Get Permission
- Renumbered subsequent steps (4-7)
- Updated both overview and detailed workflow sections
- Updated workflow diagram to show review/permission steps
|
|
- Mode 1: Fix one existing issue, test, commit, report
- Mode 2: Discover new issues with minimal documentation
- Emphasize stopping after each cycle
- Remove detailed investigation requirements
- Simplify issue documentation format
|
|
- Update production-sync-testing.md to document issues as individual markdown files in work/active-issues/ instead of polluting the tracked how-to guide
- Add issue template and workflow for creating, viewing, and resolving issues
- Document active-issues/ purpose in work/README.md
- Prevents accidental commits of transient testing issues
- Makes issue management cleaner and more focused
|
|
- Fix shebang in sanitize-logs.sh from #!/bin/bash to #!/usr/bin/env bash for NixOS compatibility
- Update sanitizer defaults from 100/20 to 200/100 chars for better log readability
- Fix CLI argument names in guide: --sync-bootstrap-relay -> --sync-bootstrap-relay-url
- Fix CLI argument names in guide: --git-path -> --git-data-path
These issues were discovered during first-time testing of the production sync testing guide.
|
|
Add infrastructure for iterative debugging of sync against production data:
- scripts/sanitize-logs.sh: Truncates verbose log lines for LLM analysis
- docs/how-to/production-sync-testing.md: Step-by-step guide for testing
sync against real relays, identifying issues, and improving logging
|
|
- Add rejected events index to architecture.md with two-tier system explanation
- Document NGIT_REJECTED_HOT_CACHE_DURATION_SECS and NGIT_REJECTED_COLD_INDEX_EXPIRY_SECS in configuration.md
- Add comprehensive rejected events metrics section to monitoring.md with Grafana queries and alerts
- Explain negentropy integration with rejected index in grasp-02-proactive-sync.md
- Document state event authorization defense-in-depth and rejection tracking in inline-authorization.md
This integrates information from work/rejected-events-index-summary.md into the main documentation,
ensuring architecture docs accurately reflect the implemented rejected events index system.
|
|
Resolves naming conflict with RelayHealthState::Degraded by using a more
explicit name that clearly indicates the connection status relates to
historic sync failures, not connection health degradation.
Changes:
- ConnectionStatus::ConnectedDegraded → ConnectedHistoricSyncFailures
- Updated all documentation and comments
- Updated Prometheus metric descriptions
- Metric value remains 4 for backward compatibility
This makes it clear that:
- ConnectedHistoricSyncFailures = connection lifecycle (missing historic data)
- RelayHealthState::Degraded = connection health (reliability issues)
These are orthogonal concerns - a relay can be ConnectedHistoricSyncFailures
but Healthy, or Connected but Degraded.
|
|
- Add ConnectionStatus::ConnectedDegraded (status=4 in metrics)
- Track batch failures via PendingBatch.failed field
- Track relay-level failures via RelayState.historic_sync_had_failures
- Transition to ConnectedDegraded when any batch fails during historic sync
- Add is_live_sync_active() helper for cleaner match patterns
- Update state machine diagram with ConnectedDegraded transitions
- Update metrics docs with status=4 and example queries
Fixes issue where relays with failed negentropy retries would
incorrectly transition to Connected status despite missing data.
Now operators can distinguish 'fully synced' vs 'degraded (partial data)'.
|
|
- Add ConnectionStatus::Syncing state between Connecting and Connected
- Track historic_sync_completed and historic_sync_completed_at in RelayState
- Auto-detect sync completion via check_and_complete_historic_sync()
- Update metrics: ngit_sync_relay_connected now shows 0-3 (disconnected/connecting/syncing/connected)
- Update Prometheus metric documentation with new status values
- Add state machine diagram showing Syncing transition
- Operators can now distinguish 'connected but catching up' vs 'fully synced'
|
|
|
|
|
|
|
|
process_newly_available_git_data
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Replace EOSE-based sync completion with negentropy reconciliation for:
- Initial connect (fresh sync)
- Daily sync (Layer 1 announcements)
- Stale reconnect (>15 min)
Key changes:
- Add NegentropySyncResult struct with remote_only, local_only, received fields
- Add supports_negentropy() using try-and-fallback approach
- Add negentropy_sync_filter() using nostr-sdk client.sync() API
- Modify handle_connect_or_reconnect() to use negentropy for fresh/stale sync
- Modify daily_sync() to use negentropy for Layer 1
- Single-warning logging per relay when negentropy fails
Quick reconnects (<15 min) unchanged - still use REQ with since filter.
If negentropy unsupported, gracefully falls back to REQ+EOSE flow.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Add SyncMetrics with full Prometheus integration
- Track sync gaps via catchup events
- Update Grafana dashboard with sync panels
- Document all sync configuration options
- Update design doc with implementation notes
|
|
|
|
- Added CRITICAL warning section to AGENTS.md about treating architecture
docs as living documents
- Mark 'Keep Architecture Docs Updated' item as fixed in grasp-01 learnings
- Mark 'Document actual architecture' technical debt item as fixed
This addresses a key learning from GRASP-01 where docs described plans
rather than implementation, causing confusion.
|
|
Split the ~900 line Nip34WritePolicy into focused sub-policies for improved
testability and maintainability:
- AnnouncementPolicy - Repository announcement validation
- StatePolicy - State event validation + ref alignment
- PrEventPolicy - PR/PR Update validation
- RelatedEventPolicy - Forward/backward reference checking
The main Nip34WritePolicy now delegates to these sub-policies via a shared
PolicyContext that provides domain, database, and git_data_path.
Also updates:
- README.md: Accurate project structure reflecting actual implementation
- docs/learnings: Marks this technical debt item as complete
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Add environment variable configuration for database backend selection:
- Added DatabaseBackend enum (memory, nostrdb, lmdb) in src/config.rs
- Updated relay builder to use configured backend in src/nostr/builder.rs
- Added NGIT_DATABASE_BACKEND to .env.example with documentation
- Updated docs/reference/configuration.md with backend comparison table
NostrDB and LMDB backends prepared for future implementation when
nostr-relay-builder adds support. Currently defaults to in-memory
database with warning logs when persistent backends are selected.
|
|
|
|
|
|
Archive valuable session documentation from test migration project:
- Phase 1: NIP-01 compliance test migration
- Phase 2: NIP-34 announcement test migration
- Phase 3: Test compliance documentation
- Final summary: Complete project overview
Session cleanup complete - work/ directory now clean (only README.md)
|
|
- Remove unnecessary 'nix' dev dependency (Unix syscalls crate, not needed)
- Migrate announcement tests to new TestRelay fixture pattern
- Delete legacy test files (announcement_tests.rs, test_relay.sh)
- Add comprehensive test documentation (docs/how-to/test-compliance.md)
- Update README.md with new test commands
- All 18 integration tests passing (NIP-01 + NIP-34)
Benefits:
- Automatic relay lifecycle management
- No manual setup required
- Pure Rust integration tests
- Better developer experience
- CI/CD ready
|
|
|
|
- Archive 5 .txt files to docs/archive/
- AUDIT_FIX_SUMMARY.txt
- PROJECT_STATUS_VISUAL.txt
- SESSION_SUMMARY.txt
- TEST_VISUAL_SUMMARY.txt
- CLEANUP_VISUAL_SUMMARY.txt
- Update AGENTS.md with file format guidelines:
- When to use .txt (ASCII art only)
- When to use .md (all documentation)
- .txt lifecycle: create → use → archive immediately
- Added to cleanup triggers and checklists
Root directory now completely clean:
- 4 .md files (README, AGENTS, CURRENT_STATUS, CLEANUP_COMPLETE)
- 0 .txt files (all archived)
Archive contains:
- 33 .md files (historical documentation)
- 5 .txt files (visual summaries)
|
|
|
|
- Archive 30 completed session documents to docs/archive/
- Extract learnings to docs/learnings/ (nix-flakes, nostr-sdk, grasp-audit)
- Create CURRENT_STATUS.md as single source of truth
- Create AGENTS.md with documentation guidelines
- Create docs/archive/README.md for archive organization
- Clean root directory: 32 files → 4 files
Root directory now contains only:
- README.md (project overview)
- AGENTS.md (documentation guidelines)
- CURRENT_STATUS.md (current state)
- CLEANUP_SUMMARY.md (cleanup report)
All historical documents preserved in docs/archive/ with proper dating.
All reusable knowledge extracted to docs/learnings/.
Benefits:
- Easy to find current information
- Clear document lifecycle
- No more documentation sprawl
- Learnings are accessible and reusable
- Better onboarding for new developers/agents
File counts:
- Root: 4 (was 32)
- Permanent docs: 7
- Learnings: 3 (new)
- Archive: 32 (new)
- Total: 49 well-organized docs
|
|
ok 2 prompts, the second one was about the test strategy so we could
reuse it. I was thinking of a tool like blossom audit. but i didnt
mention it specifically.
|