| Age | Commit message (Collapse) | Author |
|---|
|
NIP-01 places no restriction on d tag characters and NIP-34 only
recommends kebab-case without mandating it. Rejecting identifiers with
whitespace or other URL-unsafe characters was therefore overly strict.
The correct approach (per NIP-34 PR #2312 and GRASP-01) is to store
identifiers verbatim on disk and percent-encode them when constructing
URLs. The previous commit already handled the incoming direction
(percent-decoding URL paths before filesystem lookup); this commit
handles the outgoing direction and removes the validation restriction.
Changes:
- validate_identifier: drop whitespace rejection; only reject chars
that are unsafe as filesystem directory names (/, \, null, . / ..)
- git/mod.rs: add percent_encode() alongside percent_decode()
- landing.rs: percent-encode identifier in nostr:// clone URL and
gitworkshop link (also fixes a pre-existing bug where the clone URL
displayed literal '{npub}' / '{identifier}' instead of the values)
|
|
Two bugs allowed a repository announcement with a space-containing
identifier ('kuboslopp by Shakespeare') to enter purgatory and create
a bare repo on disk, but then fail to serve git data over HTTP.
Bug 1 (serving): parse_git_url and parse_repo_url did not percent-decode
the URL path before resolving the filesystem path. A client requesting
/npub.../kuboslopp%20by%20Shakespeare.git/info/refs had the identifier
extracted as 'kuboslopp%20by%20Shakespeare' (literal %20), which did not
match the on-disk directory 'kuboslopp by Shakespeare.git'.
Fix: add percent_decode() in src/git/mod.rs and apply it to the repo
component in both parse_git_url and parse_repo_url.
Bug 2 (validation): validate_announcement did not check that the
identifier is safe as a filesystem path component and URL segment.
Identifiers containing whitespace, path separators, null bytes, or
reserved names (. / ..) should be rejected at acceptance time.
Fix: add validate_identifier() in src/nostr/events.rs and call it from
validate_announcement before any other policy checks.
|
|
Fix pre-existing clippy lints:
- &PathBuf -> &Path in audit_cleanup.rs
- too_many_arguments on process_newly_available_git_data,
process_purgatory_announcements, and HttpService::new
- clone_on_copy for PublicKey (Copy type) in purgatory cleanup loop
|
|
Previously push auth failures returned HTTP 403 which git clients
display as a generic transport error. Now they return HTTP 200 with
an ERR pkt-line containing the rejection reason (e.g. 'authorisation
failed: No state events in purgatory'), which git displays directly.
Remove GitError::Unauthorized as it is no longer used. GitError
variants now represent only transport/infrastructure failures; app-level
rejections use ERR pkt-line responses.
|
|
collect_all_authorized_maintainers
Both were pub functions with no callers. Clippy doesn't flag dead pub
items because the compiler treats them as potentially used by external
crates - only private items trigger the dead_code lint.
|
|
fetch_repository_data_{excluding,with}_purgatory
The old name was ambiguous - it wasn't clear whether purgatory was
included or not. The two variants are now explicitly named:
- fetch_repository_data_excluding_purgatory: DB only
- fetch_repository_data_with_purgatory: DB + purgatory overlay
SyncContext trait method also renamed to fetch_repository_data_with_purgatory
to match the free function it delegates to.
|
|
|
|
state event copy
When git data is fetched into owner A's repo and a state event for owner B
is released from purgatory (copying OIDs from A's repo to B's repo via
process_state_with_git_data), owner B's purgatory announcement was never
promoted. process_purgatory_announcements only promotes the announcement
for the owner derived from source_repo_path (owner A), so owner B's
announcement stayed in purgatory with its 30-minute expiry timer running.
30 minutes later the cleanup task would soft-expire owner B's entry,
deleting the bare repository even though the announcement had been
effectively satisfied.
Fix: after a state event is successfully saved to the database, iterate
over all announcements in db_repo_data and promote any purgatory
announcement for owners whose repos received OIDs via the copy (i.e.
repos other than source_repo_path).
|
|
|
|
Per design doc decision #4: when git auth finds a matching state event
in purgatory that authorizes a push, extend the announcement's expiry.
The repo is actively receiving git data so the announcement should not
expire prematurely. Also triggers revival of soft-expired announcements.
|
|
When an owner announcement is promoted from purgatory via a git push,
any maintainer announcements sitting in the rejected_events_index hot
cache were never re-processed. The invalidate_and_get call only existed
in SyncManager::process_event_static (the nostr sync path); the git push
promotion path (http -> handlers -> git::sync) had no access to the
rejected_events_index at all.
Thread rejected_events_index and write_policy through the git push path:
- process_purgatory_announcements: after saving the promoted announcement,
parse its maintainers tag and call invalidate_and_get() for each, then
re-process any returned hot-cache events via admit_event + save
- process_newly_available_git_data: accept optional write_policy and
rejected_events_index, pass them through to process_purgatory_announcements
- handle_receive_pack: accept Arc<Nip34WritePolicy> and
Arc<RejectedEventsIndex>, pass them to process_newly_available_git_data
- HttpService / run_server: carry the two new fields, clone into each
handle_receive_pack call
- main.rs: obtain rejected_events_index from sync_manager before moving
it into its task; wrap write_policy in Arc for the HTTP server
- RealSyncContext::process_newly_available_git_data: pass None for both
new params (purgatory sync path already handles this via
SyncManager::process_event_static)
Also rewrite the maintainer_reprocessing integration tests to correctly
exercise the hot-cache path now that announcements require git data
before being released from purgatory:
- Start relay_b with relay_a as bootstrap so its SyncManager syncs
maintainer announcements via negentropy before the owner git push
- Use push_unique_git_data_to_relay (new helper) to give each maintainer
a distinct commit hash, preventing git from skipping pack transfer
- Make wait_for_event_on_relay poll in a retry loop so transient timing
gaps between DB write and query do not cause false negatives
|
|
after announcement promotion"
This reverts commit d76003b629a4a03dba23a8a1c41da6e4ac4c30cf.
|
|
announcement promotion
When git data arrives for a purgatory announcement and promotes it to the
database, the relay now:
1. Upgrades the announcement's sync level in RepoSyncIndex from StateOnly
to Full (git/sync.rs: process_purgatory_announcements)
2. Sends AddFilters actions to SyncManager for all connected relays, using
Full sync filters (Layer 2 #a/#A/#q) to subscribe to PR events
(purgatory/sync/context.rs: RealSyncContext.process_newly_available_git_data)
3. For user-submitted purgatory announcements, registers the repo in
RepoSyncIndex with StateOnly level and sends AddFilters to SyncManager
so it discovers and connects to relays listed in the announcement tags
(nostr/builder.rs: handle_announcement AcceptPurgatory path)
The RealSyncContext now accepts optional repo_sync_index and sync_action_tx
parameters. main.rs wires these up from SyncManager. PolicyContext gains
repo_sync_index and sync_action_tx fields for the write policy path.
|
|
Add comments explaining that PR event processing (both incoming and
purgatory) should only use database announcements, not purgatory ones.
This is intentional because:
- Incoming PR events should only be accepted for validated announcements
- Purgatory PR events should only be released when announcement is promoted
- This prevents accepting PR events for announcements that fail validation
Differs from state event processing which uses fetch_repository_data_with_purgatory
because state events check authorization without releasing from purgatory.
|
|
When processing state events from purgatory, we need to check
authorization against announcements that may still be in purgatory
(not yet promoted to the database).
Previously, process_purgatory_state_events() used fetch_repository_data()
which only queries the database. This caused authorization failures when:
1. Git data arrives
2. Announcement is promoted from purgatory to database
3. State events are processed from purgatory
4. But db_repo_data was fetched BEFORE the announcement promotion
Now uses fetch_repository_data_with_purgatory() to include both
database and purgatory announcements, ensuring authorization works
correctly regardless of promotion timing.
|
|
Route new announcements to purgatory instead of accepting immediately.
Announcements are promoted to the database when git data arrives,
ensuring we only serve announcements for repos with actual content.
Implemented:
- AnnouncementPurgatoryEntry type and DashMap store
- Route new announcements to purgatory (replacement announcements skip)
- Promote announcements on git data arrival (process_purgatory_announcements)
- Authorization checks purgatory announcements (fetch_repository_data_with_purgatory)
- State policy uses purgatory announcements for maintainer validation
- Cleanup task handles announcement expiry
- Updated count()/cleanup() to 3-tuples
Known broken:
- test_archive_read_only_creates_bare_repo fails: sync module does not
treat purgatory announcements as confirmed repos, so per-repo sync
(state events, PRs) is never triggered for purgatory announcements
- Announcement persistence (save/restore) not implemented
- SyncLevel (StateOnly vs Full) not implemented
- Soft expiry two-phase not implemented
- Expiry extension on state event / git auth not wired up
|
|
Previously, some IO errors in git handlers were logged while others were
not, leading to inconsistent observability. Additionally, the HTTP layer
logged all git errors redundantly, adding no useful context beyond what
was already logged at the source.
Changes:
- Add error logging to all previously unlogged IO operations in
handle_upload_pack and handle_receive_pack (stdin writes, stdout/stderr
reads, process waits)
- Remove redundant error logging at HTTP layer since all errors are now
logged at their source with full context
- Ensures consistent error-level logging for all git subprocess failures
This provides complete observability of git operations while eliminating
duplicate log entries that don't add value.
|
|
Only the final summary 'Aligned repository with state' remains at INFO level,
showing the total count of refs_created/refs_updated/refs_deleted.
|
|
Improves observability when pushes are rejected due to state events that
only partially match the pushed refs. Previously, logs only showed 'No
state event found' even when state events existed but didn't match.
Changes:
- Add diagnose_state_mismatch() to explain why state events don't match
- Log specific reasons: missing refs, wrong SHAs, or extra refs
- Update rejection message to 'No matching state event found' (more accurate)
- Add 4 unit tests for diagnostic function
Example diagnostic output:
WARN State event abc123 from authorized author doesn't match push:
refs/heads/main missing (state declares 9cc3d93b)
This addresses the issue where a push with only refs/heads/test was
rejected because the state event also declared refs/heads/main, but
logs didn't explain why the match failed.
|
|
Fixes race condition where user's push becomes no-op after state event
is applied between fetch and push. Now accepts these as successful
no-ops, matching Git's 'Everything up-to-date' behavior.
- Add early detection in get_state_authorization_for_specific_owner_repo
- Return success for all-noop pushes without requiring purgatory event
- Document behavior in inline-authorization.md
|
|
Previously, all git upload-pack/receive-pack failures returned HTTP 500,
but the git smart HTTP protocol requires protocol-level errors (like
"not our ref") to be returned as HTTP 200 OK with an ERR pkt-line in
the response body.
Changes:
- Add build_git_protocol_error_response() to create HTTP 200 responses
with properly formatted ERR pkt-line ("ERR <message>\n")
- Add is_git_protocol_error() to detect protocol errors (exit code 128
with stderr content) vs transport errors
- Update handle_upload_pack() and handle_receive_pack() to return
protocol errors as HTTP 200 with ERR pkt-line
- Keep HTTP 500 for actual transport errors (spawn failures, I/O errors,
signals)
This allows git clients to properly parse and display protocol error
messages instead of seeing generic HTTP 500 errors.
|
|
Add mandatory uploadpack.allowFilter capability to support partial clones
and fetches as required by GRASP-01 specification. This enables efficient
git operations for bandwidth-constrained clients (e.g., browser-based git
clients like git-natural-api).
Changes:
- Add uploadpack.allowFilter=true to git subprocess configuration
- Update SmartGitServer test helper with filter support
- Add integration tests for filter capability advertisement and functionality
- Update documentation to reflect filter as required capability
Tests verify:
- Filter capability is advertised in info/refs
- Filtered clones with blob:none work correctly
- Filtered fetches with tree:0 work correctly
|
|
Add comprehensive authorization checks to ensure state events are only
accepted from maintainers of accepted repository announcements. This
implements the core GRASP-01 requirement that pushes must match the
latest state announcement "respecting the maintainer set."
Changes:
1. StatePolicy authorization (src/nostr/policy/state.rs):
- Check authorization BEFORE git data validation (fail-fast)
- Reject if no announcement exists for repository
- Reject if author not in maintainer set
- Use existing helpers: fetch_repository_data() and
pubkey_authorised_for_repo_owners()
- Structured logging for all rejections
2. Purgatory invalidation (src/nostr/builder.rs):
- New method: check_purgatory_state_events_for_identifier()
- Called when announcements accepted (Accept and AcceptMaintainer)
- Re-evaluates state events in purgatory for the identifier
- Processes newly-authorized events (releases from purgatory)
- Keeps unauthorized events for natural expiry (30 min)
- Enables retroactive authorization when announcements arrive late
3. Purgatory sync authorization (src/git/sync.rs):
- Check authorization BEFORE processing git data
- Remove unauthorized events from purgatory (permanent rejection)
- Prevents processing even if git data arrives first
- Structured logging for monitoring
4. Rejected events tracking (src/sync/rejected_index.rs):
- Add support for tracking rejected state events
- New methods: add_state(), contains_state()
- Separate metrics for state rejections
- Enables sync to avoid re-fetching rejected states
5. Sync metrics (src/sync/metrics.rs, src/sync/mod.rs):
- Add state-specific metrics (hot cache, cold index)
- Track rejected states separately from announcements
- Support monitoring of authorization rejections
6. Comprehensive tests (tests/state_authorization.rs):
- test_reject_state_without_announcement
- test_reject_state_from_unauthorized_author
- test_accept_state_from_announcement_author
- test_accept_state_from_maintainer
Security Impact:
- Before: State events could be published by anyone
- After: Only maintainers can publish state events
- Defense-in-depth: Authorization checked at 3 points:
1. On arrival (StatePolicy)
2. On announcement acceptance (purgatory re-evaluation)
3. On git data arrival (purgatory sync)
All tests pass:
- 248 unit tests
- 51 NIP-34 announcement tests
- 4 new state authorization tests
- 9 rejected index tests
Closes: State authorization requirement from GRASP-01 spec
|
|
- Replace KIND_REPOSITORY_ANNOUNCEMENT with Kind::GitRepoAnnouncement
- Replace KIND_REPOSITORY_STATE with Kind::RepoState
- Replace KIND_PR with Kind::GitPullRequest
- Replace KIND_PR_UPDATE with Kind::GitPullRequestUpdate
- Replace KIND_USER_GRASP_LIST with Kind::GitUserGraspList
- Replace KIND_PATCH with Kind::GitPatch
- Replace KIND_ISSUE with Kind::GitIssue
- Replace KIND_COMMENT with Kind::Comment
- Replace all Kind::Custom(30617|30618|1617|1618|1619|1621|1111|10317) patterns
- Remove all hardcoded KIND_* constants from events.rs
- Update all match statements to use Kind enum directly
- Update all filter builders to use Kind variants
- Update all test helpers and assertions
Benefits:
- Type safety: compiler prevents wrong kind numbers
- Readability: Kind::GitRepoAnnouncement is self-documenting
- Maintainability: single source of truth (rust-nostr)
- IDE support: full autocompletion and refactoring
- Standards: aligns with rust-nostr best practices
Files modified: 21
Constants removed: 9
Patterns replaced: 100+
Tests passing: 222/222
|
|
- Update nostr-relay-builder, nostr-sdk, nostr-lmdb to latest revision
- Update grasp-audit nostr-sdk dependency
- Fix clippy warnings:
- Replace .clone() with std::slice::from_ref() in src/git/sync.rs
- Change &PathBuf to &Path in tests/common/git_server.rs
- Replace vec![] with array literal in src/purgatory/sync/functions.rs
- Update PR_TEST_COMMIT_HASH in grasp-audit due to event generation changes
All 249 tests passing, no breaking changes required.
|
|
|
|
Prevent GPG signing prompts (including Yubikey activation) during test runs
by explicitly disabling commit.gpgsign and tag.gpgsign in all test repository
creation helpers.
Modified:
- tests/common/purgatory_helpers.rs: create_test_repo_with_commit()
- src/git/mod.rs: create_test_repo_with_commit()
- src/purgatory/helpers.rs: create_test_repo_with_commit()
All test repositories now have GPG signing disabled regardless of global
git configuration.
|
|
Eliminates code duplication by extracting core event processing into
reusable functions. All state and PR event processing now uses the same
unified logic from src/git/process.rs.
Changes:
- Add src/git/process.rs with unified processing functions
- process_state_with_git_data() for state events
- process_pr_with_git_data() for PR events
- Pure functions with comprehensive result types
- Refactor policy handlers to use unified processing
- src/nostr/policy/state.rs: Remove ~70 lines of duplicated logic
- src/nostr/policy/pr_event.rs: Remove ~40 lines of duplicated logic
- Refactor purgatory processing to use unified functions
- src/git/sync.rs: Remove ~125 lines of duplicated logic
- Make extract_owner_from_repo_path() public for reuse
Benefits:
- DRY: Single source of truth for event processing
- Testable: Pure functions with clear contracts
- Maintainable: Changes happen in one place
- Consistent: All code paths use same logic
All 217 unit tests + 40 integration tests pass (257/257).
|
|
we forgot to add the placeholder entry
|
|
Modern git clients (2.51.0+) default to protocol v2 and send the
Git-Protocol header. The server must pass this to git processes via
the GIT_PROTOCOL environment variable for proper negotiation.
Changes:
- Extract Git-Protocol header in HTTP layer (src/http/mod.rs)
- Pass git_protocol parameter through all handler functions
- Set GIT_PROTOCOL env var when spawning git subprocesses
- Update all tests to pass None for backward compatibility
This fixes hangs/timeouts when modern git clients connect to the server.
Fixes issue discovered in work/2025-01-07-pr-clone-tag-sync-investigation.md
|
|
- Prefix unused variable auth_result with underscore
- Prefix unused field git_data_path with underscore in Purgatory struct
- Add #[allow(clippy::too_many_arguments)] to handle_receive_pack
- Replace len() >= 1 with !is_empty()
- Replace .last() with .next_back() on DoubleEndedIterator
- Fix doc list item overindentation
- Replace map_or(true, ...) with is_none_or(...)
- Replace map_or(false, ...) with is_some_and(...)
|
|
this is now handled through process_newly_available_git_data
|
|
|
|
|
|
Replace ~100 lines of duplicated post-push processing in handle_receive_pack
with a single call to the unified process_newly_available_git_data function.
The unified function handles all post-git-data-available processing:
- Discovering satisfiable events from purgatory (state and PR events)
- Syncing OIDs to authorized owner repos
- Aligning refs (+ setting HEAD) in all owner repos
- Saving events to database
- Notifying WebSocket subscribers
- Removing from purgatory
This ensures consistent behavior regardless of how git data arrives
(git push vs purgatory sync fetching from remote servers).
Also mark test-only internal methods with #[cfg(test)] to silence
dead code warnings.
|
|
Implement the unified function that handles all post-git-data-available
processing, regardless of how data arrived (git push or purgatory sync).
This function:
- Discovers satisfiable events from purgatory (state and PR events)
- Syncs OIDs to authorized owner repos
- Aligns refs and sets HEAD
- Saves events to database
- Notifies WebSocket subscribers
- Removes from purgatory
New additions:
- ProcessResult struct for tracking processing outcomes
- process_newly_available_git_data async function in src/git/sync.rs
- Helper functions: extract_identifier_from_repo_path, extract_identifier_from_pr_event
- Purgatory::find_prs_for_identifier method for PR event discovery
- Unit tests for all helper functions
Also fixes:
- Simplified extract_domain to avoid url crate dependency
- Removed unused imports in sync/loop.rs
|
|
|
|
When a push to refs/nostr/<event-id> is received (PR data), the git data
is now synced to all other owner repositories that share maintainers with
the source owner. This mirrors the behavior added for state event data.
Changes:
- Add sync_pr_refs_to_owner_repos() function in git/sync.rs
- Add PrSyncResult struct to track sync statistics
- Add copy_single_commit_between_repos() helper function
- Call PR sync in handle_receive_pack after successful push
- Add unit test for PrSyncResult default values
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Add nostr-lmdb dependency (v0.44) for persistent storage
- Create SharedDatabase type alias for database abstraction
- Update all database-related functions to use trait object
- Support runtime selection via NGIT_DATABASE_BACKEND env var
Database backends:
- memory: In-memory (default, fastest, no persistence)
- lmdb: LMDB backend (persistent, general purpose)
All 34 tests pass with the new implementation.
|
|
if we have the OIDs
|
|
|
|
incorrect ref on event receive
|
|
|
|
|
|
|
|
allow-tip-sha1-in-want
|
|
|
|
Updated get_maintainers_recursive() to properly handle maintainers listed
in accepted repository announcements:
1. Separated 'visited' set (cycle prevention) from 'maintainers' set (result)
2. Maintainers listed in an announcement's 'maintainers' tag are now added
to the maintainer set immediately, even without their own announcement
3. Recursively traverse maintainer chains to handle multi-level delegation
Also fixed RecursiveMaintainerRepoAndState fixture to publish the
maintainer's announcement (which lists the recursive maintainer) before
publishing the recursive maintainer's announcement, establishing the
proper trust chain: Owner -> Maintainer -> RecursiveMaintainer
Test results: 7/7 push authorization tests passing
|
|
|
|
|
|
but do we really nedd to create a blank commit?
I dont think ngit-relay does that.
Do we need to se the default branch or is this automatic?
|
|
|