upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/explanation/architecture.md
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2026-01-09 21:12:51 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2026-01-09 21:12:51 +0000
commit5fed2e2f32cfb15fff042a39f3ac82abe8948ca0 (patch)
tree9eeabc12bcadc43d18c772d9705dbf4b65d03ed2 /docs/explanation/architecture.md
parenta68e23733e78d33ca1d48b83414a8db63ca3d5fd (diff)
docs: integrate rejected events index into architecture documentation
- Add rejected events index to architecture.md with two-tier system explanation - Document NGIT_REJECTED_HOT_CACHE_DURATION_SECS and NGIT_REJECTED_COLD_INDEX_EXPIRY_SECS in configuration.md - Add comprehensive rejected events metrics section to monitoring.md with Grafana queries and alerts - Explain negentropy integration with rejected index in grasp-02-proactive-sync.md - Document state event authorization defense-in-depth and rejection tracking in inline-authorization.md This integrates information from work/rejected-events-index-summary.md into the main documentation, ensuring architecture docs accurately reflect the implemented rejected events index system.
Diffstat (limited to 'docs/explanation/architecture.md')
-rw-r--r--docs/explanation/architecture.md64
1 files changed, 64 insertions, 0 deletions
diff --git a/docs/explanation/architecture.md b/docs/explanation/architecture.md
index 6da2295..d2a9bf7 100644
--- a/docs/explanation/architecture.md
+++ b/docs/explanation/architecture.md
@@ -239,6 +239,22 @@ pub struct RepositoryAnnouncement { ... }
239pub struct RepositoryState { ... } 239pub struct RepositoryState { ... }
240``` 240```
241 241
242#### [`policy/state.rs`](src/nostr/policy/state.rs) - State Event Authorization
243
244State events undergo authorization checks at multiple points:
245
246```rust
247/// State event authorization checks:
248/// 1. Announcement must exist for the repository identifier
249/// 2. Author must be in maintainer set of accepted announcement
250/// 3. Validated on arrival, announcement acceptance, and git data arrival
251```
252
253**Defense-in-depth authorization:**
254- **On arrival** (StatePolicy): Initial authorization check
255- **On announcement acceptance**: Purgatory re-evaluation of waiting state events
256- **On git data arrival**: Final authorization before database save
257
242### 5. Purgatory System ([`src/purgatory/`](../../src/purgatory/)) 258### 5. Purgatory System ([`src/purgatory/`](../../src/purgatory/))
243 259
244The purgatory system solves the "which arrives first?" problem where either nostr events or git pushes can arrive in any order. It provides an in-memory holding area for events and git data awaiting their counterparts. 260The purgatory system solves the "which arrives first?" problem where either nostr events or git pushes can arrive in any order. It provides an in-memory holding area for events and git data awaiting their counterparts.
@@ -457,6 +473,7 @@ The ngit-grasp relay implements **Proactive Sync of Nostr Eevents**, which synch
457- **Health tracking** with exponential backoff for failing relays 473- **Health tracking** with exponential backoff for failing relays
458- **Daily sync** with random 23-25h timer to detect state drift 474- **Daily sync** with random 23-25h timer to detect state drift
459- **Filter consolidation** when count exceeds 70 to prevent subscription explosion 475- **Filter consolidation** when count exceeds 70 to prevent subscription explosion
476- **Rejected events index** - prevents wasteful re-fetching during negentropy sync
460 477
461**Architecture:** 478**Architecture:**
462 479
@@ -479,6 +496,14 @@ The ngit-grasp relay implements **Proactive Sync of Nostr Eevents**, which synch
479│ │ Health Tracker │ Exponential backoff, dead detection │ 496│ │ Health Tracker │ Exponential backoff, dead detection │
480│ │ (DashMap) │ │ 497│ │ (DashMap) │ │
481│ └──────────────────┘ │ 498│ └──────────────────┘ │
499│ ┌──────────────────────────────────────────────────────┐ │
500│ │ Rejected Events Index (Two-Tier) │ │
501│ │ ┌────────────────┐ ┌──────────────────────┐ │ │
502│ │ │ Hot Cache │───▶│ Cold Index │ │ │
503│ │ │ (2 min) │ │ (7 days) │ │ │
504│ │ │ Full events │ │ Metadata only │ │ │
505│ │ └────────────────┘ └──────────────────────┘ │ │
506│ └──────────────────────────────────────────────────────┘ │
482└─────────────────────────────────────────────────────────────┘ 507└─────────────────────────────────────────────────────────────┘
483``` 508```
484 509
@@ -486,6 +511,45 @@ The ngit-grasp relay implements **Proactive Sync of Nostr Eevents**, which synch
486 511
487For full design details, see [grasp-02-proactive-sync-v4.md](grasp-02-proactive-sync-v4.md). 512For full design details, see [grasp-02-proactive-sync-v4.md](grasp-02-proactive-sync-v4.md).
488 513
514### Rejected Events Index
515
516The rejected events index solves two critical problems during sync:
517
5181. **Negentropy sync efficiency**: Prevents repeatedly downloading events that will be rejected again
5192. **Race condition resolution**: Enables immediate re-processing when event dependencies are satisfied
520
521**Two-Tier Architecture:**
522
523| Tier | Duration | Storage | Purpose |
524|------|----------|---------|---------|
525| Hot Cache | 2 minutes | Full events | Immediate re-processing when dependencies arrive |
526| Cold Index | 7 days | Metadata only | Prevent re-fetch during negentropy sync |
527
528**Event Flow:**
529
530```
531Event Rejected (e.g., maintainer before owner announcement)
532
533 ├──▶ Store full event in Hot Cache (2 min expiry)
534 └──▶ Store metadata in Cold Index (7 day expiry)
535
536Dependency Arrives (e.g., owner announcement accepted)
537
538 ├──▶ Invalidate from Cold Index
539 ├──▶ Retrieve from Hot Cache (if still available)
540 └──▶ Re-process immediately (<1 second vs 24 hours)
541
542Negentropy Sync
543
544 └──▶ Exclude Cold Index IDs from "missing events" calculation
545```
546
547**Tracked Events:**
548- Repository announcements (kind 30617) rejected for not listing this service or maintainer validation failure
549- State events (kind 30618) rejected for missing announcements or unauthorized authors
550
551**Source Code:** [`src/sync/rejected_index.rs`](../../src/sync/rejected_index.rs)
552
489## Future Extensions 553## Future Extensions
490 554
491### GRASP-02: Proactive Sync 555### GRASP-02: Proactive Sync