upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/explanation/comparison.md
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2026-02-25 15:14:46 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2026-02-25 15:14:46 +0000
commit8cd232727ae31613abba7a3d0485a1cb94fda2f3 (patch)
tree7b761929d919867fee3ffc68df436c250b9c5336 /docs/explanation/comparison.md
parent5ad9d9093fcbe7037e5474a9d8fa20a0b64fb79a (diff)
docs: remove comparison doc and update architecture to reflect announcement purgatory
Diffstat (limited to 'docs/explanation/comparison.md')
-rw-r--r--docs/explanation/comparison.md379
1 files changed, 0 insertions, 379 deletions
diff --git a/docs/explanation/comparison.md b/docs/explanation/comparison.md
deleted file mode 100644
index 315f091..0000000
--- a/docs/explanation/comparison.md
+++ /dev/null
@@ -1,379 +0,0 @@
1# ngit-grasp vs ngit-relay Comparison
2
3This document compares ngit-grasp (this project) with ngit-relay (the reference implementation) based on their actual implementations.
4
5## High-Level Overview
6
7| Aspect | ngit-relay (Reference) | ngit-grasp (This Project) |
8|--------|------------------------|---------------------------|
9| **Language** | Go | Rust |
10| **Architecture** | Multi-process (nginx + fcgiwrap + khatru + sync daemon) | Single integrated process |
11| **Git Protocol** | git-http-backend (C via fcgiwrap) | HTTP layer in Rust + git subprocess |
12| **Authorization** | Pre-receive Git hook | Inline HTTP handler validation |
13| **Nostr Relay** | Khatru (Go library) | nostr-relay-builder (Rust library) |
14| **Event Store** | Badger (Go KV database) | LMDB (Rust) |
15| **Proactive Sync** | Git-only (polls DB + fetches from git servers) | Nostr event sync + git sync (event-driven) |
16| **Process Management** | supervisord (4 processes) | Single tokio runtime |
17| **Packaging** | Docker with supervisord | Single static binary or Docker |
18| **Configuration** | Environment variables | Environment variables + CLI flags |
19| **Total Code** | ~1,866 lines of Go | ~25,000 lines of Rust |
20
21## Architecture Comparison
22
23### ngit-relay (Multi-Process)
24
25```
26┌──────────────── Docker Container ────────────────┐
27│ │
28│ ┌─────────────────────────────────────────────┐ │
29│ │ supervisord │ │
30│ │ - fcgiwrap (git-http-backend wrapper) │ │
31│ │ - nginx (HTTP + reverse proxy) │ │
32│ │ - ngit-relay-khatru (Nostr relay) │ │
33│ │ - ngit-relay-proactive-sync (sync daemon) │ │
34│ └─────────────────────────────────────────────┘ │
35│ │
36│ ┌──────────┐ ┌────────────────────┐ │
37│ │ nginx │────────▶│ git-http-backend │ │
38│ │ :80 │ │ (C binary via CGI) │ │
39│ └──────┬───┘ └──────────┬─────────┘ │
40│ │ │ │
41│ │ ▼ │
42│ │ ┌──────────────────┐ │
43│ │ │ Git Repos │ │
44│ │ │ + pre-receive │ │
45│ │ │ hook (Go) │ │
46│ │ └────────┬─────────┘ │
47│ │ │ WebSocket │
48│ │ │ query │
49│ │ ▼ │
50│ │ ┌──────────────────┐ │
51│ └──────────────▶│ Khatru Relay │ │
52│ │ :3334 │ │
53│ │ (Badger DB) │ │
54│ └──────────────────┘ │
55│ │
56│ Separate sync daemon polls relay DB │
57│ and fetches from remote git servers │
58│ │
59└───────────────────────────────────────────────────┘
60```
61
62### ngit-grasp (Single Process)
63
64```
65┌────────────── ngit-grasp (Single Binary) ─────────────┐
66│ │
67│ ┌──────────────────────────────────────────────────┐ │
68│ │ hyper HTTP Server (:7334) │ │
69│ │ - WebSocket upgrade for Nostr relay │ │
70│ │ - Git Smart HTTP handlers │ │
71│ │ - Landing page + metrics endpoint │ │
72│ └───────┬──────────────────────┬───────────────────┘ │
73│ │ │ │
74│ ▼ ▼ │
75│ ┌──────────────┐ ┌────────────────────┐ │
76│ │ Git Handlers │ │ Nostr Relay │ │
77│ │ (HTTP layer) │ │ (nostr-relay- │ │
78│ │ │ │ builder library) │ │
79│ │ - info/refs │ │ - NIP-34 Policy │ │
80│ │ - upload-pk │◀─────┤ (inline query) │ │
81│ │ - receive-pk │ auth │ - LMDB/Memory │ │
82│ │ + inline │ check│ - WebSocket │ │
83│ │ validation │ │ - NIP-11 endpoint │ │
84│ └──────┬───────┘ └──────────┬─────────┘ │
85│ │ │ │
86│ ▼ ▼ │
87│ ┌──────────────┐ ┌────────────────────┐ │
88│ │ git binary │ │ Purgatory │ │
89│ │ upload-pack │ │ (in-memory queue) │ │
90│ │ receive-pk │ │ + sync loop │ │
91│ └──────────────┘ └────────────────────┘ │
92│ │
93│ ┌──────────────────────────────────────────────────┐ │
94│ │ SyncManager (tokio background task) │ │
95│ │ - Multi-relay Nostr event sync (GRASP-02) │ │
96│ │ - Negentropy + REQ/EOSE support │ │
97│ │ - Health tracking & exponential backoff │ │
98│ │ - Git fetch from remote servers (via purgatory) │ │
99│ └──────────────────────────────────────────────────┘ │
100│ │
101│ ┌──────────────────────────────────────────────────┐ │
102│ │ Shared State (Arc<T>) │ │
103│ │ - Database (LMDB/Memory) │ │
104│ │ - Purgatory (DashMap - concurrent queue) │ │
105│ │ - Metrics (Prometheus) │ │
106│ └──────────────────────────────────────────────────┘ │
107│ │
108└────────────────────────────────────────────────────────┘
109```
110
111## Feature Comparison
112
113### Key Architectural Difference: Nostr Event Sync
114
115**The biggest difference between the two implementations is how they handle Nostr events:**
116
117| Aspect | ngit-relay | ngit-grasp |
118|--------|-----------|-----------|
119| **Event Arrival** | Relies on clients to push events directly | Proactively syncs events from other relays |
120| **Discovery** | None - only stores what clients send | Discovers events from relay network |
121| **Coordination** | Events and git data handled separately | Purgatory coordinates events + git data |
122| **Completeness** | May miss events if clients don't push to this relay | Actively fetches missing events from network |
123| **Implementation** | No event sync code (~0 lines) | Full multi-relay sync system (~5,000 lines) |
124
125**Example scenario:**
126- User creates PR on relay A, pushes git data to server B
127- **ngit-relay**: Only knows about events/data pushed directly to it
128- **ngit-grasp**: Discovers PR event from relay A, fetches git data from server B
129
130This is why ngit-grasp has ~13x more code - the majority is implementing GRASP-02 proactive event sync.
131
132### Git Protocol Implementation
133
134| Feature | ngit-relay | ngit-grasp |
135|---------|-----------|-----------|
136| **HTTP Server** | nginx | hyper (Rust) |
137| **Git Backend** | git-http-backend (C) via fcgiwrap | HTTP protocol layer (Rust) + git binary |
138| **Process Model** | FastCGI spawns git-http-backend | HTTP handler spawns git subprocess |
139| **Upload Pack** | C binary passthrough | Rust parses HTTP → spawns `git upload-pack` |
140| **Receive Pack** | C binary → pre-receive hook | Rust validates → spawns `git receive-pack` |
141| **Authorization** | Go hook queries relay via WebSocket | In-process function call before git spawn |
142| **Error Reporting** | Hook stderr → git client | HTTP response body (before git runs) |
143| **CORS** | nginx config | hyper middleware |
144| **Lines of Code** | ~0 (uses C binary) + hook ~135 | ~1,000+ (HTTP protocol layer) |
145
146### Authorization Logic
147
148| Feature | ngit-relay | ngit-grasp |
149|---------|-----------|-----------|
150| **Location** | pre-receive hook (separate Go binary) | Inline HTTP handler (Rust) |
151| **Trigger** | Git invokes hook during push | HTTP handler before spawning git |
152| **State Query** | WebSocket to localhost:3334 | Direct database query (in-process) |
153| **Latency** | +50-100ms (hook spawn + WS query) | +10-20ms (function call) |
154| **Error Channel** | stderr → git client | HTTP 403 response |
155| **Ref Parsing** | Read from stdin (hook protocol) | Parse from HTTP request body |
156| **Maintainer Resolution** | Recursive Go function | Recursive Rust function (similar) |
157| **State Caching** | None (queries relay per push) | Purgatory tracks pending events |
158
159### Nostr Relay
160
161| Feature | ngit-relay | ngit-grasp |
162|---------|-----------|-----------|
163| **Implementation** | Khatru (Go library) | nostr-relay-builder (Rust library) |
164| **Database** | Badger (Go KV store) | LMDB (Rust) |
165| **Process** | Separate process on :3334 | Integrated (same binary) |
166| **Policies** | Go functions in `policies.go` | Rust traits (modular sub-policies) |
167| **Event Validation** | Single function with branches | 4 separate policy modules |
168| **WebSocket** | Khatru built-in | nostr-relay-builder + hyper |
169| **NIP-11** | Manual JSON in code | Built-in support from library |
170| **Connection** | Separate from HTTP | Shared hyper server |
171| **Lines of Code** | ~186 (policies.go) + Khatru library | ~3,000+ (policy modules) + nostr-relay-builder library |
172
173### Proactive Sync
174
175| Feature | ngit-relay | ngit-grasp |
176|---------|-----------|-----------|
177| **Architecture** | Separate daemon (`ngit-relay-proactive-sync`) | Integrated SyncManager (tokio task) |
178| **Nostr Event Sync** | ❌ None (relies on client pushes) | ✅ Multi-relay sync with negentropy/REQ |
179| **Git Data Sync** | ✅ Polls local DB + fetches from git servers | ✅ Event-driven via purgatory queue |
180| **Sync Trigger** | Timer (every 15 minutes) | Immediate on event arrival + timer for retries |
181| **Relay Discovery** | N/A (no event sync) | Dynamic from 30617 announcement events |
182| **Protocol** | Git fetch only | Nostr WebSocket + git fetch |
183| **Concurrency** | Goroutines (per-repo iteration) | Tokio async tasks (per-relay connections) |
184| **Health Tracking** | Basic retry on git fetch failures | RelayHealthTracker with exponential backoff |
185| **Connection Management** | N/A (no Nostr connections) | Persistent connections with reconnect |
186| **Coordination** | Separate process | Purgatory + SyncManager coordination |
187| **Lines of Code** | ~112 (main.go) + ~305 (git sync) | ~5,000+ (Nostr sync + git sync + coordination) |
188
189### Repository Management
190
191| Feature | ngit-relay | ngit-grasp |
192|---------|-----------|-----------|
193| **Creation** | Event hook → shell commands | Event hook → tokio::process |
194| **Trigger** | `EventReceiveHook()` in Go | `handle_announcement()` in Rust |
195| **Configuration** | `git config` via shell | `git config` via tokio::process |
196| **Hook Installation** | Symlinks to pre-receive/post-receive | Not needed (inline auth) |
197| **Permissions** | `chown nginx:nginx` | tokio::fs permissions |
198| **Path Structure** | `<npub>/<id>.git` | `<npub>/<id>.git` (same) |
199
200### Event Coordination (Purgatory)
201
202| Feature | ngit-relay | ngit-grasp |
203|---------|-----------|-----------|
204| **Implementation** | None | Dedicated Purgatory system |
205| **Purpose** | N/A | Solves "which arrives first?" problem |
206| **Storage** | N/A | In-memory DashMap (thread-safe) |
207| **Expiry** | N/A | 30 minutes default TTL |
208| **State Events** | Accepted (git sync happens later via timer) | Queued until git data arrives |
209| **PR Events** | Accepted (references may be missing) | Queued with placeholder refs |
210| **Sync Queue** | Timer-based (polls all repos) | Event-driven (only syncs needed repos) |
211| **Cleanup** | N/A | Background task (60s interval) |
212| **Lines of Code** | 0 | ~2,000+ |
213
214**Impact**: ngit-relay accepts all events and relies on periodic sync to eventually fetch git data. ngit-grasp holds events in purgatory and triggers targeted syncs, providing faster convergence and better coordination between Nostr events and git data.
215
216### Deployment & Operations
217
218| Feature | ngit-relay | ngit-grasp |
219|---------|-----------|-----------|
220| **Dependencies** | nginx, git, fcgiwrap, supervisord, Go runtime | git, Rust binary (statically linked) |
221| **Process Count** | 4 (supervisord + nginx + khatru + sync) | 1 (single tokio runtime) |
222| **Configuration** | `.env` file | `.env` + CLI flags (clap) |
223| **Docker Image Size** | ~500MB (Alpine + tools + Go runtime) | ~100MB (Debian slim + git + binary) |
224| **Startup Time** | ~2-5 seconds (multiple processes) | ~0.5 seconds (single process) |
225| **Memory (Idle)** | ~150-200MB (4 processes + Go GC) | ~50-100MB (single process, no GC) |
226| **Logs** | supervisord → stdout (4 streams) | tracing → stdout (unified) |
227| **Monitoring** | None built-in | Prometheus metrics endpoint |
228| **Binary Distribution** | Docker only | Native binary + Docker |
229
230### Development Experience
231
232| Feature | ngit-relay | ngit-grasp |
233|---------|-----------|-----------|
234| **Build Time** | Fast (~5s incremental, Go) | Slow first build (~5min), fast incremental |
235| **Type Safety** | Good (Go interfaces) | Excellent (Rust traits + ownership) |
236| **Testing** | Go tests + shell scripts | Rust unit + integration tests |
237| **Test Relay** | Manual Docker setup | `TestRelay` fixture (auto-start binary) |
238| **Debugging** | Multi-process (harder) | Single process (easier) |
239| **IDE Support** | Good (gopls) | Excellent (rust-analyzer) |
240| **Async Model** | Goroutines (simple) | Tokio (more complex) |
241| **Error Handling** | `error` interface + if checks | Result<T, E> + `?` operator |
242| **Dependencies** | Go modules | Cargo crates (larger ecosystem) |
243
244### Code Complexity
245
246| Component | ngit-relay | ngit-grasp | Notes |
247|-----------|-----------|-----------|-------|
248| Main server | 129 | 196 | ngit-relay uses supervisord |
249| Git HTTP protocol | 0 (C binary via fcgiwrap) | ~1,000 | ngit-grasp implements HTTP layer |
250| Auth logic (hooks) | 135 + 52 | 0 | ngit-grasp inline, no hooks |
251| Auth logic (inline) | 0 | ~800 | ngit-grasp authorization module |
252| Nostr relay policies | 186 | ~3,000 | Both use libraries (Khatru vs nostr-relay-builder) |
253| Git-only proactive sync | 112 + 305 | 0 | ngit-relay git sync only |
254| Nostr event proactive sync | 0 | ~5,000 | ngit-grasp adds full event sync (GRASP-02 v4) |
255| Purgatory coordination | 0 | ~2,000 | ngit-grasp event/git coordination |
256| Shared utils | 241 + 132 | ~4,000 | ngit-grasp more comprehensive |
257| Config | ~50 | ~400 | ngit-grasp CLI + validation |
258| Metrics | 0 | ~1,500 | ngit-grasp Prometheus |
259| **Total** | **~1,866** | **~25,000** | ngit-grasp 13x more code |
260
261**Why the difference?**
262- **Nostr event sync**: ngit-relay has NONE, ngit-grasp implements full multi-relay event sync (~5,000 lines)
263- **Git HTTP protocol**: ngit-relay uses C binary, ngit-grasp implements HTTP layer (~1,000 lines)
264- **Purgatory coordination**: ngit-grasp adds event/git coordination system (~2,000 lines)
265- **Metrics & observability**: ngit-grasp includes comprehensive monitoring (~1,500 lines)
266- Both use relay libraries (Khatru vs nostr-relay-builder), but ngit-grasp has more modular policies
267
268### Performance Characteristics (Estimated)
269
270| Metric | ngit-relay | ngit-grasp | Notes |
271|--------|-----------|-----------|-------|
272| **Startup** | ~2-5s | ~0.5s | Single process vs multi-process |
273| **Memory (Idle)** | ~150MB | ~75MB | No GC, single process |
274| **Memory (Active)** | ~200MB+ | ~100-150MB | Depends on event volume |
275| **CPU (Idle)** | ~1-2% | ~0.5% | Fewer processes |
276| **Push Latency** | +50-100ms | +10-20ms | No hook spawn overhead |
277| **Clone Latency** | ~same | ~same | Both passthrough to git |
278| **Concurrent Pushes** | Good (goroutines) | Excellent (tokio async) |
279| **Event Ingestion** | Good (Badger) | Excellent (LMDB zero-copy) |
280| **Sync Throughput** | Moderate (polling) | High (negentropy + async) |
281
282*These are estimates based on architecture. Actual performance depends on workload.*
283
284## Migration Path
285
286For users of ngit-relay, migration to ngit-grasp involves:
287
288### Data Migration
289
2901. **Events**: Export from Badger → Import to LMDB
291 - No direct migration tool yet (would need to be built)
292 - Alternative: Use proactive sync to re-fetch from other relays
2932. **Git Repositories**: Direct copy (same structure)
294 ```bash
295 cp -r /srv/ngit-relay/repos/* /path/to/ngit-grasp/data/git/
296 ```
2973. **Configuration**: Translate environment variables
298 - Most variables are compatible (`NGIT_DOMAIN`, etc.)
299 - Remove nginx/supervisord-specific configs
300
301### Compatibility
302
303- **Git Data**: 100% compatible (same repository structure)
304- **Nostr Events**: 100% compatible (standard NIP-34)
305- **HTTP URLs**: Compatible (same path structure)
306- **Git Hooks**: ngit-grasp doesn't use hooks (inline auth instead)
307
308### Downtime
309
310- Option 1: Run both in parallel (different domains), gradually migrate
311- Option 2: Short downtime for data copy + config update
312
313## When to Choose Each
314
315### Choose ngit-relay (Reference) if:
316
317- ✅ You need proven, production-tested code
318- ✅ You're already familiar with Go ecosystem
319- ✅ You prefer simple, minimal codebases (~1,866 lines)
320- ✅ You trust battle-tested C binaries (git-http-backend)
321- ✅ You want to stay close to the reference implementation
322- ✅ You need to deploy immediately without complexity
323- ✅ Your users will push events directly to your relay (no sync needed)
324- ✅ You only need git data sync, not Nostr event sync
325
326### Choose ngit-grasp (This Project) if:
327
328- ✅ **You need Nostr event sync from other relays** (the main differentiator)
329- ✅ You want better performance and lower resource usage
330- ✅ You prefer Rust's type safety and memory safety
331- ✅ You want simpler deployment (single binary, no supervisord)
332- ✅ You need event/git data coordination (purgatory)
333- ✅ You want inline authorization (lower latency)
334- ✅ You need comprehensive observability (Prometheus metrics)
335- ✅ You're comfortable with more complex codebase (~25,000 lines)
336- ✅ You want full GRASP-02 v4 multi-relay event discovery
337
338## Current Status
339
340### ngit-relay (Reference)
341- ✅ GRASP-01 complete and production-ready
342- ✅ Git data proactive sync (fetches from git servers)
343- ❌ No Nostr event sync (relies on client pushes)
344- ✅ Battle-tested in production
345- 🔄 Community adoption growing
346
347### ngit-grasp (This Project)
348- ✅ GRASP-01 complete with comprehensive testing
349- ✅ GRASP-02 v4 multi-relay Nostr event sync with negentropy
350- ✅ Git data proactive sync (via purgatory queue)
351- ✅ Purgatory system for event/git coordination
352- ✅ Prometheus metrics and health tracking
353- ✅ NIP-77 negentropy support
354- ✅ Full integration test suite
355- 🔄 Production deployment validation ongoing
356
357## Conclusion
358
359Both implementations are valid approaches to GRASP with different philosophies:
360
361- **ngit-relay** prioritizes simplicity - clients push events, relay syncs git data (~1,866 lines)
362- **ngit-grasp** prioritizes completeness - syncs both events and git data from network (~25,000 lines)
363
364**The fundamental difference**: ngit-relay expects clients to push Nostr events to it. ngit-grasp proactively discovers and syncs events from other relays in the network.
365
366The choice depends on your priorities:
367
368| Priority | Recommendation |
369|----------|---------------|
370| **Simplicity** | ngit-relay |
371| **Event Discovery** | ngit-grasp (syncs from network) |
372| **Production Stability** | ngit-relay (more battle-tested) |
373| **Event Completeness** | ngit-grasp (proactive sync) |
374| **Low Resources** | ngit-grasp (single binary, lower memory) |
375| **Quick Deploy** | ngit-relay (Docker Compose) |
376| **Development** | ngit-grasp (better tooling, type safety) |
377| **Network Resilience** | ngit-grasp (multi-relay sync) |
378
379For deployments where **Nostr event sync** is important (discovering events from other relays), **ngit-grasp** is required. For simpler deployments where users will push events directly, **ngit-relay** is sufficient and battle-tested.
generated by cgit v1.2.3 (git 2.39.1) at 2026-06-06 15:35:21 +0000