diff options
Diffstat (limited to 'docs/explanation/architecture.md')
| -rw-r--r-- | docs/explanation/architecture.md | 716 |
1 files changed, 175 insertions, 541 deletions
diff --git a/docs/explanation/architecture.md b/docs/explanation/architecture.md index ebf7a74..3fb895c 100644 --- a/docs/explanation/architecture.md +++ b/docs/explanation/architecture.md | |||
| @@ -2,16 +2,16 @@ | |||
| 2 | 2 | ||
| 3 | ## Executive Summary | 3 | ## Executive Summary |
| 4 | 4 | ||
| 5 | `ngit-grasp` implements the GRASP protocol in Rust with **inline authorization** rather than Git hooks. The key architectural insight is that the `git-http-backend` Rust crate provides sufficient flexibility to intercept and validate Git push operations before they reach the Git repository, eliminating the need for pre-receive hooks. | 5 | `ngit-grasp` implements the GRASP protocol in Rust with **inline authorization** rather than Git hooks. The key architectural insight is that we can intercept and validate Git push operations at the HTTP handler level before reaching the Git repository, eliminating the need for pre-receive hooks. |
| 6 | 6 | ||
| 7 | ## Architectural Decision: Inline vs. Hook-Based Authorization | 7 | ## Architectural Decision: Inline vs. Hook-Based Authorization |
| 8 | 8 | ||
| 9 | ### Investigation Summary | 9 | ### Investigation Summary |
| 10 | 10 | ||
| 11 | After examining both the reference implementation and the `git-http-backend` Rust crate, we have two options: | 11 | After examining both the reference implementation and HTTP server options, we have two options: |
| 12 | 12 | ||
| 13 | #### Option 1: Hook-Based (Reference Implementation Approach) | 13 | #### Option 1: Hook-Based (Reference Implementation Approach) |
| 14 | - Use `git-http-backend` crate as-is | 14 | - Use standard Git HTTP backend |
| 15 | - Create pre-receive and post-receive hooks | 15 | - Create pre-receive and post-receive hooks |
| 16 | - Hooks query the Nostr relay and validate pushes | 16 | - Hooks query the Nostr relay and validate pushes |
| 17 | - **Pros**: Follows reference implementation closely | 17 | - **Pros**: Follows reference implementation closely |
| @@ -28,7 +28,7 @@ After examining both the reference implementation and the `git-http-backend` Rus | |||
| 28 | 28 | ||
| 29 | **Rationale:** | 29 | **Rationale:** |
| 30 | 30 | ||
| 31 | 1. **The `git-http-backend` crate is sufficiently flexible**: Examining `src/actix/git_receive_pack.rs` shows it spawns `git receive-pack` as a subprocess and streams data. We can intercept this. | 31 | 1. **Full control over HTTP layer**: Using Hyper directly gives us complete control over request handling, WebSocket upgrades, and CORS headers. |
| 32 | 32 | ||
| 33 | 2. **Better Developer Experience**: | 33 | 2. **Better Developer Experience**: |
| 34 | - Validation errors can be returned as proper HTTP responses | 34 | - Validation errors can be returned as proper HTTP responses |
| @@ -56,7 +56,7 @@ After examining both the reference implementation and the `git-http-backend` Rus | |||
| 56 | │ │ | 56 | │ │ |
| 57 | │ ┌──────────────────┐ ┌──────────────────┐ │ | 57 | │ ┌──────────────────┐ ┌──────────────────┐ │ |
| 58 | │ │ HTTP Router │ │ Nostr Relay │ │ | 58 | │ │ HTTP Router │ │ Nostr Relay │ │ |
| 59 | │ │ (actix-web) │ │ (nostr-relay- │ │ | 59 | │ │ (Hyper) │ │ (nostr-relay- │ │ |
| 60 | │ │ │ │ builder) │ │ | 60 | │ │ │ │ builder) │ │ |
| 61 | │ └────────┬─────────┘ └────────┬─────────┘ │ | 61 | │ └────────┬─────────┘ └────────┬─────────┘ │ |
| 62 | │ │ │ │ | 62 | │ │ │ │ |
| @@ -90,299 +90,149 @@ After examining both the reference implementation and the `git-http-backend` Rus | |||
| 90 | 90 | ||
| 91 | ## Component Design | 91 | ## Component Design |
| 92 | 92 | ||
| 93 | ### 1. Main Server (`src/main.rs`) | 93 | ### 1. Main Server ([`src/main.rs`](src/main.rs)) |
| 94 | 94 | ||
| 95 | **Responsibilities:** | 95 | **Responsibilities:** |
| 96 | - Initialize configuration from environment | 96 | - Initialize configuration from environment (clap + dotenvy) |
| 97 | - Set up actix-web HTTP server | 97 | - Set up Hyper HTTP server with request routing |
| 98 | - Initialize Nostr relay builder | 98 | - Initialize Nostr relay builder with custom [`Nip34WritePolicy`](src/nostr/builder.rs:51) |
| 99 | - Set up shared storage | 99 | - Set up shared storage (LMDB, NostrDB, or Memory) |
| 100 | - Configure routes for both Git and Nostr endpoints | 100 | - Handle WebSocket upgrades for Nostr relay |
| 101 | - Handle graceful shutdown | 101 | - Handle graceful shutdown |
| 102 | 102 | ||
| 103 | **Key Dependencies:** | 103 | **Key Dependencies:** |
| 104 | ```rust | 104 | ```rust |
| 105 | actix-web = "4" | 105 | hyper = "1" |
| 106 | tokio = { version = "1", features = ["full"] } | 106 | tokio = { version = "1", features = ["full"] } |
| 107 | nostr-relay-builder = "0.43" | 107 | nostr-relay-builder = "0.43" |
| 108 | nostr-sdk = "0.43" | 108 | nostr-sdk = "0.43" |
| 109 | nostr-lmdb = "0.43" | ||
| 109 | ``` | 110 | ``` |
| 110 | 111 | ||
| 111 | ### 2. Git Module (`src/git/`) | 112 | ### 2. HTTP Module ([`src/http/mod.rs`](src/http/mod.rs)) |
| 112 | 113 | ||
| 113 | #### `handler.rs` - Git HTTP Handlers | 114 | **Responsibilities:** |
| 115 | - Route HTTP requests to appropriate handlers | ||
| 116 | - WebSocket upgrade for Nostr relay at `/` | ||
| 117 | - Git Smart HTTP endpoints at `/<npub>/<identifier>.git/*` | ||
| 118 | - Landing pages and NIP-11 document serving | ||
| 119 | - CORS headers on all responses (GRASP-01 requirement) | ||
| 114 | 120 | ||
| 115 | Implements actix-web handlers for Git Smart HTTP protocol: | 121 | **Key Implementation Details:** |
| 116 | 122 | ||
| 117 | ```rust | 123 | ```rust |
| 118 | // GET /<npub>/<identifier>.git/info/refs?service=git-upload-pack | 124 | // CORS headers required by GRASP-01 specification |
| 119 | async fn info_refs_upload_pack( | 125 | const CORS_ALLOW_ORIGIN: &str = "*"; |
| 120 | req: HttpRequest, | 126 | const CORS_ALLOW_METHODS: &str = "GET, POST"; |
| 121 | state: web::Data<AppState>, | 127 | const CORS_ALLOW_HEADERS: &str = "Content-Type"; |
| 122 | ) -> Result<HttpResponse> | 128 | |
| 123 | 129 | /// Add CORS headers to a response builder | |
| 124 | // POST /<npub>/<identifier>.git/git-upload-pack | 130 | fn add_cors_headers(builder: http::response::Builder) -> http::response::Builder { |
| 125 | async fn git_upload_pack( | 131 | builder |
| 126 | req: HttpRequest, | 132 | .header("Access-Control-Allow-Origin", CORS_ALLOW_ORIGIN) |
| 127 | body: web::Payload, | 133 | .header("Access-Control-Allow-Methods", CORS_ALLOW_METHODS) |
| 128 | state: web::Data<AppState>, | 134 | .header("Access-Control-Allow-Headers", CORS_ALLOW_HEADERS) |
| 129 | ) -> Result<HttpResponse> | 135 | } |
| 130 | |||
| 131 | // GET /<npub>/<identifier>.git/info/refs?service=git-receive-pack | ||
| 132 | async fn info_refs_receive_pack( | ||
| 133 | req: HttpRequest, | ||
| 134 | state: web::Data<AppState>, | ||
| 135 | ) -> Result<HttpResponse> | ||
| 136 | |||
| 137 | // POST /<npub>/<identifier>.git/git-receive-pack | ||
| 138 | // THIS IS WHERE THE MAGIC HAPPENS | ||
| 139 | async fn git_receive_pack( | ||
| 140 | req: HttpRequest, | ||
| 141 | body: web::Payload, | ||
| 142 | state: web::Data<AppState>, | ||
| 143 | ) -> Result<HttpResponse> | ||
| 144 | ``` | 136 | ``` |
| 145 | 137 | ||
| 146 | #### `authorization.rs` - Push Validation | 138 | See [`src/http/mod.rs:29-84`](src/http/mod.rs:29-84) for the full CORS implementation. |
| 147 | 139 | ||
| 148 | **Core Logic:** | 140 | ### 3. Git Module ([`src/git/`](src/git/)) |
| 149 | 141 | ||
| 150 | ```rust | 142 | #### [`handlers.rs`](src/git/handlers.rs) - Git HTTP Handlers |
| 151 | pub struct PushValidator { | ||
| 152 | nostr_client: Arc<Client>, | ||
| 153 | relay_url: String, | ||
| 154 | } | ||
| 155 | 143 | ||
| 156 | impl PushValidator { | 144 | Implements handlers for Git Smart HTTP protocol: |
| 157 | /// Validate a push operation against Nostr state | ||
| 158 | pub async fn validate_push( | ||
| 159 | &self, | ||
| 160 | npub: &str, | ||
| 161 | identifier: &str, | ||
| 162 | ref_updates: Vec<RefUpdate>, | ||
| 163 | ) -> Result<ValidationResult> { | ||
| 164 | // 1. Fetch announcement and state events from local relay | ||
| 165 | let events = self.fetch_events(identifier).await?; | ||
| 166 | |||
| 167 | // 2. Extract pubkey from npub | ||
| 168 | let pubkey = decode_npub(npub)?; | ||
| 169 | |||
| 170 | // 3. Get recursive maintainer set | ||
| 171 | let maintainers = get_maintainers(&events, &pubkey, identifier); | ||
| 172 | |||
| 173 | // 4. Get latest state from maintainers | ||
| 174 | let state = get_state_from_maintainers(&events, &maintainers)?; | ||
| 175 | |||
| 176 | // 5. Validate each ref update | ||
| 177 | for ref_update in ref_updates { | ||
| 178 | if ref_update.ref_name.starts_with("refs/nostr/") { | ||
| 179 | // Allow refs/nostr/<event-id> for PRs | ||
| 180 | validate_pr_ref(&ref_update)?; | ||
| 181 | } else if ref_update.ref_name.starts_with("refs/heads/pr/") { | ||
| 182 | // Reject pr/* branches - should use refs/nostr/ | ||
| 183 | return Err(Error::InvalidRef("pr/* branches must use refs/nostr/")); | ||
| 184 | } else { | ||
| 185 | // Validate against state event | ||
| 186 | validate_state_ref(&state, &ref_update)?; | ||
| 187 | } | ||
| 188 | } | ||
| 189 | |||
| 190 | Ok(ValidationResult::Accept) | ||
| 191 | } | ||
| 192 | } | ||
| 193 | ``` | ||
| 194 | |||
| 195 | **Key Functions:** | ||
| 196 | 145 | ||
| 197 | ```rust | 146 | ```rust |
| 198 | /// Parse ref updates from git-receive-pack request body | 147 | /// Handle GET /info/refs?service=git-{upload,receive}-pack |
| 199 | fn parse_ref_updates(body: &[u8]) -> Result<Vec<RefUpdate>> | 148 | pub async fn handle_info_refs( |
| 200 | 149 | repo_path: PathBuf, | |
| 201 | /// Recursively find all maintainers | 150 | service: GitService, |
| 202 | fn get_maintainers( | 151 | ) -> Result<Response<Full<Bytes>>, GitError> |
| 203 | events: &[Event], | 152 | |
| 204 | pubkey: &str, | 153 | /// Handle POST /git-upload-pack (clone/fetch) |
| 154 | pub async fn handle_upload_pack( | ||
| 155 | repo_path: PathBuf, | ||
| 156 | body: Bytes, | ||
| 157 | ) -> Result<Response<Full<Bytes>>, GitError> | ||
| 158 | |||
| 159 | /// Handle POST /git-receive-pack (push) | ||
| 160 | /// THIS IS WHERE THE MAGIC HAPPENS - validates against state before accepting | ||
| 161 | pub async fn handle_receive_pack( | ||
| 162 | repo_path: PathBuf, | ||
| 163 | body: Bytes, | ||
| 164 | database: SharedDatabase, | ||
| 165 | npub: &str, | ||
| 205 | identifier: &str, | 166 | identifier: &str, |
| 206 | ) -> Vec<String> | 167 | ) -> Result<Response<Full<Bytes>>, GitError> |
| 207 | |||
| 208 | /// Get latest state from maintainer set | ||
| 209 | fn get_state_from_maintainers( | ||
| 210 | events: &[Event], | ||
| 211 | maintainers: &[String], | ||
| 212 | ) -> Result<RepositoryState> | ||
| 213 | |||
| 214 | /// Validate a ref matches the state event | ||
| 215 | fn validate_state_ref( | ||
| 216 | state: &RepositoryState, | ||
| 217 | ref_update: &RefUpdate, | ||
| 218 | ) -> Result<()> | ||
| 219 | ``` | 168 | ``` |
| 220 | 169 | ||
| 221 | ### 3. Nostr Module (`src/nostr/`) | 170 | See [`src/git/handlers.rs:22-98`](src/git/handlers.rs:22-98) for the info-refs implementation. |
| 222 | 171 | ||
| 223 | #### `relay.rs` - Relay Configuration | 172 | #### [`authorization.rs`](src/git/authorization.rs) - Push Validation |
| 224 | 173 | ||
| 225 | ```rust | 174 | **Core Logic:** |
| 226 | pub async fn build_relay(config: &Config) -> Result<LocalRelay> { | ||
| 227 | let builder = RelayBuilder::default() | ||
| 228 | .write_policy(RepositoryAnnouncementPolicy::new(config.domain.clone())) | ||
| 229 | .write_policy(RelatedEventsPolicy::new()) | ||
| 230 | .query_policy(StandardQueryPolicy::new()) | ||
| 231 | .on_event_saved(create_repository_hook(config.git_data_path.clone())); | ||
| 232 | |||
| 233 | // Configure storage backend (LMDB or NDB) | ||
| 234 | let relay = LocalRelay::run(builder).await?; | ||
| 235 | |||
| 236 | Ok(relay) | ||
| 237 | } | ||
| 238 | ``` | ||
| 239 | |||
| 240 | #### `events.rs` - Event Handlers | ||
| 241 | 175 | ||
| 242 | ```rust | 176 | ```rust |
| 243 | /// Hook called when events are saved | 177 | /// Get authorization info for a repository owner |
| 244 | pub fn create_repository_hook( | 178 | pub async fn get_authorization_for_owner( |
| 245 | git_data_path: PathBuf, | 179 | database: &SharedDatabase, |
| 246 | ) -> impl Fn(&Event) -> BoxFuture<'static, ()> { | 180 | pubkey: &PublicKey, |
| 247 | move |event: &Event| { | 181 | identifier: &str, |
| 248 | let git_path = git_data_path.clone(); | 182 | ) -> Result<AuthorizationResult, AuthorizationError> |
| 249 | Box::pin(async move { | ||
| 250 | if event.kind == Kind::RepositoryAnnouncement { | ||
| 251 | handle_repository_announcement(event, &git_path).await; | ||
| 252 | } else if event.kind == Kind::RepositoryState { | ||
| 253 | handle_repository_state(event, &git_path).await; | ||
| 254 | } | ||
| 255 | }) | ||
| 256 | } | ||
| 257 | } | ||
| 258 | 183 | ||
| 259 | async fn handle_repository_announcement(event: &Event, git_path: &Path) { | 184 | /// Validate that pushed refs match the authorized state |
| 260 | // 1. Parse repository from event | 185 | pub fn validate_push_refs( |
| 261 | // 2. Check if listed in clone and relays tags | 186 | pushed_refs: &[PushedRef], |
| 262 | // 3. Create empty bare Git repository | 187 | state: &RepositoryState, |
| 263 | // 4. Configure uploadpack.allowTipSHA1InWant | 188 | ) -> Result<(), AuthorizationError> |
| 264 | // 5. Configure uploadpack.allowUnreachable | ||
| 265 | // 6. Configure http.receivepack | ||
| 266 | } | ||
| 267 | 189 | ||
| 268 | async fn handle_repository_state(event: &Event, git_path: &Path) { | 190 | /// Validate refs/nostr/<event-id> pushes |
| 269 | // 1. Parse state from event | 191 | pub fn validate_nostr_ref_pushes( |
| 270 | // 2. Update repository HEAD if needed | 192 | pushed_refs: &[PushedRef], |
| 271 | // 3. Trigger proactive sync (GRASP-02) | 193 | database: &SharedDatabase, |
| 272 | } | 194 | ) -> Result<(), AuthorizationError> |
| 273 | ``` | 195 | ``` |
| 274 | 196 | ||
| 275 | **Write Policies:** | 197 | ### 4. Nostr Module ([`src/nostr/`](src/nostr/)) |
| 276 | 198 | ||
| 277 | ```rust | 199 | #### [`builder.rs`](src/nostr/builder.rs) - Relay Configuration |
| 278 | /// Accept repository announcements that list this instance | ||
| 279 | pub struct RepositoryAnnouncementPolicy { | ||
| 280 | domain: String, | ||
| 281 | } | ||
| 282 | |||
| 283 | impl WritePolicy for RepositoryAnnouncementPolicy { | ||
| 284 | fn admit_event(&self, event: &Event, _addr: &SocketAddr) | ||
| 285 | -> BoxFuture<PolicyResult> | ||
| 286 | { | ||
| 287 | Box::pin(async move { | ||
| 288 | if event.kind != Kind::RepositoryAnnouncement { | ||
| 289 | return PolicyResult::Accept; // Not our concern | ||
| 290 | } | ||
| 291 | |||
| 292 | // Check if this instance is in clone and relays tags | ||
| 293 | let has_clone = event.tags.iter() | ||
| 294 | .any(|t| t.kind() == "clone" && t.content() == Some(&self.domain)); | ||
| 295 | let has_relay = event.tags.iter() | ||
| 296 | .any(|t| t.kind() == "relays" && t.content() == Some(&self.domain)); | ||
| 297 | |||
| 298 | if has_clone && has_relay { | ||
| 299 | PolicyResult::Accept | ||
| 300 | } else { | ||
| 301 | PolicyResult::Reject("instance not listed in clone and relays".into()) | ||
| 302 | } | ||
| 303 | }) | ||
| 304 | } | ||
| 305 | } | ||
| 306 | 200 | ||
| 307 | /// Accept events related to stored announcements/issues/patches | 201 | The [`Nip34WritePolicy`](src/nostr/builder.rs:51) is the core event validation logic: |
| 308 | pub struct RelatedEventsPolicy; | ||
| 309 | 202 | ||
| 310 | impl WritePolicy for RelatedEventsPolicy { | 203 | ```rust |
| 311 | fn admit_event(&self, event: &Event, _addr: &SocketAddr) | 204 | /// NIP-34 Write Policy with Full GRASP-01 Event Validation |
| 312 | -> BoxFuture<PolicyResult> | 205 | /// |
| 313 | { | 206 | /// Validates all events according to GRASP-01 specification: |
| 314 | // Accept if event tags or is tagged by stored events | 207 | /// - Repository announcements must list service in clone and relays tags |
| 315 | // Implementation requires querying the event store | 208 | /// EXCEPTION: Recursive maintainer announcements are accepted even without |
| 316 | } | 209 | /// listing the service, to enable maintainer chain discovery and GRASP-02 sync |
| 210 | /// - Repository state announcements must have valid structure | ||
| 211 | /// - Other events must reference accepted repositories or events | ||
| 212 | /// - Forward references are supported (events referenced by accepted events) | ||
| 213 | /// - Orphan events with no valid references are rejected | ||
| 214 | pub struct Nip34WritePolicy { | ||
| 215 | domain: String, | ||
| 216 | database: SharedDatabase, | ||
| 217 | git_data_path: PathBuf, | ||
| 317 | } | 218 | } |
| 318 | ``` | 219 | ``` |
| 319 | 220 | ||
| 320 | ### 4. Storage Module (`src/storage/`) | 221 | See [`src/nostr/builder.rs:38-78`](src/nostr/builder.rs:38-78) for the full policy struct. |
| 321 | 222 | ||
| 322 | #### `repository.rs` - Repository Management | 223 | #### [`events.rs`](src/nostr/events.rs) - Event Parsing |
| 224 | |||
| 225 | Provides structures for parsing NIP-34 events: | ||
| 323 | 226 | ||
| 324 | ```rust | 227 | ```rust |
| 325 | pub struct RepositoryManager { | 228 | /// Parsed repository announcement (Kind 30617) |
| 326 | git_data_path: PathBuf, | 229 | pub struct RepositoryAnnouncement { ... } |
| 327 | } | ||
| 328 | 230 | ||
| 329 | impl RepositoryManager { | 231 | /// Parsed repository state (Kind 30618) |
| 330 | /// Create a new bare Git repository | 232 | pub struct RepositoryState { ... } |
| 331 | pub async fn create_repository( | ||
| 332 | &self, | ||
| 333 | npub: &str, | ||
| 334 | identifier: &str, | ||
| 335 | ) -> Result<PathBuf> { | ||
| 336 | let repo_path = self.git_data_path | ||
| 337 | .join(npub) | ||
| 338 | .join(format!("{}.git", identifier)); | ||
| 339 | |||
| 340 | // Create directory | ||
| 341 | tokio::fs::create_dir_all(&repo_path).await?; | ||
| 342 | |||
| 343 | // Initialize bare repo | ||
| 344 | Command::new("git") | ||
| 345 | .args(&["init", "--bare"]) | ||
| 346 | .arg(&repo_path) | ||
| 347 | .output() | ||
| 348 | .await?; | ||
| 349 | |||
| 350 | // Configure | ||
| 351 | self.configure_repository(&repo_path).await?; | ||
| 352 | |||
| 353 | Ok(repo_path) | ||
| 354 | } | ||
| 355 | |||
| 356 | async fn configure_repository(&self, repo_path: &Path) -> Result<()> { | ||
| 357 | // Enable unauthenticated push (we handle auth ourselves) | ||
| 358 | git_config(repo_path, "http.receivepack", "true").await?; | ||
| 359 | |||
| 360 | // Enable tip SHA1 fetching (required for ngit) | ||
| 361 | git_config(repo_path, "uploadpack.allowTipSHA1InWant", "true").await?; | ||
| 362 | |||
| 363 | // Enable unreachable object fetching | ||
| 364 | git_config(repo_path, "uploadpack.allowUnreachable", "true").await?; | ||
| 365 | |||
| 366 | Ok(()) | ||
| 367 | } | ||
| 368 | |||
| 369 | /// Check if repository exists | ||
| 370 | pub async fn repository_exists( | ||
| 371 | &self, | ||
| 372 | npub: &str, | ||
| 373 | identifier: &str, | ||
| 374 | ) -> bool { | ||
| 375 | let repo_path = self.git_data_path | ||
| 376 | .join(npub) | ||
| 377 | .join(format!("{}.git", identifier)); | ||
| 378 | |||
| 379 | repo_path.join("HEAD").exists() && | ||
| 380 | repo_path.join("config").exists() | ||
| 381 | } | ||
| 382 | } | ||
| 383 | ``` | 233 | ``` |
| 384 | 234 | ||
| 385 | ### 5. Configuration (`src/config.rs`) | 235 | ### 5. Configuration ([`src/config.rs`](src/config.rs)) |
| 386 | 236 | ||
| 387 | ```rust | 237 | ```rust |
| 388 | pub struct Config { | 238 | pub struct Config { |
| @@ -393,34 +243,18 @@ pub struct Config { | |||
| 393 | pub git_data_path: PathBuf, | 243 | pub git_data_path: PathBuf, |
| 394 | pub relay_data_path: PathBuf, | 244 | pub relay_data_path: PathBuf, |
| 395 | pub bind_address: SocketAddr, | 245 | pub bind_address: SocketAddr, |
| 396 | pub log_level: String, | 246 | pub database_backend: DatabaseBackend, |
| 397 | } | 247 | } |
| 398 | 248 | ||
| 399 | impl Config { | 249 | pub enum DatabaseBackend { |
| 400 | pub fn from_env() -> Result<Self> { | 250 | Lmdb, // Default, production use |
| 401 | Ok(Config { | 251 | NostrDb, // Alternative |
| 402 | domain: env::var("NGIT_DOMAIN")?, | 252 | Memory, // Testing |
| 403 | owner_npub: env::var("NGIT_OWNER_NPUB")?, | ||
| 404 | relay_name: env::var("NGIT_RELAY_NAME")?, | ||
| 405 | relay_description: env::var("NGIT_RELAY_DESCRIPTION")?, | ||
| 406 | git_data_path: PathBuf::from( | ||
| 407 | env::var("NGIT_GIT_DATA_PATH") | ||
| 408 | .unwrap_or_else(|_| "./data/git".to_string()) | ||
| 409 | ), | ||
| 410 | relay_data_path: PathBuf::from( | ||
| 411 | env::var("NGIT_RELAY_DATA_PATH") | ||
| 412 | .unwrap_or_else(|_| "./data/relay".to_string()) | ||
| 413 | ), | ||
| 414 | bind_address: env::var("NGIT_BIND_ADDRESS") | ||
| 415 | .unwrap_or_else(|_| "127.0.0.1:8080".to_string()) | ||
| 416 | .parse()?, | ||
| 417 | log_level: env::var("RUST_LOG") | ||
| 418 | .unwrap_or_else(|_| "info".to_string()), | ||
| 419 | }) | ||
| 420 | } | ||
| 421 | } | 253 | } |
| 422 | ``` | 254 | ``` |
| 423 | 255 | ||
| 256 | Configuration is loaded via **clap CLI > environment variables > .env > defaults**. | ||
| 257 | |||
| 424 | ## Data Flow | 258 | ## Data Flow |
| 425 | 259 | ||
| 426 | ### Push Operation Flow | 260 | ### Push Operation Flow |
| @@ -428,327 +262,120 @@ impl Config { | |||
| 428 | ``` | 262 | ``` |
| 429 | 1. Git Client → POST /<npub>/<id>.git/git-receive-pack | 263 | 1. Git Client → POST /<npub>/<id>.git/git-receive-pack |
| 430 | ↓ | 264 | ↓ |
| 431 | 2. git_receive_pack handler receives request | 265 | 2. HttpService routes to git::handlers::handle_receive_pack() |
| 432 | ↓ | 266 | ↓ |
| 433 | 3. Parse ref updates from request body | 267 | 3. Parse ref updates from request body (pkt-line format) |
| 434 | ↓ | 268 | ↓ |
| 435 | 4. Extract npub and identifier from URL | 269 | 4. Extract npub and identifier from URL |
| 436 | ↓ | 270 | ↓ |
| 437 | 5. PushValidator::validate_push() | 271 | 5. authorization::get_authorization_for_owner() |
| 438 | ├─ Fetch events from local Nostr relay | 272 | ├─ Query database for announcements |
| 439 | ├─ Get maintainers recursively | 273 | ├─ Build recursive maintainer set |
| 440 | ├─ Get latest state from maintainers | 274 | └─ Get latest authorized state |
| 441 | └─ Validate each ref update | 275 | ↓ |
| 276 | 6. authorization::validate_push_refs() | ||
| 277 | ├─ Check each ref matches state | ||
| 278 | └─ Validate refs/nostr/ pushes | ||
| 442 | ↓ | 279 | ↓ |
| 443 | 6. If VALID: | 280 | 7. If VALID: |
| 444 | ├─ Spawn git-receive-pack subprocess | 281 | ├─ Spawn git-receive-pack subprocess |
| 445 | ├─ Stream request body to git stdin | 282 | ├─ Stream request body to git stdin |
| 446 | └─ Stream git stdout back to client | 283 | └─ Stream git stdout back to client |
| 447 | ↓ | 284 | ↓ |
| 448 | 7. If INVALID: | 285 | 8. If INVALID: |
| 449 | └─ Return HTTP 403 with error message | 286 | └─ Return HTTP 403 with error message |
| 450 | ``` | 287 | ``` |
| 451 | 288 | ||
| 452 | ### Repository Announcement Flow | 289 | ### Repository Announcement Flow |
| 453 | 290 | ||
| 454 | ``` | 291 | ``` |
| 455 | 1. Nostr Client → EVENT (Kind 30317) | 292 | 1. Nostr Client → EVENT (Kind 30617) |
| 456 | ↓ | 293 | ↓ |
| 457 | 2. Nostr relay receives event | 294 | 2. Nostr relay receives event |
| 458 | ↓ | 295 | ↓ |
| 459 | 3. RepositoryAnnouncementPolicy::admit_event() | 296 | 3. Nip34WritePolicy::admit_event() |
| 460 | ├─ Check if instance in clone tags | 297 | ├─ Check if instance in clone tags |
| 461 | ├─ Check if instance in relays tags | 298 | ├─ Check if instance in relays tags |
| 299 | ├─ OR: Check if recursive maintainer | ||
| 462 | └─ Accept or reject | 300 | └─ Accept or reject |
| 463 | ↓ | 301 | ↓ |
| 464 | 4. If ACCEPTED: | 302 | 4. If ACCEPTED: |
| 465 | ├─ Event saved to store | 303 | ├─ Event saved to database |
| 466 | └─ on_event_saved hook triggered | 304 | └─ ensure_bare_repository() called |
| 467 | ↓ | 305 | ↓ |
| 468 | 5. handle_repository_announcement() | 306 | 5. Bare Git repository created at |
| 469 | ├─ Parse repository details | 307 | <git_data_path>/<npub>/<identifier>.git |
| 470 | ├─ Create Git repository directory | ||
| 471 | ├─ Initialize bare Git repo | ||
| 472 | └─ Configure Git settings | ||
| 473 | ``` | ||
| 474 | |||
| 475 | ## Key Implementation Details | ||
| 476 | |||
| 477 | ### 1. Parsing Git Receive-Pack Protocol | ||
| 478 | |||
| 479 | The Git receive-pack protocol uses a pkt-line format. We need to parse: | ||
| 480 | |||
| 481 | ``` | ||
| 482 | 0000-0000-0000-0000 0000-0000-0000-0000 refs/heads/main\0 report-status | ||
| 483 | 0000-0000-0000-0000 0000-0000-0000-0000 refs/heads/dev | ||
| 484 | ``` | ||
| 485 | |||
| 486 | Each line has: | ||
| 487 | - Old SHA (40 hex chars) | ||
| 488 | - Space | ||
| 489 | - New SHA (40 hex chars) | ||
| 490 | - Space | ||
| 491 | - Ref name | ||
| 492 | - Optional capabilities (first line only, after \0) | ||
| 493 | |||
| 494 | ```rust | ||
| 495 | pub struct RefUpdate { | ||
| 496 | pub old_sha: String, | ||
| 497 | pub new_sha: String, | ||
| 498 | pub ref_name: String, | ||
| 499 | } | ||
| 500 | |||
| 501 | pub fn parse_ref_updates(body: &[u8]) -> Result<Vec<RefUpdate>> { | ||
| 502 | // Parse pkt-line format | ||
| 503 | // Extract ref updates | ||
| 504 | // Return structured data | ||
| 505 | } | ||
| 506 | ``` | 308 | ``` |
| 507 | 309 | ||
| 508 | ### 2. Maintainer Recursion | 310 | ### State Event Flow |
| 509 | |||
| 510 | The maintainer resolution must handle cycles and correctly build the set: | ||
| 511 | 311 | ||
| 512 | ```rust | ||
| 513 | fn get_maintainers_recursive( | ||
| 514 | events: &[Event], | ||
| 515 | pubkey: &str, | ||
| 516 | identifier: &str, | ||
| 517 | visited: &mut HashSet<String>, | ||
| 518 | ) -> HashSet<String> { | ||
| 519 | if visited.contains(pubkey) { | ||
| 520 | return HashSet::new(); | ||
| 521 | } | ||
| 522 | |||
| 523 | visited.insert(pubkey.to_string()); | ||
| 524 | |||
| 525 | let announcement = find_announcement(events, pubkey, identifier); | ||
| 526 | if announcement.is_none() { | ||
| 527 | return HashSet::new(); | ||
| 528 | } | ||
| 529 | |||
| 530 | let repo = parse_repository(announcement.unwrap()); | ||
| 531 | |||
| 532 | for maintainer in repo.maintainers { | ||
| 533 | get_maintainers_recursive(events, &maintainer, identifier, visited); | ||
| 534 | } | ||
| 535 | |||
| 536 | visited.clone() | ||
| 537 | } | ||
| 538 | ``` | 312 | ``` |
| 539 | 313 | 1. Nostr Client → EVENT (Kind 30618) | |
| 540 | ### 3. State Event Validation | 314 | ↓ |
| 541 | 315 | 2. Nostr relay receives event | |
| 542 | ```rust | 316 | ↓ |
| 543 | fn validate_state_ref( | 317 | 3. Nip34WritePolicy::admit_event() |
| 544 | state: &RepositoryState, | 318 | ├─ Check author is in maintainer set |
| 545 | ref_update: &RefUpdate, | 319 | ├─ Validate state structure |
| 546 | ) -> Result<()> { | 320 | └─ Accept or reject |
| 547 | if ref_update.ref_name.starts_with("refs/heads/") { | 321 | ↓ |
| 548 | let branch_name = &ref_update.ref_name[11..]; | 322 | 4. If ACCEPTED and is latest state: |
| 549 | if let Some(commit) = state.branches.get(branch_name) { | 323 | ├─ Align repository refs to match state |
| 550 | if commit == &ref_update.new_sha { | 324 | ├─ Create/update/delete refs as needed |
| 551 | return Ok(()); | 325 | └─ Set HEAD if commit available |
| 552 | } | ||
| 553 | return Err(Error::StateMismatch { | ||
| 554 | ref_name: ref_update.ref_name.clone(), | ||
| 555 | expected: commit.clone(), | ||
| 556 | got: ref_update.new_sha.clone(), | ||
| 557 | }); | ||
| 558 | } | ||
| 559 | return Err(Error::RefNotInState(ref_update.ref_name.clone())); | ||
| 560 | } | ||
| 561 | |||
| 562 | if ref_update.ref_name.starts_with("refs/tags/") { | ||
| 563 | let tag_name = &ref_update.ref_name[10..]; | ||
| 564 | if let Some(commit) = state.tags.get(tag_name) { | ||
| 565 | if commit == &ref_update.new_sha { | ||
| 566 | return Ok(()); | ||
| 567 | } | ||
| 568 | return Err(Error::StateMismatch { | ||
| 569 | ref_name: ref_update.ref_name.clone(), | ||
| 570 | expected: commit.clone(), | ||
| 571 | got: ref_update.new_sha.clone(), | ||
| 572 | }); | ||
| 573 | } | ||
| 574 | return Err(Error::RefNotInState(ref_update.ref_name.clone())); | ||
| 575 | } | ||
| 576 | |||
| 577 | Err(Error::InvalidRef(ref_update.ref_name.clone())) | ||
| 578 | } | ||
| 579 | ``` | ||
| 580 | |||
| 581 | ### 4. CORS Support | ||
| 582 | |||
| 583 | As per GRASP-01, we must support CORS: | ||
| 584 | |||
| 585 | ```rust | ||
| 586 | use actix_cors::Cors; | ||
| 587 | |||
| 588 | fn configure_cors() -> Cors { | ||
| 589 | Cors::default() | ||
| 590 | .allow_any_origin() | ||
| 591 | .allowed_methods(vec!["GET", "POST", "OPTIONS"]) | ||
| 592 | .allowed_headers(vec!["Content-Type"]) | ||
| 593 | .max_age(3600) | ||
| 594 | } | ||
| 595 | |||
| 596 | // In main.rs | ||
| 597 | App::new() | ||
| 598 | .wrap(configure_cors()) | ||
| 599 | .configure(git_routes) | ||
| 600 | .configure(nostr_routes) | ||
| 601 | ``` | 326 | ``` |
| 602 | 327 | ||
| 603 | ## Testing Strategy | 328 | ## Testing Strategy |
| 604 | 329 | ||
| 605 | See [TEST_STRATEGY.md](TEST_STRATEGY.md) for comprehensive testing documentation, including: | 330 | See [test-strategy.md](../reference/test-strategy.md) for comprehensive testing documentation. |
| 606 | |||
| 607 | - **GRASP Compliance Testing Tool**: Reusable test suite that validates any GRASP implementation against the spec | ||
| 608 | - **Spec-Mirrored Tests**: Test structure matches GRASP protocol documents exactly | ||
| 609 | - **Clear Failure Messages**: Test failures cite exact spec lines (e.g., "GRASP-01:12-13") | ||
| 610 | - **Multiple Test Levels**: Unit, integration, compliance, and end-to-end tests | ||
| 611 | 331 | ||
| 612 | ### Quick Overview | 332 | ### Quick Overview |
| 613 | 333 | ||
| 614 | ```rust | 334 | **Integration Tests** ([`tests/`](tests/)): |
| 615 | // Unit Tests - Individual functions | 335 | - Use [`TestRelay`](tests/common/relay.rs:14) fixture for automatic relay lifecycle |
| 616 | #[test] | 336 | - Each test file in [`tests/`](tests/) covers a GRASP-01 requirement |
| 617 | fn test_parse_ref_updates() { | ||
| 618 | let body = b"0000... 0000... refs/heads/main\0report-status\n"; | ||
| 619 | let updates = parse_ref_updates(body).unwrap(); | ||
| 620 | assert_eq!(updates.len(), 1); | ||
| 621 | assert_eq!(updates[0].ref_name, "refs/heads/main"); | ||
| 622 | } | ||
| 623 | 337 | ||
| 624 | // Integration Tests - Component interaction | 338 | **Audit Tests** ([`grasp-audit/`](grasp-audit/)): |
| 625 | #[tokio::test] | 339 | - Reusable compliance testing for any GRASP implementation |
| 626 | async fn test_full_push_flow() { | 340 | - Spec-mirrored structure in [`grasp-audit/src/specs/grasp01/`](grasp-audit/src/specs/grasp01/) |
| 627 | let app = test_app().await; | ||
| 628 | let (announcement, state) = app.create_repo_with_state() | ||
| 629 | .branch("main", "commit-123") | ||
| 630 | .build() | ||
| 631 | .await; | ||
| 632 | |||
| 633 | let result = app.git_push("main", "commit-123").await; | ||
| 634 | assert!(result.success); | ||
| 635 | } | ||
| 636 | 341 | ||
| 637 | // Compliance Tests - GRASP spec validation | 342 | ```rust |
| 343 | // Example: tests/nip01_compliance.rs | ||
| 638 | #[tokio::test] | 344 | #[tokio::test] |
| 639 | async fn test_grasp_01_compliance() { | 345 | async fn test_nip01_websocket_connection() { |
| 640 | use grasp_compliance_tests::{TestContext, Grasp01Spec}; | 346 | let relay = TestRelay::start().await; |
| 641 | 347 | // Test NIP-01 compliance... | |
| 642 | let ctx = TestContext::builder() | 348 | relay.stop().await; |
| 643 | .base_url(&server.url()) | ||
| 644 | .build(); | ||
| 645 | |||
| 646 | let results = Grasp01Spec::test_compliance(&ctx).await; | ||
| 647 | assert!(results.all_passed(), "{}", results.report()); | ||
| 648 | } | 349 | } |
| 649 | ``` | 350 | ``` |
| 650 | 351 | ||
| 651 | The compliance testing tool is designed as a **standalone crate** that can be: | ||
| 652 | - Used by ngit-grasp for self-validation | ||
| 653 | - Published for other GRASP implementations to use | ||
| 654 | - Updated as new GRASP specs are released | ||
| 655 | - Run in CI/CD for continuous compliance verification | ||
| 656 | |||
| 657 | ## Performance Considerations | 352 | ## Performance Considerations |
| 658 | 353 | ||
| 659 | ### 1. Async All The Way | 354 | ### 1. Async All The Way |
| 660 | 355 | ||
| 661 | - Use `tokio` for all I/O | 356 | - Use `tokio` for all I/O |
| 662 | - Non-blocking Git subprocess spawning | 357 | - Non-blocking Git subprocess spawning via [`GitSubprocess`](src/git/subprocess.rs) |
| 663 | - Stream large pack files without buffering | 358 | - Stream large pack files without buffering |
| 664 | 359 | ||
| 665 | ### 2. Connection Pooling | 360 | ### 2. Shared Database |
| 666 | |||
| 667 | - Reuse Nostr relay connections | ||
| 668 | - Connection pool for internal relay queries | ||
| 669 | |||
| 670 | ### 3. Caching | ||
| 671 | 361 | ||
| 672 | - Cache parsed state events (with TTL) | 362 | - Single database instance shared between relay and Git handlers |
| 673 | - Cache maintainer sets | 363 | - Direct queries for push authorization (no WebSocket round-trip) |
| 674 | - Invalidate on new state events | ||
| 675 | 364 | ||
| 676 | ```rust | 365 | ### 3. Write Policy Caching |
| 677 | pub struct StateCache { | ||
| 678 | cache: Arc<RwLock<HashMap<String, CachedState>>>, | ||
| 679 | } | ||
| 680 | |||
| 681 | struct CachedState { | ||
| 682 | state: RepositoryState, | ||
| 683 | maintainers: Vec<String>, | ||
| 684 | timestamp: Instant, | ||
| 685 | } | ||
| 686 | 366 | ||
| 687 | impl StateCache { | 367 | - Maintainer sets computed once per event validation |
| 688 | pub async fn get_or_fetch( | 368 | - State lookups use database indexes |
| 689 | &self, | ||
| 690 | identifier: &str, | ||
| 691 | fetcher: impl Future<Output = Result<(RepositoryState, Vec<String>)>>, | ||
| 692 | ) -> Result<(RepositoryState, Vec<String>)> { | ||
| 693 | // Check cache | ||
| 694 | // Return if fresh | ||
| 695 | // Otherwise fetch and cache | ||
| 696 | } | ||
| 697 | } | ||
| 698 | ``` | ||
| 699 | 369 | ||
| 700 | ## Future Extensions | 370 | ## Future Extensions |
| 701 | 371 | ||
| 702 | ### GRASP-02: Proactive Sync | 372 | ### GRASP-02: Proactive Sync |
| 703 | 373 | ||
| 704 | Add background tasks: | 374 | See [grasp-02-proactive-sync.md](grasp-02-proactive-sync.md) for detailed design. |
| 705 | |||
| 706 | ```rust | ||
| 707 | pub struct ProactiveSyncTask { | ||
| 708 | relay_client: Client, | ||
| 709 | git_manager: RepositoryManager, | ||
| 710 | } | ||
| 711 | |||
| 712 | impl ProactiveSyncTask { | ||
| 713 | pub async fn run(&self) { | ||
| 714 | loop { | ||
| 715 | tokio::time::sleep(Duration::from_secs(3600)).await; | ||
| 716 | |||
| 717 | // Fetch all announcements from our relay | ||
| 718 | let announcements = self.fetch_announcements().await; | ||
| 719 | |||
| 720 | for ann in announcements { | ||
| 721 | // Sync events from listed relays | ||
| 722 | self.sync_events(&ann).await; | ||
| 723 | |||
| 724 | // Sync git data from listed clones | ||
| 725 | self.sync_git_data(&ann).await; | ||
| 726 | |||
| 727 | // Fetch PR data | ||
| 728 | self.sync_pr_data(&ann).await; | ||
| 729 | } | ||
| 730 | } | ||
| 731 | } | ||
| 732 | } | ||
| 733 | ``` | ||
| 734 | 375 | ||
| 735 | ### GRASP-05: Archive | 376 | ### GRASP-05: Archive |
| 736 | 377 | ||
| 737 | Relax the policy: | 378 | Relax the write policy to accept all repository announcements regardless of clone/relays tags. |
| 738 | |||
| 739 | ```rust | ||
| 740 | pub struct ArchiveAnnouncementPolicy; | ||
| 741 | |||
| 742 | impl WritePolicy for ArchiveAnnouncementPolicy { | ||
| 743 | fn admit_event(&self, event: &Event, _addr: &SocketAddr) | ||
| 744 | -> BoxFuture<PolicyResult> | ||
| 745 | { | ||
| 746 | // Accept all repository announcements | ||
| 747 | // Don't check clone/relays tags | ||
| 748 | PolicyResult::Accept | ||
| 749 | } | ||
| 750 | } | ||
| 751 | ``` | ||
| 752 | 379 | ||
| 753 | ## Deployment | 380 | ## Deployment |
| 754 | 381 | ||
| @@ -756,7 +383,7 @@ impl WritePolicy for ArchiveAnnouncementPolicy { | |||
| 756 | 383 | ||
| 757 | ```bash | 384 | ```bash |
| 758 | cargo build --release | 385 | cargo build --release |
| 759 | ./target/release/ngit-grasp | 386 | ./target/release/ngit-grasp --domain example.com --owner-npub npub1... |
| 760 | ``` | 387 | ``` |
| 761 | 388 | ||
| 762 | ### Docker | 389 | ### Docker |
| @@ -799,10 +426,17 @@ WantedBy=multi-user.target | |||
| 799 | 2. **Path Traversal**: Prevent directory traversal in repository paths | 426 | 2. **Path Traversal**: Prevent directory traversal in repository paths |
| 800 | 3. **DoS Protection**: Rate limiting on both HTTP and WebSocket | 427 | 3. **DoS Protection**: Rate limiting on both HTTP and WebSocket |
| 801 | 4. **Resource Limits**: Limit pack file sizes, event sizes | 428 | 4. **Resource Limits**: Limit pack file sizes, event sizes |
| 802 | 5. **Nostr Event Validation**: Strict signature verification | 429 | 5. **Nostr Event Validation**: Strict signature verification (handled by nostr-relay-builder) |
| 803 | 430 | ||
| 804 | ## Conclusion | 431 | ## Conclusion |
| 805 | 432 | ||
| 806 | The inline authorization approach provides a cleaner, more maintainable architecture than hook-based authorization while maintaining full GRASP-01 compliance. The Rust ecosystem provides excellent libraries for both Git and Nostr protocols, enabling a high-performance, type-safe implementation. | 433 | The inline authorization approach provides a cleaner, more maintainable architecture than hook-based authorization while maintaining full GRASP-01 compliance. Using Hyper for the HTTP layer gives us complete control over request handling, WebSocket upgrades, and CORS headers. |
| 807 | 434 | ||
| 808 | The key insight is that we don't need to rely on Git's hook mechanism when we have full control over the HTTP layer that Git operates through. By intercepting at the HTTP handler level, we gain better error handling, easier testing, and tighter integration between the Git and Nostr components. | 435 | The key insight is that we don't need to rely on Git's hook mechanism when we have full control over the HTTP layer that Git operates through. By intercepting at the HTTP handler level, we gain better error handling, easier testing, and tighter integration between the Git and Nostr components. |
| 436 | |||
| 437 | ## Related Documentation | ||
| 438 | |||
| 439 | - [Inline Authorization Explanation](inline-authorization.md) - Why we chose this approach | ||
| 440 | - [GRASP-02 Proactive Sync](grasp-02-proactive-sync.md) - Future work design | ||
| 441 | - [Test Strategy](../reference/test-strategy.md) - Comprehensive testing documentation | ||
| 442 | - [GRASP-01 Implementation Learnings](../learnings/grasp-01-implementation.md) - Patterns and lessons learned \ No newline at end of file | ||