diff options
| author | DanConwayDev <DanConwayDev@protonmail.com> | 2025-11-04 09:31:57 +0000 |
|---|---|---|
| committer | DanConwayDev <DanConwayDev@protonmail.com> | 2025-11-04 09:31:57 +0000 |
| commit | 22557f15d6a7b77f72d4597fc05aa06346495a33 (patch) | |
| tree | e31e0cdecfc4cb1e28246227a7ef295b71687b09 /docs/learnings/nix-flakes.md | |
| parent | b3031800cd95601c2d9cd2d24034364d1496b073 (diff) | |
docs: major cleanup and reorganization
- Archive 30 completed session documents to docs/archive/
- Extract learnings to docs/learnings/ (nix-flakes, nostr-sdk, grasp-audit)
- Create CURRENT_STATUS.md as single source of truth
- Create AGENTS.md with documentation guidelines
- Create docs/archive/README.md for archive organization
- Clean root directory: 32 files → 4 files
Root directory now contains only:
- README.md (project overview)
- AGENTS.md (documentation guidelines)
- CURRENT_STATUS.md (current state)
- CLEANUP_SUMMARY.md (cleanup report)
All historical documents preserved in docs/archive/ with proper dating.
All reusable knowledge extracted to docs/learnings/.
Benefits:
- Easy to find current information
- Clear document lifecycle
- No more documentation sprawl
- Learnings are accessible and reusable
- Better onboarding for new developers/agents
File counts:
- Root: 4 (was 32)
- Permanent docs: 7
- Learnings: 3 (new)
- Archive: 32 (new)
- Total: 49 well-organized docs
Diffstat (limited to 'docs/learnings/nix-flakes.md')
| -rw-r--r-- | docs/learnings/nix-flakes.md | 423 |
1 files changed, 423 insertions, 0 deletions
diff --git a/docs/learnings/nix-flakes.md b/docs/learnings/nix-flakes.md new file mode 100644 index 0000000..6876647 --- /dev/null +++ b/docs/learnings/nix-flakes.md | |||
| @@ -0,0 +1,423 @@ | |||
| 1 | # Nix Flakes - Learnings and Gotchas | ||
| 2 | |||
| 3 | **Purpose:** Document Nix flake patterns, gotchas, and best practices learned during ngit-grasp development | ||
| 4 | **Last Updated:** November 4, 2025 | ||
| 5 | |||
| 6 | --- | ||
| 7 | |||
| 8 | ## Critical Gotchas | ||
| 9 | |||
| 10 | ### Always Use `nix develop`, Not `nix-shell` | ||
| 11 | |||
| 12 | **Problem:** We use `flake.nix`, not `shell.nix`. Using `nix-shell` will fail or use the wrong environment. | ||
| 13 | |||
| 14 | ```bash | ||
| 15 | # ✅ Correct - for flake.nix | ||
| 16 | cd grasp-audit | ||
| 17 | nix develop | ||
| 18 | nix develop -c cargo build | ||
| 19 | |||
| 20 | # ❌ Wrong - for shell.nix (we don't use this) | ||
| 21 | nix-shell | ||
| 22 | nix-shell --run "cargo build" | ||
| 23 | ``` | ||
| 24 | |||
| 25 | **Why:** | ||
| 26 | - `nix-shell` looks for `shell.nix` or `default.nix` | ||
| 27 | - `nix develop` looks for `flake.nix` | ||
| 28 | - We migrated from `shell.nix` to `flake.nix` on November 4, 2025 | ||
| 29 | |||
| 30 | **Related:** See `docs/archive/2025-11-04-flake-migration.md` | ||
| 31 | |||
| 32 | --- | ||
| 33 | |||
| 34 | ## Flake Structure | ||
| 35 | |||
| 36 | ### Our Standard Flake Pattern | ||
| 37 | |||
| 38 | ```nix | ||
| 39 | { | ||
| 40 | inputs = { | ||
| 41 | nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; | ||
| 42 | rust-overlay.url = "github:oxalica/rust-overlay"; | ||
| 43 | flake-utils.url = "github:numtide/flake-utils"; | ||
| 44 | }; | ||
| 45 | |||
| 46 | outputs = { nixpkgs, rust-overlay, flake-utils, ... }: | ||
| 47 | flake-utils.lib.eachDefaultSystem (system: | ||
| 48 | let | ||
| 49 | overlays = [ (import rust-overlay) ]; | ||
| 50 | pkgs = import nixpkgs { inherit system overlays; }; | ||
| 51 | manifest = pkgs.lib.importTOML ./Cargo.toml; | ||
| 52 | in with pkgs; { | ||
| 53 | # Development shell | ||
| 54 | devShells.default = mkShell { | ||
| 55 | nativeBuildInputs = [ | ||
| 56 | rust-bin.stable.latest.default | ||
| 57 | pkg-config | ||
| 58 | gitlint | ||
| 59 | ]; | ||
| 60 | buildInputs = [ | ||
| 61 | openssl | ||
| 62 | ]; | ||
| 63 | shellHook = '' | ||
| 64 | echo "🦀 Development environment loaded" | ||
| 65 | export RUST_SRC_PATH=${pkgs.rustPlatform.rustLibSrc} | ||
| 66 | ''; | ||
| 67 | }; | ||
| 68 | |||
| 69 | # Package output | ||
| 70 | packages.default = pkgs.rustPlatform.buildRustPackage { | ||
| 71 | pname = manifest.package.name; | ||
| 72 | version = manifest.package.version; | ||
| 73 | src = ./.; | ||
| 74 | cargoLock = { lockFile = ./Cargo.lock; }; | ||
| 75 | buildInputs = [ openssl ]; | ||
| 76 | nativeBuildInputs = [ pkg-config ]; | ||
| 77 | doCheck = false; # Run tests separately | ||
| 78 | }; | ||
| 79 | }); | ||
| 80 | } | ||
| 81 | ``` | ||
| 82 | |||
| 83 | ### Key Components | ||
| 84 | |||
| 85 | 1. **rust-overlay**: Provides latest stable Rust toolchain | ||
| 86 | 2. **flake-utils**: Cross-platform support helper | ||
| 87 | 3. **manifest**: Auto-read version from Cargo.toml | ||
| 88 | 4. **devShells.default**: Development environment | ||
| 89 | 5. **packages.default**: Buildable package | ||
| 90 | |||
| 91 | --- | ||
| 92 | |||
| 93 | ## Common Flake Commands | ||
| 94 | |||
| 95 | ### Essential Commands | ||
| 96 | |||
| 97 | ```bash | ||
| 98 | # Enter development shell | ||
| 99 | nix develop | ||
| 100 | |||
| 101 | # Run command in dev shell (one-off) | ||
| 102 | nix develop -c cargo build | ||
| 103 | |||
| 104 | # Show flake outputs | ||
| 105 | nix flake show | ||
| 106 | |||
| 107 | # Check flake validity | ||
| 108 | nix flake check | ||
| 109 | |||
| 110 | # Update flake inputs (like updating dependencies) | ||
| 111 | nix flake update | ||
| 112 | |||
| 113 | # Build the package directly | ||
| 114 | nix build | ||
| 115 | |||
| 116 | # Run without installing | ||
| 117 | nix run | ||
| 118 | |||
| 119 | # Show flake metadata | ||
| 120 | nix flake metadata | ||
| 121 | ``` | ||
| 122 | |||
| 123 | ### Debugging Commands | ||
| 124 | |||
| 125 | ```bash | ||
| 126 | # Show detailed evaluation trace | ||
| 127 | nix develop --show-trace | ||
| 128 | |||
| 129 | # Print flake evaluation | ||
| 130 | nix eval .#devShells.x86_64-linux.default | ||
| 131 | |||
| 132 | # Check what's in the store | ||
| 133 | nix path-info .#packages.x86_64-linux.default | ||
| 134 | ``` | ||
| 135 | |||
| 136 | --- | ||
| 137 | |||
| 138 | ## Subproject Flakes | ||
| 139 | |||
| 140 | ### grasp-audit Has Its Own Flake | ||
| 141 | |||
| 142 | **Important:** `grasp-audit/` is a subproject with its own `flake.nix` and `Cargo.toml`. | ||
| 143 | |||
| 144 | ```bash | ||
| 145 | # ✅ Correct - enter grasp-audit environment | ||
| 146 | cd grasp-audit | ||
| 147 | nix develop | ||
| 148 | cargo build | ||
| 149 | |||
| 150 | # ❌ Wrong - can't build from root | ||
| 151 | cd ngit-grasp | ||
| 152 | cargo build # This won't find grasp-audit dependencies | ||
| 153 | ``` | ||
| 154 | |||
| 155 | **Why:** | ||
| 156 | - Each Rust workspace needs its own Nix environment | ||
| 157 | - Dependencies are project-specific | ||
| 158 | - Flake inputs are locked per-project | ||
| 159 | |||
| 160 | --- | ||
| 161 | |||
| 162 | ## Migration from shell.nix to flake.nix | ||
| 163 | |||
| 164 | ### What Changed | ||
| 165 | |||
| 166 | **Before (shell.nix):** | ||
| 167 | ```nix | ||
| 168 | { pkgs ? import <nixpkgs> {} }: | ||
| 169 | |||
| 170 | pkgs.mkShell { | ||
| 171 | buildInputs = with pkgs; [ | ||
| 172 | rustc | ||
| 173 | cargo | ||
| 174 | openssl | ||
| 175 | pkg-config | ||
| 176 | ]; | ||
| 177 | } | ||
| 178 | ``` | ||
| 179 | |||
| 180 | **After (flake.nix):** | ||
| 181 | - Locked inputs (reproducible) | ||
| 182 | - Multi-output (dev shell + package) | ||
| 183 | - Cross-platform by default | ||
| 184 | - Better tooling integration | ||
| 185 | |||
| 186 | ### Migration Steps | ||
| 187 | |||
| 188 | 1. Create `flake.nix` with standard structure | ||
| 189 | 2. Run `nix flake check` to validate | ||
| 190 | 3. Update all documentation: `nix-shell` → `nix develop` | ||
| 191 | 4. Test that build works: `nix develop -c cargo build` | ||
| 192 | 5. Remove `shell.nix` | ||
| 193 | 6. Commit changes | ||
| 194 | |||
| 195 | **Reference:** See `docs/archive/2025-11-04-flake-migration.md` | ||
| 196 | |||
| 197 | --- | ||
| 198 | |||
| 199 | ## Benefits of Flakes | ||
| 200 | |||
| 201 | ### Reproducibility | ||
| 202 | |||
| 203 | **Locked inputs** ensure everyone gets the same environment: | ||
| 204 | |||
| 205 | ```bash | ||
| 206 | # flake.lock contains exact commits | ||
| 207 | $ cat flake.lock | ||
| 208 | { | ||
| 209 | "nodes": { | ||
| 210 | "nixpkgs": { | ||
| 211 | "locked": { | ||
| 212 | "lastModified": 1698611440, | ||
| 213 | "narHash": "sha256-jPjHjrerhYDy3q9+s5EAsuhyhuknNfowY6yt6pjn9pc=", | ||
| 214 | "rev": "23e89e0c8c5e2d9cf5b5e7c3e8e8e8e8e8e8e8e8" | ||
| 215 | } | ||
| 216 | } | ||
| 217 | } | ||
| 218 | } | ||
| 219 | ``` | ||
| 220 | |||
| 221 | Everyone running `nix develop` gets **exactly** this version of nixpkgs. | ||
| 222 | |||
| 223 | ### Multi-Output | ||
| 224 | |||
| 225 | Single flake provides: | ||
| 226 | - **devShells.default**: Development environment | ||
| 227 | - **packages.default**: Buildable package | ||
| 228 | - **apps.default**: Runnable application (optional) | ||
| 229 | |||
| 230 | ### Composability | ||
| 231 | |||
| 232 | Flakes can use other flakes as inputs: | ||
| 233 | |||
| 234 | ```nix | ||
| 235 | { | ||
| 236 | inputs = { | ||
| 237 | grasp-audit.url = "path:./grasp-audit"; | ||
| 238 | }; | ||
| 239 | } | ||
| 240 | ``` | ||
| 241 | |||
| 242 | --- | ||
| 243 | |||
| 244 | ## Common Issues | ||
| 245 | |||
| 246 | ### Issue: "error: getting status of '/nix/store/...': No such file or directory" | ||
| 247 | |||
| 248 | **Cause:** Flake inputs need to be updated or fetched | ||
| 249 | |||
| 250 | **Solution:** | ||
| 251 | ```bash | ||
| 252 | nix flake update | ||
| 253 | nix develop | ||
| 254 | ``` | ||
| 255 | |||
| 256 | ### Issue: "error: experimental feature 'nix-command' is not enabled" | ||
| 257 | |||
| 258 | **Cause:** Nix flakes are experimental and need to be enabled | ||
| 259 | |||
| 260 | **Solution:** | ||
| 261 | Add to `~/.config/nix/nix.conf`: | ||
| 262 | ``` | ||
| 263 | experimental-features = nix-command flakes | ||
| 264 | ``` | ||
| 265 | |||
| 266 | ### Issue: Changes to flake.nix not taking effect | ||
| 267 | |||
| 268 | **Cause:** Flake evaluation is cached | ||
| 269 | |||
| 270 | **Solution:** | ||
| 271 | ```bash | ||
| 272 | # Clear evaluation cache | ||
| 273 | nix flake update | ||
| 274 | # Or force re-evaluation | ||
| 275 | nix develop --refresh | ||
| 276 | ``` | ||
| 277 | |||
| 278 | ### Issue: "error: cannot find flake 'flake:self' in the flake registries" | ||
| 279 | |||
| 280 | **Cause:** Not in a git repository or flake.nix not committed | ||
| 281 | |||
| 282 | **Solution:** | ||
| 283 | ```bash | ||
| 284 | git add flake.nix flake.lock | ||
| 285 | git commit -m "Add flake" | ||
| 286 | ``` | ||
| 287 | |||
| 288 | **Note:** Flakes require git. Uncommitted files are ignored by default. | ||
| 289 | |||
| 290 | --- | ||
| 291 | |||
| 292 | ## Best Practices | ||
| 293 | |||
| 294 | ### 1. Always Commit flake.lock | ||
| 295 | |||
| 296 | ```bash | ||
| 297 | git add flake.lock | ||
| 298 | git commit -m "Update flake inputs" | ||
| 299 | ``` | ||
| 300 | |||
| 301 | **Why:** Ensures reproducibility across machines and CI/CD | ||
| 302 | |||
| 303 | ### 2. Use Specific Rust Versions When Needed | ||
| 304 | |||
| 305 | ```nix | ||
| 306 | # Latest stable (default) | ||
| 307 | rust-bin.stable.latest.default | ||
| 308 | |||
| 309 | # Specific version | ||
| 310 | rust-bin.stable."1.75.0".default | ||
| 311 | |||
| 312 | # Nightly | ||
| 313 | rust-bin.nightly."2024-01-01".default | ||
| 314 | ``` | ||
| 315 | |||
| 316 | ### 3. Include Helpful Shell Hooks | ||
| 317 | |||
| 318 | ```nix | ||
| 319 | shellHook = '' | ||
| 320 | echo "🦀 GRASP Audit development environment" | ||
| 321 | echo "" | ||
| 322 | echo "Common commands:" | ||
| 323 | echo " cargo build - Build project" | ||
| 324 | echo " cargo test - Run tests" | ||
| 325 | echo " cargo run - Run binary" | ||
| 326 | echo "" | ||
| 327 | export RUST_SRC_PATH=${pkgs.rustPlatform.rustLibSrc} | ||
| 328 | ''; | ||
| 329 | ``` | ||
| 330 | |||
| 331 | ### 4. Separate Build and Runtime Dependencies | ||
| 332 | |||
| 333 | ```nix | ||
| 334 | # Build-time only | ||
| 335 | nativeBuildInputs = [ | ||
| 336 | pkg-config | ||
| 337 | rustc | ||
| 338 | cargo | ||
| 339 | ]; | ||
| 340 | |||
| 341 | # Runtime needed | ||
| 342 | buildInputs = [ | ||
| 343 | openssl | ||
| 344 | ]; | ||
| 345 | ``` | ||
| 346 | |||
| 347 | ### 5. Disable Tests in Package Build | ||
| 348 | |||
| 349 | ```nix | ||
| 350 | packages.default = pkgs.rustPlatform.buildRustPackage { | ||
| 351 | # ... | ||
| 352 | doCheck = false; # Run tests separately with cargo test | ||
| 353 | }; | ||
| 354 | ``` | ||
| 355 | |||
| 356 | **Why:** Faster builds, tests run via `cargo test` in dev shell | ||
| 357 | |||
| 358 | --- | ||
| 359 | |||
| 360 | ## Workflow Examples | ||
| 361 | |||
| 362 | ### Daily Development | ||
| 363 | |||
| 364 | ```bash | ||
| 365 | # Start work | ||
| 366 | cd grasp-audit | ||
| 367 | nix develop | ||
| 368 | |||
| 369 | # Inside nix shell | ||
| 370 | cargo build | ||
| 371 | cargo test | ||
| 372 | cargo run -- --help | ||
| 373 | |||
| 374 | # Exit shell | ||
| 375 | exit | ||
| 376 | ``` | ||
| 377 | |||
| 378 | ### CI/CD | ||
| 379 | |||
| 380 | ```bash | ||
| 381 | # One-off commands (no interactive shell) | ||
| 382 | nix develop -c cargo build | ||
| 383 | nix develop -c cargo test --lib | ||
| 384 | nix develop -c cargo test -- --ignored | ||
| 385 | ``` | ||
| 386 | |||
| 387 | ### Building Release | ||
| 388 | |||
| 389 | ```bash | ||
| 390 | # Build package directly | ||
| 391 | nix build | ||
| 392 | |||
| 393 | # Result is in ./result/bin/ | ||
| 394 | ./result/bin/grasp-audit --version | ||
| 395 | ``` | ||
| 396 | |||
| 397 | --- | ||
| 398 | |||
| 399 | ## References | ||
| 400 | |||
| 401 | - **Nix Flakes Manual**: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html | ||
| 402 | - **rust-overlay**: https://github.com/oxalica/rust-overlay | ||
| 403 | - **flake-utils**: https://github.com/numtide/flake-utils | ||
| 404 | - **Our Migration**: `docs/archive/2025-11-04-flake-migration.md` | ||
| 405 | |||
| 406 | --- | ||
| 407 | |||
| 408 | ## Quick Reference | ||
| 409 | |||
| 410 | | Task | Command | | ||
| 411 | |------|---------| | ||
| 412 | | Enter dev shell | `nix develop` | | ||
| 413 | | Run one command | `nix develop -c <command>` | | ||
| 414 | | Show outputs | `nix flake show` | | ||
| 415 | | Validate flake | `nix flake check` | | ||
| 416 | | Update inputs | `nix flake update` | | ||
| 417 | | Build package | `nix build` | | ||
| 418 | | Run package | `nix run` | | ||
| 419 | |||
| 420 | --- | ||
| 421 | |||
| 422 | *Last updated: November 4, 2025* | ||
| 423 | *Status: Living document - update as we learn more* | ||