upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/learnings
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2025-11-04 09:31:57 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2025-11-04 09:31:57 +0000
commit22557f15d6a7b77f72d4597fc05aa06346495a33 (patch)
treee31e0cdecfc4cb1e28246227a7ef295b71687b09 /docs/learnings
parentb3031800cd95601c2d9cd2d24034364d1496b073 (diff)
docs: major cleanup and reorganization
- Archive 30 completed session documents to docs/archive/ - Extract learnings to docs/learnings/ (nix-flakes, nostr-sdk, grasp-audit) - Create CURRENT_STATUS.md as single source of truth - Create AGENTS.md with documentation guidelines - Create docs/archive/README.md for archive organization - Clean root directory: 32 files → 4 files Root directory now contains only: - README.md (project overview) - AGENTS.md (documentation guidelines) - CURRENT_STATUS.md (current state) - CLEANUP_SUMMARY.md (cleanup report) All historical documents preserved in docs/archive/ with proper dating. All reusable knowledge extracted to docs/learnings/. Benefits: - Easy to find current information - Clear document lifecycle - No more documentation sprawl - Learnings are accessible and reusable - Better onboarding for new developers/agents File counts: - Root: 4 (was 32) - Permanent docs: 7 - Learnings: 3 (new) - Archive: 32 (new) - Total: 49 well-organized docs
Diffstat (limited to 'docs/learnings')
-rw-r--r--docs/learnings/grasp-audit.md498
-rw-r--r--docs/learnings/nix-flakes.md423
-rw-r--r--docs/learnings/nostr-sdk.md577
3 files changed, 1498 insertions, 0 deletions
diff --git a/docs/learnings/grasp-audit.md b/docs/learnings/grasp-audit.md
new file mode 100644
index 0000000..531ebda
--- /dev/null
+++ b/docs/learnings/grasp-audit.md
@@ -0,0 +1,498 @@
1# GRASP Audit Tool - Patterns and Learnings
2
3**Purpose:** Document grasp-audit architecture, patterns, and lessons learned
4**Last Updated:** November 4, 2025
5
6---
7
8## Overview
9
10`grasp-audit` is a compliance testing tool for GRASP (Git Relays Authorized via Signed-Nostr Proofs) protocol implementations. It tests both Nostr relay compliance (NIP-01) and GRASP-specific functionality.
11
12---
13
14## Architecture Decisions
15
16### Separate Crate Strategy
17
18**Decision:** Build `grasp-audit` as a separate crate from `ngit-grasp`
19
20**Why:**
211. **Parallel Development**: Can build tests before implementation
222. **Isolated Testing**: Tests run in isolation (CI/CD safe)
233. **Production Auditing**: Can audit live production services
244. **Reusability**: Other GRASP implementations can use it
25
26**Location:** `grasp-audit/` subdirectory with own `Cargo.toml` and `flake.nix`
27
28---
29
30### Audit Event Tagging Strategy
31
32**Problem:** Test events pollute the relay and need cleanup without deletion events.
33
34**Solution:** Use special tags to mark audit events:
35
36```rust
37// Every audit event includes these tags
38[
39 ["t", "grasp-audit-test-event"], // Marker
40 ["t", "audit-{run-id}"], // Run isolation
41 ["t", "audit-cleanup-after-{timestamp}"] // Cleanup time
42]
43```
44
45**Benefits:**
46- ✅ **Queryable**: Can find all audit events via tag filter
47- ✅ **Isolated**: Each test run has unique run ID
48- ✅ **Self-cleaning**: Cleanup timestamp indicates when to delete
49- ✅ **No deletion events**: Direct database cleanup, no KIND 5 events
50- ✅ **Production safe**: Won't interfere with real events
51
52**Reference:** See `docs/archive/2025-11-04-tag-migration.md`
53
54---
55
56### Standard "t" Tags vs Custom Tags
57
58**Evolution:**
591. **Original**: Custom single-letter tags (`g`, `r`, `c`)
602. **Current**: Standard NIP-01 "t" tags with prefixed values
61
62**Why we changed:**
63- ❌ Custom tags could conflict with other systems
64- ✅ "t" tag is standard for categorization/topics
65- ✅ Multiple "t" tags are expected and supported
66- ✅ Self-documenting values (`audit-{run-id}` vs just `{run-id}`)
67- ✅ Better namespacing with prefixes
68
69**Migration:** Completed November 4, 2025
70
71---
72
73## Code Patterns
74
75### Audit Configuration
76
77```rust
78use grasp_audit::audit::AuditConfig;
79
80// CI mode - isolated test runs
81let config = AuditConfig::ci();
82// Generates UUID run ID: "ci-{uuid}"
83// Cleanup after 1 hour
84
85// Production mode - persistent run ID
86let config = AuditConfig::production("prod-server-1");
87// Uses provided run ID
88// Cleanup after 24 hours
89```
90
91**When to use:**
92- **CI mode**: Automated testing, parallel runs, temporary
93- **Production mode**: Manual audits, monitoring, persistent
94
95---
96
97### Creating Audit Events
98
99```rust
100use grasp_audit::audit::{AuditConfig, AuditEventBuilder};
101use nostr_sdk::prelude::*;
102
103let config = AuditConfig::ci();
104let keys = Keys::generate();
105
106// Create audit event
107let event = AuditEventBuilder::new(&config, Kind::TextNote, "test content")
108 .build(&keys)?;
109
110// Event automatically includes:
111// - Audit marker tag
112// - Run ID tag
113// - Cleanup timestamp tag
114```
115
116---
117
118### Querying Audit Events
119
120```rust
121use grasp_audit::client::AuditClient;
122use grasp_audit::audit::AuditConfig;
123
124let config = AuditConfig::ci();
125let client = AuditClient::new(config, keys);
126
127// Connect to relay
128client.add_relay("ws://localhost:7000").await?;
129client.connect().await;
130
131// Query audit events for this run
132let events = client.query().await?;
133
134// Events are filtered by:
135// - "grasp-audit-test-event" marker
136// - Current run ID
137```
138
139---
140
141### Test Isolation
142
143**Each test run is isolated by unique run ID:**
144
145```rust
146// CI mode generates unique UUID per run
147let config1 = AuditConfig::ci();
148let config2 = AuditConfig::ci();
149
150// config1.run_id != config2.run_id
151// Tests won't interfere with each other
152```
153
154**Benefits:**
155- ✅ Parallel CI/CD runs don't conflict
156- ✅ Can run multiple test suites simultaneously
157- ✅ Easy to identify which run created which events
158- ✅ Cleanup can target specific runs
159
160---
161
162### Cleanup Strategy
163
164**Two-phase cleanup:**
165
1661. **Automatic expiry** via cleanup timestamp tag
1672. **Manual cleanup** by querying and deleting
168
169```rust
170// Events include cleanup timestamp
171["t", "audit-cleanup-after-1730707200"]
172
173// Cleanup process:
174// 1. Query events with expired cleanup timestamp
175// 2. Delete from database directly (no KIND 5)
176// 3. Avoid deletion event pollution
177```
178
179**Implementation:** To be built in relay (not in audit tool)
180
181---
182
183## Testing Strategy
184
185### Test Organization
186
187```
188grasp-audit/src/specs/
189├── nip01_smoke.rs # NIP-01 basic functionality
190├── grasp_01_relay.rs # GRASP-01 relay requirements (planned)
191└── mod.rs # Test suite registry
192```
193
194### Unit vs Integration Tests
195
196**Unit Tests** (no relay required):
197```rust
198#[cfg(test)]
199mod tests {
200 #[test]
201 fn test_audit_config() {
202 let config = AuditConfig::ci();
203 assert!(config.run_id.starts_with("ci-"));
204 }
205}
206```
207
208**Integration Tests** (relay required):
209```rust
210#[cfg(test)]
211mod tests {
212 #[tokio::test]
213 #[ignore] // Requires relay
214 async fn test_smoke_tests_against_relay() {
215 // Test against real relay
216 }
217}
218```
219
220**Running tests:**
221```bash
222# Unit tests (fast, no dependencies)
223cargo test --lib
224
225# Integration tests (requires relay)
226docker run --rm -p 7000:7000 scsibug/nostr-rs-relay
227cargo test -- --ignored
228```
229
230---
231
232### Test Result Reporting
233
234```rust
235use grasp_audit::result::AuditResult;
236
237// Run tests
238let results = vec![
239 AuditResult::pass("websocket_connection", "Connected successfully"),
240 AuditResult::fail("invalid_event", "Expected rejection, got acceptance"),
241];
242
243// Report
244for result in &results {
245 println!("{}", result);
246}
247
248// Summary
249let passed = results.iter().filter(|r| r.is_pass()).count();
250let total = results.len();
251println!("Results: {}/{} passed ({:.1}%)",
252 passed, total, (passed as f64 / total as f64) * 100.0);
253```
254
255---
256
257## CLI Design
258
259### Command Structure
260
261```bash
262grasp-audit audit [OPTIONS]
263
264Options:
265 --relay <URL> Relay to test (required)
266 --mode <MODE> ci or production (default: ci)
267 --run-id <ID> Custom run ID (production mode only)
268 --spec <SPEC> Test spec to run (default: all)
269 --verbose Detailed output
270```
271
272### Usage Examples
273
274```bash
275# CI mode - quick smoke test
276grasp-audit audit \
277 --relay ws://localhost:7000 \
278 --mode ci \
279 --spec nip01-smoke
280
281# Production mode - full compliance audit
282grasp-audit audit \
283 --relay wss://relay.example.com \
284 --mode production \
285 --run-id "audit-2025-11-04" \
286 --verbose
287
288# Test all specs
289grasp-audit audit --relay ws://localhost:7000
290```
291
292---
293
294## Lessons Learned
295
296### 1. Tag Migration is Breaking
297
298**Lesson:** Changing tag structure breaks event queries.
299
300**Impact:** Events created with old tags won't be found by new queries.
301
302**Mitigation:**
303- ✅ Accept breaking changes in alpha stage
304- ✅ Document migration clearly
305- ✅ Old events auto-expire via cleanup
306- ✅ No production deployments affected
307
308**Reference:** `docs/archive/2025-11-04-tag-migration.md`
309
310---
311
312### 2. Test Data Lifecycle Matters
313
314**Lesson:** Test events accumulate and pollute relay.
315
316**Solution:** Built-in cleanup strategy from day one.
317
318**Implementation:**
319- Every event has cleanup timestamp
320- Relay can cleanup expired events
321- No deletion event pollution (direct DB cleanup)
322
323---
324
325### 3. Isolation Enables Parallel Testing
326
327**Lesson:** Unique run IDs enable parallel test execution.
328
329**Benefit:** CI/CD can run multiple test suites simultaneously.
330
331**Pattern:**
332```rust
333// Each CI run gets unique ID
334let config = AuditConfig::ci();
335// run_id = "ci-{uuid}"
336
337// Tests isolated by run ID
338let events = client.query().await?;
339// Only returns events for this run
340```
341
342---
343
344### 4. Standards Compliance Reduces Friction
345
346**Lesson:** Using standard NIP-01 "t" tags instead of custom tags.
347
348**Benefits:**
349- ✅ No conflicts with other systems
350- ✅ Standard relay filtering works
351- ✅ Better interoperability
352- ✅ Self-documenting
353
354---
355
356## Future Enhancements
357
358### Planned Features
359
360- [ ] **GRASP-01 Test Suite**: Repository announcement and state event tests
361- [ ] **Test Report Generation**: JSON/HTML output for CI/CD
362- [ ] **Performance Benchmarks**: Measure relay performance
363- [ ] **Relay Comparison**: Side-by-side compliance comparison
364- [ ] **Continuous Monitoring**: Periodic production audits
365
366---
367
368### Possible Improvements
369
370- [ ] **Parallel Test Execution**: Run specs in parallel
371- [ ] **Retry Logic**: Handle transient failures
372- [ ] **Custom Assertions**: Domain-specific test helpers
373- [ ] **Event Diff Tool**: Compare expected vs actual events
374- [ ] **Cleanup Automation**: Auto-cleanup after tests
375
376---
377
378## Common Issues
379
380### Issue: Integration Tests Fail
381
382**Symptoms:** Tests timeout or fail to connect
383
384**Causes:**
3851. No relay running
3862. Wrong relay URL
3873. Firewall blocking connection
388
389**Solution:**
390```bash
391# Start relay
392docker run --rm -p 7000:7000 scsibug/nostr-rs-relay
393
394# Verify relay is running
395curl http://localhost:7000
396
397# Run tests
398cargo test -- --ignored
399```
400
401---
402
403### Issue: Events Not Found in Query
404
405**Symptoms:** Query returns empty even though events were sent
406
407**Causes:**
4081. Wrong run ID (querying different run)
4092. Connection timing (query before event propagated)
4103. Tag mismatch (uppercase vs lowercase)
411
412**Solution:**
413```rust
414// Use same config for send and query
415let config = AuditConfig::ci();
416
417// Wait for event to propagate
418tokio::time::sleep(Duration::from_millis(500)).await;
419
420// Verify tags match exactly
421let t_tag = SingleLetterTag::lowercase(Alphabet::T); // Lowercase!
422```
423
424---
425
426### Issue: Build Fails in CI
427
428**Symptoms:** `cargo build` fails with dependency errors
429
430**Cause:** Not in Nix dev environment
431
432**Solution:**
433```bash
434# Enter Nix environment first
435cd grasp-audit
436nix develop
437
438# Then build
439cargo build
440```
441
442---
443
444## Quick Reference
445
446### Configuration
447
448```rust
449// CI mode
450let config = AuditConfig::ci();
451
452// Production mode
453let config = AuditConfig::production("run-id");
454```
455
456### Event Creation
457
458```rust
459let event = AuditEventBuilder::new(&config, kind, content)
460 .build(&keys)?;
461```
462
463### Client Usage
464
465```rust
466let client = AuditClient::new(config, keys);
467client.add_relay("ws://localhost:7000").await?;
468client.connect().await;
469let events = client.query().await?;
470```
471
472### Running Tests
473
474```bash
475# Unit tests
476cargo test --lib
477
478# Integration tests
479cargo test -- --ignored
480
481# CLI
482cargo run -- audit --relay ws://localhost:7000
483```
484
485---
486
487## References
488
489- **GRASP Protocol**: https://gitworkshop.dev/danconwaydev.com/grasp
490- **NIP-01**: https://github.com/nostr-protocol/nips/blob/master/01.md
491- **NIP-34**: https://github.com/nostr-protocol/nips/blob/master/34.md
492- **grasp-audit README**: `grasp-audit/README.md`
493- **Tag Migration**: `docs/archive/2025-11-04-tag-migration.md`
494
495---
496
497*Last updated: November 4, 2025*
498*Status: Living document - update as grasp-audit evolves*
diff --git a/docs/learnings/nix-flakes.md b/docs/learnings/nix-flakes.md
new file mode 100644
index 0000000..6876647
--- /dev/null
+++ b/docs/learnings/nix-flakes.md
@@ -0,0 +1,423 @@
1# Nix Flakes - Learnings and Gotchas
2
3**Purpose:** Document Nix flake patterns, gotchas, and best practices learned during ngit-grasp development
4**Last Updated:** November 4, 2025
5
6---
7
8## Critical Gotchas
9
10### Always Use `nix develop`, Not `nix-shell`
11
12**Problem:** We use `flake.nix`, not `shell.nix`. Using `nix-shell` will fail or use the wrong environment.
13
14```bash
15# ✅ Correct - for flake.nix
16cd grasp-audit
17nix develop
18nix develop -c cargo build
19
20# ❌ Wrong - for shell.nix (we don't use this)
21nix-shell
22nix-shell --run "cargo build"
23```
24
25**Why:**
26- `nix-shell` looks for `shell.nix` or `default.nix`
27- `nix develop` looks for `flake.nix`
28- We migrated from `shell.nix` to `flake.nix` on November 4, 2025
29
30**Related:** See `docs/archive/2025-11-04-flake-migration.md`
31
32---
33
34## Flake Structure
35
36### Our Standard Flake Pattern
37
38```nix
39{
40 inputs = {
41 nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
42 rust-overlay.url = "github:oxalica/rust-overlay";
43 flake-utils.url = "github:numtide/flake-utils";
44 };
45
46 outputs = { nixpkgs, rust-overlay, flake-utils, ... }:
47 flake-utils.lib.eachDefaultSystem (system:
48 let
49 overlays = [ (import rust-overlay) ];
50 pkgs = import nixpkgs { inherit system overlays; };
51 manifest = pkgs.lib.importTOML ./Cargo.toml;
52 in with pkgs; {
53 # Development shell
54 devShells.default = mkShell {
55 nativeBuildInputs = [
56 rust-bin.stable.latest.default
57 pkg-config
58 gitlint
59 ];
60 buildInputs = [
61 openssl
62 ];
63 shellHook = ''
64 echo "🦀 Development environment loaded"
65 export RUST_SRC_PATH=${pkgs.rustPlatform.rustLibSrc}
66 '';
67 };
68
69 # Package output
70 packages.default = pkgs.rustPlatform.buildRustPackage {
71 pname = manifest.package.name;
72 version = manifest.package.version;
73 src = ./.;
74 cargoLock = { lockFile = ./Cargo.lock; };
75 buildInputs = [ openssl ];
76 nativeBuildInputs = [ pkg-config ];
77 doCheck = false; # Run tests separately
78 };
79 });
80}
81```
82
83### Key Components
84
851. **rust-overlay**: Provides latest stable Rust toolchain
862. **flake-utils**: Cross-platform support helper
873. **manifest**: Auto-read version from Cargo.toml
884. **devShells.default**: Development environment
895. **packages.default**: Buildable package
90
91---
92
93## Common Flake Commands
94
95### Essential Commands
96
97```bash
98# Enter development shell
99nix develop
100
101# Run command in dev shell (one-off)
102nix develop -c cargo build
103
104# Show flake outputs
105nix flake show
106
107# Check flake validity
108nix flake check
109
110# Update flake inputs (like updating dependencies)
111nix flake update
112
113# Build the package directly
114nix build
115
116# Run without installing
117nix run
118
119# Show flake metadata
120nix flake metadata
121```
122
123### Debugging Commands
124
125```bash
126# Show detailed evaluation trace
127nix develop --show-trace
128
129# Print flake evaluation
130nix eval .#devShells.x86_64-linux.default
131
132# Check what's in the store
133nix path-info .#packages.x86_64-linux.default
134```
135
136---
137
138## Subproject Flakes
139
140### grasp-audit Has Its Own Flake
141
142**Important:** `grasp-audit/` is a subproject with its own `flake.nix` and `Cargo.toml`.
143
144```bash
145# ✅ Correct - enter grasp-audit environment
146cd grasp-audit
147nix develop
148cargo build
149
150# ❌ Wrong - can't build from root
151cd ngit-grasp
152cargo build # This won't find grasp-audit dependencies
153```
154
155**Why:**
156- Each Rust workspace needs its own Nix environment
157- Dependencies are project-specific
158- Flake inputs are locked per-project
159
160---
161
162## Migration from shell.nix to flake.nix
163
164### What Changed
165
166**Before (shell.nix):**
167```nix
168{ pkgs ? import <nixpkgs> {} }:
169
170pkgs.mkShell {
171 buildInputs = with pkgs; [
172 rustc
173 cargo
174 openssl
175 pkg-config
176 ];
177}
178```
179
180**After (flake.nix):**
181- Locked inputs (reproducible)
182- Multi-output (dev shell + package)
183- Cross-platform by default
184- Better tooling integration
185
186### Migration Steps
187
1881. Create `flake.nix` with standard structure
1892. Run `nix flake check` to validate
1903. Update all documentation: `nix-shell` → `nix develop`
1914. Test that build works: `nix develop -c cargo build`
1925. Remove `shell.nix`
1936. Commit changes
194
195**Reference:** See `docs/archive/2025-11-04-flake-migration.md`
196
197---
198
199## Benefits of Flakes
200
201### Reproducibility
202
203**Locked inputs** ensure everyone gets the same environment:
204
205```bash
206# flake.lock contains exact commits
207$ cat flake.lock
208{
209 "nodes": {
210 "nixpkgs": {
211 "locked": {
212 "lastModified": 1698611440,
213 "narHash": "sha256-jPjHjrerhYDy3q9+s5EAsuhyhuknNfowY6yt6pjn9pc=",
214 "rev": "23e89e0c8c5e2d9cf5b5e7c3e8e8e8e8e8e8e8e8"
215 }
216 }
217 }
218}
219```
220
221Everyone running `nix develop` gets **exactly** this version of nixpkgs.
222
223### Multi-Output
224
225Single flake provides:
226- **devShells.default**: Development environment
227- **packages.default**: Buildable package
228- **apps.default**: Runnable application (optional)
229
230### Composability
231
232Flakes can use other flakes as inputs:
233
234```nix
235{
236 inputs = {
237 grasp-audit.url = "path:./grasp-audit";
238 };
239}
240```
241
242---
243
244## Common Issues
245
246### Issue: "error: getting status of '/nix/store/...': No such file or directory"
247
248**Cause:** Flake inputs need to be updated or fetched
249
250**Solution:**
251```bash
252nix flake update
253nix develop
254```
255
256### Issue: "error: experimental feature 'nix-command' is not enabled"
257
258**Cause:** Nix flakes are experimental and need to be enabled
259
260**Solution:**
261Add to `~/.config/nix/nix.conf`:
262```
263experimental-features = nix-command flakes
264```
265
266### Issue: Changes to flake.nix not taking effect
267
268**Cause:** Flake evaluation is cached
269
270**Solution:**
271```bash
272# Clear evaluation cache
273nix flake update
274# Or force re-evaluation
275nix develop --refresh
276```
277
278### Issue: "error: cannot find flake 'flake:self' in the flake registries"
279
280**Cause:** Not in a git repository or flake.nix not committed
281
282**Solution:**
283```bash
284git add flake.nix flake.lock
285git commit -m "Add flake"
286```
287
288**Note:** Flakes require git. Uncommitted files are ignored by default.
289
290---
291
292## Best Practices
293
294### 1. Always Commit flake.lock
295
296```bash
297git add flake.lock
298git commit -m "Update flake inputs"
299```
300
301**Why:** Ensures reproducibility across machines and CI/CD
302
303### 2. Use Specific Rust Versions When Needed
304
305```nix
306# Latest stable (default)
307rust-bin.stable.latest.default
308
309# Specific version
310rust-bin.stable."1.75.0".default
311
312# Nightly
313rust-bin.nightly."2024-01-01".default
314```
315
316### 3. Include Helpful Shell Hooks
317
318```nix
319shellHook = ''
320 echo "🦀 GRASP Audit development environment"
321 echo ""
322 echo "Common commands:"
323 echo " cargo build - Build project"
324 echo " cargo test - Run tests"
325 echo " cargo run - Run binary"
326 echo ""
327 export RUST_SRC_PATH=${pkgs.rustPlatform.rustLibSrc}
328'';
329```
330
331### 4. Separate Build and Runtime Dependencies
332
333```nix
334# Build-time only
335nativeBuildInputs = [
336 pkg-config
337 rustc
338 cargo
339];
340
341# Runtime needed
342buildInputs = [
343 openssl
344];
345```
346
347### 5. Disable Tests in Package Build
348
349```nix
350packages.default = pkgs.rustPlatform.buildRustPackage {
351 # ...
352 doCheck = false; # Run tests separately with cargo test
353};
354```
355
356**Why:** Faster builds, tests run via `cargo test` in dev shell
357
358---
359
360## Workflow Examples
361
362### Daily Development
363
364```bash
365# Start work
366cd grasp-audit
367nix develop
368
369# Inside nix shell
370cargo build
371cargo test
372cargo run -- --help
373
374# Exit shell
375exit
376```
377
378### CI/CD
379
380```bash
381# One-off commands (no interactive shell)
382nix develop -c cargo build
383nix develop -c cargo test --lib
384nix develop -c cargo test -- --ignored
385```
386
387### Building Release
388
389```bash
390# Build package directly
391nix build
392
393# Result is in ./result/bin/
394./result/bin/grasp-audit --version
395```
396
397---
398
399## References
400
401- **Nix Flakes Manual**: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html
402- **rust-overlay**: https://github.com/oxalica/rust-overlay
403- **flake-utils**: https://github.com/numtide/flake-utils
404- **Our Migration**: `docs/archive/2025-11-04-flake-migration.md`
405
406---
407
408## Quick Reference
409
410| Task | Command |
411|------|---------|
412| Enter dev shell | `nix develop` |
413| Run one command | `nix develop -c <command>` |
414| Show outputs | `nix flake show` |
415| Validate flake | `nix flake check` |
416| Update inputs | `nix flake update` |
417| Build package | `nix build` |
418| Run package | `nix run` |
419
420---
421
422*Last updated: November 4, 2025*
423*Status: Living document - update as we learn more*
diff --git a/docs/learnings/nostr-sdk.md b/docs/learnings/nostr-sdk.md
new file mode 100644
index 0000000..57f451a
--- /dev/null
+++ b/docs/learnings/nostr-sdk.md
@@ -0,0 +1,577 @@
1# nostr-sdk - Learnings and Patterns
2
3**Purpose:** Document nostr-sdk usage patterns, upgrade notes, and gotchas
4**Last Updated:** November 4, 2025
5
6---
7
8## Current Version
9
10**We use nostr-sdk 0.43.x (latest stable)**
11
12```toml
13[dependencies]
14nostr-sdk = "0.43"
15```
16
17**Upgraded from:** 0.35.0 on November 4, 2025
18
19---
20
21## Critical Breaking Changes (0.35 → 0.43)
22
23### 1. EventBuilder API Changed
24
25**Before (0.35):**
26```rust
27let event = EventBuilder::new(kind, content, tags)
28 .to_event(keys)?;
29```
30
31**After (0.43):**
32```rust
33let event = EventBuilder::new(kind, content)
34 .tags(tags)
35 .sign_with_keys(keys)?;
36```
37
38**Changes:**
39- ❌ Removed `tags` parameter from constructor
40- ✅ Use `.tags()` builder method instead
41- ❌ Removed `.to_event()` method
42- ✅ Use `.sign_with_keys()` instead (more descriptive)
43
44---
45
46### 2. Client Ownership of Keys
47
48**Before (0.35):**
49```rust
50let keys = Keys::generate();
51let client = Client::new(&keys); // Reference
52// keys still available
53```
54
55**After (0.43):**
56```rust
57let keys = Keys::generate();
58let client = Client::new(keys.clone()); // Ownership
59// Need to clone if we want to keep keys
60```
61
62**Why:** Allows Client to own the signer, enabling more flexible signer types.
63
64---
65
66### 3. Relay Status Check No Longer Async
67
68**Before (0.35):**
69```rust
70if relay.is_connected().await {
71 // ...
72}
73```
74
75**After (0.43):**
76```rust
77if relay.is_connected() { // No await!
78 // ...
79}
80```
81
82**Why:** Status check doesn't require async operation.
83
84---
85
86### 4. Query API Redesigned
87
88**Before (0.35):**
89```rust
90let events = client
91 .get_events_of(vec![filter], EventSource::relays(Some(timeout)))
92 .await?;
93// Returns Vec<Event>
94```
95
96**After (0.43):**
97```rust
98let events = client
99 .fetch_events(filter, timeout)
100 .await?;
101// Returns Events (iterable collection)
102
103// Convert to Vec if needed
104let vec: Vec<Event> = events.into_iter().collect();
105```
106
107**Changes:**
108- ❌ Removed `get_events_of()` method
109- ✅ Use `fetch_events()` instead
110- ❌ Removed `EventSource` parameter (confusing)
111- ✅ Direct timeout parameter
112- ❌ Single filter instead of `Vec<Filter>`
113- ✅ Returns `Events` type instead of `Vec<Event>`
114
115---
116
117### 5. Filter Custom Tags Simplified
118
119**Before (0.35):**
120```rust
121filter.custom_tag(tag, ["value"])
122filter.custom_tag(tag, [&string_ref])
123```
124
125**After (0.43):**
126```rust
127filter.custom_tag(tag, "value")
128filter.custom_tag(tag, &string_ref)
129```
130
131**Why:** Simplified API for the common case of single tag value.
132
133---
134
135### 6. Send Event Takes Reference
136
137**Before (0.35):**
138```rust
139let event_id = client.send_event(event).await?;
140```
141
142**After (0.43):**
143```rust
144let output = client.send_event(&event).await?;
145let event_id = *output.id();
146```
147
148**Changes:**
149- Takes `&Event` instead of `Event` (can reuse events)
150- Returns `SendEventOutput` instead of `EventId`
151- Need to call `.id()` to get the event ID
152
153---
154
155## Common Patterns
156
157### Creating and Signing Events
158
159```rust
160use nostr_sdk::prelude::*;
161
162// Generate keys
163let keys = Keys::generate();
164
165// Create event
166let event = EventBuilder::new(Kind::TextNote, "Hello Nostr!")
167 .tags(vec![
168 Tag::custom(TagKind::SingleLetter(SingleLetterTag::lowercase(Alphabet::T)),
169 vec!["nostr"]),
170 ])
171 .sign_with_keys(&keys)?;
172
173// Send event
174let output = client.send_event(&event).await?;
175println!("Event ID: {}", output.id());
176```
177
178---
179
180### Creating Custom Tags
181
182```rust
183use nostr_sdk::prelude::*;
184
185// Single letter tag (like "t" for topics)
186let t_tag = SingleLetterTag::lowercase(Alphabet::T);
187let tag = Tag::custom(
188 TagKind::SingleLetter(t_tag),
189 vec!["my-topic"]
190);
191
192// Custom multi-letter tag
193let tag = Tag::custom(
194 TagKind::Custom("custom-tag".to_string()),
195 vec!["value1", "value2"]
196);
197
198// Hashtag (convenience method)
199let tag = Tag::hashtag("nostr"); // Creates ["t", "nostr"]
200```
201
202---
203
204### Querying Events
205
206```rust
207use nostr_sdk::prelude::*;
208
209// Build filter
210let filter = Filter::new()
211 .kind(Kind::TextNote)
212 .custom_tag(
213 SingleLetterTag::lowercase(Alphabet::T),
214 "my-topic"
215 )
216 .since(Timestamp::now() - Duration::from_secs(3600)); // Last hour
217
218// Query events
219let timeout = Duration::from_secs(10);
220let events = client.fetch_events(filter, timeout).await?;
221
222// Process events
223for event in events.into_iter() {
224 println!("Event: {}", event.id());
225}
226```
227
228---
229
230### Multiple Filters
231
232Since `fetch_events()` takes a single filter, combine multiple queries:
233
234```rust
235// Option 1: Fetch separately and combine
236let mut all_events = Vec::new();
237for filter in filters {
238 let events = client.fetch_events(filter, timeout).await?;
239 all_events.extend(events.into_iter());
240}
241
242// Option 2: Use subscription (more efficient)
243let subscription_id = SubscriptionId::new("my-sub");
244client.subscribe(filters, None).await?;
245
246// Handle events via notification handler
247let mut notifications = client.notifications();
248while let Ok(notification) = notifications.recv().await {
249 if let RelayPoolNotification::Event { event, .. } = notification {
250 println!("Event: {}", event.id());
251 }
252}
253```
254
255---
256
257### Client Setup with Relay
258
259```rust
260use nostr_sdk::prelude::*;
261
262// Create keys
263let keys = Keys::generate();
264
265// Create client
266let client = Client::new(keys.clone());
267
268// Add relay
269client.add_relay("wss://relay.example.com").await?;
270
271// Connect
272client.connect().await;
273
274// Wait for connection
275tokio::time::sleep(Duration::from_secs(2)).await;
276
277// Check connection
278if client.relay("wss://relay.example.com")
279 .await?
280 .is_connected()
281{
282 println!("Connected!");
283}
284```
285
286---
287
288## Testing Patterns
289
290### Unit Tests (No Relay Required)
291
292```rust
293#[cfg(test)]
294mod tests {
295 use super::*;
296 use nostr_sdk::prelude::*;
297
298 #[test]
299 fn test_event_creation() {
300 let keys = Keys::generate();
301 let event = EventBuilder::new(Kind::TextNote, "test")
302 .sign_with_keys(&keys)
303 .unwrap();
304
305 assert_eq!(event.kind(), Kind::TextNote);
306 assert_eq!(event.content(), "test");
307 }
308
309 #[test]
310 fn test_tag_creation() {
311 let t_tag = SingleLetterTag::lowercase(Alphabet::T);
312 let tag = Tag::custom(
313 TagKind::SingleLetter(t_tag),
314 vec!["test-topic"]
315 );
316
317 // Verify tag structure
318 assert_eq!(tag.as_vec()[0], "t");
319 assert_eq!(tag.as_vec()[1], "test-topic");
320 }
321}
322```
323
324---
325
326### Integration Tests (Relay Required)
327
328```rust
329#[cfg(test)]
330mod tests {
331 use super::*;
332 use nostr_sdk::prelude::*;
333
334 #[tokio::test]
335 #[ignore] // Requires running relay
336 async fn test_send_and_receive() -> Result<()> {
337 // Setup
338 let keys = Keys::generate();
339 let client = Client::new(keys.clone());
340 client.add_relay("ws://localhost:7000").await?;
341 client.connect().await;
342 tokio::time::sleep(Duration::from_secs(2)).await;
343
344 // Send event
345 let event = EventBuilder::new(Kind::TextNote, "test")
346 .sign_with_keys(&keys)?;
347 let output = client.send_event(&event).await?;
348
349 // Query it back
350 let filter = Filter::new()
351 .id(*output.id());
352 let events = client.fetch_events(filter, Duration::from_secs(5)).await?;
353
354 assert_eq!(events.len(), 1);
355 Ok(())
356 }
357}
358```
359
360**Running integration tests:**
361```bash
362# Start relay first
363docker run --rm -p 7000:7000 scsibug/nostr-rs-relay
364
365# Run tests
366cargo test -- --ignored
367```
368
369---
370
371## Common Gotchas
372
373### 1. Event Validation Failures
374
375**Problem:** Events fail validation with cryptic errors
376
377**Common Causes:**
378- Invalid signature (wrong keys used)
379- Invalid event ID (content/tags changed after signing)
380- Invalid timestamp (too far in future/past)
381
382**Solution:**
383```rust
384// Always sign AFTER setting all fields
385let event = EventBuilder::new(kind, content)
386 .tags(tags) // Set tags first
387 .sign_with_keys(&keys)?; // Sign last
388
389// Don't modify event after signing!
390```
391
392---
393
394### 2. Filter Not Matching Events
395
396**Problem:** Query returns no events even though they exist
397
398**Common Causes:**
399- Tag kind mismatch (uppercase vs lowercase)
400- Wrong filter field (using `.author()` when you need `.authors()`)
401- Timeout too short
402
403**Solution:**
404```rust
405// Be explicit about tag kinds
406let t_tag = SingleLetterTag::lowercase(Alphabet::T); // Lowercase!
407
408// Use correct filter methods
409let filter = Filter::new()
410 .authors(vec![keys.public_key()]) // Note: plural
411 .kinds(vec![Kind::TextNote]); // Note: plural
412
413// Increase timeout for slow relays
414let timeout = Duration::from_secs(10);
415```
416
417---
418
419### 3. Connection Timing Issues
420
421**Problem:** Events fail to send or queries return empty
422
423**Cause:** Client not fully connected to relay
424
425**Solution:**
426```rust
427// Connect
428client.connect().await;
429
430// Wait for connection to establish
431tokio::time::sleep(Duration::from_secs(2)).await;
432
433// Verify connection
434let relay = client.relay("wss://relay.example.com").await?;
435if !relay.is_connected() {
436 return Err("Not connected".into());
437}
438
439// Now safe to send/query
440```
441
442---
443
444### 4. Clone Keys When Creating Client
445
446**Problem:** Can't use keys after creating client
447
448**Cause:** Client takes ownership in 0.43+
449
450**Solution:**
451```rust
452// Clone keys if you need them later
453let keys = Keys::generate();
454let client = Client::new(keys.clone()); // Clone!
455
456// Now can still use keys
457let pubkey = keys.public_key();
458```
459
460---
461
462## Performance Tips
463
464### 1. Reuse Clients
465
466```rust
467// ✅ Good - single client
468let client = Client::new(keys);
469client.add_relay("wss://relay1.com").await?;
470client.add_relay("wss://relay2.com").await?;
471client.connect().await;
472
473// ❌ Bad - multiple clients
474for relay in relays {
475 let client = Client::new(keys.clone()); // Wasteful!
476 client.add_relay(relay).await?;
477}
478```
479
480---
481
482### 2. Use Subscriptions for Live Updates
483
484```rust
485// ✅ Good for live updates - subscription
486let filters = vec![Filter::new().kind(Kind::TextNote)];
487client.subscribe(filters, None).await?;
488
489let mut notifications = client.notifications();
490while let Ok(notification) = notifications.recv().await {
491 // Handle events as they arrive
492}
493
494// ❌ Bad for live updates - polling
495loop {
496 let events = client.fetch_events(filter, timeout).await?;
497 tokio::time::sleep(Duration::from_secs(1)).await;
498}
499```
500
501---
502
503### 3. Batch Event Creation
504
505```rust
506// ✅ Good - reuse keys
507let keys = Keys::generate();
508let events: Vec<Event> = (0..100)
509 .map(|i| {
510 EventBuilder::new(Kind::TextNote, format!("Message {}", i))
511 .sign_with_keys(&keys)
512 .unwrap()
513 })
514 .collect();
515
516// ❌ Bad - regenerate keys
517let events: Vec<Event> = (0..100)
518 .map(|i| {
519 let keys = Keys::generate(); // Wasteful!
520 EventBuilder::new(Kind::TextNote, format!("Message {}", i))
521 .sign_with_keys(&keys)
522 .unwrap()
523 })
524 .collect();
525```
526
527---
528
529## Migration Checklist (0.35 → 0.43)
530
531When upgrading from 0.35 to 0.43:
532
533- [ ] Update `Cargo.toml`: `nostr-sdk = "0.43"`
534- [ ] Fix `EventBuilder::new()` - remove tags parameter
535- [ ] Fix `EventBuilder::to_event()` → `sign_with_keys()`
536- [ ] Fix `Client::new()` - clone keys instead of reference
537- [ ] Fix `Relay::is_connected()` - remove `.await`
538- [ ] Fix `Client::get_events_of()` → `fetch_events()`
539- [ ] Remove `EventSource::relays()` usage
540- [ ] Fix `Filter::custom_tag()` - single value instead of array
541- [ ] Fix `Client::send_event()` - pass reference, handle `SendEventOutput`
542- [ ] Update tests
543- [ ] Verify all builds pass
544- [ ] Run integration tests
545
546**Reference:** See `docs/archive/2025-11-04-nostr-sdk-upgrade.md`
547
548---
549
550## Useful Resources
551
552- **nostr-sdk docs**: https://docs.rs/nostr-sdk/0.43.0
553- **rust-nostr GitHub**: https://github.com/rust-nostr/nostr
554- **NIPs**: https://github.com/nostr-protocol/nips
555- **NIP-01 (Events)**: https://github.com/nostr-protocol/nips/blob/master/01.md
556- **NIP-34 (Git)**: https://github.com/nostr-protocol/nips/blob/master/34.md
557
558---
559
560## Quick Reference
561
562| Task | Code |
563|------|------|
564| Create event | `EventBuilder::new(kind, content).sign_with_keys(&keys)?` |
565| Add tags | `.tags(vec![tag1, tag2])` |
566| Custom tag | `Tag::custom(TagKind::SingleLetter(t), vec!["value"])` |
567| Create client | `Client::new(keys.clone())` |
568| Add relay | `client.add_relay("wss://...").await?` |
569| Connect | `client.connect().await` |
570| Send event | `client.send_event(&event).await?` |
571| Query events | `client.fetch_events(filter, timeout).await?` |
572| Subscribe | `client.subscribe(filters, None).await?` |
573
574---
575
576*Last updated: November 4, 2025*
577*Status: Living document - update as nostr-sdk evolves*