diff options
| author | DanConwayDev <DanConwayDev@protonmail.com> | 2025-12-03 11:19:40 +0000 |
|---|---|---|
| committer | DanConwayDev <DanConwayDev@protonmail.com> | 2025-12-03 11:19:40 +0000 |
| commit | 2eaff5b79fed364d5eba5eb38e4b7bf76326884d (patch) | |
| tree | deacd6294f8860096ee82ee76930204efd65e33c /docs/archive/2025-11-03-test-breakdown.md | |
| parent | 57bc8cd9c021feaf08e139e8fb62800bc476068e (diff) | |
remove docs archive
Diffstat (limited to 'docs/archive/2025-11-03-test-breakdown.md')
| -rw-r--r-- | docs/archive/2025-11-03-test-breakdown.md | 203 |
1 files changed, 0 insertions, 203 deletions
diff --git a/docs/archive/2025-11-03-test-breakdown.md b/docs/archive/2025-11-03-test-breakdown.md deleted file mode 100644 index c054cd3..0000000 --- a/docs/archive/2025-11-03-test-breakdown.md +++ /dev/null | |||
| @@ -1,203 +0,0 @@ | |||
| 1 | # GRASP-01 Test Breakdown: First Requirement | ||
| 2 | |||
| 3 | **Requirement:** "MUST serve a NIP-01 compliant nostr relay at / that accepts git repository announcements and their corresponding repo state announcements." | ||
| 4 | |||
| 5 | --- | ||
| 6 | |||
| 7 | ## Test Summary | ||
| 8 | |||
| 9 | | Category | Count | Purpose | Time Investment | | ||
| 10 | |----------|-------|---------|-----------------| | ||
| 11 | | NIP-01 Smoke Tests | 6 | Verify basic relay works | 1-2 days | | ||
| 12 | | GRASP-01 Specific | 12 | Verify GRASP protocol | 3-4 days | | ||
| 13 | | **Total** | **18** | **Prove the concept** | **1 week** | | ||
| 14 | |||
| 15 | --- | ||
| 16 | |||
| 17 | ## NIP-01 Smoke Tests (6 tests) | ||
| 18 | |||
| 19 | ### Why Only Smoke Tests? | ||
| 20 | |||
| 21 | **rust-nostr already has 1000+ tests for:** | ||
| 22 | - ✅ Event structure validation | ||
| 23 | - ✅ Signature verification (Schnorr/secp256k1) | ||
| 24 | - ✅ Event ID calculation (SHA256) | ||
| 25 | - ✅ WebSocket message handling | ||
| 26 | - ✅ Subscription management | ||
| 27 | - ✅ Filter matching | ||
| 28 | |||
| 29 | **We don't need to re-test this.** We just verify the relay works at all. | ||
| 30 | |||
| 31 | ### The 6 Smoke Tests | ||
| 32 | |||
| 33 | | # | Test Name | What It Tests | Why It Matters | | ||
| 34 | |---|-----------|---------------|----------------| | ||
| 35 | | 1 | `websocket_connection` | Can connect to `/` via WebSocket | Relay is running and accepting connections | | ||
| 36 | | 2 | `send_receive_event` | Can send EVENT, get OK response | Basic event submission works | | ||
| 37 | | 3 | `create_subscription` | Can send REQ, receive EOSE | Subscription system works | | ||
| 38 | | 4 | `close_subscription` | Can close subscriptions | Cleanup works | | ||
| 39 | | 5 | `reject_invalid_event` | Rejects bad signatures | Validation is enabled | | ||
| 40 | | 6 | `reject_invalid_event_id` | Rejects wrong IDs | ID verification works | | ||
| 41 | |||
| 42 | **Coverage:** Basic Nostr relay functionality (not GRASP-specific) | ||
| 43 | |||
| 44 | --- | ||
| 45 | |||
| 46 | ## GRASP-01 Specific Tests (12 tests) | ||
| 47 | |||
| 48 | ### Why These Tests? | ||
| 49 | |||
| 50 | These test **our code**, not rust-nostr's code. They verify: | ||
| 51 | - GRASP policy enforcement | ||
| 52 | - Repository announcement acceptance | ||
| 53 | - Integration between Nostr relay and Git service | ||
| 54 | |||
| 55 | ### The 12 GRASP Tests | ||
| 56 | |||
| 57 | | # | Test Name | Spec Ref | What It Tests | | ||
| 58 | |---|-----------|----------|---------------| | ||
| 59 | | 7 | `accepts_repository_announcement` | GRASP-01:9-10 | Accepts NIP-34 kind 30617 events | | ||
| 60 | | 8 | `accepts_repository_state` | GRASP-01:9-10 | Accepts NIP-34 kind 30618 events | | ||
| 61 | | 9 | `rejects_announcement_without_clone_tag` | GRASP-01:12-13 | Enforces clone tag requirement | | ||
| 62 | | 10 | `rejects_announcement_without_relay_tag` | GRASP-01:12-13 | Enforces relay tag requirement | | ||
| 63 | | 11 | `accepts_announcement_with_multiple_clones` | GRASP-01:12-13 | Handles multiple clone URLs | | ||
| 64 | | 12 | `accepts_events_tagging_announcement` | GRASP-01:17-20 | Accepts issues/PRs tagging repos | | ||
| 65 | | 13 | `accepts_events_tagged_by_announcement` | GRASP-01:17-20 | Accepts events tagged by repos | | ||
| 66 | | 14 | `rejects_events_tagging_rejected_announcement` | GRASP-01:17-20 | Rejects orphaned events | | ||
| 67 | | 15 | `query_announcements_by_identifier` | GRASP-01 (implied) | Can query repos by identifier | | ||
| 68 | | 16 | `query_state_events` | GRASP-01 (implied) | Can query repository state | | ||
| 69 | | 17 | `state_replaces_previous` | NIP-01 replaceable | Latest state wins | | ||
| 70 | | 18 | `concurrent_event_submission` | General reliability | No race conditions | | ||
| 71 | |||
| 72 | **Coverage:** GRASP protocol requirements and policy enforcement | ||
| 73 | |||
| 74 | --- | ||
| 75 | |||
| 76 | ## What We're NOT Testing (and Why) | ||
| 77 | |||
| 78 | ### Not Testing: NIP-01 Core Protocol | ||
| 79 | |||
| 80 | **Reason:** rust-nostr already tests this extensively | ||
| 81 | |||
| 82 | | What | Why Not Testing | | ||
| 83 | |------|-----------------| | ||
| 84 | | Event signature verification | rust-nostr has 100+ tests | | ||
| 85 | | Event ID calculation | rust-nostr has 50+ tests | | ||
| 86 | | WebSocket message parsing | rust-nostr has 200+ tests | | ||
| 87 | | Subscription filter matching | rust-nostr has 150+ tests | | ||
| 88 | | Event serialization | rust-nostr has 75+ tests | | ||
| 89 | |||
| 90 | **Estimated time saved:** 2-3 weeks of redundant work | ||
| 91 | |||
| 92 | ### Not Testing: Git Protocol Details | ||
| 93 | |||
| 94 | **Reason:** Will test in separate Git service tests | ||
| 95 | |||
| 96 | | What | Where It's Tested | | ||
| 97 | |------|-------------------| | ||
| 98 | | Git pack parsing | Git service unit tests | | ||
| 99 | | Ref update parsing | Git service unit tests | | ||
| 100 | | Git authorization | Git integration tests | | ||
| 101 | | Push/pull operations | E2E tests | | ||
| 102 | |||
| 103 | --- | ||
| 104 | |||
| 105 | ## Test Implementation Estimate | ||
| 106 | |||
| 107 | ### Week 1: Test Tool Foundation | ||
| 108 | - **Day 1-2**: Set up `grasp-compliance-tests/` crate | ||
| 109 | - **Day 3**: Implement test client (HTTP/WebSocket) | ||
| 110 | - **Day 4**: Implement NIP-01 smoke tests (6 tests) | ||
| 111 | - **Day 5**: Test fixtures and builders | ||
| 112 | |||
| 113 | ### Week 2: GRASP Tests | ||
| 114 | - **Day 1-2**: Implement announcement tests (7-11) | ||
| 115 | - **Day 3**: Implement related event tests (12-14) | ||
| 116 | - **Day 4**: Implement query tests (15-17) | ||
| 117 | - **Day 5**: Implement concurrent test (18) + polish | ||
| 118 | |||
| 119 | ### Week 3: Integration | ||
| 120 | - **Day 1-2**: Create minimal ngit-grasp skeleton | ||
| 121 | - **Day 3-4**: Wire up nostr-relay-builder | ||
| 122 | - **Day 5**: First test run (expect failures) | ||
| 123 | |||
| 124 | ### Week 4: Iteration | ||
| 125 | - **Day 1-3**: Fix failing tests | ||
| 126 | - **Day 4**: Documentation | ||
| 127 | - **Day 5**: Polish and prepare for next requirement | ||
| 128 | |||
| 129 | **Total:** 4 weeks to prove the concept | ||
| 130 | |||
| 131 | --- | ||
| 132 | |||
| 133 | ## Success Criteria | ||
| 134 | |||
| 135 | ### Phase 1: Test Tool Works | ||
| 136 | - ✅ Can connect to any WebSocket relay | ||
| 137 | - ✅ Can send events and subscriptions | ||
| 138 | - ✅ Can assert on responses | ||
| 139 | - ✅ All 18 tests can execute (even if they fail) | ||
| 140 | |||
| 141 | ### Phase 2: Smoke Tests Pass | ||
| 142 | - ✅ Basic NIP-01 functionality works | ||
| 143 | - ✅ Can send/receive events | ||
| 144 | - ✅ Subscriptions work | ||
| 145 | - ✅ Invalid events rejected | ||
| 146 | |||
| 147 | ### Phase 3: GRASP Tests Pass | ||
| 148 | - ✅ Repository announcements accepted | ||
| 149 | - ✅ State events accepted | ||
| 150 | - ✅ Policy enforcement works (clone/relay tags) | ||
| 151 | - ✅ Related events accepted | ||
| 152 | - ✅ Queries work | ||
| 153 | |||
| 154 | ### Phase 4: Concept Proven | ||
| 155 | - ✅ All 18 tests pass | ||
| 156 | - ✅ Test tool is reusable | ||
| 157 | - ✅ Architecture validated | ||
| 158 | - ✅ Ready for next GRASP-01 requirements | ||
| 159 | |||
| 160 | --- | ||
| 161 | |||
| 162 | ## Comparison: Our Approach vs. Comprehensive | ||
| 163 | |||
| 164 | | Aspect | Our Approach | Comprehensive Approach | | ||
| 165 | |--------|--------------|------------------------| | ||
| 166 | | NIP-01 Tests | 6 smoke tests | 50-100 full tests | | ||
| 167 | | GRASP Tests | 12 focused tests | 12 focused tests | | ||
| 168 | | Total Tests | **18** | **62-112** | | ||
| 169 | | Time to Implement | **1 week** | **3-4 weeks** | | ||
| 170 | | Maintenance Burden | **Low** | **High** | | ||
| 171 | | Redundancy | **Minimal** | **Significant** | | ||
| 172 | | Value-Add | **High** (GRASP-specific) | **Low** (mostly redundant) | | ||
| 173 | |||
| 174 | **Conclusion:** Our approach is 3-4x faster with same GRASP coverage. | ||
| 175 | |||
| 176 | --- | ||
| 177 | |||
| 178 | ## Next Steps | ||
| 179 | |||
| 180 | 1. ✅ **Review this breakdown** - Confirm scope | ||
| 181 | 2. ✅ **Choose approach** - Test-first, parallel, or implementation-first | ||
| 182 | 3. ✅ **Start implementation** - Create test tool skeleton | ||
| 183 | 4. ✅ **Iterate** - Build until all tests pass | ||
| 184 | |||
| 185 | --- | ||
| 186 | |||
| 187 | ## Questions? | ||
| 188 | |||
| 189 | - **Q: Is 6 smoke tests enough?** | ||
| 190 | A: Yes, because rust-nostr already tests NIP-01 comprehensively. | ||
| 191 | |||
| 192 | - **Q: Should we test more NIP-01 features?** | ||
| 193 | A: Only if we find bugs in rust-nostr (unlikely). | ||
| 194 | |||
| 195 | - **Q: Can other implementations use this?** | ||
| 196 | A: Yes! That's the point of making it standalone. | ||
| 197 | |||
| 198 | - **Q: What about GRASP-02 and GRASP-05?** | ||
| 199 | A: We'll add those test modules later, same structure. | ||
| 200 | |||
| 201 | --- | ||
| 202 | |||
| 203 | **Ready to proceed?** See REPORT_COMPLIANCE_TESTING.md for full details. | ||