diff options
| -rw-r--r-- | AGENTS.md | 45 | ||||
| -rw-r--r-- | grasp-audit/README.md | 42 | ||||
| -rwxr-xr-x | grasp-audit/test-ngit-relay.sh | 175 |
3 files changed, 232 insertions, 30 deletions
| @@ -44,6 +44,39 @@ Tests marked `#[ignore]` need relay - unit tests don't. | |||
| 44 | 44 | ||
| 45 | **Note:** Always use a random available port for the relay to avoid conflicts with existing services. | 45 | **Note:** Always use a random available port for the relay to avoid conflicts with existing services. |
| 46 | 46 | ||
| 47 | ### Standard Testing Process (Recommended) | ||
| 48 | |||
| 49 | **Use test-ngit-relay.sh for automated relay management:** | ||
| 50 | |||
| 51 | This script handles all relay lifecycle management automatically: | ||
| 52 | - Starts ngit-relay in isolated Docker container | ||
| 53 | - Uses random port to avoid conflicts | ||
| 54 | - Creates isolated temporary directories | ||
| 55 | - Ensures cleanup on exit (success or failure) | ||
| 56 | - Supports both audit and test modes | ||
| 57 | |||
| 58 | **Basic Usage:** | ||
| 59 | |||
| 60 | ```bash | ||
| 61 | # Run cargo test suite (recommended for GRASP-01 development) | ||
| 62 | cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test | ||
| 63 | |||
| 64 | # Run audit CLI tool (for quick validation) | ||
| 65 | cd grasp-audit && nix develop -c bash test-ngit-relay.sh | ||
| 66 | |||
| 67 | # Get help | ||
| 68 | cd grasp-audit && ./test-ngit-relay.sh --help | ||
| 69 | ``` | ||
| 70 | |||
| 71 | **Benefits:** | ||
| 72 | - No manual relay startup required | ||
| 73 | - Automatic cleanup prevents leftover containers | ||
| 74 | - Random port selection avoids conflicts | ||
| 75 | - Consistent environment across all runs | ||
| 76 | - Proper test isolation | ||
| 77 | |||
| 78 | **Note:** Manual relay setup is still available but test-ngit-relay.sh is recommended for development workflows. | ||
| 79 | |||
| 47 | ### Running Single Test | 80 | ### Running Single Test |
| 48 | 81 | ||
| 49 | ```bash | 82 | ```bash |
| @@ -140,6 +173,7 @@ See `docs/archive/2025-11-04-nostr-sdk-upgrade.md` for full migration. | |||
| 140 | 4. **Test isolation:** Integration tests need relay, marked with `#[ignore]` | 173 | 4. **Test isolation:** Integration tests need relay, marked with `#[ignore]` |
| 141 | 5. **Work directory:** All session docs go in `work/`, NOT root | 174 | 5. **Work directory:** All session docs go in `work/`, NOT root |
| 142 | 6. **Archive naming:** Use `YYYY-MM-DD-description.md` format | 175 | 6. **Archive naming:** Use `YYYY-MM-DD-description.md` format |
| 176 | 7. **Use test-ngit-relay.sh**: Always use the test script for GRASP-01 tests - it handles cleanup and port management automatically | ||
| 143 | 177 | ||
| 144 | ## File Restrictions by Mode | 178 | ## File Restrictions by Mode |
| 145 | 179 | ||
| @@ -152,18 +186,19 @@ Code mode can only edit files matching specific patterns (enforced by system): | |||
| 152 | ## Quick Reference | 186 | ## Quick Reference |
| 153 | 187 | ||
| 154 | ```bash | 188 | ```bash |
| 189 | # Recommended: Use test-ngit-relay.sh for all testing | ||
| 190 | cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test | ||
| 191 | |||
| 155 | # Build grasp-audit | 192 | # Build grasp-audit |
| 156 | cd grasp-audit && nix develop -c cargo build | 193 | cd grasp-audit && nix develop -c cargo build |
| 157 | 194 | ||
| 158 | # Run all tests (requires relay running, set RELAY_URL to match your port) | 195 | # Manual relay testing (if needed) |
| 159 | cd grasp-audit && RELAY_URL="ws://localhost:18081" nix develop -c cargo test --ignored | 196 | # 1. Start relay: docker run --rm -p 18081:8081 ghcr.io/danconwaydev/ngit-relay:latest |
| 197 | # 2. Run tests: RELAY_URL="ws://localhost:18081" nix develop -c cargo test --ignored | ||
| 160 | 198 | ||
| 161 | # Run single test | 199 | # Run single test |
| 162 | cd grasp-audit && nix develop -c cargo test --lib test_name -- --nocapture | 200 | cd grasp-audit && nix develop -c cargo test --lib test_name -- --nocapture |
| 163 | 201 | ||
| 164 | # Verify GRASP-01 tests (cleaner output) | ||
| 165 | cd grasp-audit && RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib test_grasp01_nostr_relay_against_relay -- --ignored --nocapture 2>&1 | tail -60 | ||
| 166 | |||
| 167 | # Check session files | 202 | # Check session files |
| 168 | ls work/ # Should only have README.md when clean | 203 | ls work/ # Should only have README.md when clean |
| 169 | ``` | 204 | ``` |
diff --git a/grasp-audit/README.md b/grasp-audit/README.md index 3787fee..97a6429 100644 --- a/grasp-audit/README.md +++ b/grasp-audit/README.md | |||
| @@ -12,6 +12,28 @@ A reusable audit and compliance testing tool for GRASP protocol implementations. | |||
| 12 | 12 | ||
| 13 | ## Quick Start | 13 | ## Quick Start |
| 14 | 14 | ||
| 15 | The fastest way to run GRASP-01 compliance tests: | ||
| 16 | |||
| 17 | ```bash | ||
| 18 | # Run the test suite against ngit-relay | ||
| 19 | cd grasp-audit | ||
| 20 | nix develop -c bash test-ngit-relay.sh --mode test | ||
| 21 | ``` | ||
| 22 | |||
| 23 | This automatically: | ||
| 24 | - ✅ Starts ngit-relay in an isolated Docker container | ||
| 25 | - ✅ Runs all GRASP-01 compliance tests | ||
| 26 | - ✅ Cleans up resources when finished | ||
| 27 | |||
| 28 | **Currently Passing:** 4/18 tests (14 tests stubbed with "Not implemented yet") | ||
| 29 | |||
| 30 | For more options: | ||
| 31 | ```bash | ||
| 32 | ./test-ngit-relay.sh --help | ||
| 33 | ``` | ||
| 34 | |||
| 35 | ## Usage Examples | ||
| 36 | |||
| 15 | ### As a Library | 37 | ### As a Library |
| 16 | 38 | ||
| 17 | ```rust | 39 | ```rust |
| @@ -124,6 +146,8 @@ cargo run --example simple_audit | |||
| 124 | 146 | ||
| 125 | ## Testing | 147 | ## Testing |
| 126 | 148 | ||
| 149 | > **TL;DR:** See the [Quick Start](#quick-start) section for the fastest way to run tests. | ||
| 150 | |||
| 127 | ### Unit Tests | 151 | ### Unit Tests |
| 128 | 152 | ||
| 129 | ```bash | 153 | ```bash |
| @@ -134,20 +158,18 @@ nix develop | |||
| 134 | cargo test | 158 | cargo test |
| 135 | ``` | 159 | ``` |
| 136 | 160 | ||
| 137 | ### Integration Tests Against ngit-relay docker image | 161 | ### Integration Tests |
| 138 | 162 | ||
| 139 | Test against the reference GRASP implementation to ensure compatibility. | 163 | The recommended approach is [`test-ngit-relay.sh`](test-ngit-relay.sh), which handles all relay lifecycle management automatically. |
| 140 | 164 | ||
| 141 | **Automated Script (Recommended):** | 165 | See the [Quick Start](#quick-start) section for common usage patterns. |
| 142 | 166 | ||
| 143 | ```bash | 167 | **Advanced: Manual Relay Setup** |
| 144 | # Handles setup, testing, and cleanup automatically | ||
| 145 | ./test-ngit-relay.sh | ||
| 146 | ``` | ||
| 147 | 168 | ||
| 148 | See `test-ngit-relay.sh` for full setup/cleanup details. | 169 | <details> |
| 170 | <summary>Click to expand manual testing instructions</summary> | ||
| 149 | 171 | ||
| 150 | **Manual Testing:** | 172 | For advanced use cases where you need direct control over the relay: |
| 151 | 173 | ||
| 152 | ```bash | 174 | ```bash |
| 153 | # Start relay on a specific port (example uses 18081) | 175 | # Start relay on a specific port (example uses 18081) |
| @@ -160,6 +182,8 @@ grasp-audit audit --relay ws://localhost:18081 --mode ci | |||
| 160 | RELAY_URL="ws://localhost:18081" cargo test --lib test_grasp01_nostr_relay_against_relay -- --ignored --nocapture | 182 | RELAY_URL="ws://localhost:18081" cargo test --lib test_grasp01_nostr_relay_against_relay -- --ignored --nocapture |
| 161 | ``` | 183 | ``` |
| 162 | 184 | ||
| 185 | </details> | ||
| 186 | |||
| 163 | ## Architecture | 187 | ## Architecture |
| 164 | 188 | ||
| 165 | ``` | 189 | ``` |
diff --git a/grasp-audit/test-ngit-relay.sh b/grasp-audit/test-ngit-relay.sh index 356eed1..bd4b155 100755 --- a/grasp-audit/test-ngit-relay.sh +++ b/grasp-audit/test-ngit-relay.sh | |||
| @@ -1,36 +1,144 @@ | |||
| 1 | #!/bin/bash | 1 | #!/bin/bash |
| 2 | set -e | 2 | set -e |
| 3 | 3 | ||
| 4 | # ============================================================================= | ||
| 5 | # Script: test-ngit-relay.sh | ||
| 6 | # Purpose: Test ngit-relay against GRASP specifications | ||
| 7 | # | ||
| 8 | # This script automates the process of: | ||
| 9 | # 1. Starting a fresh ngit-relay instance in Docker | ||
| 10 | # 2. Running either grasp-audit CLI or cargo test suite | ||
| 11 | # 3. Cleaning up all resources on exit (via EXIT trap) | ||
| 12 | # | ||
| 13 | # Features: | ||
| 14 | # - Automatic cleanup via EXIT trap (runs even on failure) | ||
| 15 | # - Random port selection (20000-30000) to avoid conflicts | ||
| 16 | # - Unique temporary directories per run | ||
| 17 | # - Support for both audit and test execution modes | ||
| 18 | # ============================================================================= | ||
| 19 | |||
| 20 | # ----------------------------------------------------------------------------- | ||
| 21 | # Help Function | ||
| 22 | # ----------------------------------------------------------------------------- | ||
| 23 | show_help() { | ||
| 24 | cat << EOF | ||
| 25 | Usage: $(basename "$0") [OPTIONS] | ||
| 26 | |||
| 27 | Test ngit-relay against GRASP specifications | ||
| 28 | |||
| 29 | OPTIONS: | ||
| 30 | --mode <audit|test> Execution mode (default: audit) | ||
| 31 | audit: Run grasp-audit CLI tool | ||
| 32 | test: Run cargo test suite | ||
| 33 | --spec <spec> Specification to test (default: nip01-smoke) | ||
| 34 | Only used in audit mode | ||
| 35 | --help Show this help message | ||
| 36 | |||
| 37 | EXAMPLES: | ||
| 38 | # Run audit with default settings (current behavior) | ||
| 39 | ./test-ngit-relay.sh | ||
| 40 | |||
| 41 | # Run cargo tests instead | ||
| 42 | ./test-ngit-relay.sh --mode test | ||
| 43 | |||
| 44 | # Run audit with specific spec | ||
| 45 | ./test-ngit-relay.sh --mode audit --spec grasp01 | ||
| 46 | |||
| 47 | EOF | ||
| 48 | exit 0 | ||
| 49 | } | ||
| 50 | |||
| 51 | # ----------------------------------------------------------------------------- | ||
| 52 | # Argument Parsing | ||
| 53 | # ----------------------------------------------------------------------------- | ||
| 54 | # Default values maintain backward compatibility | ||
| 55 | MODE="audit" | ||
| 56 | SPEC="nip01-smoke" | ||
| 57 | |||
| 58 | # Parse command-line arguments | ||
| 59 | while [[ $# -gt 0 ]]; do | ||
| 60 | case $1 in | ||
| 61 | --mode) | ||
| 62 | MODE="$2" | ||
| 63 | shift 2 | ||
| 64 | ;; | ||
| 65 | --spec) | ||
| 66 | SPEC="$2" | ||
| 67 | shift 2 | ||
| 68 | ;; | ||
| 69 | --help) | ||
| 70 | show_help | ||
| 71 | ;; | ||
| 72 | *) | ||
| 73 | echo "❌ Unknown option: $1" | ||
| 74 | echo "Run with --help for usage information" | ||
| 75 | exit 1 | ||
| 76 | ;; | ||
| 77 | esac | ||
| 78 | done | ||
| 79 | |||
| 80 | # Validate mode parameter | ||
| 81 | if [ "$MODE" != "audit" ] && [ "$MODE" != "test" ]; then | ||
| 82 | echo "❌ Invalid mode: $MODE" | ||
| 83 | echo "Mode must be 'audit' or 'test'" | ||
| 84 | exit 1 | ||
| 85 | fi | ||
| 86 | |||
| 87 | # ----------------------------------------------------------------------------- | ||
| 88 | # Environment Setup | ||
| 89 | # ----------------------------------------------------------------------------- | ||
| 4 | # Create temporary directory with random name | 90 | # Create temporary directory with random name |
| 91 | # This ensures each test run is isolated and doesn't interfere with others | ||
| 5 | TEST_DIR=$(mktemp -d -t grasp-audit-run-XXXXXXXXXX) | 92 | TEST_DIR=$(mktemp -d -t grasp-audit-run-XXXXXXXXXX) |
| 6 | # Pick a random port in the range 20000-30000 | 93 | |
| 94 | # Pick a random port in the range 20000-30000 to avoid conflicts | ||
| 7 | PORT=$((20000 + RANDOM % 10000)) | 95 | PORT=$((20000 + RANDOM % 10000)) |
| 8 | # Generate a unique container name suffix | 96 | |
| 97 | # Generate a unique container name suffix for parallel runs | ||
| 9 | CONTAINER_SUFFIX=$RANDOM | 98 | CONTAINER_SUFFIX=$RANDOM |
| 10 | 99 | ||
| 11 | echo "🧹 Using temporary directory: $TEST_DIR" | 100 | echo "🧹 Using temporary directory: $TEST_DIR" |
| 12 | echo "🔌 Using port: $PORT" | 101 | echo "🔌 Using port: $PORT" |
| 102 | echo "🎯 Mode: $MODE $([ "$MODE" = "audit" ] && echo "(spec: $SPEC)" || echo "")" | ||
| 13 | 103 | ||
| 14 | # Cleanup function | 104 | # ----------------------------------------------------------------------------- |
| 105 | # Cleanup Function | ||
| 106 | # ----------------------------------------------------------------------------- | ||
| 107 | # This function is called automatically on script exit via trap | ||
| 108 | # The '|| true' pattern ensures cleanup continues even if individual steps fail | ||
| 15 | cleanup() { | 109 | cleanup() { |
| 110 | echo "" | ||
| 16 | echo "🛑 Stopping relay..." | 111 | echo "🛑 Stopping relay..." |
| 112 | # Stop container gracefully, ignore errors if already stopped | ||
| 17 | docker stop "grasp-audit-run-$CONTAINER_SUFFIX" 2>/dev/null || true | 113 | docker stop "grasp-audit-run-$CONTAINER_SUFFIX" 2>/dev/null || true |
| 18 | 114 | ||
| 19 | echo "🧹 Cleaning up temporary directory..." | 115 | echo "🧹 Cleaning up temporary directory..." |
| 116 | # Use alpine container to clean Docker-created files (may have different ownership) | ||
| 20 | docker run --rm -v "$TEST_DIR:/data" alpine sh -c "rm -rf /data/*" 2>/dev/null || true | 117 | docker run --rm -v "$TEST_DIR:/data" alpine sh -c "rm -rf /data/*" 2>/dev/null || true |
| 118 | # Remove the temporary directory itself | ||
| 21 | rm -rf "$TEST_DIR" | 119 | rm -rf "$TEST_DIR" |
| 22 | } | 120 | } |
| 23 | 121 | ||
| 24 | # Set trap to cleanup on exit | 122 | # Set trap to run cleanup on ANY exit (success, failure, or interrupt) |
| 123 | # This ensures resources are always cleaned up | ||
| 25 | trap cleanup EXIT | 124 | trap cleanup EXIT |
| 26 | 125 | ||
| 126 | # ----------------------------------------------------------------------------- | ||
| 127 | # Docker Setup | ||
| 128 | # ----------------------------------------------------------------------------- | ||
| 27 | echo "📁 Creating data directories..." | 129 | echo "📁 Creating data directories..." |
| 130 | # Create all required directories for ngit-relay in one command | ||
| 28 | mkdir -p "$TEST_DIR"/{repos,blossom,relay-db,logs} | 131 | mkdir -p "$TEST_DIR"/{repos,blossom,relay-db,logs} |
| 29 | 132 | ||
| 30 | echo "🚀 Starting ngit-relay..." | 133 | echo "🚀 Starting ngit-relay..." |
| 31 | # Remove any existing container with this name | 134 | # Remove any existing container with this name (defensive cleanup) |
| 32 | CONTAINER_NAME="grasp-audit-run-$CONTAINER_SUFFIX" | 135 | CONTAINER_NAME="grasp-audit-run-$CONTAINER_SUFFIX" |
| 33 | docker rm -f "$CONTAINER_NAME" 2>/dev/null || true | 136 | docker rm -f "$CONTAINER_NAME" 2>/dev/null || true |
| 137 | |||
| 138 | # Start ngit-relay in detached mode with all required configuration | ||
| 139 | # - Port mapping: expose internal 8081 on our random port | ||
| 140 | # - Volume mounts: persist data in temp directory | ||
| 141 | # - Proactive sync disabled: we control event submission via tests | ||
| 34 | docker run --rm -d \ | 142 | docker run --rm -d \ |
| 35 | --name "$CONTAINER_NAME" \ | 143 | --name "$CONTAINER_NAME" \ |
| 36 | -p "$PORT:8081" \ | 144 | -p "$PORT:8081" \ |
| @@ -49,19 +157,54 @@ docker run --rm -d \ | |||
| 49 | ghcr.io/danconwaydev/ngit-relay:latest | 157 | ghcr.io/danconwaydev/ngit-relay:latest |
| 50 | 158 | ||
| 51 | echo "⏳ Waiting for relay to start..." | 159 | echo "⏳ Waiting for relay to start..." |
| 160 | # Give the relay time to initialize before running tests | ||
| 52 | sleep 3 | 161 | sleep 3 |
| 53 | 162 | ||
| 54 | echo "🧪 Running tests..." | 163 | # ----------------------------------------------------------------------------- |
| 55 | echo "" | 164 | # Test Execution |
| 56 | echo "Note: ngit-relay only accepts Git-related events (NIP-34)." | 165 | # ----------------------------------------------------------------------------- |
| 57 | echo "Some NIP-01 smoke tests will fail (expected behavior)." | 166 | # Execute tests based on selected mode |
| 58 | echo "Validation tests should pass." | 167 | # The EXIT trap ensures cleanup happens regardless of test outcome |
| 59 | echo "" | ||
| 60 | 168 | ||
| 61 | # Run the CLI tool (cleanup happens via trap even on failure) | 169 | if [ "$MODE" = "audit" ]; then |
| 62 | RELAY_URL="ws://localhost:$PORT" cargo run -- audit --relay "ws://localhost:$PORT" --mode ci --spec nip01-smoke || { | 170 | echo "🧪 Running audit mode (spec: $SPEC)..." |
| 63 | echo "⚠️ Some tests failed (expected for ngit-relay)" | 171 | echo "" |
| 64 | echo " Validation tests should have passed" | 172 | echo "Note: ngit-relay only accepts Git-related events (NIP-34)." |
| 65 | } | 173 | echo "Some NIP-01 smoke tests will fail (expected behavior)." |
| 174 | echo "Validation tests should pass." | ||
| 175 | echo "" | ||
| 176 | |||
| 177 | # Run grasp-audit CLI tool | ||
| 178 | # - RELAY_URL: Environment variable used by audit tool | ||
| 179 | # - --relay: Command-line parameter for relay address | ||
| 180 | # - --mode ci: Continuous integration mode (structured output) | ||
| 181 | # - --spec: Which specification to test | ||
| 182 | # The '|| { }' block provides user-friendly messaging on failure | ||
| 183 | RELAY_URL="ws://localhost:$PORT" cargo run -- audit \ | ||
| 184 | --relay "ws://localhost:$PORT" \ | ||
| 185 | --mode ci \ | ||
| 186 | --spec "$SPEC" || { | ||
| 187 | echo "⚠️ Some tests failed (expected for ngit-relay)" | ||
| 188 | echo " Validation tests should have passed" | ||
| 189 | } | ||
| 190 | |||
| 191 | elif [ "$MODE" = "test" ]; then | ||
| 192 | echo "🧪 Running cargo test mode..." | ||
| 193 | echo "" | ||
| 194 | |||
| 195 | # Run cargo test suite | ||
| 196 | # - RELAY_URL: Environment variable tests use to connect | ||
| 197 | # - --lib: Only library tests (not integration tests in tests/) | ||
| 198 | # - test_grasp01_nostr_relay_against_relay: Specific test module | ||
| 199 | # - --ignored: Run tests marked with #[ignore] (these need relay) | ||
| 200 | # - --nocapture: Show println! output from tests | ||
| 201 | RELAY_URL="ws://localhost:$PORT" cargo test \ | ||
| 202 | --lib test_grasp01_nostr_relay_against_relay \ | ||
| 203 | -- --ignored --nocapture || { | ||
| 204 | echo "⚠️ Some tests failed" | ||
| 205 | echo " Review output above for details" | ||
| 206 | } | ||
| 207 | fi | ||
| 66 | 208 | ||
| 209 | echo "" | ||
| 67 | echo "✅ Done!" | 210 | echo "✅ Done!" |