upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/work/phase1-baseline.md
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2025-12-19 16:02:04 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2025-12-19 16:02:04 +0000
commit1adbd93e5bb8e14403ba64a76d5dc93209227514 (patch)
treecf21d9ec7958ac1918143b72649c53eb14804283 /work/phase1-baseline.md
parent02a90c109d4d08c6a54184f821c100f4eba92545 (diff)
docs: cleanup
Diffstat (limited to 'work/phase1-baseline.md')
-rw-r--r--work/phase1-baseline.md159
1 files changed, 0 insertions, 159 deletions
diff --git a/work/phase1-baseline.md b/work/phase1-baseline.md
deleted file mode 100644
index 8bd3902..0000000
--- a/work/phase1-baseline.md
+++ /dev/null
@@ -1,159 +0,0 @@
1# Phase 1: Sync Test Baseline
2
3**Timestamp:** 2025-12-18T16:50:07Z (UTC)
4**Git Commit:** (pre-refactoring baseline)
5
6## Test Execution Command
7```bash
8cargo test --test sync
9```
10
11## Summary Statistics
12
13- **Total Tests:** 40
14- **Passed:** 38 (95%)
15- **Failed:** 2 (5%)
16- **Ignored:** 0
17- **Filtered Out:** 0
18- **Execution Time:** 8.05s
19
20## Passing Tests (38)
21
22### Common Module Tests (7)
23- `common::relay::tests::test_find_free_port`
24- `common::sync_helpers::tests::test_parse_empty_metrics`
25- `common::sync_helpers::tests::test_parse_counter_with_labels`
26- `common::sync_helpers::tests::test_parse_gauge_without_labels`
27- `common::sync_helpers::tests::test_parse_metric_with_relay_url_label`
28- `common::sync_helpers::tests::test_repo_coord_format`
29- `common::sync_helpers::tests::test_build_layer3_comment_with_uppercase_e`
30
31### Sync Helper Builder Tests (6)
32- `common::sync_helpers::tests::test_build_layer3_comment_kind_1`
33- `common::sync_helpers::tests::test_build_layer3_quote_with_q`
34- `common::sync_helpers::tests::test_build_layer3_comment_kind_1111`
35- `common::sync_helpers::tests::test_build_layer2_issue_event`
36- `common::sync_helpers::tests::test_build_layer3_reply_with_e_tag`
37- `common::sync_helpers::tests::test_build_layer2_issue_with_uppercase_a`
38- `common::sync_helpers::tests::test_build_layer2_issue_with_q_tag`
39
40### Metrics Tests (6 passing)
41- `sync::metrics::test_metric_values_are_numeric`
42- `sync::metrics::test_concurrent_metrics_requests`
43- `sync::metrics::test_metrics_availability_during_sync`
44- `sync::metrics::test_connection_failure_increments_counter`
45- `sync::metrics::test_prometheus_format_valid`
46- `sync::metrics::test_relay_connected_status`
47- `sync::metrics::test_health_state_degrades_on_failure`
48- `sync::metrics::test_startup_sync_event_count`
49
50### Live Sync Tests (3)
51- `sync::live_sync::test_live_sync_layer2_events`
52- `sync::live_sync::test_live_sync_layer3_events`
53- `sync::live_sync::test_live_sync_event_ordering`
54
55### Bootstrap Tests (3)
56- `sync::bootstrap::test_announcement_not_listing_relay_is_not_synced`
57- `sync::bootstrap::test_history_sync_without_negentropy`
58- `sync::bootstrap::test_bootstrap_syncs_existing_layer2_events`
59- `sync::bootstrap::test_relay_replays_events_after_restart`
60
61### Discovery Tests (3)
62- `sync::discovery::test_layer2_discovery_with_chain`
63- `sync::discovery::test_discovers_layer3_via_layer2`
64- `sync::discovery::test_recursive_relay_discovery_syncs_announcement`
65
66### Tag Variations Tests (6)
67- `sync::tag_variations::test_layer2_sync_with_lowercase_a_tag`
68- `sync::tag_variations::test_layer2_sync_with_q_tag`
69- `sync::tag_variations::test_layer2_sync_with_uppercase_a_tag`
70- `sync::tag_variations::test_layer3_sync_with_lowercase_e_tag`
71- `sync::tag_variations::test_layer3_sync_with_q_tag`
72- `sync::tag_variations::test_layer3_sync_with_uppercase_e_tag`
73
74## Failing Tests (2)
75
76### 1. sync::metrics::test_live_sync_event_count
77
78**Location:** `tests/sync/metrics.rs:444`
79
80**Error Type:** Assertion failure
81
82**Details:**
83```
84assertion `left == right` failed: Should have 2 live events
85 left: None
86 right: Some(2)
87```
88
89**Root Cause:** Live event counting metric is not being populated. The metric parser is returning `None` when it should find a count of 2 live synced events.
90
91**Output Sample:**
92```
93Live events synced: None
94```
95
96**Impact:** This suggests that the `sync_events_total{sync_type="live"}` metric either:
97- Is not being incremented correctly during live sync
98- Is using a different metric name/label than expected
99- Is not being exposed in the metrics endpoint
100
101---
102
103### 2. sync::metrics::test_multi_source_aggregate_counts
104
105**Location:** `tests/sync/metrics.rs:603`
106
107**Error Type:** Assertion failure
108
109**Details:**
110```
111assertion `left == right` failed: Should have 0 connected
112 left: Some(1)
113 right: Some(0)
114```
115
116**Root Cause:** After stopping a relay connection, the `sync_relays_connected_total` metric is not being decremented. The test expects 0 connected relays after calling stop, but the metric still shows 1.
117
118**Output Sample:**
119```
120Tracked total: Some(1)
121Connected total: Some(1)
122After stop - Tracked total: Some(1)
123After stop - Connected total: Some(1)
124```
125
126**Impact:** This indicates that relay disconnection is not properly updating the connection count metric. This could be:
127- A lifecycle issue where the metric update happens asynchronously after the test assertion
128- A bug where the disconnect handler doesn't decrement the counter
129- A race condition in the test timing
130
131---
132
133## Analysis
134
135### Test Health
136The sync test suite is in relatively good shape with a 95% pass rate. The failures are both isolated to the metrics module and appear to be either timing/synchronization issues or metric collection bugs rather than fundamental sync logic problems.
137
138### Pre-existing Issues
139Both failing tests appear to be pre-existing issues unrelated to the planned refactoring work. They should be tracked separately and not conflated with any issues introduced during the refactor.
140
141### Refactoring Risk Assessment
142- **Low Risk Areas:** Bootstrap, discovery, live_sync, tag_variations modules are all passing
143- **Medium Risk Area:** Metrics tests have 2 failures, but they're specific to metric collection, not sync functionality
144- **Safe to Refactor:** The core sync logic tests are passing, so structural refactoring of test helpers and organization should not affect test outcomes
145
146## Next Steps
147
148This baseline will be used to:
1491. Verify that refactoring doesn't introduce new failures
1502. Distinguish pre-existing failures from regressions
1513. Track if the refactoring inadvertently fixes the existing failures
1524. Ensure that after refactoring, we still have 38 passing tests (or more if we fix the failing ones)
153
154## Notes
155
156- Both failures are in `tests/sync/metrics.rs`
157- The failures appear to be metric collection/timing issues rather than sync logic bugs
158- All functional sync tests (bootstrap, discovery, live_sync, tag_variations) are passing
159- The refactoring should not affect these test results unless we accidentally change metric collection timing \ No newline at end of file