084f70df360a67fc458ff63874d37418ae620fc0
1
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 happy5318
|
dcac71c74e |
fix(run-journal): match echo suffixes through an incremental folded index
Review follow-up for #7569: change 4 of the live-snapshot perf PR replaced one quadratic with another. The window-free backward walk is correct, but it re-walks the raw reasoning transcript's whitespace once per interim event — on a whitespace-heavy transcript that span is megabytes, and the replay went ~55x slower than master on the production shape the gate measured (1MB / 20-interim journal: 0.035s master → 1.922s branch). ## Thinking Path The gate's finding was that the cost is structural, not a tweak: the matcher consults the *whole* accumulated transcript on every interim event, so any raw-text rescan of it is O(interim_count x whitespace_span). The fix is the data-structure change the review asked for — maintain the folded view and the raw cut offsets incrementally, and match/strip from that indexed state instead of rescanning the raw buffer. ## What Changed - `api/streaming.py`: new `_CompactEchoIndex` — an append-only folded index that records, per non-whitespace character, its raw index. `append` folds a chunk once (cost proportional to the chunk, never the buffer), `matches_tail`/`cut_to` compare only the candidate against the folded tail (O(len(candidate)), no raw walk), `reset` drops the state after the raw text is truncated. The old raw helpers (`_find_compact_echo_suffix_start`, `_strip_compact_echo_suffix`) stay as the equivalence oracle. - `api/routes.py::_run_journal_live_snapshot`: reasoning chunks feed the index as they are appended; `reasoning_echo_tail_matches` and `strip_reasoning_echo_tail` consult the index. The strip re-indexes the truncated transcript so later probes match against it. - `api/streaming.py` live path: the same swap for the live-stream echo strip. The three live reasoning buffers (STREAM_REASONING_TEXT, the coalescing buffer, the per-message segments) each carry a mirrored index, fed on every append and re-indexed after a strip. The buffer flush resets the buffer's index with the text it drops, so the index never describes text that is no longer there. - `docs/rfcs/webui-run-state-consistency-contract.md`: the echo-match clause now names the indexed representation and the scaling regression that pins it. ## Why It Matters Removing one quadratic by introducing another in the same hot path is the exact risk the gate flagged — the headline benchmark used a journal shape where the window-free walk's cost does not dominate, so the aggregate number still looked like a win. The index keeps the original 8.9x and adds the scaling guarantee on the shape that was regressing. ## Verification - Measured on the reproduced shape (1MB whitespace span, 20 interim events): 0.972s → 0.049s. Doubling both span and interim count scales ~2x per step (x1 0.041s / x2 0.090s / x4 0.175s) — linear, not quadratic. - Behaviour equivalence: 300 randomized journals (whitespace-heavy, Unicode, chunked, flagged/unflagged interims) rebuilt with the indexed implementation and with the raw walk swapped in — sha256 of the canonical snapshot JSON matched on all 300. - `tests/test_compact_echo_index.py` (new, 6 tests): oracle equivalence over 2000+ randomized cases, chunked-append equivalence, matches_tail/cut_to agreement, reset-after-truncation, and two cost probes that fail the raw walk. - `tests/test_live_snapshot_echo_scan_scaling.py` (new, 4 tests): the proportional-scaling regression the review asked for — verified to FAIL on the pre-fix matcher (14.6x time for 4x rows, the quadratic signature) and pass on the indexed one. - Existing echo/journal suites unchanged and green: 19 items across `test_compact_echo_suffix_linear_scan` / `test_live_snapshot_echo_dedup` / `test_live_snapshot_single_parse`, plus 66 neighbouring items (`test_issue1361_cancel_data_loss`, `test_issue2565_reasoning_accumulation`, `test_issue2713_streaming_segment_flush`, `test_cancel_interrupt`, `test_run_journal`). ## Contract Change None. The matcher's observable behaviour is unchanged (same cut points, same whitespace-insensitive semantics); only its cost profile changes. No config knobs, no API surface. ## Risks - Index drift: if a future writer mutates one of the reasoning buffers without feeding (or resetting) its index, the matcher would silently miss echoes. Mitigated by keeping the index feed adjacent to every existing write site and by the reset-on-flush/reset-on-strip symmetry; the equivalence test would catch a dropped feed as a missed echo. - Memory: the index stores one char plus one int per non-whitespace character. On a whitespace-heavy transcript this is bounded by the folded length, which is what the transcript would cost folded anyway; the raw text is still the larger object. ## Model Used Hermes Agent (step-5-preview) with direct file edits and pytest verification. Cost properties were measured with a purpose-built scaling probe (proportional span x interim count), not a fixed-size fixture; the probe is kept as a regression test rather than a script. ## Related #7569 (this PR), #7310 (slow /api/session under load), #7534 (incremental journal read cache — complementary), #7273/#7291 (metering write-side skip). |