Files
hermes-webui/api
Anthony Ruiz c6a2c1b7d2 perf(streaming): fold the echo search window once instead of per cut index
`_strip_compact_echo_suffix` probed every candidate cut index and re-folded
the whole remaining tail for each probe, so the cost grew with
`window x suffix` rather than with the data actually inspected. It runs up to
four times per interim assistant message, on the reasoning buffer, the shared
stream mirror and up to two per-message segments.

The practical effect is worst on the longest message of a turn: 297ms for a
200-character message but 5.6s for a 6000-character one, measured on the live
process. That time is spent holding the GIL, so a single long final message
also stalls every other stream served by the same worker. Profiling a live
turn attributed ~5% of total CPU to this one helper.

Fold the window once, then walk backwards across exactly as many non-space
characters as the folded suffix holds. `str.isspace` and the `\s` pattern used
by `_compact_for_echo_compare` agree on all 1114112 Unicode code points, so
the folded view and the walk stay consistent.

Behaviour is unchanged, including the leftmost-cut tie-break on repeated text
and the bounded search window. The new test pins equivalence against a naive
reference oracle over ~2020 cases (exact, partial and absent echoes, mixed
whitespace, unicode, empty and None inputs, repetitions, out-of-window
echoes), asserts the fold no longer runs once per cut index, and caps the
runtime of a long-conclusion call.

Measured: 200 chars 297ms -> 0.149ms, 6000 chars 5645ms -> 0.720ms.
2026-08-27 23:58:18 +00:00
..