0 Fork 0

Files

T

happy5318 dcac71c74e fix(run-journal): match echo suffixes through an incremental folded index

Review follow-up for #7569: change 4 of the live-snapshot perf PR replaced
one quadratic with another. The window-free backward walk is correct, but it
re-walks the raw reasoning transcript's whitespace once per interim event —
on a whitespace-heavy transcript that span is megabytes, and the replay went
~55x slower than master on the production shape the gate measured
(1MB / 20-interim journal: 0.035s master → 1.922s branch).

## Thinking Path

The gate's finding was that the cost is structural, not a tweak: the matcher
consults the *whole* accumulated transcript on every interim event, so any
raw-text rescan of it is O(interim_count x whitespace_span). The fix is the
data-structure change the review asked for — maintain the folded view and the
raw cut offsets incrementally, and match/strip from that indexed state
instead of rescanning the raw buffer.

## What Changed

- `api/streaming.py`: new `_CompactEchoIndex` — an append-only folded index
  that records, per non-whitespace character, its raw index. `append` folds a
  chunk once (cost proportional to the chunk, never the buffer),
  `matches_tail`/`cut_to` compare only the candidate against the folded tail
  (O(len(candidate)), no raw walk), `reset` drops the state after the raw
  text is truncated. The old raw helpers (`_find_compact_echo_suffix_start`,
  `_strip_compact_echo_suffix`) stay as the equivalence oracle.
- `api/routes.py::_run_journal_live_snapshot`: reasoning chunks feed the
  index as they are appended; `reasoning_echo_tail_matches` and
  `strip_reasoning_echo_tail` consult the index. The strip re-indexes the
  truncated transcript so later probes match against it.
- `api/streaming.py` live path: the same swap for the live-stream echo strip.
  The three live reasoning buffers (STREAM_REASONING_TEXT, the coalescing
  buffer, the per-message segments) each carry a mirrored index, fed on every
  append and re-indexed after a strip. The buffer flush resets the buffer's
  index with the text it drops, so the index never describes text that is no
  longer there.
- `docs/rfcs/webui-run-state-consistency-contract.md`: the echo-match clause
  now names the indexed representation and the scaling regression that pins
  it.

## Why It Matters

Removing one quadratic by introducing another in the same hot path is the
exact risk the gate flagged — the headline benchmark used a journal shape
where the window-free walk's cost does not dominate, so the aggregate number
still looked like a win. The index keeps the original 8.9x and adds the
scaling guarantee on the shape that was regressing.

## Verification

- Measured on the reproduced shape (1MB whitespace span, 20 interim events):
  0.972s → 0.049s. Doubling both span and interim count scales ~2x per step
  (x1 0.041s / x2 0.090s / x4 0.175s) — linear, not quadratic.
- Behaviour equivalence: 300 randomized journals (whitespace-heavy,
  Unicode, chunked, flagged/unflagged interims) rebuilt with the indexed
  implementation and with the raw walk swapped in — sha256 of the canonical
  snapshot JSON matched on all 300.
- `tests/test_compact_echo_index.py` (new, 6 tests): oracle equivalence over
  2000+ randomized cases, chunked-append equivalence, matches_tail/cut_to
  agreement, reset-after-truncation, and two cost probes that fail the raw
  walk.
- `tests/test_live_snapshot_echo_scan_scaling.py` (new, 4 tests): the
  proportional-scaling regression the review asked for — verified to FAIL on
  the pre-fix matcher (14.6x time for 4x rows, the quadratic signature) and
  pass on the indexed one.
- Existing echo/journal suites unchanged and green: 19 items across
  `test_compact_echo_suffix_linear_scan` / `test_live_snapshot_echo_dedup` /
  `test_live_snapshot_single_parse`, plus 66 neighbouring items
  (`test_issue1361_cancel_data_loss`, `test_issue2565_reasoning_accumulation`,
  `test_issue2713_streaming_segment_flush`, `test_cancel_interrupt`,
  `test_run_journal`).

## Contract Change

None. The matcher's observable behaviour is unchanged (same cut points, same
whitespace-insensitive semantics); only its cost profile changes. No config
knobs, no API surface.

## Risks

- Index drift: if a future writer mutates one of the reasoning buffers
  without feeding (or resetting) its index, the matcher would silently miss
  echoes. Mitigated by keeping the index feed adjacent to every existing
  write site and by the reset-on-flush/reset-on-strip symmetry; the
  equivalence test would catch a dropped feed as a missed echo.
- Memory: the index stores one char plus one int per non-whitespace
  character. On a whitespace-heavy transcript this is bounded by the folded
  length, which is what the transcript would cost folded anyway; the raw
  text is still the larger object.

## Model Used

Hermes Agent (step-5-preview) with direct file edits and pytest verification.
Cost properties were measured with a purpose-built scaling probe (proportional
span x interim count), not a fixed-size fixture; the probe is kept as a
regression test rather than a script.

## Related

#7569 (this PR), #7310 (slow /api/session under load), #7534 (incremental
journal read cache — complementary), #7273/#7291 (metering write-side skip).

2026-09-21 12:34:30 +08:00

17 KiB

Raw Blame History

WebUI Run State Consistency Contract

Status: Proposed
Author: @franksong2702
Created: 2026-05-16
Updated: 2026-08-22
Tracking issue: #2361
Related architecture: #1925, hermes-run-adapter-contract.md, stable-assistant-turn-anchors.md

Problem

A single WebUI agent turn is represented by several overlapping state layers:

the visible transcript the user can read,
the model context / context_messages the agent actually receives,
pending_user_message and active stream metadata,
live SSE events and in-memory stream state,
durable run journal / replay state,
automatic compression summaries and active-task handoff text,
the browser's live timeline DOM/cache,
sidebar ordering, unread state, and updated_at metadata.

Those layers are not independent. When they drift apart, the user sees failures that look unrelated: a prompt is visible but missing from recovered model context, a live run loses or reorders thinking/tool cards after switching sessions, cleanup makes old sessions look newly active, replay duplicates content, or automatic compression reference material appears inside the active turn.

This RFC defines a consistency contract for those layers. It complements the larger run adapter direction in #1925 by documenting what must remain coherent while WebUI still has multiple overlapping state stores.

Inactive compression continuation recovery

The Agent profile's SQLite compression lineage owns the canonical continuation, including when Desktop/CLI compressed a session without updating WebUI's pre_compression_snapshot sidecar flag. GET /api/session may expose the existing continuation_session_id hint from that read-only lineage. Automatic idle_timeout closure does not hide the continuation; explicit/unknown terminal reasons, foreign-profile rows and delegated/tool children do not authorize it. This read does not reopen sessions or copy ancestor display history into context.

A stale POST /api/chat/start returns HTTP 409 with code=session_rotated and the continuation hint before workspace, model, pending-turn or worker mutation. The browser loads the continuation through normal session access checks and restores the rejected text and attachments as a draft. The user sends again explicitly; there is no automatic POST replay or migration of the parent's workspace binding. Clients without this handling must reload the session before retrying. Server wakeups, regeneration semantics and Gateway routing are not silently retargeted by this recovery path.

Goals

Define the state layers involved in active and recovered WebUI turns.
Make the source-of-truth expectations explicit for each layer.
Give reviewers a checklist for streaming, replay, compression, recovery, model-context, and sidebar changes.
Map recent real issues to reusable invariants so future fixes do not solve the same class of bug one symptom at a time.

Non-goals

Do not implement a runner process, sidecar, or new runtime boundary here.
Do not replace #1925 or the run adapter contract.
Do not rewrite the streaming protocol in this RFC.
Do not reopen already-fixed narrow bugs.
Do not make this a catch-all for unrelated UI polish.

Current implementation relationship

Stable Assistant Turn Anchors now implement the presentation/reconciliation portion of this contract for one assistant turn. The run journal and settled transcript provide durable observations; the Anchor registry and activity_scene_v1 reconcile those observations into Compact Worklog, Transparent Stream, or Final answer only; S.messages, INFLIGHT, renderer caches, and DOM remain projections or recovery caches rather than independent semantic owners.

This RFC remains Proposed because its broader cross-layer contract also covers model-context reconstruction, compression handoff, session metadata, and future runtime-adapter migration. Shipped Anchor coverage strengthens invariants 2, 3, and 5; it does not mark every run-state boundary implemented.

State Layers

Layer	Purpose	Source-of-truth expectation	Must not do
Visible transcript	Shows what the user and assistant said	Session transcript plus live replay should produce one chronological user-visible story	Hide the user turn that started active work, or show internal recovery text as current user intent
Model context / `context_messages`	Supplies conversation state to the agent	Must include the current visible user turn unless deliberately excluded with a user-visible reason	Let the agent resume from context that contradicts what the user can see
Pending turn metadata	Bridges submitted-but-not-yet-finalized user input	Must identify the user turn and stream that own active work	Become a permanent duplicate transcript row after recovery
Live stream / SSE	Delivers active runtime events to the browser	Must remain an observation path, not the only durable truth for already-emitted events	Lose the visible scene on refresh, reconnect, or session switch
Worker lifecycle registry (`ACTIVE_RUNS`)	Tracks whether a worker still occupies the session, so a successor turn cannot start on top of it	Broader than "attachable UI work": a cancelled worker stays registered while it unwinds	Be read directly as the set of runs a browser may attach to
Run journal / replay	Rebuilds emitted runtime events after reconnect or restart	Must be cursor-safe and idempotent	Duplicate assistant text, thinking text, tool cards, or compression cards
Compression summary / handoff	Gives the agent recovery context after automatic compression	Must remain agent-facing recovery material unless explicitly rendered as history	Pollute the active turn or become implicit current user intent
Live UI scene/cache	Preserves expanded rows, in-progress cards, local scroll, and transient grouping	May optimize presentation but must be rebuildable or degradable from transcript/replay	Become the only place where chronological ordering exists
Sidebar/session metadata	Helps the user find active and recent sessions	Must reflect meaningful user or assistant activity	Treat background cleanup as a fresh user-facing update

Core Invariants

Visible current turns enter model context. If the user can see a current prompt and WebUI asks the model to continue that work, the prompt must be in the reconstructed model context unless WebUI shows an explicit reason it was excluded.
Active turn UI keeps its owner. The user turn that started active work must remain visible before assistant text, thinking cards, tool cards, or activity groups that belong to that work.
Reattach preserves order or degrades clearly. Refresh, reconnect, and session switch must preserve chronological live-scene order. If WebUI cannot restore the exact live scene, it should downgrade to an explicit structured replay state instead of silently reordering content.
Maintenance is not activity. Runtime maintenance such as stale-stream cleanup, orphan repair, or background compression must not refresh sidebar ordering, unread markers, or active-session affordances as if the user or assistant just acted.
Replay is idempotent. Replaying a run from a cursor must not duplicate transcript rows, thinking content, interim assistant text, tool cards, or compression cards. Replayed long-task events should enter the same browser-facing timeline renderer as live SSE events so recovery does not downgrade a structured Thinking / progress / tool / compression turn into a separate flattened presentation. When session loading combines a WebUI sidecar with Hermes Agent state.db, a native-image user turn may appear as both rich multipart content and scalar text that replaces each image part with [screenshot]. Reconciliation may treat those rows as one turn only when the multipart value contains text and recognized native-image parts, its exact scalar projection matches, role and tool shape match, timestamps match exactly, stable IDs and provider metadata do not conflict, and the pairing is unambiguous. Keep the rich sidecar row; if any requirement is missing or contradictory, preserve both rows rather than deduplicating. Literal scalar [screenshot] text alone is not identity evidence. Visible interim assistant progress must remain visible timeline content; a compact Activity disclosure may summarize adjacent tool/debug detail, but it must not be the only place where the user can see emitted progress text. Interim assistant text that duplicates the tail of the accumulated reasoning transcript is stripped from the reasoning copy so the restored snapshot shows the content once. That echo match is whitespace-insensitive and carries no fixed search window: a compact-equivalent suffix is recognized however much interior whitespace stretches its raw span. Both consumers (the live-stream echo path and the journal replay in api/routes.py) match through an incremental folded index (_CompactEchoIndex in api/streaming.py): the folded view and its raw cut offsets are built as each chunk is appended, so a probe costs O(len(candidate)) and never rescans the transcript's whitespace. The retired windowed variants could drop the strip when the span exceeded the window, duplicating the interim text; the retired raw backward walk was correct but re-walked the span per interim event, quadratic on whitespace-heavy transcripts. Regressions: tests/test_live_snapshot_echo_dedup.py pins the single-occurrence result, tests/test_live_snapshot_echo_scan_scaling.py pins the scaling property (a fixed-size fixture cannot catch a per-interim rescan), and tests/test_compact_echo_index.py pins index/oracle equivalence.
Compression is not current intent. Automatic compression summaries and reference cards are recovery/handoff material. They must not be treated as a new user request, active-turn content, or the default visible explanation for the current answer. Automatic compression may appear during a live turn only as a quiet, non-interactive context divider in the Worklog timeline, not as a clickable tool row. It should use action wording: Compressing context while active and Context auto-compressed when the agent has continued past the compression barrier or when a completion event arrives. The timer is diagnostic detail, not the source of truth for the divider's running state. Later tool, reasoning, or interim assistant events prove the compression barrier has passed even if no explicit completion event was delivered. Settled final history should omit live-only automatic-compression rows unless there is a user-visible recovery or error state to explain.
Observation has a degraded path. Long-running or many-session observation should expose enough heartbeat/degraded status that the UI does not appear silent and ordinary APIs do not stall behind active streams.
Every mutation names its layer. A PR touching streaming, recovery, context reconstruction, compression, replay, or sidebar metadata should state which layer it changes and what regression proves the invariant still holds.
Lifecycle-busy is not client-attachable. ACTIVE_RUNS answers "may a new turn start?", not "may a browser attach a renderer?". Cancellation splits the two: cancel_stream() keeps the row as phase="cancelling" so a successor cannot overlap the unwinding worker, but the client has already reached a terminal state for that stream because its run journal ends in a terminal event. Recovery paths that hand a stream id to a renderer — session SSE recovery and hidden-tab status polling — must therefore exclude cancelling rows, while busy/admission checks must keep counting them. Reading the registry with a single meaning resurrects a cancelled run on every fresh subscription: the client attaches, consumes the terminal event, tears the renderer down, resubscribes, and the loop repeats indefinitely.

Because a cancelling row can otherwise persist forever, cancellation unwind is bounded: a cancelling row older than that window and owning no live STREAMS channel is reclaimed from ACTIVE_RUNS along with its stream-owner entry, so a wedged worker cannot suppress background wakeups permanently. Reclamation requires both conditions — age alone must not evict a row that still owns a live channel. Staleness is measured from the cancellation timestamp (falling back to run start), so a long-running turn cancelled moments ago is never mistaken for an orphan.

Review Checklist

Use this checklist for PRs that touch run state, streaming, replay, compression, context reconstruction, or session metadata:

Which state layers does this PR read or write?
Which layer is the source of truth after this change?
Can the visible transcript and model context diverge? If yes, is that deliberate and user-visible?
What happens after browser refresh, session switch, SSE reconnect, and WebUI restart?
Does replay rebuild the same scene without duplicates?
Does replay use the same timeline-rendering path as live SSE for thinking, interim assistant text, tool cards, compression cards, and terminal states?
Can this change move a session in the sidebar without meaningful user or assistant activity?
Does this change read ACTIVE_RUNS for admission ("may a turn start?") or for attachment ("may a browser render this?"), and does it use the matching predicate for that question?
If it introduces or changes a reclamation window, what proves an in-flight cancellation is not evicted early, and that a wedged one is eventually freed?
Can automatic compression or recovery text become visible active-turn content?
What test or manual evidence proves the invariant?

Existing Issue Map

Example	State boundary exposed	Relevant invariant
#2341 / #2342	Active reattach could show agent activity without the pending user turn that started it	2
#2344 / #2347	Session switching could lose or reorder the live thinking/tool/interim timeline	3, 5
#2345 / #2349	Stale stream cleanup could mutate `updated_at` and resurface old sessions	4
#2346 / #2348	Thinking cards could repeat interim assistant progress text	5
#2353 / #2354	Recovered pending user turns could be visible but missing from model context	1
#2355 / #2357	Auto-compression rotation could leave reference-only cards in the active conversation tail	3, 6
#2308 / #2309	Compressed sessions could resume stale agent tasks when the user starts an ordinary fresh chat	6
#2283	Run event journal replay provides the foundation for ordered recovery	5

These references are evidence for the contract. This RFC does not make the linked implementation PRs dependent on this document, and it does not close the tracking issue by itself.

Relationship To The Run Adapter RFC

The run adapter RFC defines the longer-term event/control boundary for WebUI and Hermes runtime ownership. This RFC defines the consistency rules that the current WebUI and any future adapter-backed implementation must preserve.

The two documents should be read together:

The adapter contract answers: "Where should execution ownership live?"
This consistency contract answers: "How do transcript, context, streams, replay, compression, and UI metadata stay coherent while execution is active or being recovered?"

Rollout Plan

Land this RFC as a reviewable draft and refine it through PR discussion.
Link future streaming/recovery/compression/sidebar PRs back to the invariant they intentionally preserve or change.
Convert recurring checklist items into focused regression tests where practical.
If #1925 introduces a new adapter-backed runtime layer, update this RFC or replace it with the accepted implementation contract so these invariants do not live only in historical discussion.

17 KiB Raw Blame History