Files
hermes-webui/tests
ruizanthonyandAnthony Ruiz 0e5a0fadae perf(streaming): skip redundant sidecar rewrites during a turn (#7172)
The periodic checkpoint thread fires whenever a tool call completes and
unconditionally rewrites the whole session sidecar. A completed tool call
is not proof that anything the checkpoint persists actually changed:
`run_conversation()` mutates an internal copy of the transcript, and the
turn bookkeeping fields (`active_stream_id`, `pending_user_message`,
`pending_started_at`) are written once before the run starts. So the
recurring rewrite normally re-persists byte-identical state.

Each rewrite serializes the entire session while holding the GIL, so its
cost scales with transcript size. Because the HTTP server and the agent
workers share one interpreter, that cost is added latency for every
concurrent request — not just for the session being written.

Measured on this deployment (1,622 sidecars, 6.65 GB, median 2.31 MB,
28.8% above 5 MB, max 35.9 MB):

  sidecar    write    per 10-min turn (40 wakeups)
  1.8 MB    ~100 ms    4.1 s  ->  0.2 s
  24.7 MB   ~1.4 s    41.6 s  ->  2.1 s

A checkpoint now writes only when the state it persists actually moved,
compared through a cheap fingerprint that allocates nothing. The
fingerprint covers every field the checkpoint can legitimately advance
mid-run: session identity (compression rotates the SID), turn
bookkeeping, and the length of the persisted message projections.

Fail-closed on both sides: an unreadable fingerprint returns None and
forces the write, and a 5-minute idle refresh keeps `updated_at` from
going stale on a long quiet turn (the sidebar sorts on it). Crash
recovery is unchanged — any real mutation is still persisted within one
15 s interval.

Co-authored-by: Anthony Ruiz <ruizanthony@users.noreply.github.com>
2026-08-20 04:47:38 -07:00
..