* perf(session-load): memoize the compression-lineage display stitch
_webui_sidecar_lineage_messages_for_display re-merged multi-thousand-row
parent+child transcripts on EVERY GET /api/session (measured 5.1s per
request on a real 5.2k-message lineage, py-spy: merge_session_messages_
append_only + _matching_visible_duplicate dominate). The merge result is
now cached per session id, validity keyed by the exact stat signature of
the child sidecar AND every snapshot parent involved, bounded LRU(16).
Any write to any involved sidecar invalidates naturally; cache hits and
the populating call hand out shallow-copied rows so caller-side display
metadata cannot corrupt cached state.
Measured: 5.17s cold -> 0.002s warm on the real session; invalidation
after touch re-merges. Tests: tests/test_lineage_display_cache.py
(correctness, copy isolation, child/parent invalidation, no pollution
for lineage-less sessions).
* perf(api): memoize inactive-session display merges
* perf(api): skip state load on cached transcript reopen
* test(api): remove unused lineage-cache import
* fix(api): reject streaming key without state database
* test(api): isolate display cache fixture
* fix(api): bind cache publication to read signature
* fix(api): harden display cache invalidation
* fix(api): bind cache hits to lineage provenance
* fix(api): close remaining display cache scope races
* fix(api): scope display cache revision to target
* fix(api): evict unverifiable lineage provenance
* fix(api): keep incomplete lineage provenance uncached
---------
Co-authored-by: Anthony Ruiz <ruizanthony@users.noreply.github.com>
* fix(workspace): new chat inherits the current conversation's workspace
`newSession()` resolved the workspace of a new chat as:
switchWs || S._profileDefaultWorkspace || S.session.workspace
`S._profileDefaultWorkspace` is global to the profile, hydrated from
`/api/profile/active`. Preferring it over the current conversation meant a new
chat opened from a MES conversation landed on the profile-global value instead
of MES. With several conversations running in parallel on different workspaces,
the context the user was actually in was systematically lost.
The precedence is worse than it looks, because that "profile default" is not a
stable configured value: `get_profile_default_workspace()` reads
`last_workspace.txt` BEFORE the configured key (api/workspace.py ~419-427), and
for the `default` profile that file is the GLOBAL one. `set_last_workspace()`
rewrites it on every `/api/chat/start` and `/api/session/update` of ANY
conversation. So "profile default wins over the current session" degrades in
practice to "whatever workspace another conversation last used wins over the
conversation I am in" -- observed live: the pointer flipped mid-operation
because a concurrent turn wrote to it.
New precedence:
switchWs || S.session.workspace || S._profileDefaultWorkspace
Preserved invariants:
- the one-shot profile-switch flag still wins first and is still consumed (#823);
- a blank page (no loaded session) still falls back to the profile default,
which remains required for the empty-composer workspace chip (#804/#5169);
- `_profileDefaultWorkspace` is never nulled by `newSession()` (#823).
This deliberately diverges from upstream #4755, which made the profile default
win. That decision assumed a stable configured default; on a shared/global
`last_workspace.txt` it inverts its own intent. The upstream test is rewritten
with the rationale inline so a future rebase does not silently restore the bug,
plus a new test that evaluates the real expression under node and proves the
blank-page fallback still holds.
Pre-existing failures in tests/test_workspace_stale_recovery.py (3) were
verified identical with and without this change.
* fix(workspace): recover stale inherited workspace
---------
Co-authored-by: Anthony Ruiz <ruizanthony@users.noreply.github.com>
The periodic checkpoint thread fires whenever a tool call completes and
unconditionally rewrites the whole session sidecar. A completed tool call
is not proof that anything the checkpoint persists actually changed:
`run_conversation()` mutates an internal copy of the transcript, and the
turn bookkeeping fields (`active_stream_id`, `pending_user_message`,
`pending_started_at`) are written once before the run starts. So the
recurring rewrite normally re-persists byte-identical state.
Each rewrite serializes the entire session while holding the GIL, so its
cost scales with transcript size. Because the HTTP server and the agent
workers share one interpreter, that cost is added latency for every
concurrent request — not just for the session being written.
Measured on this deployment (1,622 sidecars, 6.65 GB, median 2.31 MB,
28.8% above 5 MB, max 35.9 MB):
sidecar write per 10-min turn (40 wakeups)
1.8 MB ~100 ms 4.1 s -> 0.2 s
24.7 MB ~1.4 s 41.6 s -> 2.1 s
A checkpoint now writes only when the state it persists actually moved,
compared through a cheap fingerprint that allocates nothing. The
fingerprint covers every field the checkpoint can legitimately advance
mid-run: session identity (compression rotates the SID), turn
bookkeeping, and the length of the persisted message projections.
Fail-closed on both sides: an unreadable fingerprint returns None and
forces the write, and a 5-minute idle refresh keeps `updated_at` from
going stale on a long quiet turn (the sidebar sorts on it). Crash
recovery is unchanged — any real mutation is still persisted within one
15 s interval.
Co-authored-by: Anthony Ruiz <ruizanthony@users.noreply.github.com>
* fix(transcript): recognize agent row ids in replay merge
* Strip imported _row_id so replay merge cannot trust caller provenance.
Session import already dropped _state_db_row_id aliases. After treating
Agent _row_id as identity, the same public/import scrubber must drop that
field too, or a crafted import can collide with a later state.db turn.
---------
Co-authored-by: Anthony Ruiz <ruizanthony@users.noreply.github.com>
Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
* fix(branch): slice fork transcript in GET /api/session merged-view coordinates
keep_count sent by forkFromMessage is an index into the merged display
view (sidecar lineage stitched across compression snapshots, append-only
merged with state.db rows, parent backfill). The branch handler sliced the
raw sidecar instead, which diverges from that view in both length and
content after compactions: forks from the final conclusion landed rows
too early, ending the fork mid tool-run and dropping the conclusion.
Build source_messages through the exact same pipeline as the
GET /api/session load_messages path for non-messaging sessions.
Regression test: test_branch_route_slices_merged_display_view_not_raw_sidecar
(fails on the old code with the fork ending on a tool row).
* fix(branch): align fork source with display bounds
---------
Co-authored-by: Anthony Ruiz <ruizanthony@users.noreply.github.com>
Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
