A user prompt could render at the very bottom of a conversation, below the
assistant turn that had already answered it.
merge_session_messages_append_only reconciles the sidecar with state.db. When a
state.db row carries a typed row id that is unique to state.db and absent from
the sidecar (the two stores do not share an id space),
row_id_preserves_source_multiplicity is true and the row bypasses the
sidecar-timestamp-range block -- including the
_insert_state_message_chronologically call that block owns. It then fell through
to the terminal merged_messages.append and landed at the tail regardless of its
timestamp.
The full display path re-sorts by timestamp afterwards
(_merged_webui_lineage_messages_for_display), which hid the defect. The
paginated path used for the initial render (msg_limit) does not, so the user saw
their own message below its own answer until a full reload reshuffled it.
Place such a row in its chronological slot when its timestamp predates the
sidecar tail, falling back to append when no safe slot exists. Rows that are
genuinely newest still append at the tail, so the append-only contract (never
drop a local row) is preserved.
Verified on the reported transcript: the merged tail now ends on the assistant
conclusion instead of the orphaned prompt, with the same message count.
The webhook second-pass loop in _load_cli_sessions_uncached called
get_last_workspace(profile=_cli_profile) directly per row instead of the
_cli_workspace() closure the main/cron/kanban passes already use, so it
paid its own redundant file reads/is_dir() probe on every webhook row even
though the loop already sits inside the profile-home resolve cache scope.
Flagged in PR #7636 review (greptile): this is the one concrete gap in an
otherwise-correct fix.
Also adds an integration-level regression test proving the actual N+1 this
PR targets -- Session.__init__ constructed via
_state_projection_sidecar_metadata's Session.load_metadata_only() fallback
on a cold sidecar-metadata cache -- is deduped by
@profile_home_resolve_cache_scope() on _load_cli_sessions_uncached, with a
companion test showing the same fixture scales linearly (one filesystem
resolve per row) when the scope is bypassed. This directly answers the
review claim that the cache scope doesn't reach any Session construction:
it does, verified by counting real _safe_resolve() filesystem calls rather
than call counts on the memoizing wrapper alone.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The process-lifetime cache from the previous commit was flagged in PR
review (nesquena-hermes): Path.resolve() is a filesystem call, not a
pure function of process-startup constants. A profile-home symlink can
be retargeted while the long-running webui process stays up, and
_safe_resolve() deliberately falls back to the unresolved input after a
transient OSError/RuntimeError/ValueError -- a process-lifetime cache
would serve either a stale symlink target or a stale fallback forever.
Redesign: replace the module-level dict with a contextvars.ContextVar
that defaults to None (no active scope -> resolve fresh, exactly like
pre-PR behavior). profile_home_resolve_cache_scope() sets it to a fresh
dict for the duration of one call and resets it in a finally block, so
no cache entry survives past that call. It now wraps only
_load_cli_sessions_uncached as a decorator -- the actual N+1 hot loop
(hundreds of Session.__init__ calls per sidebar build, each resolving
the same profile argument). All other call sites of
_resolve_profile_home_param (confirmed via repo-wide grep: all internal
to api/workspace.py, none elsewhere) get fresh resolution on every call,
same as before this PR existed.
Rewrote tests/test_profile_home_resolve_caching.py to prove: (a)
repeated resolves within one scope still hit the cache once, (b) two
separate scopes (simulating two requests) do not share state -- a
symlink change between them is observed on the second scope, and (c)
calls outside any scope never cache at all.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The all-profiles sidebar scan walks every profile's state.db in one
request. When it lazily minted a "Webhooks" / "Cron Jobs" project for a
profile that had webhook/cron rows, ensure_webhook_project() and
ensure_cron_project() tagged the new entry with get_active_profile_name()
-- the UI-selected profile -- instead of the profile whose rows triggered
it. A webhook-only profile therefore leaked a "Webhooks" project onto
whichever profile was selected; deleting it did nothing because the next
sidebar poll re-minted it, and switching profiles minted one per profile.
ensure_webhook_project / ensure_cron_project / _profile_has_user_projects
now take an explicit profile (default: active profile), and the scan
passes _cli_profile through, so the opt-in gate and the tag both key on
the scanned profile. Existing stubs in tests updated to accept the kwargs.
Read the metadata prefix in staged geometric reads so one oversized metadata field no longer forces a full re-parse of the whole sidecar.
Co-authored-by: rodrigogs <rodrigogs@users.noreply.github.com>
Decode the state.db content sentinel in the WebUI message projection so structured/multimodal content renders instead of a placeholder, with an out-of-band tuple identity that no scalar can forge and a per-call ContextVar memo for the canonical serialisation.
Co-authored-by: totalitarian <totalitarian@users.noreply.github.com>
Closes the remote POSIX workspace path preservation gap across Session and streaming. Profile-aware runtime path resolution.
Co-authored-by: alfred-rootson <alfred.wojtasordzik@gmail.com>
Preserve nested request isolation, make the response allowlist single-source,
and strip heavy fields from existing index rows after reconciliation.
Co-authored-by: Martin Dell <martindell@users.noreply.github.com>
* fix(transcript): recognize agent row ids in replay merge
* Strip imported _row_id so replay merge cannot trust caller provenance.
Session import already dropped _state_db_row_id aliases. After treating
Agent _row_id as identity, the same public/import scrubber must drop that
field too, or a crafted import can collide with a later state.db turn.
---------
Co-authored-by: Anthony Ruiz <ruizanthony@users.noreply.github.com>
Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
* fix: prevent kanban rows from hiding CLI sessions
Keep worker-heavy Kanban history out of the bounded interactive session window and recover it through a separate capped pass. Add regression coverage and update query-pass contract tests.
* docs: describe imported session projection bounds
---------
Co-authored-by: zicochaos <23367003+zicochaos@users.noreply.github.com>
Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
* fix(#6335): filter unconfigured providers from model selector
When a known provider (e.g. openai-api) is listed in config.yaml's
providers section with only metadata (name, display_name) and no route
configuration or credentials, it should not appear in the chat model
selector dropdown.
Previously, any provider recognised as a 'known provider' (present in
_PROVIDER_MODELS or _PROVIDER_DISPLAY) would be admitted to the picker
simply by being listed in the config.yaml providers block — even after
the user removed its API key. This caused a persistent ghost provider
entry in the dropdown (#6335).
The fix gates known-provider admission on credential evidence: the
provider must already have been detected from a credential source (env
vars, hermes auth, or credential pool) OR carry explicit route
configuration (api/base_url/api_key/key_env) OR be the active provider
with a configured models allowlist.
* fix(#6338): normalize provider aliases in credential check
The credential check compared _canonical (config key like 'x-ai',
'google') directly against detected_providers (alias-resolved values
like 'xai', 'gemini'). Alias-named authenticated providers were
therefore incorrectly treated as unconfigured and removed from the
model selector.
Fix: resolve _canonical through _resolve_provider_alias() before the
membership test, so config keys like 'x-ai' match credential evidence
reported as 'xai'. Also check the raw canonical for cases where the
config key is itself the alias target.
Refs: #6335, #6338
* fix(#6338): normalize detected_providers through alias resolver
Normalize both _canonical and detected_providers entries through
_resolve_provider_alias before the membership test, so that a WebUI
canonical form in detected_providers (e.g. x-ai added by a prior
loop iteration) also matches the alias-resolved form used in
credential detection (e.g. xai from hermes_cli).
This is belt-and-braces on the existing _resolve_provider_alias(_canonical)
check: both sides now map into the same alias-resolved namespace,
closing any edge case where detected_providers contains a WebUI-canonical
form that happens to differ from the alias target.
* test(ui): catalog-admission regression for credentialed providers (#6338)
* test(ui): pin config path state in catalog-admission regression (#6338)
* fix(system-health): expose runtime memory diagnostics without blocking owner locks (#6351)
* fix(system-health): keep owner contention local to runtime status (#6351)
* docs(changelog): stamp #6338#6363 (certified batch)
---------
Co-authored-by: webtecnica <webtecnica@users.noreply.github.com>
Co-authored-by: webtecnica <webtecnica@gmail.com>
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
PR #6922 made /api/media revalidate on every use, so overwriting a file in
place (same filename) also rewrote every historical chat preview that
referenced it — the old/new comparison was lost.
At settle time, snapshot each local-file MEDIA: reference into a
content-addressed store (<STATE_DIR>/media_snapshots/<sha256>.snap) and stamp
the message with _media_snapshots {path: digest}. The frontend appends
&snap=<digest> to historical preview URLs; /api/media serves the frozen bytes
with immutable caching when the digest is valid, and falls back to the live
file otherwise (eviction/never-captured). The existing path allow/deny checks
still gate the request — snap only selects which bytes to serve, it never
widens access.
- api/media_snapshots.py: content-addressed store (sha256 key, dedup, 50MB
per-file / 2GB total caps with FIFO eviction), MEDIA ref resolver, capture
allow-list mirroring the /api/media roots + #3234 deny list
- streaming.py: annotate settled assistant messages (idempotent, best-effort)
- routes.py: snap= serving branch + download_name override + deny direct
store-path access
- models.py: _media_snapshots registered in display-metadata merge keys
- ui.js: _stampMediaSnapshots post-cache stamping (main transcript,
transparent ordered segments, worklog scene prose) + data-snap threading
through the five lazy preview loaders
25 new tests incl. node-executed frontend behavior tests; reverse-verified.
Ships PR #6757 with a maintainer hardening pass folded in (authorship preserved).
Preserves the exact provider-facing reply text (api_content) through the internal
session store for byte-accurate agent replay, while guaranteeing it never reaches
a browser: every client-facing surface strips it via the public projection.
Maintainer fixes (2 defects my adversarial re-gate reproduced):
- Selective session redaction restored: only transcript fields are scrubbed;
operational fields (e.g. workspace path) pass through unchanged, so a path
containing credential-shaped text is no longer masked into an invalid path.
- Runner-backed SSE relay now projects payloads before emit, closing an
api_content leak through _stream_runner_run_events.
+5 regression tests. Codex adversarial re-gate: SAFE TO SHIP. Full suite clean.
Co-authored-by: franksong2702 <franksong2702@users.noreply.github.com>
* feat(sidebar): add show_kanban_sessions toggle to hide kanban worker sessions
* fix(sidebar): kanban source-filter override + filtered-view cap (cron/webhook parity)
Gate re-review (Codex) found the kanban-hide addition missed cron/webhook
parity for the source-filter case:
- An explicit kanban source_filter now overrides the default hide (threaded
source_filter into _dedupe_cli_sidebar_sessions_for_api, normalized with
strip().lower() to match the loader), so a deliberate kanban view shows
those rows even with the toggle off. Scoped to the matching source only.
- A dedicated KANBAN_PROJECT_CHIP_LIMIT bounds a filtered kanban-only view,
mirroring CRON/WEBHOOK_PROJECT_CHIP_LIMIT.
Deliberately NOT changing the general exclude_sources: the original PR lets
kanban load on the source_filter=None path so the toggle-ON case works (the
hide is applied by _hide_from_default_sidebar only when the toggle is off).
Excluding kanban there would save the 20-row budget but break toggle-on
unless a bounded kanban merge is added (cron/webhook don't do that either) —
that query-budget optimization is deferred as a separate cross-cutting change.
The pinned-survives-hide finding is also out of scope (cron/webhook hide pinned
identically, confirmed by Codex). Adds bite-tested regressions.
Co-authored-by: vduke <vduke@users.noreply.github.com>
---------
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
Render structured background-process completions and watch matches as accessible collapsed cards while preserving raw fallback, verbatim output, and full watch patterns.
Co-authored-by: b3nw <b3nw@duck.com>
Record resolved_model on assistant messages as _usedModel, expose used_model
in the usage SSE payload, and render model chips in transparent and settled
turn footers while suppressing duplicates when gateway routing already shows
used_model.
* fix(webui): keep the content search from evicting the user's working set
/api/sessions/search?content=1 walks EVERY session and pulls each one through
get_session(), which inserts it into the SESSIONS LRU and marks it
recently-used. On any install with more sessions than sessions_cache_max
(default 300), a single search therefore flushes the whole cache and refills it
with sessions the user is not looking at — the classic buffer-pool
scan-pollution problem. The sessions actually open in the UI are exactly the
ones evicted, and the search is keystroke-debounced, so it repeats while typing.
A scan reads each session exactly once, so nothing it touches has earned
"recently used". get_session_for_scan() reuses a resident session without
promoting it, and reads a cold one straight from disk without caching it. It
returns None rather than raising, since a scan skips what it cannot open.
This is a correctness fix for cache behaviour, not a latency fix. The
multi-second searches that led here were contention, not scan cost: a trivial
/api/profiles took 9.2s in the same window, and a full read+parse of ~1700 real
sessions measures ~4s total.
test_sessions_search_depth_validation patched api.routes.get_session. With the
search reading through the scan accessor it now patches get_session_for_scan —
left unfixed, two of its cases fail and the third passes vacuously against an
empty result set.
Validation:
pytest tests/test_issue4765_sessions_lru_eviction.py
tests/test_sessions_search_depth_validation.py -> 13 passed
Both added eviction tests fail on the pre-fix accessor (verified by revert):
the working set drops from 4/4 to 0/4 resident after one scan.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(changelog): stamp #6084 content-search working-set preservation
---------
Co-authored-by: ai-ag2026 <261867348+ai-ag2026@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
Memory: get_state_db_session_messages had no SQL LIMIT — it full-scanned the
state.db messages table and fetchall()'d into a Python list. The display path
(GET /api/session) already has a careful since_timestamp optimization that
bounds the common uncompressed tail load, but it deliberately bails (returns
None) on compressed sessions (truncation_watermark/boundary) and msg_before
paging — in those cases the read used to full-scan with no cap. A
pathological/huge state.db could then materialize unbounded rows into memory.
Add an opt-in `limit` parameter to get_state_db_session_messages (applied as a
SQL LIMIT that keeps the NEWEST rows via a DESC subquery re-sorted ASC). It is
a BACKSTOP, not a semantic window: the display window counts visible rows
post-reconciliation, so a true window LIMIT here would corrupt the
sidecar/state.db merge (see _state_db_since_timestamp_for_limited_display,
which deliberately does NOT SQL-LIMIT raw rows for exactly that reason).
Thread a generous _STATE_DB_DISPLAY_ROW_BACKSTOP (50000) from the GET /api/session
display caller only. The full-history model-context callers (reconciliation at
models.py:3437, context reconstruction at :8576, new-turn context at
streaming.py:8443) do NOT pass a limit — their semantics require the full
history, unchanged.
Test: tests/test_state_db_read_backstop.py — no-limit returns full history
oldest-first (model-context contract unchanged); limit=N keeps the newest N
re-sorted oldest-first; limit > row count returns all; limit<=0 clamps to 1
(newest); invalid limit falls back to unbounded; limit composes with
since_timestamp. Existing state.db active-filter + reconciliation suites (54
tests) unchanged.
Contract Routing
- Task type: memory-bound fix (unbounded state.db row materialization on the
display path's bail cases).
- Touched areas: state.db message reader (api/models.py) + GET /api/session
display caller (api/routes.py). NOT the model-context callers.
- Relevant public docs: AGENTS.md, CONTRIBUTING.md, docs/CONTRACTS.md,
docs/rfcs/webui-run-state-consistency-contract.md (Model context layer).
- State layer mutated: none (read-only). The model-context read path is
UNCHANGED (no limit passed) — Invariant 1 (visible current turns enter model
context) and the reconciliation contract are preserved. Only the display path
gets the defensive cap, which does not affect what the agent receives.
- Scope boundaries: this is narrower than a naive "add LIMIT everywhere" —
_state_db_since_timestamp_for_limited_display's docstring explains why a
semantic window LIMIT would corrupt the merge; this cap is purely a
pathological-input safety net above any legitimate session size.
Release note: GET /api/session's state.db read now has a defensive row backstop,
capping memory when the bounded since_timestamp optimization bails on
compressed/paged sessions.
Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent).
Memory: the CLI/cron session projection cache was a plain dict with TTL storage
but NO size cap and only lazy, same-key eviction. Each value is a copy.deepcopy()
of the full CLI/cron session list (the expensive projection behind #4842/#4672).
The cache key folds in a state.db content fingerprint that advances on every
streamed message, so the next poll builds a NEW key; the previous key's heavy
deepcopy was never looked up again, so its lazy-expiry path never ran. A
long-lived process under churn could accumulate orphaned heavy deepcopies until
the next structural clear_cli_sessions_cache().
Make it an OrderedDict capped at _CLI_SESSIONS_CACHE_MAX_ENTRIES (8) with
drop-oldest on write (while len > MAX: popitem(last=False)) and move_to_end on
fresh hits, mirroring the _CLAUDE_CODE_PARSE_CACHE / _SIDECAR_METADATA_CACHE LRU
pattern in the same file. TTL remains the primary freshness control; the cap is
the backstop the plain dict lacked. clear_cli_sessions_cache() keeps .clear().
Test: tests/test_cli_sessions_cache_cap.py — cache stays at len == MAX after
MAX+5 distinct keys (oldest 5 evicted), a fresh hit marks the entry
most-recently-used so it survives over colder entries, and the cache is an
OrderedDict (the regression: it was an unbounded plain dict). Existing
streaming-cache-TTL/cron-session tests unchanged.
Contract Routing
- Task type: memory-bound fix (unbounded projection cache).
- Touched areas: CLI/cron sidebar projection cache (api/models.py
_CLI_SESSIONS_CACHE).
- Relevant public docs: AGENTS.md, CONTRIBUTING.md, docs/CONTRACTS.md,
docs/rfcs/webui-run-state-consistency-contract.md (Sidebar/session metadata
layer).
- State layer mutated: Sidebar/session metadata layer — specifically the
in-memory projection cache that backs the CLI/cron sidebar rows. The cache is
a read-derivation of state.db content; it is NOT a source of truth. Dropped
entries are rebuilt on the next poll (TTL/invalidation already drive rebuilds).
- Invariant preserved — Invariant 4 (Maintenance is not activity): the cache is a
derived projection; evicting an entry cannot move a session in the sidebar or
change a row's content, because the next poll recomputes the projection from
state.db. TTL remains the primary freshness control; the LRU cap only bounds
how many STALE orphaned deepcopies linger between invalidations. Regression:
test_cli_sessions_cache_cap.py plus the existing streaming-cache-TTL and
cron-session tests (which exercise the real invalidate/rebuild path).
- Scope boundaries: does not change sidebar ordering, unread state, updated_at,
or the projection result. The cap is a memory backstop; correctness still
flows from the existing TTL + clear-on-mutation invalidation.
Release note: the CLI/cron sidebar projection cache is now LRU-bounded, capping
worst-case memory for long-lived processes under heavy streaming churn.
Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent). No AI-specific tool
use mattered beyond the repo's standard read/edit/test workflow.
Codex re-gate asked to distinguish a confirmed-absent sidecar from an
indeterminate/unreadable one before entering the abandoned-shell grace path.
Add _session_sidecar_exists() (True/False/None) and only grace-evict when the
sidecar is confirmed absent (False); a present-or-indeterminate sidecar keeps
the session resident. Age-window bound retained (fully closes the leak, no
data-loss edge); an SSE-mounted-lease refinement is noted as optional follow-up.
Codex gate found the eviction-reorder made never-saved empty shells immortal
(abandoned /api/session/new tabs accumulate unbounded past sessions_cache_max).
Add a grace-window bound: protect a never-persisted empty shell only while fresh
(<30min old) OR while it carries a composer draft; a stale, empty, draftless,
never-saved shell is treated as abandoned and becomes evictable again. A real
compose persists a draft on the first keystroke (disk-backed + reloadable), so
the brick fix is preserved while the leak is closed. +2 regression tests.
Two independent defects that chain into "new conversations cannot be started":
every POST /api/session/draft and /api/chat/start 404s a few seconds after
POST /api/session/new returned 200.
1. Search inputs are parsed as username fields.
index.html contains no <form> at all, but does contain password inputs (the
Settings "Access Password" panel). Chromium therefore groups the document's
unowned fields into one synthetic password form and picks the first text input
in the DOM — #sessionSearch, the sidebar conversation filter — as that form's
username field, then autofills the saved account name into it on load.
autocomplete="off" was already present; it is ignored for credential
heuristics.
The autofill fires oninput -> filterSessions() -> GET /api/sessions/search
?content=1&depth=5: a full content search the user never asked for, repeated
on every page load.
Fix: give the four search inputs type="search" plus the per-manager opt-outs
(data-1p-ignore / data-lpignore / data-bwignore / data-form-type), and give the
Settings password inputs a real <form> owner with proper autocomplete tokens so
Chromium scopes the credential form to them instead of to the whole document.
The form uses display:contents, so layout is unchanged. CSS suppresses WebKit's
native search clear button, since these fields ship their own
(#sessionSearchClear).
2. A never-persisted session is treated as evictable.
new_session() intentionally keeps a session in RAM until its first message
(#1171), so the SESSIONS cache is its only copy. _session_is_evictable()
(#4765) short-circuited on zero messages, reasoning that an empty shell "is
recreated on next access". It is not: get_session() has no recreate path and
raises KeyError, so both routes 404 and the session can never be started.
The content search from (1) pulls every hit through get_session(), which blows
past sessions_cache_max (default 300) and evicts the session being composed.
That is why this reproduces within seconds on an install with ~1700 sessions,
and why it grew worse as the session count rose.
Fix: require proof of persistence before evicting, for empty sessions too —
which is what the function's own docstring already promised ("Its full state is
already persisted to the JSON sidecar"). The zero-message branch was the single
path that bypassed it. #4765's memory bound is unaffected: sessions loaded from
disk remain evictable, and its existing tests still pass.
Trade-off: an unsaved empty session now stays resident for the process
lifetime (one per "New Conversation" click, an empty object each). That is a
deliberate application of the function's stated invariant — "slightly more RAM
for a session we are unsure about is strictly better than evicting an unsaved
session". Bounding it by age instead would also work if maintainers prefer.
Validation:
pytest tests/test_issue4765_sessions_lru_eviction.py -> 9 passed
The added regression test fails on the unpatched predicate (verified by
reverting api/models.py alone) and passes with the fix.
Note: (1) fixes what triggers the churn, (2) fixes what turns that churn into
an unstartable session. Either fix alone leaves a real defect in place, so
they are submitted together.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nesquena-hermes