Closes the remote POSIX workspace path preservation gap across Session and streaming. Profile-aware runtime path resolution.
Co-authored-by: alfred-rootson <alfred.wojtasordzik@gmail.com>
Preserve nested request isolation, make the response allowlist single-source,
and strip heavy fields from existing index rows after reconciliation.
Co-authored-by: Martin Dell <martindell@users.noreply.github.com>
* fix(transcript): recognize agent row ids in replay merge
* Strip imported _row_id so replay merge cannot trust caller provenance.
Session import already dropped _state_db_row_id aliases. After treating
Agent _row_id as identity, the same public/import scrubber must drop that
field too, or a crafted import can collide with a later state.db turn.
---------
Co-authored-by: Anthony Ruiz <ruizanthony@users.noreply.github.com>
Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
* fix: prevent kanban rows from hiding CLI sessions
Keep worker-heavy Kanban history out of the bounded interactive session window and recover it through a separate capped pass. Add regression coverage and update query-pass contract tests.
* docs: describe imported session projection bounds
---------
Co-authored-by: zicochaos <23367003+zicochaos@users.noreply.github.com>
Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
* fix(#6335): filter unconfigured providers from model selector
When a known provider (e.g. openai-api) is listed in config.yaml's
providers section with only metadata (name, display_name) and no route
configuration or credentials, it should not appear in the chat model
selector dropdown.
Previously, any provider recognised as a 'known provider' (present in
_PROVIDER_MODELS or _PROVIDER_DISPLAY) would be admitted to the picker
simply by being listed in the config.yaml providers block — even after
the user removed its API key. This caused a persistent ghost provider
entry in the dropdown (#6335).
The fix gates known-provider admission on credential evidence: the
provider must already have been detected from a credential source (env
vars, hermes auth, or credential pool) OR carry explicit route
configuration (api/base_url/api_key/key_env) OR be the active provider
with a configured models allowlist.
* fix(#6338): normalize provider aliases in credential check
The credential check compared _canonical (config key like 'x-ai',
'google') directly against detected_providers (alias-resolved values
like 'xai', 'gemini'). Alias-named authenticated providers were
therefore incorrectly treated as unconfigured and removed from the
model selector.
Fix: resolve _canonical through _resolve_provider_alias() before the
membership test, so config keys like 'x-ai' match credential evidence
reported as 'xai'. Also check the raw canonical for cases where the
config key is itself the alias target.
Refs: #6335, #6338
* fix(#6338): normalize detected_providers through alias resolver
Normalize both _canonical and detected_providers entries through
_resolve_provider_alias before the membership test, so that a WebUI
canonical form in detected_providers (e.g. x-ai added by a prior
loop iteration) also matches the alias-resolved form used in
credential detection (e.g. xai from hermes_cli).
This is belt-and-braces on the existing _resolve_provider_alias(_canonical)
check: both sides now map into the same alias-resolved namespace,
closing any edge case where detected_providers contains a WebUI-canonical
form that happens to differ from the alias target.
* test(ui): catalog-admission regression for credentialed providers (#6338)
* test(ui): pin config path state in catalog-admission regression (#6338)
* fix(system-health): expose runtime memory diagnostics without blocking owner locks (#6351)
* fix(system-health): keep owner contention local to runtime status (#6351)
* docs(changelog): stamp #6338#6363 (certified batch)
---------
Co-authored-by: webtecnica <webtecnica@users.noreply.github.com>
Co-authored-by: webtecnica <webtecnica@gmail.com>
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
PR #6922 made /api/media revalidate on every use, so overwriting a file in
place (same filename) also rewrote every historical chat preview that
referenced it — the old/new comparison was lost.
At settle time, snapshot each local-file MEDIA: reference into a
content-addressed store (<STATE_DIR>/media_snapshots/<sha256>.snap) and stamp
the message with _media_snapshots {path: digest}. The frontend appends
&snap=<digest> to historical preview URLs; /api/media serves the frozen bytes
with immutable caching when the digest is valid, and falls back to the live
file otherwise (eviction/never-captured). The existing path allow/deny checks
still gate the request — snap only selects which bytes to serve, it never
widens access.
- api/media_snapshots.py: content-addressed store (sha256 key, dedup, 50MB
per-file / 2GB total caps with FIFO eviction), MEDIA ref resolver, capture
allow-list mirroring the /api/media roots + #3234 deny list
- streaming.py: annotate settled assistant messages (idempotent, best-effort)
- routes.py: snap= serving branch + download_name override + deny direct
store-path access
- models.py: _media_snapshots registered in display-metadata merge keys
- ui.js: _stampMediaSnapshots post-cache stamping (main transcript,
transparent ordered segments, worklog scene prose) + data-snap threading
through the five lazy preview loaders
25 new tests incl. node-executed frontend behavior tests; reverse-verified.
Ships PR #6757 with a maintainer hardening pass folded in (authorship preserved).
Preserves the exact provider-facing reply text (api_content) through the internal
session store for byte-accurate agent replay, while guaranteeing it never reaches
a browser: every client-facing surface strips it via the public projection.
Maintainer fixes (2 defects my adversarial re-gate reproduced):
- Selective session redaction restored: only transcript fields are scrubbed;
operational fields (e.g. workspace path) pass through unchanged, so a path
containing credential-shaped text is no longer masked into an invalid path.
- Runner-backed SSE relay now projects payloads before emit, closing an
api_content leak through _stream_runner_run_events.
+5 regression tests. Codex adversarial re-gate: SAFE TO SHIP. Full suite clean.
Co-authored-by: franksong2702 <franksong2702@users.noreply.github.com>
* feat(sidebar): add show_kanban_sessions toggle to hide kanban worker sessions
* fix(sidebar): kanban source-filter override + filtered-view cap (cron/webhook parity)
Gate re-review (Codex) found the kanban-hide addition missed cron/webhook
parity for the source-filter case:
- An explicit kanban source_filter now overrides the default hide (threaded
source_filter into _dedupe_cli_sidebar_sessions_for_api, normalized with
strip().lower() to match the loader), so a deliberate kanban view shows
those rows even with the toggle off. Scoped to the matching source only.
- A dedicated KANBAN_PROJECT_CHIP_LIMIT bounds a filtered kanban-only view,
mirroring CRON/WEBHOOK_PROJECT_CHIP_LIMIT.
Deliberately NOT changing the general exclude_sources: the original PR lets
kanban load on the source_filter=None path so the toggle-ON case works (the
hide is applied by _hide_from_default_sidebar only when the toggle is off).
Excluding kanban there would save the 20-row budget but break toggle-on
unless a bounded kanban merge is added (cron/webhook don't do that either) —
that query-budget optimization is deferred as a separate cross-cutting change.
The pinned-survives-hide finding is also out of scope (cron/webhook hide pinned
identically, confirmed by Codex). Adds bite-tested regressions.
Co-authored-by: vduke <vduke@users.noreply.github.com>
---------
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
Render structured background-process completions and watch matches as accessible collapsed cards while preserving raw fallback, verbatim output, and full watch patterns.
Co-authored-by: b3nw <b3nw@duck.com>
Record resolved_model on assistant messages as _usedModel, expose used_model
in the usage SSE payload, and render model chips in transparent and settled
turn footers while suppressing duplicates when gateway routing already shows
used_model.
* fix(webui): keep the content search from evicting the user's working set
/api/sessions/search?content=1 walks EVERY session and pulls each one through
get_session(), which inserts it into the SESSIONS LRU and marks it
recently-used. On any install with more sessions than sessions_cache_max
(default 300), a single search therefore flushes the whole cache and refills it
with sessions the user is not looking at — the classic buffer-pool
scan-pollution problem. The sessions actually open in the UI are exactly the
ones evicted, and the search is keystroke-debounced, so it repeats while typing.
A scan reads each session exactly once, so nothing it touches has earned
"recently used". get_session_for_scan() reuses a resident session without
promoting it, and reads a cold one straight from disk without caching it. It
returns None rather than raising, since a scan skips what it cannot open.
This is a correctness fix for cache behaviour, not a latency fix. The
multi-second searches that led here were contention, not scan cost: a trivial
/api/profiles took 9.2s in the same window, and a full read+parse of ~1700 real
sessions measures ~4s total.
test_sessions_search_depth_validation patched api.routes.get_session. With the
search reading through the scan accessor it now patches get_session_for_scan —
left unfixed, two of its cases fail and the third passes vacuously against an
empty result set.
Validation:
pytest tests/test_issue4765_sessions_lru_eviction.py
tests/test_sessions_search_depth_validation.py -> 13 passed
Both added eviction tests fail on the pre-fix accessor (verified by revert):
the working set drops from 4/4 to 0/4 resident after one scan.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(changelog): stamp #6084 content-search working-set preservation
---------
Co-authored-by: ai-ag2026 <261867348+ai-ag2026@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
Memory: get_state_db_session_messages had no SQL LIMIT — it full-scanned the
state.db messages table and fetchall()'d into a Python list. The display path
(GET /api/session) already has a careful since_timestamp optimization that
bounds the common uncompressed tail load, but it deliberately bails (returns
None) on compressed sessions (truncation_watermark/boundary) and msg_before
paging — in those cases the read used to full-scan with no cap. A
pathological/huge state.db could then materialize unbounded rows into memory.
Add an opt-in `limit` parameter to get_state_db_session_messages (applied as a
SQL LIMIT that keeps the NEWEST rows via a DESC subquery re-sorted ASC). It is
a BACKSTOP, not a semantic window: the display window counts visible rows
post-reconciliation, so a true window LIMIT here would corrupt the
sidecar/state.db merge (see _state_db_since_timestamp_for_limited_display,
which deliberately does NOT SQL-LIMIT raw rows for exactly that reason).
Thread a generous _STATE_DB_DISPLAY_ROW_BACKSTOP (50000) from the GET /api/session
display caller only. The full-history model-context callers (reconciliation at
models.py:3437, context reconstruction at :8576, new-turn context at
streaming.py:8443) do NOT pass a limit — their semantics require the full
history, unchanged.
Test: tests/test_state_db_read_backstop.py — no-limit returns full history
oldest-first (model-context contract unchanged); limit=N keeps the newest N
re-sorted oldest-first; limit > row count returns all; limit<=0 clamps to 1
(newest); invalid limit falls back to unbounded; limit composes with
since_timestamp. Existing state.db active-filter + reconciliation suites (54
tests) unchanged.
Contract Routing
- Task type: memory-bound fix (unbounded state.db row materialization on the
display path's bail cases).
- Touched areas: state.db message reader (api/models.py) + GET /api/session
display caller (api/routes.py). NOT the model-context callers.
- Relevant public docs: AGENTS.md, CONTRIBUTING.md, docs/CONTRACTS.md,
docs/rfcs/webui-run-state-consistency-contract.md (Model context layer).
- State layer mutated: none (read-only). The model-context read path is
UNCHANGED (no limit passed) — Invariant 1 (visible current turns enter model
context) and the reconciliation contract are preserved. Only the display path
gets the defensive cap, which does not affect what the agent receives.
- Scope boundaries: this is narrower than a naive "add LIMIT everywhere" —
_state_db_since_timestamp_for_limited_display's docstring explains why a
semantic window LIMIT would corrupt the merge; this cap is purely a
pathological-input safety net above any legitimate session size.
Release note: GET /api/session's state.db read now has a defensive row backstop,
capping memory when the bounded since_timestamp optimization bails on
compressed/paged sessions.
Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent).
Memory: the CLI/cron session projection cache was a plain dict with TTL storage
but NO size cap and only lazy, same-key eviction. Each value is a copy.deepcopy()
of the full CLI/cron session list (the expensive projection behind #4842/#4672).
The cache key folds in a state.db content fingerprint that advances on every
streamed message, so the next poll builds a NEW key; the previous key's heavy
deepcopy was never looked up again, so its lazy-expiry path never ran. A
long-lived process under churn could accumulate orphaned heavy deepcopies until
the next structural clear_cli_sessions_cache().
Make it an OrderedDict capped at _CLI_SESSIONS_CACHE_MAX_ENTRIES (8) with
drop-oldest on write (while len > MAX: popitem(last=False)) and move_to_end on
fresh hits, mirroring the _CLAUDE_CODE_PARSE_CACHE / _SIDECAR_METADATA_CACHE LRU
pattern in the same file. TTL remains the primary freshness control; the cap is
the backstop the plain dict lacked. clear_cli_sessions_cache() keeps .clear().
Test: tests/test_cli_sessions_cache_cap.py — cache stays at len == MAX after
MAX+5 distinct keys (oldest 5 evicted), a fresh hit marks the entry
most-recently-used so it survives over colder entries, and the cache is an
OrderedDict (the regression: it was an unbounded plain dict). Existing
streaming-cache-TTL/cron-session tests unchanged.
Contract Routing
- Task type: memory-bound fix (unbounded projection cache).
- Touched areas: CLI/cron sidebar projection cache (api/models.py
_CLI_SESSIONS_CACHE).
- Relevant public docs: AGENTS.md, CONTRIBUTING.md, docs/CONTRACTS.md,
docs/rfcs/webui-run-state-consistency-contract.md (Sidebar/session metadata
layer).
- State layer mutated: Sidebar/session metadata layer — specifically the
in-memory projection cache that backs the CLI/cron sidebar rows. The cache is
a read-derivation of state.db content; it is NOT a source of truth. Dropped
entries are rebuilt on the next poll (TTL/invalidation already drive rebuilds).
- Invariant preserved — Invariant 4 (Maintenance is not activity): the cache is a
derived projection; evicting an entry cannot move a session in the sidebar or
change a row's content, because the next poll recomputes the projection from
state.db. TTL remains the primary freshness control; the LRU cap only bounds
how many STALE orphaned deepcopies linger between invalidations. Regression:
test_cli_sessions_cache_cap.py plus the existing streaming-cache-TTL and
cron-session tests (which exercise the real invalidate/rebuild path).
- Scope boundaries: does not change sidebar ordering, unread state, updated_at,
or the projection result. The cap is a memory backstop; correctness still
flows from the existing TTL + clear-on-mutation invalidation.
Release note: the CLI/cron sidebar projection cache is now LRU-bounded, capping
worst-case memory for long-lived processes under heavy streaming churn.
Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent). No AI-specific tool
use mattered beyond the repo's standard read/edit/test workflow.
Codex re-gate asked to distinguish a confirmed-absent sidecar from an
indeterminate/unreadable one before entering the abandoned-shell grace path.
Add _session_sidecar_exists() (True/False/None) and only grace-evict when the
sidecar is confirmed absent (False); a present-or-indeterminate sidecar keeps
the session resident. Age-window bound retained (fully closes the leak, no
data-loss edge); an SSE-mounted-lease refinement is noted as optional follow-up.
Codex gate found the eviction-reorder made never-saved empty shells immortal
(abandoned /api/session/new tabs accumulate unbounded past sessions_cache_max).
Add a grace-window bound: protect a never-persisted empty shell only while fresh
(<30min old) OR while it carries a composer draft; a stale, empty, draftless,
never-saved shell is treated as abandoned and becomes evictable again. A real
compose persists a draft on the first keystroke (disk-backed + reloadable), so
the brick fix is preserved while the leak is closed. +2 regression tests.
Two independent defects that chain into "new conversations cannot be started":
every POST /api/session/draft and /api/chat/start 404s a few seconds after
POST /api/session/new returned 200.
1. Search inputs are parsed as username fields.
index.html contains no <form> at all, but does contain password inputs (the
Settings "Access Password" panel). Chromium therefore groups the document's
unowned fields into one synthetic password form and picks the first text input
in the DOM — #sessionSearch, the sidebar conversation filter — as that form's
username field, then autofills the saved account name into it on load.
autocomplete="off" was already present; it is ignored for credential
heuristics.
The autofill fires oninput -> filterSessions() -> GET /api/sessions/search
?content=1&depth=5: a full content search the user never asked for, repeated
on every page load.
Fix: give the four search inputs type="search" plus the per-manager opt-outs
(data-1p-ignore / data-lpignore / data-bwignore / data-form-type), and give the
Settings password inputs a real <form> owner with proper autocomplete tokens so
Chromium scopes the credential form to them instead of to the whole document.
The form uses display:contents, so layout is unchanged. CSS suppresses WebKit's
native search clear button, since these fields ship their own
(#sessionSearchClear).
2. A never-persisted session is treated as evictable.
new_session() intentionally keeps a session in RAM until its first message
(#1171), so the SESSIONS cache is its only copy. _session_is_evictable()
(#4765) short-circuited on zero messages, reasoning that an empty shell "is
recreated on next access". It is not: get_session() has no recreate path and
raises KeyError, so both routes 404 and the session can never be started.
The content search from (1) pulls every hit through get_session(), which blows
past sessions_cache_max (default 300) and evicts the session being composed.
That is why this reproduces within seconds on an install with ~1700 sessions,
and why it grew worse as the session count rose.
Fix: require proof of persistence before evicting, for empty sessions too —
which is what the function's own docstring already promised ("Its full state is
already persisted to the JSON sidecar"). The zero-message branch was the single
path that bypassed it. #4765's memory bound is unaffected: sessions loaded from
disk remain evictable, and its existing tests still pass.
Trade-off: an unsaved empty session now stays resident for the process
lifetime (one per "New Conversation" click, an empty object each). That is a
deliberate application of the function's stated invariant — "slightly more RAM
for a session we are unsure about is strictly better than evicting an unsaved
session". Bounding it by age instead would also work if maintainers prefer.
Validation:
pytest tests/test_issue4765_sessions_lru_eviction.py -> 9 passed
The added regression test fails on the unpatched predicate (verified by
reverting api/models.py alone) and passes with the fix.
Note: (1) fixes what triggers the churn, (2) fixes what turns that churn into
an unstartable session. Either fix alone leaves a real defect in place, so
they are submitted together.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codex found a verified lifecycle hole: the bare boolean model_explicitly_picked
was not cleared on /api/session/update (api/routes.py:14438-14460 changes+persists
the model/provider without touching it), so after an update+reload the flag stayed
True and a cold #433 stale leftover (openai/gpt-5.4) was wrongly preserved instead
of stripped to gpt-5.4.
Fix (Codex's prescription): replace the unscoped boolean with a SIGNATURE bound to
the routing context.
- api/models.py: new model_explicit_pick_signature(model, provider) ->
'<model>\x1f<provider>'; Session.model_explicit_pick_signature persisted field
(replaces the boolean; in METADATA_FIELDS, restored via kwargs, survives restart).
- api/routes.py /api/chat/start: on a fresh explicit_model_pick, stamp the signature
of the resolved model+provider.
- api/streaming.py: treat the send as a deliberate pick ONLY when the persisted
signature equals the signature recomputed from the session's CURRENT persisted
model+provider. Any model/provider change (chat-start, session-update,
normalization, provider repair) yields a different signature and auto-invalidates
the stale pick — no per-mutation-site clearing needed.
So: deliberate pick on unchanged context -> preserve (b3nw); any switch away ->
signature mismatch -> unmarked -> legacy strip (#433 stays correct even after a
session-update+cold-restart). Adds signature match/invalidate/persist tests.
75 focused tests pass (resolver + sprint40 + the 2 streaming-mock suites).
Nathan's call: adopt Codex's complete mechanism instead of the unconditional
cold-preserve. The cold-catalog custom-proxy decision is now gated on whether the
user DELIBERATELY selected the model this session, closing the residual where a
stale #433 leftover would preserve+400 on a cold+no-disk send.
Mechanism:
- resolve_model_provider(model_id, *, explicitly_picked=False): in the custom-arm
cold branch (no warm provenance, not config-declared), explicitly_picked=True
preserves the vendor namespace verbatim (deliberate pick, proxy routes on it);
otherwise the legacy redundant-prefix strip runs (stale cross-provider leftover,
#433). Warm endpoint-advertised provenance still wins over the flag.
- Session.model_explicitly_picked: new persisted field (default False, restored
from metadata via **kwargs, listed in METADATA_FIELDS so it survives a restart
— b3nw's exact cold scenario).
- /api/chat/start stamps it: True on a fresh explicit_model_pick; cleared only
when the resolved model actually CHANGED without a pick (real switch away);
preserved across plain follow-up sends of the same model (the onchange marker
is one-shot, so per-send explicit_model_pick alone would drop the intent).
- streaming.py reads s.model_explicitly_picked and passes explicitly_picked= into
resolve_model_provider (inside the profile scope).
Independent end-to-end matrix (all pass): ben default cold, ben picked non-default
cold+warm, ben stale cold (strips), #433 warm+cold, #3872 bedrock, #548 zai-org.
67 resolver+sprint40 tests pass incl. new explicit-pick + stale-companion +
persistence round-trip.
* fix(#4685): meter post-compression model context
* Release exp-v0.52.48: meter post-compression model context (#5872, #4685)
---------
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
Adversarial security re-gate on the rebased head found 4 issues; fixed:
1. [CORE] api/shares.py public snapshot could leak credentials/paths/tool
payloads. The sanitizer relied on redact_session_data (gated on the
user-toggleable api_redact_enabled) — disabling it exposed API keys, a dict
assistant content was stringified as raw tool data, and session.workspace
survived in transcript text. Now the per-message sanitizer applies ALWAYS-ON
credential redaction (_redact_fn_cached, force=True, ignores the setting) +
scrubs known session/workspace/home paths, and refuses to stringify non-text
structured content. Title is force-redacted too. Regression test proves the
boundary holds with api_redact_enabled=False.
2. [SILENT] api/models.py: my rebase wrongly kept anchor_activity_scenes in
METADATA_FIELDS, moving 250-480KB scene bodies before messages and undoing
master's metadata fast-path (#5854). Removed; 17 #5854 tests pass again.
3. [SILENT] static/index.html: the Share/StopSharing/ExportHTML button triplet
was double-applied by the rebase (duplicate IDs → only the first wired).
Deduped to one each + a count assertion test.
4. tests: fixed the stale Session.load-after-/api/session/new setup (master
keeps new sessions memory-only until first message).
Also (UX, found in the visual flow sweep): a clipboard-copy failure after a
SUCCESSFUL share showed a red 'Share failed' toast and skipped opening the page.
The copy is now isolated — the share succeeds, the page opens, and the toast
shows the link if the clipboard write fails, never a false failure.
Co-authored-by: MicoRobot.
Rebased 363 commits forward (branch-point 2026-07-05). Applied the PR's code
diff cleanly across 16 files; resolved api/models.py additively (both master's
process_wakeup_pause/anchor_scene_index fields AND the PR's share_token/
share_created_at fields kept in the Session constructor, metadata list, and
to_dict). CI workflow (.github/workflows/tests.yml) intentionally NOT taken from
the stale branch — master owns it (the branch's apparent removal of Playwright/
Office steps was a PAT-workflow-scope revert; rebase restores master's CI).
Co-authored-by: MicoRobot. Concept approved by maintainer (opt-in public share,
2026-07-11). Note: 4 dynamic share tests fail identically on the original
unrebased head (pre-existing test-env Session.load fragility, not rebase-caused);
to be resolved during the security re-gate in a proper sandbox.
Reasoning-heavy sessions accumulate 250-480KB of anchor_activity_scenes serialized
BEFORE the messages array, so the 64KB metadata-prefix read always overflowed ->
full-parse fallback on every read -> memory growth. Move scene data AFTER messages;
metadata prefix now ~2KB, modern reads skip the full parse. Old-layout files still
load (one authoritative parse then cached); round-trip preserves scenes exactly.
Independent review: nesquena APPROVED (end-to-end, on head). Gate-pass: Codex SAFE
(933KB legacy sidecar loads clean + round-trip verified), 17 own + 2945 regression green.
Co-authored-by: t <a@b>
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
alfred-rootson