Commit Graph
2514 Commits
Author SHA1 Message Date
b470793473 Release: msg_limit ceiling + row-loss-safe pagination (#6152 + #6154, @rh-id) (#6213)
* fix(routes): clamp GET /api/session msg_limit to a server-side ceiling

Memory: ?msg_limit= had no upper bound — a client could request
msg_limit=1000000 (or the frontend's msg_limit=9999 outline-jump path) and
force the server to assemble and serialize an unbounded message payload. The
frontend's real pagination grows windows by ~30 at a time (initial load 30,
load-older +30, msg_before paging 30), so only the outline-jump and
pathological/hostile cases ask for huge values.

Add _MAX_MSG_LIMIT (500) and clamp at the query-parse site
(max(1, min(int(_msg_limit), _MAX_MSG_LIMIT))). The ceiling is generous — far
above any legitimate visible-row window — so real pagination is unaffected. The
existing _messages_truncated signal already covers the clamped case: when the
request exceeds the ceiling, _message_window_for_display returns
_messages_offset > 0 and the client learns more rows exist (load-older still
works). The bare no-msg_limit path is intentionally unchanged — it is the
documented "give me everything" escape hatch used by branch/undo/jump-to-start
flows that need the full transcript for correct global indices.

Test: tests/test_session_msg_limit_ceiling.py — the ceiling constant is
reasonable (100..2000), an over-ceiling request is clamped and signals
truncation, and legit sizes (5/30/60/100) are unaltered. Existing window/tail
tests (19) unchanged.

Contract Routing
- Task type: memory-bound fix (unbounded msg_limit request payload).
- Touched areas: GET /api/session query parsing (api/routes.py).
- Relevant public docs: AGENTS.md, CONTRIBUTING.md, docs/CONTRACTS.md.
- State layer mutated: none (read-only endpoint response shaping).
- Contract change (additive): ?msg_limit=N is now clamped to _MAX_MSG_LIMIT=500.
  Compatibility: the only frontend caller above the ceiling is outline.js's
  msg_limit=9999, which now receives up to 500 rows + _messages_truncated=true
  (load-older still works). All other callers (30/60) are unaffected. The bare
  no-msg_limit path is unchanged.

Release note: GET /api/session now caps ?msg_limit at 500 rows server-side,
bounding the worst-case payload for oversized requests.

Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent).

* fix(frontend): route msg_limit-heavy paths around the server ceiling

PR #6152 clamps GET /api/session ?msg_limit= to a max of 500 server-side. Two
frontend paths relied on unbounded msg_limit and silently broke for sessions
> 500 visible rows:

1. outline.js _jumpToMessage sent msg_limit=9999 as a "give me everything" hack
   so it could scroll to any message by absolute index (msg-user-<rawIdx>).
   Clamped to 500, the target row's DOM id doesn't exist for early messages in
   long sessions, and the jump silently no-ops (line 119 `if (r)` is falsy).
   Fix: drop msg_limit=9999 — use the bare messages=1 path, which #6152 left
   unbounded precisely for callers that need the full transcript. This restores
   exact pre-#6152 behavior for outline jumps.

2. sessions.js _loadOlderMessages grows msg_limit by +_INITIAL_MSG_LIMIT each
   load (30 -> 60 -> ... -> 500 -> clamped). Past 500 the server returns the
   same clamped tail, olderMsgs shrinks to 0, and endless-scroll / "Load
   earlier" silently stalls — the head becomes unreachable. Fix: once the grown
   requestedLimit meets/exceeds the new _MSG_LIMIT_MAX mirror (500), switch the
   primary fetch to msg_before paging (a fixed _INITIAL_MSG_LIMIT backward page
   keyed off _oldestIdx) — the same bounded paging the existing race-fallback
   uses, so the head stays reachable for arbitrarily long transcripts. The
   response is handled by the existing prepend path (tailMatches is naturally
   false for a page, so the !tailMatches branch reuses the fetched data).

Added _MSG_LIMIT_MAX = 500 constant in sessions.js mirroring backend
_MAX_MSG_LIMIT, documented as kept-in-sync.

Verification: node --check passes on both files; the backend window-contract
tests (test_session_tail_payload, test_session_message_window_renderable_tail,
19 tests) still pass. No JS test harness exists in this repo (vanilla JS,
manual browser verification per TESTING.md) — manual verification needed: open a
> 500-message session, (a) click an outline entry for an early message and
confirm it scrolls+flashes, (b) scroll up past 500 loaded rows and confirm
"Load earlier" keeps producing older messages without stalling.

Contract Routing
- Task type: frontend follow-up to a backend memory fix (#6152).
- Touched areas: outline jump-to-message (static/outline.js), endless-scroll /
  load-older (static/sessions.js _loadOlderMessages).
- Relevant public docs: AGENTS.md, CONTRIBUTING.md, docs/CONTRACTS.md.
- State layer mutated: client-side S.messages (the active-session transcript),
  same as the existing load-older path. No backend state change.
- Scope boundaries: does not weaken the #6152 clamp. The bare messages=1 path
  (outline) was intentionally left unbounded by #6152; the msg_before paging
  path (load-older) is bounded by _INITIAL_MSG_LIMIT and never hits the ceiling.

Release note: fixes outline jump-to-message and endless scroll silently
breaking on sessions longer than 500 messages after the server-side msg_limit
ceiling (companion to #6152).

Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent).

* test(frontend): update static-source assertion for refactored _loadOlderMessages fetch

The static-source test test_session_message_loads_keep_explicit_longer_timeouts
asserted the exact multi-line shape of the _loadOlderMessages fetch strings.
The #6154 refactor wrapped them in a useBeforePaging ternary (tail-growth vs
msg_before paging), reindenting them — the contract the test guards (both
fetches keep timeoutMs:120000 for large state.db loads) is preserved, but the
literal source shape changed, so the assertion failed on shard 0.

Updated the assertions to match the new source shape while keeping the test's
intent: assert each fetch URL + its {timeoutMs:120000} survive in sessions.js.

Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent).

* test(msg-limit): stopgap drift guard for _MSG_LIMIT_MAX === _MAX_MSG_LIMIT

Reviewer guidance (#6154 round-2): the stopgap drift-test belongs in THIS PR
(#6152) because _MAX_MSG_LIMIT is defined here. The frontend _MSG_LIMIT_MAX
mirror lives on #6154 (not master yet), so the assertion SKIPS when the JS
constant is absent and activates automatically once both branches land on master.

The guard: extract _MAX_MSG_LIMIT from api/routes.py and _MSG_LIMIT_MAX from
static/sessions.js via regex; assert equality when both present. If they drift,
load-older silently stalls at the wrong value (the exact regression #6154 fixed).
Retired by #6177 (metadata-exposure), which removes the mirror entirely.

Verified: on this branch the cross-check SKIPS (_MSG_LIMIT_MAX absent, as
expected — #6154 not landed) and the backend-existence test PASSES. Extraction
logic confirmed correct against simulated match (500==500 passes) and drift
(999!=500 fails) cases.

Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent).

* fix(session): prevent silent row-loss in msg_limit ceiling paging (gate follow-up)

Adversarial review of #6152/#6154 reproduced two silent row-loss regressions in
static/sessions.js; both fixed here, plus a stale source-string assertion update
and the CHANGELOG entry:

1. _loadOlderMessages: gate the suffix-continuity heuristic on !useBeforePaging so
   a raw-row-heavy msg_before cursor page whose visible text repeats the current
   tail can't be misclassified as the cumulative tail and wholesale-replace it
   (previously dropped 470 of 940 rows). msg_before pages are always prepended.

2. loadSession same-session refresh: only apply the width-hint msg_limit when it
   is <= _MSG_LIMIT_MAX; above the ceiling the backend would clamp and silently
   shrink an already-loaded >500-row transcript to the last 500. Over-ceiling
   reloads now use the bare full-transcript path (no msg_limit / no expand_renderable).

3. tests/test_webui_external_refresh_frontend.py: update the reloadLimitParam
   source assertion to the boundedReloadLimit form.

Re-gate: Codex adversarial SAFE TO SHIP (940/940 rows retained on cursor paging,
1000/1000 on refresh, no seam duplication, small sessions still use the fast tail
window); targeted + sessions/pagination/regression suites green.

Co-authored-by: Ruby Hartono <rh-id@users.noreply.github.com>

* test: fix two node-harness tests broken by the boundedReloadLimit change

- test_issue3162_ensure_messages_loaded: replace the brittle fixed 4500-char
  window (which the boundedReloadLimit lines pushed the carry-forward past) with
  robust brace-balance function-body extraction.
- test_cross_session_message_load_isolation: define _MSG_LIMIT_MAX=500 in the node
  harness env; _ensureMessagesLoaded now references it, and an undefined value made
  boundedReloadLimit=null -> the fetch URL dropped msg_limit/expand_renderable and
  mismatched the ordered api() harness -> stall/timeout.

---------

Co-authored-by: Ruby Hartono <58564005+rh-id@users.noreply.github.com>
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
Co-authored-by: Ruby Hartono <rh-id@users.noreply.github.com>
2026-07-17 20:15:40 -07:00
Hermes Agent 64f98dadd0 fix: preserve lazy alignment semantics 2026-07-17 15:36:55 -06:00
Hermes Agent d8360bfe7d refactor: clarify alignment safety checks 2026-07-17 15:36:55 -06:00
Hermes Agent 509263e161 perf: linearize session truncate alignment 2026-07-17 15:36:55 -06:00
Frank Song 5c34dd9248 fix(streaming): skip reasoning rows for tool anchors 2026-07-18 04:36:00 +08:00
Frank Song e0dee876a0 fix(streaming): keep recovered tool call anchors 2026-07-18 03:23:45 +08:00
Frank Song 3330bb4bd1 fix(streaming): preserve recovered context shape 2026-07-18 03:23:45 +08:00
Frank Song 59b6022dd2 fix(streaming): harden recovered reasoning context ownership 2026-07-18 03:23:45 +08:00
Frank Song b741e17275 fix(streaming): recover interrupted reasoning 2026-07-18 03:23:45 +08:00
mol 04783d11d8 fix: normalize Codex handoff base URL 2026-07-17 16:33:52 +00:00
mol 6c863ae9c9 fix: omit unsupported Codex handoff token cap 2026-07-17 16:17:19 +00:00
nesquena-hermes a3d27eb7ca Merge remote-tracking branch 'origin/master' into pr6182-g 2026-07-17 09:47:27 +00:00
Rod Boev 73305706c6 fix(#6108): bound SessionDB retries to transient sqlite contention 2026-07-17 05:42:00 -04:00
Hermes Agent 1bcf8f8b46 perf(sessions): cache reconciliation message keys 2026-07-17 02:11:31 -06:00
Rod Boev 2e1867c6e5 fix(#6108): keep session_search alive through transient state.db lock races 2026-07-17 03:55:22 -04:00
nesquena-hermes c8edc97e89 Merge remote-tracking branch 'origin/master' into p6153b 2026-07-17 01:16:08 +00:00
nesquena-hermes 13efcbe06c Merge remote-tracking branch 'origin/master' into p6153b 2026-07-17 01:10:27 +00:00
nesquena-hermes a8143affe2 Merge remote-tracking branch 'origin/master' into p6137b 2026-07-17 01:09:51 +00:00
Ruby Hartono f445c3ae39 fix(routes): gate state.db row backstop on absence of truncation_boundary prefix
Reviewer regression (reproduced): _STATE_DB_DISPLAY_ROW_BACKSTOP (newest-50k SQL
cap) was applied UNCONDITIONALLY to the GET /api/session display path's state.db
read. But merge_session_messages_append_only needs the rows at/around the
session's truncation_boundary (the preserved prefix) to reconcile correctly. A
newest-N-only cap drops those boundary rows for a >N-row session and corrupts
the merge (silently losing the preserved prefix), and also prevents older-page
(msg_before) requests from ever reaching them. So a pathological-but-real >50k-
row compressed session got a corrupted merge / unreachable older history rather
than a clean bound.

Fix: gate the backstop on the SAME conditions
_state_db_since_timestamp_for_limited_display uses to decide a read is
boundary-free — not msg_before paging, and no truncation_watermark /
truncation_boundary. Those cases stay on the full (uncapped) read; the backstop
is now a defensive net for the common uncompressed initial-tail path only.
Extracted into _state_db_backstop_limit_for_display for direct test coverage.

The get_state_db_session_messages(limit=) signature + newest-rows subquery are
unchanged — only the display-path APPLICATION of the cap is gated.

Test: tests/test_state_db_backstop_boundary_gate.py — backstop applies with no
boundary prefix; SKIPPED when truncation_boundary / truncation_watermark set or
msg_before paging; empty-string markers treated as unset (matching the existing
helper). Existing backstop + reconciliation suites (53 tests) unchanged.

Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent).
2026-07-17 07:19:47 +07:00
Ruby Hartono d57fa7ba3b fix(stream): retry (not break) on queue.Empty so terminal frames survive a concurrent drain
Reviewer regression (reproduced): the broadcast drop-oldest loop did
put_nowait -> on Full, get_nowait -> on Empty, break. But Empty here means a
concurrent consumer (the SSE handler thread) DRAINED the queue between our Full
and get_nowait — the queue now has space, which is exactly the condition to
retry the put. `break` silently discarded `item`, and when `item` was a terminal
frame (stream_end/error/cancel) the subscriber never received it, leaving the
client attached indefinitely (spinner-forever). The "# cap 0 defensive guard;
never reached" comment was the flawed assumption: any concurrent drain makes
the Empty branch reachable at any cap.

Fix: on queue.Empty, `continue` (retry the put into the now-empty queue) in
both the broadcast path and the replay path (for consistency; the replay path
runs under self._lock with a fresh queue so it's unreachable in practice, but
the same race-safe rule applies). No infinite-loop risk: each iteration either
succeeds a put (break) or succeeds a get (progress); the continue only fires on
Empty, after which the next put has space.

Test: test_terminal_frame_survives_concurrent_drain_during_drop_oldest
deterministically simulates the interleaving with a custom queue whose first
post-Full get_nowait raises Empty (as if a concurrent consumer drained it),
then runs BOTH the pre-fix `break` logic (drops the terminal frame — proving
the test bites) and the post-fix `continue` logic (delivers it).

Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent).
2026-07-17 07:04:04 +07:00
nesquena-hermes ea3cba3823 Merge remote-tracking branch 'origin/master' into pr6092-gate 2026-07-16 21:05:14 +00:00
nesquena-hermes 1101da4663 Merge remote-tracking branch 'origin/master' into p6140 2026-07-16 20:04:54 +00:00
nesquena-hermesandcarlotestor dcdcfb9977 fix(streaming): degrade self-heal SessionDB to None on failed rebuild
Codex gate on #6143 found a SILENT regression the reuse-path fix introduced:
_replace_session_db_in_kwargs early-returned the OLD handle when the rebuild
failed, but master assigned None. If the old handle was already CLOSED, both
credential self-heal callers (streaming.py:9032/10253) then built a fresh agent
around a closed SessionDB -> every persist/search fails with 'NoneType has no
attribute execute' while the chat keeps going. Guard on _session_db_is_open:
keep a live handle (subagents hold it by ref), else degrade to None (clean lazy
reinit, master behaviour). +2 regression tests.

Co-authored-by: carlotestor <carlotestor@users.noreply.github.com>
2026-07-16 19:14:53 +00:00
Ruby Hartono dec041997f fix(models): add defensive row backstop to display-path state.db message read
Memory: get_state_db_session_messages had no SQL LIMIT — it full-scanned the
state.db messages table and fetchall()'d into a Python list. The display path
(GET /api/session) already has a careful since_timestamp optimization that
bounds the common uncompressed tail load, but it deliberately bails (returns
None) on compressed sessions (truncation_watermark/boundary) and msg_before
paging — in those cases the read used to full-scan with no cap. A
pathological/huge state.db could then materialize unbounded rows into memory.

Add an opt-in `limit` parameter to get_state_db_session_messages (applied as a
SQL LIMIT that keeps the NEWEST rows via a DESC subquery re-sorted ASC). It is
a BACKSTOP, not a semantic window: the display window counts visible rows
post-reconciliation, so a true window LIMIT here would corrupt the
sidecar/state.db merge (see _state_db_since_timestamp_for_limited_display,
which deliberately does NOT SQL-LIMIT raw rows for exactly that reason).

Thread a generous _STATE_DB_DISPLAY_ROW_BACKSTOP (50000) from the GET /api/session
display caller only. The full-history model-context callers (reconciliation at
models.py:3437, context reconstruction at :8576, new-turn context at
streaming.py:8443) do NOT pass a limit — their semantics require the full
history, unchanged.

Test: tests/test_state_db_read_backstop.py — no-limit returns full history
oldest-first (model-context contract unchanged); limit=N keeps the newest N
re-sorted oldest-first; limit > row count returns all; limit<=0 clamps to 1
(newest); invalid limit falls back to unbounded; limit composes with
since_timestamp. Existing state.db active-filter + reconciliation suites (54
tests) unchanged.

Contract Routing
- Task type: memory-bound fix (unbounded state.db row materialization on the
  display path's bail cases).
- Touched areas: state.db message reader (api/models.py) + GET /api/session
  display caller (api/routes.py). NOT the model-context callers.
- Relevant public docs: AGENTS.md, CONTRIBUTING.md, docs/CONTRACTS.md,
  docs/rfcs/webui-run-state-consistency-contract.md (Model context layer).
- State layer mutated: none (read-only). The model-context read path is
  UNCHANGED (no limit passed) — Invariant 1 (visible current turns enter model
  context) and the reconciliation contract are preserved. Only the display path
  gets the defensive cap, which does not affect what the agent receives.
- Scope boundaries: this is narrower than a naive "add LIMIT everywhere" —
  _state_db_since_timestamp_for_limited_display's docstring explains why a
  semantic window LIMIT would corrupt the merge; this cap is purely a
  pathological-input safety net above any legitimate session size.

Release note: GET /api/session's state.db read now has a defensive row backstop,
capping memory when the bounded since_timestamp optimization bails on
compressed/paged sessions.

Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent).
2026-07-16 23:40:26 +07:00
pxxD1998 aae2830459 fix(compression): preserve running job reuse 2026-07-16 22:56:41 +08:00
carlotestor 7d2ca8cabb fix: address greptile review on SessionDB adopt helper
- Log failed closes in _adopt_session_db_for_cached_agent with
  logger.debug(..., exc_info=True) instead of bare pass, matching
  _replace_session_db_in_kwargs so a failed close (EMFILE risk) is
  diagnosable.
- Drop the fragile exec/source-slice fallback in the test helper import;
  real import or pytest.skip. Source-level pins still cover reverts
  without importing streaming.
2026-07-16 16:53:42 +02:00
pxxD1998 798bc5fb51 fix(compression): reject stale runtime before async job 2026-07-16 22:43:19 +08:00
carlotestor 0cee7dba47 fix: stop closing SessionDB under live background subagents
Background subagents (delegate_tool) capture parent_agent._session_db by
reference. The cached-agent reuse path created a fresh SessionDB every
stream request and unconditionally closed the previous handle (PR #1421
EMFILE fix). A server-side wakeup / new turn for the same parent session
could therefore close the shared handle while children were still
flushing messages, producing:

  Session DB append_message failed: 'NoneType' object has no attribute 'execute'

Reproduce (observed in production):
1. Parent WebUI session runs delegate_task with N background subagents
2. Parent turn tears down / idle-hook redelivers a deferred wakeup
3. New stream for the parent adopts a fresh SessionDB and closes the old one
4. Live subagents' append_message hits conn=None

Fix: _adopt_session_db_for_cached_agent reuses a still-open handle (and
closes the unused new one — still EMFILE-safe) and only replaces when
the existing handle is missing or already closed. Same guard applied to
_replace_session_db_in_kwargs (credential self-heal). LRU eviction still
closes at true session boundaries.

Tests updated to pin the adopt helper and the subagent-safe semantics.
2026-07-16 16:38:43 +02:00
pxxD1998 2415995204 fix(runtime): ignore in-memory agent module swaps 2026-07-16 22:28:35 +08:00
pxxD1998 0bab6a3d7a fix: handle remaining stale runtime entrypoints 2026-07-16 22:28:35 +08:00
pxxD1998 66dceb10e5 fix: ignore untracked enclosing agent repositories 2026-07-16 22:28:35 +08:00
pxxD1998 f722f68738 fix: scope stale runtime guard by execution owner 2026-07-16 22:28:35 +08:00
pxxD1998 5f1791e08d fix: reject stale runtime before chat mutation 2026-07-16 22:28:35 +08:00
pxxD1998 5df6b1fe8a fix: stop stale agent runtime after source updates 2026-07-16 22:28:35 +08:00
Ruby Hartono bbda885aa6 fix(stream): cap StreamChannel per-subscriber queues (drop-oldest on full)
Memory: each connected tab's subscriber queue was queue.Queue() with no
maxsize. A slow/backpressured/backgrounded tab accumulated every coalesced
token frame for the WHOLE turn (producer = agent token stream) — unbounded
per-tab growth, OOM risk with many tabs x long agentic turns. The offline
buffer was already capped (#4633), but once a subscriber attached the
live-broadcast path flowed into an unbounded per-subscriber queue.

Cap per-subscriber queues at _SUBSCRIBER_QUEUE_MAXSIZE (== _OFFLINE_BUFFER_MAXLEN)
and drop the OLDEST frame on queue.Full in both the broadcast (put_nowait) and
offline-buffer replay (subscribe_with_snapshot) paths, so a slow tab keeps the
most recent tail (older frames stay recoverable via the run journal by
last_event_id). Mirrors SessionChannel.emit's drop-on-full contract.

The cap intentionally equals the offline buffer bound, not SessionChannel's
smaller 16: StreamChannel carries the live token stream and has a #4633
reconnect-replay contract (a resubscribing tab must receive the full retained
offline tail), so capping below the buffer would force every flaky-network
reconnect through the run journal. Matching the buffer bound preserves that
contract while bounding live-broadcast memory to the same worst-case #4633
already accepts. The SSE write deadline (SSE_WRITE_DEADLINE_SECONDS) breaks a
stuck socket within ~20s, so the overflow window is short; this also bounds it
by frame count.

Adds a cumulative subscriber_dropped_events diagnostic counter (ops visibility).

Test: tests/test_stream_subscriber_queue_cap.py — bounded queue never exceeds
cap, drop-oldest keeps newest tail, draining vs non-draining subscribers, and
the #4633 reconnect-replay contract still delivers the full offline tail.
Existing offline-buffer/gap-recovery/multitab tests unchanged.

Contract Routing
- Task type: memory-bound fix (unbounded per-subscriber queue).
- Touched areas: live chat SSE broadcast (StreamChannel).
- Relevant public docs: AGENTS.md, CONTRIBUTING.md, docs/CONTRACTS.md,
  docs/rfcs/webui-run-state-consistency-contract.md (Live stream/SSE layer),
  docs/rfcs/session-sse-contract-v1.md (distinct: that RFC governs the PROPOSED
  per-session endpoint /api/sessions/{id}/events, NOT StreamChannel, which is
  the /api/chat/stream live token broadcast; no text in that RFC is changed).
- State layer mutated: Live stream / SSE layer — specifically the per-subscriber
  queue inside StreamChannel (api/config.py). NOT the durable run journal, NOT
  the transcript, NOT replay.
- Invariant preserved: the #4633 reconnect-replay contract — a resubscribing
  tab must receive the full retained offline tail. The cap is set EQUAL to
  _OFFLINE_BUFFER_MAXLEN so replay delivery is unchanged; only the previously-
  unbounded live-broadcast-to-slow-tab growth is bounded. Regression:
  test_reconnect_replay_still_delivers_full_offline_tail (and the pre-existing
  test_stream_offline_buffer_cap.py suite) pass unchanged.
- Scope boundaries: does not touch replay idempotency (Invariant 5), the turn
  journal, compression, or sidebar metadata. Slow-tab frames dropped at the cap
  stay recoverable via the run journal by last_event_id.

Release note: StreamChannel per-subscriber SSE queues are now bounded
(drop-oldest), capping memory for slow/backgrounded tabs during long turns.

Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent). No AI-specific tool
use mattered beyond the repo's standard read/edit/test workflow.
2026-07-16 20:18:39 +07:00
Ruby Hartono 015171eba3 fix(models): cap _CLI_SESSIONS_CACHE with LRU eviction (drop-oldest)
Memory: the CLI/cron session projection cache was a plain dict with TTL storage
but NO size cap and only lazy, same-key eviction. Each value is a copy.deepcopy()
of the full CLI/cron session list (the expensive projection behind #4842/#4672).
The cache key folds in a state.db content fingerprint that advances on every
streamed message, so the next poll builds a NEW key; the previous key's heavy
deepcopy was never looked up again, so its lazy-expiry path never ran. A
long-lived process under churn could accumulate orphaned heavy deepcopies until
the next structural clear_cli_sessions_cache().

Make it an OrderedDict capped at _CLI_SESSIONS_CACHE_MAX_ENTRIES (8) with
drop-oldest on write (while len > MAX: popitem(last=False)) and move_to_end on
fresh hits, mirroring the _CLAUDE_CODE_PARSE_CACHE / _SIDECAR_METADATA_CACHE LRU
pattern in the same file. TTL remains the primary freshness control; the cap is
the backstop the plain dict lacked. clear_cli_sessions_cache() keeps .clear().

Test: tests/test_cli_sessions_cache_cap.py — cache stays at len == MAX after
MAX+5 distinct keys (oldest 5 evicted), a fresh hit marks the entry
most-recently-used so it survives over colder entries, and the cache is an
OrderedDict (the regression: it was an unbounded plain dict). Existing
streaming-cache-TTL/cron-session tests unchanged.

Contract Routing
- Task type: memory-bound fix (unbounded projection cache).
- Touched areas: CLI/cron sidebar projection cache (api/models.py
  _CLI_SESSIONS_CACHE).
- Relevant public docs: AGENTS.md, CONTRIBUTING.md, docs/CONTRACTS.md,
  docs/rfcs/webui-run-state-consistency-contract.md (Sidebar/session metadata
  layer).
- State layer mutated: Sidebar/session metadata layer — specifically the
  in-memory projection cache that backs the CLI/cron sidebar rows. The cache is
  a read-derivation of state.db content; it is NOT a source of truth. Dropped
  entries are rebuilt on the next poll (TTL/invalidation already drive rebuilds).
- Invariant preserved — Invariant 4 (Maintenance is not activity): the cache is a
  derived projection; evicting an entry cannot move a session in the sidebar or
  change a row's content, because the next poll recomputes the projection from
  state.db. TTL remains the primary freshness control; the LRU cap only bounds
  how many STALE orphaned deepcopies linger between invalidations. Regression:
  test_cli_sessions_cache_cap.py plus the existing streaming-cache-TTL and
  cron-session tests (which exercise the real invalidate/rebuild path).
- Scope boundaries: does not change sidebar ordering, unread state, updated_at,
  or the projection result. The cap is a memory backstop; correctness still
  flows from the existing TTL + clear-on-mutation invalidation.

Release note: the CLI/cron sidebar projection cache is now LRU-bounded, capping
worst-case memory for long-lived processes under heavy streaming churn.

Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent). No AI-specific tool
use mattered beyond the repo's standard read/edit/test workflow.
2026-07-16 19:31:47 +07:00
6ed474ca54 fix(models): route list-shaped custom provider models (#6127) (#6128)
Centralize custom-provider model-ID extraction into _configured_model_ids()
+ _configured_model_options(), used by both catalog construction and runtime
routing so the picker and resolver can't drift. Fixes list-shaped
custom_providers[].models allowlists silently staying on the default provider
instead of routing to their own (#6121). Existing mapping / list-of-strings
configs unaffected.

Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
Co-authored-by: franksong2702 <franksong2702@users.noreply.github.com>
2026-07-16 00:22:50 -07:00
c39e8edaf1 fix(#6100): clear stale gateway approval attention badge (#6117) (#6125)
Reconcile the gateway pending-mirror in _session_attention_summary()
before counting sidebar approval attention, so a parent session's red
approval dot clears once the underlying gateway approval has drained.
Live approvals stay visible; clarify fallback unchanged. WebUI read-path
mitigation; child-session context-propagation root cause stays in #6100.

Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
2026-07-15 22:34:19 -07:00
nesquena-hermes 500fa485d2 Merge commit 'refs/pull/6013/head' of github.com:nesquena/hermes-webui into stage/6013 2026-07-16 02:28:15 +00:00
nesquena-hermes b877c9aa40 Merge commit 'refs/pull/6102/head' of github.com:nesquena/hermes-webui into stage/6102-cachettl 2026-07-15 22:40:21 +00:00
nesquena-hermes 38119fac25 fix(#6056): extend sidebar title-field redaction to /api/sessions/search (Codex class-fix)
Codex re-gate found the same derived-title leak on the sibling /api/sessions/search
endpoint (3 branches redacted only 'title'). Extract _redact_sidebar_title_fields()
shared helper, apply it in both _sidebar_session_response_item and all 3 search
branches, read api_redact_enabled once per search response (mirrors #4662). +helper
test + source-guard that every search branch calls the redactor.
2026-07-15 21:06:36 +00:00
starship-s f5ef3688c6 fix(cache): keep streaming TTLs above poll cadence 2026-07-15 20:59:59 +00:00
nesquena-hermes ffbc91343a fix(#6056): redact display_title/_state_db_title/parent_title on the sidebar API
Codex gate found the #6056 derived display_title (from raw user-message content)
bypassed sidebar redaction — the serializer redacted only 'title', so a credential
in a delegated goal leaked to the sidebar even with api_redact_enabled=True.
Redact display_title + _state_db_title + parent_title on the same path; +regression
test (proven to fail on the unfixed serializer).
2026-07-15 20:57:31 +00:00
nesquena-hermes b96405ef29 Merge commit 'refs/pull/6056/head' of github.com:nesquena/hermes-webui into stage/6056-delegtitles 2026-07-15 20:46:34 +00:00
nesquena-hermes 754dcca048 Merge commit 'refs/pull/6088/head' of github.com:nesquena/hermes-webui into stage/6088-manualupd 2026-07-15 20:02:02 +00:00
chsong 139c76535b fix tool-limit final answer classification 2026-07-15 19:38:25 +00:00
nesquena-hermes 30059085c2 fix(#6083): only grace-evict CONFIRMED never-persisted shells (Codex finding #2)
Codex re-gate asked to distinguish a confirmed-absent sidecar from an
indeterminate/unreadable one before entering the abandoned-shell grace path.
Add _session_sidecar_exists() (True/False/None) and only grace-evict when the
sidecar is confirmed absent (False); a present-or-indeterminate sidecar keeps
the session resident. Age-window bound retained (fully closes the leak, no
data-loss edge); an SSE-mounted-lease refinement is noted as optional follow-up.
2026-07-15 19:06:10 +00:00
nesquena-hermes 02e5acb890 fix(#6083 follow-up): bound never-persisted empty session shells
Codex gate found the eviction-reorder made never-saved empty shells immortal
(abandoned /api/session/new tabs accumulate unbounded past sessions_cache_max).
Add a grace-window bound: protect a never-persisted empty shell only while fresh
(<30min old) OR while it carries a composer draft; a stale, empty, draftless,
never-saved shell is treated as abandoned and becomes evictable again. A real
compose persists a draft on the first keystroke (disk-backed + reloadable), so
the brick fix is preserved while the leak is closed. +2 regression tests.
2026-07-15 18:44:21 +00:00
Sean 2a191007f1 Merge branch 'master' into fix/6010-cache-providers 2026-07-16 00:54:24 +08:00
webtecnica 40d5e17006 fix: allow manual force-check even when auto-update is disabled (#6082) 2026-07-15 12:18:04 -03:00