Reviewer regression (reproduced): the broadcast drop-oldest loop did
put_nowait -> on Full, get_nowait -> on Empty, break. But Empty here means a
concurrent consumer (the SSE handler thread) DRAINED the queue between our Full
and get_nowait — the queue now has space, which is exactly the condition to
retry the put. `break` silently discarded `item`, and when `item` was a terminal
frame (stream_end/error/cancel) the subscriber never received it, leaving the
client attached indefinitely (spinner-forever). The "# cap 0 defensive guard;
never reached" comment was the flawed assumption: any concurrent drain makes
the Empty branch reachable at any cap.
Fix: on queue.Empty, `continue` (retry the put into the now-empty queue) in
both the broadcast path and the replay path (for consistency; the replay path
runs under self._lock with a fresh queue so it's unreachable in practice, but
the same race-safe rule applies). No infinite-loop risk: each iteration either
succeeds a put (break) or succeeds a get (progress); the continue only fires on
Empty, after which the next put has space.
Test: test_terminal_frame_survives_concurrent_drain_during_drop_oldest
deterministically simulates the interleaving with a custom queue whose first
post-Full get_nowait raises Empty (as if a concurrent consumer drained it),
then runs BOTH the pre-fix `break` logic (drops the terminal frame — proving
the test bites) and the post-fix `continue` logic (delivers it).
Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent).
Codex gate on #6143 found a SILENT regression the reuse-path fix introduced:
_replace_session_db_in_kwargs early-returned the OLD handle when the rebuild
failed, but master assigned None. If the old handle was already CLOSED, both
credential self-heal callers (streaming.py:9032/10253) then built a fresh agent
around a closed SessionDB -> every persist/search fails with 'NoneType has no
attribute execute' while the chat keeps going. Guard on _session_db_is_open:
keep a live handle (subagents hold it by ref), else degrade to None (clean lazy
reinit, master behaviour). +2 regression tests.
Co-authored-by: carlotestor <carlotestor@users.noreply.github.com>
- Log failed closes in _adopt_session_db_for_cached_agent with
logger.debug(..., exc_info=True) instead of bare pass, matching
_replace_session_db_in_kwargs so a failed close (EMFILE risk) is
diagnosable.
- Drop the fragile exec/source-slice fallback in the test helper import;
real import or pytest.skip. Source-level pins still cover reverts
without importing streaming.
Background subagents (delegate_tool) capture parent_agent._session_db by
reference. The cached-agent reuse path created a fresh SessionDB every
stream request and unconditionally closed the previous handle (PR #1421
EMFILE fix). A server-side wakeup / new turn for the same parent session
could therefore close the shared handle while children were still
flushing messages, producing:
Session DB append_message failed: 'NoneType' object has no attribute 'execute'
Reproduce (observed in production):
1. Parent WebUI session runs delegate_task with N background subagents
2. Parent turn tears down / idle-hook redelivers a deferred wakeup
3. New stream for the parent adopts a fresh SessionDB and closes the old one
4. Live subagents' append_message hits conn=None
Fix: _adopt_session_db_for_cached_agent reuses a still-open handle (and
closes the unused new one — still EMFILE-safe) and only replaces when
the existing handle is missing or already closed. Same guard applied to
_replace_session_db_in_kwargs (credential self-heal). LRU eviction still
closes at true session boundaries.
Tests updated to pin the adopt helper and the subagent-safe semantics.
Memory: each connected tab's subscriber queue was queue.Queue() with no
maxsize. A slow/backpressured/backgrounded tab accumulated every coalesced
token frame for the WHOLE turn (producer = agent token stream) — unbounded
per-tab growth, OOM risk with many tabs x long agentic turns. The offline
buffer was already capped (#4633), but once a subscriber attached the
live-broadcast path flowed into an unbounded per-subscriber queue.
Cap per-subscriber queues at _SUBSCRIBER_QUEUE_MAXSIZE (== _OFFLINE_BUFFER_MAXLEN)
and drop the OLDEST frame on queue.Full in both the broadcast (put_nowait) and
offline-buffer replay (subscribe_with_snapshot) paths, so a slow tab keeps the
most recent tail (older frames stay recoverable via the run journal by
last_event_id). Mirrors SessionChannel.emit's drop-on-full contract.
The cap intentionally equals the offline buffer bound, not SessionChannel's
smaller 16: StreamChannel carries the live token stream and has a #4633
reconnect-replay contract (a resubscribing tab must receive the full retained
offline tail), so capping below the buffer would force every flaky-network
reconnect through the run journal. Matching the buffer bound preserves that
contract while bounding live-broadcast memory to the same worst-case #4633
already accepts. The SSE write deadline (SSE_WRITE_DEADLINE_SECONDS) breaks a
stuck socket within ~20s, so the overflow window is short; this also bounds it
by frame count.
Adds a cumulative subscriber_dropped_events diagnostic counter (ops visibility).
Test: tests/test_stream_subscriber_queue_cap.py — bounded queue never exceeds
cap, drop-oldest keeps newest tail, draining vs non-draining subscribers, and
the #4633 reconnect-replay contract still delivers the full offline tail.
Existing offline-buffer/gap-recovery/multitab tests unchanged.
Contract Routing
- Task type: memory-bound fix (unbounded per-subscriber queue).
- Touched areas: live chat SSE broadcast (StreamChannel).
- Relevant public docs: AGENTS.md, CONTRIBUTING.md, docs/CONTRACTS.md,
docs/rfcs/webui-run-state-consistency-contract.md (Live stream/SSE layer),
docs/rfcs/session-sse-contract-v1.md (distinct: that RFC governs the PROPOSED
per-session endpoint /api/sessions/{id}/events, NOT StreamChannel, which is
the /api/chat/stream live token broadcast; no text in that RFC is changed).
- State layer mutated: Live stream / SSE layer — specifically the per-subscriber
queue inside StreamChannel (api/config.py). NOT the durable run journal, NOT
the transcript, NOT replay.
- Invariant preserved: the #4633 reconnect-replay contract — a resubscribing
tab must receive the full retained offline tail. The cap is set EQUAL to
_OFFLINE_BUFFER_MAXLEN so replay delivery is unchanged; only the previously-
unbounded live-broadcast-to-slow-tab growth is bounded. Regression:
test_reconnect_replay_still_delivers_full_offline_tail (and the pre-existing
test_stream_offline_buffer_cap.py suite) pass unchanged.
- Scope boundaries: does not touch replay idempotency (Invariant 5), the turn
journal, compression, or sidebar metadata. Slow-tab frames dropped at the cap
stay recoverable via the run journal by last_event_id.
Release note: StreamChannel per-subscriber SSE queues are now bounded
(drop-oldest), capping memory for slow/backgrounded tabs during long turns.
Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent). No AI-specific tool
use mattered beyond the repo's standard read/edit/test workflow.
Memory: the CLI/cron session projection cache was a plain dict with TTL storage
but NO size cap and only lazy, same-key eviction. Each value is a copy.deepcopy()
of the full CLI/cron session list (the expensive projection behind #4842/#4672).
The cache key folds in a state.db content fingerprint that advances on every
streamed message, so the next poll builds a NEW key; the previous key's heavy
deepcopy was never looked up again, so its lazy-expiry path never ran. A
long-lived process under churn could accumulate orphaned heavy deepcopies until
the next structural clear_cli_sessions_cache().
Make it an OrderedDict capped at _CLI_SESSIONS_CACHE_MAX_ENTRIES (8) with
drop-oldest on write (while len > MAX: popitem(last=False)) and move_to_end on
fresh hits, mirroring the _CLAUDE_CODE_PARSE_CACHE / _SIDECAR_METADATA_CACHE LRU
pattern in the same file. TTL remains the primary freshness control; the cap is
the backstop the plain dict lacked. clear_cli_sessions_cache() keeps .clear().
Test: tests/test_cli_sessions_cache_cap.py — cache stays at len == MAX after
MAX+5 distinct keys (oldest 5 evicted), a fresh hit marks the entry
most-recently-used so it survives over colder entries, and the cache is an
OrderedDict (the regression: it was an unbounded plain dict). Existing
streaming-cache-TTL/cron-session tests unchanged.
Contract Routing
- Task type: memory-bound fix (unbounded projection cache).
- Touched areas: CLI/cron sidebar projection cache (api/models.py
_CLI_SESSIONS_CACHE).
- Relevant public docs: AGENTS.md, CONTRIBUTING.md, docs/CONTRACTS.md,
docs/rfcs/webui-run-state-consistency-contract.md (Sidebar/session metadata
layer).
- State layer mutated: Sidebar/session metadata layer — specifically the
in-memory projection cache that backs the CLI/cron sidebar rows. The cache is
a read-derivation of state.db content; it is NOT a source of truth. Dropped
entries are rebuilt on the next poll (TTL/invalidation already drive rebuilds).
- Invariant preserved — Invariant 4 (Maintenance is not activity): the cache is a
derived projection; evicting an entry cannot move a session in the sidebar or
change a row's content, because the next poll recomputes the projection from
state.db. TTL remains the primary freshness control; the LRU cap only bounds
how many STALE orphaned deepcopies linger between invalidations. Regression:
test_cli_sessions_cache_cap.py plus the existing streaming-cache-TTL and
cron-session tests (which exercise the real invalidate/rebuild path).
- Scope boundaries: does not change sidebar ordering, unread state, updated_at,
or the projection result. The cap is a memory backstop; correctness still
flows from the existing TTL + clear-on-mutation invalidation.
Release note: the CLI/cron sidebar projection cache is now LRU-bounded, capping
worst-case memory for long-lived processes under heavy streaming churn.
Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent). No AI-specific tool
use mattered beyond the repo's standard read/edit/test workflow.
Centralize custom-provider model-ID extraction into _configured_model_ids()
+ _configured_model_options(), used by both catalog construction and runtime
routing so the picker and resolver can't drift. Fixes list-shaped
custom_providers[].models allowlists silently staying on the default provider
instead of routing to their own (#6121). Existing mapping / list-of-strings
configs unaffected.
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
Co-authored-by: franksong2702 <franksong2702@users.noreply.github.com>
Reconcile the gateway pending-mirror in _session_attention_summary()
before counting sidebar approval attention, so a parent session's red
approval dot clears once the underlying gateway approval has drained.
Live approvals stay visible; clarify fallback unchanged. WebUI read-path
mitigation; child-session context-propagation root cause stays in #6100.
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
Codex found the body-reparented picker (z-200, later body child) could be revealed
UNDER a full-screen drawer/panel via multiple paths (swipe, breakpoint resize,
profile-switch). Rather than guard each open-site (whack-a-mole + my resize guard
over-fired on keyboard reflows), fix the class once: .model-dropdown--floating gets
z-index:190 — above the composer-mobile-config-panel (z-180) it opens from, below
the mobile sidebar + workspace panel (both z-200) — so any drawer/panel that opens
covers the picker, matching master's stacking intent. Reverted the two boot.js
guards. Co-authored-by webtecnica.
Codex found a sibling of the swipe-stacking edge: a direct landscape/tablet->phone
resize can re-reveal an already-open mobile sidebar or workspace panel under the
body-reparented picker (z-200 siblings, no click to trigger outside-close). The
window resize handler now guardedly closes+restores the picker before re-syncing
drawer state, gated on entering phone width AND a drawer/panel actually being open,
so ordinary keyboard/URL-bar reflows don't dismiss the picker mid-use.
Codex re-gate found the body-reparented picker (z-index:200, body child) would
paint on top of the mobile sidebar drawer when a left-edge swipe opened the
sidebar while the picker was open. _openMobileSidebarFromGesture() now calls
guarded closeModelDropdown() first (which also restores the dropdown to the
footer), so the swipe reveals the session drawer cleanly. Co-authored-by webtecnica.
The composer model dropdown (#composerModelDropdown) is a child of
.composer-footer, which sets container-type:inline-size (and, under the Geist
Contrast skin, a backdrop-filter). Both establish a fixed containing block, so
the phone position:fixed dropdown resolved against the FOOTER (pinned to the
bottom of the screen) instead of the visual viewport — dropping the menu below
the fold on all skins (the "tapping the model picker does nothing / opens
off-screen" symptom in #6080).
Fix (Fable UX gate recommendation on PR #6105): on the phone path
(<=640px) reparent the dropdown to <body> and switch it to position:fixed via a
.model-dropdown--floating class, exactly the working #profileDropdown idiom, so
fixed positioning is viewport-relative on every skin. The visualViewport math
from PR #6105 is preserved (viewport-relative coordinates, above/below flip
preferring the larger space, width = viewport - 16px, titlebar-bottom clamp) and
so are the rAF-coalesced visualViewport resize/scroll listeners.
Desktop (>640px) is unchanged: closeModelDropdown() and the desktop branch of
_positionModelDropdown() restore the element into its original .composer-footer
slot and clear the phone inline styles, so the desktop popover stays an
absolutely positioned footer child, byte-for-byte identical to master. Because
the menu now leaves the footer on phones, no per-skin backdrop-filter override
is needed.
Adds tests/test_issue6080_model_picker_reparent.py, including a node-executed
DOM-parentage guard that runs _positionModelDropdown()/closeModelDropdown()
against a stub DOM and asserts the open phone dropdown has NO .composer-footer
ancestor (escaping the container-type trap) and is restored into the footer on
close and on desktop.
Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com>
Scoped to the two stated fixes only: strips the #composerModelDropdown position:fixed
mobile override that #6104's branch had bundled in (that's the #6080 model-picker
mechanism, being reworked separately to reparent to body level — the fixed override
is trapped by .composer-footer's container-type so it doesn't belong here), and drops
the <=360px tool-name-hiding rules (header overflow-clip prevents the spill without
losing tool identity). Thanks @webtecnica.
Three fixes in one commit:
1. **CI test mock** — Added addEventListener/removeEventListener to the
Node.js makeElement mock so viewport.addEventListener() calls in the
touch handler registration don't crash with 'not a function'.
2. **Two-finger pan during pinch** — Replaced _setScale() call with
direct transform math. The new formula computes scale and position
simultaneously:
ratio = boundedScale / pinchStartScale
x = currMidX - (startMidX - startPosX) * ratio
This keeps the diagram under both fingers regardless of midpoint
drift, enabling simultaneous zoom + pan.
3. **Post-pinch click suppression** — _onTouchEnd now sets dragged=true
when exiting pinch mode so any synthesized click event is caught
by the existing dragged guard in _openViewerOnClick instead of
opening the lightbox.
Adds touch event handlers (touchstart/touchmove/touchend/touchcancel) to
the Mermaid viewer viewport for two-finger pinch zoom, using the existing
_setScale() function with midpoint anchoring.
- Guards pointer handlers (down/move/click) during active pinch to prevent
interference between touch and pointer event systems
- Uses {passive: false} on touchstart/touchmove to allow preventDefault()
- Falls back gracefully on non-touch devices — touch listeners are inert
without touch hardware
Fixes: Mermaid diagrams in Hermes WebUI were not zoomable via multi-touch
pinch on mobile/tablet devices, frustrating users with complex diagrams.
Codex re-gate found the same derived-title leak on the sibling /api/sessions/search
endpoint (3 branches redacted only 'title'). Extract _redact_sidebar_title_fields()
shared helper, apply it in both _sidebar_session_response_item and all 3 search
branches, read api_redact_enabled once per search response (mirrors #4662). +helper
test + source-guard that every search branch calls the redactor.
nesquena-hermes