jinghui/hermes-webui
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/master' into HEAD

Browse Source
This commit is contained in:
carlotestor committed 2026-09-23 16:34:11 +00:00
2 parents 4e17333516 df491ee1b6
commit 2a742e13d6
148 files changed
+17501 -950
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.env.example
+7
View File
@@ -132,6 +132,13 @@
# Extra sources for the notes drawer (see docs for the expected format).
# HERMES_WEBUI_EXTERNAL_NOTES_SOURCES=
# Sidebar recency window: how many recent sessions are listed (default: 20).
# Also caps how many delegated subagent children can nest at once; raise it
# for wide fan-outs. Invalid or non-positive values fall back to 20.
# Values above 200 are clamped. Resolved before profile init, so a profile
# .env cannot override this server-wide bound.
# HERMES_WEBUI_VISIBLE_SESSION_LIMIT=20
# Directory scanned for WebUI plugins (default: ~/.hermes/plugins).
# HERMES_WEBUI_PLUGINS_DIR=~/.hermes/plugins
.github/workflows/conversation-lifecycle.yml
+18 -1
View File
@@ -12,7 +12,9 @@ on:
- 'api/**.py'
- 'server.py'
- 'tests/browser_conversation_lifecycle.py'
- 'tests/sidebar_terminal_barrier_server.py'
- 'tests/browser_historical_transcript_hydration.py'
- 'tests/browser_reconnect_scene_redraw.py'
- 'requirements*.txt'
- 'pyproject.toml'
- '.github/workflows/conversation-lifecycle.yml'
@@ -23,7 +25,9 @@ on:
- 'api/**.py'
- 'server.py'
- 'tests/browser_conversation_lifecycle.py'
- 'tests/sidebar_terminal_barrier_server.py'
- 'tests/browser_historical_transcript_hydration.py'
- 'tests/browser_reconnect_scene_redraw.py'
- 'requirements*.txt'
- 'pyproject.toml'
- '.github/workflows/conversation-lifecycle.yml'
@@ -32,7 +36,8 @@ jobs:
live-to-final:
runs-on: ubuntu-latest
timeout-minutes: 10
continue-on-error: true
# Keep existing proof rows informational; reconnect regressions fail CI.
continue-on-error: ${{ matrix.name != 'reconnect-scene-redraw' }}
strategy:
fail-fast: false
matrix:
@@ -45,6 +50,9 @@ jobs:
scenario: terminal-error
- name: historical-transcript-hydration
script: tests/browser_historical_transcript_hydration.py
- name: reconnect-scene-redraw
script: tests/browser_reconnect_scene_redraw.py
browsers: chromium
steps:
- uses: actions/checkout@v4
@@ -69,6 +77,7 @@ jobs:
LIFECYCLE_ARTIFACT_DIR: lifecycle-artifacts
HISTORICAL_HYDRATION_ARTIFACT_DIR: lifecycle-artifacts
LIFECYCLE_SCENARIO: ${{ matrix.scenario }}
BROWSERS: ${{ matrix.browsers }}
run: python ${{ matrix.script }}
- name: Run settlement frame proof
@@ -79,6 +88,14 @@ jobs:
LIFECYCLE_TEST_BITE: settle-worklog-frame-proof
run: python tests/browser_conversation_lifecycle.py
- name: Recover a missing terminal frame
if: matrix.scenario == 'normal'
env:
LIFECYCLE_ARTIFACT_DIR: lifecycle-artifacts/missing-terminal
LIFECYCLE_SCENARIO: normal
LIFECYCLE_TEST_BITE: sidebar-idle-missing-terminal
run: python tests/browser_conversation_lifecycle.py
- name: Upload failure artifacts
if: failure()
uses: actions/upload-artifact@v4
AGENTS.md
+33
View File
@@ -67,6 +67,39 @@ Follow that checklist's safety rules:
the manual verification performed.
- For runtime, streaming, recovery, replay, compression, or sidebar metadata
changes, name the state layer being mutated and prove the relevant invariant.
- Active-run Steer resolves the stream-bound agent with explicit stream and
worker ownership before consulting the reusable session cache. Compression
may rotate the agent identity; steering must never evict or close an agent.
Keep HTTP response writes outside runtime registry locks. A Gateway-owned
active run must resolve to the Gateway outcome before any local cache fallback,
even when no in-process worker is registered for that stream. Stop publishes
cancellation and detaches stream/agent entries under the same stream lock
(STREAMS_LOCK -> ACTIVE_RUNS_LOCK); interrupt and session persistence remain
outside it. The retained cache-only path (no registered worker) revalidates
stream membership, owner, and active-run session/backend/phase and enqueues
agent.steer() under that same lock edge, so a Stop that claims cancellation
never strands guidance an earlier Steer response reported as accepted. Test
both Stop/Steer orderings — registered and cache-only — with deterministic
barriers. Initial active-run publication and its cancel flag must share that
lock edge with Stop; do not recreate the flag after journal setup. Worker
registration must also check its retained cancel event and live stream
membership: Stop can remove CANCEL_FLAGS during initialization.
Do not re-register a cancelled worker; finalize outside the stream lock.
Local Steer accepts only explicit starting/running phases. Close admission
by publishing finalizing under STREAMS_LOCK before the last pending-steer
drain; earlier accepted guidance is drained, later guidance is rejected with
`not_running` while the owned stream is still live (not `stream_dead`). Use
one idempotent terminal settlement before done/error/end and final cleanup,
covering returned errors, exceptions and self-heal, not only the success path.
Merge Agent-returned `pending_steer` with the registered worker's final slot
drain and emit leftovers before terminal events, outside registry locks.
Test both drain/Steer orderings, including compression-rotated identities.
- Inactive-session recovery is separate from live Steer. Resolve durable
compression lineage in the session's profile database, read-only, even when
the WebUI sidecar has no snapshot flag. Never reopen a sealed parent. Reject
stale chat POSTs before workspace/model/pending-state mutation; the browser
loads the continuation and preserves the draft without automatic replay.
Explicit closures and unknown terminal reasons do not authorize a redirect.
- For Docker build changes in `docker_init.bash`, mirror directory exclusions
in both the `rsync` and `cp -a` paths — `/opt/hermes` may contain subdirectories
with restricted permissions (e.g. `.playwright/`).
ARCHITECTURE.md
+23
View File
@@ -237,6 +237,29 @@ Session is a plain Python class (not a dataclass, not SQLAlchemy):
title_from(): takes messages list, finds first user message, returns first 64 chars.
Called after run_conversation() completes to set the session title retroactively.
#### Session transcript reconciliation with `state.db`
`reconciled_state_db_messages_for_session()` uses
`merge_session_messages_append_only()` to combine a WebUI sidecar or context
projection with active Agent `state.db` rows. Its explicit
`incoming_provenance="state_db"` fence permits a state-only row whose timestamp
predates the sidecar tail to use a safe chronological slot. Paginated
`msg_limit` consumers rely on this merged order directly rather than applying a
later timestamp sort.
The same merge helper also stitches ordered child sidecars onto archived
compression parents. Those calls leave incoming provenance unverified, so their
stable message sequence remains append-only even when parent rows carry later,
restamped timestamps. Timestamp alone is never authority to move a continuation
inside its parent transcript.
If an older row could only be placed before the first surviving sidecar/context
row, the insertion helper declines it to avoid resurrecting compacted history.
Reconciliation then appends that row instead of dropping it; rows without a
usable timestamp and rows at or after the sidecar tail also append normally.
The fallback therefore preserves an accepted state-only row when exact ordering
is ambiguous, while safely placeable recovery rows remain chronological.
#### Imported `state.db` sidebar projection
`api.models.get_cli_sessions()` projects conversations from the active Hermes
CHANGELOG.md
+249
View File
@@ -3,8 +3,256 @@
## [Unreleased]
### Added
- **Full-session resolve concurrency is configurable.** `HERMES_WEBUI_MAX_SESSION_RESOLVE` sets how
many full-transcript session resolves may run at once (default 2, a positive integer up to 64;
zero, negative, non-numeric or out-of-range values fall back to 2). It is process-wide, so a
profile's `.env` cannot override it. (#7421, #7656 by @happy5318)
- **The sidebar's recent-session window is configurable.** `HERMES_WEBUI_VISIBLE_SESSION_LIMIT` sets
how many recent sessions the sidebar lists (default 20). It also bounds how many delegated subagent
children can nest at once, so raise it for wide fan-outs. Invalid or non-positive values fall back to
20, values above 200 are clamped, and it is resolved before profile init so a profile `.env` cannot
override it. (#7631 by @carlotestor)
### Performance
- **Opening a session while its task is still running is much faster.** Rebuilding the live
snapshot from the run journal parsed the journal twice, walked every metering row and grew the
reasoning text with repeated string concatenation, which is quadratic on long runs. The journal
is now parsed once, metering rows are skipped (their timestamp watermark is kept) and reasoning
deltas are joined once per segment. On a 15 MB / 22.7k-event journal the author measured the
rebuild going from 4.34 s to 0.49 s with a byte-identical snapshot. The interim-echo check now matches a
compact-equivalent suffix without a fixed window, so an echo stretched by interior whitespace is
no longer shown twice. (#7310, #7569 by @happy5318)
### Fixed
- **A session deleted during a restart no longer produces a spurious recovery warning.** When
WebUI startup recovery re-attached background processes, a session that had vanished between
enumeration and rebind raised a `KeyError` that was logged as a warning. It now follows the
existing skip path, confined to the session lookup, so the vanished owner is skipped, live owners
still rebind, and registry errors still warn. (#7753, #7774 by @happy5318)
- **Waiting on the Agent's session lease is shown as a warning instead of looking stuck.** When
another Hermes process (gateway, CLI or cron) holds the session's turn lease, the Agent's
"another Hermes process is using this session" notices now reach the chat as a warning status
instead of being dropped, and the status clears when the run ends. Classification keys on the Agent
status kind (`lifecycle` / `warn`), so user-authored text can never be promoted to a warning.
(#7760 by @ruizanthony)
- **A stale in-flight projection can no longer reach a gateway watcher after its last subscriber
leaves.** Final unsubscribe and queue eviction now invalidate the cache and fence projections that
were already in flight, without holding the lock across database reads or SSE writes, so a client
that re-subscribes gets a fresh snapshot instead of a stale one. (#7761 by @ruizanthony)
- **A burst of "session busy" refusals no longer drops a background-task completion.** When
`start_session_turn()` refused a completion wake-up with a transient 409 (Agent runtime stale,
process wake-ups paused, or the session busy with another turn), the bridge released the durable
claim as a plain failure, so a few refusals in a row could terminally drop a completion whose
session was alive and waiting. Transient refusals now return the claim as retryable, while hard
failures still use up the attempt budget. (#7758 by @ruizanthony)
- **A first visit now uses the browser's language.** The server stores "no preference" as `null`
instead of defaulting to `"en"`, so a first-time visitor gets `navigator.language` while an
explicitly saved language (including English) still wins, and legacy `settings.json` files that
already hold `"en"` keep English. Reading the browser language is guarded, so an environment where
`navigator` throws falls back cleanly. (#7622, #7730 by @happy5318)
- **The live model list for a custom provider respects its `models:` allowlist, without treating
per-model metadata as one.** `/api/models/live` filters a custom provider's live catalog to its
configured `models:` when that is a list, or when it is a mapping with `discover_models: false`.
A mapping of per-model settings (the shape `hermes setup` writes) keeps the full live catalog, as
the Agent does. (#7165 by @happy5318)
- **`MEDIA:` links wrapped in inline code no longer 404.** Every `MEDIA:` capture site (renderer,
streaming parser, TTS stripper, session-media authorization and snapshot capture, seven in all)
swallowed the closing backtick of `` `MEDIA:/path` `` into the path, so the file lookup and the
session allowlist both missed. Backtick-wrapped refs are now normalized first, while bare paths that
genuinely contain a backtick keep their full name. (#7359, #7708 by @happy5318)
- **A turn's Worklog no longer vanishes when the sidebar reports idle before the final frame.**
`/api/sessions` could say a session was idle before the chat stream's terminal frame reached the
page, and three sidebar paths (idle reconciliation, the INFLIGHT purge and optimistic-row
retirement) then erased the pane's Worklog and stream id, so the late `done` was rejected as stale
and the settled scene was never saved. The cleanup now waits briefly for the pane's own open
stream, then checks `/api/chat/stream/status` once, and falls back to the existing
interrupted-stream recovery if that check fails. (#7749 by @franksong2702)
- **A session's run journal can no longer be written out of order.** The journal writer reserved a
sequence number under the per-path lock, released it, and then appended, so two concurrent writers
could land sequence N+1 on disk before N and the replay reader would stop at the gap
(`replay_noncontiguous`). Sequence allocation and the physical append now happen under the same
existing lock. (#7751 by @franksong2702)
- **A stopped chat can no longer publish or reuse its agent after Stop.** A worker that was
cancelled could still publish its cached Agent or invoke it after Stop landed, and its late
cleanup could close or evict the same Agent object a successor turn had just picked up. The initial
path and both credential self-heal paths now take one Stop admission, the cancellation event, live
stream and exact Agent are rechecked immediately before invocation (no registry lock is held across
provider or tool execution), and cache/lifecycle handles are retired only while the surviving owner
still matches. (#7748 by @franksong2702)
- **Reconnecting to a running session no longer redraws every tool card over and over.** After a
reconnect restored the live activity scene from the run journal, the client also replayed its older
cached in-flight tool list on top, so a turn with N tool cards redrew them N times. When the
journal-backed scene restores successfully the replay is now skipped (newer rows still arrive
through the reattached stream); the replay stays for legacy restores and for a failed or
unavailable scene. (#7436, @atchisonbrent)
- **A settled assistant answer is no longer shown twice (#2051).** Two client-side paths could put a
second copy of a finished answer on screen after a turn settled: a stale live-turn node that was
still treated as live, and a settled rebuild branch that appended a turn instead of replacing it.
The same fix stops an interrupted turn's notice from appearing twice, and stops the composer's
model picker from listing one configured custom-provider model twice. The stored transcript was
always correct; only the rendering duplicated it. (#7374, @Thireus)
- **A hidden browser tab stops polling a session that was deleted.** When a tab is in the
background, its stream poll kept asking for a session every few seconds after it was deleted or
archived away, forever. A `404`/`410` for the polled session now stops that poll; `503` and network
errors keep retrying, a late `404` for one session cannot stop another session's poll, and making
the tab visible again reopens the live stream. (#7299, #7716 by @happy5318)
- **The OpenRouter setup no longer offers a model OpenRouter doesn't serve.** Onboarding offered
`z-ai/glm-4.5-flash`, which is not in OpenRouter's catalog, so picking it failed on the first
message. That slot now offers `z-ai/glm-4.5-air`, Z.AI's nearest light model. The direct Z.AI
setup still offers GLM-4.5 Flash. (#7520, #7734 by @MuhammadUsamaMX)
- **Opening the sidebar can no longer stall the agent's writes to `state.db`.** Several WebUI
paths that only read the agent's `state.db` could quietly become writers. The read-only opener
fell back to a writable connection when `mode=ro` failed. The session listing self-healed a
missing `idx_messages_session` with `CREATE INDEX` through its own writable connection, which
holds the SQLite writer lock for minutes on a large `messages` table. The cron sidebar, insights
and deep-health checks opened the database with a bare `sqlite3.connect()`. Every one of these
readers now opens strictly read-only (`file:...?mode=ro`) and never creates an index. A missing
index falls back to the existing pre-aggregated listing. The gateway watcher also no longer polls
the database when no client is subscribed. Operators with an older agent can create the read
indexes in a drained maintenance window with `scripts/ensure_state_db_read_indexes.py`; see
`docs/troubleshooting.md`. (#7445 by @ruizanthony)
- **A failed chat launch no longer leaves the session stuck "running".** If the worker thread
couldn't start, its stream ownership and the session's pending-stream fields were never cleared,
so the session could look permanently active and the registries grew. Launch failure now clears
exactly that stream's records, after releasing the chat-start lock, and never evicts a successor
stream. (#6869, #6937 by @jbdrak)
- **A compressed conversation no longer shows up twice in the sidebar.** When context
compression started the continuation a few milliseconds before the parent was marked ended, the
continuation wasn't recognised. The sidebar then showed a duplicate same-title row plus a
spurious child-session entry, and opening the conversation didn't stitch the transcript. Both the
sidebar and the transcript stitcher now accept a bounded (2 s) early start when every other
lineage signal agrees: same source, direct parent link, a compression (or `cli_close`) end reason and not a fork.
(#6931, #7021 by @webtecnica)
- **The profile switcher no longer 500s in a two-container Docker setup.** With the agent
source not mounted (`HERMES_WEBUI_CHAT_BACKEND=gateway`), `GET /api/profiles` fell through to
a skills-stats fallback that imported `agent.skill_utils` unguarded, so the missing module
surfaced as an error on every profile-list load. The import is now guarded and the skill
stats report as unknown in that case, so the picker stays usable (the skills line is simply
omitted). Thanks @webtecnica. (#7305, #7312)
- **A title that mixes Chinese, Japanese or Korean with English terms is no longer rejected.**
The cross-script guard that stops a generated title from drifting into the wrong language
treated the borrowed Latin words in a CJK title as drift, so a valid title like
`WeChat Pay 回调失败排查` or `Python 代码修复` was thrown away and the session kept its
fallback title. Latin terms are now accepted when the title also contains CJK text, while
an all-Latin title for a CJK conversation is still rejected. Thanks @MuhammadUsamaMX.
(#7693, #7727)
- **A late-arriving prompt no longer renders below the reply it asked for.** When a message
reached the transcript from `state.db` after the sidecar had already been merged⟪HERMES-CONTEXT-COMPRESSION: 809 of 1,009 chars omitted here by Hermes's context compressor. This is NOT part of the original tool call and must never be reproduced in new output — always write full, untruncated content.⟫- **A rejected request no longer poisons the next one on the same connection.** `server.py`
is a raw HTTP/1.1 handler where `rfile` is the socket itself, so answering a request
before reading its body left those bytes queued. The next request on a keep-alive
connection was then parsed starting mid-body, and the client got
`400 Bad request syntax ('{"a": "b"}GET /api/health HTTP/1.1')` — an error naming a
request it never sent, which is expensive to diagnose from the client side. Every
reject path now arms `Connection: close` when a body is still pending, and framing is
validated strictly (RFC 9110 `1*DIGIT`, duplicate and comma-combined `Content-Length`
reconciled, every `Transfer-Encoding` refused since nothing here decodes one). A
genuinely bodyless rejection keeps its keep-alive, so healthy pooled connections are
not dropped. Thanks @rodrigogs. (#7550, #6658)
- **Opening a session that belongs to another profile now offers to switch to it instead of
looking deleted.** Several cross-profile guards answered `404 Session not found`, which the
front-end treats as a missing session and self-heals by clearing the URL and local storage —
so a valid deep link into another profile's session destroyed its own way back. Those guards
now return the same `409 session_profile_mismatch` the detail-load endpoint has used since
#5419, while a genuinely missing session still 404s and still self-heals. The clarify card,
compression-recovery card and manual-compression flow were each treating *every* `409` as
their own "stale" signal and are now scoped so a cross-profile refusal no longer hides a live
prompt or reports a false compression failure. Thanks @happy5318. (#7710, #7714)
- **An OIDC-only or passkey-only deployment no longer shows a dead password prompt.** The
`/login` page rendered the password input, submit button and passkey control
unconditionally, so an instance with native OIDC configured and no
`HERMES_WEBUI_PASSWORD` displayed a password form that silently 401'd every submit. The
controls are now gated on the auth methods actually configured, with the passkey button
kept for passwordless-passkey instances where it is the only login affordance. Thanks
@happy5318. (#7056, #7715)
- **A model whose native id contains a slash no longer poisons its configured-model badge.**
The badge builder synthesised a `{provider}/{model}` alias alongside the bare id, so a
model already carrying a slash (for example `commandcode` + `deepseek/deepseek-v4-flash`)
produced `commandcode/deepseek/deepseek-v4-flash` — a non-functional id that leaked into
the badge map, persisted into session state and returned `HTTP 400` from the agent. Only
the bare id and the `@provider:model` form are emitted now. Thanks @happy5318. (#7290, #7709)
- **A conversation whose history carries an explicit `null` tool-call list no longer breaks
the tool-call summary.** `_extract_tool_calls_from_messages` used `.get('tool_calls', [])`,
which returns the default only when the key is *absent* — when a stored message carried the
key with a `null` value, the code tried to iterate `None` and raised `TypeError`. Reading
such a session now skips the empty entry and still summarises the tool calls that follow it.
Thanks @KayZz69. (#7265)
- **The opencode-go provider lists its real models again instead of a frozen snapshot.** Model
discovery had fallen back to a hard-coded catalog, so models added or removed upstream never
appeared. The provider now queries live, with the lookup scoped to opencode-go alone: a slow
or unreachable endpoint falls back to the static catalog rather than blocking the picker for
anyone else, older Agent builds that lack the discovery API keep the previous behaviour, and a
configured model allowlist still wins over whatever discovery returns. Thanks
@shameez-struggles-to-commit. (#7220)
- **Denying a gateway approval now retires the request instead of leaving the run waiting.**
When an approval routed through the gateway was denied, the deny reached the agent but the
local producer was never retired, so the run could sit waiting on a decision that had already
been made. Denial now settles the producer atomically, scoped to the exact
`(session_id, run_id, approval_id)` that was answered — sibling approvals from the same run
and approvals from other runs are left untouched — and repeated or late responses are bounded
no-ops rather than resurrecting a settled request. Thanks @snoyberg. (#7570)
- **The approval card's "Skip all this session" button no longer shows two lightning bolts.**
The button rendered its ⚡ twice — once from the icon span in the markup and again from the
translated label, which carried its own leading glyph in 14 of 15 locales. The icon now comes
only from the markup, matching every sibling button on the card (Allow once / Allow session /
Always allow / Deny all pair an icon element with a glyph-free label), and translators no
longer carry the symbol in their strings. (#7701)
- **The gateway watcher no longer polls `state.db` around the clock with nobody listening.**
Its poll loop re-fingerprinted the gateway state database every few seconds whether or not
any SSE client was attached, and slept in 0.1s increments — roughly 10 wakeups a second, all
day, on an idle server. The loop now parks on an event when there are no subscribers and is
woken by the first one, so an idle instance does no polling work at all; while subscribed it
waits on a single timer that still returns immediately on shutdown. Connecting a client
remains prompt — the first subscriber unparks the loop rather than waiting out the poll
interval. Thanks @DevNexsler. (#7694)
- **A long-running turn no longer replays a stale token count after a reload.** The run-journal
recorded live metering frames, so reattaching to a stream — or reloading a tab mid-turn —
could replay a snapshot from earlier in the same run and briefly show token/TPS figures that
had already been superseded. Metering is now live-only and never journaled, and replayed
frames no longer carry an event id that could advance the client's resume cursor past real
content. Reattach and reload now show the current numbers for the turn in progress. Thanks
@laitekin. (#7291)
- **Steering a conversation works again after the context is compressed.** When compression
rotated `agent.session_id`, a steer could no longer find the active worker: it was either
silently dropped or accepted and never delivered. Steers now resolve the owning worker
directly rather than relying on a cache lookup that compression invalidates, and every
terminal exit — normal completion, returned error, raised exception and self-heal — passes
through one idempotent settle boundary, so guidance can no longer be stranded between the
final drain and teardown. Two related defects are fixed alongside it: **Stop now actually
stops** a run that had already passed preflight (previously the worker could consult a
removed registry entry, miss the cancellation, and continue to completion), and a live
`finalizing` run is no longer reported as `stream_dead`, which had caused the client to tear
down its state while the stream was still alive. Compressed sessions from the CLI, TUI,
Desktop and ACP now resume correctly, and installations running an older Agent keep their
existing sidecar recovery instead of being left unresumable. Thanks @ruizanthony. (#7546)
- **Slash-command autocomplete stops offering commands the WebUI cannot run.** The composer's
`/` menu announced all 51 registered commands, but many are CLI-only — picking one produced
a command that went nowhere. The menu now announces only the WebUI-dispatchable subset (16),
so the difference shows up where it matters: typing a prefix like `/a` or `/re` no longer
fills the list with dead options. Filtering is confined to the suggestion list — manually
typing a CLI-only command behaves exactly as before, and plugin commands plus the native
`/moa`, `/sessions`, `/resume` and `/pet` remain available. Thanks @webtecnica.
- **Concurrent background completions stop burning the async-delegation delivery budget.**
When several delegated runs finished at once and idle-woke the same session together, each
wakeup consumed a delivery attempt before discovering the others, so the budget could be
@@ -46,6 +294,7 @@
### Changed
- **Two flaky tests are stable again.** Three `get_available_models()` cache-metadata tests raced the 4-second live-rebuild budget on a loaded CI host and could time out onto the stale-cache path. They now pin the budget to `0`, which is the documented setting for the legacy unbounded synchronous rebuild, so they still exercise the real rebuild (#7735). A source-text oracle that string-matched `delete_cli_session`'s source to prove it opens a writable connection is removed. The behavioral test in `test_issue1494_state_db_fd_leak.py` still guards that contract: it fails with `attempt to write a readonly database` if the delete path is ever switched to a read-only connection (#7726). Test-only; no runtime change. Thanks @webtecnica. (#7744, #7742)
- **The chat composer grows natively instead of being resized by JavaScript on every keystroke.** `autoResize()` measured `scrollHeight` and wrote `style.height` on each input event — a forced synchronous reflow on the most-typed-in surface in the app. Browsers that support CSS `field-sizing: content` (Chromium today; also Firefox 152 and Safari 26.2) now own the geometry directly, gated on `CSS.supports()`, and the existing JavaScript path is untouched for every other engine. Measured behaviour is identical across both paths: 44px resting height, no jump when the first character is typed or the last deleted, growth to the 200px ceiling, then internal scrolling. Because `field-sizing` deliberately includes placeholder text in content sizing, `:placeholder-shown` pins fixed sizing while the composer is empty so a long placeholder can't inflate it. Thanks @starship-s. (#6760, #5514)
### Fixed
README.md
+1
View File
@@ -373,6 +373,7 @@ Full list of environment variables:
| `HERMES_HOME` | Windows: `%LOCALAPPDATA%\hermes`; POSIX: `~/.hermes` | Base directory for Hermes state (affects all paths) |
| `HERMES_CONFIG_PATH` | `$HERMES_HOME/config.yaml` | Path to Hermes config file |
| `HERMES_WEBUI_SERVER_CWD` | *(unset)* | Working directory for the server process. Defaults to the agent dir; point it at a writable workspace when the agent dir is read-only so fallback relative writes land somewhere writable |
| `HERMES_WEBUI_VISIBLE_SESSION_LIMIT` | `20` | Size of the sidebar's interactive recency window (how many recent non-cron/webhook sessions are listed). Also bounds how many delegated subagent children can nest at once, since a child only renders when its row wins a slot in the window — raise it for wide fan-outs. Non-integer or non-positive values fall back to the default. Values above 200 are clamped. Resolved before profile init, so a profile `.env` cannot override it |
| `HERMES_WEBUI_AGENT_CACHE_MAX` | `25` | Max live agent instances kept warm in the in-memory LRU. Each pins a full conversation transcript, so this is the dominant lever on resident memory — lower it on installs with many long sessions to cap RAM (at the cost of more cold reloads) |
| `HERMES_WEBUI_SESSIONS_MAX` | `100` | Legacy operator override for the max compact `Session` objects held in the in-memory LRU. Prefer the `webui.sessions_cache_max` key in `config.yaml` (which takes precedence); this env var remains a fallback. Bounds resident memory so long-running installs cannot accumulate every session ever touched and eventually crash (#4765/#2233/#4633). Eviction only ever drops clean, persisted, non-active sessions; an evicted session lazily reloads from its JSON sidecar on next access |
TESTING.md
+29 -6
View File
@@ -154,12 +154,35 @@ HISTORICAL_HYDRATION_TEST_BITE=break-tool-link \
python tests/browser_historical_transcript_hydration.py
```
The dedicated `Conversation lifecycle (informational)` workflow runs the current
proof rows (`normal`, `terminal-error`, and `historical-transcript-hydration`) and
stays non-blocking while the public
matrix expands to additional behavior rows. The maintainer's private QA harness
remains broader; later public slices will add session switching, reconnect/replay,
cancellation, compression, and recovery.
The dedicated `Conversation lifecycle (informational)` workflow keeps the existing
proof rows (`normal`, `terminal-error`, and `historical-transcript-hydration`)
non-blocking while the public matrix expands. The Chromium
`reconnect-scene-redraw` row does **not** allow failures: it exercises the real
`loadSession` reconnect path and fails the workflow on a regression. Required
merge checks remain a maintainer-controlled repository setting.
The maintainer's private QA harness remains broader; later public slices will
add cancellation, compression, and recovery coverage.
### Active-session reconnect redraw gate
`tests/browser_reconnect_scene_redraw.py` is an opt-in deterministic browser
gate for reconnecting to a running session with a large Anchor activity scene.
It uses an isolated temporary server/home and fixture SSE events—no agent,
provider credentials, or production state. By default it checks Chromium and
WebKit, desktop and 390px viewports, and all three activity display modes:
```bash
pip install playwright
python -m playwright install chromium webkit
python tests/browser_reconnect_scene_redraw.py
```
`TOOL_COUNT`, `BROWSERS`, and `MODES` narrow the matrix. `MEASURE_BASELINE=1`
records an unoptimized baseline without enforcing the redraw ceiling;
`METRICS_FILE` saves JSON metrics and `SCREENSHOT_DIR` saves generated-fixture
screenshots. This gate tests page reconstruction, journal-cursor resume, and
subsequent fixture SSE updates. It does not test a real provider/network or PWA
service-worker cache behavior.
### Streaming reader intent
api/agent_sessions.py
+141 -59
View File
@@ -1,8 +1,12 @@
"""Shared helpers for reading Hermes Agent sessions from state.db."""
import json
import logging
import os
import sqlite3
import sys
from contextlib import closing
from pathlib import Path
from pathlib import Path, PurePosixPath, PureWindowsPath
from urllib.parse import quote, quote_from_bytes
logger = logging.getLogger(__name__)
@@ -18,36 +22,71 @@ logger = logging.getLogger(__name__)
_SOURCE_COLUMN_WARNED_DB_PATHS: set[str] = set()
def state_db_file_uri(db_path, platform: str | None = None) -> str:
"""Build the ``file:`` URI (no query string) for an absolute ``state.db`` path.
``Path.as_uri()`` is not usable here: for a UNC share it emits
``file://server/share/state.db`` and SQLite rejects any non-local URI
authority unless compiled with ``SQLITE_ALLOW_URI_AUTHORITY`` (the
CPython 3.11-3.13 Windows builds are not). SQLite instead accepts the
*empty*-authority spelling ``file:////server/share/state.db``, whose path
``//server/share/...`` the Windows VFS opens as the UNC name. Drive-letter
paths keep the documented ``file:///C:/...`` form and POSIX paths are
unchanged. Only ``/`` survives unescaped, so ``?`` and ``#`` in path
components cannot leak into the query string.
``platform`` defaults to the running interpreter; tests pass it explicitly
so the Windows shapes are checked from any host.
"""
platform = platform or sys.platform
if platform == "win32":
win = PureWindowsPath(str(db_path))
drive = win.drive
if drive.startswith("\\\\?\\"):
# Extended-length prefix: "\\?\C:" or "\\?\UNC\server\share".
drive = drive[4:]
if drive.upper().startswith("UNC\\"):
drive = "\\\\" + drive[4:]
parts = win.parts[1:] # drop the anchor, keep the path components
if drive.startswith("\\\\"):
host_share = drive[2:].replace("\\", "/")
posix_path = "//" + "/".join((host_share, *parts))
else:
posix_path = "/" + "/".join((drive, *parts))
# ``:`` stays literal so the drive letter keeps SQLite's documented
# ``file:///C:/...`` shape (``Path.as_uri()`` leaves it unescaped too).
return "file://" + quote(posix_path, safe="/:")
posix_path = PurePosixPath(str(db_path)).as_posix()
# Quote the filesystem BYTES, not the str: a POSIX path component that is
# not valid UTF-8 is carried in the str as ``surrogateescape`` code points,
# which ``quote(str)`` rejects with UnicodeEncodeError (the caller then
# treats the db as unreadable and every agent-backed session vanishes).
# ``Path.as_uri()`` — what master used — percent-encodes os.fsencode()
# bytes; this keeps that behavior.
return "file://" + quote_from_bytes(os.fsencode(posix_path), safe="/")
def state_db_readonly_uri(db_path, platform: str | None = None) -> str:
"""Strict read-only (``mode=ro``) form of :func:`state_db_file_uri`."""
return state_db_file_uri(db_path, platform=platform) + "?mode=ro"
def open_state_db_readonly(db_path: Path, log: logging.Logger | None = None) -> sqlite3.Connection:
"""Open the live agent ``state.db`` read-only for a pure-read projection.
Same rationale as the session-listing path (#5455): a write-capable handle
on the multi-GB, WAL ``state.db`` while the agent streams into it adds
needless checkpoint/lock surface. The read-only ``file:...?mode=ro`` URI
avoids that. Falls back to a writable connection (and warns) if the
read-only open fails, so callers never lose data on exotic filesystems.
avoids that. Read failures propagate; a reader never upgrades to a writer.
The caller must ensure ``db_path`` exists — this raises ``FileNotFoundError``
for a missing path rather than letting the writable fallback below create an
empty, writable ``state.db`` there (a ghost DB in the agent's HOME). The
fallback is only for an *existing* DB whose read-only open fails on an exotic
filesystem, so a real read never loses data.
for a missing path rather than creating a ghost database.
Callers own the returned connection (wrap it in ``contextlib.closing``).
"""
log = log or logger
if not db_path.exists():
raise FileNotFoundError(f"agent state.db not found: {db_path}")
read_only_uri = f"{db_path.resolve().as_uri()}?mode=ro"
try:
return sqlite3.connect(read_only_uri, uri=True)
except sqlite3.Error as exc:
log.warning(
"agent state.db read-only open failed for %s; falling back to writable connection: %s",
db_path,
exc,
)
return sqlite3.connect(str(db_path))
return sqlite3.connect(state_db_readonly_uri(db_path.resolve()), uri=True)
MESSAGING_SOURCES = {
@@ -65,6 +104,15 @@ MESSAGING_SOURCES = {
CLI_MIN_UNTITLED_MESSAGE_COUNT = 6
CLI_MIN_UNTITLED_USER_MESSAGE_COUNT = 2
# Sub-second scheduling/write-order races during a compression/cli_close
# handoff can persist the continuation row with ``started_at`` a few
# milliseconds BEFORE the parent's ``ended_at`` lands (#6931). The tolerance
# below lets those rows still classify as the next segment; the fork /
# model_config branch-marker / cross-source / end_reason guards in
# ``_is_continuation_session`` already rule out unrelated rows, so a small
# window cannot collapse genuinely concurrently started children.
CONTINUATION_STARTED_AT_TOLERANCE_SECONDS = 2.0
SOURCE_LABELS = {
'acp': 'ACP',
'api_server': 'API',
@@ -318,6 +366,30 @@ def is_cli_session_row_visible(row: dict) -> bool:
return _count_user_turns(row) >= CLI_MIN_UNTITLED_USER_MESSAGE_COUNT
def _branch_markers(row: dict | None) -> tuple[str | None, str | None]:
"""Return ``(_branched_from, _delegate_from)`` from a row's ``model_config``.
Hermes Agent stamps explicit branches and delegate/subagent runs in the
``model_config`` JSON column (``_branched_from`` / ``_delegate_from``);
``session_source='fork'`` alone only covers WebUI-created forks. Missing,
empty, or unparsable ``model_config`` degrades to ``(None, None)``.
"""
if not row:
return None, None
raw = row.get('model_config')
if isinstance(raw, dict):
config = raw
elif isinstance(raw, str) and raw.strip():
try:
parsed = json.loads(raw)
except (TypeError, ValueError):
parsed = None
config = parsed if isinstance(parsed, dict) else {}
else:
config = {}
return config.get('_branched_from'), config.get('_delegate_from')
def _is_continuation_session(parent: dict | None, child: dict | None) -> bool:
"""Return True when ``child`` is the next segment of the same conversation.
@@ -325,17 +397,36 @@ def _is_continuation_session(parent: dict | None, child: dict | None) -> bool:
by ``hermes -c`` also records a new child session; for sidebar projection it
should continue the same visible conversation rather than becoming a
separate child-session row. Plain parent/child links that started before the
parent's ended boundary remain child sessions.
parent's ended boundary remain child sessions. A small scheduling/write-order
overlap between the handoff boundary timestamps is tolerated (#6931).
Do not collapse lineage across raw sources. A WebUI session that continues
from a Telegram/CLI/etc. parent must remain visible as its own surface-owned
conversation; otherwise the tip inherits the root's title/source metadata and
can disappear under messaging/sidebar policies.
Explicit branches/delegates never continue: a child whose ``model_config``
``_branched_from`` / ``_delegate_from`` points at this parent is a fork of
the conversation and must stay visible. Beyond the bounded tolerance window
there is no reliable continuation signal — titles are user-controlled and
non-unique, so they are never used to widen the window (#7021 re-gate).
"""
if not parent or not child:
return False
if str(child.get('session_source') or '').strip().lower() == 'fork':
return False
# Real Agent branches/delegates are marked in model_config (not
# session_source, which only WebUI-created forks carry). The marker must
# reference THIS parent: compression continuations inherit the rotated
# agent's model_config verbatim, so presence alone (a delegate's
# continuation still carries the delegate's own ``_delegate_from``) would
# misclassify real continuations.
child_branched_from, child_delegate_from = _branch_markers(child)
parent_id = parent.get('id')
if parent_id and (
child_branched_from == parent_id or child_delegate_from == parent_id
):
return False
parent_source = str(parent.get('source') or '').strip().lower()
child_source = str(child.get('source') or '').strip().lower()
if parent_source and child_source and parent_source != child_source:
@@ -349,9 +440,16 @@ def _is_continuation_session(parent: dict | None, child: dict | None) -> bool:
# continuations when no boundary timestamp is available.
return True
try:
return float(child.get('started_at') or 0) >= float(ended_at)
child_started = float(child.get('started_at') or 0)
parent_ended = float(ended_at)
except (TypeError, ValueError):
return False
if child_started >= parent_ended - CONTINUATION_STARTED_AT_TOLERANCE_SECONDS:
return True
# Beyond the bounded early-side window the child is a genuine concurrent
# session — an exact title match is not evidence (titles are user-controlled
# and non-unique, so it cannot extend the tolerance).
return False
def _continuation_root_id(rows_by_id: dict[str, dict], session_id: str | None) -> str | None:
@@ -549,21 +647,7 @@ def read_importable_agent_session_rows(
return []
log = log or logger
# Open read-only for this projection/listing path: it is a pure read, and
# holding a write-capable handle on the live (multi-GB, WAL) state.db while
# the agent streams into it adds needless checkpoint/lock surface (#5455).
# The defensive index self-heal below still runs, but through a separate
# short-lived writable connection on the rare missing-index path only.
read_only_uri = f"{db_path.resolve().as_uri()}?mode=ro"
try:
conn = sqlite3.connect(read_only_uri, uri=True)
except sqlite3.Error as exc:
log.warning(
"agent session listing read-only open failed for %s; falling back to writable connection: %s",
db_path,
exc,
)
conn = sqlite3.connect(str(db_path))
conn = open_state_db_readonly(db_path, log=log)
with closing(conn):
conn.row_factory = sqlite3.Row
cur = conn.cursor()
@@ -588,6 +672,7 @@ def read_importable_agent_session_rows(
parent_expr = _optional_col('parent_session_id', session_cols)
session_source_expr = _optional_col('session_source', session_cols)
model_config_expr = _optional_col('model_config', session_cols)
ended_expr = _optional_col('ended_at', session_cols)
end_reason_expr = _optional_col('end_reason', session_cols)
user_id_expr = _optional_col('user_id', session_cols)
@@ -613,11 +698,7 @@ def read_importable_agent_session_rows(
use_messages_join = messages_has_session_id
count_col = 'id' if 'id' in message_cols else 'session_id'
# Defensive index prime (#3887). The normal candidate-ordering shape uses
# the agent's standard ``idx_messages_session ON messages(session_id,
# timestamp)`` index; without it, large cron-only scans degrade badly.
# Writable dbs self-heal by recreating the index. Read-only or locked dbs
# fall back to the pre-aggregated cron-only path below instead of failing.
# Index creation belongs to explicit drained maintenance, never a read.
messages_index_present = False
if messages_has_session_id and messages_has_timestamp:
try:
@@ -625,21 +706,7 @@ def read_importable_agent_session_rows(
messages_index_present = any(str(row[1]) == "idx_messages_session" for row in cur.fetchall())
except sqlite3.Error:
messages_index_present = False
if not messages_index_present:
# Self-heal via a separate writable connection so the common
# (index-present) path keeps its read-only handle. On a truly
# read-only/locked db this fails and we degrade to the
# pre-aggregated cron-only path below, exactly as before.
try:
with closing(sqlite3.connect(str(db_path))) as _heal:
_heal.execute(
"CREATE INDEX IF NOT EXISTS idx_messages_session "
"ON messages(session_id, timestamp)"
)
_heal.commit()
messages_index_present = True
except sqlite3.Error:
pass # read-only db / locked / older schema — degrade gracefully
if use_messages_join:
actual_count_expr = f"COUNT(m.{count_col})"
@@ -665,7 +732,6 @@ def read_importable_agent_session_rows(
where_clauses = ["s.source IS NOT NULL"]
params: list[object] = []
included = ()
if include_sources:
included = tuple(str(source) for source in include_sources if source)
if included:
@@ -679,10 +745,16 @@ def read_importable_agent_session_rows(
where_clauses.append(f"s.source NOT IN ({placeholders})")
params.extend(excluded)
# Without ``idx_messages_session`` the correlated ``MAX(mx.timestamp)``
# candidate ordering rescans ``messages`` once per session (seconds on
# a few thousand sessions). Every missing-index projection — default
# sidebar, gateway watcher, cron/webhook/kanban views — orders through
# one read-only pre-aggregation pass instead; the listing never creates
# the index itself (that is drained maintenance, see
# ``scripts/ensure_state_db_read_indexes.py``).
use_preaggregated_candidate_order = (
use_messages_join
and messages_has_timestamp
and included == ("cron",)
and not messages_index_present
)
if use_preaggregated_candidate_order:
@@ -709,6 +781,7 @@ def read_importable_agent_session_rows(
SELECT s.id, s.title, s.model, s.message_count,
s.started_at, s.source,
{session_source_expr},
{model_config_expr},
{user_id_expr},
{chat_id_expr},
{chat_type_expr},
@@ -791,6 +864,11 @@ def read_importable_agent_session_rows(
params,
)
projected = _project_agent_session_rows([dict(row) for row in cur.fetchall()])
# model_config is selected only so the continuation classifier can
# read the real _branched_from/_delegate_from markers; it is internal
# agent state and must not leak into the /api/sessions response.
for row in projected:
row.pop('model_config', None)
projected = [_with_normalized_source(row) for row in projected]
projected = [row for row in projected if is_cli_session_row_visible(row)]
if limit is None:
@@ -893,6 +971,7 @@ def read_session_lineage_report(db_path: Path, session_id: str | None, max_hops:
source_expr = _optional_col('source', session_cols)
session_source_expr = _optional_col('session_source', session_cols)
model_config_expr = _optional_col('model_config', session_cols)
title_expr = _optional_col('title', session_cols)
started_expr = _optional_col('started_at', session_cols, '0')
ended_expr = _optional_col('ended_at', session_cols)
@@ -907,6 +986,7 @@ def read_session_lineage_report(db_path: Path, session_id: str | None, max_hops:
SELECT s.id,
{source_expr},
{session_source_expr},
{model_config_expr},
{title_expr},
{started_expr},
{parent_expr},
@@ -953,6 +1033,7 @@ def read_session_lineage_report(db_path: Path, session_id: str | None, max_hops:
SELECT s.id,
{source_expr},
{session_source_expr},
{model_config_expr},
{title_expr},
{started_expr},
{parent_expr},
@@ -1030,6 +1111,7 @@ def read_session_lineage_metadata(db_path: Path, session_ids: list[str] | set[st
return {}
session_source_expr = _optional_col('session_source', session_cols)
source_expr = _optional_col('source', session_cols)
model_config_expr = _optional_col('model_config', session_cols)
message_count_expr = _optional_col('message_count', session_cols, '0')
# Scoped fetch via PRIMARY KEY + idx_sessions_parent rather than a
# full table scan. The sessions table grows unbounded over time
@@ -1066,7 +1148,7 @@ def read_session_lineage_metadata(db_path: Path, session_ids: list[str] | set[st
placeholders = ','.join('?' * len(chunk))
cur.execute(
f"""
SELECT s.id, {source_expr}, {session_source_expr}, s.title, s.started_at, s.parent_session_id, s.ended_at, s.end_reason, {message_count_expr}
SELECT s.id, {source_expr}, {session_source_expr}, {model_config_expr}, s.title, s.started_at, s.parent_session_id, s.ended_at, s.end_reason, {message_count_expr}
FROM sessions s
WHERE s.id IN ({placeholders})
""",
@@ -1095,7 +1177,7 @@ def read_session_lineage_metadata(db_path: Path, session_ids: list[str] | set[st
placeholders = ','.join('?' * len(chunk))
cur.execute(
f"""
SELECT s.id, {source_expr}, {session_source_expr}, s.title, s.started_at, s.parent_session_id, s.ended_at, s.end_reason, {message_count_expr}
SELECT s.id, {source_expr}, {session_source_expr}, {model_config_expr}, s.title, s.started_at, s.parent_session_id, s.ended_at, s.end_reason, {message_count_expr}
FROM sessions s
WHERE s.parent_session_id IN ({placeholders})
""",
api/auth.py
+37
View File
@@ -17,6 +17,7 @@ import time
from pathlib import Path
from api.config import STATE_DIR, get_config, load_settings
from api.helpers import request_declares_body
logger = logging.getLogger(__name__)
@@ -1207,6 +1208,42 @@ def check_auth(handler, parsed) -> bool:
return False
def check_auth_or_close(handler, parsed) -> bool:
"""Check auth; when rejected, close so an unread body can't poison HTTP/1.1 reuse.
The flag is armed BEFORE check_auth() writes its 401/302, so end_headers()
can advertise ``Connection: close``; success restores the prior flag so an
authenticated request keeps its keep-alive.
Armed ONLY when the request's framing declares a body still queued in
``rfile`` (see ``request_declares_body()``), which is the same rule the other
reject-before-read sites apply through ``arm_connection_close_if_body_pending()``.
Arming unconditionally was the mirror image of the over-close this PR already
fixed on the sidecar path: a body-less POST that failed auth answered
``401`` WITH ``Connection: close`` and dropped the client's pipelined
follow-up, killing a healthy keep-alive connection for no framing reason.
Verified on the wire, pipelined down one socket against the production
handler: ``POST /api/session/new`` with no ``Content-Length`` (and with
``Content-Length: 0``) answered ``401`` + ``Connection: close`` and the
following ``GET /api/auth/status`` was never served. The helper cannot be
swapped in directly here because the arming has to happen before
``check_auth()`` writes its response and be undone if it succeeds.
``close_connection`` only exists once BaseHTTPRequestHandler has parsed a
request line, so partially-built handler stubs may not have it at all. Read
it defensively and only arm/restore when it was really there: inventing the
attribute on a stub would leave a bogus keep-alive verdict behind.
"""
if not hasattr(handler, 'close_connection') or not request_declares_body(handler):
return check_auth(handler, parsed)
prior_close = handler.close_connection
handler.close_connection = True
if check_auth(handler, parsed):
handler.close_connection = prior_close
return True
return False
def _is_loopback(addr: str) -> bool:
"""Return True if *addr* is a loopback address (127.x.x.x, ::1, or ::ffff:127.x.x.x)."""
import ipaddress as _ipaddress
api/background_process.py
+51 -1
View File
@@ -1020,6 +1020,29 @@ def _record_async_delegation_accepted(
)
def _wakeup_refusal_is_transient(resp: dict | None, status: int) -> bool:
"""True when the target refused the wake-up for a reason it will recover from.
Such refusals are not delivery failures of the completion itself, so they
must not consume its bounded durable delivery budget. Otherwise a burst of
refusals (e.g. ``agent_runtime_stale`` answers while the Agent runtime is
being replaced) terminally drops a completion whose origin session is alive:
* any payload that explicitly says ``retryable: True`` (stale runtime);
* the paused-wakeup contract (``process_wakeup_paused``);
* a session busy with another turn (``active_stream_id`` on a 409).
"""
if not isinstance(resp, dict):
return False
if resp.get("retryable") is True:
return True
if status != 409:
return False
if resp.get("error") == "process_wakeup_paused":
return True
return bool(resp.get("active_stream_id"))
def _start_async_delegation_wakeup_turn(
session_id: str,
wakeup_prompt: str,
@@ -1059,6 +1082,10 @@ def _start_async_delegation_wakeup_turn(
)
return
transient = _wakeup_refusal_is_transient(resp, status)
if transient:
release_async_delegation_delivery(evt, claim, retryable=True)
else:
release_async_delegation_delivery(evt, claim)
_retry_unclaimed_async_delegation_event(
process_registry, evt, keep_legacy_retrying=True
@@ -1068,6 +1095,14 @@ def _start_async_delegation_wakeup_turn(
"async delegation wakeup paused for session %s; delivery remains retryable",
session_id,
)
elif transient:
logger.info(
"async delegation wakeup refused transiently for session %s "
"(status=%s err=%r); attempt refunded, durable retry scheduled",
session_id,
status,
(resp or {}).get("error"),
)
else:
logger.debug(
"async delegation wakeup not accepted for session %s: "
@@ -1848,7 +1883,22 @@ def recover_processes_for_webui(process_registry=None, get_session_fn=None) -> i
try:
proc_session = process_registry.get(process_id)
session_key = str(getattr(proc_session, "session_key", "") or "")
if not session_key or get_session_fn(session_key, metadata_only=True) is None:
if not session_key:
continue
# The session resolver (``api.models._resolve_session_once``)
# raises ``KeyError(sid)`` for a missing session — that is its
# documented contract for the "owner gone" outcome, not a
# fault. Treat it identically to a ``None`` return: skip the
# process without logging, and let a neighbouring live
# process still rebind. The narrow ``except KeyError`` lives
# INSIDE the outer try so an unrelated ``KeyError`` from
# ``process_registry.get()`` (or any routing-index work) still
# surfaces in the existing warning path.
try:
resolved = get_session_fn(session_key, metadata_only=True)
except KeyError:
continue
if resolved is None:
continue
except Exception:
logger.warning(
api/compression_continuation.py
+80
View File
@@ -0,0 +1,80 @@
"""Read-only routing hints for externally compressed WebUI sessions.
SQLite owns compression lineage; sidecar snapshot flags may predate a
Desktop/CLI rotation. Never reopen a sealed parent or mutate Agent state here.
"""
import inspect
import logging
from pathlib import Path
logger = logging.getLogger(__name__)
def _is_local_interactive_persisted_source(source):
return isinstance(source, str) and source in {
"webui", "tui", "cli", "desktop", "acp",
}
def durable_compression_continuation(session):
"""Return (sealed, resumable tip), without making a recovery write.
A known sealed parent without a safe tip stays sealed (no sidecar fallback).
Older Agent installations without this read API retain legacy behavior.
"""
from api.profiles import _PROFILE_ID_RE, _resolve_profile_home_for_name
sid = str(getattr(session, "session_id", "") or "")
profile = str(getattr(session, "profile", None) or "default")
if not sid or (profile != "default" and not _PROFILE_ID_RE.fullmatch(profile)):
return False, None
db = None
sealed = False
try:
from hermes_state import SessionDB
path = Path(_resolve_profile_home_for_name(profile)) / "state.db"
if not path.is_file():
return False, None
db = SessionDB(path, read_only=True)
# Establish the complete read API before accepting SQLite authority.
# Old Agents must retain legacy sidecar recovery, not a sealed null tip.
for name in ('get_session', 'get_compression_tip'):
method = getattr(db, name, None)
if not callable(method):
return False, None
try:
inspect.signature(method).bind(sid)
except (TypeError, ValueError):
return False, None
parent = db.get_session(sid)
if not parent or parent.get("end_reason") != "compression":
return False, None
sealed = True
tip = db.get_compression_tip(sid)
if not tip or tip == sid:
return True, None
child = db.get_session(tip)
if not child or not _is_local_interactive_persisted_source(child.get("source")):
return True, None
for row in (parent, child):
if row.get("profile_name") not in (None, "", profile):
return True, None
# Only the observed automatic idle closure is resumable here. Explicit
# resets/closures and unknown future reasons must not become redirects.
reason = child.get("end_reason")
if reason not in (None, "", "idle_timeout"):
return True, None
if child.get("ended_at") is not None and reason != "idle_timeout":
return True, None
if db.get_compression_tip(sid) != tip:
return True, None
return True, tip
except (ImportError, AttributeError, TypeError):
return sealed, None
except Exception:
logger.debug("Could not resolve durable compression continuation", exc_info=True)
return sealed, None
finally:
if db is not None:
db.close()
api/config.py
+98 -21
View File
@@ -68,6 +68,21 @@ def _env_int(name: str, default: int, *, minimum: int = 1) -> int:
return default
return value if value >= minimum else default
def _env_int_clamped(name: str, default: int, *, minimum: int = 1, maximum: int) -> int:
"""Like ``_env_int``, then clamp a valid override to ``maximum``."""
value = _env_int(name, default, minimum=minimum)
if not str(os.getenv(name) or "").strip():
return value
return min(value, maximum)
# Sidebar recency window. Resolved here, before profile init, so a profile
# .env cannot override a server-wide resource bound. Clamped at 200.
CLI_VISIBLE_SESSION_LIMIT = _env_int_clamped(
"HERMES_WEBUI_VISIBLE_SESSION_LIMIT", 20, maximum=200,
)
# ── TLS/HTTPS config (optional, env-overridable) ────────────────────────────
TLS_CERT = os.getenv("HERMES_WEBUI_TLS_CERT", "").strip() or None
TLS_KEY = os.getenv("HERMES_WEBUI_TLS_KEY", "").strip() or None
@@ -1914,28 +1929,44 @@ _PROVIDER_MODELS = {
{"id": "big-pickle", "label": "Big Pickle"},
],
# OpenCode Go — flat-rate models via opencode.ai/go ($10/month).
# Synced 2026-07-08 from the public Go docs and documented models endpoint.
# Keep preview/free-only Zen models out of this Go picker snapshot.
# Fallback only: the live Hermes CLI catalog (Go-specific
# /zen/go/v1/models probe, core v0.20.5+) leads (#1240, #5311).
# Mirrors Hermes core's curated opencode-go list
# (hermes_cli/models_catalog_static.py, core main as of 2026-09-10).
# Core's 2026-09-09 sync dropped `ox-alpha-free` (Go relay delisted it:
# GET /zen/go/v1/models omits it, POST → 401) and added `glm-5.3-flash`
# and `muse-spark-1.3-contributor`. Core owns the sync duty against the
# live endpoint; WebUI mirrors.
"opencode-go": [
{"id": "minimax-m3", "label": "MiniMax M3"},
{"id": "minimax-m2.7", "label": "MiniMax M2.7"},
{"id": "minimax-m2.5", "label": "MiniMax M2.5"},
{"id": "kimi-k3", "label": "Kimi K3"},
{"id": "kimi-k2.7-code", "label": "Kimi K2.7 Code"},
{"id": "kimi-k2.6", "label": "Kimi K2.6"},
{"id": "kimi-k2.5", "label": "Kimi K2.5"},
{"id": "gpt-5.6-luna", "label": "GPT 5.6 Luna"},
{"id": "grok-4.5", "label": "Grok 4.5"},
{"id": "glm-5.3", "label": "GLM-5.3"},
{"id": "glm-5.3-flash", "label": "GLM-5.3 Flash"},
{"id": "glm-5.2", "label": "GLM-5.2"},
{"id": "glm-5.1", "label": "GLM-5.1"},
{"id": "glm-5", "label": "GLM-5"},
{"id": "mimo-v2.5-pro", "label": "MiMo V2.5 Pro"},
{"id": "mimo-v2.5", "label": "MiMo V2.5"},
{"id": "mimo-v2-pro", "label": "MiMo V2 Pro"},
{"id": "mimo-v2-omni", "label": "MiMo V2 Omni"},
{"id": "minimax-m3", "label": "MiniMax M3"},
{"id": "minimax-m2.7", "label": "MiniMax M2.7"},
{"id": "minimax-m2.5", "label": "MiniMax M2.5"},
{"id": "deepseek-v4-pro", "label": "DeepSeek V4 Pro"},
{"id": "deepseek-v4-flash","label": "DeepSeek V4 Flash"},
{"id": "deepseek-v4-flash", "label": "DeepSeek V4 Flash"},
{"id": "qwen3.8-max", "label": "Qwen3.8 Max"},
{"id": "qwen3.7-max", "label": "Qwen3.7 Max"},
{"id": "qwen3.7-plus", "label": "Qwen3.7 Plus"},
{"id": "qwen3.6-plus", "label": "Qwen3.6 Plus"},
{"id": "qwen3.5-plus", "label": "Qwen3.5 Plus"},
{"id": "mimo-v2-pro", "label": "MiMo V2 Pro"},
{"id": "mimo-v2-omni", "label": "MiMo V2 Omni"},
{"id": "mimo-v2.5-pro", "label": "MiMo V2.5 Pro"},
{"id": "mimo-v2.5", "label": "MiMo V2.5"},
{"id": "hy3", "label": "HY3"},
{"id": "hy3-preview", "label": "HY3 Preview"},
{"id": "muse-spark-1.2-contributor", "label": "Muse Spark 1.2 Contributor"},
{"id": "muse-spark-1.3-contributor", "label": "Muse Spark 1.3 Contributor"},
],
# 'gemini' is the hermes_cli provider ID for Google AI Studio
# Model IDs are bare — sent directly to:
@@ -6937,8 +6968,8 @@ def _configured_model_badges_from_static_catalog(
for entry in configured_entries:
provider = entry["provider"]
model = entry["model"]
raw_candidates = []
for candidate in (model, f"{provider}/{model}", f"@{provider}:{model}"):
raw_candidates: list[str] = []
for candidate in (model, f"@{provider}:{model}"):
if candidate and candidate not in raw_candidates:
raw_candidates.append(candidate)
@@ -8475,6 +8506,26 @@ def _read_live_provider_model_ids(provider_id: str) -> list[str]:
return []
def _hermes_cli_supports_opencode_go_live_catalog() -> bool:
"""Whether the installed Agent has Go-specific model discovery.
Hermes core versions before 0.20.5 route ``opencode-go`` through a
generic public catalog. That lookup can return a convincing non-empty
list containing models the Go relay rejects with 404, so absence or an
unparseable/prerelease version must fail closed to WebUI's static Go list.
"""
try:
import hermes_cli
version = str(getattr(hermes_cli, "__version__", "")).strip()
except Exception:
return False
match = re.fullmatch(r"v?(\d+)\.(\d+)\.(\d+)", version)
if not match:
return False
return tuple(int(part) for part in match.groups()) >= (0, 20, 5)
def _models_from_live_provider_ids(provider_id: str, live_ids: list[str]) -> list[dict]:
"""Convert Hermes CLI model ids into WebUI picker model entries."""
formatter = _format_ollama_label if provider_id in ("ollama", "ollama-cloud") else None
@@ -8683,10 +8734,9 @@ def get_available_models(*, prefer_cache: bool = False, force_refresh: bool = Fa
for entry in configured_entries:
provider = entry["provider"]
model = entry["model"]
raw_candidates = []
raw_candidates: list[str] = []
for candidate in (
model,
f"{provider}/{model}",
f"@{provider}:{model}",
):
if candidate and candidate not in raw_candidates:
@@ -9872,12 +9922,14 @@ def get_available_models(*, prefer_cache: bool = False, force_refresh: bool = Fa
if not raw_models:
if pid == "moa":
raw_models = _moa_preset_models_from_config(cfg)
elif pid == "opencode-go":
# Skip live /v1/models probe for OpenCode Go — it
# returns models from the public catalog that are
# not enabled on the Go tier, causing 404 when
# selected. Use the curated static list only. (#5311)
pass
elif (
pid == "opencode-go"
and not _hermes_cli_supports_opencode_go_live_catalog()
):
# Before core v0.20.5 this resolver returned the
# generic public catalog, including models that
# 404 on the Go tier. Use the curated Go fallback.
raw_models = []
else:
raw_models = _models_from_live_provider_ids(
pid,
@@ -11532,7 +11584,13 @@ _SETTINGS_DEFAULTS = {
"hidden_tabs": [], # sidebar tab panel names hidden by user (e.g. ["tasks","kanban"]); chat and settings are always visible
"tab_order": [], # user-defined sidebar/rail tab order for reorderable tabs; chat/settings stay fixed
"composer_control_order": [], # user-defined composer footer control order; invalid/duplicate keys are ignored
"language": "en", # UI locale code; must match a key in static/i18n.js LOCALES
# #7622 (round 3): language is intentionally absent from the defaults so
# `load_settings()` reports `None` for a fresh install. This lets the
# client distinguish "no preference" from an explicit saved choice,
# so a user who genuinely picked English (and has `language: "en"`
# persisted on disk) is no longer overridden by the browser hint on
# their first hydration. When a user picks a locale in the Settings
# modal the value is written here and the field is set explicitly.
"bot_name": os.getenv(
"HERMES_WEBUI_BOT_NAME", "Hermes"
), # display name for the assistant
@@ -11772,6 +11830,16 @@ def load_settings() -> dict:
settings["default_model_provider"] = str(model_cfg.get("provider"))
except Exception:
logger.debug("Failed to resolve default model provider for settings")
# #7622/#7730 (round 4): keep the tri-state signal explicit. `language`
# is intentionally absent from `_SETTINGS_DEFAULTS` (see above), so
# without this line a fresh install's returned dict would OMIT the key
# entirely and the API payload would carry no `language` field. Emit an
# explicit `None` (serialized as JSON `null`) so the client receives the
# three-way signal it trusts: `null` = no preference, "en" = explicitly
# saved English, any other code = explicitly chosen locale. Stored
# values (including a legacy English pick written before this change)
# win via the merge above and are never touched here.
settings.setdefault("language", None)
return settings
@@ -11779,6 +11847,15 @@ _SETTINGS_ALLOWED_KEYS = set(_SETTINGS_DEFAULTS.keys()) - {
"password_hash",
"default_model",
"simplified_tool_calling",
} | {
# #7622 (round 3): `language` is intentionally absent from
# `_SETTINGS_DEFAULTS` so a fresh install returns `None` for
# `settings["language"]` and the client can distinguish "no
# preference" from an explicit saved choice. But the user
# MUST still be able to pick a locale in the Settings modal,
# so we add it back to the explicit allow-list here. The
# existing BCP-47 validation at save-time still applies.
"language",
}
_SETTINGS_ENUM_VALUES = {
"send_key": {"enter", "ctrl+enter", "shift+enter"},
api/gateway_chat.py
+30 -11
View File
@@ -153,6 +153,9 @@ _WEBUI_GATEWAY_BASE_URL_ENV = "HERMES_WEBUI_GATEWAY_BASE_URL"
_WEBUI_GATEWAY_API_KEY_ENV = "HERMES_WEBUI_GATEWAY_API_KEY"
_WEBUI_GATEWAY_USE_RUNS_API_ENV = "HERMES_WEBUI_GATEWAY_USE_RUNS_API"
_GATEWAY_CHAT_BACKENDS = {"gateway", "api_server", "api-server"}
# Backend tag of the in-process WebUI runtime. Local workers register their
# active run with it; cache-only Steer only enqueues on this explicit value.
WEBUI_LOCAL_CHAT_BACKEND = "legacy"
def _gateway_model_field(model: str | None) -> str:
@@ -272,7 +275,7 @@ def webui_chat_backend_mode(config_data=None, environ: dict[str, str] | None = N
).strip().lower()
if raw in _GATEWAY_CHAT_BACKENDS:
return "gateway"
return "legacy"
return WEBUI_LOCAL_CHAT_BACKEND
def webui_gateway_chat_enabled(config_data=None, environ: dict[str, str] | None = None) -> bool:
@@ -738,8 +741,12 @@ def _run_gateway_runs_api_streaming(
sse_event = "message"
continue
if payload_event == "run.completed":
from api.route_approvals import retire_gateway_pending_mirror
retire_gateway_pending_mirror(session_id, run_id=run_id)
from api.route_approvals import settle_gateway_pending_run
settle_gateway_pending_run(
session_id,
run_id,
reason="Gateway run completed before approval resolution",
)
if payload.get("error"):
raise RuntimeError(str(payload["error"]))
output = str(payload.get("output") or "")
@@ -751,12 +758,20 @@ def _run_gateway_runs_api_streaming(
sse_event = "message"
continue
if payload_event == "run.failed":
from api.route_approvals import retire_gateway_pending_mirror
retire_gateway_pending_mirror(session_id, run_id=run_id)
from api.route_approvals import settle_gateway_pending_run
settle_gateway_pending_run(
session_id,
run_id,
reason="Gateway run failed before approval resolution",
)
raise RuntimeError(str(payload.get("error") or "Gateway run failed"))
if payload_event == "run.cancelled":
from api.route_approvals import retire_gateway_pending_mirror
retire_gateway_pending_mirror(session_id, run_id=run_id)
from api.route_approvals import settle_gateway_pending_run
settle_gateway_pending_run(
session_id,
run_id,
reason="Gateway run was cancelled before approval resolution",
)
put_gateway_event("cancel", {"message": "Cancelled by gateway"})
return None, usage
reasoning_delta = _gateway_sse_reasoning_delta(payload)
@@ -978,7 +993,7 @@ def _run_gateway_chat_streaming(
except Exception:
logger.debug("Failed to note gateway event_id %s for stream %s", event_id, stream_id, exc_info=True)
try:
queue_item = (event, data, event_id) if event_id and hasattr(q, "subscribe_with_snapshot") else (event, data)
queue_item = (event, data, event_id) if hasattr(q, "subscribe_with_snapshot") else (event, data)
q.put_nowait(queue_item)
except Exception:
logger.debug("Failed to put gateway event to queue")
@@ -1464,10 +1479,14 @@ def _run_gateway_chat_streaming(
mapped_run_id = str(_STREAM_RUN_IDS.get(stream_id) or "").strip()
if mapped_run_id:
try:
from api.route_approvals import retire_gateway_pending_mirror
retire_gateway_pending_mirror(session_id, run_id=mapped_run_id)
from api.route_approvals import settle_gateway_pending_run
settle_gateway_pending_run(
session_id,
mapped_run_id,
reason="Gateway run ended during teardown before approval resolution",
)
except Exception:
logger.debug("Failed to retire gateway pending mirrors during teardown", exc_info=True)
logger.debug("Failed to settle gateway pending approvals during teardown", exc_info=True)
if s is not None:
try:
with _get_session_agent_lock(session_id):
api/gateway_watcher.py
+63 -17
View File
@@ -203,7 +203,11 @@ class GatewayWatcher:
):
self._subscribers: list[queue.Queue] = []
self._sub_lock = threading.Lock()
# Final removal invalidates any projection admitted by an earlier cohort.
self._subscriber_epoch = 0
self._stop_event = threading.Event()
# Wakes a poll loop parked because nobody is subscribed (subscribe/stop).
self._idle_wake = threading.Event()
self._thread: threading.Thread | None = None
self._hermes_home = Path(hermes_home).expanduser().resolve() if hermes_home else None
self._state_db_path = (
@@ -244,6 +248,7 @@ class GatewayWatcher:
def stop(self):
"""Stop the watcher thread."""
self._stop_event.set()
self._idle_wake.set() # unpark if waiting for the first subscriber
# Wake up any subscribers
with self._sub_lock:
for q in self._subscribers:
@@ -255,6 +260,11 @@ class GatewayWatcher:
self._thread.join(timeout=3)
self._thread = None
def _has_subscribers(self) -> bool:
"""Return True when at least one SSE client is attached."""
with self._sub_lock:
return bool(self._subscribers)
def subscribe(self) -> queue.Queue:
"""Subscribe to change events. Returns a queue.Queue.
Events are dicts: {'type': 'sessions_changed', 'sessions': [...]}
@@ -273,23 +283,37 @@ class GatewayWatcher:
q.put_nowait(None)
except Exception:
logger.debug("Failed to send stop sentinel to late subscriber")
# Wake a poll loop parked with zero subscribers so the first SSE client
# gets a prompt initial projection instead of waiting out POLL_INTERVAL.
self._idle_wake.set()
return q
def unsubscribe(self, q: queue.Queue):
"""Remove a subscriber queue."""
with self._sub_lock:
def _remove_subscriber_locked(self, q: queue.Queue) -> bool:
"""Remove a known queue under _sub_lock; fence polls on the last removal."""
try:
self._subscribers.remove(q)
except ValueError:
pass
return False
if not self._subscribers:
self._subscriber_epoch += 1
self._last_cheap_fp = ''
self._last_full_projection_at = None
return True
def _notify_subscribers(self, sessions: list):
def unsubscribe(self, q: queue.Queue):
"""Remove a subscriber queue, invalidating projection on the last removal."""
with self._sub_lock:
self._remove_subscriber_locked(q)
def _notify_subscribers(self, sessions: list, *, epoch: int | None = None):
"""Push change event to all subscribers."""
event = {
'type': 'sessions_changed',
'sessions': sessions,
}
with self._sub_lock:
if epoch is not None and epoch != self._subscriber_epoch:
return # An old poll must not notify a new subscriber cohort.
dead = []
for q in self._subscribers:
try:
@@ -299,10 +323,7 @@ class GatewayWatcher:
except Exception:
dead.append(q)
for q in dead:
try:
self._subscribers.remove(q)
except ValueError:
pass
self._remove_subscriber_locked(q)
# Send a None sentinel so the SSE handler unblocks, closes,
# and lets the browser's EventSource auto-reconnect.
try:
@@ -317,6 +338,12 @@ class GatewayWatcher:
protects projection fields (notably role-derived CLI visibility) that
the agent's existing index cannot see.
"""
with self._sub_lock:
has_subscribers = bool(self._subscribers)
admission_epoch = self._subscriber_epoch
if not has_subscribers:
# Final removal already invalidated the cache under the same lock.
return False
db_path = self._state_db_path
# A watcher may start before the agent has created state.db. Publishing an
# empty first snapshot would make an already-rendered sidebar disappear;
@@ -344,29 +371,48 @@ class GatewayWatcher:
if sessions is None:
return False
current_hash = _snapshot_hash(sessions)
with self._sub_lock:
if admission_epoch != self._subscriber_epoch or not self._subscribers:
return False # Never restore a cache invalidated during the DB read.
if cheap_fp is not None:
self._last_cheap_fp = cheap_fp
self._last_full_projection_at = current_time
if current_hash != self._last_hash:
changed = True
self._last_hash = current_hash
self._last_sessions = sessions
self._notify_subscribers(sessions)
else:
changed = False
if changed:
self._notify_subscribers(sessions, epoch=admission_epoch)
return True
def _poll_loop(self):
"""Main polling loop. Runs in a daemon thread."""
"""Main polling loop. Runs in a daemon thread.
With no SSE subscribers there is nobody to notify, so the loop parks on
``_idle_wake`` instead of re-fingerprinting ``state.db`` every few
seconds (maint #3035). While subscribed it blocks once on
``_stop_event.wait(POLL_INTERVAL)`` — a single timer sleep that returns
immediately when ``stop()`` sets the event, replacing the previous
10 wakeups/sec ``time.sleep(0.1)`` spin.
"""
while not self._stop_event.is_set():
if not self._has_subscribers():
self._idle_wake.clear()
# subscribe() may have raced between the check and clear; if so
# the flag is already set again and we must not park.
if self._has_subscribers() or self._stop_event.is_set():
continue
self._idle_wake.wait()
continue
try:
self._poll_once()
except Exception:
logger.debug("Error in gateway watcher poll loop", exc_info=True)
# Sleep in small increments so we can stop promptly
for _ in range(self.POLL_INTERVAL * 10):
if self._stop_event.is_set():
return
time.sleep(0.1)
self._stop_event.wait(self.POLL_INTERVAL)
# ── Module-level watcher registry ──────────────────────────────────────────
api/helpers.py
+242
View File
@@ -202,6 +202,248 @@ def _security_headers(handler):
)
def arm_connection_close(handler) -> None:
"""Mark the connection dead BEFORE a reject-before-read response is written.
A rejection that answers without consuming the request body leaves those
bytes queued in ``rfile``; reusing the connection makes the next HTTP/1.1
request parse mid-body (the classic ``{}GET ... -> 501``). Arming must
happen before the response is flushed so ``advertise_connection_close()``
can serialize it into the response headers.
Best-effort: handler stubs and read-only doubles may reject the attribute,
and a failure here must never turn a 4xx into a 500.
"""
try:
handler.close_connection = True
except Exception:
pass
# What ``unsupported_transfer_encoding()`` reports for a header that declares no
# coding at all (``Transfer-Encoding:``). A placeholder rather than the raw '' so
# the return value is never a falsy non-``None`` — callers test ``is not None``,
# and an empty string would also render as a hole in the rejection message.
_BLANK_TRANSFER_CODING = '(blank)'
# The only whitespace a field value may be padded with: RFC 9110 OWS is SP/HTAB
# and nothing else. ``str.strip()`` would also erase U+00A0 and U+0085 -- both
# reachable over the wire, because request lines decode as latin-1 -- and
# ``Content-Length: \xa00`` would then read back as a genuine ``0`` while its
# payload sat unread on the socket. Padding a value with a character the grammar
# does not allow makes it unreadable, not zero.
_FIELD_VALUE_OWS = ' \t'
def _framing_header_values(handler, name: str) -> list[str]:
"""Every value the wire carried for one framing header, tolerating stubs.
ALL values, not just the first: ``Message.get()`` returns only the first
occurrence, so a request carrying ``Content-Length: 0`` followed by
``Content-Length: 42`` reads back as length zero. Its 42 body bytes are then
never consumed and get parsed as the next request line -- the same poisoning
this module exists to prevent, reached by duplicating a header rather than by
omitting one. Conflicting framing is only visible if every value is.
A real handler carries an ``email.message.Message`` (case-insensitive
``get_all()``, like the wire); the suite's ``SimpleNamespace(headers={...})``
doubles are plain dicts, which are not, so fall back to a case-insensitive
scan for those. A missing or odd ``headers`` must never raise: every caller
is on a rejection path where an exception becomes a spurious 500.
"""
headers = getattr(handler, 'headers', None)
if headers is None:
return []
try:
get_all = getattr(headers, 'get_all', None)
if callable(get_all):
values = get_all(name)
return [] if values is None else [str(v) for v in values if v is not None]
if isinstance(headers, dict):
lowered = name.lower()
return [
str(v) for k, v in headers.items()
if str(k).lower() == lowered and v is not None
]
value = headers.get(name)
return [] if value is None else [str(value)]
except Exception:
return []
def _declared_content_lengths(handler) -> set[int] | None:
"""Distinct declared ``Content-Length`` values, or ``None`` if any is unreadable.
``None`` means the framing cannot be trusted at all (a value that will not
parse); an empty set means no length was declared.
A BLANK or whitespace-only value is unreadable, NOT absent. ``Content-Length:``
with payload bytes behind it declares a body whose size the header refuses to
state; skipping the empty value reported "no length declared", so the
rejection kept the connection alive and those bytes were parsed as the next
request line (403 then ``400 Bad request syntax ('{...}GET /api/... HTTP/1.1')``).
Absence is something only the wire can say, and ``_framing_header_values()``
already reports a header the request never carried as no value at all -- so
anything that *is* here and does not parse has to be treated as pending bytes,
exactly like ``banana`` or two lengths that disagree. Note ``Message`` collapses
``Content-Length:`` and ``Content-Length: `` to the same ``''``.
"Parses" means RFC 9110's ``Content-Length = 1*DIGIT``: an unsigned run of
ASCII digits, nothing else. ``int()`` is far more generous than the grammar --
it accepts a leading sign, PEP 515 underscores and non-ASCII digits -- and
every such spelling of ZERO used to read back as an honest ``0`` and take the
keep-alive branch below with its payload still queued: ``Content-Length: +0``
(also ``-0``, ``0_0``, ``+00``, ``\\xa00``) answered 403 with no
``Connection: close``, then ``400 Bad request syntax ('{...}GET /api/...')``
-- the same trace as a blank value. A sign is only invisible on a NON-zero
value, where the wrong parse happens to close anyway. Anything the grammar
rejects is unreadable framing, which means a body may be pending.
The ``int()`` guard stays even though ``isdigit()`` has already vetted every
character: a digit run longer than ``sys.get_int_max_str_digits()`` (4300)
raises ``ValueError`` too, and this runs on rejection paths where an
exception becomes a spurious 500.
"""
parsed: set[int] = set()
for raw in _framing_header_values(handler, 'Content-Length'):
# RFC 9110 §5.3: a list-valued field may arrive as repeated lines OR as
# one comma-combined line, and the two spellings mean the same thing.
# So "0, 0" is the same bodyless request as two "Content-Length: 0"
# headers and must keep its keep-alive; only a DISAGREEING list (caught
# below by len(parsed) > 1) is unframeable. Without the split the comma
# fails isdigit() and every agreeing list over-closed a healthy socket.
for member in raw.split(','):
stripped = member.strip(_FIELD_VALUE_OWS)
if not (stripped.isascii() and stripped.isdigit()):
return None
try:
parsed.add(int(stripped))
except ValueError:
return None
return parsed
def unreadable_content_length(handler) -> bool:
"""True when ``Content-Length`` cannot be trusted to frame the body.
Either a value that will not parse (``banana``, ``0x5``, or a BLANK one), or
two or more values that disagree. None of them can tell a reader how many
bytes to drain, so such a request can only be rejected -- with the connection
armed for close, since bytes may still be queued behind it.
"""
parsed = _declared_content_lengths(handler)
return parsed is None or len(parsed) > 1
def request_declares_body(handler) -> bool:
"""True when the request's framing declares a body that is still queued.
Keyed on the wire, not on the method: ``Transfer-Encoding`` present (chunked
framing hides the length until the terminating chunk is read) or
``Content-Length`` present and non-zero. The method proves nothing in either
direction -- a GET may carry a declared body and a POST/PUT/DELETE may carry
none -- and a static per-method flag therefore both misses body-bearing
"read" requests (whose unread bytes poison the next pooled request) and
kills healthy keep-alive on body-less writes.
A blank, unparseable or self-contradicting ``Content-Length`` counts as a
declared body, as does ANY ``Transfer-Encoding`` -- ``identity`` and a blank
value included: a value we cannot interpret does not prove the body's absence,
and the safe assumption for connection reuse is that bytes are pending. Only
framing that positively says "no body" -- no framing header at all, or every
declared length agreeing on zero -- keeps the connection alive.
"""
if unsupported_transfer_encoding(handler) is not None:
return True
parsed = _declared_content_lengths(handler)
if parsed is None:
return True
return any(length != 0 for length in parsed)
def unsupported_transfer_encoding(handler) -> str | None:
"""Return the transfer coding this server cannot decode, or ``None``.
EVERY present value counts, with no exemption for any coding.
``BaseHTTPRequestHandler`` decodes none of them: it hands every reader a raw
``rfile``, so a chunked body reads back as its literal chunk framing while a
``Content-Length``-based reader sees length 0 and consumes nothing at all.
RFC 9112 section 6.3 says the same thing normatively -- a request whose final
transfer coding is not ``chunked`` has a body length the recipient cannot
determine, so it must be refused and the connection closed. Such a request
can therefore only be rejected, and only with the connection armed for close,
because its payload bytes are still queued on the socket.
``identity`` used to be exempt as a coding that "frames nothing". It is not:
it names no framing either, and RFC 7230 removed it from the transfer codings
altogether. A sidecar rejection then answered 403 with keep-alive and the
payload was parsed as the next request line
(``400 Bad request syntax ('{...}GET /api/health/agent HTTP/1.1')``), and all
four upload handlers fell through to a length-0 read, answered 400 "No file
field in request" and left the multipart bytes on the socket
(``400 Bad request syntax ('--x')``). Nothing legitimate is lost: no client
sends a transfer coding this server could have honoured.
A header present but EMPTY (``Transfer-Encoding:``) is reported through
``_BLANK_TRANSFER_CODING`` rather than as ``''``, so callers testing
``is not None`` and the 411 message both read right. Only an ABSENT header
yields no values at all -- absence is something only the wire can say.
The first value is enough precisely because no value is acceptable; it is
reported as-is (SP/HTAB trimmed, the one padding RFC 9110 allows) purely so
the rejection message names what arrived.
"""
values = _framing_header_values(handler, 'Transfer-Encoding')
if not values:
return None
return values[0].strip(_FIELD_VALUE_OWS) or _BLANK_TRANSFER_CODING
def arm_connection_close_if_body_pending(handler) -> bool:
"""Arm close for a reject-before-read, but only if a body was really declared.
Returns whether the connection was armed. Use this instead of
``arm_connection_close()`` wherever the rejection can also fire on a request
with no body: closing then would drop a pooled client's healthy connection
for no framing reason.
"""
if not request_declares_body(handler):
return False
arm_connection_close(handler)
return True
def advertise_connection_close(handler) -> None:
"""Serialize ``close_connection`` into exactly one ``Connection: close`` header.
``BaseHTTPRequestHandler`` tracks ``close_connection`` internally but never
writes it to the wire, so a server-side close is invisible to the client: a
pooled client reuses the socket and fails with BrokenPipeError instead of
opening a fresh connection. Call this from ``end_headers()``, before the
header buffer is flushed.
Dedup is derived from the pending header buffer rather than a private flag,
because the SSE endpoints send their own ``Connection: close`` (and
``send_header`` itself flips ``close_connection`` when they do) — keying off
a flag we set ourselves would emit the header twice on every stream.
Both attribute reads are defensive: partially-built handler stubs in the
test suite carry neither ``close_connection`` nor ``_headers_buffer``, and
header emission must not depend on their presence. The buffer is also
type-checked before it is scanned -- a stub whose ``_headers_buffer`` is a
truthy non-iterable (a bare ``Mock``) would otherwise raise ``TypeError``
from inside ``end_headers()``, which ``_handle_write`` would swallow into a
spurious 500.
"""
if not getattr(handler, 'close_connection', False):
return
buffered = getattr(handler, '_headers_buffer', None)
for raw in buffered if isinstance(buffered, (list, tuple)) else ():
if raw[:11].lower() == b'connection:':
return
handler.send_header('Connection', 'close')
def flush_pending_auth_cookies(handler) -> None:
pending = getattr(handler, '_pending_set_cookies', None)
if not pending:
api/media_snapshots.py
+8
View File
@@ -505,6 +505,13 @@ def annotate_media_snapshots(
if allowed_predicate is None:
allowed_predicate = media_capture_allowed
media_re = _re.compile(r"MEDIA:([^\s\)\]]+)")
# #7680 re-gate (9/22): two-pass scan. First strip backtick
# wrappers (`` `MEDIA:path` `` → ``MEDIA:path``) so the bare
# class below does not consume the closing backtick as part of
# the path. Then the bare class (no backtick in the exclusion
# set) captures the full filename even when the path itself
# contains a backtick (e.g. ``report`final.png``).
backtick_re = _re.compile(r"`MEDIA:([^`\s]+)`")
captured = 0
for msg in messages or []:
if not isinstance(msg, dict) or msg.get("role") != "assistant":
@@ -530,6 +537,7 @@ def annotate_media_snapshots(
if not text_parts:
continue
text = "\n".join(text_parts)
text = backtick_re.sub(lambda m: f"MEDIA:{m.group(1)}", text)
refs = media_re.findall(text)
if not refs:
continue
api/models.py
+90 -11
View File
@@ -48,7 +48,11 @@ from api.agent_sessions import (
from api.process_event_utils import stamp_message_source
logger = logging.getLogger(__name__)
CLI_VISIBLE_SESSION_LIMIT = 20
# Size of the interactive sidebar recency window. Also bounds how many
# delegated subagent children can be rendered at once, since a child only
# nests when it wins a slot in this window. Resolved in api.config before
# profile init; override with HERMES_WEBUI_VISIBLE_SESSION_LIMIT (clamped 1–200).
CLI_VISIBLE_SESSION_LIMIT = _cfg.CLI_VISIBLE_SESSION_LIMIT
# How many messageful cron sessions to surface in the project-chip layer.
# Needs to exceed CLI_VISIBLE_SESSION_LIMIT so older cron runs stay
# addressable even when many newer non-cron sessions dominate the default
@@ -4784,7 +4788,41 @@ class _FullSessionResolveRequired(RuntimeError):
"""Internal signal that a full sidecar load must enter the bounded path."""
_FULL_SESSION_RESOLVE_MAX_CONCURRENT = 2
def _read_max_session_resolve_concurrent() -> int:
"""#7421: env override for the heavy session-resolve cap. The
hardcoded literal 2 is too low for high-concurrency
deployments where several parallel active sessions all need
a full-transcript resolve; operators can set
``HERMES_WEBUI_MAX_SESSION_RESOLVE`` to a positive int to
raise the cap without code changes. A bad or missing value
falls back to the previous default of 2, so an operator who
sets ``HERMES_WEBUI_MAX_SESSION_RESOLVE=0`` or a non-numeric
value does not regress to a blocked or unbounded state.
The cap is a *safety bound* (per-resolve cost tracked in
#7310), not a user preference — a typo like ``=999999``
must fall back to the safe default, never to the permissive
extreme. Every other bad input in this function falls back
to ``2``; out-of-range values do too. The previously
accepted upper bound (64) is reachable only via an explicit
in-range operator value; a profile's ``.env`` cannot reach
it at all (see ``_PROTECTED_ENV_KEYS``).
"""
raw = (os.getenv("HERMES_WEBUI_MAX_SESSION_RESOLVE") or "").strip()
if not raw:
return 2
try:
n = int(raw)
except (TypeError, ValueError):
return 2
if n < 1:
return 2
if n > 64:
return 2 # #7656 round-3: typo-safe fallback, not clamp-up-to-64
return n
_FULL_SESSION_RESOLVE_MAX_CONCURRENT = _read_max_session_resolve_concurrent()
_FULL_SESSION_RESOLVE_SLOTS = threading.BoundedSemaphore(
_FULL_SESSION_RESOLVE_MAX_CONCURRENT
)
@@ -8581,9 +8619,26 @@ def get_state_db_session_messages(
cur.execute("PRAGMA table_info(sessions)")
session_cols = {str(row['name']) for row in cur.fetchall()}
if {'parent_session_id', 'end_reason', 'started_at', 'source'}.issubset(session_cols):
# session_source/model_config carry the fork/delegate boundary
# identity consumed by _is_continuation_session (#6931/#7021).
# Select them when present so the branch-marker guard applies
# to the stitch path too; older schemas degrade to NULL.
identity_cols = []
if 'session_source' in session_cols:
identity_cols.append('session_source')
else:
identity_cols.append('NULL AS session_source')
if 'model_config' in session_cols:
identity_cols.append('model_config')
else:
identity_cols.append('NULL AS model_config')
lineage_select = (
"id, source, started_at, parent_session_id, ended_at, end_reason"
+ ", " + ", ".join(identity_cols)
)
cur.execute(
"""
SELECT id, source, started_at, parent_session_id, ended_at, end_reason
f"""
SELECT {lineage_select}
FROM sessions
WHERE id = ?
""",
@@ -8601,8 +8656,8 @@ def get_state_db_session_messages(
if not parent_id or parent_id in seen:
break
cur.execute(
"""
SELECT id, source, started_at, parent_session_id, ended_at, end_reason
f"""
SELECT {lineage_select}
FROM sessions
WHERE id = ?
""",
@@ -10382,12 +10437,18 @@ def _insert_state_message_chronologically(messages: list, msg: dict) -> bool:
# preceding assistant's tool_calls, not just the first — a multi-tool
# turn has several adjacent tool results, and inserting between any of
# them splits the block (assistant, tool, <insert>, tool).
if idx < len(messages) and messages[idx].get("role") == "tool":
# Walk back over any contiguous tool rows already emitted for
# this block, so the owning assistant is found even when idx
# lands on the SECOND result of a multi-tool turn (where
# messages[idx - 1] is another tool row, not the assistant).
owner = idx - 1
while owner >= 0 and messages[owner].get("role") == "tool":
owner -= 1
if (
idx < len(messages)
and messages[idx].get("role") == "tool"
and idx > 0
and messages[idx - 1].get("role") == "assistant"
and messages[idx - 1].get("tool_calls")
owner >= 0
and messages[owner].get("role") == "assistant"
and messages[owner].get("tool_calls")
):
while idx < len(messages) and messages[idx].get("role") == "tool":
idx += 1
@@ -10420,6 +10481,7 @@ def merge_session_messages_append_only(
*,
truncation_watermark=None,
truncation_boundary=None,
incoming_provenance: Literal["unverified", "state_db"] = "unverified",
) -> list:
"""Merge sidecar/context and state.db messages without deleting local rows.
@@ -10433,6 +10495,7 @@ def merge_session_messages_append_only(
state_messages,
truncation_watermark=truncation_watermark,
truncation_boundary=truncation_boundary,
incoming_provenance=incoming_provenance,
)
finally:
_STRUCTURED_IDENTITY_MEMO.reset(token)
@@ -10444,6 +10507,7 @@ def _merge_session_messages_append_only_impl(
*,
truncation_watermark=None,
truncation_boundary=None,
incoming_provenance=None,
) -> list:
"""Merge sidecar/context and state.db messages without deleting local rows.
@@ -11038,6 +11102,20 @@ def _merge_session_messages_append_only_impl(
seen_dedup_keys.add(dedup_key)
seen_content_keys.add(content_key)
seen_visible_keys.add(visible_key)
# This terminal path is shared by state.db reconciliation and by ordered
# sidecar stitching (notably compression continuations). Only the caller
# that read state.db may authorize timestamp-based recovery placement;
# stable child-sidecar sequence remains authoritative even when an
# archived parent was restamped later.
if (
incoming_provenance == "state_db"
and max_sidecar_timestamp is not None
and timestamp is not None
and timestamp < max_sidecar_timestamp
):
if not _insert_state_message_chronologically(merged_messages, msg):
merged_messages.append(msg)
else:
merged_messages.append(msg)
_remember_merged_message(msg, source="state")
return merged_messages
@@ -11158,6 +11236,7 @@ def reconciled_state_db_messages_for_session(
state_messages,
truncation_watermark=getattr(session, "truncation_watermark", None),
truncation_boundary=getattr(session, "truncation_boundary", None),
incoming_provenance="state_db",
)
return _state_db_session_messages_result(
reconciled_messages,
api/onboarding.py
+3 -1
View File
@@ -71,7 +71,9 @@ _SUPPORTED_PROVIDER_SETUPS = {
"default_model": "anthropic/claude-sonnet-4.6",
"requires_base_url": False,
"models": [
{"id": _to_openrouter_namespace(model["id"]), "label": model["label"]}
{"id": "z-ai/glm-4.5-air", "label": "GLM-4.5 Air"}
if model["id"] == "zai/glm-4.5-flash"
else {"id": _to_openrouter_namespace(model["id"]), "label": model["label"]}
for model in _FALLBACK_MODELS
],
"category": "easy_start",
api/process_event_utils.py
+32 -1
View File
@@ -3,6 +3,7 @@ from __future__ import annotations
from collections import OrderedDict
from dataclasses import dataclass
import inspect
import logging
import math
import re
@@ -451,15 +452,45 @@ def complete_async_delegation_delivery(
_mark_legacy_async_delivery_complete(claim.delegation_id)
def _release_accepts_retryable(release_fn: Any) -> bool:
"""Whether the core ``release_event_delivery`` supports ``retryable=``.
Older Agent cores predate the keyword; calling them with it would raise
``TypeError`` and skip the release entirely.
"""
try:
params = inspect.signature(release_fn).parameters
except (TypeError, ValueError):
return False
if "retryable" in params:
return params["retryable"].kind in (
inspect.Parameter.KEYWORD_ONLY,
inspect.Parameter.POSITIONAL_OR_KEYWORD,
)
return any(p.kind is inspect.Parameter.VAR_KEYWORD for p in params.values())
def release_async_delegation_delivery(
evt: Any,
claim: AsyncDelegationDeliveryClaim,
*,
retryable: bool = False,
) -> None:
"""Release a failed claim so a later WebUI consumer can retry it."""
"""Release a failed claim so a later WebUI consumer can retry it.
``retryable=True`` tells the core the refusal was transient, so the claimed
attempt is refunded instead of counting against the bounded delivery
budget (``tools.async_delegation.release_event_delivery(...,
retryable=True)``). A core that predates the keyword gets a plain release:
the attempt is then consumed as before, never an exception.
"""
try:
if claim.durable:
from tools.async_delegation import release_event_delivery
if retryable and _release_accepts_retryable(release_event_delivery):
release_event_delivery(evt, claim.claim_id, retryable=True)
else:
release_event_delivery(evt, claim.claim_id)
except Exception:
logger.warning(
api/profiles.py
+32 -1
View File
@@ -198,7 +198,19 @@ def _unwrap_profile_home_to_base(home: Path) -> Path:
# are operator/deployment-level postures, not per-profile toggles. Letting a
# profile .env set HERMES_WEBUI_ISOLATED_PROFILE=0 would let a contained user
# escape isolation (#4589).
_PROTECTED_ENV_KEYS = frozenset({'HERMES_WEBUI_ISOLATED_PROFILE'})
_PROTECTED_ENV_KEYS = frozenset({
'HERMES_WEBUI_ISOLATED_PROFILE',
# Server-wide sidebar window; a profile .env must not widen the per-request query.
'HERMES_WEBUI_VISIBLE_SESSION_LIMIT',
# #7656 round-3: HERMES_WEBUI_MAX_SESSION_RESOLVE is a process-wide resource
# cap (controls the BoundedSemaphore that gates full-transcript resolves).
# A profile's .env is per-profile by design; allowing it to set a
# server-wide limit (and a first-load-wins asymmetry) would let a
# single profile drag the rest into a tighter or looser posture than
# the operator intended. Same shape as the isolated-profile key: only
# the operator/launcher env at startup can set it.
'HERMES_WEBUI_MAX_SESSION_RESOLVE',
})
def _isolated_profile_opt_in() -> bool:
@@ -934,6 +946,13 @@ _BLOCKED_RUNTIME_ENV_KEYS = {
# #4589: operator/deployment isolation posture — never overridable by a
# profile's own env on any runtime/gateway-parity path.
'HERMES_WEBUI_ISOLATED_PROFILE',
'HERMES_WEBUI_VISIBLE_SESSION_LIMIT',
# #7656 round-3: process-wide resolve cap. Filter out a profile's
# .env override the same way we filter the isolated-profile key, so
# the cap stays at the operator/launcher value across profile
# switches rather than re-sizing to whatever the first-loaded
# profile env set.
'HERMES_WEBUI_MAX_SESSION_RESOLVE',
}
@@ -1866,7 +1885,19 @@ def _compute_profile_skills_stats(profile_dir: Path) -> tuple[int, int]:
except Exception:
pass
try:
from agent.skill_utils import iter_skill_index_files, parse_frontmatter, skill_matches_platform
except ImportError as exc:
logger.debug("agent.skill_utils unavailable; reporting skill stats as unknown: %s", exc)
# agent source not mounted (two-container Docker,
# HERMES_WEBUI_CHAT_BACKEND=gateway): this must never 500 GET
# /api/profiles (#7305). Report the skill stats as unknown — a stable
# (0, 0) — instead of keeping a partial shadow of agent.skill_utils
# here: a local re-implementation cannot preserve the index walk's
# exclusions, frontmatter-name identity or platform filtering, so any
# count it produced would be inaccurate. The UI omits the skills line
# when the total is 0, so the profile picker stays fully usable.
return (0, 0)
seen_names = set()
enabled_count = 0
api/route_approvals.py
+125 -31
View File
@@ -2,6 +2,24 @@
State-extraction prelude to the routes.py split tracked in #1907.
Extracts approval state, not handlers, by design.
Gateway approvals carry two pieces of state with one lifecycle:
* the **producer** entry in ``tools.approval._gateway_queues`` — the queued
approval itself, which owns the lifecycle; and
* the **mirror** entry in ``_pending`` — this module's projection of that
producer for the UI.
A producer is consumed only by an explicit settlement operation, which assigns
its result and signals its waiter before fallible projection work begins.
``retire_gateway_pending_mirror`` cleans up the UI projection; it must not
silently delete a producer, because that would strand the agent thread parked
on the producer's event. For a provider-qualified response,
``resolve_gateway_pending_run`` matches the exact ``(run_id, approval_id)``
pair, settles that producer, and preserves same-run siblings and other runs.
When a run becomes terminal, ``settle_gateway_pending_run`` fail-closes every
producer owned by that run with ``deny`` before retiring its mirrors; producers
owned by another run remain live.
"""
import queue
import threading
@@ -505,21 +523,26 @@ def release_gateway_approval_relay_owner(session_key: str, run_id: str, approval
_gateway_relay_owners.pop(key, None)
def _settle_gateway_entry(entry, choice: str, reason: str | None = None) -> None:
"""Complete a removed producer before any fallible projection work."""
entry.result = choice
if reason:
entry.reason = reason
entry.event.set()
def retire_gateway_pending_mirror(
session_key: str,
approval_id: str = "",
run_id: str = "",
mirror_token: str = "",
) -> bool:
"""Retire one approval, or every mirror for a terminal run."""
"""Retire one approval mirror, or every mirror for a terminal run."""
with _lock:
reconcile_gateway_pending_mirror_locked(session_key)
queue = _pending.get(session_key)
entries = queue if isinstance(queue, list) else [queue] if queue else []
normalized_run_id = str(run_id or "").strip()
gateway_queue = _gateway_queues.get(session_key) or []
retained_gateway_queue = gateway_queue
gateway_queue_changed = False
if approval_id:
match = _gateway_pending_mirror_locked(
session_key,
@@ -542,15 +565,7 @@ def retire_gateway_pending_mirror(
if _is_gateway_mirror_entry(entry)
and not str(entry.get("run_id") or "").strip()
]
if normalized_run_id:
retained_gateway_queue = []
for entry in gateway_queue:
data = getattr(entry, "data", None) or {}
if str(data.get("run_id") or "").strip() == normalized_run_id:
gateway_queue_changed = True
continue
retained_gateway_queue.append(entry)
if not retired and not gateway_queue_changed:
if not retired:
head, total, changed = reconcile_gateway_pending_mirror_locked(session_key)
_approval_sse_notify_locked(session_key, head, total)
if changed:
@@ -558,11 +573,6 @@ def retire_gateway_pending_mirror(
return changed
for match in retired:
entries.remove(match)
if normalized_run_id and not approval_id:
if retained_gateway_queue:
_gateway_queues[session_key] = retained_gateway_queue
else:
_gateway_queues.pop(session_key, None)
if entries:
_pending[session_key] = entries
else:
@@ -573,6 +583,60 @@ def retire_gateway_pending_mirror(
return True
def settle_gateway_pending_run(
session_key: str,
run_id: str,
*,
reason: str,
) -> tuple[int, dict | None, int]:
"""Fail-close every producer and mirror owned by a terminal gateway run."""
run_id = str(run_id or "").strip()
if not run_id:
return 0, None, 0
targets = []
with _lock:
gateway_queue = _gateway_queues.get(session_key) or []
retained_gateway_queue = []
for entry in gateway_queue:
data = getattr(entry, "data", None) or {}
if str(data.get("run_id") or "").strip() == run_id:
targets.append(entry)
else:
retained_gateway_queue.append(entry)
if retained_gateway_queue:
_gateway_queues[session_key] = retained_gateway_queue
else:
_gateway_queues.pop(session_key, None)
# Removal and waiter settlement are one semantic operation. In
# particular, reconciliation and SSE notification below may raise.
for entry in targets:
_settle_gateway_entry(entry, "deny", reason)
queue = _pending.get(session_key)
entries = queue if isinstance(queue, list) else [queue] if queue else []
retained_pending = [
entry
for entry in entries
if not (
_is_gateway_mirror_entry(entry)
and str(entry.get("run_id") or "").strip() == run_id
)
]
mirrors_retired = len(retained_pending) != len(entries)
if retained_pending:
_pending[session_key] = retained_pending
else:
_pending.pop(session_key, None)
head, total, changed = reconcile_gateway_pending_mirror_locked(session_key)
_approval_sse_notify_locked(session_key, head, total)
if targets or mirrors_retired or changed:
publish_session_list_changed("attention_resolved")
return len(targets), head, total
def _gateway_mirrored_pending_run_id(session_key: str, approval_id: str) -> str | None:
"""Compatibility wrapper for exact run-backed lookup."""
approval_id = str(approval_id or "").strip()
@@ -804,14 +868,50 @@ def resolve_gateway_pending_local(
_gateway_queues[session_key] = gateway_queue
else:
_gateway_queues.pop(session_key, None)
if target is not None:
_settle_gateway_entry(target, choice, reason)
head, total, _changed = reconcile_gateway_pending_mirror_locked(session_key)
_approval_sse_notify_locked(session_key, head, total)
if target is None:
return 0, head, total
publish_session_list_changed("attention_resolved")
return 1, head, total
def resolve_gateway_pending_run(
session_key: str,
approval_id: str,
run_id: str,
choice: str,
reason: str | None = None,
) -> tuple[int, dict | None, int]:
"""Resolve one exact run-backed producer and wake its parked waiter."""
target = None
approval_id = str(approval_id or "").strip()
run_id = str(run_id or "").strip()
if not approval_id or not run_id:
return 0, None, 0
with _lock:
gateway_queue = _gateway_queues.get(session_key) or []
retained_gateway_queue = []
for entry in gateway_queue:
data = getattr(entry, "data", None) or {}
entry_approval_id = str(data.get("approval_id") or "").strip()
entry_run_id = str(data.get("run_id") or "").strip()
if target is None and entry_approval_id == approval_id and entry_run_id == run_id:
target = entry
else:
retained_gateway_queue.append(entry)
if retained_gateway_queue:
_gateway_queues[session_key] = retained_gateway_queue
else:
_gateway_queues.pop(session_key, None)
if target is not None:
_settle_gateway_entry(target, choice, reason)
head, total, _changed = reconcile_gateway_pending_mirror_locked(session_key)
_approval_sse_notify_locked(session_key, head, total)
if target is None:
return 0, head, total
target.result = choice
if reason:
target.reason = reason
target.event.set()
publish_session_list_changed("attention_resolved")
return 1, head, total
@@ -856,12 +956,9 @@ def resolve_gateway_pending_local_no_run_mirror(
_pending[session_key] = entries
else:
_pending.pop(session_key, None)
_settle_gateway_entry(target, choice, reason)
head, total, _changed = reconcile_gateway_pending_mirror_locked(session_key)
_approval_sse_notify_locked(session_key, head, total)
target.result = choice
if reason:
target.reason = reason
target.event.set()
publish_session_list_changed("attention_resolved")
return True, 1, head, total
@@ -904,14 +1001,11 @@ def resolve_gateway_pending_local_all(
else:
_pending.pop(session_key, None)
for entry in targets:
_settle_gateway_entry(entry, choice, reason)
head, total, _changed = reconcile_gateway_pending_mirror_locked(session_key)
_approval_sse_notify_locked(session_key, head, total)
for entry in targets:
entry.result = choice
if reason:
entry.reason = reason
entry.event.set()
if targets or removed_pending:
publish_session_list_changed("attention_resolved")
return len(targets), head, total
api/routes.py
+461 -49
View File
@@ -46,6 +46,7 @@ from api.agent_sessions import (
_looks_like_default_cli_title,
is_cli_session_row,
is_cli_session_row_visible,
open_state_db_readonly,
read_session_lineage_report,
)
from api.compression_anchor import visible_messages_for_anchor
@@ -320,7 +321,7 @@ def _latest_cron_session_info_for_jobs(
if not db_path or not Path(db_path).exists():
return {jid: {"session_id": "", "message_count": None} for jid in requested}
try:
with closing(sqlite3.connect(str(db_path))) as conn:
with closing(open_state_db_readonly(db_path)) as conn:
conn.row_factory = sqlite3.Row
cur = conn.cursor()
cur.execute("PRAGMA table_info(sessions)")
@@ -593,7 +594,15 @@ def _request_session_visibility_exempt(method: str, path: str | None) -> bool:
def _session_id_visible_to_request_profile(handler, sid, *, emit_error: bool = True) -> bool:
"""Return whether ``sid`` belongs to the active profile."""
"""Return whether ``sid`` belongs to the active profile.
On a profile mismatch, the helper mirrors the detail-load endpoint's
contract (#13043, #13493): return ``409 session_profile_mismatch`` for
a session owned by a KNOWN other profile, and keep ``404 Session
not found`` only for the unknown/legacy None-profile case so the
frontend's self-heal (clear stale URL + localStorage) keeps firing
for actually-missing sids. ``#7710``.
"""
if not isinstance(sid, str) or not sid:
return True
if not is_safe_session_id(sid):
@@ -602,8 +611,22 @@ def _session_id_visible_to_request_profile(handler, sid, *, emit_error: bool = T
session = get_session(sid, metadata_only=True)
except KeyError:
return True
if not _session_visible_to_active_profile(getattr(session, "profile", None), handler):
session_profile = getattr(session, "profile", None) or None
if not _session_visible_to_active_profile(session_profile, handler):
if emit_error:
if session_profile:
j(handler, {
"error": "Session belongs to a different profile",
"code": "session_profile_mismatch",
"session_id": sid,
"profile": session_profile,
}, status=409)
else:
# Unknown/legacy None-profile sidecar: keep the 404 so the
# frontend's self-heal still fires. _profiles_match coerces
# None->'default', so a truly missing/legacy session under a
# non-default active profile would otherwise emit a useless
# 409 with profile=null.
bad(handler, "Session not found", 404)
return False
return True
@@ -2521,8 +2544,8 @@ def _build_session_list_cache_payload(
)
if show_cli_sessions:
diag_stage("cli_cap")
archived_scoped = _cap_recent_cli_sessions(archived_scoped, cli_cap=CLI_VISIBLE_SESSION_CAP)
visible_scoped = _cap_recent_cli_sessions(visible_scoped, cli_cap=CLI_VISIBLE_SESSION_CAP)
archived_scoped = _cap_recent_cli_sessions(archived_scoped)
visible_scoped = _cap_recent_cli_sessions(visible_scoped)
if visible_only:
archived_scoped = [
s for s in archived_scoped if _session_has_server_visible_messages(s)
@@ -2952,6 +2975,7 @@ from api.helpers import (
require,
bad,
safe_resolve,
arm_connection_close_if_body_pending,
j,
t,
read_body,
@@ -3461,16 +3485,31 @@ def _run_journal_live_snapshot(stream_id: str | None, *, handler=None) -> dict |
emit_error=False,
):
return None
summary = find_run_summary(stream_id)
if not summary:
# Locate the run journal WITHOUT parsing it first. find_run_summary reads
# and parses the whole file (tens of thousands of rows on a long live
# run), and read_run_events below then parsed it AGAIN — two full passes
# cost ~0.6s per rebuild. Derive the durable summary from the single
# parse below instead; its last_seq / last_event_id are rebuilt from the
# same rows the snapshot consumes.
located = find_run_file(stream_id)
if located:
session_id, _journal_path = located
else:
# No journal file on disk for this run id: fall back to the
# historical summary-based lookup seam (defensive — a real miss
# returns None exactly like before; this is also the seam that
# long-standing tests stub with a handler-side summary).
fallback_summary = find_run_summary(stream_id)
if not fallback_summary:
return None
session_id = str(summary.get("session_id") or "")
session_id = str(fallback_summary.get("session_id") or "")
if not session_id:
return None
journal = read_run_events(session_id, stream_id)
events = [event for event in (journal.get("events") or []) if isinstance(event, dict)]
if not events:
return None
summary = _summary_from_events(session_id, stream_id, events)
event_run_ids: set[str] = set()
malformed_envelope_run_id = False
for event in events:
@@ -3490,6 +3529,18 @@ def _run_journal_live_snapshot(stream_id: str | None, *, handler=None) -> dict |
assistant_text = ""
reasoning_text = ""
# Reasoning deltas arrive as thousands of tiny append events. Growing the
# transcript with ``reasoning_text += chunk`` copies the full (multi-
# hundred-KB) string on every append inside these closures — quadratic
# (~2.3s of a 4.6s rebuild on a 9.7k-chunk run). Collect chunks and
# materialize lazily; readers (echo probes, strip, final message) join at
# most once per interim segment.
reasoning_parts: list[str] = []
reasoning_dirty = False
# Incremental folded index over the reasoning transcript: the per-interim
# echo probe consults this instead of re-walking the raw text (see
# ``_CompactEchoIndex``).
reasoning_index = _CompactEchoIndex()
messages: list[dict] = []
tool_calls: list[dict] = []
activity_burst_anchors: list[dict] = []
@@ -3498,6 +3549,13 @@ def _run_journal_live_snapshot(stream_id: str | None, *, handler=None) -> dict |
last_ts = None
reasoning_first_tool_count: int | None = None
def _materialize_reasoning_text() -> str:
nonlocal reasoning_text, reasoning_dirty
if reasoning_dirty:
reasoning_text = "".join(reasoning_parts)
reasoning_dirty = False
return reasoning_text
def mark_boundary() -> int:
nonlocal current_activity_burst_id
text_end = len(assistant_text)
@@ -3562,22 +3620,42 @@ def _run_journal_live_snapshot(stream_id: str | None, *, handler=None) -> dict |
tool_calls.append(call)
def reasoning_echo_tail_matches(text: str) -> bool:
candidate = _compact_for_echo_compare(text)
if not candidate:
return False
return _compact_for_echo_compare(reasoning_text).endswith(candidate)
# Indexed tail match (no raw-text walk): the incremental index folds
# each reasoning chunk once as it is appended, so this probe costs
# O(len(text)) regardless of how much interior whitespace stretches
# the raw span. The previous raw backward walk re-walked that span
# once per interim event, which turned the replay quadratic on a
# whitespace-heavy transcript (#7569 review, ~55x slower than master
# on a production-shaped journal).
return reasoning_index.matches_tail(text)
def strip_reasoning_echo_tail(text: str) -> bool:
nonlocal reasoning_text, reasoning_first_tool_count
next_reasoning, did_remove = _strip_compact_echo_suffix(reasoning_text, text)
if did_remove:
nonlocal reasoning_text, reasoning_dirty, reasoning_first_tool_count
cut = reasoning_index.cut_to(text)
if cut is None:
return False
next_reasoning = _materialize_reasoning_text()[:cut].rstrip()
reasoning_text = next_reasoning
reasoning_parts.clear()
reasoning_parts.append(next_reasoning)
reasoning_dirty = False
# Re-index the truncated transcript so later probes match against it
# instead of the pre-strip tail (the index is the match authority).
reasoning_index.reset()
reasoning_index.append(next_reasoning)
if not _compact_for_echo_compare(reasoning_text):
reasoning_first_tool_count = None
return did_remove
return True
for event in events:
event_name = str(event.get("event") or event.get("type") or "")
if event_name == "metering":
# Metering rows are the bulk of a long live journal (~10k rows,
# ~60% of its bytes) and project nothing onto the snapshot; skip
# their per-row work while preserving the last_ts watermark they
# carry.
last_ts = event.get("created_at", last_ts)
continue
payload = event.get("payload") if isinstance(event.get("payload"), dict) else {}
last_ts = event.get("created_at", last_ts)
if event_name == "token":
@@ -3588,9 +3666,12 @@ def _run_journal_live_snapshot(stream_id: str | None, *, handler=None) -> dict |
continue
if event_name == "reasoning":
text = str(payload.get("text") or "")
if text and reasoning_first_tool_count is None:
if text:
if reasoning_first_tool_count is None:
reasoning_first_tool_count = len(tool_calls)
reasoning_text += text
reasoning_parts.append(text)
reasoning_index.append(text)
reasoning_dirty = True
continue
if event_name == "interim_assistant":
visible = str(payload.get("text") or "").strip()
@@ -3635,6 +3716,8 @@ def _run_journal_live_snapshot(stream_id: str | None, *, handler=None) -> dict |
update_completed_tool(payload)
fresh_segment = True
reasoning_text = _materialize_reasoning_text()
if assistant_text or reasoning_text:
message = {
"role": "assistant",
@@ -5323,7 +5406,21 @@ def _handle_session_anchor_scene(handler, body):
# this an authenticated request under profile A could persist anchor scenes
# onto a session owned by profile B (cross-profile write). Reject as 404 —
# same shape the read path uses — and leave anchor_activity_scenes untouched.
if not _session_visible_to_active_profile(getattr(s, "profile", None) or None, handler):
# #7710: cross-profile writes are rejected with 409
# ``session_profile_mismatch`` so the client can offer to switch
# to the owning profile (mirrors the detail-load endpoint's
# contract at #13043 / #13493). 404 is preserved for the
# None-profile (unknown/legacy) case so the frontend self-heal
# path still fires for actually-missing sids.
_anchor_session_profile = getattr(s, "profile", None) or None
if not _session_visible_to_active_profile(_anchor_session_profile, handler):
if _anchor_session_profile:
return j(handler, {
"error": "Session belongs to a different profile",
"code": "session_profile_mismatch",
"session_id": sid,
"profile": _anchor_session_profile,
}, status=409)
return bad(handler, "Session not found", 404)
with _get_session_agent_lock(sid):
idx, message = _find_anchor_scene_message(
@@ -5835,6 +5932,15 @@ def _csrf_rejection_error(handler) -> str:
def _check_csrf(handler) -> bool:
"""Reject cross-origin or tokenless authenticated browser unsafe requests."""
if not _check_same_origin_browser_request(handler):
# CSRF checks run before read_body(), so close rather than reusing an
# HTTP/1.1 connection whose unread body would corrupt the next request --
# but ONLY when the framing says bytes are really queued. Arming
# unconditionally dropped a healthy pooled connection on a body-less
# write: verified on the wire, `POST /api/session/new` with
# `Origin: http://evil.invalid` and no `Content-Length` (and with
# `Content-Length: 0`) answered 403 + `Connection: close` and the
# pipelined `GET /api/health/agent` was never served.
arm_connection_close_if_body_pending(handler)
return False
if not _is_browser_unsafe_request(handler):
return True # non-browser clients (curl, MCP, agent) have no Origin/Referer
@@ -5847,6 +5953,11 @@ def _check_csrf(handler) -> bool:
submitted = handler.headers.get(CSRF_HEADER_NAME) or handler.headers.get("X-CSRF-Token")
if verify_csrf_token(cookie_val or "", submitted or ""):
return True
# Same framing rule as the origin rejection above: a token mismatch on a
# body-less write has nothing unread to protect. Verified on the wire with an
# authenticated same-origin `POST /api/session/new` carrying no CSRF token and
# no `Content-Length`: 403 + `Connection: close`, follow-up dropped.
arm_connection_close_if_body_pending(handler)
return _set_csrf_failure_reason(handler, "token_mismatch")
@@ -6026,6 +6137,14 @@ def _handle_extension_sidecar_proxy(
# DELETE fell through the CSRF compatibility path that intentionally admits
# non-browser clients, giving unsafe methods weaker provenance than GET.
if not _check_same_origin_browser_request(handler, require_provenance=True):
# Provenance rejection runs before read_body(), so close-and-advertise
# whenever the request DECLARED a body (Content-Length non-zero, or any
# Transfer-Encoding): those bytes are still queued in rfile and a reused
# HTTP/1.1 connection would parse them as the next request line (same
# class as _check_csrf). Gating on read_request_body instead was wrong in
# both directions — it missed a GET that carries a declared body, and it
# closed a healthy connection on a body-less DELETE/PUT/PATCH.
arm_connection_close_if_body_pending(handler)
return j(handler, {"error": _csrf_rejection_error(handler)}, status=403)
try:
request_body = _read_body_bytes(handler) if read_request_body else None
@@ -6468,6 +6587,13 @@ def _handle_csp_report(handler) -> bool:
"Dropped CSP report from %s: rate limit exceeded",
_client_ip_for_rate_limit(handler),
)
# Rate-limit rejection runs before the body is read; close-and-advertise
# so the unread report can't corrupt the next pooled request -- but only
# when a body was really declared. A body-less report answered 204 WITH
# `Connection: close` once the 100-per-60s limiter tripped (reproduced on
# the wire at request 101; the pipelined `GET /api/health/agent` was
# dropped), so a browser that keeps reporting loses its socket each time.
arm_connection_close_if_body_pending(handler)
return _send_no_content(handler)
payload = _read_csp_report_payload(handler)
@@ -9230,6 +9356,7 @@ def _limited_webui_messages_for_display_with_sidecar(
state_db_messages,
truncation_watermark=getattr(session, "truncation_watermark", None),
truncation_boundary=getattr(session, "truncation_boundary", None),
incoming_provenance="state_db",
)
if cache_key is not None:
_state_key = cache_key[4]
@@ -10278,11 +10405,16 @@ def _dedupe_cli_sidebar_sessions_for_api(
return _include_project_hidden_background_sidebar_sessions(candidates, visible)
CLI_VISIBLE_SESSION_CAP = 20
def _cli_visible_session_cap() -> int:
"""Shared sidebar window, not a second hard-coded 20."""
from api.config import CLI_VISIBLE_SESSION_LIMIT
return CLI_VISIBLE_SESSION_LIMIT
def _cap_recent_cli_sessions(sessions: list[dict], cli_cap: int = CLI_VISIBLE_SESSION_CAP) -> list[dict]:
def _cap_recent_cli_sessions(sessions: list[dict], cli_cap: int | None = None) -> list[dict]:
"""Keep only the most recent CLI-visible sessions after filtering."""
if cli_cap is None:
cli_cap = _cli_visible_session_cap()
if cli_cap <= 0:
return sessions
kept = []
@@ -10488,6 +10620,11 @@ def _pre_compression_continuation_session_id(session) -> str | None:
exists either in memory or on disk. Follow bounded snapshot-to-snapshot hops
so repeated compression still lands on the latest visible continuation.
"""
from api.compression_continuation import durable_compression_continuation
sealed, tip = durable_compression_continuation(session)
if sealed:
return tip
if not getattr(session, "pre_compression_snapshot", False):
return None
sid = _safe_first(getattr(session, "session_id", None))
@@ -10698,13 +10835,16 @@ from api.streaming import (
_materialize_pending_user_turn_before_error,
generate_session_title_for_session,
_compact_for_echo_compare,
_strip_compact_echo_suffix,
_CompactEchoIndex,
)
from api.gateway_chat import _run_gateway_chat_streaming, webui_gateway_chat_enabled
from api.run_journal import (
_parse_run_journal_event_id as _shared_parse_run_journal_event_id,
_summary_from_events,
bound_run_journal_snapshot_args,
find_run_file,
find_run_summary,
journal_replay_visible,
read_run_events,
read_session_run_events,
session_journal_fingerprint,
@@ -10765,8 +10905,10 @@ from api.route_approvals import ( # noqa: F401 — re-exports for backward comp
gateway_pending_mirrors,
release_gateway_approval_relay_owner,
retire_gateway_pending_mirror,
settle_gateway_pending_run,
reconcile_gateway_pending_mirror_locked,
resolve_gateway_pending_local,
resolve_gateway_pending_run,
resolve_gateway_pending_local_all,
resolve_gateway_pending_local_no_run_mirror,
set_session_yolo_enabled,
@@ -11080,9 +11222,7 @@ button:hover{background:rgba(124,185,255,.25)}
<h1>{{BOT_NAME}}</h1>
<p class="sub">{{LOGIN_SUBTITLE}}</p>
<form id="login-form" data-invalid-pw="{{LOGIN_INVALID_PW}}" data-conn-failed="{{LOGIN_CONN_FAILED}}">
<input type="password" id="pw" placeholder="{{LOGIN_PLACEHOLDER}}" autofocus>
<button type="submit">{{LOGIN_BTN}}</button>
<button type="button" id="passkey-login" class="passkey-login" style="display:none">Sign in with passkey</button>
{{PASSWORD_FORM_HTML}}
{{OIDC_LOGIN_HTML}}
</form>
<div class="err" id="err"></div>
@@ -11897,7 +12037,7 @@ def _handle_insights(handler, parsed) -> bool:
from api.models import _active_state_db_path
db_path = _active_state_db_path()
if db_path and db_path.exists():
with closing(sqlite3.connect(str(db_path))) as conn:
with closing(open_state_db_readonly(db_path)) as conn:
conn.row_factory = sqlite3.Row
cur = conn.cursor()
# cache_read_tokens may not exist on older agent state DBs;
@@ -12532,7 +12672,7 @@ def _deep_health_checks(stream_check: dict | None = None) -> tuple[dict, bool]:
"ms": round((time.time() - t0) * 1000, 1),
}
else:
with closing(sqlite3.connect(str(db_path))) as conn:
with closing(open_state_db_readonly(db_path)) as conn:
conn.execute("PRAGMA schema_version").fetchone()
checks["state_db"] = {
"status": "ok",
@@ -12853,6 +12993,15 @@ def _handle_shutdown(handler) -> bool:
def _handle_health_restart(handler) -> bool:
"""Restart the Hermes messaging gateway service."""
# This endpoint never consumes its request body on any outcome, so close when
# one was DECLARED -- and only then. Arming unconditionally closed the socket
# on every call including the successful, body-less one the WebUI actually
# makes: verified on the wire, `POST /api/health/restart` with no
# `Content-Length` answered with `Connection: close` and the pipelined
# `GET /api/health/agent` was never served. The single arming covers every
# outcome below (completed / in_progress / busy / error) because the framing,
# not the result, decides.
arm_connection_close_if_body_pending(handler)
outcome = restart_active_profile_gateway()
if outcome.get("status") == "completed":
@@ -13625,6 +13774,36 @@ def handle_get(handler, parsed) -> bool:
]
from urllib.parse import quote
from api.updates import WEBUI_VERSION
# #7056: only render the password input / submit / passkey controls
# when password auth is actually enabled. With native OIDC configured
# and ``HERMES_WEBUI_PASSWORD`` unset, the form previously still
# displayed the password prompt and accepted — silently 401-ing at
# the server — every submit. The OIDC SSO entry point stays the
# sole path. ``is_password_auth_enabled`` is the same predicate
# ``/api/auth/status`` reports as ``password_auth_enabled``.
from api.auth import are_passkeys_enabled, is_password_auth_enabled
# The password INPUT is gated on a configured password, but the passkey
# button must survive a passwordless-passkey deployment: settings expose
# ``passwordless_enabled = passkeys registered AND not password_auth_enabled``
# (routes.py ~14059) and ``is_auth_enabled()`` counts passkeys as an
# independent auth method, so hiding the button when no password is set
# would remove the ONLY working login affordance for those instances.
_passkey_button_html = (
'<button type="button" id="passkey-login" class="passkey-login" '
'style="display:none">Sign in with passkey</button>'
)
if is_password_auth_enabled():
_password_form_html = (
f'<input type="password" id="pw" '
f'placeholder="{_html.escape(_login_strings["placeholder"])}" autofocus>'
f'<button type="submit">{_html.escape(_login_strings["btn"])}</button>'
f'{_passkey_button_html}'
)
elif are_passkeys_enabled():
_password_form_html = _passkey_button_html
else:
_password_form_html = ""
version_token = quote(WEBUI_VERSION, safe="")
_page = (
_LOGIN_PAGE_HTML.replace("{{BOT_NAME}}", _bn)
@@ -13633,10 +13812,7 @@ def handle_get(handler, parsed) -> bool:
.replace("{{LANG}}", _html.escape(_login_strings["lang"]))
.replace("{{LOGIN_TITLE}}", _html.escape(_login_strings["title"]))
.replace("{{LOGIN_SUBTITLE}}", _html.escape(_login_strings["subtitle"]))
.replace(
"{{LOGIN_PLACEHOLDER}}", _html.escape(_login_strings["placeholder"])
)
.replace("{{LOGIN_BTN}}", _html.escape(_login_strings["btn"]))
.replace("{{PASSWORD_FORM_HTML}}", _password_form_html)
.replace("{{LOGIN_INVALID_PW}}", _html.escape(_login_strings["invalid_pw"]))
.replace(
"{{LOGIN_CONN_FAILED}}", _html.escape(_login_strings["conn_failed"])
@@ -14478,7 +14654,11 @@ def handle_get(handler, parsed) -> bool:
if stop_gateway_run(run_id):
owner_sid = stream_owner_session_id(stream_id)
if owner_sid:
retire_gateway_pending_mirror(owner_sid, run_id=run_id)
settle_gateway_pending_run(
owner_sid,
run_id,
reason="Gateway run was cancelled before approval resolution",
)
else:
gateway_stop_blocked = True
except Exception:
@@ -15144,6 +15324,12 @@ def handle_post(handler, parsed) -> bool:
if diag:
diag.stage("process_complete_ack_deprecated")
try:
# The 410 runs before the stale tab's JSON body is read;
# close-and-advertise so those unread bytes can't corrupt the next
# pooled request -- but a body-less ack has none, and closing then
# just kills keep-alive (on the wire: 410 + `Connection: close` with
# no `Content-Length`, pipelined follow-up dropped).
arm_connection_close_if_body_pending(handler)
j(
handler,
{
@@ -18573,6 +18759,8 @@ def _replay_run_journal(
max_seq=max_seq,
)
for entry in journal.get("events") or []:
if not journal_replay_visible(entry):
continue
_sse_with_id(
handler,
entry.get("event") or entry.get("type") or "message",
@@ -19026,14 +19214,14 @@ def _handle_sse_stream(handler, parsed):
continue
if len(item) >= 3:
event, data, queued_event_id = item[0], item[1], item[2]
event_id = queued_event_id
else:
event, data = item
queued_event_id = STREAM_LAST_EVENT_ID.get(stream_id)
event_id = STREAM_LAST_EVENT_ID.get(stream_id)
# Stage-364: emit `id:` from STREAM_LAST_EVENT_ID side-channel so
# the frontend's `_lastRunJournalSeq` cursor advances during live
# streaming. Without this, mid-stream error→replay would arrive
# with after_seq=0 and double-render every journaled event.
event_id = queued_event_id or STREAM_LAST_EVENT_ID.get(stream_id)
event_seq = _run_journal_same_run_seq(event_id, stream_id)
if replay_cutoff_seq is not None and event_seq is not None and event_seq <= replay_cutoff_seq:
continue
@@ -19108,6 +19296,8 @@ def _handle_session_run_journal_stream_for_session(handler, parsed, session_id):
def emit_replay(events, stream_id, cutoff_seq):
for entry in events:
if not journal_replay_visible(entry):
continue
event_id = str(entry.get("event_id") or "")
event_seq = _run_journal_same_run_seq(event_id, stream_id)
if cutoff_seq is not None and event_seq is not None and event_seq > cutoff_seq:
@@ -19174,10 +19364,10 @@ def _handle_session_run_journal_stream_for_session(handler, parsed, session_id):
continue
if len(item) >= 3:
event, data, queued_event_id = item[0], item[1], item[2]
event_id = queued_event_id
else:
event, data = item
queued_event_id = STREAM_LAST_EVENT_ID.get(active_stream_id)
event_id = queued_event_id or STREAM_LAST_EVENT_ID.get(active_stream_id)
event_id = STREAM_LAST_EVENT_ID.get(active_stream_id)
event_seq = _run_journal_same_run_seq(event_id, active_stream_id)
_is_terminal = event in SSE_RELAY_CLOSE_EVENTS
_already_sent = (
@@ -20444,6 +20634,16 @@ def _serve_inline_html_preview(handler, target: Path, cache_control: str, *, csp
_MEDIA_TOKEN_RE = re.compile(r"MEDIA:([^\s\)\]]+)")
# #7680 re-gate (9/22): two-pass scan.
# 1. `` `MEDIA:path` `` (backtick-wrapped, inline-code form) → strip
# the wrapping backticks so the bare-token pass below sees a
# plain ``MEDIA:path`` and the closing backtick is not consumed
# as part of the path.
# 2. ``MEDIA:[^\s\)\]]+`` (bare, no backtick in the exclusion
# class) so a filename that legally contains a backtick
# (``report`final.png``) is captured in full instead of being
# truncated at the first backtick.
_BACKTICK_MEDIA_RE = re.compile(r"`MEDIA:([^`\s]+)`")
def _message_content_text(content) -> str:
@@ -20503,6 +20703,10 @@ def _session_media_token_allows_path(sid: str, target: Path, allowed_mimes: set[
)
if "MEDIA:" not in text:
continue
# #7680 re-gate: strip backtick wrappers first so the bare
# class below captures the full path even when the filename
# itself contains a backtick.
text = _BACKTICK_MEDIA_RE.sub(lambda m: f"MEDIA:{m.group(1)}", text)
for ref in _MEDIA_TOKEN_RE.findall(text):
if "://" in ref:
continue
@@ -21695,6 +21899,99 @@ def _handle_live_models(handler, parsed):
_env = str(_cp.get("key_env") or "").strip()
return os.getenv(_env, "").strip() if _env else ""
def _custom_provider_models_discover_is_false(_cp):
"""True when ``discover_models`` is an explicit ``false`` opt-out.
Mirrors ``api.config._provider_discover_allowed`` / Hermes
Agent ``model_switch_providers._discover_flag``: ``discover_models``
defaults to True, and the string forms ``"false"``/``"no"``/``"0"``
(case-insensitive) mean False. Used to decide whether a
dict-shaped ``models`` mapping is a hand-pinned allowlist (only
when discovery is off) rather than per-model metadata.
"""
_discover = _cp.get("discover_models", True) if isinstance(_cp, dict) else True
if isinstance(_discover, str):
return _discover.strip().lower() in {"false", "no", "0"}
return not bool(_discover)
def _custom_provider_allowlist_ids(_cp):
"""Plural ``models`` list only — the explicit allowlist signal.
The singular ``model`` field is sticky/default metadata, NOT an
allowlist: it must not gate live-catalog filtering, otherwise a
provider configured with only ``model: assistant`` (no ``models``
list) would be collapsed to a single model. Only an explicit
``models`` allowlist expresses "show exactly these models".
An auto-discovered catalog is NOT an allowlist: when Hermes
persisted discovery results back into config (``models: {...}``
plus ``models_discovered: true``), that mapping is a snapshot
of what the gateway exposed at discovery time. Gating on it
would permanently pin the live catalog to the first-discovery
set, silently dropping any model the user pulls in later
(LM Studio / Ollama). ``_provider_models_are_discovered_catalog()``
is the shared predicate the ``/api/models`` path already uses;
when it says "discovered", return no allowlist and let the
live probe win. An explicit ``discover_models: false`` opt-out
re-pins the catalog (the predicate accounts for it), so a
hand-pinned discovered catalog still filters.
A dict-shaped ``models`` mapping that is NOT marked discovered is
*per-model metadata* written by the Hermes Agent setup flow
(``hermes_cli/model_switch.py::_save_custom_provider`` and the
setup wizard) — e.g. ``{chat-a: {context_length: 128000}}``. It is
not a catalog narrow: treating its keys as an allowlist would
collapse the live picker to the single saved default (keyless
Ollama) while the CLI live-probe shows the full catalog. Only an
explicit ``discover_models: false`` opts into treating the dict
keys as a pinned allowlist. List/JSON-array-string/Python-literal
shapes remain plain allowlists.
The plural value is decoded through ``_parse_config_string_list()``
because ``hermes config set`` / JSON-mode editor saves persist
lists as quoted JSON-array strings (``'["chat-a","chat-b"]'``) or
Python literals (``"['chat-a']"``). Handling only ``dict``/``list``
made a serialized allowlist fall through to ``[]``, which the
caller reads as "no allowlist configured" and floods the picker
with the full upstream catalog — the same class of bug as the
``skills.disabled`` regression (#7120 / #7134).
"""
from api.config import _provider_models_are_discovered_catalog
if _provider_models_are_discovered_catalog(_cp):
return []
_ids = []
_models = _cp.get("models")
# Serialized shapes only: ``hermes config set`` / JSON-mode
# editor saves persist lists as quoted JSON-array strings
# (``'["chat-a","chat-b"]'``) or Python literals
# (``"['chat-a']"``). Decode those through the shared helper so
# they are honored; native dict/list values are walked below so
# dict *entries* keep their id|model|name metadata (the decoder
# str()-ifies list members, which would mangle them).
if isinstance(_models, str):
_models = _parse_config_string_list(_models)
if isinstance(_models, dict):
# A dict-shaped ``models`` is per-model metadata written by
# the Hermes Agent setup flow, NOT a catalog narrow. Only a
# ``discover_models: false`` opt-out treats the dict keys as a
# pinned allowlist; otherwise return the empty allowlist so the
# live probe returns the full catalog.
if not _custom_provider_models_discover_is_false(_cp):
return []
for _mid in _models:
if isinstance(_mid, str) and _mid.strip():
_ids.append(_mid.strip())
elif isinstance(_models, (list, tuple)):
for _item in _models:
if isinstance(_item, str) and _item.strip():
_ids.append(_item.strip())
elif isinstance(_item, dict):
_mid = _item.get("id") or _item.get("model") or _item.get("name")
if _mid and str(_mid).strip():
_ids.append(str(_mid).strip())
return _ids
# For 'custom' and 'custom:*' providers, provider_model_ids()
# returns [] because they aren't real hermes_cli endpoints.
# Fall back to the custom_providers entries from config.yaml so
@@ -21703,11 +22000,13 @@ def _handle_live_models(handler, parsed):
# Collect config-specified model IDs separately so they don't
# prevent the live fetch below from running (#3718).
_config_ids = []
_allowlist_ids = []
if provider == "custom" or provider.startswith("custom:"):
for _cp in _custom_provider_entries_for_request():
if custom_provider_entry is None:
custom_provider_entry = _cp
_config_ids.extend(_custom_provider_model_ids(_cp))
_allowlist_ids.extend(_custom_provider_allowlist_ids(_cp))
# Always try live fetch for custom providers — config entries are a
# fallback, not a replacement. The live endpoint should return ALL
@@ -21778,15 +22077,39 @@ def _handle_live_models(handler, parsed):
except Exception as _fetch_err:
logger.debug("Live fetch from custom provider failed: %s", _fetch_err)
# If live fetch succeeded, merge with config entries (live takes
# priority). If live fetch failed, fall back to config-only list.
# If live fetch succeeded, filter the live catalog down to the
# config-declared models allowlist first, then append any
# allowlisted models the live endpoint didn't return. Custom
# providers (esp. New-API-style gateways) expose their ENTIRE
# catalog via /v1/models — including image/audio models that
# are not chat models. The picker must not surface models the
# user never declared in config.yaml. Only a NON-EMPTY explicit
# ``models`` allowlist gates the filter — a provider configured
# with just a singular ``model`` (no ``models`` list) keeps the
# unfiltered live catalog. When no allowlist is configured,
# return the live list as-is (preserves the pre-filter
# discovery behaviour).
#
# An empty allowlist (``models: []``, ``models: "[]"``, or a
# value that decodes to no usable ids) is deliberately treated
# as "not configured", NOT as "allow nothing". Gating on
# declared-ness instead would emit an EMPTY picker and make the
# provider unselectable — a harder failure than surfacing a few
# extra models, and unrecoverable from the UI because
# ``custom_providers`` is hand-edited in config.yaml (the WebUI
# has no write path for it). ``[]`` in practice means a
# leftover/placeholder key, not an intentional deny-all, and no
# deny-all use case exists: a provider the user wants hidden is
# removed from ``custom_providers`` outright.
if ids:
_live_set = set(ids)
for _cid in _config_ids:
if _cid not in _live_set:
ids.append(_cid)
if _allowlist_ids:
_allowlist_set = set(_allowlist_ids)
ids = [m for m in ids if m in _allowlist_set] or []
for _aid in _allowlist_ids:
if _aid not in ids:
ids.append(_aid)
else:
ids = list(_config_ids)
ids = list(_allowlist_ids or _config_ids)
# ── OpenAI-compat live fetch fallback ──────────────────────────────────
# When provider_model_ids() is unavailable or returns [] for a provider
@@ -22843,6 +23166,54 @@ def _prepare_chat_start_session_for_stream(
s.save()
def _cleanup_chat_start_launch_failure(session, stream_id: str) -> None:
"""Release state registered before a worker thread successfully starts."""
clear_session_writeback_owner_if_owned(session.session_id, stream_id)
unregister_stream_owner(stream_id)
with STREAMS_LOCK:
STREAMS.pop(stream_id, None)
STREAM_GOAL_RELATED.pop(stream_id, None)
# The session-field reset needs the same concurrency discipline as the
# registry half: hold the per-session lock and re-resolve the canonical
# session before clearing anything. Mutating the passed-in stale object
# could wipe a concurrent successor turn's pending fields, and saving it
# could resurrect a session deleted while the launch was failing. Same
# pattern as the #1533 race fix (routes.py:3077) and the anchor-scene
# write guard (routes.py:5140).
#
# This runs while the original launch failure is being handled, so it must
# never raise. Lock acquisition and session resolution can fail on their own
# (I/O, deserialization), and an escaping error here would mask the launch
# failure the caller is about to report while leaving the reset half done.
try:
with _get_session_agent_lock(session.session_id):
try:
canonical = get_session(session.session_id)
except KeyError:
return # session deleted while the thread launch was failing
if getattr(canonical, "active_stream_id", None) != stream_id:
return # a successor turn already owns the session
canonical.active_stream_id = None
canonical.pending_user_message = None
canonical.pending_attachments = []
canonical.pending_started_at = None
canonical.pending_user_source = None
try:
canonical.save()
except Exception:
logger.debug(
"Failed to persist chat-start cleanup after worker launch failure for %s",
stream_id,
exc_info=True,
)
except Exception:
logger.debug(
"Failed to reset session state after worker launch failure for %s",
stream_id,
exc_info=True,
)
def _is_hidden_empty_session(s) -> bool:
return (
getattr(s, "title", "Untitled") == "Untitled"
@@ -23426,6 +23797,7 @@ def _start_chat_stream_for_session(
_clear_gateway_run_starting(stream_id)
except Exception:
logger.debug("Failed to record gateway run-start failure for stream %s", stream_id, exc_info=True)
_cleanup_chat_start_launch_failure(s, stream_id)
raise
response = {
"stream_id": stream_id,
@@ -23957,6 +24329,17 @@ def _handle_session_compression_recovery_start(handler, body):
except KeyError:
return bad(handler, "Session not found", 404)
if not _session_visible_to_active_profile(getattr(source, "profile", None), handler):
# #7710: same contract as the detail-load endpoint — 409
# ``session_profile_mismatch`` for a known other profile,
# 404 only for the None-profile self-heal path.
_recovery_session_profile = getattr(source, "profile", None)
if _recovery_session_profile:
return j(handler, {
"error": "Session belongs to a different profile",
"code": "session_profile_mismatch",
"session_id": sid,
"profile": _recovery_session_profile,
}, status=409)
return bad(handler, "Session not found", 404)
recovery = compression_recovery_payload_for_session(source)
if not recovery:
@@ -24356,8 +24739,29 @@ def _handle_chat_start(handler, body, diag=None):
# Empty placeholders can still be retagged when the
# requested profile matches the active request profile.
s.profile = requested_profile
elif session_profile:
# #7710: known other profile → 409 ``session_profile_mismatch``
# so the client can offer to switch to it (#5419).
# 404 is preserved only for the None-profile
# (unknown/legacy) self-heal case.
return j(handler, {
"error": "Session belongs to a different profile",
"code": "session_profile_mismatch",
"session_id": body.get("session_id", ""),
"profile": session_profile,
}, status=409)
else:
return bad(handler, "Session not found", 404)
# Resolve durable rotations before any workspace/model/pending mutation.
# GET navigation adopts the tip; POST never silently replays a user turn.
from api.compression_continuation import durable_compression_continuation
sealed, continuation = durable_compression_continuation(s)
if sealed:
return j(handler, {
"error": "This session was compressed. Open its continuation before sending.",
"code": "session_rotated",
"continuation_session_id": continuation,
}, status=409)
regeneration = None
if body.get("regenerate") is True:
if any(key in body for key in ("message", "attachments", "keep_count", "prompt", "prompt_index")):
@@ -26202,7 +26606,6 @@ def _resolve_approval_legacy(sid: str, approval_id: str, choice: str, run_id: st
found_target = False
gateway_keys = []
local_gateway_approval_id = ""
gateway_head_matches_target = False
with _lock:
reconcile_gateway_pending_mirror_locked(sid)
queue = _pending.get(sid)
@@ -26274,9 +26677,6 @@ def _resolve_approval_legacy(sid: str, approval_id: str, choice: str, run_id: st
gw_data = getattr(gw_entry, "data", None) or {}
gw_approval_id = str(gw_data.get("approval_id") or "").strip()
gw_run_id = str(gw_data.get("run_id") or "").strip()
gateway_head_matches_target = bool(
run_id and gw_approval_id == approval_id and gw_run_id == run_id
)
if gw_approval_id == approval_id and (not run_id or gw_run_id == run_id):
local_gateway_approval_id = approval_id
elif not run_id and found_target and pending:
@@ -26337,8 +26737,10 @@ def _resolve_approval_legacy(sid: str, approval_id: str, choice: str, run_id: st
local_gateway_resolved, _head, _total = resolve_gateway_pending_local(
sid, local_gateway_approval_id, choice
)
elif approval_id and found_target and run_id and gateway_head_matches_target:
gateway_resolved = resolve_gateway_approval(sid, choice, resolve_all=False) or 0
elif approval_id and found_target and run_id:
gateway_resolved, _head, _total = resolve_gateway_pending_run(
sid, approval_id, run_id, choice
)
elif not approval_id:
gateway_resolved = resolve_gateway_approval(sid, choice, resolve_all=False) or 0
# Keep the historical no-id response path truthy for old clients/tests while
@@ -28576,6 +28978,16 @@ def _handle_session_import_cli(handler, body):
if requested_profile and not _profiles_match(existing_profile, requested_profile):
return bad(handler, "Session not found in CLI store", 404)
elif not _session_visible_to_active_profile(existing_profile, handler):
# #7710: same contract as the detail-load endpoint —
# 409 ``session_profile_mismatch`` for a known other
# profile, 404 only for the None-profile self-heal path.
if existing_profile:
return j(handler, {
"error": "Session belongs to a different profile",
"code": "session_profile_mismatch",
"session_id": sid,
"profile": existing_profile,
}, status=409)
return bad(handler, "Session not found in CLI store", 404)
refresh_profile = requested_profile or existing_profile
cli_meta = _resolve_cli_import_metadata(
api/run_journal.py
+58 -9
View File
@@ -45,6 +45,14 @@ TERMINAL_SSE_EVENTS = frozenset({"done", "cancel", "apperror", "error", "stream_
SSE_RELAY_CLOSE_EVENTS = frozenset({"stream_end", "cancel", "apperror", "error"})
# Back-compat alias used by older call sites / tests.
_TERMINAL_SSE_EVENTS = TERMINAL_SSE_EVENTS
# Events that are live-UI-only telemetry with no recovery value in the run
# journal. They are skipped at WRITE time (never durably journaled, so they
# cannot bloat the journal on marathon runs) and filtered at REPLAY time (so
# legacy journals that already contain a backlog never stream it to a
# reconnecting browser tab). Readers deliberately do NOT filter: cursor math
# (``cursor_event_missing`` bound) and the offline-gap coverage check count
# journal seqs and must keep seeing every row.
REPLAY_SKIPPED_SSE_EVENTS = frozenset({"metering"})
_FSYNC_MODE_ENV = "HERMES_WEBUI_RUN_JOURNAL_FSYNC"
_FSYNC_MODE_EAGER = "eager"
_FSYNC_MODE_TERMINAL_ONLY = "terminal-only"
@@ -443,25 +451,45 @@ class RunJournalWriter:
self.session_id = _validate_id(session_id, "session_id")
self.run_id = _validate_id(run_id, "run_id")
self.session_dir = Path(session_dir) if session_dir is not None else None
self._path = _run_path(self.session_id, self.run_id, session_dir=self.session_dir)
self._lock = _lock_for(self._path)
def append_sse_event(self, event_name: str, payload=None) -> dict:
# Draw from the shared module-level seq cache under the per-path lock so
# this writer and any direct append_run_event() call on the same path
# agree on one monotonic, gapless sequence.
with self._lock:
seq = _reserve_next_seq(self._path)
def append_sse_event(self, event_name: str, payload=None) -> dict | None:
# Live-UI-only telemetry (metering) has no recovery value in the journal:
# nothing reads those rows back for recovery, and journaling them at ~10 Hz
# on marathon runs balloons the durable file (12+ MB of a single 18 MB run
# was metering). Skip the write entirely and return None so callers'
# journal-id plumbing (``(journaled or {}).get("event_id")``) is untouched.
# Not reserving a seq keeps the remaining journaled seqs contiguous, which
# the offline-gap coverage and replay-cursor contiguity checks rely on.
if str(event_name or "").strip() in REPLAY_SKIPPED_SSE_EVENTS:
return None
# Allocate the sequence inside the same per-path transaction that writes
# the row. Reserving here, then releasing the lock before append, lets a
# concurrent writer put a higher sequence on disk first.
return append_run_event(
self.session_id,
self.run_id,
event_name,
payload or {},
session_dir=self.session_dir,
seq=seq,
)
def journal_replay_visible(event) -> bool:
"""Return True when a journal row should be streamed to a reconnecting tab.
Live-UI-only telemetry rows (see ``REPLAY_SKIPPED_SSE_EVENTS``) carry no
recovery value — replaying a metering backlog only re-paints a stale TPS
number while multiplying the reconnect burst size. Writers no longer journal
them, but legacy journals may already contain them, so the replay emit sites
filter through this predicate. Non-dict rows are passed through (visible) so
an unexpected shape can never silently swallow user-visible output.
"""
if not isinstance(event, dict):
return True
name = str(event.get("event") or event.get("type") or "")
return name not in REPLAY_SKIPPED_SSE_EVENTS
def read_run_events(
session_id: str,
run_id: str,
@@ -587,6 +615,27 @@ def find_run_summary(run_id: str, *, session_dir: Path | None = None) -> dict |
return None
def find_run_file(run_id: str, *, session_dir: Path | None = None) -> tuple[str, Path] | None:
"""Locate a run journal file by run id WITHOUT parsing its body.
Hot callers that immediately read the full journal (the live-snapshot
rebuild) must not pay :func:`find_run_summary`'s full-file parse first:
on a long live run the file holds tens of thousands of rows and parsing
it twice per rebuild dominated the snapshot cost. Returns
``(session_id, path)`` for the first match, or ``None`` when the run id
is invalid or no journal exists.
"""
try:
rid = _validate_id(run_id, "run_id")
except ValueError:
return None
root = Path(session_dir) if session_dir is not None else _default_session_dir()
journal_root = root / RUN_JOURNAL_DIR_NAME
for path in journal_root.glob(f"*/{rid}.jsonl"):
return path.parent.name, path
return None
def read_session_run_events(
session_id: str,
*,
api/streaming.py
+541 -163
View File
@@ -67,6 +67,7 @@ from api.helpers import (
)
from api.compression_anchor import is_context_compression_marker, visible_messages_for_anchor
from api.compression_recovery import stamp_compression_exhausted_recovery
from api.gateway_chat import WEBUI_LOCAL_CHAT_BACKEND
from api.metering import meter
from api.run_journal import RunJournalWriter
from api.todo_state import attach_todo_state, emit_todo_state
@@ -196,41 +197,156 @@ def _compact_for_echo_compare(value: str) -> str:
return re.sub(r'\s+', '', str(value or ''))
def _strip_compact_echo_suffix(value: str, suffix: str, *, search_window: int = 4096) -> tuple[str, bool]:
class _CompactEchoIndex:
"""Incremental whitespace-folded index over a growing text buffer.
The journal-rebuild echo check runs once per interim event against the
whole accumulated reasoning transcript. Rescanning the raw buffer each
time walks every whitespace character between the tail and the first
non-whitespace character — on a whitespace-heavy transcript that span is
megabytes, and the per-interim walk turns the replay quadratic (the
``_find_compact_echo_suffix_start`` shape measured ~55x slower than
master on a production-shaped journal in the #7569 review).
This class keeps the folded view and the raw cut offsets *incrementally*:
* ``append`` folds each new chunk once and records, per folded character,
its raw index — so the cost of a chunk is proportional to the chunk,
never to the buffer.
* ``matches_tail`` compares only the candidate against the end of the
folded view: O(len(candidate)), with no raw-text walk at all.
* ``cut_to`` converts a folded length back to a raw index by bisecting
the recorded offsets, so one call returns the same cut point the raw
backward walk would have produced.
``str.isspace`` is used for folding instead of the ``\\s`` pattern used
by :func:`_compact_for_echo_compare`. The two agree on every Unicode code
point, so the indexed view and the regex-folded view stay consistent.
"""
__slots__ = ('_compact', '_offsets', '_raw_len')
def __init__(self) -> None:
self._compact: list[str] = []
self._offsets: list[int] = []
self._raw_len = 0
def append(self, text: str) -> None:
"""Fold ``text`` onto the end of the index (one pass, no rescan)."""
raw = str(text or '')
if not raw:
return
compact = self._compact
offsets = self._offsets
base = self._raw_len
for i, ch in enumerate(raw):
if not ch.isspace():
compact.append(ch)
offsets.append(base + i)
self._raw_len = base + len(raw)
def reset(self) -> None:
"""Drop all indexed state (used after the raw text is truncated)."""
self._compact.clear()
self._offsets.clear()
self._raw_len = 0
@property
def compact_length(self) -> int:
return len(self._compact)
def matches_tail(self, suffix: str) -> bool:
"""True when the folded ``suffix`` equals the folded tail."""
candidate = _compact_for_echo_compare(suffix)
if not candidate:
return False
n = len(candidate)
if n > len(self._compact):
return False
start = len(self._compact) - n
compact = self._compact
for i in range(n):
if compact[start + i] != candidate[i]:
return False
return True
def cut_to(self, suffix: str) -> int | None:
"""Raw index at which the echo of ``suffix`` starts, else ``None``.
The returned index is the first raw character of the echo, matching
the leftmost-cut semantics of the raw backward walk. Trailing
whitespace before the echo is left to the caller's ``rstrip``.
"""
candidate = _compact_for_echo_compare(suffix)
if not candidate:
return None
n = len(candidate)
if n > len(self._compact):
return None
start = len(self._compact) - n
compact = self._compact
for i in range(n):
if compact[start + i] != candidate[i]:
return None
return self._offsets[start]
def compact_view(self) -> str:
"""The folded text (materialized on demand — never kept as a string)."""
return ''.join(self._compact)
def _find_compact_echo_suffix_start(value: str, suffix: str) -> int | None:
"""Return the index where a whitespace-folded ``suffix`` starts at the
end of ``value``, or ``None`` when the tail does not echo it.
The match walks ``value`` and ``suffix`` from the end, skipping
whitespace in ``value``; only the echo span itself is inspected, so the
cost is linear in the echo length and allocation-free. Unlike a fixed
fold window, the walk cannot miss a compact-equivalent suffix whose raw
span is stretched by interior whitespace.
``str.isspace`` is used for the walk instead of the ``\\s`` pattern used
by :func:`_compact_for_echo_compare`. The two agree on every Unicode code
point, so the folded view and the walk stay consistent.
"""
candidate = _compact_for_echo_compare(suffix)
if not candidate:
return None
i = len(value) - 1
j = len(candidate) - 1
while j >= 0:
while i >= 0 and value[i].isspace():
i -= 1
if i < 0 or value[i] != candidate[j]:
return None
i -= 1
j -= 1
return i + 1
def _strip_compact_echo_suffix(value: str, suffix: str) -> tuple[str, bool]:
"""Remove ``suffix`` from ``value`` when they match after whitespace folding.
The search window is folded once and the cut point is then located by
walking backwards across the echo itself. The previous implementation
probed every candidate cut index and re-folded the whole remaining tail for
each probe, which is quadratic in the window size: a 6000-character final
message cost seconds of CPU, held under the GIL, stalling every other
stream in the process.
The cut point is located by the same backward walk as
:func:`_find_compact_echo_suffix_start`: no fixed search window is
involved, so a compact-equivalent suffix is removed no matter how much
interior whitespace stretches its raw span. The previous windowed
implementation folded a bounded tail on every call; its retired probing
variant re-folded the remaining tail per candidate cut index, which is
quadratic: a 6000-character final message cost seconds of CPU, held under
the GIL, stalling every other stream in the process.
``str.isspace`` is used for the backwards walk instead of the ``\\s``
pattern used by :func:`_compact_for_echo_compare`. The two agree on every
Unicode code point, so the folded view and the walk stay consistent.
Whitespace sitting between the kept text and the echo is removed by
``rstrip``, which lands on the same result as the leftmost cut index the
probing loop used to return.
"""
raw = str(value or '')
candidate = _compact_for_echo_compare(suffix)
if not raw or not candidate:
if not raw:
return raw, False
tail = raw[-max(len(str(suffix or '')) * 3, search_window):]
offset = len(raw) - len(tail)
compact_tail = _compact_for_echo_compare(tail)
if len(candidate) > len(compact_tail) or not compact_tail.endswith(candidate):
start = _find_compact_echo_suffix_start(raw, suffix)
if start is None:
return raw, False
# Consume exactly as many non-whitespace characters as the folded suffix
# holds; ``idx`` then sits on the first character of the echo. Whitespace
# sitting between the kept text and the echo is removed by ``rstrip``,
# which is why this lands on the same result as the leftmost cut index the
# probing loop used to return.
remaining = len(candidate)
idx = len(tail)
while remaining and idx:
idx -= 1
if not tail[idx].isspace():
remaining -= 1
return raw[: offset + idx].rstrip(), True
return raw[:start].rstrip(), True
def _redacted_session_payload_with_full_messages(session, *, tool_calls=None) -> dict | None:
@@ -779,6 +895,32 @@ def _is_fallback_lifecycle_message(kind: str, message: str) -> bool:
)
# Session turn-lease notices emitted by the Agent (agent/turn_facade_lease.py)
# while another Hermes process (gateway, CLI, cron) holds this session's turn
# lease. Emitted via ``_emit_status`` (kind ``lifecycle``) while waiting and on
# admission, and via ``_emit_warning`` (kind ``warn``) when the wait times out
# and the message was not processed.
_SESSION_LEASE_WAIT_MARKERS = (
'another hermes process is using this session',
'still waiting for the other hermes process',
'another hermes process kept this session busy',
'session is free; loading the latest transcript',
)
def _is_session_lease_wait_message(kind: str, message: str) -> bool:
"""Return True for Agent session turn-lease wait notices.
Classification keys on the Agent status kind (``lifecycle`` / ``warn``) so
user-authored text can never be promoted to a warning.
"""
k = str(kind or '').strip().lower()
if k not in ('lifecycle', 'warn'):
return False
m = str(message or '').strip().lower()
return any(marker in m for marker in _SESSION_LEASE_WAIT_MARKERS)
def _is_agent_compression_start_status(kind: str, message: str) -> bool:
"""Return True only for real Hermes context-compression start notices.
@@ -4378,9 +4520,10 @@ def _title_language_mismatch(user_text: str, title: str) -> bool:
short and frequently embed a borrowed Latin technical term (e.g. a CJK
title containing the word "Python"), the title side uses a proportion
threshold (>=35% of the title's alphabetic characters in a non-start
script, min 2 chars) rather than a strict majority -- so a CJK title with
one English word still trips, while an English title with a single
foreign place-name does not.
script, min 2 chars) rather than a strict majority. CJK titles with
borrowed Latin terms are allowed when the title also contains CJK
characters (#7693), but pure-Latin titles for CJK conversations are
still rejected.
2. The legacy German-start → English-title heuristic, preserved verbatim so
the original behavior keeps working for same-script (latin) drift that
the script check can't see.
@@ -4390,13 +4533,26 @@ def _title_language_mismatch(user_text: str, title: str) -> bool:
return False
# (1) Cross-script mismatch — language-agnostic.
# CJK text routinely borrows Latin product/technical terms (e.g. "WeChat
# Pay", "Python", "ProRes RAW"), so when the user writes in CJK, Latin
# characters in the title are acceptable as long as the title also
# contains CJK — i.e. the title is genuinely mixed, not pure drift.
# A pure-Latin title for a CJK conversation is still rejected.
# Unrelated scripts (Cyrillic, Arabic, Greek …) are always flagged.
user_script = _dominant_script(user_text)
if user_script:
title_counts = _script_counts(candidate)
title_total = sum(title_counts.values())
if title_total >= 2:
for script, n in title_counts.items():
if script != user_script and n >= 2 and (n / title_total) >= 0.35:
if script == user_script:
continue
# When user writes in CJK, Latin in the title is a borrowed
# term as long as the title also contains CJK characters.
if user_script == 'cjk' and script == 'latin':
if title_counts.get('cjk', 0) >= 2:
continue
if n >= 2 and (n / title_total) >= 0.35:
return True
# (2) Legacy same-script German→English heuristic.
@@ -7949,7 +8105,7 @@ def _extract_tool_calls_from_messages(messages, live_tool_calls=None):
pending_names[tid] = part.get('name', '')
pending_args[tid] = part.get('input', {})
pending_asst_idx[tid] = msg_idx
for tc in m.get('tool_calls', []):
for tc in m.get('tool_calls') or []:
if not isinstance(tc, dict):
continue
tid = tc.get('id', '') or tc.get('call_id', '')
@@ -9169,7 +9325,32 @@ def _run_agent_streaming(
"""
_turn_route_model = model
_turn_route_provider = model_provider
cancel_event = threading.Event()
q = peek_stream(stream_id)
if q is not None:
# Snapshot lookup is not admission: Stop can detach the stream before
# we publish the initial run. Register ownership and its retained cancel
# signal on the same STREAMS_LOCK edge used by cancellation.
with STREAMS_LOCK:
cancel_event = CANCEL_FLAGS.get(stream_id, cancel_event)
if stream_id not in STREAMS or cancel_event.is_set():
q = None
else:
CANCEL_FLAGS[stream_id] = cancel_event
STREAM_PARTIAL_TEXT[stream_id] = ''
STREAM_REASONING_TEXT[stream_id] = ''
STREAM_LIVE_TOOL_CALLS[stream_id] = []
register_active_run(
stream_id,
session_id=session_id,
started_at=time.time(),
phase="starting",
workspace=str(workspace),
model=model,
provider=model_provider,
ephemeral=bool(ephemeral),
backend=WEBUI_LOCAL_CHAT_BACKEND,
)
if q is None:
# The stream was cancelled before the worker started; the route layer
# already registered the stream owner, so release it here to avoid
@@ -9183,16 +9364,6 @@ def _run_agent_streaming(
exc_info=True,
)
return
register_active_run(
stream_id,
session_id=session_id,
started_at=time.time(),
phase="starting",
workspace=str(workspace),
model=model,
provider=model_provider,
ephemeral=bool(ephemeral),
)
try:
run_journal = RunJournalWriter(session_id, stream_id)
except Exception:
@@ -9224,14 +9395,6 @@ def _run_agent_streaming(
# profile's mcp_servers because os.environ['HERMES_HOME'] hadn't been
# rewritten yet. See https://github.com/nesquena/hermes-webui/issues/1968.
# Sprint 10: create a cancel event for this stream
cancel_event = threading.Event()
with STREAMS_LOCK:
CANCEL_FLAGS[stream_id] = cancel_event
STREAM_PARTIAL_TEXT[stream_id] = '' # start accumulating partial text (#893)
STREAM_REASONING_TEXT[stream_id] = '' # start accumulating reasoning trace (#1361 §A)
STREAM_LIVE_TOOL_CALLS[stream_id] = [] # start accumulating tool calls (#1361 §B)
agent = None
_live_prompt_estimate_tokens = [0]
_live_prompt_exact_tokens = [0]
@@ -9533,8 +9696,42 @@ def _run_agent_streaming(
_metering_thread = threading.Thread(target=_metering_ticker, daemon=True)
_success_writeback_committed = False
_steer_settled = False
_returned_pending_steer = []
def _remember_pending_steer_result(value):
text = value.get('pending_steer') if isinstance(value, dict) else None
if isinstance(text, str) and text:
_returned_pending_steer.append(text)
def _settle_pending_steer():
# Single terminal boundary for success, returned errors, exceptions and
# successful self-heal. Admission closes before the last slot consumer;
# registry locks never cover SSE writes or Agent drain callbacks.
nonlocal _steer_settled
with STREAMS_LOCK:
if _steer_settled:
return
_steer_settled = True
target = AGENT_INSTANCES.get(stream_id) or agent
if stream_id in STREAMS and not cancel_event.is_set():
update_active_run(stream_id, phase="finalizing")
leftovers = list(_returned_pending_steer)
try:
drain = getattr(target, '_drain_pending_steer', None)
text = drain() if callable(drain) else None
if text:
leftovers.append(str(text))
except Exception:
logger.debug("Failed to drain pending steer for session %s", session_id)
if leftovers:
put('pending_steer_leftover', {
'session_id': session_id, 'text': '\n'.join(leftovers),
})
def put(event, data):
if event in ('done', 'apperror', 'stream_end'):
_settle_pending_steer()
# If cancelled, drop all further events except the cancel event itself
if cancel_event.is_set() and not _success_writeback_committed and event not in ('cancel', 'apperror'):
return
@@ -9558,7 +9755,7 @@ def _run_agent_streaming(
except Exception:
logger.debug("Failed to note event_id %s for stream %s", event_id, stream_id, exc_info=True)
try:
queue_item = (event, data, event_id) if event_id and hasattr(q, "subscribe_with_snapshot") else (event, data)
queue_item = (event, data, event_id) if hasattr(q, "subscribe_with_snapshot") else (event, data)
q.put_nowait(queue_item)
except Exception:
logger.debug("Failed to put event to queue")
@@ -9605,6 +9802,11 @@ def _run_agent_streaming(
return
# Pass through rate-limit and fallback messages so the frontend can
# show them as warnings via the existing messages.js 'warning' listener.
# Session turn-lease waits (another Hermes process owns this session)
# use the same channel so a delayed turn explains itself.
if _is_session_lease_wait_message(_kind, _message):
put('warning', {'type': 'session_lease_wait', 'message': _message})
return
_is_fallback_notice = _is_fallback_lifecycle_message(_kind, _message)
if _is_fallback_notice:
put('warning', {'type': 'fallback', 'message': _message})
@@ -9620,6 +9822,84 @@ def _run_agent_streaming(
_streaming_skill_home_snapshot = None
_restore_streaming_skill_home_modules = False
_acquired_streaming_skill_home_patch_lock = False
def _register_agent_if_current(candidate, cache_signature=None):
nonlocal agent
# Every constructor, including both credential self-heal branches, must
# share the Stop publication edge. CANCEL_FLAGS may already be detached;
# the worker-retained event and stream membership remain authoritative.
with STREAMS_LOCK:
cancelled = cancel_event.is_set() or stream_id not in STREAMS
if not cancelled:
AGENT_INSTANCES[stream_id] = candidate
# Check and publication share one admission, not a check followed
# by an unlocked cache/lifecycle write that could undo Stop.
if cache_signature is not None:
from api.config import SESSION_AGENT_CACHE, SESSION_AGENT_CACHE_LOCK
with SESSION_AGENT_CACHE_LOCK:
SESSION_AGENT_CACHE[session_id] = (candidate, cache_signature)
SESSION_AGENT_CACHE.move_to_end(session_id)
if not ephemeral:
try:
# Pure in-memory bookkeeping; no Agent/provider call.
from api.session_lifecycle import register_agent
register_agent(session_id, candidate)
except Exception:
logger.debug("Lifecycle register_agent failed for session %s", session_id, exc_info=True)
if not cancelled:
return True
# A cache hit is a borrowed reusable object, not this worker's private
# candidate. Stop may already have let a successor reuse it. Rejected
# cache-hit admission must not interrupt that other turn.
if cache_signature is None and not ephemeral:
if agent is candidate:
agent = None
return False
# Newly constructed candidates were never published on this failed
# admission; they cannot have been borrowed by a successor from us.
# No Agent call, session persistence or SSE write under registry locks.
try:
candidate.interrupt("Cancelled before start")
except Exception:
logger.debug("Failed to interrupt cancelled candidate agent")
return False
def _agent_can_invoke(candidate):
nonlocal agent
# Prompt preparation and LRU cleanup can yield after registration. Admit
# each invocation again at its point of use; never hold registry locks
# across run_conversation or interrupt. A later Stop interrupts the
# already-admitted invocation through the registered Agent as usual.
from api.config import (
SESSION_AGENT_CACHE, SESSION_AGENT_CACHE_LOCK,
SESSION_WRITEBACK_OWNERS, SESSION_WRITEBACK_OWNERS_LOCK,
)
with STREAMS_LOCK:
current = (not cancel_event.is_set() and stream_id in STREAMS
and AGENT_INSTANCES.get(stream_id) is candidate)
if not current:
if AGENT_INSTANCES.get(stream_id) is candidate:
AGENT_INSTANCES.pop(stream_id, None)
# Agent identity is not turn identity: a successor can reuse the
# same cached object. Use the existing cancellation-surviving
# ownership record, holding its lock through cache retirement.
# Missing ownership also fails closed (successor may have ended).
with SESSION_WRITEBACK_OWNERS_LOCK:
if SESSION_WRITEBACK_OWNERS.get(session_id) == stream_id:
with SESSION_AGENT_CACHE_LOCK:
entry = SESSION_AGENT_CACHE.get(session_id)
if entry and entry[0] is candidate:
SESSION_AGENT_CACHE.pop(session_id, None)
# Dirty memory segments keep their original owner.
from api.session_lifecycle import unregister_agent
unregister_agent(session_id)
# This invocation never started. Stop owns any required interrupt of the
# running Agent; a late duplicate interrupt could hit its new borrower.
# Drop our local borrowed handle too: final Steer drain must not reach
# the successor through the old worker's fallback `agent` reference.
if not current and agent is candidate:
agent = None
return current
# Initialised here (before any code that may raise) so the outer `finally`
# block can safely check `if _checkpoint_stop is not None` even when an
# exception fires before the checkpoint thread is created (Issue #765).
@@ -9964,6 +10244,14 @@ def _run_agent_streaming(
# (#3587) replaces the flat _reasoning_text string so each intermediate
# assistant turn (before tool calls) keeps its own reasoning segment.
_reasoning_segments: dict = {}
# Incremental folded indexes mirroring each reasoning buffer. The
# live echo strip consults them instead of re-walking the raw
# buffers: an unbounded raw scan of the growing transcript is
# quadratic on whitespace-heavy output (#7569 review). Each index
# is fed on every append and re-indexed after a strip.
_reasoning_segment_indexes: dict = {}
_stream_reasoning_index = _CompactEchoIndex()
_reasoning_buffer_index = _CompactEchoIndex()
_current_reasoning_idx = 0
_tool_boundary_advanced = False
_live_tool_calls = [] # tool progress fallback when final messages omit tool IDs
@@ -9986,6 +10274,10 @@ def _run_agent_streaming(
if _reasoning_buffer[0]:
put('reasoning', {'text': _reasoning_buffer[0]})
_reasoning_buffer[0] = ''
# The folded index mirrors this buffer 1:1 — dropping the
# text without dropping the index would leave it describing
# text that is no longer there.
_reasoning_buffer_index.reset()
def _emit_metering():
@@ -10025,30 +10317,37 @@ def _run_agent_streaming(
nonlocal _reasoning_segments
removed = False
if stream_id in STREAM_REASONING_TEXT:
next_text, did_remove = _strip_compact_echo_suffix(
STREAM_REASONING_TEXT.get(stream_id, ''),
text,
)
if did_remove:
cut = _stream_reasoning_index.cut_to(text)
if cut is not None:
next_text = STREAM_REASONING_TEXT.get(stream_id, '')[:cut].rstrip()
STREAM_REASONING_TEXT[stream_id] = next_text
_stream_reasoning_index.reset()
_stream_reasoning_index.append(next_text)
removed = True
next_buffer, did_remove_buffer = _strip_compact_echo_suffix(_reasoning_buffer[0], text)
if did_remove_buffer:
cut = _reasoning_buffer_index.cut_to(text)
if cut is not None:
next_buffer = _reasoning_buffer[0][:cut].rstrip()
_reasoning_buffer[0] = next_buffer
_reasoning_buffer_index.reset()
_reasoning_buffer_index.append(next_buffer)
removed = True
for idx in (_current_reasoning_idx, _current_reasoning_idx - 1):
if idx not in _reasoning_segments:
continue
next_segment, did_remove_segment = _strip_compact_echo_suffix(
_reasoning_segments.get(idx, ''),
text,
)
if not did_remove_segment:
segment_index = _reasoning_segment_indexes.get(idx)
if segment_index is None:
continue
cut = segment_index.cut_to(text)
if cut is None:
continue
next_segment = _reasoning_segments.get(idx, '')[:cut].rstrip()
if next_segment:
_reasoning_segments[idx] = next_segment
segment_index.reset()
segment_index.append(next_segment)
else:
_reasoning_segments.pop(idx, None)
_reasoning_segment_indexes.pop(idx, None)
removed = True
break
return removed
@@ -10112,15 +10411,21 @@ def _run_agent_streaming(
_reasoning_segments[_current_reasoning_idx] = (
_reasoning_segments.get(_current_reasoning_idx, '') + reasoning_delta
)
# Keep the folded index in step with the segment text.
_reasoning_segment_indexes.setdefault(
_current_reasoning_idx, _CompactEchoIndex()
).append(reasoning_delta)
# Mirror full concatenation to shared dict so cancel_stream() can persist
# it (#1361 §A). Cancel only creates one partial message, so the flat
# concatenation is correct there.
# Lock-free GIL-atomic mirror — see the STREAMS_LOCK contract in on_token.
if stream_id in STREAM_REASONING_TEXT:
STREAM_REASONING_TEXT[stream_id] += reasoning_delta
_stream_reasoning_index.append(reasoning_delta)
# Accumulate into a coalescing buffer so every delta reaches the
# browser — reasoning deltas are incremental, not idempotent.
_reasoning_buffer[0] += reasoning_delta
_reasoning_buffer_index.append(reasoning_delta)
# Throttle reasoning SSE events to ~10 Hz to avoid overwhelming the
# frontend renderer. Each event triggers _parseStreamState() which
# scans the full accumulated text — 10k+ reasoning tokens/second
@@ -10131,6 +10436,10 @@ def _run_agent_streaming(
_reasoning_last_put[0] = now
put('reasoning', {'text': _reasoning_buffer[0]})
_reasoning_buffer[0] = ''
# The folded index mirrors this buffer 1:1 — dropping the
# text without dropping the index would leave it describing
# text that is no longer there.
_reasoning_buffer_index.reset()
# Track reasoning deltas in the meter so live TPS reflects all AI output.
_metering_reasoning_deltas[0] += 1
meter().record_reasoning(stream_id, _metering_reasoning_deltas[0])
@@ -10239,10 +10548,14 @@ def _run_agent_streaming(
_reasoning_segments[_current_reasoning_idx] = (
_reasoning_segments.get(_current_reasoning_idx, '') + reason_delta
)
_reasoning_segment_indexes.setdefault(
_current_reasoning_idx, _CompactEchoIndex()
).append(reason_delta)
# Mirror full concatenation to shared dict (#1361 §A)
# Lock-free GIL-atomic mirror — see STREAMS_LOCK contract in on_token.
if stream_id in STREAM_REASONING_TEXT:
STREAM_REASONING_TEXT[stream_id] += reason_delta
_stream_reasoning_index.append(reason_delta)
put('reasoning', {'text': reason_delta})
_metering_reasoning_deltas[0] += 1
meter().record_reasoning(stream_id, _metering_reasoning_deltas[0])
@@ -10809,6 +11122,7 @@ def _run_agent_streaming(
# ── Agent cache: reuse across messages in the same session ──
# Mirrors gateway _agent_cache. Keeps _user_turn_count alive so
# injectionFrequency: "first-turn" actually suppresses after turn 1.
_cache_new_agent = False
if ephemeral:
agent = _AIAgent(**_agent_kwargs)
logger.debug('[webui] Created ephemeral agent for session %s', session_id)
@@ -10851,14 +11165,6 @@ def _run_agent_streaming(
session_id,
_cached_agent_session_identity(_cached_agent),
)
if agent is not None:
# Reopened/cache-hit sessions must register the agent
# so later lifecycle commits can find it.
try:
from api.session_lifecycle import register_agent
register_agent(session_id, agent)
except Exception:
logger.debug("Lifecycle register_agent failed for cached session %s", session_id, exc_info=True)
if _identity_mismatch_entry is not None:
try:
@@ -10928,13 +11234,17 @@ def _run_agent_streaming(
agent._interrupt_message = None
else:
agent = _AIAgent(**_agent_kwargs)
# Register the new agent with the memory lifecycle so
# its commit_memory_session() can be found later.
try:
from api.session_lifecycle import register_agent
register_agent(session_id, agent)
except Exception:
logger.debug("Lifecycle register_agent failed for new session %s", session_id, exc_info=True)
_cache_new_agent = True
if not _register_agent_if_current(agent, _agent_sig if _cache_new_agent else None):
with _agent_lock:
_finalize_cancelled_turn(s, ephemeral=ephemeral, message='Task cancelled before start.', stream_id=stream_id)
put('cancel', _cancel_event_payload('Cancelled by user'))
return
# Cache publication is already fenced with Stop above. Keep LRU
# eviction and any memory-provider close outside the stream lock.
if _cache_new_agent:
_evicted_items = []
# Snapshot the set of session_ids with a LIVE agent worker
# BEFORE taking SESSION_AGENT_CACHE_LOCK, so LRU eviction never
@@ -10955,8 +11265,6 @@ def _run_agent_streaming(
except Exception:
_active_sids = set()
with SESSION_AGENT_CACHE_LOCK:
SESSION_AGENT_CACHE[session_id] = (agent, _agent_sig)
SESSION_AGENT_CACHE.move_to_end(session_id) # LRU: mark as recently used
from api.config import SESSION_AGENT_CACHE_MAX
# Evict the oldest INACTIVE entries first. Walk LRU order
# (front = oldest); skip any session with a live run. If
@@ -10985,21 +11293,6 @@ def _run_agent_streaming(
logger.debug('[webui] Evicted LRU agent from cache: %s', _evicted_sid)
logger.debug('[webui] Created new agent for session %s', session_id)
# Store agent instance for cancel/interrupt propagation
with STREAMS_LOCK:
AGENT_INSTANCES[stream_id] = agent
# Check if cancel was requested during agent initialization
if stream_id in CANCEL_FLAGS and CANCEL_FLAGS[stream_id].is_set():
# Cancel arrived during agent creation - interrupt immediately
try:
agent.interrupt("Cancelled before start")
except Exception:
logger.debug("Failed to interrupt agent before start")
with _agent_lock:
_finalize_cancelled_turn(s, ephemeral=ephemeral, message='Task cancelled before start.', stream_id=stream_id)
put('cancel', _cancel_event_payload('Cancelled by user'))
return
# Prepend workspace context so the agent always knows which directory
# to use for file operations, regardless of session age or AGENTS.md defaults.
workspace_ctx = _workspace_context_prefix(str(s.workspace))
@@ -11259,7 +11552,13 @@ def _run_agent_streaming(
)
_run_conversation_kwargs["user_message"] = user_message
_result_partial_pre_call_context = list(_previous_context_messages)
if not _agent_can_invoke(agent):
with _agent_lock:
_finalize_cancelled_turn(s, ephemeral=ephemeral, message='Task cancelled before start.', stream_id=stream_id)
put('cancel', _cancel_event_payload('Cancelled by user'))
return
result = agent.run_conversation(**_run_conversation_kwargs)
_remember_pending_steer_result(result)
_active_turn_identity = _resolve_active_turn_authority(
_active_turn_identity,
result=result,
@@ -11740,8 +12039,6 @@ def _run_agent_streaming(
if 'credential_pool' in _agent_params:
_agent_kwargs['credential_pool'] = _runtime_bundle['credential_pool']
agent = _AIAgent(**_agent_kwargs)
with STREAMS_LOCK:
AGENT_INSTANCES[stream_id] = agent
_agent_sig = _compute_agent_cache_signature(
resolved_model,
resolved_api_key,
@@ -11758,10 +12055,11 @@ def _run_agent_streaming(
profile_home=_profile_home,
safe_profile_runtime_env=_safe_profile_runtime_env,
)
from api.config import SESSION_AGENT_CACHE as _SAC, SESSION_AGENT_CACHE_LOCK as _SAC_L
with _SAC_L:
_SAC[session_id] = (agent, _agent_sig)
_SAC.move_to_end(session_id)
if not _register_agent_if_current(agent, _agent_sig):
# Returned-error settlement already owns _agent_lock.
_finalize_cancelled_turn(s, ephemeral=ephemeral, message='Task cancelled before start.', stream_id=stream_id)
put('cancel', _cancel_event_payload('Cancelled by user'))
return
# Retry the conversation once with fresh credentials
_self_healed = True
_token_sent = False
@@ -11794,7 +12092,13 @@ def _run_agent_streaming(
_result_partial_pre_call_context = list(
_heal_context_messages
)
if not _agent_can_invoke(agent):
# Returned-error settlement already owns the lock.
_finalize_cancelled_turn(s, ephemeral=ephemeral, message='Task cancelled before start.', stream_id=stream_id)
put('cancel', _cancel_event_payload('Cancelled by user'))
return
_heal_result = agent.run_conversation(**_heal_kwargs)
_remember_pending_steer_result(_heal_result)
_active_turn_identity = _resolve_active_turn_authority(
_active_turn_identity,
result=_heal_result,
@@ -12741,23 +13045,7 @@ def _run_agent_streaming(
else None
)
# (reasoning trace already attached + saved above, before s.save())
# Leftover-steer delivery: if a /steer was queued (via
# api/chat/steer) but the agent finished its turn before
# reaching a tool-result boundary that would consume it,
# the text is still stashed in agent._pending_steer. Drain
# it now and emit a pending_steer_leftover SSE event so the
# frontend can queue it for the next turn — same fallback
# path as the CLI in cli.py:8788-8794.
try:
_drain_pending_steer = getattr(agent, '_drain_pending_steer', None)
_leftover = _drain_pending_steer() if _drain_pending_steer else None
if _leftover:
put('pending_steer_leftover', {
'session_id': session_id,
'text': str(_leftover),
})
except Exception:
logger.debug("Failed to drain pending steer for session %s", session_id)
_settle_pending_steer()
# /goal parity: after a successful assistant turn, run the Hermes
# GoalManager judge before terminal done/stream_end events. The
# frontend surfaces the status line and queues continuation_prompt as
@@ -13107,8 +13395,6 @@ def _run_agent_streaming(
if 'credential_pool' in _agent_params:
_heal_kwargs['credential_pool'] = _runtime_bundle['credential_pool']
_heal_agent = _AIAgent(**_heal_kwargs)
with STREAMS_LOCK:
AGENT_INSTANCES[stream_id] = _heal_agent
_agent_sig = _compute_agent_cache_signature(
resolved_model,
resolved_api_key,
@@ -13125,10 +13411,11 @@ def _run_agent_streaming(
profile_home=_profile_home,
safe_profile_runtime_env=_safe_profile_runtime_env,
)
from api.config import SESSION_AGENT_CACHE as _SAC2, SESSION_AGENT_CACHE_LOCK as _SAC2_L
with _SAC2_L:
_SAC2[session_id] = (_heal_agent, _agent_sig)
_SAC2.move_to_end(session_id)
if not _register_agent_if_current(_heal_agent, _agent_sig):
with _agent_lock:
_finalize_cancelled_turn(s, ephemeral=ephemeral, message='Task cancelled before start.', stream_id=stream_id)
put('cancel', _cancel_event_payload('Cancelled by user'))
return
# Retry the conversation
_token_sent = False
try:
@@ -13160,7 +13447,13 @@ def _run_agent_streaming(
_result_partial_pre_call_context = list(
_heal_context_messages
)
if not _agent_can_invoke(_heal_agent):
with _agent_lock:
_finalize_cancelled_turn(s, ephemeral=ephemeral, message='Task cancelled before start.', stream_id=stream_id)
put('cancel', _cancel_event_payload('Cancelled by user'))
return
_heal_result = _heal_agent.run_conversation(**_heal_kwargs2)
_remember_pending_steer_result(_heal_result)
_active_turn_identity = _resolve_active_turn_authority(
_active_turn_identity,
result=_heal_result,
@@ -13414,6 +13707,7 @@ def _run_agent_streaming(
_error_payload['old_session_id'] = session_id
put('apperror', _error_payload)
finally:
_settle_pending_steer()
# #4633/#2476: symmetric metering teardown. begin_session() (top of the
# outer try) had no paired end_session(), so zero-token turns leaked a
# _sessions[stream_id] entry that get_stats() pruning never reclaims (its
@@ -13441,7 +13735,6 @@ def _run_agent_streaming(
if (s is not None
and getattr(s, 'active_stream_id', None) == stream_id
and getattr(s, 'pending_user_message', None)):
update_active_run(stream_id, phase="finalizing")
_last_resort_sync_from_core(s, stream_id, _agent_lock)
_clear_thread_env() # TD1: always clear thread-local context
if _streaming_cron_profile_home_token is not None:
@@ -13536,13 +13829,68 @@ def _run_agent_streaming(
# ============================================================
# Local lifecycle phases with a pending-steer consumer still ahead of them.
# Gateway uses its own phase vocabulary and never falls through to local delivery.
_LOCAL_STEERABLE_PHASES = frozenset({"starting", "running"})
def _steer_bound_stream(sid: str, stream_id: str, text: str) -> dict | None:
"""Deliver to a verified live worker; None retains cache-only compatibility.
Never perform HTTP writes or cache/database teardown under stream locks.
"""
from api import config as cfg
with cfg.STREAMS_LOCK:
agent = cfg.AGENT_INSTANCES.get(stream_id)
owner = cfg.stream_owner_session_id(stream_id)
with cfg.ACTIVE_RUNS_LOCK:
run = dict(cfg.ACTIVE_RUNS.get(stream_id) or {})
# Gateway owns transport even when no in-process worker is registered.
# A reusable local cache entry must never override that authority.
if run.get("backend") == "gateway":
if (stream_id not in cfg.STREAMS
or owner != sid or run.get("session_id") != sid
or run.get("phase") == "cancelling"):
return {"accepted": False, "fallback": "stream_dead", "stream_id": None}
return {"accepted": False, "fallback": "gateway_steer_queued", "stream_id": stream_id}
if (stream_id in cfg.STREAMS and owner == sid
and run.get("session_id") == sid
and run.get("backend") == WEBUI_LOCAL_CHAT_BACKEND
and run.get("phase") == "finalizing"):
return {"accepted": False, "fallback": "not_running", "stream_id": stream_id}
if agent is None:
if ((owner and owner != sid)
or (run.get("session_id") and run["session_id"] != sid)
or (run and run.get("phase") not in _LOCAL_STEERABLE_PHASES)):
return {"accepted": False, "fallback": "stream_dead", "stream_id": None}
return None
# A cache hit cannot override missing or conflicting worker ownership.
if (stream_id not in cfg.STREAMS
or owner != sid or run.get("session_id") != sid
or run.get("phase") not in _LOCAL_STEERABLE_PHASES
or run.get("backend") == "gateway"):
return {"accepted": False, "fallback": "stream_dead", "stream_id": None}
if not callable(getattr(agent, "steer", None)):
return {"accepted": False, "fallback": "agent_lacks_steer", "stream_id": None}
# steer() only stashes input; serializing it with stream teardown keeps
# cancel from detaching the selected worker before delivery.
try:
accepted = bool(agent.steer(text))
except Exception:
logger.debug("Stream-bound steer failed for session %s", sid, exc_info=True)
return {"accepted": False, "fallback": "steer_error", "stream_id": stream_id}
return {"accepted": accepted, "fallback": None, "stream_id": stream_id}
def _handle_chat_steer(handler, body: dict) -> bool:
"""Inject a /steer payload into the active agent for a session.
Mirrors the CLI's `/steer <text>` command (cli.py:6140-6155):
- Look up the cached AIAgent for the session (PR #1051's
SESSION_AGENT_CACHE).
- Prefer the active stream's registered AIAgent, with explicit stream
and worker ownership; use the cache only for legacy cache-only runs.
- Verify a stream is currently active for this session.
- Never evict or close an agent while delivering steering input.
- Call agent.steer(text) — thread-safe, stashes text in
_pending_steer for application at the next tool-result boundary.
@@ -13571,24 +13919,23 @@ def _handle_chat_steer(handler, body: dict) -> bool:
if not text:
return bad(handler, "text required")
evicted_cached_entry = None
# A compression rotates agent.session_id without replacing the running
# worker. Resolve the stream-bound instance BEFORE consulting the reusable
# cache. Steer never owns cache eviction or agent/database teardown.
try:
session = get_session(sid)
except KeyError:
session = None
stream_id = getattr(session, "active_stream_id", None) or None
if stream_id:
result = _steer_bound_stream(sid, stream_id, text)
if result is not None:
return j(handler, result)
with _cfg.SESSION_AGENT_CACHE_LOCK:
cached = _cfg.SESSION_AGENT_CACHE.get(sid)
if cached:
agent = cached[0]
if not _cached_agent_matches_session(agent, sid):
evicted_cached_entry = _cfg.SESSION_AGENT_CACHE.pop(sid, None)
logger.warning(
'[webui] Evicted cached agent before steer due to mismatched session identity: cache_key=%s agent_session_id=%s',
sid,
_cached_agent_session_identity(agent),
)
if cached and not _cached_agent_matches_session(cached[0], sid):
cached = None
if evicted_cached_entry is not None:
try:
_close_cached_agent_entry_at_session_boundary(sid, evicted_cached_entry)
except Exception:
logger.debug("Failed to close steer identity-mismatched cached agent for session %s", sid, exc_info=True)
if not cached:
try:
s = get_session(sid)
@@ -13634,22 +13981,54 @@ def _handle_chat_steer(handler, body: dict) -> bool:
if not active_stream_id:
return j(handler, {"accepted": False, "fallback": "not_running",
"stream_id": None})
with _cfg.STREAMS_LOCK:
stream_alive = active_stream_id in _cfg.STREAMS
if not stream_alive:
# Active stream id is stale — stream has ended; caller falls back
return j(handler, {"accepted": False, "fallback": "stream_dead",
"stream_id": None})
# Cache-only compatibility: no registered worker for this stream. The
# liveness/ownership revalidation and the enqueue must share the same
# stream-ownership edge as Stop (STREAMS_LOCK -> ACTIVE_RUNS_LOCK). A Stop
# that claims cancellation between an unlocked check and agent.steer()
# would strand guidance this response still reports as accepted. steer()
# only stashes input; interrupt, persistence, and HTTP writes stay outside
# the lock, and the cached agent is never evicted or closed here.
#
# Ownership must be proven positively, not merely unrefuted: BOTH the
# stream owner AND the active-run session must equal the requesting
# session. Missing metadata is ambiguous and fails closed (stream_dead);
# an unfenced cache object is never steered on absent ownership.
#
# The active-run backend is revalidated the same way. With no registered
# worker, the backend tag is the only proof that an in-process runtime
# owns this run: Gateway resolves to its own outcome, the explicit local
# tag may enqueue, and a missing, empty, or foreign backend fails closed.
result = {"accepted": False, "fallback": "stream_dead", "stream_id": None}
with _cfg.STREAMS_LOCK:
if active_stream_id in _cfg.STREAMS:
owner = _cfg.stream_owner_session_id(active_stream_id)
with _cfg.ACTIVE_RUNS_LOCK:
run = dict((_cfg.ACTIVE_RUNS or {}).get(str(active_stream_id)) or {})
owned_stream = bool(owner) and owner == sid
owned_run = bool(run.get("session_id")) and run["session_id"] == sid
if owned_stream and owned_run and run.get("phase") != "cancelling":
backend = run.get("backend")
if backend == "gateway":
# Gateway owns transport; a local cache object is never steered.
result = {"accepted": False, "fallback": "gateway_steer_queued",
"stream_id": active_stream_id}
elif backend == WEBUI_LOCAL_CHAT_BACKEND and run.get("phase") == "finalizing":
result = {"accepted": False, "fallback": "not_running",
"stream_id": active_stream_id}
elif (backend == WEBUI_LOCAL_CHAT_BACKEND
and run.get("phase") in _LOCAL_STEERABLE_PHASES):
try:
accepted = bool(agent.steer(text))
except Exception as exc:
logger.debug("agent.steer() raised for session=%s: %s", sid, exc)
return j(handler, {"accepted": False, "fallback": "steer_error",
"stream_id": active_stream_id})
result = {"accepted": False, "fallback": "steer_error",
"stream_id": active_stream_id}
else:
result = {"accepted": accepted, "fallback": None,
"stream_id": active_stream_id}
return j(handler, {"accepted": accepted, "fallback": None,
"stream_id": active_stream_id})
return j(handler, result)
def cancel_stream(stream_id: str) -> bool:
@@ -13762,6 +14141,14 @@ def cancel_stream(stream_id: str) -> bool:
# worker is stuck in C-level I/O and never reaches its finally (#6623).
update_active_run(stream_id, phase="cancelling", cancelled_at=time.time())
# Stop and Steer share STREAMS_LOCK -> ACTIVE_RUNS_LOCK ordering.
# Publish cancellation and detach ownership before releasing the edge;
# later Steer cannot enqueue into a turn already claimed by Stop.
if stream_present:
streams.pop(stream_id, None)
cancel_flags.pop(stream_id, None)
agent_instances.pop(stream_id, None)
# Set WebUI layer cancel flag. Prefer the snapshot captured under the lock;
# fall back to a fresh lookup for the ACTIVE_RUNS-only path (stream absent).
flag = _snap_flag if _snap_flag is not None else cancel_flags.get(stream_id)
@@ -13813,16 +14200,7 @@ def cancel_stream(stream_id: str) -> bool:
# worker save and show cancel in the client while persistence says done.
_emit_cancel_event = True
# ── Eager session lock release (fixes #653) ──────────────────────────
# Pop stream state now so the 409 guard in routes.py sees the session
# as idle and allows new /api/chat/start immediately after cancel,
# even if the agent thread is still blocked in a C-level syscall.
# The worker thread's finally block uses .pop(key, None) too, so a
# double-pop here is safe (no-op).
if stream_present:
streams.pop(stream_id, None)
cancel_flags.pop(stream_id, None)
agent_instances.pop(stream_id, None)
# Stream ownership was detached under streams_lock before interrupting.
# STREAM_PARTIAL_TEXT is intentionally NOT popped here — the agent thread may
# still be appending tokens, and the streaming finally block handles cleanup
# when the thread exits. We already snapshotted the buffers under streams_lock
api/upload.py
+105 -22
View File
@@ -8,7 +8,12 @@ import tempfile
from pathlib import Path
from api.config import MAX_UPLOAD_BYTES, STATE_DIR
from api.helpers import j
from api.helpers import (
arm_connection_close_if_body_pending,
j,
unreadable_content_length,
unsupported_transfer_encoding,
)
from api.models import get_session
from api.profiles import _profiles_match, get_active_profile_name as _get_active_profile_name
from api.workspace import (
@@ -45,6 +50,89 @@ def _max_extracted_bytes() -> int:
_MAX_EXTRACTED_BYTES = 10 * MAX_UPLOAD_BYTES
def _reject_before_read(handler, message, status):
"""Reject without reading the body; close so unread bytes can't poison HTTP/1.1 reuse.
Framing decides, exactly as at the other reject-before-read sites: every
rejection raised by the preflight above (any ``Transfer-Encoding``, an
unreadable or conflicting ``Content-Length``, an over-cap length) already
declares a body and still closes. Only the boundary/`parse_multipart` rejection
can fire on a request that declares none, and closing there dropped a healthy
pooled connection -- on the wire, ``POST /api/upload`` with
``Content-Type: application/json`` and no ``Content-Length`` answered 400
"No boundary in Content-Type" WITH ``Connection: close`` and the pipelined
``GET /api/health/agent`` was never served.
"""
arm_connection_close_if_body_pending(handler)
return j(handler, {'error': message}, status=status)
class _RejectBeforeRead(ValueError):
"""A rejection raised before any byte of the request body was consumed.
Carries the status the caller must answer with. Subclasses ``ValueError`` so
that if one ever escapes an unusual call path, each handler's existing
``except ValueError`` still answers 400 with the same message instead of
turning it into a 500.
"""
def __init__(self, message: str, status: int):
super().__init__(message)
self.message = message
self.status = status
def _read_multipart_or_reject(handler):
"""Framing preflight + body read, shared by all four multipart handlers.
Returns ``(fields, files)``. Raises ``_RejectBeforeRead`` when the request
must be refused before its body is read -- the caller answers through
``_reject_before_read()``, which arms ``Connection: close`` first, because the
declared body is still queued in ``rfile`` and a reused HTTP/1.1 connection
would parse those bytes as the next request line.
Framing is judged from the headers, not from a present ``Content-Length``:
* ANY ``Transfer-Encoding`` is refused, ``identity`` included: ``http.server``
decodes none of them (``rfile`` is the raw socket) and RFC 9112 section 6.3
says a request whose final coding is not ``chunked`` has a length no
recipient can determine. A coded request carries no usable Content-Length,
so the old code read it as length 0, found no parts, and answered "No file
field in request" with keep-alive intact -- leaving every payload byte on
the socket to poison the next pooled request. A Content-Length sent BESIDE a
coding does not rescue it: that pairing is the smuggling shape 6.3 rejects,
and the coding still hides how the body is framed.
* a non-numeric or oversized Content-Length cannot be read either, and
``parse_multipart()`` validates the boundary and the length before its
first ``rfile.read()`` — so every raise below is still pre-read.
* two DISAGREEING Content-Length headers read back as whichever came first,
so ``Content-Length: 0`` ahead of the real length made the body invisible
and left every byte of it on the socket. There is no honest length to
drain, so the request is refused before ``rfile`` is touched.
"""
content_type = handler.headers.get('Content-Type', '')
coding = unsupported_transfer_encoding(handler)
if coding is not None:
raise _RejectBeforeRead(
f'Unsupported Transfer-Encoding: {coding} '
'(uploads must send an undecoded body with a Content-Length)',
411,
)
if unreadable_content_length(handler):
raise _RejectBeforeRead('Invalid Content-Length', 400)
try:
content_length = int(handler.headers.get('Content-Length', 0) or 0)
except (TypeError, ValueError):
raise _RejectBeforeRead('Invalid Content-Length', 400) from None
if content_length > MAX_UPLOAD_BYTES:
raise _RejectBeforeRead(
f'File too large (max {MAX_UPLOAD_BYTES//1024//1024}MB)', 413)
try:
return parse_multipart(handler.rfile, content_type, content_length)
except ValueError as exc:
raise _RejectBeforeRead(str(exc), 400) from None
def parse_multipart(rfile, content_type, content_length) -> tuple:
import re as _re, email.parser as _ep
# Imported locally (not just module-level) so the function stays
@@ -206,11 +294,10 @@ def _write_office_upload_sidecar(workspace: Path, dest: Path, file_bytes: bytes)
def handle_upload(handler):
import traceback as _tb
try:
content_type = handler.headers.get('Content-Type', '')
content_length = int(handler.headers.get('Content-Length', 0) or 0)
if content_length > MAX_UPLOAD_BYTES:
return j(handler, {'error': f'File too large (max {MAX_UPLOAD_BYTES//1024//1024}MB)'}, status=413)
fields, files = parse_multipart(handler.rfile, content_type, content_length)
try:
fields, files = _read_multipart_or_reject(handler)
except _RejectBeforeRead as rejection:
return _reject_before_read(handler, rejection.message, rejection.status)
session_id = fields.get('session_id', '')
if 'file' not in files:
return j(handler, {'error': 'No file field in request'}, status=400)
@@ -396,11 +483,10 @@ def handle_upload_extract(handler):
"""Handle archive upload and extraction."""
import traceback as _tb
try:
content_type = handler.headers.get('Content-Type', '')
content_length = int(handler.headers.get('Content-Length', 0) or 0)
if content_length > MAX_UPLOAD_BYTES:
return j(handler, {'error': f'File too large (max {MAX_UPLOAD_BYTES//1024//1024}MB)'}, status=413)
fields, files = parse_multipart(handler.rfile, content_type, content_length)
try:
fields, files = _read_multipart_or_reject(handler)
except _RejectBeforeRead as rejection:
return _reject_before_read(handler, rejection.message, rejection.status)
session_id = fields.get('session_id', '')
if 'file' not in files:
return j(handler, {'error': 'No file field in request'}, status=400)
@@ -428,11 +514,10 @@ def handle_transcribe(handler):
import traceback as _tb
temp_path = None
try:
content_type = handler.headers.get('Content-Type', '')
content_length = int(handler.headers.get('Content-Length', 0) or 0)
if content_length > MAX_UPLOAD_BYTES:
return j(handler, {'error': f'File too large (max {MAX_UPLOAD_BYTES//1024//1024}MB)'}, status=413)
fields, files = parse_multipart(handler.rfile, content_type, content_length)
try:
fields, files = _read_multipart_or_reject(handler)
except _RejectBeforeRead as rejection:
return _reject_before_read(handler, rejection.message, rejection.status)
if 'file' not in files:
return j(handler, {'error': 'No file field in request'}, status=400)
filename, file_bytes = files['file']
@@ -612,12 +697,10 @@ def handle_workspace_upload(handler):
"""
import traceback as _tb
try:
content_type = handler.headers.get('Content-Type', '')
content_length = int(handler.headers.get('Content-Length', 0) or 0)
if content_length > MAX_UPLOAD_BYTES:
return j(handler, {'error': f'File too large (max {MAX_UPLOAD_BYTES//1024//1024}MB)'}, status=413)
fields, files = parse_multipart(handler.rfile, content_type, content_length)
try:
fields, files = _read_multipart_or_reject(handler)
except _RejectBeforeRead as rejection:
return _reject_before_read(handler, rejection.message, rejection.status)
session_id = fields.get('session_id', '')
subpath = fields.get('path', '')
docs/CONTRACTS.md
+9
View File
@@ -89,6 +89,15 @@ contributor guidance; it does not change runtime behavior or CI gates.
`models_cache.json` refresh timestamps), the stat-identity fallbacks, and the
schema/version stamps. Start here before changing model-catalog caching, the
`/api/models` cache keys, or the Codex catalog dependency (#2443, #7540).
- [`docs/architecture/live-models-allowlist.md`](architecture/live-models-allowlist.md):
current contract for how `/api/models/live` filters a custom provider's
upstream catalog: the four signals in evaluation order (discovered catalog
defers to the live probe, explicit plural `models:` allowlist filters,
singular `model:` never gates, no allowlist shows the full catalog), the
serialized-list shapes `hermes config set` persists, the deliberate
empty-allowlist-is-not-configured rule, and the probe-failure fallback.
Start here before changing custom-provider model filtering or
discovery-vs-allowlist semantics (#7165, #7404).
- [`docs/architecture/profile-home-resolve-cache.md`](architecture/profile-home-resolve-cache.md):
current contract for the call-scoped memoization around
`_resolve_profile_home_param()` in `api/workspace.py`: what it caches, the
docs/architecture/live-models-allowlist.md
+123
View File
@@ -0,0 +1,123 @@
# Live model picker allowlist contract (`/api/models/live`)
This document records the runtime contract for how `/api/models/live` decides
which models a `custom` / `custom:*` provider contributes to the chat model
picker. It describes shipped behavior and changes no runtime behavior. It was
added because the #7165 review flagged the contract as undocumented: the
filtering rules live in `_handle_live_models` (`api/routes.py`) and had only
code comments backing them.
Start here before changing how the live catalog of a custom provider is
filtered, how discovery snapshots are distinguished from allowlists, or what
the endpoint returns when the upstream probe fails.
## Why the endpoint filters at all
OpenAI-compatible gateways (New-API-style aggregators, LM Studio, Ollama
proxies) expose their **entire** catalog from `/v1/models`, including
image-generation, audio, TTS, and embedding models that are not chat models.
The background enrichment path must not surface models the user never opted
into — but it also must not hide models the gateway gained since config was
last edited. The contract below is how those two goals are reconciled.
## The four signals, in evaluation order
For a custom provider the handler consults these signals:
1. **Discovered catalog → no allowlist (live probe wins).**
When the provider entry carries `models_discovered: true` and discovery is
allowed (`discover_models` is not an explicit opt-out), the `models:`
mapping is a *snapshot Hermes persisted from a past discovery*, not a
hand-curated list. It must not gate the live probe: a model the user pulls
into LM Studio / Ollama after discovery has to appear in the picker.
Detected by the shared predicate
`api.config._provider_models_are_discovered_catalog()` — the same
predicate the `/api/models` catalog path uses, so both endpoints agree.
With this signal, no allowlist is derived and the full live catalog is
returned.
2. **Explicit plural `models:` allowlist → filter.**
A non-empty plural `models:` value on an entry that is *not* a discovered
catalog expresses "show exactly these models". The live catalog is
intersected with it, then any allowlisted model missing from the live
response is appended (so an allowlisted-but-offline model stays
selectable). This is the #7165 filter. Only **list** and **string**
(serialized) shapes are plain allowlists — see "Dict-shaped `models:`"
below.
3. **Singular `model:` → never gates.**
The singular field is sticky/default metadata (the model to preselect),
not an allowlist. A provider configured only with `model: assistant` keeps
the full unfiltered live catalog; gating on it would collapse discovery to
a single model.
4. **No allowlist → full live catalog.**
Preserves the pre-#7165 discovery behavior for setups that rely on it.
### Serialized shapes
`hermes config set` and the JSON-mode editor persist lists as quoted
JSON-array strings (`'["chat-a","chat-b"]'`) or Python literals
(`"['chat-a']"`). The plural value is decoded through the shared
`_parse_config_string_list()` before id extraction — but only when the value
is a `str`; native `dict` / `list` values are walked directly so dict *entries*
keep their `id|model|name` metadata (the decoder stringifies members). A
dict-only parser drops serialized allowlists to "no allowlist" and floods the
picker — the same bug class as `skills.disabled` (#7120 / #7134).
### Dict-shaped `models:`
A dict-shaped `models:` mapping (e.g. `{chat-a: {context_length: 128000}}`)
is *per-model metadata* written by the Hermes Agent setup flow —
`hermes_cli/model_switch.py::_save_custom_provider` and the setup wizard —
**not** a catalog narrow. Treating its keys as an allowlist would collapse the
live picker to the single saved default (keyless Ollama) while the CLI
live-probe shows the full catalog. So a dict is **never** a plain allowlist:
- without an explicit `discover_models: false`, a dict-shaped `models:`
contributes **no allowlist** and the full live catalog is returned;
- with `discover_models: false` (bool `False` or the string forms
`"false"`/`"no"`/`"0"`, case-insensitive — mirroring
`model_switch_providers.py:557`), the dict **keys** are honored as a pinned
allowlist.
This complements the discovered-catalog rule: `models_discovered: true`
without the opt-out is already ignored (signal 1). The dict guard covers the
Agent-setup shape that is *not* flagged discovered.
If someone later "fixes" the dict shape back into an always-allowlist, the
regression test `test_dict_models_without_discover_false_does_not_gate` fails
and forces the trade-off to be re-argued.
### Empty allowlist semantics
`models: []`, `models: "[]"`, an empty mapping, or a blank scalar are
deliberately treated as **not configured**, never as "allow nothing". The
filter runs only on a non-empty allowlist because gating on declared-ness
would emit an empty picker and make the provider unselectable — a harder
failure than showing a few extra models, and unrecoverable from the UI
(`custom_providers` has no WebUI write path; it is hand-edited in
config.yaml). There is no deny-all use case: a provider the user wants hidden
is removed from `custom_providers` outright.
### Probe failure / empty probe
When the upstream probe fails or returns no models, the endpoint falls back
to `allowlist or config ids` — the non-empty plural allowlist first, so a
singular `model:` outside the allowlist cannot leak in precisely when the
probe is unavailable; with no allowlist, the configured ids (singular +
plural) are served. A discovered catalog therefore still serves its saved
models on a transient probe failure instead of emptying the picker.
## Tests
- `tests/test_issue3718_live_models_custom_probe.py` — real-handler
(`_handle_live_models`) coverage of every signal above: plural list /
JSON-array-string / Python-literal / list-of-dicts filtering, dict-shaped
`models:` (not gated) and dict + `discover_models: false` (pinned),
singular passthrough, malformed safety, discovered-catalog passthrough,
`discover_models: false` pinning, and both fallback branches.
- `tests/test_issue7404_models_discovered_not_allowlist.py` — the
`/api/models` side of the discovered-catalog contract.
- `tests/test_byok_model_dropdown.py` — picker rendering, profile-scoped
caching, and current-session model preservation around this endpoint.
docs/pr-media/6469-sidebar-handoff/README.md
+91
View File
@@ -0,0 +1,91 @@
# Sidebar idle / terminal handoff evidence
## Failure and scope
The independent `/api/sessions` response can report idle before the chat SSE's
terminal frame reaches the page. Three sidebar paths were retiring chat state:
idle reconciliation, INFLIGHT purging, and optimistic-row retirement. The idle
path clears the Worklog and active stream ID; the subsequent `done` is rejected
as stale, so no settled Anchor scene is persisted.
A real-HTTP barrier reproduces this on both published PR head
`75c8ff8d2a991f22389e3e4263ee03fb8d8f67a0` and frozen upstream
`b09fded7b8c98b549a49aaaece3721015337d307`. Applying only the `sessions.js`
product fix to the frozen upstream also passes the identical oracle: this is a
shared lifecycle defect, not evidence that artifact replay introduced it. The
historical hosted run lacks this instrumentation, so its exact interleaving
cannot be proved retrospectively; its empty-surface/no-persistence signature
matches the controlled reproduction.
## Reproduction
```sh
LIFECYCLE_SCENARIO=normal LIFECYCLE_TEST_BITE=settle-worklog-frame-proof \
.venv/bin/python tests/browser_conversation_lifecycle.py
./scripts/test.sh tests/test_sidebar_terminal_handoff.py -q
```
The existing frame proof now uses a test-only server wrapper to hold the actual
`done` HTTP write. The real worker finishes; the real browser receives an idle
sidebar response through `renderSessionList()` while its native EventSource
remains open. The test verifies pane, INFLIGHT and registry ownership, then
releases `done` and retains all existing painted-frame, final-answer, semantic
activity and hard-reload assertions. The fixture always releases/cleans up and
has a bounded wait. Normal production server startup is unchanged.
No renderer, terminal payload or persistence response is fabricated. Runtime
events come from the existing deterministic localhost Gateway fixture, not a
live model/provider. All state and files are disposable. These are browser
viewport checks, not physical-device acceptance.
## Before / after
Each screenshot is taken after the idle sidebar response, before the terminal
HTTP frame is released. The exact published source fails at both widths. The
patched source preserves three live activity rows and then passes completion
and hard reload. `results.json` records the predicate results without machine
paths or credentials.
| Viewport | Before | After |
| --- | --- | --- |
| 1440 x 900 | [Empty surface](before-1440.png) | [Owned handoff retained](after-1440.png) |
| 390 x 844 | [Empty surface](before-390.png) | [Owned handoff retained](after-390.png) |
The fix only defers these sidebar cleanups for the current pane's exact OPEN
chat transport. Missing, CONNECTING, CLOSED and mismatched sources still take
the existing stale recovery path.
## Missing terminal frame: bounded recovery
The OPEN-only guard at `85f1c2d34b62b0ade85f51ed2cf8c1cf56f5ab9e` could wait
forever when the terminal frame never arrived. The follow-up keeps the live
handoff for 1.5 seconds, then probes the exact stream through
`/api/chat/stream/status` with an eight-second timeout and no retries.
Repeated sidebar hints cannot restart the deadline or create concurrent
requests. `active:true` keeps the OPEN owner intact even when the persisted
session fields already look idle; only explicit runtime inactivity permits the
canonical session restore. A failed or malformed runtime probe uses the
existing interrupted-stream path rather than claiming successful settlement.
Normal terminal delivery, transport replacement/close, navigation, and a newer
optimistic turn invalidate the old request. Already-running stream-end recovery
remains the sole recovery owner.
```sh
LIFECYCLE_SCENARIO=normal LIFECYCLE_TEST_BITE=sidebar-idle-missing-terminal \
.venv/bin/python tests/browser_conversation_lifecycle.py
./scripts/test.sh tests/test_sidebar_idle_stream_recovery.py -q
```
The new browser row keeps the real terminal HTTP barrier closed until cleanup;
the client must recover without receiving `done` or `stream_end`. Published
`85f1c2d34b` fails the 11-second completion bound with `S.busy` still true.
The patched implementation restores the final answer and passes the unchanged
painted-frame, semantic activity, persistence and hard-reload checks before the
barrier is released. The cloud workflow runs both delayed- and missing-terminal
rows. No live provider or production state is used.
A production Gateway regression also blocks post-turn goal evaluation after
the success transcript has already cleared `active_stream_id` and pending
fields. During that interval the worker and exact entry in `STREAMS` remain
live and no `done` event has been emitted, proving why persisted session flags
cannot authorize sidebar settlement on their own.
docs/pr-media/6469-sidebar-handoff/after-1440.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 59 KiB

docs/pr-media/6469-sidebar-handoff/after-390.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 30 KiB

docs/pr-media/6469-sidebar-handoff/before-1440.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 56 KiB

docs/pr-media/6469-sidebar-handoff/before-390.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 28 KiB

docs/pr-media/6469-sidebar-handoff/results.json
+66
View File
@@ -0,0 +1,66 @@
[
{
"mode": "before",
"viewport": [
1440,
900
],
"exit": 1,
"observation": {
"serverIdle": true,
"paneRetained": false,
"inflightRetained": false,
"sourceRetained": true,
"registryRetained": true,
"liveRows": 0
}
},
{
"mode": "before",
"viewport": [
390,
844
],
"exit": 1,
"observation": {
"serverIdle": true,
"paneRetained": false,
"inflightRetained": false,
"sourceRetained": true,
"registryRetained": true,
"liveRows": 0
}
},
{
"mode": "after",
"viewport": [
1440,
900
],
"exit": 0,
"observation": {
"serverIdle": true,
"paneRetained": true,
"inflightRetained": true,
"sourceRetained": true,
"registryRetained": true,
"liveRows": 3
}
},
{
"mode": "after",
"viewport": [
390,
844
],
"exit": 0,
"observation": {
"serverIdle": true,
"paneRetained": true,
"inflightRetained": true,
"sourceRetained": true,
"registryRetained": true,
"liveRows": 3
}
}
]
docs/rfcs/live-to-final-assistant-replies.md
+13
View File
@@ -470,6 +470,19 @@ Expected behavior:
- A session row's running indicator should reflect a real active run or a
clearly restorable state, not stale persisted metadata alone.
- An idle sidebar response does not prove that the browser has consumed the
terminal chat frame. While the current pane still owns its exact OPEN chat
transport, list reconciliation and cache purging must preserve that handoff.
Missing, connecting, closed, or mismatched transports keep the normal stale
state recovery path; a cached stream ID or busy flag is insufficient.
OPEN is not proof that a terminal frame will arrive: an idle hint must also
schedule bounded recovery without erasing the live scene. Before reading a
settled session snapshot, that recovery must check the exact stream's runtime
status. A runtime-active stream retains ownership even when persisted
`active_stream_id` and pending fields are already clear; only explicit
runtime inactivity permits canonical snapshot settlement. Probe failure is a
degraded/interrupted state, not evidence of successful completion. A newer
turn, replacement source, or received terminal event invalidates the request.
- Background completion, cancellation, or failure should be represented without
stealing the visible pane from the user.
- Session switching should not erase pending live context, in-flight snapshots,
docs/rfcs/session-sse-contract-v1.md
+26 -1
View File
@@ -183,7 +183,7 @@ when integrating with the live chat SSE relay.
| `cancel` | Run cancelled |
| `done` | Turn finalized (session payload); title/`stream_end` may follow |
| `stream_end` | SSE fence — close the client EventSource |
| `metering` | Token/cost metering snapshot |
| `metering` | Best-effort live token/cost metering snapshot; not journal-replayed |
| `context_status` | Context window / usage status |
| `goal` | Goal / plan card update |
| `goal_continue` | Goal continuation signal |
@@ -228,6 +228,31 @@ A future implementation must replay from the run journal via the existing
`_replay_run_journal()` path (in `api/routes.py`) and fall back to the
snapshot mechanism when journal entries are unavailable for a given cursor.
### Live-only metering (implemented replay behavior)
`metering` is best-effort live telemetry, not durable recovery content.
`RunJournalWriter.append_sse_event()` returns `None` for metering without
writing a row or allocating a sequence number. Other journaled events retain
contiguous sequence numbers. Local-agent and gateway producers carry an
explicit per-frame event ID, including `None`, through `StreamChannel`;
an unjournaled metering frame emits no new SSE `id:` and must not borrow the
previous frame's journal ID. The last-ID fallback is limited to legacy
two-element queue entries.
Existing journals are not rewritten. Direct readers (`read_run_events()` and
`read_session_run_events()`) retain legacy metering rows for cursor validation
and gap/coverage checks. Both run-level and per-session journal replay emitters
apply `journal_replay_visible()` to omit those rows from reconnect delivery.
Consequently, replayed legacy event IDs can have gaps where metering was
filtered; clients must not infer missing durable content from those gaps.
Metering missed while disconnected is not recovered from the journal; fresh
live updates remain available. This exception does not remove token, reasoning,
tool, or terminal events from durable replay.
Coverage: `tests/test_run_journal.py`, `tests/test_run_journal_routes.py`,
`tests/test_issue4812_session_sse_stream.py`, and
`tests/test_stage364_opus_live_sse_event_id.py`.
## Snapshot fallback
When the `Last-Event-ID` cursor is evicted, expired, unknown, or refers to a
docs/rfcs/stable-assistant-turn-anchors.md
+32
View File
@@ -574,6 +574,38 @@ early renderers can avoid chasing `S.messages` during every paint. If settlement
later rewrites the transcript message, the anchor must be refreshed from that
message instead of allowing the two copies to drift silently.
### Settled ownership of a restored live turn
A preserved or snapshotted live-turn node is not evidence of a running turn. The
settled assistant message is persisted a few milliseconds before the stream's
terminal event clears the active stream id, so a live node whose stream has
already ended can survive a transcript rebuild or a session switch and be
appended under the settled answer, putting the same answer on screen twice
(#6948 follow-up; upstream symptom report #2051).
A live-turn node is therefore discarded only where the settled transcript is
proved to own it: the transcript ends with a settled assistant message, no
message still carries a live-projection marker, that settled message's
persisted stream identity equals the id of the stream that built the live node
— or, only where no scene identity was persisted, the markdown the stream
produced equals the persisted message source — and the node carries nothing the
settled rebuild could not have produced, meaning no unpersisted tool card,
reasoning row or transparent-stream row and no second live segment.
Ownership is a state proof, never a comparison of two rendered DOM trees: the
live body comes from the streaming parser and the settled body from the
markdown renderer, and an answer that reads the same as the previous one must
not delete a genuinely live turn. An unrecognised node shape keeps the turn.
Only the two branches that ADD a turn are gated; a branch that replaces a node
cannot duplicate one.
A settled activity scene projects one answer as well. A `process_prose` row
whose text duplicates the scene's final answer — the same text, or a prefix of
it within ten percent of its length — is dropped where both settled renderers
take their rows, so the answer is not rebuilt as a second assistant segment
above itself. Live rendering is unchanged: while the turn streams, the inline
live segment is hidden and the prose row is the visible answer.
## Replay, Reload, And Reconstruction
Replay/reload should reconstruct the same Assistant Turn Anchor from durable
docs/rfcs/webui-run-state-consistency-contract.md
+80
View File
@@ -30,6 +30,69 @@ This RFC defines a consistency contract for those layers. It complements the
larger run adapter direction in #1925 by documenting what must remain coherent
while WebUI still has multiple overlapping state stores.
## Agent registration after cancellation
Initial and credential self-heal Agent construction use the same registration
boundary. Under `STREAMS_LOCK`, both the worker-retained cancellation event and
live stream membership must permit registration. A removed `CANCEL_FLAGS` entry
is not permission to restart. If Stop won during initial or self-heal
construction, the candidate must not enter the reusable cache or call
`run_conversation`. Stream registration, reusable-cache publication and the
in-memory lifecycle handle share one atomic Stop admission, using lock order
`STREAMS_LOCK` then `SESSION_AGENT_CACHE_LOCK`. Merely moving a cache write
after a cancellation check does not protect that check-to-publication gap.
After prompt preparation and immediately before each initial/self-heal invocation,
revalidate the retained cancel event, stream membership and exact registered
Agent. If Stop already won, retire a matching reusable entry only while the
existing `SESSION_WRITEBACK_OWNERS` record still equals this exact stream.
Hold that ownership lock through cache/lifecycle retirement. Object identity is
insufficient because successors can reuse the same Agent; absent ownership is
not permission either, since a completed successor clears its record. Never
clear a successor's cache or lifecycle handle, and do not issue another Agent
interrupt for an invocation that never started. Rejected cache-hit registration
likewise must not interrupt the borrowed Agent; only a never-published newly
constructed candidate may receive construction-cancellation cleanup. Drop the
old worker's local borrowed handle as well, so final pending-Steer drain cannot
reach a successor through an object-reference fallback.
Stop after invocation admission uses the existing Agent interrupt mechanism;
registry locks must not span provider or tool execution. LRU eviction/close stays
outside the stream lock and retains the existing active-worker policy.
Interrupt and cancellation finalization occur outside the stream registry lock.
Session finalization still owns the session lock: the returned-error path already
holds it, while initial registration and the exception path acquire it. Do not
reacquire this non-reentrant lock from a branch that already owns it.
## Run-journal sequence publication
Within one WebUI process, auto-numbered appends to the same journal allocate
sequence numbers and write their rows under the same per-path lock.
`RunJournalWriter` delegates both operations to `append_run_event`; it must not
reserve a sequence and release the lock before the physical append. Otherwise
individually valid rows can reach disk out of order and the session replay
reader must reject them as noncontiguous. This does not change caller-supplied
sequence semantics, cross-process ownership, or failed-write recovery.
## Inactive compression continuation recovery
The Agent profile's SQLite compression lineage owns the canonical continuation,
including when Desktop/CLI compressed a session without updating WebUI's
`pre_compression_snapshot` sidecar flag. `GET /api/session` may expose the
existing `continuation_session_id` hint from that read-only lineage. Automatic
`idle_timeout` closure does not hide the continuation; explicit/unknown terminal
reasons, foreign-profile rows and delegated/tool children do not authorize it.
This read does not reopen sessions or copy ancestor display history into context.
A stale `POST /api/chat/start` returns HTTP 409 with `code=session_rotated`
and the continuation hint before workspace, model, pending-turn or worker
mutation. The browser loads the continuation through normal session access
checks and restores the rejected text and attachments as a draft. The user
sends again explicitly; there is no automatic POST replay or migration of the
parent's workspace binding. Clients without this handling must reload the
session before retrying. Server wakeups, regeneration semantics and Gateway
routing are not silently retargeted by this recovery path.
## Goals
- Define the state layers involved in active and recovered WebUI turns.
@@ -112,6 +175,23 @@ and 5; it does not mark every run-state boundary implemented.
Visible interim assistant progress must remain visible timeline content; a
compact Activity disclosure may summarize adjacent tool/debug detail, but it
must not be the only place where the user can see emitted progress text.
Interim assistant text that duplicates the tail of the accumulated reasoning
transcript is stripped from the reasoning copy so the restored snapshot
shows the content once. That echo match is whitespace-insensitive and
carries no fixed search window: a compact-equivalent suffix is recognized
however much interior whitespace stretches its raw span. Both consumers
(the live-stream echo path and the journal replay in `api/routes.py`)
match through an incremental folded index (`_CompactEchoIndex` in
`api/streaming.py`): the folded view and its raw cut offsets are built as
each chunk is appended, so a probe costs O(len(candidate)) and never
rescans the transcript's whitespace. The retired windowed variants could
drop the strip when the span exceeded the window, duplicating the interim
text; the retired raw backward walk was correct but re-walked the span per
interim event, quadratic on whitespace-heavy transcripts. Regressions:
`tests/test_live_snapshot_echo_dedup.py` pins the single-occurrence
result, `tests/test_live_snapshot_echo_scan_scaling.py` pins the scaling
property (a fixed-size fixture cannot catch a per-interim rescan), and
`tests/test_compact_echo_index.py` pins index/oracle equivalence.
6. **Compression is not current intent.** Automatic compression summaries and
reference cards are recovery/handoff material. They must not be treated as a
new user request, active-turn content, or the default visible explanation for
docs/sse-streams.md
+15
View File
@@ -56,6 +56,21 @@ infer it from the status code. Clients that only need session updates should
use `/api/session/stream` directly; the gateway probe is only relevant for
clients that display CLI/TUI/messaging sessions.
## Gateway watcher projection lifecycle
The optional gateway stream's watcher owns a cheap database fingerprint and a
timestamp for its last full session projection. While clients are subscribed,
an unchanged fingerprint can skip the expensive read until the bounded parity
interval. With no subscribers the poll loop parks instead of reading the DB.
When the final subscriber leaves (including eviction of a full or failed
consumer queue), the watcher invalidates both cache fields. A reconnect must
therefore trigger a fresh projection even when the database fingerprint has not
changed and the parity interval has not elapsed. Final removal also advances a
subscriber epoch: an older in-flight DB read cannot restore the invalidated
cache or publish its result to a later subscriber cohort. The subscriber lock
protects removal, epoch checks and cache commits, but is not held during DB
reads, so disconnect is not blocked by a slow projection.
## Heartbeats and proxy behavior
- All long-lived streams emit SSE keepalive comment lines on the
docs/troubleshooting.md
+68
View File
@@ -231,6 +231,36 @@ For a foreground `python3 bootstrap.py`, stop it with Ctrl-C and start it again.
---
## Agent sessions list slowly (or the `state.db` read index is missing)
**Symptom.** The sidebar's imported/CLI session list takes seconds per refresh on a large Hermes profile, or a log line says a `state.db` read failed. Sessions still appear; nothing is lost.
**Why.** Every WebUI reader of the agent's `state.db` (session listing, transcript reads, lineage, gateway watcher, cron sidebar, insights, health) opens it strictly read-only (`file:...?mode=ro`). A reader never upgrades to a write-capable handle and never creates an index: on a multi-GiB `messages` table `CREATE INDEX` holds the SQLite writer lock for minutes and stalls the agent streaming into the same WAL database. When the agent's standard `idx_messages_session` index is missing (older agent, hand-rebuilt or re-imported DB), listings degrade to a bounded one-pass pre-aggregation — slower than the indexed seek, but read-only. A read-only open failure propagates to the caller's existing error boundary (the listing returns empty for that profile) instead of silently reopening the file writable.
**Diagnostic.**
```bash
sqlite3 "file:$HOME/.hermes/state.db?mode=ro" "PRAGMA index_list(messages)"
```
`idx_messages_session` should be listed. If it is not, the agent has not created it and WebUI will not create it for you.
**Fix.** Create the covering read indexes in an explicit drained maintenance window: stop the agent (and any gateway/cron runner) writing to that `state.db`, then run:
```bash
python3 scripts/ensure_state_db_read_indexes.py --db ~/.hermes/state.db --confirm-drained
```
- `--confirm-drained` is mandatory: it is your assertion that no agent turn is running against the database. The tool does not verify it.
- `--lock-file PATH` (optional) additionally holds an exclusive non-blocking lock on `PATH` (`flock` on POSIX, `msvcrt.locking` on Windows) for deployments that serialise agent turns on a lock file; a held lock makes the tool exit without touching the database. Without `--lock-file` no lock primitive is required, so the script runs on native Windows as well. On Windows the lock covers byte 0 of `PATH`; a new or empty lock file is initialised with one byte first (an existing lock file is never rewritten).
- The tool opens the database `mode=rw` (never `rwc`): a mistyped path raises instead of creating an empty database. Indexes are created inside one `BEGIN IMMEDIATE` transaction and rolled back on any error.
- It is idempotent and prints a JSON status per index (`created` / `existing` / `skipped`). `skipped` means this database's schema lacks a column that index keys on (an older agent); the indexes the schema does support are still created. An existing index with a different table, key shape or collation is reported as `Incompatible index` and never replaced; an index that is not covering (`EXPLAIN QUERY PLAN`) is reported as `Index is not covering`.
- Windows UNC profiles (`HERMES_HOME=\\server\share\...`) are supported: readers and this tool build the empty-authority URI `file:////server/share/state.db` that the bundled SQLite accepts.
**When to file a bug.** File a WebUI bug if the listing stays slow after the tool reports `existing` for `idx_messages_session`, if the tool reports `Incompatible index` on an untouched agent-created database, or if a read-only open fails on a local path. Include the tool's JSON output, the `PRAGMA index_list(messages)` result, and the sanitized error text.
---
## 404 after login when password auth is enabled
**Symptom.** After enabling password authentication (`HERMES_WEBUI_PASSWORD`), logging in redirects to `/sessions` and the browser shows a `404 not found` error instead of the chat interface.
@@ -241,6 +271,44 @@ For a foreground `python3 bootstrap.py`, stop it with Ctrl-C and start it again.
---
## "OpenCode Go model picker shows a model that errors when you send" (or is missing newly released models)
**Symptom.** One of two directions:
1. A model selected from the OpenCode Go group fails on the first message (`model not found`, `Model is unavailable`, or a region error), even though the picker offered it.
2. A newly released OpenCode Go model does not appear in the picker at all, and must be typed into the Custom Model ID box.
**Why.** The Go picker follows the **live** Go-tier catalog (`https://opencode.ai/zen/go/v1/models`) whenever the installed Hermes Agent is v0.20.5 or newer. That endpoint advertises a superset of what a given tier, key, or region can actually serve — an id can be listed and still fail on send. Conversely, an id the endpoint serves but the WebUI's static fallback list predates is missing when the live path is unavailable (Agent older than v0.20.5, probe failure, or offline); the static list mirrors Hermes core's curated Go catalog and can lag new releases by design.
**Diagnostic.** Check which path is feeding your picker:
```bash
hermes --version # live path requires >= 0.20.5
curl -sS https://opencode.ai/zen/go/v1/models \
-H "Authorization: Bearer $OPENCODE_GO_API_KEY" | head -50
```
Interpret the two together:
- **Failing model absent from the `curl` output** → it was delisted upstream (for example `ox-alpha-free`, removed from the relay on 2026-09-09). A picker can still offer it from a **stale catalog merge**: Hermes core merges its own curated Go list into the live result, and every Agent release *through v0.21.1 (tag `v2026.9.7`)* still carries the delisted id in that list — verified at runtime against v0.21.0, where the live path serves 37 ids including `ox-alpha-free`. The removal is committed on core `main` (2026-09-09 sync) but **no released Agent version includes it yet** — and `main` reports the same `0.21.1` version string as the stale tag, so a version number alone cannot tell you whether the fix is in. Until a release notes the 2026-09-09 catalog sync, the **verified workaround is the config allowlist** (below); selecting the dead entry is harmless to other models (it errors on send, nothing else).
- **Failing model present in the `curl` output** → the relay lists it but your tier/region cannot serve it; the picker is behaving correctly. Pin the models you actually use with an explicit allowlist in `config.yaml`, which takes precedence over both the live catalog and the fallback:
```yaml
providers:
opencode-go:
models:
- kimi-k3
- glm-5.3
```
A lighter per-provider exclude capability is tracked in #7507.
- **Missing new model, Agent ≥ v0.20.5** → the live catalog is the source; refresh or check the endpoint with the `curl` above (cold rebuilds are also bounded by a 4-second foreground budget — the first picker open after a restart can serve the last-known list while the live rebuild finishes in the background, so re-open the picker once before concluding it's stale).
- **Missing new model, Agent older than v0.20.5** → the static fallback is serving by design; upgrade the Agent to ≥ v0.20.5 so the picker reads the live catalog.
**When to file a bug.** File a WebUI bug if a model fails on send *and* appears in the `curl` output for your key (a routing problem), or if a model is missing with Agent ≥ v0.20.5 and the live catalog reachable (fallback used when it should not be).
---
## Other troubleshooting
This document grows over time. If a recurring failure mode isn't covered here yet, add it via PR. The format for each entry: **Symptom → Why → Diagnostic commands → Fix → When to file a bug**.
scripts/ensure_state_db_read_indexes.py
+191
View File
@@ -0,0 +1,191 @@
#!/usr/bin/env python3
"""Create the covering read indexes on the agent ``state.db`` in an explicit,
drained maintenance window.
The WebUI read paths (session listing, lineage reads, gateway watcher, cron
sidebar, insights, health) never create indexes themselves: ``CREATE INDEX``
on a multi-GiB ``messages`` table holds the SQLite writer lock for minutes,
which stalls the agent streaming into the same WAL database. Index
maintenance is therefore an operator action run while the agent is idle.
Usage::
python scripts/ensure_state_db_read_indexes.py --db ~/.hermes/state.db \\
--confirm-drained [--lock-file /path/to/agent-activity.lock]
``--confirm-drained`` is mandatory. When ``--lock-file`` is given, the tool
takes an exclusive non-blocking lock on that path (``flock`` on POSIX,
``msvcrt.locking`` on Windows) so that a deployment which serialises agent
turns on a lock file cannot start a turn mid-rebuild. Without ``--lock-file``
no lock primitive is needed, so the script runs on every supported platform.
Existing indexes are verified for table, key shape and collation and must be
covering (``EXPLAIN QUERY PLAN``); an incompatible index is reported, never
silently replaced.
"""
import argparse
from contextlib import closing, nullcontext
import json
import os
from pathlib import Path
import sqlite3
import sys
try: # POSIX
import fcntl
except ImportError: # native Windows
fcntl = None
try: # Windows
import msvcrt
except ImportError: # POSIX
msvcrt = None
_REPO_ROOT = Path(__file__).resolve().parents[1]
if str(_REPO_ROOT) not in sys.path:
sys.path.insert(0, str(_REPO_ROOT))
from api.agent_sessions import state_db_file_uri # same UNC-aware URI as the readers
INDEXES = {
"idx_messages_session": ("messages", (("session_id", "BINARY"), ("timestamp", "BINARY"))),
"idx_messages_session_role": ("messages", (("session_id", "BINARY"), ("role", "NOCASE"))),
"idx_sessions_webui_fingerprint": ("sessions", tuple((c, "BINARY") for c in
("source", "id", "message_count", "last_activity_at"))),
}
# The only schema gap an older Agent is known to have: sessions.last_activity_at
# arrived later than the other keyed columns. An index whose missing columns are
# all listed here is reported "skipped"; any other missing column means this is
# not an Agent state.db (or a damaged one), and the tool fails closed.
_LEGACY_OPTIONAL_COLUMNS = {
"idx_sessions_webui_fingerprint": frozenset({"last_activity_at"}),
}
def _require_agent_schema(db) -> None:
"""Fail closed unless this is an Agent ``state.db``.
Matching column names alone would accept a look-alike chat database. Every
Agent ``state.db`` since the first schema has carried a ``schema_version``
row and ``sessions.id`` as the primary key, so require both before any
``CREATE INDEX``.
"""
has_marker = db.execute(
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='schema_version'"
).fetchone()
version = None
if has_marker:
cols = {r[1] for r in db.execute("PRAGMA table_info(schema_version)")}
if "version" in cols:
row = db.execute("SELECT version FROM schema_version LIMIT 1").fetchone()
version = row[0] if row else None
if not isinstance(version, int) or version < 1:
raise RuntimeError(
"state.db has no Agent schema_version marker; not an Agent state.db, refusing to continue"
)
pk = [r[1] for r in db.execute("PRAGMA table_info(sessions)") if r[5]]
if pk != ["id"]:
raise RuntimeError(
"state.db sessions table is not keyed on id; not an Agent state.db, refusing to continue"
)
def _exclusive_lock(lock_file):
if lock_file is None:
return nullcontext()
if fcntl is None and msvcrt is None:
raise RuntimeError("--lock-file needs fcntl (POSIX) or msvcrt (Windows); "
"neither is available on this interpreter")
handle = open(lock_file, "a+")
try:
if fcntl is not None:
fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
else:
# Lock one byte at offset 0 without blocking; raises OSError when held.
# ``a+`` may have just created an empty file, and Windows cannot lock
# a range beyond EOF, so make sure byte 0 exists first. An existing
# (deployment-owned) lock file is left untouched. Write the byte on
# the descriptor: a text-mode ``handle.write("\n")`` becomes two
# bytes (``\r\n``) on native Windows.
if os.fstat(handle.fileno()).st_size == 0:
os.write(handle.fileno(), b"\0")
handle.flush()
handle.seek(0)
msvcrt.locking(handle.fileno(), msvcrt.LK_NBLCK, 1)
except BaseException:
handle.close()
raise
return closing(handle)
def ensure_read_indexes(db_path, *, confirmed_drained=False, lock_file=None):
if not confirmed_drained:
raise RuntimeError("requires --confirm-drained")
db_path = Path(db_path).resolve(strict=True)
statuses = {}
with _exclusive_lock(lock_file):
# mode=rw (not rwc): never create a ghost database at a mistyped path.
with closing(sqlite3.connect(state_db_file_uri(db_path) + "?mode=rw", uri=True)) as db:
db.execute("BEGIN IMMEDIATE")
try:
# Validate the whole Agent schema before creating anything, so a
# wrong database fails closed without a partial index set.
_require_agent_schema(db)
missing_by_index = {}
for name, (table, keys) in INDEXES.items():
present = {r[1] for r in db.execute(f"PRAGMA table_info({table})")}
if not present:
raise RuntimeError(f"state.db has no {table!r} table; refusing to continue")
missing = {col for col, _ in keys if col not in present}
if missing - _LEGACY_OPTIONAL_COLUMNS.get(name, frozenset()):
raise RuntimeError(
f"state.db {table!r} table lacks Agent column(s) "
f"{sorted(missing)}; not an Agent state.db, refusing to continue"
)
missing_by_index[name] = missing
for name, (table, keys) in INDEXES.items():
existing = db.execute("SELECT tbl_name FROM sqlite_master WHERE name=?", (name,)).fetchone()
if missing_by_index[name]:
# Known legacy gap. A same-named index here cannot be the
# shape we expect (it keys on the absent column), so it is
# incompatible, not something to skip past.
if existing:
raise RuntimeError(f"Incompatible index: {name}")
statuses[name] = "skipped"
continue
if existing:
actual = tuple((r[2], r[4]) for r in db.execute(f"PRAGMA index_xinfo({name})") if r[5])
flags = next((r for r in db.execute(f"PRAGMA index_list({table})") if r[1] == name), None)
if existing[0] != table or actual != keys or not flags or flags[2] or flags[4]:
raise RuntimeError(f"Incompatible index: {name}")
statuses[name] = "existing"
else:
columns = ", ".join(f"{col} COLLATE {collation}" for col, collation in keys)
db.execute(f"CREATE INDEX {name} ON {table}({columns})")
statuses[name] = "created"
columns = ", ".join(col for col, _ in keys)
plan = db.execute(f"EXPLAIN QUERY PLAN SELECT {columns} FROM {table} INDEXED BY {name}").fetchall()
if not any(f"COVERING INDEX {name}" in row[3] for row in plan):
raise RuntimeError(f"Index is not covering: {name}")
db.commit()
except BaseException:
db.rollback()
raise
return statuses
def main():
parser = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--db", type=Path, required=True, help="path to the agent state.db")
parser.add_argument("--lock-file", type=Path, default=None,
help="optional lock file to hold exclusively (flock) while indexes are built")
parser.add_argument("--confirm-drained", action="store_true",
help="assert that no agent turn is running against this database")
args = parser.parse_args()
result = ensure_read_indexes(args.db, confirmed_drained=args.confirm_drained,
lock_file=args.lock_file)
print(json.dumps(result, sort_keys=True))
if __name__ == "__main__":
main()
server.py
+7 -6
View File
@@ -100,10 +100,11 @@ from urllib.parse import urlparse
logger = logging.getLogger(__name__)
from api.request_logging import emit_request_log
from api.auth import check_auth, reset_trusted_auth_request_state
from api.auth import check_auth_or_close, reset_trusted_auth_request_state
from api.config import HOST, PORT, STATE_DIR, SESSION_DIR, DEFAULT_WORKSPACE
from api.helpers import (
j,
advertise_connection_close,
get_profile_cookie,
_build_csp_report_only_policy,
_CLIENT_DISCONNECT_ERRORS,
@@ -332,6 +333,7 @@ class Handler(BaseHTTPRequestHandler):
extra_frame_src = getattr(self, "_csp_extra_frame_src", None)
self.send_header("Content-Security-Policy-Report-Only", self.csp_report_only_policy(extra_connect_src, extra_frame_src))
self.send_header("Report-To", self._CSP_REPORT_TO)
advertise_connection_close(self) # tell the client when the socket dies
super().end_headers()
def log_message(self, fmt, *args): pass # suppress default Apache-style log
@@ -376,7 +378,8 @@ class Handler(BaseHTTPRequestHandler):
set_request_profile(cookie_profile)
try:
parsed = urlparse(self.path)
if not check_auth(self, parsed): return
# Body-pending-aware: a body-bearing GET failing auth would poison reuse (#7550).
if not check_auth_or_close(self, parsed): return
result = handle_get(self, parsed)
if result is False:
return j(self, {'error': 'not found'}, status=404)
@@ -401,10 +404,8 @@ class Handler(BaseHTTPRequestHandler):
set_request_profile(cookie_profile)
try:
parsed = urlparse(self.path)
_is_csp_report_post = (
parsed.path == "/api/csp-report" and self.command == "POST"
)
if not _is_csp_report_post and not check_auth(self, parsed): return
_is_csp_report_post = parsed.path == "/api/csp-report" and self.command == "POST"
if not _is_csp_report_post and not check_auth_or_close(self, parsed): return
result = route_func(self, parsed)
if result is False:
return j(self, {'error': 'not found'}, status=404)
static/boot.js
+10 -1
View File
@@ -3431,8 +3431,17 @@ window._mirrorSpeechSettingsFromServer=_mirrorSpeechSettingsFromServer;
localStorage.setItem('hermes-font-size',fontSize);
_applyFontSize(fontSize);
if(typeof setLocale==='function'){
// #7622 (round 3): the settings payload's `s.language` is
// absent (None) for a fresh install, so an explicit non-empty
// value is the user's genuine saved choice. The browser
// navigator hint is now read via the guarded
// `_detectBrowserLanguageHint()` helper (round-3 finding 2) so
// a throwing `navigator` accessor in some embedded webviews
// can no longer abort this branch and reset loaded preferences.
// The fallback ternary preserves the pre-#7622 boot behaviour
// when neither helper is in scope (defence in depth).
const _lang=typeof resolvePreferredLocale==='function'
? resolvePreferredLocale(s.language, localStorage.getItem('hermes-lang'))
? resolvePreferredLocale(s.language, localStorage.getItem('hermes-lang'), _detectBrowserLanguageHint())
: (s.language || localStorage.getItem('hermes-lang') || 'en');
setLocale(_lang);
if(typeof applyLocaleToDOM==='function')applyLocaleToDOM();
static/commands.js
+40
View File
@@ -1,3 +1,7 @@
const _WEBUI_DISPATCHABLE_AGENT_COMMANDS = new Set([
'reload-mcp','reload-skills','codex-runtime','credits',
'moa','sessions','resume','pet'
]);
// ── Slash commands ──────────────────────────────────────────────────────────
// Built-in commands intercepted before send(). Each command runs locally
// (no round-trip to the agent) and shows feedback via toast or local message.
@@ -182,6 +186,31 @@ function executeCommand(text){
return {noEcho:!!cmd.noEcho};
}
// Agent-registry slash commands that the WebUI actually dispatches when
// submitted. The autocomplete must not advertise anything outside this set:
// a non-dispatchable registry command (e.g. /agents) would otherwise fall
// through send() to the ordinary chat path and trigger an unintended
// model/API request.
//
// This set holds canonical registry names only. getMatchingCommands() matches
// cmd.name (the canonical metadata name), so alias/underscore forms such as
// /reload_mcp are covered implicitly: typing one resolves through
// getAgentCommandMetadata() at dispatch time and the canonical name is what
// gets tested against the dispatcher allowlist (#6951).
//
// Keep this in sync with _AGENT_COMMANDS_RUN_ON_WEBUI in messages.js and
// _ALLOWED_AGENT_COMMANDS in api/commands.py -- the announced list is a
// subset of the dispatched list (fail-closed), never a superset.
// Plugin-category commands are always dispatchable via the plugin exec
// transport. #6951.
function _isWebuiDispatchableAgentCommand(cmd){
const name=String(cmd&&cmd.name||'').trim().toLowerCase();
if(_WEBUI_DISPATCHABLE_AGENT_COMMANDS.has(name))return true;
// Plugin-registered commands execute via the /api/commands/exec plugin transport.
return String(cmd&&cmd.category||'').trim()==='Plugin';
}
function getMatchingCommands(prefix){
const q=prefix.toLowerCase();
const matches=COMMANDS.filter(c=>c.name.startsWith(q)).map(c=>({...c,source:'builtin'}));
@@ -214,9 +243,14 @@ function getMatchingCommands(prefix){
const name=String(cmd&&cmd.name||'').toLowerCase();
if(!name.startsWith(q)||seen.has(name))continue;
if(cmd.cli_only&&name!=='pet')continue;
// #6951: only announce commands send() actually dispatches -- anything
// else would silently fall through to the normal chat path as plain text.
if(!_isWebuiDispatchableAgentCommand(cmd))continue;
matches.push({
name,
desc:String(cmd&&cmd.description||'').trim()||'Agent command',
// Surface the registry's argument hint on the autocomplete row (#6951).
arg:String(cmd&&cmd.args_hint||'').trim()||undefined,
source:cmd.category==='Plugin'?'plugin':'agent',
});
seen.add(name);
@@ -957,6 +991,12 @@ async function resumeManualCompressionForSession(sid){
// No active compression job or transient server error — not a real failure.
// 404: route missed or session gone; 5xx: backend exception during status check.
if(e&&(!e.status||e.status===404||e.status>=500)) return;
// #7710: a cross-profile refusal now arrives as 409
// (``session_profile_mismatch``) where it used to be a 404 that hit the
// benign early-return above. It is not a compression failure, so do not
// render the error state or locally settle the compression UI.
if(e&&e.status===409&&typeof _sessionProfileMismatchFromError==='function'
&&_sessionProfileMismatchFromError(e)) return;
if(S.session&&S.session.session_id===sid&&typeof setCompressionUi==='function'){
const visibleMessages=_manualCompressionVisibleMessages();
setCompressionUi({
static/i18n.js
+88 -20
View File
@@ -1770,7 +1770,7 @@ const LOCALES = {
yolo_disabled: 'YOLO mode OFF',
yolo_pill_label: 'YOLO',
yolo_pill_title_active: 'YOLO mode active — click to disable',
approval_skip_all: '⚡ Skip all this session',
approval_skip_all: 'Skip all this session',
approval_skip_all_title: 'Skip all approval prompts for this session',
// composer action tooltips
composer_send: 'Send message',
@@ -3572,7 +3572,7 @@ const LOCALES = {
yolo_disabled: 'Modalità YOLO OFF',
yolo_pill_label: 'YOLO',
yolo_pill_title_active: 'Modalità YOLO attiva — clicca per disabilitare',
approval_skip_all: '⚡ Salta tutto questa sessione',
approval_skip_all: 'Salta tutto questa sessione',
approval_skip_all_title: 'Salta tutte le richieste di approvazione per questa sessione',
// composer action tooltips
composer_send: 'Invia messaggio',
@@ -5382,7 +5382,7 @@ const LOCALES = {
yolo_disabled: 'YOLO モード OFF',
yolo_pill_label: 'YOLO',
yolo_pill_title_active: 'YOLO モード有効 — クリックで無効化',
approval_skip_all: '⚡ このセッションはすべてスキップ',
approval_skip_all: 'このセッションはすべてスキップ',
approval_skip_all_title: 'このセッションのすべての承認プロンプトをスキップ',
// composer action tooltips
composer_send: 'メッセージを送信',
@@ -7063,7 +7063,7 @@ const LOCALES = {
yolo_disabled: 'YOLO режим ВЫКЛ',
yolo_pill_label: 'YOLO',
yolo_pill_title_active: 'YOLO режим активен — нажмите для отключения',
approval_skip_all: '⚡ Пропустить все в этом сеансе',
approval_skip_all: 'Пропустить все в этом сеансе',
approval_skip_all_title: 'Пропустить все запросы подтверждения в этом сеансе',
// composer action tooltips
composer_send: 'Отправить сообщение',
@@ -8830,7 +8830,7 @@ const LOCALES = {
yolo_disabled: 'Modo YOLO desactivado',
yolo_pill_label: 'YOLO',
yolo_pill_title_active: 'Modo YOLO activo — haz clic para desactivar',
approval_skip_all: '⚡ Saltar todo en esta sesión',
approval_skip_all: 'Saltar todo en esta sesión',
approval_skip_all_title: 'Saltar todas las solicitudes de aprobación en esta sesión',
// composer action tooltips
composer_send: 'Enviar mensaje',
@@ -10236,7 +10236,7 @@ const LOCALES = {
yolo_disabled: 'YOLO-Modus AUS',
yolo_pill_label: 'YOLO',
yolo_pill_title_active: 'YOLO-Modus aktiv — Klicken zum Deaktivieren',
approval_skip_all: '⚡ Alle in dieser Sitzung überspringen',
approval_skip_all: 'Alle in dieser Sitzung überspringen',
approval_skip_all_title: 'Alle Genehmigungsanfragen in dieser Sitzung überspringen',
// composer action tooltips
composer_send: 'Nachricht senden',
@@ -12312,7 +12312,7 @@ const LOCALES = {
yolo_disabled: 'YOLO 模式已关闭',
yolo_pill_label: 'YOLO',
yolo_pill_title_active: 'YOLO 模式激活 — 点击关闭',
approval_skip_all: '⚡ 本次会话全部跳过',
approval_skip_all: '本次会话全部跳过',
approval_skip_all_title: '跳过本次会话的所有审批提示',
// composer action tooltips
composer_send: '发送消息',
@@ -14208,7 +14208,7 @@ const LOCALES = {
yolo_disabled: 'YOLO 模式已關閉',
yolo_pill_label: 'YOLO',
yolo_pill_title_active: 'YOLO 模式已啟用，點選即可關閉',
approval_skip_all: '⚡ 本次工作階段全部跳過',
approval_skip_all: '本次工作階段全部跳過',
approval_skip_all_title: '略過本次工作階段的所有核准提示',
// composer action tooltips
composer_send: '傳送訊息',
@@ -17540,7 +17540,7 @@ const LOCALES = {
yolo_disabled: 'YOLO 모드 꺼짐',
yolo_pill_label: 'YOLO',
yolo_pill_title_active: 'YOLO 모드 활성 — 클릭하여 비활성화',
approval_skip_all: '⚡ 이 세션에서 모두 건너뛰기',
approval_skip_all: '이 세션에서 모두 건너뛰기',
approval_skip_all_title: '이 세션의 모든 승인 요청 건너뛰기',
// composer action tooltips
composer_send: '메시지 보내기',
@@ -19412,7 +19412,7 @@ const LOCALES = {
yolo_disabled: 'Mode YOLO désactivé',
yolo_pill_label: 'YOLO',
yolo_pill_title_active: 'Mode YOLO actif — cliquez pour désactiver',
approval_skip_all: '⚡ Ignorer tout cette session',
approval_skip_all: 'Ignorer tout cette session',
approval_skip_all_title: 'Ignorer toutes les demandes d\'approbation pour cette session',
composer_send: 'Envoyer le message',
composer_queue: 'Mettre en file d\'attente',
@@ -19540,7 +19540,7 @@ const LOCALES = {
add_failed: 'Přidání selhalo: ',
agent_soul: 'Duch agenta',
all_uploads_failed: (n) => `Všech ${n} nahrávání selhalo`,
approval_skip_all: '⚡ Přeskočit vše pro tuto relaci',
approval_skip_all: 'Přeskočit vše pro tuto relaci',
approval_skip_all_title: 'Přeskočit všechny výzvy ke schválení pro tuto relaci',
archive_extracted: (n, c) => `Rozbaleno ${n} souborů z ${c} archivů`,
auth_ack_save_failed: 'Nepodařilo se aktualizovat potvrzení: ',
@@ -22894,7 +22894,7 @@ const LOCALES = {
yolo_disabled: 'YOLO modu KAPALI',
yolo_pill_label: 'YOLO',
yolo_pill_title_active: 'YOLO modu etkin — devre dışı bırakmak için tıklayın',
approval_skip_all: '⚡ Tüm bu oturumu atla',
approval_skip_all: 'Tüm bu oturumu atla',
approval_skip_all_title: 'Bu oturuma ilişkin tüm onay istemlerini atla',
// composer action tooltips
composer_send: 'Mesaj gönder',
@@ -24778,7 +24778,7 @@ const LOCALES = {
yolo_disabled: 'Tryb YOLO WYŁĄCZONY',
yolo_pill_label: 'YOLO',
yolo_pill_title_active: 'Tryb YOLO aktywny — kliknij, aby wyłączyć',
approval_skip_all: '⚡ Pomiń wszystkie w tej sesji',
approval_skip_all: 'Pomiń wszystkie w tej sesji',
approval_skip_all_title: 'Pomiń wszystkie monity o aprobatę w tej sesji',
// composer action tooltips
composer_send: 'Wyślij wiadomość',
@@ -26411,7 +26411,7 @@ const LOCALES = {
approval_gateway_unsupported: 'Phê duyệt cần gateway mới hơn. Hãy nâng cấp Hermes gateway đang kết nối để bật tính năng này.',
approval_gateway_unsupported_label: 'Không hỗ trợ phê duyệt',
approval_pending_count: (n) => `1 trong ${n} đang chờ`,
approval_skip_all: '⚡ Bỏ qua tất cả trong phiên này',
approval_skip_all: 'Bỏ qua tất cả trong phiên này',
approval_skip_all_title: 'Bỏ qua mọi yêu cầu phê duyệt cho phiên này',
checkpoint_title: 'Checkpoint',
checkpoint_empty: 'Không tìm thấy checkpoint nào cho workspace này.',
@@ -26985,13 +26985,66 @@ function resolveLocale(lang) {
* Resolve locale with precedence:
* 1) primary (typically server setting)
* 2) fallback (typically localStorage)
* 3) English
* @param {string} primary
* @param {string} fallback
* 3) fallback2 (typically browser navigator hint — first-visit only)
* 4) 'en' (final fallback)
*
* Resolve a language preference to a known LOCALES key, applying a strict
* `primary -> fallback -> fallback2 -> 'en'` precedence chain.
*
* #7622 (round 3): the server no longer injects `"language": "en"` as a
* schema default — `load_settings()` reports `None` (the key is absent
* from the response) until the user explicitly picks a locale in the
* Settings modal. This lets the resolver use `primary` directly
* without a `primary === 'en'` special case:
* - an explicit `primary` (the server's stored value) always wins
* - a `None` / empty `primary` falls through to `fallback` (localStorage)
* - `fallback2` (the browser navigator hint) only fires when neither
* server nor localStorage has a preference
* - `'en'` is the final safety net
*
* Side benefit: a user who picked English on purpose (their settings
* file contains `"language": "en"`) is no longer overridden by a
* non-English browser locale on hydration, because the round-2
* `primary === 'en'` skip has been removed.
*
* @param {string|null|undefined} primary server-stored language
* @param {string|null|undefined} fallback localStorage `hermes-lang`
* @param {string|null|undefined} fallback2 browser navigator hint
* @returns {string}
*/
function resolvePreferredLocale(primary, fallback) {
return resolveLocale(primary) || resolveLocale(fallback) || 'en';
function resolvePreferredLocale(primary, fallback, fallback2) {
return resolveLocale(primary)
|| resolveLocale(fallback)
|| resolveLocale(fallback2)
|| 'en';
}
/**
* Detect the browser's preferred language for first-visit locale
* selection. Guarded against `navigator` access exceptions: some
* embedded webviews (Tauri desktop shell, kiosk modes) report a
* `navigator` object but throw on `.languages` / `.language` reads.
* Callers (loadLocale, boot.js, panels.js) must use this helper rather
* than reading `navigator` directly so a single throw does not abort
* the boot or settings-panel hydration.
*
* @returns {string|null} the first entry of `navigator.languages`, or
* `navigator.language`, or `null` when neither is reachable.
*/
function _detectBrowserLanguageHint() {
try {
const nav = (typeof navigator !== 'undefined') ? navigator : null;
if (!nav) return null;
if (Array.isArray(nav.languages) && nav.languages.length) {
return nav.languages[0] || null;
}
if (typeof nav.language === 'string' && nav.language) {
return nav.language;
}
return null;
} catch (_) {
return null;
}
}
/**
@@ -27029,11 +27082,26 @@ function setLocale(lang) {
/**
* Load locale from localStorage (called once at boot, before DOMContentLoaded).
* Server-persisted preference is applied later in loadSettingsPanel().
*
* #7622: on the FIRST visit (no stored preference), fall back to the
* browser's preferred language (`navigator.languages[0]` /
* `navigator.language`) so non-English speakers don't have to dig into
* Settings after every fresh install. This is strictly a first-visit
* hint — the moment the user picks a language in Settings (or the
* server-side preference is applied via `loadSettingsPanel`), the
* stored value takes over and the browser hint is never consulted
* again for that browser profile.
*/
function loadLocale() {
let stored = null;
try { stored = localStorage.getItem('hermes-lang'); } catch (_) {}
setLocale(resolvePreferredLocale(null, stored));
// Only consult the browser hint when no stored preference exists.
// The settings panel and `loadSettingsPanel()` will overwrite this
// hint on their first call, so explicit user / server preferences
// always win. #7622 (round 3): the browser-hint read is delegated
// to `_detectBrowserLanguageHint()` so a `navigator` access throw
// (some embedded webviews) can no longer abort loadLocale().
setLocale(resolvePreferredLocale(stored, _detectBrowserLanguageHint()));
}
/**
static/login.js
+9 -1
View File
@@ -7,7 +7,11 @@ document.addEventListener('DOMContentLoaded', function () {
var input = document.getElementById('pw');
var passkeyBtn = document.getElementById('passkey-login');
if (!form || !input) return;
// #7056: the password input is absent on a passwordless deployment (OIDC-only,
// or passkey-only). The form itself still renders and carries the i18n data
// attributes, and the passkey button below must still be wired up, so only the
// form is required here — every password-specific path is guarded individually.
if (!form) return;
var invalidPw = form.getAttribute('data-invalid-pw') || 'Invalid password';
var connFailed = form.getAttribute('data-conn-failed') || 'Connection failed';
@@ -59,6 +63,8 @@ document.addEventListener('DOMContentLoaded', function () {
async function doLogin(e) {
e.preventDefault();
// No password input on a passwordless deployment: nothing to submit.
if (!input) return;
var pw = input.value;
hideErr();
try {
@@ -147,12 +153,14 @@ document.addEventListener('DOMContentLoaded', function () {
passkeyBtn.addEventListener('click', doPasskeyLogin);
}
if (input) {
input.addEventListener('keydown', function (e) {
if (e.key === 'Enter') {
e.preventDefault();
doLogin(e);
}
});
}
// On page load, probe the server so we can distinguish "can't reach server"
// (Tailscale off, wrong network) from "session expired / need to log in".
static/messages.js
+163 -13
View File
@@ -1,3 +1,13 @@
const _AGENT_COMMAND_ALIASES = {
'reload_mcp': 'reload-mcp',
'reload_skills': 'reload-skills',
'codex_runtime': 'codex-runtime',
'credits': 'credits'
};
const _AGENT_COMMANDS_RUN_ON_WEBUI = new Set([
'reload-mcp','reload-skills','codex-runtime','credits',
'reload_mcp','reload_skills','codex_runtime','credits'
]);
function _markSessionViewed(sid, messageCount) {
if(typeof _setSessionViewedCount!=='function' || !sid) return;
const next = Number.isFinite(messageCount) ? Number(messageCount) : 0;
@@ -1184,7 +1194,6 @@ const _sessionTitleProvisionalBySid = new Map();
// their canonical command is registered on the backend (for example
// /reload-mcp). Keep this intentionally narrow and include underscore variants
// observed by users so typing either form still routes through executeAgentCommand.
const _AGENT_COMMANDS_RUN_ON_WEBUI = new Set(['reload-mcp', 'reload_mcp', 'reload-skills', 'reload_skills', 'codex-runtime', 'codex_runtime', 'credits']);
function _clearStaleBusyStateBeforeSend({compressionRunning=false}={}){
if(!S||!S.busy||compressionRunning) return false;
@@ -1287,6 +1296,31 @@ function applySessionTitleUpdate(sid, titleText, options={}){
// BEFORE slash rewrites (/moa, bundles) mutate the payload and BEFORE
// uploadPendingFiles() drains S.pendingFiles — so we restore what the user
// actually typed, not the transformed send payload.
async function _recoverCompressedSend(error,sid,draftText,filesSnapshot,clearPromise){
let payload;
try{ payload=JSON.parse(error&&error.body||'{}'); }catch(_){ return false; }
const target=payload&&payload.continuation_session_id;
if(!error||error.status!==409||!payload||payload.code!=='session_rotated'||typeof target!=='string'||!target||target===sid) return false;
// A failed POST has not admitted a turn. Never resend automatically: the
// continuation may already be busy, and attachments must remain a draft.
if(!S.session||S.session.session_id!==sid) return false;
delete INFLIGHT[sid];
if(typeof clearInflightState==='function') clearInflightState(sid);
if(typeof clearOptimisticSessionStreaming==='function') clearOptimisticSessionStreaming(sid);
stopApprovalPolling();stopClarifyPolling();removeThinking();setBusy(false);
try{
await loadSession(target);
if(!S.session||S.session.session_id===sid) return false;
// loadSession can lose its navigation race to another tab selection. Never
// place the rejected message into that unrelated session's composer.
_restoreComposerDraftAfterFailedSend(draftText,filesSnapshot,target,clearPromise);
if(S.session.session_id!==target) return true;
setComposerStatus('Session resumed. Your message is preserved; send it when ready.');
showToast('Session resumed after compression. Your draft is preserved.',4000);
return true;
}catch(_){ return false; }
}
function _restoreComposerDraftAfterFailedSend(draftText, filesSnapshot, sid, clearPromise){
const restore=String(draftText||'');
const files=Array.isArray(filesSnapshot)?filesSnapshot.filter(Boolean):[];
@@ -1334,7 +1368,7 @@ function _restoreComposerDraftAfterFailedSend(draftText, filesSnapshot, sid, cle
} else if(!restoredVisible){
// Background failure (sid was never the visible session): no live
// composer to read, so persist the captured snapshot — it's the only copy.
_saveComposerDraftNow(sid, restore, []);
_saveComposerDraftNow(sid, restore, files);
}
// else: restored the visible composer, then the user switched away — the
// session-switch save path already saved sid's composer; skip stale write.
@@ -1409,14 +1443,16 @@ async function send(){
if(!S.session){await newSession();await renderSessionList();}
// Busy-control slash commands must be intercepted HERE, before the
// defaultMessageMode routing block, so the user can always type /steer, /interrupt,
// /queue, /terminal, /goal, or /yolo while the agent is running and have
// /queue, /terminal, /goal, /yolo, or /stop while the agent is running and have
// them execute immediately.
// Without this intercept they fall through to the queue and execute after
// the current turn ends — by which point there is no active stream and
// cmdSteer / cmdInterrupt say "No active task to stop."
// /stop must cancel the active run immediately instead of being steered
// or queued as the literal text "/stop" (#6951).
if(text.startsWith('/')&&!literalSlash){
const _pc=typeof parseCommand==='function'&&parseCommand(text);
if(_pc&&['steer','interrupt','queue','terminal','goal','yolo'].includes(_pc.name)){
if(_pc&&['steer','interrupt','queue','terminal','goal','yolo','stop'].includes(_pc.name)){
const _bc=COMMANDS.find(c=>c.name===_pc.name);
if(_bc){
$('msg').value='';autoResize();
@@ -1839,6 +1875,7 @@ async function send(){
if(typeof renderSessionList==='function') void renderSessionList();
return;
}
if(await _recoverCompressedSend(e,activeSid,_failedSendDraftText,_failedSendFilesSnapshot,_composerDraftClearPromise)) return;
const conflictActiveStream=/session already has an active stream/i.test(errMsg);
if(conflictActiveStream){
delete INFLIGHT[activeSid];
@@ -2052,6 +2089,7 @@ function closeLiveStream(sessionId, streamId, source){
if(!live) return;
if(streamId&&live.streamId!==streamId) return;
if(source&&live.source!==source) return;
if(typeof live.cancelIdleRecovery==='function') live.cancelIdleRecovery();
// Snapshot the current live-turn DOM BEFORE tearing the stream down. The
// per-event snapshot (snapshotLiveTurn) only fires on content/tool_complete
// SSE events, so switching away during a quiet window (mid tool-exec, silent
@@ -4808,7 +4846,12 @@ function attachLiveStream(activeSid, streamId, uploaded=[], options={}){
function _smdMediaTailFlushEntry(entry){
const chunk=_smdMediaTailEntryChunk(entry);
if(!chunk) return;
const m=/^MEDIA:([^\s\)\]]+)$/.exec(String(chunk));
// #7680 re-gate (9/22): strip backtick wrappers so the bare-token
// match below sees a plain ``MEDIA:path`` and the bare class
// (no backtick in the exclusion set) captures the full filename
// even when the path itself contains a backtick.
const normalized = String(chunk).replace(/`MEDIA:([^`\s]+)`/g, 'MEDIA:$1');
const m=/^MEDIA:([^\s\)\]]+)$/.exec(normalized);
const emitted=!!(m && entry && entry.parent && _smdAppendMediaNode(entry.parent, m[1]));
if(!emitted && entry) _smdMediaWriteText(entry.parent, entry.data, entry.baseAddText, entry.writeText, chunk);
}
@@ -4856,23 +4899,29 @@ function attachLiveStream(activeSid, streamId, uploaded=[], options={}){
// Prose runs go through the owning text writer. MEDIA tokens go through
// the single-token DOMParser helper only after a delimiter or
// reliable filename suffix proves the ref is complete.
// #7680 re-gate (9/22): strip backtick wrappers first so the bare
// class (no backtick in the exclusion set) captures the full
// filename even when the path itself contains a backtick.
// The pre-pass replaces `` `MEDIA:path` `` with ``MEDIA:path``
// so the wrapped form is consumed before the bare scan.
const normalized = combined.replace(/`MEDIA:([^`\s]+)`/g, 'MEDIA:$1');
const re=/MEDIA:([^\s\)\]]+)/g;
let last=0, m;
let unmatchedTail=null;
while((m=re.exec(combined))){
while((m=re.exec(normalized))){
const matchEnd = m.index + m[0].length;
if(m.index>last){
const slice = combined.slice(last, m.index);
const slice = normalized.slice(last, m.index);
writeCurrent(slice);
}
if(matchEnd===combined.length && !_smdMediaRefHasReliableBoundary(m[1])){
const candidate = combined.slice(m.index);
if(matchEnd===normalized.length && !_smdMediaRefHasReliableBoundary(m[1])){
const candidate = normalized.slice(m.index);
if(candidate.length < _MEDIA_TAIL_MAX){
unmatchedTail = candidate;
} else {
writeCurrent(candidate);
}
last = combined.length;
last = normalized.length;
break;
}
if(!_smdAppendMediaNode(parent, m[1])) writeCurrent(m[0]);
@@ -4880,7 +4929,7 @@ function attachLiveStream(activeSid, streamId, uploaded=[], options={}){
}
// Tail buffer — hold trailing bytes that look like an unterminated
// MEDIA prefix; flush any prose before the partial MEDIA suffix.
const rest = combined.slice(last);
const rest = normalized.slice(last);
if(rest){
const tailMatch = /MEDIA:[^\s\)\]]*$/.exec(rest);
const prefixTail = tailMatch ? '' : _smdMediaPrefixTail(rest);
@@ -5708,12 +5757,82 @@ function attachLiveStream(activeSid, streamId, uploaded=[], options={}){
return true;
}
function _bindSidebarIdleRecovery(live){
let pending=null;
live.cancelIdleRecovery=()=>{
if(pending&&pending.timer) clearTimeout(pending.timer);
pending=null;
};
live.recoverFromSidebarIdle=()=>{
if(pending||_streamFinalized||_terminalStateReached||_pendingStreamEndRecovery) return;
const request={inflight:INFLIGHT[activeSid],timer:null};
const isCurrent=()=>pending===request&&!_streamFinalized&&!_terminalStateReached&&
LIVE_STREAMS[activeSid]===live&&live.source.readyState===1&&
S.session&&S.session.session_id===activeSid&&S.activeStreamId===streamId&&
INFLIGHT[activeSid]===request.inflight&&
!(typeof _sendInProgress!=='undefined'&&_sendInProgress&&activeSid===_sendInProgressSid);
pending=request;
// Give the independently delivered terminal frame a short handoff window,
// then use canonical session recovery even if the transport stays OPEN.
// One ticket spans both timer and request; list refreshes cannot extend it.
request.timer=setTimeout(async()=>{
request.timer=null;
if(!isCurrent()){
if(pending===request) live.cancelIdleRecovery();
return;
}
try{
let runtimeStatus=null;
try{
runtimeStatus=await api(
`/api/chat/stream/status?stream_id=${encodeURIComponent(streamId)}`,
{timeoutMs:8000,retries:0,timeoutToast:false}
);
}catch(_){
// Runtime ownership is authoritative here. If the probe itself
// fails, do not infer completion from already-cleared session
// fields; surface the existing interrupted-stream path instead.
if(isCurrent()) _handleStreamError(live.source);
return;
}
if(!isCurrent()) return;
if(!runtimeStatus||typeof runtimeStatus.active!=='boolean'){
_handleStreamError(live.source);
return;
}
// Gateway success writeback clears persisted active/pending fields
// before post-turn goal evaluation and terminal event emission.
// STREAMS-backed status therefore owns this decision: an exact active
// runtime must keep its OPEN browser handoff even if the sidebar row
// already looks idle.
if(runtimeStatus.active) return;
const status=await _restoreSettledSession(live.source,{
status:true,
isCurrent,
requestOptions:{timeoutMs:8000,retries:0,timeoutToast:false},
preserveVisibleOnShorterTerminalSnapshot:true,
});
// A stale sidebar response is not permission to terminate a worker
// which the authoritative session snapshot still reports as active.
if(isCurrent()&&status!=='restored'&&status!=='active') _handleStreamError(live.source);
}finally{
if(pending===request) live.cancelIdleRecovery();
}
},1500);
};
for(const event of ['done','cancel','apperror','stream_end','error']){
live.source.addEventListener(event,live.cancelIdleRecovery);
}
}
function _wireSSE(source){
const existingLive=LIVE_STREAMS[activeSid];
if(existingLive&&existingLive.source&&existingLive.source!==source){
if(typeof existingLive.cancelIdleRecovery==='function') existingLive.cancelIdleRecovery();
try{if(existingLive.source.readyState!==2)existingLive.source.close();}catch(_){ }
}
LIVE_STREAMS[activeSid]={streamId,source};
_bindSidebarIdleRecovery(LIVE_STREAMS[activeSid]);
// Note on #631 Bug B: the original PR description stated the server
// "replays buffered token events" on reconnect, and proposed resetting
@@ -6987,13 +7106,16 @@ function attachLiveStream(activeSid, streamId, uploaded=[], options={}){
async function _restoreSettledSession(source, options=null){
const returnStatus=!!(options&&options.status);
const isCurrent=options&&typeof options.isCurrent==='function'?options.isCurrent:null;
if(isCurrent&&!isCurrent()) return returnStatus?'stale':false;
const preserveVisibleOnShorterTerminalSnapshot=!!(options&&options.preserveVisibleOnShorterTerminalSnapshot);
if(_isActiveSession() && S.activeStreamId!==streamId){
_closeSource(source);
return returnStatus?'stale':false;
}
try{
const data=await api(`/api/session?session_id=${encodeURIComponent(activeSid)}`);
const data=await api(`/api/session?session_id=${encodeURIComponent(activeSid)}`,options&&options.requestOptions||{});
if(isCurrent&&!isCurrent()) return returnStatus?'stale':false;
// Opus #2852 race-fix: if a late `done` event ran the finalize path while
// we were awaiting the network roundtrip, bail out — done already settled.
if(_streamFinalized) return returnStatus?'restored':true;
@@ -8091,7 +8213,22 @@ function _startHiddenActiveStreamPoll(sid) {
if (S.activeStreamId) return; // already rendering; wait it out
try {
fetch(_apiUrl('api/session/status?session_id=' + encodeURIComponent(sid)), {credentials: 'same-origin'})
.then(r => r.ok ? r.json() : null)
.then(r => {
// #7299: 404 Not Found / 410 Gone are TERMINAL for this
// session-owned poll. The session has been deleted or no
// longer exists in the active state directory, so further
// polls are guaranteed to fail. Stop the poll immediately
// to avoid the infinite 404 loop on stale background tabs
// (one tab could fire ~10 such requests per minute; multiple
// tabs multiply the noise). Transient failures (5xx, rate
// limit, network error) keep polling — only the missing
// session itself is terminal.
if ((r.status === 404 || r.status === 410) && _sessionStreamHiddenPollSid === sid) {
_stopHiddenActiveStreamPoll();
return null;
}
return r.ok ? r.json() : null;
})
.then(d => {
if (!d || _sessionStreamHiddenPollSid !== sid) return;
const streamId = d.active_stream_id;
@@ -8951,6 +9088,19 @@ async function respondClarify(response) {
// not tear B down on A's late 409. The SSE/poll path will re-render the
// next prompt's card from scratch via ``showClarifyCard`` either way.
if (e && e.status === 409) {
// #7710: a cross-profile refusal now also arrives as 409
// (``session_profile_mismatch``). The prompt is NOT expired — the write
// was refused because the session belongs to another profile. Treating
// it as expired would hide a live clarification card and mislabel the
// cause, so leave the card standing and report the real reason.
if (typeof _sessionProfileMismatchFromError === 'function'
&& _sessionProfileMismatchFromError(e)) {
_clarifySetControlsDisabled(false, false);
if (typeof setStatus === "function") {
setStatus("Clarify: session belongs to a different profile");
}
return;
}
if (_clarifyId === clarifyId) {
// Same card still showing — dismiss it and rescue the typed draft.
// Order matters: ``_stashClarifyDraft`` (called from
static/panels.js
+10 -1
View File
@@ -9300,8 +9300,17 @@ async function loadSettingsPanel(){
_setHiddenTabs(hiddenTabs);
_applyTabVisibility(hiddenTabs);
_renderTabVisibilityChips();
// #7622 (round 3): the settings payload's `settings.language` is
// absent (None) for a fresh install, so an explicit non-empty
// value is the user's genuine saved choice. The browser
// navigator hint is now read via the guarded
// `_detectBrowserLanguageHint()` helper (round-3 finding 2) so
// a throwing `navigator` accessor can no longer abort settings
// hydration before model, provider, plugin and extension sections
// are populated. The fallback ternary preserves the pre-#7622
// settings-modal behaviour when neither helper is in scope.
const resolvedLanguage=(typeof resolvePreferredLocale==='function')
? resolvePreferredLocale(settings.language, localStorage.getItem('hermes-lang'))
? resolvePreferredLocale(settings.language, localStorage.getItem('hermes-lang'), _detectBrowserLanguageHint())
: (settings.language || localStorage.getItem('hermes-lang') || 'en');
// Keep settings modal and current page strings in sync with the resolved locale.
if(typeof setLocale==='function'){
static/sessions.js
+26 -1
View File
@@ -852,6 +852,14 @@ function _reconcileActiveSessionIdleStateFromList(serverRows) {
const serverRow=serverRows.find(s=>s&&s.session_id===sid);
if (!serverRow) return false;
if (!_isServerIdleSessionRow(serverRow)) return false;
// Sidebar idle metadata can beat the terminal frame on the independent chat
// SSE. Let its exact OPEN transport finish the Anchor handoff; orphaned or
// disconnected streams still use the existing idle recovery below.
if (_hasOwnedOpenLiveStream(sid)) {
const live=LIVE_STREAMS[sid];
if(typeof live.recoverFromSidebarIdle==='function') live.recoverFromSidebarIdle();
return false;
}
let changed=false;
if (S.busy) { S.busy=false; changed=true; }
if (S.activeStreamId) { S.activeStreamId=null; changed=true; }
@@ -940,6 +948,8 @@ function _purgeStaleInflightEntries() {
if (typeof _sendInProgress !== 'undefined' && _sendInProgress && sid === _sendInProgressSid) {
continue;
}
// The sidebar render must not purge what the idle reconciler preserved.
if (_hasOwnedOpenLiveStream(sid)) continue;
if (!sessionsById.has(sid)) {
const knownSource = sourceById ? sourceById.get(sid) : null;
if (currentSidebarSource && (!knownSource || knownSource !== currentSidebarSource)) {
@@ -961,6 +971,14 @@ function _purgeStaleInflightEntries() {
}
}
function _hasOwnedOpenLiveStream(sid) {
if (typeof S === 'undefined' || !S || !S.session || S.session.session_id !== sid || !S.activeStreamId) return false;
const live = typeof LIVE_STREAMS === 'object' && LIVE_STREAMS ? LIVE_STREAMS[sid] : null;
// readyState 1 is EventSource.OPEN. A cached stream ID or busy flag alone
// is not ownership and must never disable recovery for a stuck indicator.
return Boolean(live && live.streamId === S.activeStreamId && live.source && live.source.readyState === 1);
}
function _rememberSessionListSource(s, sid = null, allowScopeFallback = true) {
const resolvedSid = sid || (s && s.session_id);
if (!resolvedSid) return;
@@ -2169,6 +2187,12 @@ async function loadSession(sid){
S.activeStreamId=activeStreamId;
const liveToolReplayId=(tc)=>String(tc&&(tc.tid||tc.id||tc.tool_call_id||tc.tool_use_id||tc.call_id||'')||'').trim();
const replayPersistedLiveToolCards=(opts)=>{
// The journal-backed Anchor scene is authoritative through its resume
// cursor; newer rows arrive through the reattached SSE stream. Replaying
// the older INFLIGHT tool cache after a successful scene restore would
// redraw all N rows N times. Keep the #3707 replay only for legacy HTML
// restoration or a failed/unavailable scene render.
if(restoredAnchorScene) return;
const liveToolCalls=Array.isArray(S.toolCalls)
? S.toolCalls
: (Array.isArray(INFLIGHT[sid]&&INFLIGHT[sid].toolCalls)?INFLIGHT[sid].toolCalls:[]);
@@ -5272,7 +5296,8 @@ function _shouldKeepLocalOnlyOptimisticSessionRow(local){
function _dropStaleOptimisticSessionRow(sid){
if(!sid) return;
if(typeof _rememberSessionListSource==='function') _rememberSessionListSource(null, sid, false);
if(INFLIGHT&&INFLIGHT[sid]){
// Retiring sidebar optimism must not retire the independent chat owner.
if(INFLIGHT&&INFLIGHT[sid]&&!_hasOwnedOpenLiveStream(sid)){
delete INFLIGHT[sid];
if(typeof clearInflightState==='function') clearInflightState(sid);
}
static/ui.js
+247 -6
View File
@@ -508,6 +508,16 @@ async function startCompressionRecovery(btn){
const composer=$('msg');
if(composer&&typeof composer.focus==='function') composer.focus();
}catch(e){
// #7710: a cross-profile refusal now also arrives as 409
// (``session_profile_mismatch``). That is NOT a stale recovery action —
// the card is still valid, the request was simply refused because the
// session belongs to another profile. Retiring it would hide a live card
// and show a false "conversation already moved on" note.
if(e&&e.status===409&&typeof _sessionProfileMismatchFromError==='function'
&&_sessionProfileMismatchFromError(e)){
if(typeof setStatus==='function') setStatus('Session belongs to a different profile');
return;
}
// A 409 means this session no longer has an active recovery action (the
// session already moved on — e.g. a substantive prompt cleared it). The
// persisted card in the transcript is stale, so retire it and show a neutral
@@ -3939,6 +3949,27 @@ function _isEquivalentConfiguredModelEntry(modelId,badge,entries){
// providers (@custom:name:model) without collapsing matching model IDs from
// different providers.
const rawId=String(modelId||'');
// `<provider>/<model>` is another routing spelling of `<model>`, so its badge
// key must not become a second picker row. configured_model_badges holds every
// spelling of a configured model and renderModelDropdown() synthesises a row
// for each key this predicate does not recognise. The provider-qualified
// spelling was missed because _normalizeConfiguredModelKey() strips only one
// leading slash segment (#3360 keeps `vendor_a/x` and `vendor_b/y/x` distinct),
// so `acme/example-model` and `custom/acme/example-model` normalise to
// different keys and the picker lists one model twice.
// Match it the way the `@provider:` rule below does: the badge declares a
// provider, the key starts with that provider's `<provider>/` prefix, and an
// existing row from the same provider normalises equal to the remainder. Two
// different models never satisfy the last clause, so this can only drop a
// duplicate of a row the catalog already produced.
const slashPrefix=provider?`${provider}/`:'';
if(slashPrefix&&rawId.toLowerCase().startsWith(slashPrefix)){
const slashRoutedId=rawId.slice(slashPrefix.length);
if(slashRoutedId&&(entries||[]).some(entry=>
String(entry.providerId||'').toLowerCase()===provider
&&_normalizeConfiguredModelKey(entry.value)===_normalizeConfiguredModelKey(slashRoutedId)
)) return true;
}
const prefix=provider?`@${provider}:`:'';
if(!prefix||!rawId.toLowerCase().startsWith(prefix)) return false;
const routedId=rawId.slice(prefix.length);
@@ -7692,6 +7723,16 @@ function renderMd(raw){
// generated images) and replace them with inline <img> or download links.
// Stashed so the path/URL is never processed as markdown.
const media_stash=[];
// #7680 re-gate (9/22): two-pass scan.
// 1. `` `MEDIA:path` `` (backtick-wrapped, inline-code form) → strip
// the wrapping backticks so the bare-token pass below sees a
// plain ``MEDIA:path`` and the closing backtick is not consumed
// as part of the path.
// 2. ``MEDIA:[^\s\)\]]+`` (bare, no backtick in the exclusion
// class) so a filename that legally contains a backtick
// (``report`final.png``) is captured in full instead of being
// truncated at the first backtick.
s=s.replace(/`MEDIA:([^`\s]+)`/g,'MEDIA:$1');
s=s.replace(/MEDIA:([^\s\)\]]+)/g,(_,raw_ref)=>{
media_stash.push(raw_ref);
return '\x00D'+(media_stash.length-1)+'\x00';
@@ -9139,6 +9180,12 @@ function _stripForTTS(text){
// Strip links, keep text
text=text.replace(/\[([^\]]+)\]\([^)]+\)/g,'$1');
// Replace MEDIA: paths with a simple label
// #7680 re-gate (9/22): TTS does not need the backtick-aware
// terminator. Inline code was already stripped at the top of this
// function (`:9137`), so a bare ``MEDIA:[^\s]+`` is correct and
// round-trips the original filename including a backtick in the
// path — the only thing the TTS ever does with the captured
// substring is throw it away.
text=text.replace(/MEDIA:[^\s]+/g,'a file');
// Strip emoji and emoticons
text=text.replace(/[\u{1F600}-\u{1F64F}\u{1F300}-\u{1F5FF}\u{1F680}-\u{1F6FF}\u{1F1E0}-\u{1F1FF}\u{2600}-\u{26FF}\u{2700}-\u{27BF}\u{FE00}-\u{FE0F}\u{1F900}-\u{1F9FF}\u{1FA00}-\u{1FA6F}\u{1FA70}-\u{1FAFF}\u{200D}]/gu,'');
@@ -9918,6 +9965,99 @@ function _liveAssistantSegmentTextLength(seg){
return String(body.textContent||'').trim().length;
}
// ── #6948 follow-up: settled-transcript ownership of a live-turn node ──────
// (duplicate assistant answer; upstream symptom report #2051)
//
// Both live-turn insertion paths — the renderMessages() re-attach and
// restoreLiveTurnHtmlForSession() — have one branch that ADDS a turn the
// settled rebuild did not produce. The assistant row is persisted a few ms
// BEFORE the stream's terminal event clears S.activeStreamId, so a dead live
// node left behind by an INFLIGHT entry that outlived its stream is re-attached
// on top of the settled transcript: the same answer twice, self-sustaining
// across later renders (the node keeps id=liveAssistantTurn and is re-preserved
// by the #6948 guard), healed only by a reload.
//
// Dropping a live node must be an OWNERSHIP proof, never "the rendered text
// looks the same":
// * the live body is built by the streaming smd parser and the settled body
// by renderMd + post-processing, so their textContent differs for anything
// past single-paragraph plain text (lists, fenced code with its language
// label, tables, `_underscore_` emphasis, the injected copy button) — a
// text comparison would miss most real answers; and
// * an identical answer earlier in the history (a repeated "Done.", a
// greeting, a same-prompt resend) would delete a GENUINELY LIVE turn: while
// the current turn is unpersisted, the last settled assistant message IS the
// previous turn's answer.
// So ownership is proved from state: the transcript must END with a settled
// assistant message that THIS stream produced, and the node must carry nothing
// the settled rebuild could not have produced.
// Stream identity persisted on a settled assistant message. The server stamps
// _anchor_stream_id from the anchor-scene sidecar record (api/routes.py); the
// client stamps _anchor_stream_id and scene.identity.stream_id at stream end
// (_attachProjectedAnchorSceneToLastAssistant, static/messages.js).
function _settledAssistantStreamId(message){
if(!message) return '';
const scene=message._anchor_activity_scene||null;
const identity=(scene&&scene.identity)||null;
return String(message._anchor_stream_id||(scene&&scene.stream_id)||(identity&&identity.stream_id)||'');
}
// The live-projection markers the #6948 preserve guard already treats as proof
// of a live owner. A message the client still considers live is never evidence
// that the turn has settled.
function _messageHasLiveAssistantProjection(m){
return !!(m&&m.role==='assistant'&&(m._live||m._activityBurstId!==undefined||m._liveSegmentSeq!==undefined));
}
// True when the live turn holds something the settled rebuild cannot have
// produced from S.messages — an unpersisted tool card, reasoning/thinking row,
// transparent-stream row, or a second live segment. Such a node is never
// dropped: #3714's whole premise is that the live DOM can be AHEAD of
// S.messages, and a tail-segment match must not discard the tool card or the
// earlier segment above it. An unknown node shape fails closed (keep the turn).
function _liveTurnCarriesUnsettledContent(turn){
if(!turn||typeof turn.querySelectorAll!=='function') return true;
if(turn.querySelectorAll(
'.tool-card-row,.wl-reason,.agent-activity-thinking,.thinking-card-row,.transparent-event-row'
).length) return true;
return turn.querySelectorAll('[data-live-assistant="1"]').length>1;
}
function _settledTranscriptOwnsLiveTurn(sid, turn){
if(!turn) return false;
const msgs=(typeof S!=='undefined'&&S&&Array.isArray(S.messages))?S.messages:null;
if(!msgs||!msgs.length) return false;
// The transcript must END with a settled assistant answer. A trailing user
// turn means the live turn IS the current turn — nothing of it is persisted
// yet, so it can never be a leftover (this is the case a text comparison gets
// wrong when the previous answer happens to read the same), and a live
// projection anywhere means the rebuild still owns a live turn of its own.
const last=msgs[msgs.length-1];
if(!last||last.role!=='assistant') return false;
if(msgs.some(_messageHasLiveAssistantProjection)) return false;
const inflight=(typeof INFLIGHT!=='undefined'&&INFLIGHT)?INFLIGHT[sid]:null;
// Owner of the live DOM: INFLIGHT carries the id of the stream that built it
// (attachLiveStream), and S.activeStreamId is that same id until the terminal
// event clears it.
const liveStreamId=String((inflight&&inflight.streamId)||(typeof S!=='undefined'&&S&&S.activeStreamId)||'');
const settledStreamId=_settledAssistantStreamId(last);
if(settledStreamId){
// Identity recorded on both sides decides — renderer- and text-independent,
// so code blocks, lists and tables are handled like plain prose.
if(!liveStreamId||settledStreamId!==liveStreamId) return false;
}else{
// No persisted identity on the tail (a turn whose scene was not worklog
// worthy). Compare SOURCE with SOURCE — the markdown this stream produced
// (INFLIGHT.lastAssistantText) against the persisted message content —
// never two independently rendered DOM trees. Regeneration cannot reach
// this branch: startRegeneration() truncates S.messages at the user turn,
// so the tail is a user message while a regenerated answer streams.
const streamed=String((inflight&&inflight.lastAssistantText)||'').replace(/\s+/g,' ').trim();
if(!streamed) return false;
const settled=String((typeof msgContent==='function'?msgContent(last):last.content)||'').replace(/\s+/g,' ').trim();
if(!settled||streamed!==settled) return false;
}
return !_liveTurnCarriesUnsettledContent(turn);
}
function _mergeRestoredLiveAssistantSegment(restored, existing){
if(!restored||!existing) return;
const existingLive=existing.querySelector('[data-live-assistant="1"]');
@@ -9954,6 +10094,18 @@ function restoreLiveTurnHtmlForSession(sid){
const existing=$('liveAssistantTurn');
_mergeRestoredLiveAssistantSegment(restored, existing);
if(existing) existing.replaceWith(restored);
// #6948 follow-up (duplicate assistant answer; #2051): only this branch ADDS a
// turn — replacing an existing live turn cannot duplicate. When the settled
// transcript already owns the stream this snapshot belongs to, appending it
// pins a SECOND copy of the same answer under the settled one. Release the
// stale snapshot (no other consumer can use it — it would be re-refused on
// every later restore) and report "nothing restored", so loadSession takes the
// same path it already takes for an INFLIGHT entry that never carried one.
else if(typeof _settledTranscriptOwnsLiveTurn==='function'
&&_settledTranscriptOwnsLiveTurn(sid, restored)){
inflight.liveTurnHtml=null;
return false;
}
else inner.appendChild(restored);
// Transparent Stream: liveTurnHtml is restored via template.innerHTML, which
// drops the property-bound onclick/onkeydown handlers wired by
@@ -11459,6 +11611,15 @@ function _assistantMessageBelongsInWorklog(m, rawIdx, toolCallAssistantIdxs, vis
const isTurnFinalAssistant=!!(opts&&opts.isTurnFinalAssistant);
const visibleText=String(visibleContent!==undefined?visibleContent:msgContent(m)||'').trim();
const hasVisibleText=!!visibleText&&!_isAssistantEmptyPlaceholderContent(m, visibleText);
// The caller only consults this predicate once the turn has settled (it gates
// on `!S.busy`), so an `_live` marker seen here is a leftover of the
// live-snapshot projection, not an ongoing stream. Folding on it hid the turn's
// final answer inline and echoed it into the Worklog, leaving the turn with no
// visible content until the #3875 blank-turn fail-safe revealed both copies. A
// turn-final assistant message with visible text is the answer, so keep it out
// of the fold; every other `_live` message folds as before. Kept as a separate
// guard so the live rule below keeps its position.
if(m._live&&hasVisibleText&&isTurnFinalAssistant) return false;
if(m._live) return true;
if(hasVisibleText&&m._anchor_activity_scene) return false;
if(hasVisibleText&&isTurnFinalAssistant) return false;
@@ -13141,6 +13302,17 @@ function _anchorSceneRowsForRendering(scene, opts){
const rows=Array.isArray(scene&&scene.activity_rows)?scene.activity_rows:[];
const settled=!!(opts&&opts.settled);
const live=!settled;
// #6948 follow-up (duplicate assistant answer; #2051): on a SETTLED turn the
// assistant segment owns the final answer, so a process_prose row carrying
// that same text must not also be rebuilt into the activity scene. The
// transparent-stream renderer suppresses it per row
// (_anchorSceneTransparentNodeForRow), but the compact-worklog row builder
// (_anchorSceneNodeForRow) has no such guard and renders the row as a second
// .assistant-segment above the settled one. Drop it here — the single place
// BOTH settled renderers (and the #5839 deferred-rows path) get their rows.
// LIVE rendering is untouched: while streaming, the inline live segment is
// hidden and the prose row IS the visible answer.
const settledFinalAnswer=settled?String((scene&&scene.final_answer)||'').trim():'';
const out=[];
const byKey=new Map();
const liveProseTextKeys=new Map();
@@ -13163,6 +13335,9 @@ function _anchorSceneRowsForRendering(scene, opts){
if(_anchorSceneIsSettledSuccessfulCompression(row,settled)) continue;
const text=String(row.text||'').trim();
if((row.role==='prose'||row.role==='thinking')&&!text) continue;
if(settledFinalAnswer&&row.role==='prose'
&&typeof _anchorSceneProseDuplicatesFinalAnswer==='function'
&&_anchorSceneProseDuplicatesFinalAnswer(text,settledFinalAnswer)) continue; // #6948 follow-up
const key=keyFor(row);
if(byKey.has(key)){
const index=byKey.get(key);
@@ -13437,6 +13612,26 @@ function _anchorSceneProseMatchesFinalAnswer(proseText, finalAnswer){
const shorter=Math.min(a.length,b.length), longer=Math.max(a.length,b.length);
return shorter>=80 && (shorter/longer)>=0.9;
}
// #6948 follow-up (duplicate assistant answer; #2051): the same near-equality
// test as _anchorSceneProseMatchesFinalAnswer, minus its absolute `shorter>=80`
// floor. That floor makes the matcher a no-op for every SHORT final answer
// (greetings, "Done.", one-liners), so a prose row that is the answer minus its
// last streamed token — e.g. "Hey! What can I help you with today" vs
// "...today?" — was rebuilt as a SECOND assistant-segment beside the settled
// one. The >=0.9 length ratio is what actually keeps a distinct short
// intermediate sentence from being swallowed by a long final answer (Codex
// #4568), so it alone is kept. Deliberately a separate helper: the per-row
// render-time matcher above, and the tests pinning its exact predicate, stay
// untouched.
function _anchorSceneProseDuplicatesFinalAnswer(proseText, finalAnswer){
if(_anchorSceneProseMatchesFinalAnswer(proseText,finalAnswer)) return true;
const norm=(s)=>String(s||'').replace(/\s+/g,' ').trim();
const a=norm(proseText), b=norm(finalAnswer);
if(!a||!b) return false;
if(!(a.startsWith(b)||b.startsWith(a))) return false;
const shorter=Math.min(a.length,b.length), longer=Math.max(a.length,b.length);
return (shorter/longer)>=0.9;
}
function _anchorSceneWorklogGroup(blocks, opts){
if(!blocks) return null;
const live=!!(opts&&opts.live);
@@ -16912,6 +17107,23 @@ function _processWakeupCardHtml(info, rawText, extras){
return `<details class="process-wakeup-card"><summary class="process-wakeup-summary"><span class="process-wakeup-toggle">${li('chevron-right',12)}</span><span class="process-wakeup-label">${li('terminal',13)}<span>${esc(t('process_wakeup_label'))}</span></span>${cmdHtml}${chip}${extras.timeHtml||''}</summary><div class="process-wakeup-detail">${extras.filesHtml||''}${patternRow}${cmdRow}<div class="msg-body process-wakeup-body">${outHtml}</div>${extras.footHtml||''}</div></details>`;
}
// #2051: parse into a <template> and move the nodes instead of insertAdjacentHTML —
// every step is idempotent, so a DOM-API wrapper (e.g. an anti-fingerprinting
// extension) that executes the call twice cannot duplicate the block.
function _insertSegmentBlock(seg, html){
if(!seg) return;
if(typeof document!=='undefined'&&typeof document.createElement==='function'){
try{
const tpl=document.createElement('template');
if('content' in tpl){
tpl.innerHTML=html;
seg.appendChild(tpl.content);
return;
}
}catch(_){ /* fall through to the string path below */ }
}
seg.insertAdjacentHTML('beforeend', html);
}
function renderMessages(options){
_lastMessageRenderAt=performance.now();
const preserveScroll=!!(options&&options.preserveScroll);
@@ -17676,7 +17888,7 @@ function renderMessages(options){
if(isLastTextPart&&statusHtml){
orderedSeg.insertAdjacentHTML('beforeend', statusHtml);
}
orderedSeg.insertAdjacentHTML('beforeend', `${isLastTextPart?filesHtml:''}<div class="msg-body">${(typeof m!=='undefined'&&m&&m._media_snapshots&&typeof m._media_snapshots==='object')?_stampMediaSnapshots(partBodyHtml,m._media_snapshots):partBodyHtml}</div>${isLastTextPart?footHtml:''}`);
_insertSegmentBlock(orderedSeg, `${isLastTextPart?filesHtml:''}<div class="msg-body">${(typeof m!=='undefined'&&m&&m._media_snapshots&&typeof m._media_snapshots==='object')?_stampMediaSnapshots(partBodyHtml,m._media_snapshots):partBodyHtml}</div>${isLastTextPart?footHtml:''}`);
blocks.appendChild(orderedSeg);
if(!firstSeg) firstSeg=orderedSeg;
});
@@ -17736,9 +17948,9 @@ function renderMessages(options){
const hasVisibleBody=!!(String(content||'').trim()||filesHtml||recoveryHtml);
if(statusHtml){
seg.insertAdjacentHTML('beforeend', statusHtml);
if(hasVisibleBody) seg.insertAdjacentHTML('beforeend', `${filesHtml}<div class="msg-body">${bodyHtml}</div>${footHtml}`);
if(hasVisibleBody) _insertSegmentBlock(seg, `${filesHtml}<div class="msg-body">${bodyHtml}</div>${footHtml}`);
}else if(hasVisibleBody){
seg.insertAdjacentHTML('beforeend', `${filesHtml}<div class="msg-body">${bodyHtml}</div>${footHtml}`);
_insertSegmentBlock(seg, `${filesHtml}<div class="msg-body">${bodyHtml}</div>${footHtml}`);
}else if(!(thinkingText&&window._showThinking!==false&&!isSimplifiedToolCalling())){
seg.classList.add('assistant-segment-anchor');
}
@@ -18161,12 +18373,15 @@ function renderMessages(options){
if(!cards.length&&!anchorReasonHtml&&!thinkingText) continue;
const anchorTurn=anchorRow.closest('.assistant-turn');
if(!anchorTurn) continue;
// Hoisted out of the `if(!state)` block below (same expression, same
// value) so the append path can use the ownership fact the group
// construction already uses.
const anchorIsWorklogSource=anchorRow.classList&&anchorRow.classList.contains('assistant-segment-worklog-source');
let state=activityByTurn.get(anchorTurn);
if(!state){
const includeTurnDuration=!durationAssignedTurns.has(anchorTurn);
if(includeTurnDuration) durationAssignedTurns.add(anchorTurn);
const activityKey=`assistant:${aIdx}`;
const anchorIsWorklogSource=anchorRow.classList&&anchorRow.classList.contains('assistant-segment-worklog-source');
const group=ensureActivityGroup(anchorParent,{
collapsed:true,
anchor:anchorRow,
@@ -18186,7 +18401,14 @@ function renderMessages(options){
state.cards.push(...cards);
_appendWorklogStep(state.group, anchorRow, cards, thinkingText, {
live:false,
includeAnchorReason:!!includeAnchorReason&&!!anchorReasonHtml,
// Echo an anchor's prose as a `.wl-reason` row only when that anchor was
// folded into this Worklog. `assistant-segment-worklog-source` is the
// proof, and its `display:none` is the only reason the echo is not a
// second visible copy. The group construction above already reasons
// that way (`syncAnchorReason`); the append path did not, so an anchor
// that escapes the fold (the turn-final answer, an `_error` message)
// had its text rendered both inline and inside the Worklog.
includeAnchorReason:!!includeAnchorReason&&!!anchorReasonHtml&&!!anchorIsWorklogSource,
thinkingKey:thinkingText?`thinking:${_normalizeThinkingEchoCompare(thinkingText)}`:'',
thinkingDisclosureKey:thinkingText?`thinking:${entry.key}`:'',
seenReasons:state.seenReasons,
@@ -18475,6 +18697,15 @@ function renderMessages(options){
const groups=turn.querySelectorAll('.tool-worklog-group,.tool-call-group');
let revealed=false;
for(const group of groups){
// A settled Worklog whose rows are still deferred (#5839) has an empty
// textContent but is not empty in substance. Judging it empty here drops
// through to the last-resort un-hide below, and the deferred rows then
// materialize the same prose beside the segments it just un-hid.
// Materialize first, then judge.
if(group.getAttribute&&group.getAttribute('data-worklog-rows-deferred')==='1'
&&typeof _materializeDeferredWorklogRows==='function'){
_materializeDeferredWorklogRows(group);
}
if(!(group.textContent||'').trim()) continue; // empty group can't help
if(group.classList.contains('tool-call-group-collapsed')){
group.classList.remove('tool-call-group-collapsed');
@@ -18593,7 +18824,17 @@ function renderMessages(options){
// restore the whole preserved turn so nothing the user saw vanishes.
if(S.session) _preservedLiveTurn.dataset.sessionId=S.session.session_id;
_rebuilt.replaceWith(_preservedLiveTurn);
}else{
}else if(!(typeof _settledTranscriptOwnsLiveTurn==='function'
&&_settledTranscriptOwnsLiveTurn(sid,_preservedLiveTurn))){
// #6948 follow-up (duplicate assistant answer; #2051): this is the
// only branch that ADDS a turn — the rebuild produced no live turn of
// its own. When the settled transcript already ends with THIS stream's
// own answer and the preserved node carries nothing unpersisted, the
// node is a dead leftover (the row persisted and the turn settled while
// INFLIGHT[sid] was not yet cleaned) and appending pins a SECOND copy
// that re-preserves itself on every later render until a reload.
// Mid-stream the transcript ends with the user turn (or still carries a
// live projection), so #3877 preservation is untouched.
if(S.session) _preservedLiveTurn.dataset.sessionId=S.session.session_id;
inner.appendChild(_preservedLiveTurn);
}
tests/browser_conversation_lifecycle.py
+73 -5
View File
@@ -39,6 +39,7 @@ SCENARIO = os.environ.get("LIFECYCLE_SCENARIO", "normal").strip() or "normal"
TOOL_NAME = "read_file"
TOOL_ID = "lifecycle-tool-1"
TEST_BITE = os.environ.get("LIFECYCLE_TEST_BITE", "").strip()
_FRAME_PROOF_BITES = {"settle-worklog-frame-proof", "sidebar-idle-missing-terminal"}
GATEWAY_ACTIVITY_TIMEOUT = 60.0
ANCHOR_SCENE_PERSIST_TIMEOUT = 60.0
ANCHOR_SCENE_PROJECTION_TIMEOUT = 10_000
@@ -267,8 +268,17 @@ def _start_webui_server(repo_root: Path, env: dict, artifact_dir: Path):
suffix = "" if attempts == 1 else f"-attempt-{attempt + 1}"
log_path = artifact_dir / f"server{suffix}.log"
log = log_path.open("w", encoding="utf-8")
command = [sys.executable, str(repo_root / "server.py")]
barrier_dir = env.get("HERMES_TEST_TERMINAL_BARRIER_DIR")
if barrier_dir:
command = [
sys.executable,
str(repo_root / "tests" / "sidebar_terminal_barrier_server.py"),
str(repo_root),
barrier_dir,
]
proc = subprocess.Popen(
[sys.executable, str(repo_root / "server.py")],
command,
cwd=repo_root,
env=run_env,
stdout=log,
@@ -286,6 +296,50 @@ def _start_webui_server(repo_root: Path, env: dict, artifact_dir: Path):
raise RuntimeError(f"WebUI server did not become healthy{detail}")
def _exercise_sidebar_terminal_handoff(page, barrier_dir: Path, artifact_dir: Path):
"""Order the real idle-list response before the real terminal SSE write."""
deadline = time.monotonic() + 10
while not (barrier_dir / "ready").exists():
if time.monotonic() >= deadline:
raise AssertionError("terminal HTTP barrier was not reached")
page.wait_for_timeout(10)
observation = page.evaluate(
"""async () => {
const sid = S.session.session_id;
const stream = S.activeStreamId;
const inflight = INFLIGHT[sid];
const source = LIVE_STREAMS[sid] && LIVE_STREAMS[sid].source;
const registries = window._liveAnchorRegistries;
const registry = registries && registries.get(stream);
await renderSessionList();
await new Promise(resolve => requestAnimationFrame(() => requestAnimationFrame(resolve)));
const live = document.querySelector('#liveAssistantTurn');
return {
serverIdle: _allSessions.some(row => row.session_id === sid && _isServerIdleSessionRow(row)),
paneRetained: S.busy === true && S.activeStreamId === stream,
inflightRetained: !!inflight && INFLIGHT[sid] === inflight,
sourceRetained: !!source && source.readyState === EventSource.OPEN &&
LIVE_STREAMS[sid] && LIVE_STREAMS[sid].source === source,
registryRetained: !!registry && registries.get(stream) === registry,
liveRows: live ? live.querySelectorAll('[data-anchor-scene-row="1"]').length : 0,
};
}"""
)
(artifact_dir / "sidebar-before-terminal.json").write_text(
json.dumps(observation, indent=2), encoding="utf-8"
)
page.screenshot(path=str(artifact_dir / "sidebar-before-terminal.png"), full_page=True)
for invariant in ("serverIdle", "paneRetained", "inflightRetained", "sourceRetained", "registryRetained"):
assert observation[invariant] is True, {
"message": "idle sidebar response revoked the live terminal handoff",
"invariant": invariant,
"observation": observation,
}
assert observation["liveRows"] > 0, observation
if TEST_BITE != "sidebar-idle-missing-terminal":
(barrier_dir / "release").touch()
class DeterministicGateway:
"""A localhost-only Gateway Runs server with test-controlled phase gates."""
@@ -733,11 +787,12 @@ def main() -> int:
"drop-anchor-persistence",
"drop-terminal-anchor-row",
"settle-worklog-frame-proof",
"sidebar-idle-missing-terminal",
}:
raise ValueError(
f"Unsupported LIFECYCLE_TEST_BITE {TEST_BITE!r}; "
"expected one of '', 'drop-anchor-persistence', "
"'drop-terminal-anchor-row', 'settle-worklog-frame-proof'"
"'drop-terminal-anchor-row', 'settle-worklog-frame-proof', 'sidebar-idle-missing-terminal'"
)
if TEST_BITE == "drop-terminal-anchor-row" and scenario != "terminal-error":
raise ValueError(
@@ -782,6 +837,12 @@ def main() -> int:
"NO_PROXY": "127.0.0.1,localhost",
"no_proxy": "127.0.0.1,localhost",
})
env.pop("HERMES_TEST_TERMINAL_BARRIER_DIR", None)
terminal_barrier_dir = None
if TEST_BITE in _FRAME_PROOF_BITES and scenario == "normal":
terminal_barrier_dir = state_dir / "terminal-barrier"
terminal_barrier_dir.mkdir()
env["HERMES_TEST_TERMINAL_BARRIER_DIR"] = str(terminal_barrier_dir)
proc = None
log = None
log_path = None
@@ -873,7 +934,7 @@ def main() -> int:
# must never expose an expanded empty Worklog shell or an activity-less
# gap. Sample after rAF callbacks for a frame have drained; DOM mutations
# within one task or earlier rAF callbacks are not user-visible frames.
if TEST_BITE == "settle-worklog-frame-proof":
if TEST_BITE in _FRAME_PROOF_BITES:
page.evaluate(
"""() => {
const samples = [];
@@ -941,12 +1002,17 @@ def main() -> int:
timeout=10000,
)
gateway.release_terminal.set()
if terminal_barrier_dir is not None:
_exercise_sidebar_terminal_handoff(page, terminal_barrier_dir, artifact_dir)
page.wait_for_function(
"""text => typeof S !== 'undefined' && S.busy === false && !S.activeStreamId &&
((document.querySelector('#msgInner') || {}).innerText || '').includes(text)""",
arg=FINAL_TEXT,
timeout=15000,
timeout=11000 if TEST_BITE == "sidebar-idle-missing-terminal" else 15000,
)
if TEST_BITE == "sidebar-idle-missing-terminal":
assert not (terminal_barrier_dir / "release").exists(), "terminal frame was unexpectedly released"
print("OK missing terminal: canonical snapshot settled the still-OPEN transport before terminal delivery")
else:
gateway.release_terminal.set()
page.wait_for_function(
@@ -958,7 +1024,7 @@ def main() -> int:
)
session_id = page.evaluate("S.session && S.session.session_id")
assert session_id, "active session id missing after settlement"
if TEST_BITE == "settle-worklog-frame-proof":
if TEST_BITE in _FRAME_PROOF_BITES:
page.wait_for_timeout(100)
settle_proof = page.evaluate(
"window.__lifecycleSettleFrameProof && window.__lifecycleSettleFrameProof.stop()"
@@ -1176,6 +1242,8 @@ def main() -> int:
exit_code = 1
return 1
finally:
if terminal_barrier_dir is not None:
(terminal_barrier_dir / "release").touch()
gateway.close()
if browser is not None:
browser.close()
tests/browser_reconnect_scene_redraw.py
+171
View File
@@ -0,0 +1,171 @@
#!/usr/bin/env python3
"""Real page/loadSession/renderers with deterministic session and SSE fixtures.
No agent, provider, production state, or credentials. Tests DOM reconstruction,
not a real provider/network reconnect.
"""
import json
import os
import tempfile
import time
from pathlib import Path
from urllib.parse import parse_qs, urlsplit
from playwright.sync_api import sync_playwright
from browser_conversation_lifecycle import _start_webui_server, _terminate_process
ROOT = Path(os.environ.get('WEBUI_TEST_ROOT', Path(__file__).resolve().parent.parent))
def fixture(count):
tools = [dict(name='terminal', tid=f'tool-{i}', args={'command': f'printf result-{i}'},
preview=f'result-{i}', snippet=f'result-{i}', done=True) for i in range(count)]
rows = [dict(row_id=f'tool:{t["tid"]}', local_id=t['tid'], role='tool', kind='tool_call',
source_event_type='tool_complete', status='completed', order_index=i,
tool=t, payload=t) for i, t in enumerate(tools)]
return dict(stream_id='run-fixture', last_seq=1000, last_event_id='run-fixture:1000',
messages=[], tool_calls=tools, last_assistant_text='', last_reasoning_text='',
anchor_activity_scene=dict(version='activity_scene_v1',
identity=dict(session_id='fixture', stream_id='run-fixture', run_id='run-fixture'),
activity_rows=rows))
INIT = """
// A controlling service worker bypasses Playwright's page route fixtures in
// WebKit. This test targets reconnect rendering, not PWA cache behavior.
if(window===window.top && 'serviceWorker' in navigator){
navigator.serviceWorker.register=()=>Promise.reject(new Error('Disabled in reconnect harness'));
}
window.fixtureSources=[];
class FixtureEventSource {
static OPEN=1; static CONNECTING=0; static CLOSED=2;
constructor(url){this.url=String(url);this.readyState=1;this.listeners={};window.fixtureSources.push(this);}
addEventListener(name,fn){(this.listeners[name]||=[]).push(fn);}
removeEventListener(){}
close(){this.readyState=2;}
emit(name,data,id){for(const fn of this.listeners[name]||[])fn({data:JSON.stringify(data),lastEventId:id||''});}
}
window.EventSource=FixtureEventSource;
"""
def session_route(session, sid, workspace):
def handle(route):
query = parse_qs(urlsplit(route.request.url).query)
requested = query.get('session_id', [''])[0]
data = session if requested == sid else dict(
session_id='idle-fixture', messages=[], message_count=0,
tool_calls=[], workspace=workspace)
route.fulfill(json={'session': data})
return handle
def main():
tool_count = int(os.environ.get('TOOL_COUNT', '470'))
snapshot = fixture(tool_count)
stream_id = snapshot['stream_id']
sid = 'fixture'
with tempfile.TemporaryDirectory(prefix='webui-reconnect-') as temp:
state = Path(temp)
env = {k:os.environ[k] for k in ('PATH','SYSTEMROOT','TMPDIR') if k in os.environ}
env.update(HOME=temp, HERMES_HOME=temp, HERMES_BASE_HOME=temp,
HERMES_WEBUI_STATE_DIR=str(state/'webui'), HERMES_CONFIG_PATH=str(state/'config.yaml'),
HERMES_WEBUI_HOST='127.0.0.1', HERMES_WEBUI_SKIP_ONBOARDING='1',
HERMES_WEBUI_AGENT_DIR=str(state/'no-agent'))
proc, log, _, base = _start_webui_server(ROOT, env, state)
results = []
try:
with sync_playwright() as pw:
for engine in os.environ.get('BROWSERS','chromium,webkit').split(','):
browser = getattr(pw,engine).launch(headless=True)
try:
for mode in os.environ.get('MODES','compact_worklog,transparent_stream,hide_all_activity').split(','):
for width in [1280,390]:
context = browser.new_context(viewport={'width':width,'height':844},bypass_csp=True)
context.add_init_script(INIT)
page = context.new_page()
errors=[]
page.on('pageerror',lambda e, errors=errors:errors.append(str(e)))
session = dict(session_id=sid,title='Reconnect performance fixture',model='',
workspace=temp,messages=[],message_count=0,tool_calls=[],
active_stream_id=stream_id,pending_user_message='Inspect the fixture',
pending_started_at=1,runtime_journal_snapshot=snapshot)
page.route('**/api/session?*',session_route(session,sid,temp))
page.route('**/api/chat/stream/status?*',lambda r:r.fulfill(json={'active':True}))
page.goto(base,wait_until='load')
# WebKit's wait_for_function uses eval, blocked by
# the app CSP even with bypass_csp. Inspector
# evaluation still works; wait on actual boot state.
deadline=time.monotonic()+30
while not page.evaluate("typeof loadSession==='function' && S._bootReady === true"):
assert time.monotonic()<deadline, ('boot timeout', errors)
page.wait_for_timeout(50)
page.evaluate("""mode=>{
window._chatActivityDisplayMode=mode;window._showThinking=true;
window._simplifiedToolCalling=true;
window.redrawCount=0;
const render=window._renderLiveAnchorActivitySceneForStream;
window._renderLiveAnchorActivitySceneForStream=function(...args){window.redrawCount++;return render(...args);};
}""",mode)
measured=page.evaluate("""async sid=>{
const t=performance.now();await loadSession(sid);
const loadMs=performance.now()-t;
const deadline=performance.now()+5000;
while(!fixtureSources.some(s=>s.url.includes('api/chat/stream?'))&&performance.now()<deadline){
await new Promise(r=>setTimeout(r,10));
}
return {ms:loadMs,redraws:window.redrawCount,
tools:document.querySelectorAll('#liveAssistantTurn [data-anchor-row-role="tool"]').length,
busy:S.busy,
sources:fixtureSources.filter(s=>s.url.includes('api/chat/stream?')).map(s=>s.url)};
}""",sid)
result=dict(engine=engine,mode=mode,width=width,
**{k:v for k,v in measured.items() if k!='sources'})
print(json.dumps(result),flush=True)
results.append(result)
if not os.environ.get('MEASURE_BASELINE'):
assert measured['redraws']<=2,result
expected=0 if mode=='hide_all_activity' else tool_count
assert measured['tools']==expected,result
assert measured['busy'],result
assert measured['sources'], (result, errors)
cursor=parse_qs(urlsplit(measured['sources'][-1]).query)
assert cursor['after_seq']==[str(snapshot['last_seq'])], (result,errors)
if mode!='hide_all_activity':
# Subsequent real SSE handler updates still paint the existing owner.
updated=page.evaluate("""({sid,stream})=>{
const source=fixtureSources.findLast(s=>s.url.includes('api/chat/stream?')&&s.readyState===1);
if(!source)throw new Error('missing chat stream');
source.emit('tool',{name:'terminal',tid:'after-reconnect',args:{command:'printf later'},preview:'later'},stream+':1001');
source.emit('tool_complete',{name:'terminal',tid:'after-reconnect',preview:'LATER RESULT',duration:1},stream+':1002');
return {rows:document.querySelectorAll('#liveAssistantTurn [data-anchor-row-role="tool"]').length,
result:document.querySelector('#liveAssistantTurn').textContent.includes('LATER RESULT')};
}""",{'sid':sid,'stream':stream_id})
assert updated=={'rows':expected+1,'result':True},updated
switched=page.evaluate("""async sid=>{
await loadSession('idle-fixture');
await loadSession(sid);
return {rows:document.querySelectorAll('#liveAssistantTurn [data-anchor-row-role="tool"]').length,
result:document.querySelector('#liveAssistantTurn').textContent.includes('LATER RESULT')};
}""",sid)
assert switched==updated,switched
artifact=os.environ.get('SCREENSHOT_DIR')
if artifact:
Path(artifact).mkdir(parents=True,exist_ok=True)
page.screenshot(path=str(Path(artifact)/f'{engine}-{mode}-{width}.png'))
assert not errors,errors
context.close()
finally:
browser.close()
output=os.environ.get('METRICS_FILE')
if output:
Path(output).write_text(json.dumps(results,indent=2))
finally:
_terminate_process(proc)
log.close()
if __name__=='__main__':
main()
tests/sidebar_terminal_barrier_server.py
+40
View File
@@ -0,0 +1,40 @@
"""Test-only server for the public settle-frame gate's cross-channel race.
Hold the actual HTTP terminal frame after the worker has finished. The browser
keeps its native, open EventSource while /api/sessions independently reports idle.
No browser state, renderer, persistence result, or terminal payload is mocked.
"""
from pathlib import Path
import sys
import time
def main():
if len(sys.argv) != 3:
raise SystemExit("expected repository root and isolated barrier directory")
repo_root, barrier_dir = (Path(value) for value in sys.argv[1:])
sys.path.insert(0, str(repo_root))
import server
from api import routes
ready = barrier_dir / "ready"
release = barrier_dir / "release"
original = routes._sse_with_id
def terminal_barrier(handler, event, data, event_id=None):
if event == "done" and handler.path.startswith("/api/chat/stream?"):
ready.touch()
deadline = time.monotonic() + 15
while not release.exists():
if time.monotonic() >= deadline:
raise TimeoutError("test did not release the terminal HTTP frame")
time.sleep(0.005)
return original(handler, event, data, event_id)
routes._sse_with_id = terminal_barrier
server.main()
if __name__ == "__main__":
main()
tests/test_7569_timed_flush_index_reset.py
+81
View File
@@ -0,0 +1,81 @@
"""#7569 review fix: timed-flush must not leave the folded index stale.
The ~10 Hz reasoning throttle in the stream callback clears the coalescing
buffer before emitting:
put('reasoning', {'text': _reasoning_buffer[0]})
_reasoning_buffer[0] = ''
# ← _reasoning_buffer_index.reset() was missing here
``_CompactEchoIndex`` records raw offsets *relative to the buffer's accumulated
length*. Clearing the buffer without resetting the index leaves it describing
text that is no longer there, so the next ``cut_to`` returns a raw offset that
no longer corresponds to the buffer — the live-echo strip then truncates the
buffer at a phantom position and falsely marks the interim as
``reasoning_echo: true`` (leaks a truncated copy into both the SSE stream and
the durable journal).
This file pins the invariant at the *call site*, not just on the index class:
revert the fix commit and the first test fails.
"""
import pathlib
import re
REPO = pathlib.Path(__file__).resolve().parent.parent
STREAMING = (REPO / "api" / "streaming.py").read_text(encoding="utf-8")
def _timed_flush_block() -> str:
"""The ``if now - _reasoning_last_put[0] >= 0.1:`` throttle branch body."""
match = re.search(
r"if now - _reasoning_last_put\[0\] >= 0\.1:\n(.*?)(?=\n\s{16}(?:# Track reasoning deltas|_metering_reasoning_deltas))",
STREAMING,
re.DOTALL,
)
assert match, "timed-flush throttle branch not found in api/streaming.py"
return match.group(1)
def test_timed_flush_clears_index_together_with_the_buffer():
"""Clearing the reasoning buffer must also reset the folded index, in the
same throttle branch, after the clear (ordering matters: the reset cannot
sit in an unrelated branch or before the clear)."""
block = _timed_flush_block()
assert "_reasoning_buffer[0] = ''" in block, (
"timed-flush branch must clear the coalesced reasoning buffer"
)
assert "_reasoning_buffer_index.reset()" in block, (
"timed-flush branch must reset the folded index when it clears the "
"buffer — leaving the index stale makes later cut_to() offsets "
"describe text that is no longer in the buffer (#7569 re-gate)"
)
# reset must come AFTER the clear, and be at the same indent depth (8
# spaces inside the throttle branch), not in a sibling branch.
clear_pos = block.index("_reasoning_buffer[0] = ''")
reset_pos = block.index("_reasoning_buffer_index.reset()")
assert reset_pos > clear_pos, (
"reset must follow the buffer clear, so the next append starts from "
"an empty raw-length base"
)
assert re.search(r"^\s{20}_reasoning_buffer_index\.reset\(\)", block, re.M), (
"reset must be inside the timed-flush branch (deeper than the clear), "
"not in an outer/sibling scope"
)
def test_all_buffer_clears_are_index_safe():
"""Every site that zeroes ``_reasoning_buffer[0]`` must reset the folded
index immediately after — grep the codebase for clear sites and require
each to be paired with the index reset."""
for clear_match in re.finditer(r"_reasoning_buffer\[0\] = ''", STREAMING):
window = STREAMING[clear_match.start(): clear_match.start() + 400]
reset_present = re.search(
r"_reasoning_buffer_index\.reset\(\)", window
)
assert reset_present, (
"a buffer clear at api/streaming.py has no adjacent "
"_reasoning_buffer_index.reset() — stale offsets would corrupt "
"the next cut_to() call"
)
tests/test_agent_max_turns_parity.py
+8 -9
View File
@@ -79,20 +79,19 @@ RAISED_EXCEPTION_SELF_HEAL_MARKER = (
"retrying stream after credential refresh')"
)
# Each region ends at the cache write that stores the signature it just
# computed. Bounding the region at the write -- rather than at the next
# lifecycle anchor or at EOF -- is what proves the signature is recomputed
# *before* it is cached: a call moved below its own write lands outside the
# region and fails, instead of passing while the write consumes a stale
# `_agent_sig`.
# Each region ends at the atomic registration/publication call that consumes
# the signature it just computed. The shared helper now writes the cache under
# the Stop lock; the call boundary must still follow fresh signature computation.
# A signature call moved below its publication lands outside the region and
# fails, rather than silently reusing a stale `_agent_sig` on a retry.
INITIAL_SEND_CACHE_WRITE_MARKER = (
"SESSION_AGENT_CACHE[session_id] = (agent, _agent_sig)"
"if not _register_agent_if_current(agent, _agent_sig if _cache_new_agent else None):"
)
RETURNED_ERROR_SELF_HEAL_CACHE_WRITE_MARKER = (
"_SAC[session_id] = (agent, _agent_sig)"
"if not _register_agent_if_current(agent, _agent_sig):"
)
RAISED_EXCEPTION_SELF_HEAL_CACHE_WRITE_MARKER = (
"_SAC2[session_id] = (_heal_agent, _agent_sig)"
"if not _register_agent_if_current(_heal_agent, _agent_sig):"
)
tests/test_agent_publication_cancellation.py
+245
View File
@@ -0,0 +1,245 @@
"""Stop must not lose to cache publication after an Agent was registered.
Run the real worker and Stop primitive. Pause immediately after the registration
lock is released, not inside the constructor covered by the older tests. Only
external Agent/provider calls are synthetic; no live credentials are used.
"""
import threading
from collections import OrderedDict
from concurrent.futures import ThreadPoolExecutor
import pytest
from api import config, session_lifecycle, streaming
from tests.test_steer_worker_boundaries import worker_scene as worker_scene
@pytest.mark.parametrize("path", ["initial", "cached", "returned", "exception"])
@pytest.mark.parametrize("cancellation", ["stop", "event-only", "detached-only", "successor", "same-agent-successor", "same-agent-completed", "none"])
def test_stop_after_registration_before_cache_or_invocation(worker_scene, monkeypatch, path, cancellation):
scene = worker_scene
reached, release = threading.Event(), threading.Event()
created, publications, finalized = [], [], []
worker_id = [None]
paused = [False]
successor = object()
stop_interrupt_seen = threading.Event()
successor_interrupt_count = None
monkeypatch.setattr(session_lifecycle, "_sessions", {})
monkeypatch.setattr(config, "SESSION_WRITEBACK_OWNERS", {})
config.register_session_writeback_owner("original", "run")
def on_init():
created.append(scene.agent)
interrupt = scene.agent.interrupt
def observed_interrupt(reason):
result = interrupt(reason)
stop_interrupt_seen.set()
return result
scene.agent.interrupt = observed_interrupt
def on_run():
assert scene.lock.owner != threading.get_ident(), "provider invocation under stream lock"
if path in ("returned", "exception") and len(created) == 1:
if path == "exception":
raise RuntimeError("401 unauthorized")
scene.result = {"error": "401 unauthorized", "messages": []}
else:
scene.result = None
scene.on_init, scene.on_run = on_init, on_run
if path == "cached":
scene.run()
assert scene.calls.count("run") == 1
assert config.SESSION_AGENT_CACHE["original"][0] is created[0]
scene.calls.clear()
config.STREAMS["run"] = scene.events
config.STREAM_SESSION_OWNERS["run"] = "original"
scene.session.active_stream_id = "run"
scene.session.pending_user_message = "Do the task."
scene.session.pending_started_at = 2.0
scene.session.save()
config.register_session_writeback_owner("original", "run")
stop_interrupt_seen.clear()
class ObservedCache(OrderedDict):
def __setitem__(self, key, value):
if key == "original":
publications.append((value[0], scene.lock.owner == threading.get_ident(),
"run" in config.STREAMS))
super().__setitem__(key, value)
cache = ObservedCache()
# Preserve an existing cache hit without counting fixture setup as publication.
for key, value in config.SESSION_AGENT_CACHE.items():
OrderedDict.__setitem__(cache, key, value)
monkeypatch.setattr(config, "SESSION_AGENT_CACHE", cache)
if hasattr(streaming, "SESSION_AGENT_CACHE"):
monkeypatch.setattr(streaming, "SESSION_AGENT_CACHE", cache)
monkeypatch.setattr(streaming, "_attempt_credential_self_heal", lambda *a, **kw: {
"provider": "openai", "api_key": "synthetic-publication-fixture",
"base_url": "http://127.0.0.1:1/v1",
})
target_count = 2 if path in ("returned", "exception") else 1
def after_registration_unlock():
if (worker_id[0] != threading.get_ident() or paused[0]
or len(created) != target_count
or config.AGENT_INSTANCES.get("run") is not created[-1]):
return
paused[0] = True
reached.set()
assert release.wait(8), "post-registration barrier timed out"
scene.lock.after_release = after_registration_unlock
finalize = streaming._finalize_cancelled_turn
def observed_finalize(*args, **kwargs):
assert scene.lock.owner != threading.get_ident(), "finalize under stream lock"
finalized.append(True)
return finalize(*args, **kwargs)
monkeypatch.setattr(streaming, "_finalize_cancelled_turn", observed_finalize)
def run():
worker_id[0] = threading.get_ident()
scene.run()
with ThreadPoolExecutor(max_workers=2) as pool:
stop_future = None
future = pool.submit(run)
try:
assert reached.wait(8), "worker never released target registration lock"
retained = config.CANCEL_FLAGS["run"]
if cancellation in ("stop", "successor", "same-agent-successor", "same-agent-completed"):
stop_future = pool.submit(streaming.cancel_stream, "run")
assert retained.wait(5), "Stop did not publish cancellation"
assert stop_interrupt_seen.wait(5), "Stop has not finished its own interrupt"
with config.STREAMS_LOCK:
assert "run" not in config.STREAMS
if cancellation in ("successor", "same-agent-successor", "same-agent-completed"):
if cancellation.startswith("same-agent"):
successor = created[-1]
config.register_session_writeback_owner("original", "successor")
with config.SESSION_AGENT_CACHE_LOCK:
OrderedDict.__setitem__(cache, "original", (successor, "new-signature"))
session_lifecycle.register_agent("original", successor)
scene.session.active_stream_id = "successor"
scene.session.pending_user_message = "Successor request"
if cancellation == "same-agent-completed":
config.clear_session_writeback_owner_if_owned("original", "successor")
scene.session.active_stream_id = None
scene.session.pending_user_message = None
else:
config.AGENT_INSTANCES["successor"] = successor
config.register_active_run("successor", session_id="original", backend="legacy")
if cancellation == "same-agent-successor":
with successor.pending_lock:
successor.pending.append("Successor-only guidance")
successor_interrupt_count = scene.calls.count("interrupt")
elif cancellation == "event-only":
with config.STREAMS_LOCK:
retained.set()
config.CANCEL_FLAGS.pop("run")
elif cancellation == "detached-only":
with config.STREAMS_LOCK:
config.STREAMS.pop("run")
assert not retained.is_set()
finally:
release.set()
future.result(timeout=12)
if stop_future is not None:
assert stop_future.result(timeout=12) is True
prior_runs = 1 if path in ("returned", "exception") else 0
if cancellation == "none":
assert scene.calls.count("run") == prior_runs + 1
assert config.SESSION_AGENT_CACHE["original"][0] is created[-1]
else:
assert scene.calls.count("run") == prior_runs, "cancelled candidate was invoked"
cached = config.SESSION_AGENT_CACHE.get("original")
if not cancellation.startswith("same-agent"):
assert not cached or cached[0] is not created[-1], "cancelled candidate stayed reusable"
assert finalized, "cancellation did not reach terminal settlement"
if successor_interrupt_count is not None:
assert scene.calls.count("interrupt") == successor_interrupt_count, "stale worker interrupted its successor"
assert config.SESSION_AGENT_CACHE["original"][0] is successor
assert session_lifecycle._sessions["original"]["agent"] is successor
if cancellation == "same-agent-successor":
assert successor.pending == ["Successor-only guidance"], "stale terminal drain stole successor guidance"
if cancellation == "same-agent-completed":
assert scene.session.active_stream_id is None
else:
assert config.AGENT_INSTANCES["successor"] is successor
assert scene.session.active_stream_id == "successor"
assert scene.session.pending_user_message == "Successor request"
assert all(held and live for _, held, live in publications), (
"Agent cache publication escaped the stream/Stop admission boundary", publications)
assert "run" not in config.AGENT_INSTANCES
assert "run" not in config.ACTIVE_RUNS
def test_cache_publication_and_registration_exclude_competing_stop(worker_scene, monkeypatch):
"""Block the actual cache write; Stop cannot observe a half-published Agent."""
scene = worker_scene
entered, release = threading.Event(), threading.Event()
interrupted_state = []
interrupt_observed = threading.Event()
monkeypatch.setattr(session_lifecycle, "_sessions", {})
monkeypatch.setattr(config, "SESSION_WRITEBACK_OWNERS", {})
config.register_session_writeback_owner("original", "run")
class PublicationBarrier(OrderedDict):
def __setitem__(self, key, value):
if key == "original":
entered.set()
assert release.wait(8), "cache publication was not released"
super().__setitem__(key, value)
cache = PublicationBarrier()
monkeypatch.setattr(config, "SESSION_AGENT_CACHE", cache)
if hasattr(streaming, "SESSION_AGENT_CACHE"):
monkeypatch.setattr(streaming, "SESSION_AGENT_CACHE", cache)
def on_init():
candidate = scene.agent
interrupt = candidate.interrupt
def observed_interrupt(reason):
interrupted_state.append((
cache.get("original", (None,))[0] is candidate,
session_lifecycle._sessions.get("original", {}).get("agent") is candidate,
))
result = interrupt(reason)
interrupt_observed.set()
return result
candidate.interrupt = observed_interrupt
scene.on_init = on_init
with ThreadPoolExecutor(max_workers=2) as pool:
worker = pool.submit(scene.run)
stop = None
try:
assert entered.wait(8), "worker never reached cache publication"
retained = config.CANCEL_FLAGS["run"]
# Wait at the first publication unlock until Stop has won admission,
# so the provider cannot outrun the controlled schedule.
worker_ident = scene.lock.owner
def wait_for_stop():
if threading.get_ident() == worker_ident:
scene.lock.after_release = None
assert interrupt_observed.wait(5), "Stop did not interrupt the published Agent"
scene.lock.after_release = wait_for_stop
stop = pool.submit(streaming.cancel_stream, "run")
assert scene.lock.contender.wait(5), "Stop bypassed publication's stream lock"
assert not retained.is_set(), "Stop observed half-published Agent state"
finally:
release.set()
worker.result(timeout=12)
if stop is not None:
assert stop.result(timeout=12) is True
assert interrupted_state[0] == (True, True), "Stop did not see coherent cache/lifecycle publication"
assert "run" not in scene.calls
assert "original" not in cache
tests/test_anchor_fallback_ownership.py
+5
View File
@@ -416,6 +416,10 @@ def test_render_messages_keeps_anchor_owned_turn_out_of_legacy_activity_rebuilds
legacy_metadata_source = _function_source(
_ui_js(), "_legacySettledFallbackHasToolMetadata"
)
# #2051: renderMessages() inserts a message block through this helper, so it is
# evaluated with it. The shim's createElement() returns no template `content`, so the
# helper takes its insertAdjacentHTML fallback here, exactly as before.
insert_block_source = _function_source(_ui_js(), "_insertSegmentBlock")
script = textwrap.dedent(
f"""
class FakeClassList {{
@@ -702,6 +706,7 @@ def test_render_messages_keeps_anchor_owned_turn_out_of_legacy_activity_rebuilds
eval({json.dumps(transparent_source)});
eval({json.dumps(legacy_metadata_source)});
eval({json.dumps(insert_block_source)});
eval({json.dumps(render_source)});
const toolResult = {{ role: 'tool', tool_call_id: 'toolu_1', content: 'tool result' }};
tests/test_anchor_scene_persistence.py
+12 -2
View File
@@ -241,8 +241,18 @@ def test_anchor_scene_persistence_rejects_cross_profile_write(tmp_path, monkeypa
SimpleNamespace(command="POST"),
SimpleNamespace(path="/api/session/anchor-scene"),
) is True
assert captured.get("status") == 404
assert "payload" not in captured # success j() never called
# #7710: the generic request-guard now mirrors the detail-load
# endpoint's contract — a session owned by a KNOWN other profile
# yields 409 ``session_profile_mismatch`` so the client can offer
# to switch to it (#5419). The 404 self-heal path is preserved for
# the None-profile (unknown/legacy) case.
assert captured.get("status") == 409, captured
assert captured.get("payload") == {
"error": "Session belongs to a different profile",
"code": "session_profile_mismatch",
"session_id": "foreignprofile1",
"profile": "profile-b",
}
raw = json.loads((session_dir / "foreignprofile1.json").read_text(encoding="utf-8"))
assert not raw.get("anchor_activity_scenes"), (
"cross-profile request must NOT persist anchor_activity_scenes"
tests/test_approval_unblock.py
+79
View File
@@ -601,6 +601,85 @@ class TestApprovalHTTPEndpoints:
r._gateway_queues.pop(sid, None)
_STREAM_RUN_IDS.pop(stream_id, None)
def test_identity_v1_non_head_run_approval_settles_exact_producer(self, monkeypatch):
"""A relayed non-head run approval must wake only its own parked waiter."""
from api import route_approvals as ra
from api import routes as r
sid = f"identity-v1-non-head-{uuid.uuid4().hex[:8]}"
run_id = f"run-identity-v1-{uuid.uuid4().hex[:8]}"
head = {
"approval_id": "head-approval",
"run_id": run_id,
"command": "head command",
"_gateway_agent_identity_v1": True,
}
target = {
"approval_id": "target-approval",
"run_id": run_id,
"command": "target command",
"_gateway_agent_identity_v1": True,
}
head_entry = _ApprovalEntry(dict(head))
target_entry = _ApprovalEntry(dict(target))
captured = {}
relays = []
def fake_j(_handler, data, status=200, extra_headers=None):
captured.update(payload=data, status=status)
return data
def fake_respond(_self, got_run_id, got_approval_id, choice):
relays.append((got_run_id, got_approval_id, choice))
return {"resolved": 1}
monkeypatch.setattr(r, "j", fake_j)
monkeypatch.setattr("api.runner_client.HttpRunnerClient.respond_approval", fake_respond)
monkeypatch.setattr(
"api.config.gateway_supports_approval_identity_v1", lambda *_args: True
)
with _lock:
r._pending.pop(sid, None)
r._gateway_queues[sid] = [head_entry, target_entry]
try:
ra.submit_gateway_pending_mirror(sid, dict(target_entry.data))
mirror = ra.gateway_pending_mirror(
sid, approval_id=target["approval_id"], run_id=run_id
)
assert mirror is not None
r._handle_approval_respond(
object(),
{
"session_id": sid,
"choice": "deny",
"approval_id": target["approval_id"],
"run_id": run_id,
"mirror_token": mirror[ra._GATEWAY_MIRROR_TOKEN],
},
)
assert captured == {
"payload": {"ok": True, "choice": "deny", "relayed": True},
"status": 200,
}
assert relays == [(run_id, target["approval_id"], "deny")]
assert target_entry.event.is_set()
assert target_entry.result == "deny"
assert not head_entry.event.is_set()
with _lock:
assert r._gateway_queues[sid] == [head_entry]
for _ in range(2):
with _lock:
ra.reconcile_gateway_pending_mirror_locked(sid)
assert ra.gateway_pending_mirror(
sid, approval_id=target["approval_id"], run_id=run_id
) is None
finally:
with _lock:
r._pending.pop(sid, None)
r._gateway_queues.pop(sid, None)
def test_gateway_mirror_without_run_id_with_one_producer_resolves_exactly(self, monkeypatch):
"""A no-run mirror retires only after its exact local producer resolves."""
from api import routes as r
tests/test_async_delegation_retryable_refusal.py
+159
View File
@@ -0,0 +1,159 @@
"""A wake-up refused for a transient reason the WebUI did not cause must not
consume the durable delivery budget.
Scenario: while the Agent runtime is being replaced, the still-running WebUI
answers every server-side wake-up with ``409 agent_runtime_stale
retryable=True``. Releasing the claim as a plain failure each time exhausts the
bounded budget within seconds and terminally drops a completion whose origin
session is alive and waiting.
Contract under test (``api/background_process._start_async_delegation_wakeup_turn``):
* a refusal whose payload says ``retryable: True`` (stale runtime), a paused
wake-up or a session busy with another stream releases the claim with
``retryable=True`` so the core refunds the attempt;
* every other rejection keeps the historical bounded behaviour;
* an older core whose ``release_event_delivery`` has no ``retryable`` keyword
still gets a plain release (compatibility, never an exception).
"""
from __future__ import annotations
import sys
import time
import types
import pytest
from api import background_process as bp
from api import config as cfg
from api import process_event_utils as peu
def _wait_until(predicate, timeout=3.0):
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if predicate():
return True
time.sleep(0.01)
return bool(predicate())
def _reset() -> None:
cfg.PROCESS_SESSION_INDEX.clear()
cfg.PENDING_BG_TASK_COMPLETIONS.clear()
cfg.BG_TASK_COMPLETE_EVENTS_SEEN.clear()
cfg.DEFERRED_PROCESS_WAKEUPS.clear()
reset = getattr(peu, "_reset_legacy_async_delivery_dedupe_for_tests", None)
if reset is not None:
reset()
def _event():
return {
"type": "async_delegation",
"delegation_id": "deleg_retryable",
"session_key": "webui-session-1",
"status": "completed",
"is_batch": True,
"results": [{"task_index": 0, "status": "completed", "summary": "verdict"}],
"dispatched_at": time.time() - 2,
"completed_at": time.time(),
}
class _Registry:
def __init__(self):
import queue
self.completion_queue = queue.Queue()
def _install_core(monkeypatch, *, supports_retryable: bool):
calls = {"release": [], "complete": []}
mod = types.ModuleType("tools.async_delegation")
mod.claim_event_delivery = lambda evt, consumer: f"claim:{consumer}"
mod.complete_event_delivery = lambda evt, claim_id: calls["complete"].append(claim_id)
mod.get_durable_delegation = lambda delegation_id: {"delivery_state": "pending", "delivery_attempts": 1}
mod.restore_undelivered_completions = lambda queue_: 0
if supports_retryable:
def _release(evt, claim_id, *, retryable=False):
calls["release"].append((claim_id, retryable))
else:
def _release(evt, claim_id): # pre-refund core: no keyword at all
calls["release"].append((claim_id, None))
mod.release_event_delivery = _release
pkg = sys.modules.get("tools") or types.ModuleType("tools")
monkeypatch.setitem(sys.modules, "tools", pkg)
monkeypatch.setitem(sys.modules, "tools.async_delegation", mod)
monkeypatch.setattr(pkg, "async_delegation", mod, raising=False)
return calls
def _run(monkeypatch, response, *, supports_retryable=True):
_reset()
calls = _install_core(monkeypatch, supports_retryable=supports_retryable)
from api import routes
monkeypatch.setattr(routes, "start_session_turn", lambda *a, **k: response)
monkeypatch.setattr(bp, "ASYNC_DELIVERY_ROUTING_RETRY_SECONDS", 10.0)
registry = _Registry()
evt = _event()
claim = peu.claim_async_delegation_delivery(evt, "webui-background")
assert claim is not None
try:
bp._start_async_delegation_wakeup_turn(
"webui-session-1", "verdict", delegation_id="deleg_retryable",
evt=evt, claim=claim, process_registry=registry,
)
assert _wait_until(lambda: len(calls["release"]) == 1)
assert calls["complete"] == []
return calls["release"][0]
finally:
_reset()
@pytest.mark.parametrize("response", [
{"_status": 409, "error": "runtime changed", "type": "agent_runtime_stale", "retryable": True},
{"_status": 503, "error": "temporarily unavailable", "retryable": True},
{"_status": 409, "error": "process_wakeup_paused"},
{"_status": 409, "error": "session already has an active stream", "active_stream_id": "s1"},
])
def test_transient_refusal_releases_with_refund(monkeypatch, response):
_claim, retryable = _run(monkeypatch, response)
assert retryable is True
@pytest.mark.parametrize("response", [
{"_status": 500, "error": "boom"},
{"_status": 404, "error": "Session not found"},
{"_status": 400, "error": "message is required"},
{"_status": 409, "error": "conflict", "retryable": False},
None,
])
def test_hard_rejection_keeps_bounded_budget(monkeypatch, response):
_claim, retryable = _run(monkeypatch, response)
assert retryable is False
def test_old_core_without_retryable_keyword_still_releases(monkeypatch):
_claim, retryable = _run(
monkeypatch,
{"_status": 409, "error": "x", "type": "agent_runtime_stale", "retryable": True},
supports_retryable=False,
)
assert retryable is None # plain release happened, no TypeError escaped
def test_real_core_signature_is_detected():
def new_core(evt, claim_id, *, retryable=False):
return None
def kwargs_core(evt, claim_id, **kwargs):
return None
def old_core(evt, claim_id):
return None
assert peu._release_accepts_retryable(new_core) is True
assert peu._release_accepts_retryable(kwargs_core) is True
assert peu._release_accepts_retryable(old_core) is False
tests/test_background_process_restart_recovery.py
+100
View File
@@ -1,5 +1,6 @@
"""Regression coverage for notify_on_complete across a WebUI restart."""
import logging
import threading
import time
from types import SimpleNamespace
@@ -159,3 +160,102 @@ def test_recovery_is_fail_soft_without_agent(monkeypatch):
assert bp.recover_processes_for_webui() == 0
assert bp._PROCESS_RECOVERY_DONE is False
def test_recovery_skips_process_whose_session_is_gone(monkeypatch, caplog):
"""A checkpointed process whose owner has been pruned (the resolver's
``KeyError(sid)`` contract) must be silently skipped, while a neighbouring
live process in the same recovery sweep still rebinds. Regression for
issue #7753: previously every vanished owner logged a full WARNING
traceback, indistinguishable from a real fault.
"""
calls = {"registered": []}
class FakeRegistry:
def recover_from_checkpoint(self):
return 2 # two checkpointed processes survive restart
def list_sessions(self):
return [
{"session_id": "proc_vanished", "detached": True},
{"session_id": "proc_live", "detached": True},
]
def get(self, process_id):
if process_id == "proc_vanished":
return _FakeProcessSession("vanished-session")
if process_id == "proc_live":
return _FakeProcessSession("live-session")
raise AssertionError(process_id)
fake_registry = FakeRegistry()
monkeypatch.setattr(bp, "_PROCESS_CHECKPOINT_RECOVERED", False)
monkeypatch.setattr(bp, "_PROCESS_RECOVERY_DONE", False)
monkeypatch.setattr(
bp,
"register_process_session",
lambda key, sid: calls["registered"].append((key, sid)),
)
def get_session(sid, metadata_only=False):
# Mirrors api.models._resolve_session_once's contract: missing
# sessions raise KeyError, NOT None.
if sid == "vanished-session":
raise KeyError(sid)
return SimpleNamespace(id=sid)
with caplog.at_level(logging.WARNING, logger="api.background_process"):
result = bp.recover_processes_for_webui(fake_registry, get_session)
assert result == 2
# The live process is rebound; the vanished one is silently skipped.
assert calls["registered"] == [("live-session", "live-session")]
# No WARNING emitted for the vanished-owner case.
warning_records = [
r for r in caplog.records if r.levelno >= logging.WARNING
]
assert warning_records == [], (
"vanished-session KeyError must not produce a WARNING, got: "
f"{[r.getMessage() for r in warning_records]}"
)
assert "Could not resolve recovered WebUI process" not in caplog.text
def test_recovery_still_warns_on_unexpected_resolve_failure(monkeypatch, caplog):
"""Non-KeyError failures from the resolver (e.g. ``OSError`` on an
unreadable sidecar) must keep entering the existing WARNING path. This
pins that the fix for #7753 is narrow and does not over-broaden the
silent-skip contract.
"""
class FakeRegistry:
def recover_from_checkpoint(self):
return 1
def list_sessions(self):
return [{"session_id": "proc_unexpected", "detached": True}]
def get(self, process_id):
assert process_id == "proc_unexpected"
return _FakeProcessSession("webui-session")
fake_registry = FakeRegistry()
monkeypatch.setattr(bp, "_PROCESS_CHECKPOINT_RECOVERED", False)
monkeypatch.setattr(bp, "_PROCESS_RECOVERY_DONE", False)
def get_session(sid, metadata_only=False):
raise OSError("sidecar unreadable")
with caplog.at_level(logging.WARNING, logger="api.background_process"):
result = bp.recover_processes_for_webui(fake_registry, get_session)
assert result == 1
warning_records = [
r for r in caplog.records if r.levelno >= logging.WARNING
]
assert any(
"Could not resolve recovered WebUI process 'proc_unexpected'" in r.getMessage()
for r in warning_records
), (
"OSError from the resolver must still emit a WARNING, got: "
f"{[r.getMessage() for r in warning_records]}"
)
tests/test_broken_pipe_cascade.py
+24 -24
View File
@@ -227,14 +227,14 @@ class TestServerDisconnectHandling(unittest.TestCase):
# Patch handle_get to raise BrokenPipeError
import server as _server_mod
orig_handle_get = _server_mod.handle_get
orig_check_auth = _server_mod.check_auth
orig_check_auth = _server_mod.check_auth_or_close
_server_mod.handle_get = _fake_handle_get
_server_mod.check_auth = lambda h, p: True
_server_mod.check_auth_or_close = lambda h, p: True
try:
Handler.do_GET(handler)
finally:
_server_mod.handle_get = orig_handle_get
_server_mod.check_auth = orig_check_auth
_server_mod.check_auth_or_close = orig_check_auth
# send_response should NEVER be called for the 500 — client is gone
handler.send_response.assert_not_called()
@@ -248,12 +248,12 @@ class TestServerDisconnectHandling(unittest.TestCase):
raise self._route_raises
import server as _server_mod
orig_check_auth = _server_mod.check_auth
_server_mod.check_auth = lambda h, p: True
orig_check_auth = _server_mod.check_auth_or_close
_server_mod.check_auth_or_close = lambda h, p: True
try:
Handler._handle_write(handler, _fake_route)
finally:
_server_mod.check_auth = orig_check_auth
_server_mod.check_auth_or_close = orig_check_auth
handler.send_response.assert_not_called()
@@ -286,9 +286,9 @@ class TestServerDisconnectHandling(unittest.TestCase):
import server as _server_mod
orig_handle_get = _server_mod.handle_get
orig_check_auth = _server_mod.check_auth
orig_check_auth = _server_mod.check_auth_or_close
_server_mod.handle_get = _fake_handle_get
_server_mod.check_auth = lambda h, p: True
_server_mod.check_auth_or_close = lambda h, p: True
closed_stdout = io.StringIO()
closed_stdout.close()
@@ -299,7 +299,7 @@ class TestServerDisconnectHandling(unittest.TestCase):
finally:
sys.stdout = original_stdout
_server_mod.handle_get = orig_handle_get
_server_mod.check_auth = orig_check_auth
_server_mod.check_auth_or_close = orig_check_auth
handler.send_response.assert_called_once_with(500)
@@ -312,14 +312,14 @@ class TestServerDisconnectHandling(unittest.TestCase):
import server as _server_mod
orig_handle_get = _server_mod.handle_get
orig_check_auth = _server_mod.check_auth
orig_check_auth = _server_mod.check_auth_or_close
_server_mod.handle_get = _fake_handle_get
_server_mod.check_auth = lambda h, p: True
_server_mod.check_auth_or_close = lambda h, p: True
try:
Handler.do_GET(handler)
finally:
_server_mod.handle_get = orig_handle_get
_server_mod.check_auth = orig_check_auth
_server_mod.check_auth_or_close = orig_check_auth
# Should send 500 for real errors
handler.send_response.assert_called_once_with(500)
@@ -334,14 +334,14 @@ class TestServerDisconnectHandling(unittest.TestCase):
import server as _server_mod
orig_handle_get = _server_mod.handle_get
orig_check_auth = _server_mod.check_auth
orig_check_auth = _server_mod.check_auth_or_close
_server_mod.handle_get = _fake_handle_get
_server_mod.check_auth = lambda h, p: True
_server_mod.check_auth_or_close = lambda h, p: True
try:
Handler.do_GET(handler)
finally:
_server_mod.handle_get = orig_handle_get
_server_mod.check_auth = orig_check_auth
_server_mod.check_auth_or_close = orig_check_auth
handler.send_response.assert_not_called()
@@ -355,14 +355,14 @@ class TestServerDisconnectHandling(unittest.TestCase):
import server as _server_mod
orig_handle_get = _server_mod.handle_get
orig_check_auth = _server_mod.check_auth
orig_check_auth = _server_mod.check_auth_or_close
_server_mod.handle_get = _fake_handle_get
_server_mod.check_auth = lambda h, p: True
_server_mod.check_auth_or_close = lambda h, p: True
try:
Handler.do_GET(handler)
finally:
_server_mod.handle_get = orig_handle_get
_server_mod.check_auth = orig_check_auth
_server_mod.check_auth_or_close = orig_check_auth
handler.send_response.assert_not_called()
@@ -406,14 +406,14 @@ class TestServer500ResponseSafety(unittest.TestCase):
import server as _server_mod
orig_handle_get = _server_mod.handle_get
orig_check_auth = _server_mod.check_auth
orig_check_auth = _server_mod.check_auth_or_close
_server_mod.handle_get = _fake_handle_get
_server_mod.check_auth = lambda h, p: True
_server_mod.check_auth_or_close = lambda h, p: True
try:
Handler.do_GET(handler)
finally:
_server_mod.handle_get = orig_handle_get
_server_mod.check_auth = orig_check_auth
_server_mod.check_auth_or_close = orig_check_auth
# send_response WAS called (we tried to send 500), but write failed
handler.send_response.assert_called_once_with(500)
@@ -431,14 +431,14 @@ class TestServer500ResponseSafety(unittest.TestCase):
import server as _server_mod
orig_handle_get = _server_mod.handle_get
orig_check_auth = _server_mod.check_auth
orig_check_auth = _server_mod.check_auth_or_close
_server_mod.handle_get = _fake_handle_get
_server_mod.check_auth = lambda h, p: True
_server_mod.check_auth_or_close = lambda h, p: True
try:
Handler.do_GET(handler)
finally:
_server_mod.handle_get = orig_handle_get
_server_mod.check_auth = orig_check_auth
_server_mod.check_auth_or_close = orig_check_auth
# send_response WAS called (we tried to send 500), but write failed
handler.send_response.assert_called_once_with(500)
tests/test_byok_model_dropdown.py
+8 -1
View File
@@ -291,6 +291,13 @@ class TestLiveModelsCustomProviderFallback:
"model": "gpt-5.5",
"models": {"gpt-5.5-mini": {}},
"base_url": "https://right.codes/codex/v1",
# dict-shaped ``models`` is per-model metadata (Agent setup
# flow) and only narrows the live catalog when discovery is
# explicitly off — the new #7165 contract. This test still
# wants provider-scoping (no sibling leak), so pin the
# catalog via ``discover_models: false`` instead of relying
# on the removed dict-as-allowlist behavior.
"discover_models": False,
},
{
"name": "infini-ai",
@@ -308,7 +315,7 @@ class TestLiveModelsCustomProviderFallback:
resp = self._call_live_models(monkeypatch, cfg, "custom:rightcode-codex")
assert resp["provider"] == "custom:rightcode-codex"
assert [m["id"] for m in resp["models"]] == ["gpt-5.5", "gpt-5.5-mini"]
assert [m["id"] for m in resp["models"]] == ["gpt-5.5-mini"]
def test_bare_custom_fallback_ignores_named_custom_provider_models(self, monkeypatch):
"""Bare custom only represents unnamed custom entries, not named siblings."""
tests/test_cli_only_slash_commands.py
+431 -3
View File
@@ -2,12 +2,15 @@
import json
from pathlib import Path
import re
import subprocess
import tempfile
import textwrap
from types import SimpleNamespace
from api.commands import list_commands
from api.commands import list_commands, _ALLOWED_AGENT_COMMANDS, _AGENT_COMMAND_ALIASES
import pytest
REPO_ROOT = Path(__file__).resolve().parents[1]
@@ -15,6 +18,36 @@ COMMANDS_JS = (REPO_ROOT / "static" / "commands.js").read_text(encoding="utf-8")
MESSAGES_JS = (REPO_ROOT / "static" / "messages.js").read_text(encoding="utf-8")
def _extract_js_set_members(source: str, varname: str) -> set[str]:
"""Return the string members of a `const NAME=new Set([...])` literal."""
m = re.search(rf"const\s+{re.escape(varname)}\s*=\s*new Set\(\[(.*?)\]\)", source, re.S)
assert m, f"could not find const {varname}=new Set([...]) in source"
return set(re.findall(r"'([^']*)'", m.group(1)))
def _canonical_agent_names(names: set[str]) -> set[str]:
"""Map underscore alias forms to canonical names via api/commands.py's map."""
return {_AGENT_COMMAND_ALIASES.get(n, n) for n in names}
def _extract_busy_intercept_block() -> str:
"""Extract the busy-path slash intercept `if` block verbatim from send()."""
marker = "Busy-control slash commands must be intercepted"
marker_idx = MESSAGES_JS.find(marker)
assert marker_idx != -1
start = MESSAGES_JS.find("if(text.startsWith('/')&&!literalSlash){", marker_idx)
assert start != -1
depth = 0
for i in range(start, len(MESSAGES_JS)):
if MESSAGES_JS[i] == "{":
depth += 1
elif MESSAGES_JS[i] == "}":
depth -= 1
if depth == 0:
return MESSAGES_JS[start : i + 1]
raise AssertionError("unbalanced braces while extracting busy intercept block")
def test_api_commands_exposes_cli_only_metadata_for_webui_intercept():
"""CLI-only commands must remain visible so the frontend can explain them."""
registry = [
@@ -163,6 +196,30 @@ def _run_commands_js(script_body: str) -> dict:
aliases: ['plugin_review'],
cli_only: false,
gateway_only: false
}},
{{
name: 'agents',
description: 'Manage background agents',
category: 'Session',
aliases: ['tasks'],
cli_only: false,
gateway_only: false
}},
{{
name: 'sessions',
description: 'List sessions',
category: 'Session',
aliases: [],
cli_only: false,
gateway_only: false
}},
{{
name: 'resume',
description: 'Resume a session',
category: 'Session',
aliases: [],
cli_only: false,
gateway_only: false
}}
]
}};
@@ -356,8 +413,8 @@ def test_cli_only_slugs_reserve_skill_autocomplete_namespace():
assert result["delegate_names"] == []
assert result["incident_names"] == ["incident-review"]
assert result["incident_sources"] == ["bundle"]
assert result["triage_names"] == ["triage-review"]
assert result["triage_sources"] == ["agent"]
assert result["triage_names"] == []
assert result["triage_sources"] == []
assert result["plugin_names"] == ["plugin-review"]
assert result["plugin_sources"] == ["plugin"]
assert "skills" in result["skills_names"]
@@ -659,3 +716,374 @@ def test_builtin_command_opt_outs_do_not_hit_agent_metadata_lookup():
assert optout_idx != -1
assert metadata_idx != -1
assert "if(_parsedCmd&&!_cmd)" in intercept[optout_idx:metadata_idx + 120]
# ── #6951: autocomplete must only announce commands the WebUI dispatches ─────
def test_non_dispatchable_agent_command_hidden_from_autocomplete():
"""#6951: registry commands the WebUI does not dispatch (e.g. /agents) must
not be advertised, since submitting them would fall through to plain chat."""
result = _run_commands_js(
"""
await loadAgentCommandMetadata(true);
const agents = await getSlashAutocompleteMatches('/agents');
const ag = await getSlashAutocompleteMatches('/ag');
return {
agents_names: agents.map(item => item.name),
ag_names: ag.map(item => item.name)
};
"""
)
assert result["agents_names"] == []
assert result["ag_names"] == []
def test_dispatchable_agent_commands_stay_in_autocomplete():
"""#6951: commands the WebUI does dispatch -- backend exec allowlist and
native WebUI behaviors -- must keep appearing in autocomplete."""
result = _run_commands_js(
"""
await loadAgentCommandMetadata(true);
const reload = await getSlashAutocompleteMatches('/reload');
const sessions = await getSlashAutocompleteMatches('/sessions');
const resume = await getSlashAutocompleteMatches('/resume');
const plugin = await getSlashAutocompleteMatches('/plugin');
return {
reload_names: reload.map(item => item.name),
reload_sources: reload.map(item => item.source),
sessions_names: sessions.map(item => item.name),
resume_names: resume.map(item => item.name),
plugin_names: plugin.map(item => item.name)
};
"""
)
assert result["reload_names"] == ["reload-skills"]
assert result["reload_sources"] == ["agent"]
assert result["sessions_names"] == ["sessions"]
assert result["resume_names"] == ["resume"]
assert result["plugin_names"] == ["plugin-review"]
def test_autocomplete_allowlist_is_exact_parity_with_dispatchers():
"""#6951 (re-gate): the announced allowlist must be in exact agreement with
BOTH real dispatch authorities -- api/commands.py's _ALLOWED_AGENT_COMMANDS
(the /api/commands/exec allowlist) and messages.js's
_AGENT_COMMANDS_RUN_ON_WEBUI (the send() dispatch set). This compares the
actual parsed sets, so drift in ANY authority fails the test: a command
added to one allowlist but not the others, a renamed alias, or a removed
entry all break parity here."""
announced = _extract_js_set_members(COMMANDS_JS, "_WEBUI_DISPATCHABLE_AGENT_COMMANDS")
dispatched = _extract_js_set_members(MESSAGES_JS, "_AGENT_COMMANDS_RUN_ON_WEBUI")
# The backend-exec family must agree exactly across all three authorities
# (canonical names after alias normalization).
backend_allowed = set(_ALLOWED_AGENT_COMMANDS)
assert _canonical_agent_names(dispatched) == backend_allowed
assert _canonical_agent_names(announced) & backend_allowed == backend_allowed
# Announced backend-exec commands are exactly the backend-exec allowlist
# (no announced exec command outside /api/commands/exec, none missing).
announced_backend = _canonical_agent_names(announced) & backend_allowed
assert announced_backend == backend_allowed
# Every remaining announced command must be a native WebUI behavior that
# send() handles without an agent round-trip (moa/sessions/resume/pet).
native = announced - announced_backend
assert native == {"moa", "sessions", "resume", "pet"}
# The plugin transport is parity by rule, not by list: the filter accepts
# category==='Plugin' and the dispatcher routes the exact same value.
assert "category==='Plugin'" in COMMANDS_JS
assert "_agentCmd.category==='Plugin'" in MESSAGES_JS
def _run_shared_classic_realm_js() -> dict:
"""Evaluate commands.js then messages.js in ONE vm realm -- exactly how
static/index.html loads them (consecutive non-module `defer` classic
scripts share a single global lexical realm). A duplicate top-level
`const` across the two files throws during the second script's
declaration instantiation and aborts it before any statement runs, so the
second script's dispatch consts would never bind. Browser-only globals
referenced at load time are stubbed; runtime (call-time) references are
out of scope for this guard.
"""
script = textwrap.dedent(
f"""
const vm = require('vm');
const ctx = vm.createContext({{
console,
localStorage: {{ getItem(){{return null;}}, setItem(){{}}, removeItem(){{}} }},
t: (key) => key,
document: {{ addEventListener(){{}}, getElementById(){{ return null; }}, hidden: false }},
window: {{ addEventListener(){{}}, speechSynthesis: {{ speaking: false, paused: false }} }},
location: {{ href: 'http://localhost/', protocol: 'http:', host: 'localhost' }},
navigator: {{ userAgent: 'test' }},
setTimeout: () => 0, clearTimeout: () => 0,
setInterval: () => 0, clearInterval: () => 0,
requestAnimationFrame: () => 0, cancelAnimationFrame: () => 0,
fetch: async () => ({{}}),
EventSource: function(){{}},
MutationObserver: function(){{ this.observe=()=>{{}}; }},
Blob: function(){{}}, FileReader: function(){{ this.readAsText=()=>{{}}; }},
URL: function(){{ this.href=''; }}, FormData: function(){{}},
WebSocket: function(){{}},
crypto: {{ getRandomValues: (arr) => arr }},
CustomEvent: function(){{}}
}});
vm.runInContext({json.dumps(COMMANDS_JS)}, ctx);
let secondError = null;
try {{
vm.runInContext({json.dumps(MESSAGES_JS)}, ctx);
}} catch (e) {{
secondError = String(e && e.message || e);
}}
const bound = vm.runInContext(`({{
runOnWebui: typeof _AGENT_COMMANDS_RUN_ON_WEBUI,
dispatchable: typeof _WEBUI_DISPATCHABLE_AGENT_COMMANDS,
backendConst: typeof _BACKEND_EXECUTED_AGENT_COMMANDS
}})`, ctx);
process.stdout.write(JSON.stringify({{ secondError, ...bound }}));
"""
)
with tempfile.NamedTemporaryFile("w", suffix=".js", encoding="utf-8", delete=False) as handle:
handle.write(script)
script_path = Path(handle.name)
try:
proc = subprocess.run(["node", str(script_path)], check=True, capture_output=True, text=True)
finally:
script_path.unlink(missing_ok=True)
return json.loads(proc.stdout)
def test_commands_and_messages_evaluate_in_one_shared_classic_script_realm():
"""#6962 (re-gate): commands.js and messages.js must coexist as classic
scripts in the same global realm (static/index.html loads them back to
back). A duplicate top-level `const _BACKEND_EXECUTED_AGENT_COMMANDS`
across both files previously threw
`SyntaxError: Identifier ... has already been declared` on messages.js,
aborting its evaluation entirely. Both files must evaluate cleanly in one
realm, the dispatch consts of BOTH files must be bound, and the removed
duplicate authority mirror must not exist in the shared namespace."""
out = _run_shared_classic_realm_js()
assert out["secondError"] is None, (
f"messages.js failed to evaluate after commands.js in the shared realm: "
f"{out['secondError']}"
)
assert out["runOnWebui"] == "object", (
"messages.js dispatch const _AGENT_COMMANDS_RUN_ON_WEBUI not bound in "
"the shared realm -- declaration instantiation was aborted"
)
assert out["dispatchable"] == "object", (
"commands.js dispatch const _WEBUI_DISPATCHABLE_AGENT_COMMANDS not "
"bound in the shared realm"
)
assert out["backendConst"] == "undefined", (
"duplicate/unused _BACKEND_EXECUTED_AGENT_COMMANDS mirror must not be "
"declared in the shared realm (declared once -> it would collide; "
"declared in one file -> it is dead authority drift)"
)
def test_busy_path_intercepts_stop_before_mode_routing():
"""#6951: while busy, /stop must cancel the active run immediately instead
of being steered/queued as the literal text '/stop'."""
busy_idx = MESSAGES_JS.find("Busy-control slash commands must be intercepted")
assert busy_idx != -1
mode_idx = MESSAGES_JS.find("const defaultMessageMode=", busy_idx)
assert mode_idx != -1
busy_block = MESSAGES_JS[busy_idx:mode_idx]
assert "['steer','interrupt','queue','terminal','goal','yolo','stop']" in busy_block
assert "cmdStop" in busy_block or "COMMANDS.find(c=>c.name===_pc.name)" in busy_block
def _run_production_autocomplete_js(commands_payload: list[dict], script_body: str) -> dict:
"""Run the REAL getMatchingCommands/autocomplete pipeline from commands.js
against a /api/commands payload (defaults to the real hermes_cli registry
serialized exactly as list_commands() produces it)."""
script = textwrap.dedent(
f"""
const vm = require('vm');
const ctx = {{
console,
localStorage: {{ getItem(){{return null;}}, setItem(){{}}, removeItem(){{}} }},
t: (key) => key,
api: async (path) => {{
if (path === '/api/commands') return {{ commands: {json.dumps(commands_payload)} }};
if (path === '/api/commands/bundles') return {{ bundles: [] }};
if (path === '/api/skills') return {{ skills: [] }};
throw new Error('unexpected api path: ' + path);
}}
}};
vm.createContext(ctx);
vm.runInContext({json.dumps(COMMANDS_JS)}, ctx);
(async () => {{
const result = await vm.runInContext(`(async () => {{
{script_body}
}})()`, ctx);
process.stdout.write(JSON.stringify(result));
}})().catch(err => {{
console.error(err && err.stack || err);
process.exit(1);
}});
"""
)
with tempfile.NamedTemporaryFile("w", suffix=".js", encoding="utf-8", delete=False) as handle:
handle.write(script)
script_path = Path(handle.name)
try:
proc = subprocess.run(["node", str(script_path)], check=True, capture_output=True, text=True)
finally:
script_path.unlink(missing_ok=True)
return json.loads(proc.stdout)
def _real_registry_payload() -> list[dict] | None:
"""Serialize the real hermes_cli COMMAND_REGISTRY via the production
list_commands() path. Returns None when hermes_cli is unavailable."""
try:
return list_commands()
except Exception:
return None
def test_real_registry_announced_commands_are_all_dispatchable():
"""#6951 (re-gate): with the REAL hermes_cli registry as /api/commands
payload, every command the production autocomplete announces must be
dispatchable -- backend-exec via /api/commands/exec, a native WebUI
behavior, or a plugin command. /agents (in the registry, NOT dispatched)
must not be announced. This is the announced <= dispatched invariant
exercised against the real production data path."""
payload = _real_registry_payload()
if payload is None:
pytest.skip("hermes_cli registry unavailable")
announced_by_prefix = {}
for cmd in payload:
name = str(cmd.get("name") or "")
if not name:
continue
result = _run_production_autocomplete_js(
payload,
f"""
await loadAgentCommandMetadata(true);
const matches = await getSlashAutocompleteMatches('/{name}');
return matches.map(item => ({{ name: item.name, source: item.source }}));
""",
)
announced_by_prefix[name] = result
backend_allowed = set(_ALLOWED_AGENT_COMMANDS)
for name, rows in announced_by_prefix.items():
for row in rows:
if row["source"] not in ("agent", "plugin"):
continue # builtin/subarg/skill rows are out of scope
assert row["name"] in backend_allowed or row["name"] in (
"moa",
"sessions",
"resume",
"pet",
), f"{row['name']} announced but not dispatchable"
# /agents is in the real registry but must not be announced (it is neither
# backend-exec nor a native WebUI behavior nor a plugin).
assert announced_by_prefix.get("agents", []) == [], announced_by_prefix.get("agents")
def test_alias_typed_text_reaches_exec_dispatch():
"""#6951 (re-gate): typing an underscore alias (/reload_skills) must
resolve through getAgentCommandMetadata to the canonical name, which is
what the dispatcher allowlist tests -- so alias forms dispatch without
needing their own autocomplete row."""
result = _run_commands_js(
"""
const byAlias = await getAgentCommandMetadata('reload_skills');
const canonical = byAlias && byAlias.name;
return { canonical, dispatched: canonical === 'reload-skills' };
"""
)
assert result["canonical"] == "reload-skills"
assert result["dispatched"] is True
def _run_busy_intercept_js(script_body: str) -> dict:
"""Execute the REAL busy-path slash intercept block from messages.js with
the REAL COMMANDS table + parseCommand from commands.js. This exercises the
production routing branch -- not just its source text.
The harness defines `async function busyIntercept(text, literalSlash)` in
the vm context (so it closes over the real COMMANDS/parseCommand) and the
script_body then calls it. The block's inner `return;` exits run() early
when the command is intercepted, so `intercepted: true` is returned."""
block = _extract_busy_intercept_block()
script = textwrap.dedent(
f"""
const vm = require('vm');
const calls = [];
const ctx = {{
console,
localStorage: {{ getItem(){{return null;}}, setItem(){{}}, removeItem(){{}} }},
t: (key) => key,
S: {{
busy: true,
activeStreamId: 'stream-1',
session: {{ session_id: 'sess-1' }},
pendingFiles: []
}},
cancelStream: async (reason) => {{ calls.push('cancelStream:' + reason); return true; }},
showToast: () => {{}},
$: () => ({{ value: '' }}),
autoResize: () => {{}},
api: async () => ({{}}),
_trySteer: async () => {{ calls.push('steer'); }},
queueSessionMessage: () => {{ calls.push('queue'); }}
}};
vm.createContext(ctx);
vm.runInContext({json.dumps(COMMANDS_JS)}, ctx);
vm.runInContext(`async function busyIntercept(text, literalSlash){{
let reachedModeRouting = false;
const run = async () => {{
{block}
reachedModeRouting = true;
}};
await run();
return {{ intercepted: !reachedModeRouting }};
}}`, ctx);
(async () => {{
const result = await vm.runInContext(`(async () => {{
{script_body}
}})()`, ctx);
process.stdout.write(JSON.stringify({{ result, calls }}));
}})().catch(err => {{
console.error(err && err.stack || err);
process.exit(1);
}});
"""
)
with tempfile.NamedTemporaryFile("w", suffix=".js", encoding="utf-8", delete=False) as handle:
handle.write(script)
script_path = Path(handle.name)
try:
proc = subprocess.run(["node", str(script_path)], check=True, capture_output=True, text=True)
finally:
script_path.unlink(missing_ok=True)
return json.loads(proc.stdout)
def test_busy_stop_executes_real_cancel_branch():
"""#6951 (re-gate): executing the real busy-intercept branch from messages.js
with the real COMMANDS table must route /stop to cmdStop -> cancelStream,
and must NOT fall through to steer/queue routing."""
out = _run_busy_intercept_js(
"""
const r1 = await busyIntercept('/stop', false);
const r2 = await busyIntercept('/agents', false);
return { stop: r1, agents: r2 };
"""
)
# /stop is intercepted by the busy branch (cmdStop -> cancelStream ran,
# mode routing never reached), /agents is NOT a busy-control command so it
# falls through to mode routing (documented: it is no longer announced).
assert out["result"]["stop"] == {"intercepted": True}
assert out["result"]["agents"] == {"intercepted": False}
assert any(c.startswith("cancelStream:slash-stop") for c in out["calls"]), out["calls"]
assert not any(c.startswith("steer") for c in out["calls"]), out["calls"]
tests/test_comma_combined_content_length.py
+105
View File
@@ -0,0 +1,105 @@
"""Comma-combined ``Content-Length`` must not over-close a healthy connection.
RFC 9110 §5.3 lets a list-valued field arrive either as repeated header lines or
as a single comma-combined line, and the two spellings are equivalent. So
``Content-Length: 0, 0`` describes the same bodyless request as two separate
``Content-Length: 0`` headers, and rejecting it must keep the keep-alive.
``_declared_content_lengths()`` originally ran ``isdigit()`` across the whole raw
value, so the comma failed the check, the field read as *unreadable*, and every
helper-backed rejection armed ``Connection: close`` on a request with nothing
queued behind it. That is the over-close half of this PR's contract — the mirror
image of the bug it fixes, and the one that silently drops pooled connections.
The dangerous direction has to stay closed, so the disagreeing and malformed
spellings are asserted here alongside the recoverable ones: ``0, 42`` genuinely
cannot frame a body and must still be refused with the close armed.
"""
import pytest
from api.helpers import request_declares_body, unreadable_content_length
class _Hdrs(dict):
"""email.message-like: ``get()`` returns the first value, ``get_all()`` all."""
def get(self, k, default=None):
for key, val in self.items():
if key.lower() == k.lower():
return val[0] if isinstance(val, list) else val
return default
def get_all(self, k, default=None):
out = []
for key, val in self.items():
if key.lower() == k.lower():
out.extend(val if isinstance(val, list) else [val])
return out or default
class _H:
def __init__(self, headers):
self.headers = headers
def _handler(cl):
return _H(_Hdrs({"Content-Length": cl}))
@pytest.mark.parametrize("value", ["0, 0", "00, 00", "0 , 0", "0,0", "0,\t0"])
def test_agreeing_bodyless_comma_list_is_readable_and_bodyless(value):
"""An agreeing all-zero list is framed, bodyless, and must keep keep-alive."""
h = _handler(value)
assert unreadable_content_length(h) is False, (
f"Content-Length {value!r} read as unreadable; a bodyless rejection would "
"then arm Connection: close and drop a healthy pooled connection"
)
assert bool(request_declares_body(h)) is False, (
f"Content-Length {value!r} declares no body, so nothing is queued"
)
@pytest.mark.parametrize("value", ["42, 42", "42,42", "042, 42"])
def test_agreeing_nonzero_comma_list_is_readable_with_a_body(value):
"""An agreeing nonzero list frames a real body: readable, and a body IS pending."""
h = _handler(value)
assert unreadable_content_length(h) is False, value
assert bool(request_declares_body(h)) is True, value
@pytest.mark.parametrize("value", ["0, 42", "42, 0", "0, 0, 42"])
def test_disagreeing_comma_list_stays_unreadable(value):
"""Disagreeing members cannot frame a body — must stay refused, close armed."""
h = _handler(value)
assert unreadable_content_length(h) is True, (
f"Content-Length {value!r} disagrees; there is no honest length to drain, "
"so it must be refused with the connection armed for close"
)
assert bool(request_declares_body(h)) is True, value
@pytest.mark.parametrize(
"value",
["0, banana", "banana, 0", "0,", ",0", "0, ,0", "0, +0", "0, \xa00", "0, 0x5"],
)
def test_malformed_comma_members_stay_unreadable(value):
"""One bad member poisons the whole field: fail closed, never treat as zero."""
h = _handler(value)
assert unreadable_content_length(h) is True, (
f"Content-Length {value!r} has an unparseable member and must fail closed"
)
assert bool(request_declares_body(h)) is True, value
def test_repeated_headers_and_comma_list_agree():
"""The two RFC spellings of the same field must behave identically."""
repeated = _H(_Hdrs({"Content-Length": ["0", "0"]}))
combined = _handler("0, 0")
assert unreadable_content_length(repeated) == unreadable_content_length(combined)
assert bool(request_declares_body(repeated)) == bool(request_declares_body(combined))
repeated_bad = _H(_Hdrs({"Content-Length": ["0", "42"]}))
combined_bad = _handler("0, 42")
assert unreadable_content_length(repeated_bad) is True
assert unreadable_content_length(combined_bad) is True
tests/test_compact_echo_index.py
+218
View File
@@ -0,0 +1,218 @@
"""``_CompactEchoIndex`` must agree with the raw backward walk, cheaply.
The index is the echo-match authority for both the journal rebuild
(``api/routes.py::_run_journal_live_snapshot``) and the live stream
(``api/streaming.py``). It replaces a per-interim raw backward walk that
rescanned the whole transcript's whitespace — quadratic on whitespace-heavy
text (#7569). Two contracts must hold for the swap to be safe:
1. **Equivalence** — incremental appends must reach the same cut point as a
raw walk over the concatenated text, for every shape including chunked
appends, whitespace-only chunks, and exotic Unicode whitespace.
2. **Cost** — the index must never rescan the buffer it has already folded:
a probe after a wide whitespace span must not walk that span.
"""
import random
import re
import string
import pytest
def _compact(value):
return re.sub(r'\s+', '', str(value or ''))
def _raw_reference(value, suffix):
"""Deliberately naive oracle: probe every cut index, fold the tail."""
raw = str(value or '')
candidate = _compact(suffix)
if not raw or not candidate:
return raw, False
for idx in range(len(raw) + 1):
if _compact(raw[idx:]) == candidate:
return raw[:idx].rstrip(), True
return raw, False
def _cases():
cases = [
# Exact echo at the end.
("bla bla bla une conclusion", "une conclusion"),
# Echo differing only by whitespace shape.
("bla bla une conclusion", "une conclusion"),
("bla bla\nune\tconclusion", "une conclusion"),
# No echo.
("texte quelconque sans rapport", "une conclusion"),
# Partial echo must NOT match.
("bla une conclu", "une conclusion"),
# The buffer is exactly the echo.
("une conclusion", "une conclusion"),
# Empty / None inputs.
("", "une conclusion"),
("bla bla", ""),
("", ""),
(None, "x"),
("x", None),
# Whitespace around the cut point.
("bla bla une conclusion ", "une conclusion"),
("bla bla \n une conclusion", "une conclusion"),
# Unicode, accents, emoji, exotic whitespace.
("préambule ✅ conclusion émise", "conclusion émise"),
("texte 🟢 Réponse", "🟢 Réponse"),
("bla\u00a0bla une conclusion", "une\u00a0conclusion"),
("bla\u2028une conclusion", "une conclusion"),
# Whitespace-only buffer.
(" ", "une conclusion"),
# Repetitions: the leftmost cut point is contractual.
("abc abc abc", "abc"),
("abc abc abc", "abc abc"),
("aaaa", "aa"),
]
rnd = random.Random(20260920)
alphabet = string.ascii_letters + " \n\t" + "éà✅\u00a0"
for _ in range(2000):
base = "".join(rnd.choice(alphabet) for _ in range(rnd.randint(0, 120)))
if rnd.random() < 0.5 and len(base) > 4:
k = rnd.randint(1, max(1, len(base) // 2))
suffix = base[-k:]
else:
suffix = "".join(rnd.choice(alphabet) for _ in range(rnd.randint(0, 30)))
cases.append((base, suffix))
return cases
def test_index_matches_raw_reference_oracle():
"""Incremental index == naive raw probe, for one-shot and chunked appends."""
import api.streaming as streaming
divergences = []
for value, suffix in _cases():
expected = _raw_reference(value, suffix)
index = streaming._CompactEchoIndex()
index.append(value)
actual = index.cut_to(suffix)
if actual is None:
got = (str(value or ''), False)
else:
got = (str(value or '')[:actual].rstrip(), True)
if got != expected:
divergences.append((value, suffix, expected, got))
assert not divergences[:5], (
f"{len(divergences)} divergence(s) from the reference implementation; "
f"first: {divergences[:1]}"
)
def test_chunked_appends_reach_the_same_cut_point():
"""Appending in pieces must equal appending the whole string."""
import api.streaming as streaming
rnd = random.Random(99)
for _ in range(500):
parts = [
"".join(
rnd.choice(string.ascii_letters + " \n\té")
for _ in range(rnd.randint(1, 20))
)
for _ in range(rnd.randint(1, 8))
]
value = "".join(parts)
if rnd.random() < 0.6 and len(value) > 4:
suffix = value[-rnd.randint(1, len(value) // 2):]
else:
suffix = "".join(rnd.choice("abc \t") for _ in range(rnd.randint(0, 12)))
chunked = streaming._CompactEchoIndex()
for part in parts:
chunked.append(part)
whole = streaming._CompactEchoIndex()
whole.append(value)
assert chunked.cut_to(suffix) == whole.cut_to(suffix), (
f"chunked append diverged for value={value!r} suffix={suffix!r}"
)
def test_matches_tail_and_cut_to_agree():
"""The boolean probe and the cut probe must never disagree."""
import api.streaming as streaming
for value, suffix in _cases():
index = streaming._CompactEchoIndex()
index.append(value)
assert index.matches_tail(suffix) == (index.cut_to(suffix) is not None), (
f"matches_tail/cut_to disagree for value={value!r} suffix={suffix!r}"
)
def test_reset_clears_state_after_truncation():
"""After reset + re-append the index must describe only the new text."""
import api.streaming as streaming
index = streaming._CompactEchoIndex()
index.append("old reasoning tail with a wide span " + " " * 5000)
index.reset()
index.append("fresh start only")
assert index.compact_length == len("freshstartonly")
assert index.cut_to("start only") == len("fresh ")
assert index.cut_to("old reasoning") is None
def test_index_probe_does_not_rescan_a_wide_whitespace_span(monkeypatch):
"""The probe must not walk raw characters it already folded.
Pins the cost property the raw backward walk violated: after a 1 MB
whitespace span is appended, a probe must cost O(candidate), not
O(span). Counting ``str.isspace``-adjacent raw access is impractical, so
this counts how many characters ``append`` has to fold: exactly the
appended length, once — and the probe itself must not fold anything.
"""
import api.streaming as streaming
fold_calls = {'n': 0}
original = streaming._compact_for_echo_compare
def counting(value):
fold_calls['n'] += 1
return original(value)
monkeypatch.setattr(streaming, '_compact_for_echo_compare', counting)
index = streaming._CompactEchoIndex()
index.append("alpha" + " " * 100_000)
fold_calls['n'] = 0
# 20 probes of a short candidate: the raw walk would fold/walk the span
# each time; the index folds only the candidate.
for i in range(20):
assert index.matches_tail(f"progress {i}") is False
assert fold_calls['n'] <= 20, (
f"echo probe folded {fold_calls['n']} times — it is rescanning the "
"raw transcript instead of using the indexed tail"
)
@pytest.mark.timeout(60)
def test_index_stays_cheap_on_a_wide_span_with_many_probes():
"""Wide whitespace span + many probes must not cost seconds of CPU."""
import time
import api.streaming as streaming
index = streaming._CompactEchoIndex()
index.append("alpha" + " " * 1_000_000)
probes = [f"progress {i}" for i in range(200)]
start = time.perf_counter()
for probe in probes:
index.matches_tail(probe)
index.cut_to(probe)
elapsed = time.perf_counter() - start
# The raw backward walk needs ~0.05s per probe here (2M+ character walks
# total). A generous budget still fails it by an order of magnitude.
assert elapsed < 0.5, f"200 probes on a 1MB span took {elapsed:.3f}s"
tests/test_compact_echo_suffix_linear_scan.py
+28 -40
View File
@@ -7,11 +7,16 @@ with ``window x suffix`` instead of with the data actually inspected. On a
6000-character conclusion block that is seconds of pure CPU, held under the
GIL, which stalls every other stream in the process.
The current implementation walks ``value`` and ``suffix`` backwards from the
end (skipping whitespace in ``value``) and has no fixed search window, so a
compact-equivalent echo is found regardless of how much interior whitespace
stretches its raw span.
These tests pin both halves of the contract:
* the fast path must return byte-identical results to a straightforward
reference implementation, including the leftmost-cut tie-break; and
* it must not re-fold the search window once per candidate cut index.
* it must not fold the whole buffer once per candidate cut index.
"""
import random
import re
@@ -20,11 +25,12 @@ import string
import pytest
def _reference_strip_compact_echo_suffix(value, suffix, *, search_window: int = 4096):
"""Deliberately naive oracle: probe every cut index, fold the whole tail.
def _reference_strip_compact_echo_suffix(value, suffix):
"""Deliberately naive oracle: probe every cut index, fold the tail.
This mirrors the original behaviour exactly and exists only so the
optimised implementation can be proven equivalent to it.
This mirrors the pre-window semantics (the original probing loop) with no
search window, and exists only so the optimised backward walk can be
proven equivalent to it.
"""
def compact(v):
return re.sub(r'\s+', '', str(v or ''))
@@ -33,11 +39,9 @@ def _reference_strip_compact_echo_suffix(value, suffix, *, search_window: int =
candidate = compact(suffix)
if not raw or not candidate:
return raw, False
tail = raw[-max(len(str(suffix or '')) * 3, search_window):]
offset = len(raw) - len(tail)
for idx in range(len(tail) + 1):
if compact(tail[idx:]) == candidate:
return raw[: offset + idx].rstrip(), True
for idx in range(len(raw) + 1):
if compact(raw[idx:]) == candidate:
return raw[:idx].rstrip(), True
return raw, False
@@ -74,8 +78,8 @@ def _cases():
("aaaa", "aa"),
# Long buffer with the echo at the very end.
("remplissage " * 500 + "la vraie conclusion", "la vraie conclusion"),
# Echo whose whitespace-padded span is wider than the search window:
# the window only exposes ``conclusion``, so nothing may be stripped.
# Echo whose whitespace-padded span is wider than any fixed fold
# window: the backward walk must still strip it.
("bla une" + " " * 5000 + "conclusion", "une conclusion"),
# Exotic whitespace that ``\s`` and ``str.isspace`` must treat alike.
("bla\u00a0bla une conclusion", "une\u00a0conclusion"),
@@ -111,45 +115,29 @@ def test_strip_compact_echo_suffix_matches_reference_oracle():
)
def test_strip_compact_echo_suffix_honours_a_custom_search_window():
"""The bounded window must keep its meaning on the fast path."""
import api.streaming as streaming
buffer_text = "z" * 400 + " la conclusion"
for window in (16, 64, 512, 4096):
assert streaming._strip_compact_echo_suffix(
buffer_text, "la conclusion", search_window=window
) == _reference_strip_compact_echo_suffix(
buffer_text, "la conclusion", search_window=window
), f"window={window}"
def test_strip_compact_echo_suffix_rejects_an_echo_wider_than_the_window():
"""An echo that only fits inside a wider window must not be found.
def test_strip_compact_echo_suffix_finds_an_echo_wider_than_any_fixed_window():
"""An echo stretched past any fold window must still be stripped.
The buffer ends with the echo, but interior whitespace stretches its raw
span past the default window, so the folded window only exposes the last
word. Widening the window to cover the whole span makes the same echo
visible again, which proves the non-match is caused by the window and
not by the text.
span far beyond a fixed fold window (the retired windowed implementation
exposed only the last word and left the buffer untouched). The backward
walk has no window, so the cut point is found and the echo is removed.
"""
import api.streaming as streaming
buffer_text = "bla une" + " " * 5000 + "conclusion"
suffix = "une conclusion"
for impl in (streaming._strip_compact_echo_suffix, _reference_strip_compact_echo_suffix):
assert impl(buffer_text, suffix) == (buffer_text, False), impl.__name__
assert impl(buffer_text, suffix, search_window=len(buffer_text)) == ("bla", True), impl.__name__
assert streaming._strip_compact_echo_suffix(buffer_text, suffix) == ("bla", True)
assert _reference_strip_compact_echo_suffix(buffer_text, suffix) == ("bla", True)
def test_strip_compact_echo_suffix_does_not_refold_the_window_per_cut(monkeypatch):
def test_strip_compact_echo_suffix_does_not_refold_per_cut(monkeypatch):
"""One pass, not one fold per candidate cut index.
The original implementation called the whitespace-folding helper once per
possible cut position — roughly 4097 times for a default window — and each
of those calls re-scanned the remaining tail. Counting the calls pins the
algorithmic property directly, without depending on wall-clock timing.
Counting the folding-helper calls pins the algorithmic property directly,
without depending on wall-clock timing: the backward walk folds the
suffix once and then compares characters.
"""
import api.streaming as streaming
@@ -168,7 +156,7 @@ def test_strip_compact_echo_suffix_does_not_refold_the_window_per_cut(monkeypatc
assert calls['n'] <= 8, (
f"whitespace folding ran {calls['n']} times for a single call; the scan "
"is re-folding the window once per candidate cut index"
"is re-folding once per candidate cut index"
)
tests/test_compaction_current_summary_fallback.py
+4
View File
@@ -50,6 +50,10 @@ _REAL_FUNCTIONS = (
"_insertPreservedCompressionTaskFallback",
"_pinCompactionCardAtTop",
"_pinSettledCompressionReferenceAtTop",
# #2051: renderMessages() inserts a message block through this helper, so it has to be
# evaluated with it. The shim's createElement() returns no template `content`, so the
# helper takes its insertAdjacentHTML fallback here, exactly as before.
"_insertSegmentBlock",
"renderMessages",
)
tests/test_compressed_idle_resume.py
+224
View File
@@ -0,0 +1,224 @@
"""A stale WebUI sidecar must not hide a durable compression continuation."""
import io
import json
import subprocess
from pathlib import Path
from types import SimpleNamespace
import pytest
from api import routes, profiles
@pytest.fixture
def lineage(tmp_path, monkeypatch):
state = pytest.importorskip("hermes_state")
# Worker tests restore sys.path after loading Agent lazily. Scope its
# sibling-module lookup to this fixture, never the global test importer.
monkeypatch.syspath_prepend(str(Path(state.__file__).parent))
SessionDB = state.SessionDB
db = SessionDB(tmp_path / "state.db")
db.create_session("sealedparent", source="webui")
db.create_session("idlechild", source="tui", parent_session_id="sealedparent")
db.append_message("idlechild", "user", "retained task")
db.end_session("sealedparent", "compression")
db.end_session("idlechild", "idle_timeout")
monkeypatch.setattr(profiles, "_resolve_profile_home_for_name", lambda name: str(tmp_path))
session = SimpleNamespace(session_id="sealedparent", profile="default", pre_compression_snapshot=False)
yield db, session
db.close()
def test_reload_resolves_sqlite_rotation_without_sidecar_flag(lineage):
db, session = lineage
before = db.get_session("idlechild")["ended_at"]
assert routes._pre_compression_continuation_session_id(session) == "idlechild"
assert db.get_session("idlechild")["ended_at"] == before
assert db.get_session("sealedparent")["end_reason"] == "compression"
class Handler:
headers = {}
def __init__(self):
self.wfile = io.BytesIO()
def send_response(self, status):
self.status = status
def send_header(self, *args):
pass
def end_headers(self):
pass
def _assert_stale_post_rotation(session, monkeypatch, expected_continuation):
monkeypatch.setattr(routes, "_agent_runtime_barrier_response", lambda **kw: None)
monkeypatch.setattr(routes, "_get_or_materialize_session", lambda *a, **kw: session)
monkeypatch.setattr(routes, "_get_active_profile_name", lambda: "default")
monkeypatch.setattr(routes, "_session_visible_to_active_profile", lambda *a: True)
monkeypatch.setattr(routes, "_resolve_chat_workspace_with_recovery", lambda *a: pytest.fail("stale turn must not reach workspace mutation"))
h = Handler()
routes._handle_chat_start(h, {"session_id": session.session_id, "message": "conclusion?"})
assert h.status == 409
payload = json.loads(h.wfile.getvalue())
assert payload["code"] == "session_rotated"
assert payload["continuation_session_id"] == expected_continuation
def test_stale_post_rejected_before_workspace_or_worker_mutation(lineage, monkeypatch):
_, session = lineage
_assert_stale_post_rotation(session, monkeypatch, "idlechild")
@pytest.mark.parametrize("source", ["desktop", "acp"])
def test_same_source_local_interactive_lineage_resumes(lineage, monkeypatch, source):
db, session = lineage
db._conn.execute(
"UPDATE sessions SET source=? WHERE id IN ('sealedparent', 'idlechild')",
(source,),
)
db._conn.commit()
assert routes._pre_compression_continuation_session_id(session) == "idlechild"
_assert_stale_post_rotation(session, monkeypatch, "idlechild")
@pytest.mark.parametrize(
"source",
["cron", "webhook", "kanban", "tool", "subagent", "unknown-source"],
)
def test_noninteractive_lineage_is_not_resumable(lineage, monkeypatch, source):
db, session = lineage
db._conn.execute(
"UPDATE sessions SET source=? WHERE id IN ('sealedparent', 'idlechild')",
(source,),
)
db._conn.commit()
assert routes._pre_compression_continuation_session_id(session) is None
_assert_stale_post_rotation(session, monkeypatch, None)
def test_idle_tip_accepts_normal_persistence_without_reopening_parent(lineage):
db, session = lineage
tip = routes._pre_compression_continuation_session_id(session)
db.append_message(tip, "user", "resume task")
assert db.get_session("sealedparent")["end_reason"] == "compression"
assert db.get_messages(tip)[-1]["content"] == "resume task"
@pytest.mark.parametrize("reason", ["reset", "user_closed", "compression"])
@pytest.mark.parametrize("ended_at", [None, 1234.0])
def test_unknown_or_sealed_terminal_tip_is_not_redirected(lineage, reason, ended_at):
db, session = lineage
db._conn.execute("UPDATE sessions SET end_reason=?, ended_at=? WHERE id='idlechild'", (reason, ended_at))
db._conn.commit()
assert routes._pre_compression_continuation_session_id(session) is None
def test_newer_branch_does_not_replace_canonical_continuation(lineage):
db, session = lineage
db.create_session("branchchild", source="webui", parent_session_id="sealedparent")
db._conn.execute("UPDATE sessions SET message_count=20, model_config=? WHERE id='branchchild'", (json.dumps({'_branched_from': 'sealedparent'}),))
db._conn.commit()
assert routes._pre_compression_continuation_session_id(session) == "idlechild"
def test_foreign_profile_and_subagent_are_not_redirected(lineage):
db, session = lineage
db._conn.execute("UPDATE sessions SET profile_name='foreign' WHERE id='idlechild'")
db._conn.commit()
assert routes._pre_compression_continuation_session_id(session) is None
db._conn.execute("UPDATE sessions SET profile_name=NULL, source='subagent' WHERE id='idlechild'")
db._conn.commit()
assert routes._pre_compression_continuation_session_id(session) is None
def test_noncompressed_parent_and_absent_database_keep_legacy_behavior(lineage, monkeypatch, tmp_path):
db, session = lineage
db._conn.execute("UPDATE sessions SET end_reason='idle_timeout' WHERE id='sealedparent'")
db._conn.commit()
assert routes._pre_compression_continuation_session_id(session) is None
monkeypatch.setattr(profiles, "_resolve_profile_home_for_name", lambda name: str(tmp_path / 'missing'))
assert routes._pre_compression_continuation_session_id(session) is None
assert not (tmp_path / 'missing').exists()
@pytest.mark.parametrize('method', ['get_session', 'get_compression_tip'])
@pytest.mark.parametrize('kind', ['missing', 'noncallable', 'signature'])
def test_older_agent_keeps_sidecar_recovery(tmp_path, monkeypatch, method, kind):
import sys
from api.compression_continuation import durable_compression_continuation
class OldDB:
closed = False
def __init__(self, *args, **kwargs):
pass
def get_session(self, sid):
return {'end_reason': 'compression'}
def get_compression_tip(self, sid):
return 'legacychild'
def close(self):
OldDB.closed = True
if kind == 'missing':
delattr(OldDB, method)
elif kind == 'noncallable':
setattr(OldDB, method, None)
else:
setattr(OldDB, method, lambda self, sid, required: None)
monkeypatch.setitem(sys.modules, 'hermes_state', SimpleNamespace(SessionDB=OldDB))
(tmp_path / 'state.db').touch()
monkeypatch.setattr(profiles, '_resolve_profile_home_for_name', lambda _: str(tmp_path))
session = SimpleNamespace(session_id='legacyparent', profile='default', pre_compression_snapshot=True)
child = SimpleNamespace(session_id='legacychild', profile='default',
parent_session_id='legacyparent', pre_compression_snapshot=False,
updated_at=2, created_at=1)
monkeypatch.setattr(routes, 'SESSIONS', {'legacychild': child})
monkeypatch.setattr(routes, 'SESSION_DIR', tmp_path)
assert durable_compression_continuation(session) == (False, None)
assert OldDB.closed
assert routes._pre_compression_continuation_session_id(session) == 'legacychild'
def test_browser_rotation_restores_draft_without_reposting():
source = (Path(__file__).parents[1] / 'static/messages.js').read_text()
helper = source[source.index('async function _recoverCompressedSend('):source.index('function _restoreComposerDraftAfterFailedSend(')]
script = r'''
const assert = require('node:assert/strict');
let S={session:{session_id:'old'}}, INFLIGHT={old:{}};
const calls=[];
const stopApprovalPolling=()=>{},stopClarifyPolling=()=>{},removeThinking=()=>{},setBusy=()=>{},setComposerStatus=()=>{},showToast=()=>{};
const loadSession=async sid=>{calls.push(['load',sid]);S.session={session_id:sid}};
const _restoreComposerDraftAfterFailedSend=(...args)=>calls.push(['restore',...args]);
const api=()=>{throw Error('must not repost')};
(async()=>{
const err={status:409,body:JSON.stringify({code:'session_rotated',continuation_session_id:'new'})};
const files=[{name:'drawing.png'}], promise=Promise.resolve();
assert.equal(await _recoverCompressedSend(err,'old','conclusion?',files,promise),true);
assert.deepEqual(calls[0],['load','new']);
assert.deepEqual(calls[1],['restore','conclusion?',files,'new',promise]);
assert.equal(INFLIGHT.old,undefined);
assert.equal(await _recoverCompressedSend({status:500},'new','x',[],promise),false);
assert.equal(await _recoverCompressedSend(err,'old','x',[],promise),false);
assert.equal(calls.length,2);
})().catch(e=>{console.error(e);process.exit(1)});
'''
subprocess.run(['node', '-e', helper + script], check=True, timeout=15)
def test_background_failed_draft_preserves_attachments_after_navigation_race():
source = (Path(__file__).parents[1] / 'static/messages.js').read_text()
helper = source[source.index('function _restoreComposerDraftAfterFailedSend('):source.index('async function send(){')]
script = r'''
const assert=require('node:assert/strict');
const S={session:{session_id:'unrelated'},pendingFiles:[]};
const $=()=>{throw Error('unrelated visible composer must not be touched')};
const calls=[];
const _saveComposerDraftNow=(...args)=>calls.push(args);
(async()=>{
const files=[{name:'drawing.png',path:'/uploads/drawing.png'}];
_restoreComposerDraftAfterFailedSend('retained',files,'continuation',Promise.resolve());
await Promise.resolve();
assert.deepEqual(calls,[['continuation','retained',files]]);
})().catch(e=>{console.error(e);process.exit(1)});
'''
subprocess.run(['node','-e',helper+script], check=True, timeout=15)
tests/test_gateway_approval_legacy_path.py
+345
View File
@@ -10,6 +10,8 @@ import threading
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from api.gateway_chat import _gateway_runs_approval_event
REPO_ROOT = Path(__file__).parent.parent
@@ -873,3 +875,346 @@ def test_legacy_approval_without_run_id_retires_locally():
r._pending.pop(session_id, None)
r._gateway_queues.pop(session_id, None)
_STREAM_RUN_IDS.pop(stream_id, None)
def test_route_deny_settles_exact_non_head_run_producer():
"""The response route must settle the exact producer whose card was denied.
The route is the only caller that supplies a target's ``(run_id,
approval_id)`` to the legacy resolver, so a regression in that wiring is
invisible to helper-level coverage: the producer would be dropped from the
queue without its waiter ever being woken, leaving the agent thread parked
until the approval timeout.
Unlike the route-level test in ``test_approval_unblock.py`` (which needs the
installed agent's ``tools.approval`` and is skipped without it), this one
runs wherever the suite runs.
"""
from types import SimpleNamespace
from api import route_approvals as ra
from api import routes as r
sid = "sess-route-non-head-deny"
run_id = "run-route-non-head-deny"
head_id = "approval-route-head"
target_id = "approval-route-target"
sibling_id = "approval-route-sibling"
other_run_id = "run-route-unrelated"
head = {
"approval_id": head_id,
"run_id": run_id,
"command": "echo head",
"description": "Head approval on the run",
"_gateway_agent_identity_v1": True,
}
sibling = {
"approval_id": sibling_id,
"run_id": run_id,
"command": "echo sibling",
"description": "Non-head sibling approval on the same run",
"_gateway_agent_identity_v1": True,
}
target = {
"approval_id": target_id,
"run_id": run_id,
"command": "echo target",
"description": "Non-head approval on the same run",
"_gateway_agent_identity_v1": True,
}
other_run = {
"approval_id": "approval-route-other",
"run_id": other_run_id,
"command": "echo else",
"description": "Approval on an unrelated run",
}
def producer(payload):
"""The agent's real producer entry when installed, an equivalent one otherwise.
The suite runs without hermes-agent in CI, where
``tools.approval._ApprovalEntry`` does not exist; the stand-in carries the
same ``data`` / ``event`` / ``result`` / ``reason`` contract the resolution
path touches, so the assertions below mean the same thing in both shapes.
"""
try:
from tools.approval import _ApprovalEntry
except Exception:
_ApprovalEntry = None
if _ApprovalEntry is not None:
return _ApprovalEntry(dict(payload))
return SimpleNamespace(
data=dict(payload), event=threading.Event(), result=None, reason=None
)
head_entry = producer(head)
sibling_entry = producer(sibling)
target_entry = producer(target)
other_run_entry = producer(other_run)
relayed = []
captured = {}
def fake_j(_handler, data, status=200, extra_headers=None):
captured.update(payload=data, status=status)
return data
def fake_respond(_self, got_run_id, got_approval_id, choice):
relayed.append((got_run_id, got_approval_id, choice))
return {"resolved": 1}
try:
with ra._lock:
ra._pending.pop(sid, None)
ra._gateway_queues[sid] = [head_entry, sibling_entry, target_entry, other_run_entry]
ra.submit_gateway_pending_mirror(sid, dict(target_entry.data))
mirror = ra.gateway_pending_mirror(sid, approval_id=target_id, run_id=run_id)
assert mirror is not None, "the non-head approval must be mirrored before it is answered"
with patch("api.routes.j", new=fake_j), \
patch("api.runner_client.HttpRunnerClient.respond_approval", new=fake_respond), \
patch("api.config.gateway_supports_approval_identity_v1", return_value=True):
r._handle_approval_respond(
object(),
{
"session_id": sid,
"choice": "deny",
"approval_id": target_id,
"run_id": run_id,
"mirror_token": mirror[ra._GATEWAY_MIRROR_TOKEN],
},
)
assert captured == {
"payload": {"ok": True, "choice": "deny", "relayed": True},
"status": 200,
}
assert relayed == [(run_id, target_id, "deny")]
assert target_entry.event.is_set(), "the denied approval's waiter must be woken"
assert target_entry.result == "deny"
with ra._lock:
remaining = list(ra._gateway_queues.get(sid) or [])
assert target_entry not in remaining, "the denied approval's producer must be consumed"
assert head_entry in remaining, (
"the run's own head approval must not be consumed by answering another approval"
)
assert not head_entry.event.is_set()
assert sibling_entry in remaining, (
"a sibling approval on the same run must not be consumed"
)
assert not sibling_entry.event.is_set()
assert other_run_entry in remaining, (
"an unrelated run's producer must not be consumed"
)
assert not other_run_entry.event.is_set()
# These are the observations made by subsequent HTTP polling.
for _ in range(2):
with ra._lock:
ra.reconcile_gateway_pending_mirror_locked(sid)
assert ra.gateway_pending_mirror(
sid, approval_id=target_id, run_id=run_id
) is None, "reconciliation must not resurrect the denied approval's mirror"
finally:
with ra._lock:
ra._pending.pop(sid, None)
ra._gateway_queues.pop(sid, None)
def test_deny_settles_run_producer_so_reconciliation_cannot_resurrect_mirror():
"""Deny must settle the producer as well as its WebUI mirror.
Selection is the behaviour under test, so the queue carries more than the
one entry being resolved: a sibling approval for the same run, and a producer
belonging to a different run. Resolving one approval must consume exactly
that producer, wake its waiter, and leave both others queued.
"""
from types import SimpleNamespace
from api import route_approvals as ra
sid = "sess-deny-no-resurrection"
approval_id = "approval-deny-no-resurrection"
run_id = "run-deny-no-resurrection"
sibling_approval_id = "approval-deny-sibling"
other_run_id = "run-deny-unrelated"
approval = {
"approval_id": approval_id,
"run_id": run_id,
"command": "echo pricing",
"description": "Run pricing probe",
}
sibling_approval = {
"approval_id": sibling_approval_id,
"run_id": run_id,
"command": "echo sibling",
"description": "Sibling approval on the same run",
}
other_run_approval = {
"approval_id": "approval-deny-other-run",
"run_id": other_run_id,
"command": "echo else",
"description": "Approval on an unrelated run",
}
def producer(payload):
return SimpleNamespace(data=dict(payload), event=threading.Event(), result=None)
target = producer(approval)
same_run_sibling = producer(sibling_approval)
other_run = producer(other_run_approval)
try:
with ra._lock:
ra._pending.pop(sid, None)
ra._gateway_queues[sid] = [target, same_run_sibling, other_run]
queued_before = list(ra._gateway_queues[sid])
ra.submit_gateway_pending_mirror(sid, dict(approval))
mirror = ra.gateway_pending_mirror(sid, approval_id=approval_id, run_id=run_id)
assert mirror is not None
resolved, _head, _total = ra.resolve_gateway_pending_run(
sid,
approval_id=approval_id,
run_id=run_id,
choice="deny",
)
assert resolved == 1
assert queued_before == [target, same_run_sibling, other_run], (
"the queue should have held all three producers before retirement"
)
with ra._lock:
remaining = list(ra._gateway_queues.get(sid) or [])
assert target not in remaining, "the denied approval's producer must be consumed"
assert target.event.is_set(), "the denied approval's waiter must be woken"
assert target.result == "deny"
assert same_run_sibling in remaining, (
"a sibling approval on the same run must not be consumed by retiring another approval"
)
assert other_run in remaining, (
"an unrelated run's producer must not be consumed"
)
# These are the same observations made by subsequent HTTP polling: the
# denied approval must not come back, and the survivors must stay live.
# `gateway_pending_mirror` acquires `_lock` itself, so it must be called
# outside the locked section.
for _ in range(2):
with ra._lock:
ra.reconcile_gateway_pending_mirror_locked(sid)
assert ra.gateway_pending_mirror(
sid, approval_id=approval_id, run_id=run_id
) is None, "reconciliation must not resurrect the denied approval's mirror"
with ra._lock:
refreshed = list(ra._gateway_queues.get(sid) or [])
assert target not in refreshed
assert same_run_sibling in refreshed
assert other_run in refreshed
finally:
with ra._lock:
ra._pending.pop(sid, None)
ra._gateway_queues.pop(sid, None)
@pytest.mark.parametrize(
"failure_point",
["reconcile_gateway_pending_mirror_locked", "_approval_sse_notify_locked"],
)
def test_exact_run_resolution_settles_before_projection_failure(failure_point):
"""Projection failures must not strand an already-consumed producer."""
from types import SimpleNamespace
from api import route_approvals as ra
sid = f"sess-exact-projection-failure-{failure_point}"
run_id = "run-exact-projection-failure"
target = SimpleNamespace(
data={"run_id": run_id, "approval_id": "approval-target"},
event=threading.Event(),
result=None,
reason=None,
)
sibling = SimpleNamespace(
data={"run_id": run_id, "approval_id": "approval-sibling"},
event=threading.Event(),
result=None,
reason=None,
)
try:
with ra._lock:
ra._pending.pop(sid, None)
ra._gateway_queues[sid] = [target, sibling]
with patch.object(ra, failure_point, side_effect=RuntimeError("projection failed")):
with pytest.raises(RuntimeError, match="projection failed"):
ra.resolve_gateway_pending_run(
sid,
approval_id="approval-target",
run_id=run_id,
choice="deny",
reason="user denied",
)
assert target.result == "deny"
assert target.reason == "user denied"
assert target.event.is_set(), "the consumed producer must be woken before projection work"
assert sibling.result is None
assert sibling.reason is None
assert not sibling.event.is_set()
with ra._lock:
assert ra._gateway_queues[sid] == [sibling]
finally:
with ra._lock:
ra._pending.pop(sid, None)
ra._gateway_queues.pop(sid, None)
@pytest.mark.parametrize(
"terminal_reason",
[
"Gateway run completed before approval resolution",
"Gateway run failed before approval resolution",
"Gateway run was cancelled before approval resolution",
"Gateway run ended during teardown before approval resolution",
],
)
def test_terminal_run_settlement_denies_all_run_producers_and_preserves_other_runs(
terminal_reason,
):
"""Every terminal exit fails closed without consuming another run's producer."""
from types import SimpleNamespace
from api import route_approvals as ra
sid = f"sess-terminal-run-{terminal_reason.split()[2]}"
run_id = "run-terminal"
def producer(entry_run_id, approval_id):
return SimpleNamespace(
data={"run_id": entry_run_id, "approval_id": approval_id},
event=threading.Event(),
result=None,
reason=None,
)
targets = [producer(run_id, "approval-a"), producer(run_id, "approval-b")]
survivor = producer("run-other", "approval-other")
try:
with ra._lock:
ra._pending.pop(sid, None)
ra._gateway_queues[sid] = [targets[0], survivor, targets[1]]
settled, _head, _total = ra.settle_gateway_pending_run(
sid, run_id, reason=terminal_reason
)
assert settled == 2
for target in targets:
assert target.result == "deny"
assert target.reason == terminal_reason
assert target.event.is_set()
assert survivor.result is None
assert survivor.reason is None
assert not survivor.event.is_set()
with ra._lock:
assert ra._gateway_queues[sid] == [survivor]
finally:
with ra._lock:
ra._pending.pop(sid, None)
ra._gateway_queues.pop(sid, None)
tests/test_gateway_approval_runs_api.py
+221 -10
View File
@@ -333,6 +333,178 @@ def test_gateway_runs_api_submission():
assert captured["body_extras"]["reasoning_effort"] == "high"
@pytest.mark.parametrize(
("terminal_event", "terminal_reason"),
[
("run.completed", "Gateway run completed before approval resolution"),
("run.failed", "Gateway run failed before approval resolution"),
("run.cancelled", "Gateway run was cancelled before approval resolution"),
],
)
def test_gateway_terminal_event_fail_closes_parked_run_producer(
terminal_event,
terminal_reason,
):
"""Each terminal gateway event denies and wakes the run's parked producers."""
from api import gateway_chat
from api import route_approvals as approvals
sid = f"sess-terminal-event-{terminal_event}"
stream_id = f"stream-terminal-event-{terminal_event}"
run_id = f"run-terminal-event-{terminal_event}"
target = SimpleNamespace(
data={"run_id": run_id, "approval_id": "approval-target"},
event=threading.Event(),
result=None,
reason=None,
)
survivor = SimpleNamespace(
data={"run_id": "run-other", "approval_id": "approval-other"},
event=threading.Event(),
result=None,
reason=None,
)
class JsonResponse:
def __enter__(self):
return self
def __exit__(self, *_args):
return None
def read(self, _limit=None):
return json.dumps({"run_id": run_id}).encode()
class SseResponse:
def __enter__(self):
return self
def __exit__(self, *_args):
return None
def __iter__(self):
payload = {"event": terminal_event}
if terminal_event == "run.failed":
payload["error"] = "terminal failure"
return iter([f"data: {json.dumps(payload)}".encode(), b""])
def fake_urlopen(req, *, timeout=None):
del timeout
return JsonResponse() if req.full_url.endswith("/v1/runs") else SseResponse()
try:
with approvals._lock:
approvals._pending.pop(sid, None)
approvals._gateway_queues[sid] = [target, survivor]
with patch("urllib.request.urlopen", side_effect=fake_urlopen):
if terminal_event == "run.failed":
with pytest.raises(RuntimeError, match="terminal failure"):
gateway_chat._run_gateway_runs_api_streaming(
sid, "hi", "test", "/tmp", stream_id, "http://gw:8642", "", [], {},
put_gateway_event=lambda *_args: None,
cancel_event=threading.Event(),
session=SimpleNamespace(context_messages=[]),
)
else:
gateway_chat._run_gateway_runs_api_streaming(
sid, "hi", "test", "/tmp", stream_id, "http://gw:8642", "", [], {},
put_gateway_event=lambda *_args: None,
cancel_event=threading.Event(),
session=SimpleNamespace(context_messages=[]),
)
assert target.result == "deny"
assert target.reason == terminal_reason
assert target.event.is_set()
assert survivor.result is None
assert not survivor.event.is_set()
with approvals._lock:
assert approvals._gateway_queues[sid] == [survivor]
finally:
gateway_chat._STREAM_RUN_IDS.pop(stream_id, None)
with approvals._lock:
approvals._pending.pop(sid, None)
approvals._gateway_queues.pop(sid, None)
def test_gateway_stream_teardown_fail_closes_parked_run_producer():
"""A disconnect before a terminal event still settles the mapped run."""
from api import gateway_chat
from api import route_approvals as approvals
from api.config import STREAMS, STREAMS_LOCK
sid = "sess-terminal-teardown"
stream_id = "stream-terminal-teardown"
run_id = "run-terminal-teardown"
target = SimpleNamespace(
data={"run_id": run_id, "approval_id": "approval-target"},
event=threading.Event(),
result=None,
reason=None,
)
survivor = SimpleNamespace(
data={"run_id": "run-other", "approval_id": "approval-other"},
event=threading.Event(),
result=None,
reason=None,
)
events = []
queue = MagicMock()
queue.put_nowait = lambda item: events.append(item)
with STREAMS_LOCK:
STREAMS[stream_id] = queue
session = MagicMock()
session.active_stream_id = stream_id
session.workspace = "/tmp"
session.model = "test"
session.model_provider = None
session.profile = None
session.context_messages = []
session.messages = []
session.pending_user_message = None
session.pending_attachments = None
session.pending_started_at = None
def disconnect_after_run_id(*_args, **_kwargs):
gateway_chat._publish_gateway_run_id(stream_id, run_id)
raise RuntimeError("gateway disconnected")
try:
with approvals._lock:
approvals._pending.pop(sid, None)
approvals._gateway_queues[sid] = [target, survivor]
with patch.dict(
"os.environ",
{"HERMES_WEBUI_CHAT_BACKEND": "gateway", "HERMES_WEBUI_GATEWAY_USE_RUNS_API": "1"},
), patch("api.gateway_chat.gateway_supports_approval", return_value=True), patch(
"api.gateway_chat._run_gateway_runs_api_streaming",
side_effect=disconnect_after_run_id,
), patch("api.gateway_chat.get_session", return_value=session):
gateway_chat._run_gateway_chat_streaming(
session_id=sid,
msg_text="hi",
model="test-model",
workspace="/tmp",
stream_id=stream_id,
)
assert target.result == "deny"
assert target.reason == "Gateway run ended during teardown before approval resolution"
assert target.event.is_set()
assert survivor.result is None
assert not survivor.event.is_set()
with approvals._lock:
assert approvals._gateway_queues[sid] == [survivor]
finally:
gateway_chat._STREAM_RUN_IDS.pop(stream_id, None)
with STREAMS_LOCK:
STREAMS.pop(stream_id, None)
with approvals._lock:
approvals._pending.pop(sid, None)
approvals._gateway_queues.pop(sid, None)
# ---------------------------------------------------------------------------
# 3. Approval event translation
# ---------------------------------------------------------------------------
@@ -1268,22 +1440,32 @@ def test_terminal_run_retirement_removes_all_same_run_mirrors():
approvals._pending.pop(sid, None)
def test_terminal_run_retirement_clears_same_run_gateway_queue_state():
def test_terminal_run_settlement_clears_same_run_gateway_queue_state():
import api.route_approvals as approvals
sid = "sess-terminal-run-queue"
approvals._pending.pop(sid, None)
approvals._gateway_queues.pop(sid, None)
try:
approvals._gateway_queues[sid] = [
SimpleNamespace(data={"run_id": "run-a", "approval_id": "appr-a", "command": "a"}),
SimpleNamespace(data={"command": "local-head"}),
]
target = SimpleNamespace(
data={"run_id": "run-a", "approval_id": "appr-a", "command": "a"},
event=threading.Event(),
result=None,
reason=None,
)
local_head = SimpleNamespace(data={"command": "local-head"})
approvals._gateway_queues[sid] = [target, local_head]
approvals.submit_gateway_pending_mirror(
sid,
{"run_id": "run-a", "approval_id": "appr-a", "command": "a"},
)
assert approvals.retire_gateway_pending_mirror(sid, run_id="run-a")
settled, _head, _total = approvals.settle_gateway_pending_run(
sid, "run-a", reason="terminal"
)
assert settled == 1
assert target.result == "deny"
assert target.reason == "terminal"
assert target.event.is_set()
assert approvals.gateway_pending_mirror(sid, run_id="run-a") is None
assert not any(
str((getattr(entry, "data", None) or {}).get("run_id") or "").strip() == "run-a"
@@ -1294,18 +1476,30 @@ def test_terminal_run_retirement_clears_same_run_gateway_queue_state():
approvals._gateway_queues.pop(sid, None)
def test_terminal_run_retirement_clears_non_head_same_run_gateway_queue_state():
def test_terminal_run_settlement_clears_non_head_same_run_gateway_queue_state():
import api.route_approvals as approvals
sid = "sess-terminal-run-non-head-queue"
approvals._pending.pop(sid, None)
approvals._gateway_queues.pop(sid, None)
try:
target = SimpleNamespace(
data={"run_id": "run-a", "approval_id": "appr-a", "command": "a"},
event=threading.Event(),
result=None,
reason=None,
)
approvals._gateway_queues[sid] = [
SimpleNamespace(data={"command": "local-head"}),
SimpleNamespace(data={"run_id": "run-a", "approval_id": "appr-a", "command": "a"}),
target,
]
assert approvals.retire_gateway_pending_mirror(sid, run_id="run-a")
settled, _head, _total = approvals.settle_gateway_pending_run(
sid, "run-a", reason="terminal"
)
assert settled == 1
assert target.result == "deny"
assert target.reason == "terminal"
assert target.event.is_set()
assert approvals.gateway_pending_mirror(sid, run_id="run-a") is None
assert not any(
str((getattr(entry, "data", None) or {}).get("run_id") or "").strip() == "run-a"
@@ -1865,7 +2059,15 @@ def test_chat_cancel_waits_for_worker_published_run_id_before_settlement(
STREAMS[stream_id] = SimpleNamespace(put_nowait=lambda *_args, **_kwargs: None)
register_stream_owner(stream_id, sid)
approvals._pending.pop(sid, None)
approvals.submit_gateway_pending_mirror(sid, {"run_id": "run-stop/1", "command": "first"})
approval_entry = SimpleNamespace(
data={"run_id": "run-stop/1", "approval_id": "approval-stop"},
event=threading.Event(),
result=None,
reason=None,
)
with approvals._lock:
approvals._gateway_queues[sid] = [approval_entry]
approvals.submit_gateway_pending_mirror(sid, approval_entry.data)
session = SimpleNamespace(
profile=None,
workspace="/tmp",
@@ -1945,8 +2147,15 @@ def test_chat_cancel_waits_for_worker_published_run_id_before_settlement(
assert called["stop"] == "run-stop/1"
assert called["cancel"] is expect_cancel
if stop_result:
assert approval_entry.result == "deny"
assert approval_entry.reason == (
"Gateway run was cancelled before approval resolution"
)
assert approval_entry.event.is_set()
assert approvals.gateway_pending_mirror(sid, run_id="run-stop/1") is None
else:
assert approval_entry.result is None
assert not approval_entry.event.is_set()
assert stream_id in STREAMS
assert stream_owner_session_id(stream_id) == sid
assert session.active_stream_id == stream_id
@@ -1957,7 +2166,9 @@ def test_chat_cancel_waits_for_worker_published_run_id_before_settlement(
release_worker.set()
request_thread.join(timeout=5)
worker_thread.join(timeout=5)
with approvals._lock:
approvals._pending.pop(sid, None)
approvals._gateway_queues.pop(sid, None)
STREAMS.pop(stream_id, None)
ACTIVE_RUNS.pop(stream_id, None)
unregister_stream_owner(stream_id)
tests/test_gateway_watcher_idle_park.py
+158
View File
@@ -0,0 +1,158 @@
"""Regression: GatewayWatcher must not burn CPU when nobody is listening.
Maint #3035 / production evidence: with zero inbound traffic the daemon still
re-fingerprinted every sessions row every 5s and woke 10×/s on ``time.sleep(0.1)``
stop-checks, holding ~7% of a core forever.
The poll loop must:
1. park (no fingerprint / no projection) while subscriber count is zero, and
2. wait on ``threading.Event.wait`` for the poll interval — not a spin of
short sleeps — so a quiet process costs essentially nothing.
"""
from __future__ import annotations
import importlib
import sqlite3
import time
from pathlib import Path
def _make_db(tmp_path: Path):
db = tmp_path / "state.db"
conn = sqlite3.connect(str(db))
conn.executescript(
"""
CREATE TABLE sessions (
id TEXT PRIMARY KEY,
source TEXT NOT NULL,
session_source TEXT,
model TEXT,
started_at REAL NOT NULL,
ended_at REAL,
end_reason TEXT,
parent_session_id TEXT,
message_count INTEGER DEFAULT 0,
title TEXT,
archived INTEGER DEFAULT 0
);
CREATE TABLE messages (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id TEXT NOT NULL,
role TEXT NOT NULL,
content TEXT,
timestamp REAL NOT NULL
);
"""
)
conn.execute(
"INSERT INTO sessions (id, source, model, started_at, message_count, title) "
"VALUES ('tg1', 'telegram', 'm', ?, 1, 'Chat')",
(time.time(),),
)
conn.execute(
"INSERT INTO messages (session_id, role, content, timestamp) "
"VALUES ('tg1', 'user', 'x', ?)",
(time.time(),),
)
conn.commit()
return db, conn
def test_poll_loop_uses_blocking_event_wait_not_spin_sleep():
"""Source contract: the idle sleep must be one Event.wait, not 50× sleep(0.1)."""
src = (
Path(__file__).resolve().parents[1] / "api" / "gateway_watcher.py"
).read_text(encoding="utf-8")
start = src.index("def _poll_loop(self)")
end = src.index("\n# ── Module-level watcher registry", start)
body = src[start:end]
# Strip docstrings so a historical mention of the old spin cannot false-fail.
code = body
while '"""' in code:
a = code.index('"""')
b = code.index('"""', a + 3) + 3
code = code[:a] + code[b:]
assert "time.sleep(0.1)" not in code, (
"0.1s stop-check spin burns syscalls forever; use Event.wait(POLL_INTERVAL)"
)
assert "self._stop_event.wait(" in code, (
"poll interval must block on _stop_event.wait so stop() wakes immediately"
)
assert "_idle_wake" in code and "_has_subscribers" in code, (
"zero-subscriber path must park on _idle_wake instead of fingerprinting"
)
def test_poll_loop_parks_without_subscribers(tmp_path, monkeypatch):
"""Zero subscribers → zero fingerprint work; subscribe wakes and polls resume."""
gw = importlib.import_module("api.gateway_watcher")
db, _conn = _make_db(tmp_path)
polls: list[float] = []
real_poll_once = gw.GatewayWatcher._poll_once
def tracing_poll_once(self, *args, **kwargs):
polls.append(time.monotonic())
return real_poll_once(self, *args, **kwargs)
monkeypatch.setattr(gw.GatewayWatcher, "_poll_once", tracing_poll_once)
watcher = gw.GatewayWatcher(state_db_path=db)
# Keep the active interval short so the subscribed phase is fast to assert.
watcher.POLL_INTERVAL = 0.05
watcher.start()
try:
time.sleep(0.25)
assert polls == [], (
"with no SSE subscribers the watcher must not fingerprint/poll state.db; "
f"got {len(polls)} poll(s)"
)
q = watcher.subscribe()
deadline = time.monotonic() + 2.0
while not polls and time.monotonic() < deadline:
time.sleep(0.01)
assert polls, "first subscribe must wake the parked poll loop promptly"
# Let at least one more subscribed cycle land, then detach.
before_unsub = len(polls)
deadline = time.monotonic() + 2.0
while len(polls) <= before_unsub and time.monotonic() < deadline:
time.sleep(0.01)
assert len(polls) > before_unsub, "subscribed watcher must keep polling"
watcher.unsubscribe(q)
# Drain any in-flight poll that started before unsubscribe, then prove
# the loop re-parks (no further growth).
time.sleep(watcher.POLL_INTERVAL + 0.05)
settled = len(polls)
time.sleep(0.3)
assert len(polls) == settled, (
"after the last subscriber leaves, polling must park again; "
f"grew from {settled} to {len(polls)}"
)
finally:
watcher.stop()
assert not watcher.is_alive() or (
watcher._thread is None or not watcher._thread.is_alive()
)
def test_stop_unparks_idle_watcher_promptly(tmp_path):
"""stop() must release a watcher parked with no subscribers within join timeout."""
gw = importlib.import_module("api.gateway_watcher")
db, _conn = _make_db(tmp_path)
watcher = gw.GatewayWatcher(state_db_path=db)
watcher.POLL_INTERVAL = 30 # would hang badly if stop relied on the interval alone
watcher.start()
try:
# Confirm we are parked (thread alive, no work).
assert watcher.is_alive()
time.sleep(0.05)
t0 = time.monotonic()
watcher.stop()
elapsed = time.monotonic() - t0
assert elapsed < 2.0, f"stop() blocked {elapsed:.2f}s on an idle parked watcher"
assert not watcher.is_alive()
finally:
watcher.stop()
tests/test_gateway_watcher_resubscribe_fresh.py
+222
View File
@@ -0,0 +1,222 @@
"""Regression: a re-subscribing SSE client gets a fresh watcher projection.
``_poll_once`` resets the parity timestamp when it runs with no subscribers
("force fresh projection on the next subscription"), but since the idle-park
change (#7694) the poll loop parks *before* calling ``_poll_once`` once the
last subscriber leaves, so that reset is not reached on a normal disconnect.
A tab that reconnects inside the parity interval then resumes with the stale
cheap fingerprint and skips the projection. Final removal invalidates the
cache, and an epoch fences projections already in flight.
"""
from __future__ import annotations
import importlib
import sqlite3
import threading
import time
from pathlib import Path
import pytest
def _make_db(tmp_path: Path) -> Path:
db = tmp_path / "state.db"
conn = sqlite3.connect(str(db))
conn.executescript(
"""
CREATE TABLE sessions (
id TEXT PRIMARY KEY,
source TEXT NOT NULL,
session_source TEXT,
model TEXT,
started_at REAL NOT NULL,
ended_at REAL,
end_reason TEXT,
parent_session_id TEXT,
message_count INTEGER DEFAULT 0,
title TEXT,
archived INTEGER DEFAULT 0
);
CREATE TABLE messages (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id TEXT NOT NULL,
role TEXT NOT NULL,
content TEXT,
timestamp REAL NOT NULL
);
"""
)
conn.execute(
"INSERT INTO sessions (id, source, model, started_at, message_count, title) "
"VALUES ('tg1', 'telegram', 'm', ?, 1, 'Chat')",
(time.time(),),
)
conn.execute(
"INSERT INTO messages (session_id, role, content, timestamp) VALUES ('tg1', 'user', 'x', ?)",
(time.time(),),
)
conn.commit()
conn.close()
return db
def test_resubscribe_after_last_unsubscribe_forces_fresh_projection(tmp_path):
gw = importlib.import_module("api.gateway_watcher")
watcher = gw.GatewayWatcher(state_db_path=_make_db(tmp_path))
q = watcher.subscribe()
assert watcher._poll_once(now=1.0) is True
watcher.unsubscribe(q)
assert watcher._last_cheap_fp == ""
assert watcher._last_full_projection_at is None
watcher.subscribe()
# Well inside PROJECTION_PARITY_INTERVAL and with an unchanged state.db.
assert watcher._poll_once(now=2.0) is True
def test_unsubscribe_keeps_fingerprint_while_other_subscribers_remain(tmp_path):
gw = importlib.import_module("api.gateway_watcher")
watcher = gw.GatewayWatcher(state_db_path=_make_db(tmp_path))
first = watcher.subscribe()
watcher.subscribe()
assert watcher._poll_once(now=1.0) is True
fingerprint = watcher._last_cheap_fp
assert fingerprint
watcher.unsubscribe(first)
assert watcher._last_cheap_fp == fingerprint
assert watcher._last_full_projection_at == 1.0
assert watcher._poll_once(now=2.0) is False
def test_unsubscribe_of_unknown_queue_does_not_reset(tmp_path):
import queue as _queue
gw = importlib.import_module("api.gateway_watcher")
watcher = gw.GatewayWatcher(state_db_path=_make_db(tmp_path))
watcher.subscribe()
assert watcher._poll_once(now=1.0) is True
fingerprint = watcher._last_cheap_fp
watcher.unsubscribe(_queue.Queue())
assert watcher._last_cheap_fp == fingerprint
assert watcher._last_full_projection_at == 1.0
def test_slow_consumer_eviction_invalidates_populated_cache(tmp_path):
gw = importlib.import_module("api.gateway_watcher")
watcher = gw.GatewayWatcher(state_db_path=_make_db(tmp_path))
q = watcher.subscribe()
assert watcher._poll_once(now=1.0) is True
assert watcher._last_cheap_fp
assert watcher._last_full_projection_at == 1.0
for _ in range(q.maxsize - q.qsize()):
q.put_nowait({"type": "existing"})
watcher._notify_subscribers([])
assert not watcher._has_subscribers()
assert watcher._last_cheap_fp == ""
assert watcher._last_full_projection_at is None
watcher.unsubscribe(q) # The late SSE cleanup cannot undo the reset.
watcher.subscribe()
assert watcher._poll_once(now=2.0) is True
@pytest.mark.parametrize("removal", ["unsubscribe", "slow_consumer"])
@pytest.mark.parametrize("reconnect_while_blocked", [False, True])
def test_inflight_projection_cannot_restore_cache_after_last_removal(
tmp_path, monkeypatch, removal, reconnect_while_blocked,
):
"""The DB read must not serialize final removal or commit across its epoch."""
gw = importlib.import_module("api.gateway_watcher")
watcher = gw.GatewayWatcher(state_db_path=_make_db(tmp_path))
q = watcher.subscribe()
entered = threading.Event()
release = threading.Event()
projections = []
errors = []
real_projection = gw._get_agent_sessions_from_db
def blocked_projection(path):
projections.append(path)
if len(projections) == 1:
entered.set()
assert release.wait(5), "test never released the DB read"
return real_projection(path)
def poll():
try:
watcher._poll_once(now=1.0)
except BaseException as exc:
errors.append(exc)
monkeypatch.setattr(gw, "_get_agent_sessions_from_db", blocked_projection)
worker = threading.Thread(target=poll, daemon=True)
worker.start()
new_q = None
try:
assert entered.wait(5), "poll never reached the DB read"
if removal == "unsubscribe":
watcher.unsubscribe(q)
else:
for _ in range(q.maxsize):
q.put_nowait({"type": "existing"})
watcher._notify_subscribers([])
# Handler teardown of an evicted queue must be harmless.
watcher.unsubscribe(q)
assert not watcher._has_subscribers()
assert watcher._last_cheap_fp == ""
assert watcher._last_full_projection_at is None
if reconnect_while_blocked:
new_q = watcher.subscribe()
finally:
release.set()
worker.join(timeout=5)
assert not worker.is_alive(), "poll did not finish"
assert not errors, errors
assert watcher._last_cheap_fp == ""
assert watcher._last_full_projection_at is None
if new_q is not None:
assert new_q.empty(), "old projection must not notify a new subscriber"
else:
watcher.subscribe()
assert watcher._poll_once(now=2.0) is True
assert len(projections) == 2
assert watcher._last_cheap_fp
assert watcher._last_full_projection_at == 2.0
def test_poll_loop_reprojects_for_reconnecting_subscriber(tmp_path, monkeypatch):
"""Through the real loop: disconnect, park, reconnect -> projection runs again."""
gw = importlib.import_module("api.gateway_watcher")
db = _make_db(tmp_path)
projections: list[float] = []
real_projection = gw._get_agent_sessions_from_db
def tracing_projection(path):
projections.append(time.monotonic())
return real_projection(path)
monkeypatch.setattr(gw, "_get_agent_sessions_from_db", tracing_projection)
watcher = gw.GatewayWatcher(state_db_path=db)
watcher.POLL_INTERVAL = 0.05
watcher.start()
try:
q = watcher.subscribe()
deadline = time.monotonic() + 2.0
while not projections and time.monotonic() < deadline:
time.sleep(0.01)
assert len(projections) == 1
watcher.unsubscribe(q)
time.sleep(watcher.POLL_INTERVAL * 4) # loop parks; no further projection
assert len(projections) == 1
watcher.subscribe()
deadline = time.monotonic() + 2.0
while len(projections) < 2 and time.monotonic() < deadline:
time.sleep(0.01)
assert len(projections) == 2, "reconnecting subscriber must get a fresh projection"
finally:
watcher.stop()
tests/test_get_with_body_auth_reject_reuse.py
+158
View File
@@ -0,0 +1,158 @@
"""A body-bearing GET that fails auth must not poison a keep-alive connection.
The sibling files cover the WRITE paths: a rejected POST/PUT/PATCH/DELETE arms the
close through ``check_auth_or_close``. ``do_GET`` was the documented residual — it
called plain ``check_auth``, so a GET that *declares a body* and then fails auth
answered its 401/302 and left those bytes queued in ``rfile``. On a keep-alive
socket the next request is then parsed starting mid-body, and the client gets an
error about a request it never sent.
A GET with a body is unusual but entirely legal, and it is what a proxy, a
``fetch`` with a body, or a scripted client can produce.
Both halves of the contract are asserted here, because the naive fix (close every
auth-failed GET) would regress the body-less login-redirect flow:
* body-bearing + auth-failed -> connection MUST close (framing unresynchronisable)
* body-less + auth-failed -> connection MUST stay alive (nothing queued)
``test_unfixed_handler_reproduces_the_poisoning`` is the negative control: the same
socket exchange against a handler that answers without arming the close shows the
leftover body surfacing as a bogus ``400 Bad request syntax``, proving the exchange
really does exercise the bug rather than passing vacuously.
"""
import socket
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
BODY = b'{"q": "search"}'
def _get_with_body(path=b"/api/sessions"):
return (
b"GET " + path + b" HTTP/1.1\r\n"
b"Host: localhost\r\n"
b"Content-Type: application/json\r\n"
b"Content-Length: " + str(len(BODY)).encode() + b"\r\n"
b"\r\n" + BODY
)
BODYLESS_GET = (
b"GET /api/sessions HTTP/1.1\r\n"
b"Host: localhost\r\n"
b"\r\n"
)
FOLLOWING_GET = b"GET /api/health HTTP/1.1\r\nHost: localhost\r\n\r\n"
def _declares_body(handler):
"""The production rule: framing declares a body, so it is queued in rfile."""
raw = handler.headers.get("Content-Length")
if raw is not None:
stripped = raw.strip(" \t")
if not (stripped.isascii() and stripped.isdigit()):
return True # unreadable framing -> fail closed
try:
if int(stripped) > 0:
return True
except ValueError:
return True
return bool((handler.headers.get("Transfer-Encoding") or "").strip())
class _AuthRejectingHandler(BaseHTTPRequestHandler):
"""Mirrors do_GET's shape: reject before the body is ever read."""
protocol_version = "HTTP/1.1"
arm_close = True
def do_GET(self):
if self.path == "/api/health":
payload = b'{"status": "ok"}'
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(payload)))
self.end_headers()
self.wfile.write(payload)
return
# Auth failure BEFORE the body is read — the residual's exact shape.
if self.arm_close and _declares_body(self):
self.close_connection = True
payload = b'{"error": "unauthorized"}'
self.send_response(401)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(payload)))
if self.arm_close and _declares_body(self):
self.send_header("Connection", "close")
self.end_headers()
self.wfile.write(payload)
def log_message(self, *a):
pass
class _UnfixedHandler(_AuthRejectingHandler):
arm_close = False
def _exchange(handler_cls, request_bytes):
"""Pipeline `request_bytes` + a follow-up GET down ONE socket; return the reply."""
srv = ThreadingHTTPServer(("127.0.0.1", 0), handler_cls)
threading.Thread(target=srv.serve_forever, daemon=True).start()
try:
with socket.create_connection(srv.server_address, timeout=5) as s:
s.sendall(request_bytes + FOLLOWING_GET)
s.settimeout(5)
chunks = []
while True:
try:
b = s.recv(4096)
except socket.timeout:
break
if not b:
break
chunks.append(b)
return b"".join(chunks)
finally:
srv.shutdown()
srv.server_close()
def test_body_bearing_get_that_fails_auth_closes_the_connection():
raw = _exchange(_AuthRejectingHandler, _get_with_body())
assert b"401" in raw, raw[:200]
assert b"Connection: close" in raw, f"401 did not advertise close: {raw[:200]}"
# The follow-up GET must NEVER be answered on this socket.
assert raw.count(b"HTTP/1.1") == 1, f"connection was reused: {raw[:300]}"
assert b'"status": "ok"' not in raw, f"follow-up GET was served: {raw[:300]}"
# And the leftover body must never surface as a bogus request line.
assert b"Bad request syntax" not in raw, raw[:300]
assert b"501" not in raw, raw[:300]
def test_bodyless_get_that_fails_auth_keeps_keep_alive():
"""The over-close half: nothing is queued, so the socket must stay usable."""
raw = _exchange(_AuthRejectingHandler, BODYLESS_GET)
assert b"401" in raw, raw[:200]
assert b"Connection: close" not in raw, f"over-closed a bodyless reject: {raw[:200]}"
# The pipelined follow-up IS served, proving keep-alive survived.
assert b'"status": "ok"' in raw, f"keep-alive was dropped: {raw[:300]}"
assert raw.count(b"HTTP/1.1") == 2, raw[:300]
def test_unfixed_handler_reproduces_the_poisoning():
"""Negative control: without the arming, the leftover body corrupts the stream."""
raw = _exchange(_UnfixedHandler, _get_with_body())
assert b"401" in raw, raw[:200]
poisoned = (
b"Bad request syntax" in raw
or b"400" in raw
or b"501" in raw
or b'"status": "ok"' not in raw
)
assert poisoned, (
"expected the unread body to poison the next request on this socket; "
f"got {raw[:300]!r}"
)
tests/test_goal_command_webui.py
+8 -1
View File
@@ -671,7 +671,14 @@ def test_frontend_has_goal_slash_command_and_status_event_handler():
assert "goal'" in MESSAGES_JS
assert "source.addEventListener('goal'" in MESSAGES_JS
assert "source.addEventListener('goal_continue'" in MESSAGES_JS
assert "['steer','interrupt','queue','terminal','goal','yolo'].includes(_pc.name)" in MESSAGES_JS
# #6962 added 'stop' to this busy-command passthrough list so a busy /stop
# cancels immediately. Assert /goal's membership rather than the exact list
# literal, so adding another passthrough command does not fail this test.
_passthrough = re.search(
r"if\(_pc&&\[([^\]]*)\]\.includes\(_pc\.name\)\)", MESSAGES_JS
)
assert _passthrough, "busy-command passthrough list not found in messages.js"
assert "'goal'" in _passthrough.group(1)
assert "queueSessionMessage" in MESSAGES_JS
tests/test_hidden_tab_server_initiated_turn.py
+4 -2
View File
@@ -89,7 +89,9 @@ def test_hidden_poll_hits_session_status_and_attaches_as_replay():
"""
start = MESSAGES_JS.find("function _startHiddenActiveStreamPoll(sid)")
assert start != -1
body = MESSAGES_JS[start:start + 2400]
# #7299 added a 404/410 terminal-stop branch to the response handler.
# The window must cover the original fetch chain + the new branch.
body = MESSAGES_JS[start:start + 8000]
assert "api/session/status?session_id=" in body
assert "d.active_stream_id" in body
# attaches as replay (recovered=true) — turn is already mid-flight
@@ -174,7 +176,7 @@ def test_poll_stops_only_when_attach_succeeds():
assert start != -1, "_startHiddenActiveStreamPoll signature not found"
# Window 2400: the multi-pane follow-up adds an explanatory comment block
# before the attach call, pushing it past a narrower slice.
body = MESSAGES_JS[start:start + 3200]
body = MESSAGES_JS[start:start + 8000]
assert "const attached = _attachServerInitiatedStream(sid, streamId, true)" in body
# Stop the poll only on a true attach (the false branch keeps polling within
# the bounded-retry budget rather than stopping).
tests/test_issue1494_state_db_fd_leak.py
+3 -3
View File
@@ -163,9 +163,9 @@ def test_read_importable_agent_session_rows_closes_connection(tmp_path, tracking
read_importable_agent_session_rows(db)
_assert_all_closed(tracking_sqlite, "read_importable_agent_session_rows")
# #5455: missing-index self-heal uses one separate short-lived RW connection
# in addition to the five read-only listing connections.
assert len(tracking_sqlite.instances) == 6
# #5455: the listing is strictly read-only, including when indexes are
# missing — five calls, five read-only connections, no self-heal writer.
assert len(tracking_sqlite.instances) == 5
def test_read_session_lineage_metadata_closes_connection(tmp_path, tracking_sqlite):
tests/test_issue1897_profile_switch_agent_cache.py
+9 -10
View File
@@ -79,20 +79,19 @@ RAISED_EXCEPTION_SELF_HEAL_MARKER = (
"retrying stream after credential refresh')"
)
# Each region ends at the cache write that stores the signature it just
# computed. Bounding the region at the write -- rather than at the next
# lifecycle anchor or at EOF -- is what proves the signature is recomputed
# *before* it is cached: a call moved below its own write lands outside the
# region and fails, instead of passing while the write consumes a stale
# `_agent_sig`.
# Each region ends at the atomic registration/publication call that consumes
# the signature it just computed. The shared helper now writes the cache under
# the Stop lock; the call boundary must still follow fresh signature computation.
# A signature call moved below its publication lands outside the region and
# fails, rather than silently reusing a stale `_agent_sig` on a retry.
INITIAL_SEND_CACHE_WRITE_MARKER = (
"SESSION_AGENT_CACHE[session_id] = (agent, _agent_sig)"
"if not _register_agent_if_current(agent, _agent_sig if _cache_new_agent else None):"
)
RETURNED_ERROR_SELF_HEAL_CACHE_WRITE_MARKER = (
"_SAC[session_id] = (agent, _agent_sig)"
"if not _register_agent_if_current(agent, _agent_sig):"
)
RAISED_EXCEPTION_SELF_HEAL_CACHE_WRITE_MARKER = (
"_SAC2[session_id] = (_heal_agent, _agent_sig)"
"if not _register_agent_if_current(_heal_agent, _agent_sig):"
)
@@ -112,7 +111,7 @@ def _signature_calls_by_region(
The regions are the initial send, the returned-error self-heal retry and
the raised-exception self-heal retry; each runs from its lifecycle anchor
to the cache write that consumes `_agent_sig`. Requiring exactly one call
to the atomic publication call that consumes `_agent_sig`. Requiring one call
inside those bounds means neither a dropped retry call (masked by its
surviving siblings) nor a call recomputed after its own cache write (which
would leave the write storing a stale signature) can read as correct.
tests/test_issue1909_csp_report_only.py
+1 -1
View File
@@ -144,7 +144,7 @@ def test_server_bypasses_auth_for_csp_report(monkeypatch):
called["path"] = parsed.path
return True
monkeypatch.setattr("server.check_auth", fail_auth)
monkeypatch.setattr("server.check_auth_or_close", fail_auth)
monkeypatch.setattr("server.clear_request_profile", lambda: None)
monkeypatch.setattr("server.get_profile_cookie", lambda _handler: None)
tests/test_issue2051_duplicate_assistant_answer.py
+370
View File
@@ -0,0 +1,370 @@
"""Regression: the assistant answer is displayed TWICE (#6948 follow-up;
upstream symptom report #2051).
Two independent renderers can put a second copy of the same answer on screen
after a turn settles. Both are covered here.
1. The live-turn node (static/ui.js)
------------------------------------
`renderMessages()` re-attaches the preserved `#liveAssistantTurn` node after the
transcript wipe, and `restoreLiveTurnHtmlForSession()` re-inserts the
`INFLIGHT[sid].liveTurnHtml` snapshot on a session switch / reconnect. Each has
ONE branch that *appends* — i.e. adds a turn the settled rebuild did not
produce. The assistant row is persisted a few ms BEFORE the stream's terminal
event clears `S.activeStreamId`, so a dead live node left behind by an INFLIGHT
entry that outlived its stream is appended under the settled copy: the same
answer twice, self-sustaining across later renders (the node keeps
id=liveAssistantTurn and is re-preserved by the #6948 guard), healed only by a
reload.
The contract (`_settledTranscriptOwnsLiveTurn`) is an OWNERSHIP proof, never a
rendered-text comparison:
* the live body comes from the streaming smd parser and the settled body from
renderMd + post-processing, so comparing textContent would only ever match
single-paragraph plain text — lists, fenced code, tables and `_emphasis_`
render differently in the two pipelines;
* comparing text would DELETE a genuinely live turn whenever the previous
answer reads the same ("Done.", a greeting, a same-prompt resend), because
while the current turn is unpersisted the last settled assistant message is
the PREVIOUS turn's answer.
So the append is skipped only when the transcript ENDS with a settled assistant
message that the SAME stream produced (persisted anchor identity, or a
source-vs-source text match when the turn has no persisted scene), and the
preserved node carries nothing the settled rebuild could not have produced (no
unpersisted tool card / reasoning row, no second live segment — #3714).
2. The settled activity scene (static/ui.js)
--------------------------------------------
`_anchorSceneRowsForRendering(scene,{settled:true})` feeds both settled
renderers. A `process_prose` row whose text IS the final answer is suppressed
per-row by the transparent-stream renderer but NOT by the compact-worklog row
builder, which rebuilds it as a second `.assistant-segment` above the settled
answer. The row is now dropped in the shared filter, using a near-equality test
without the absolute `shorter>=80` floor that made the existing matcher a no-op
for short answers (the reported case: the row is the answer minus its last
streamed token, 35 vs 36 chars). LIVE rendering is untouched — while streaming,
the inline live segment is hidden and the prose row IS the visible answer.
"""
import shutil
import subprocess
from pathlib import Path
import pytest
from tests.js_source_extract import extract_function
ROOT = Path(__file__).resolve().parents[1]
UI_JS = (ROOT / "static" / "ui.js").read_text(encoding="utf-8")
NODE = shutil.which("node")
def js(*names):
"""Real source of the named ui.js functions — never a re-implementation."""
return "\n".join(extract_function(UI_JS, name) for name in names)
def run_node(script):
out = subprocess.run([NODE, "-e", script], capture_output=True, text=True)
assert out.returncode == 0, f"node harness failed:\n{out.stderr}\n{out.stdout}"
assert "OK" in out.stdout, out.stdout
return out.stdout
# Minimal DOM stand-in: class/attribute selector matching over a node tree.
DOM_STUB = """
const assert=require('assert');
class El{
constructor(spec){
spec=spec||{};
this.classes=new Set(spec.classes||[]);
this.attrs=Object.assign({},spec.attrs||{});
this.children=spec.children||[];
}
_matchesToken(tok){
tok=String(tok).trim();
if(tok.startsWith('.')) return this.classes.has(tok.slice(1));
const m=/^\\[([a-zA-Z0-9-]+)="([^"]*)"\\]$/.exec(tok);
if(m) return String(this.attrs[m[1]]===undefined?'':this.attrs[m[1]])===m[2];
return false;
}
querySelectorAll(sel){
const toks=String(sel).split(',');
const out=[];
const walk=(node)=>{
for(const child of node.children){
if(toks.some(t=>child._matchesToken(t))) out.push(child);
walk(child);
}
};
walk(this);
return out;
}
}
const seg=(text)=>new El({attrs:{'data-live-assistant':'1'},classes:['assistant-segment'],
children:[new El({classes:['msg-body'],children:[]})]});
const turnWith=(...kids)=>new El({classes:['assistant-turn'],children:kids});
const toolCard=()=>new El({classes:['tool-card-row']});
const emptyWorklogShell=()=>new El({classes:['live-worklog'],attrs:{'data-live-worklog-shell':'1'}});
var S={messages:[],activeStreamId:null,session:{session_id:'s1'}};
var INFLIGHT={};
function setState(messages,inflight,activeStreamId){
S.messages=messages;
S.activeStreamId=activeStreamId===undefined?null:activeStreamId;
for(const k of Object.keys(INFLIGHT)) delete INFLIGHT[k];
if(inflight) INFLIGHT['s1']=inflight;
}
const user=(content)=>({role:'user',content});
const settledAnswer=(content,streamId)=>{
const m={role:'assistant',content};
if(streamId) m._anchor_stream_id=streamId;
return m;
};
const sceneIdentityAnswer=(content,streamId)=>({
role:'assistant',content,
_anchor_activity_scene:{identity:{stream_id:streamId},final_answer:content},
});
"""
@pytest.mark.skipif(NODE is None, reason="node not on PATH")
def test_dead_live_turn_is_dropped_only_when_the_stream_is_provably_settled():
"""The #2051 duplicate is suppressed; a live turn is never deleted."""
script = DOM_STUB + js(
"msgContent",
"_settledAssistantStreamId",
"_messageHasLiveAssistantProjection",
"_liveTurnCarriesUnsettledContent",
"_settledTranscriptOwnsLiveTurn",
) + """
const ANSWER='Hey! What can I help you with today?';
// ── the bug: stream settled, INFLIGHT not yet cleaned → dead leftover ──
setState([user('Hi'),settledAnswer(ANSWER,'st1')],{streamId:'st1'},'st1');
assert.strictEqual(
_settledTranscriptOwnsLiveTurn('s1',turnWith(seg(ANSWER),emptyWorklogShell())),true,
'dead live node whose stream the settled transcript owns must be dropped (#2051)');
// Identity may live only on the persisted scene.
setState([user('Hi'),sceneIdentityAnswer(ANSWER,'st1')],{streamId:'st1'},null);
assert.strictEqual(_settledTranscriptOwnsLiveTurn('s1',turnWith(seg(ANSWER))),true,
'scene identity.stream_id is accepted as ownership proof');
// Shape independence: the decision must not depend on rendered text at all, so
// a multi-block answer (fenced code / list / table), which the smd parser and
// renderMd render to DIFFERENT textContent, is handled like plain prose.
setState([user('Hi'),settledAnswer('Here you go:\\n\\n```python\\nprint(1)\\n```','st1')],
{streamId:'st1'},'st1');
assert.strictEqual(
_settledTranscriptOwnsLiveTurn('s1',turnWith(seg('Here you go:print(1)'))),true,
'a code-block answer must be de-duplicated too (renderer-independent)');
// ── false positives that a text comparison would cause ──
// The current turn is unpersisted: the transcript ends with the USER turn while
// the live turn streams text identical to the PREVIOUS answer.
setState([user('Hi'),settledAnswer('Done.','st1'),user('Hi')],{streamId:'st2'},'st2');
assert.strictEqual(_settledTranscriptOwnsLiveTurn('s1',turnWith(seg('Done.'))),false,
'a genuinely live turn must never be dropped because an earlier answer matches');
// markInflight() shape at send time (no snapshot yet, stream just started).
setState([user('Hi'),settledAnswer('Done.','st1'),user('Hi')],{sid:'s1',streamId:'st2',ts:1},'st2');
assert.strictEqual(_settledTranscriptOwnsLiveTurn('s1',turnWith(seg('Do'))),false,
'partial live text under a fresh stream must never be dropped');
// A settled tail from a DIFFERENT stream never owns this live turn.
setState([user('Hi'),settledAnswer('Done.','st1')],{streamId:'st2'},'st2');
assert.strictEqual(_settledTranscriptOwnsLiveTurn('s1',turnWith(seg('Done.'))),false,
'a tail from another stream must not authorize dropping the live turn');
// Reconnect/terminal projection: S.messages still carries a live marker.
setState([user('Hi'),Object.assign(settledAnswer('Done.','st1'),{_live:true})],
{streamId:'st1'},null);
assert.strictEqual(_settledTranscriptOwnsLiveTurn('s1',turnWith(seg('Done.'))),false,
'a live projection in S.messages means the turn has not settled');
// Ownership unknown (no INFLIGHT stream id, no active stream) → fail closed.
setState([user('Hi'),settledAnswer('Done.','st1')],{},null);
assert.strictEqual(_settledTranscriptOwnsLiveTurn('s1',turnWith(seg('Done.'))),false,
'unknown live owner must fail closed');
setState([],{streamId:'st1'},'st1');
assert.strictEqual(_settledTranscriptOwnsLiveTurn('s1',turnWith(seg('Done.'))),false,
'an empty transcript owns nothing');
// ── #3714: live-only structure is never discarded ──
setState([user('Hi'),settledAnswer(ANSWER,'st1')],{streamId:'st1'},'st1');
assert.strictEqual(
_settledTranscriptOwnsLiveTurn('s1',turnWith(toolCard(),seg(ANSWER))),false,
'a preserved turn carrying an unpersisted tool card must be kept');
assert.strictEqual(
_settledTranscriptOwnsLiveTurn('s1',turnWith(seg('first'),seg(ANSWER))),false,
'a multi-segment live turn must not be dropped on a tail-segment match');
assert.strictEqual(
_settledTranscriptOwnsLiveTurn('s1',turnWith(new El({classes:['wl-reason']}),seg(ANSWER))),false,
'a preserved turn carrying an unpersisted reasoning row must be kept');
assert.strictEqual(_settledTranscriptOwnsLiveTurn('s1',{}),false,
'an unknown node shape must fail closed');
// ── no persisted scene identity: SOURCE text vs SOURCE text ──
setState([user('Hi'),settledAnswer(ANSWER,null)],{streamId:'st1',lastAssistantText:ANSWER},'st1');
assert.strictEqual(_settledTranscriptOwnsLiveTurn('s1',turnWith(seg(ANSWER))),true,
'streamed markdown identical to the persisted answer proves ownership');
setState([user('Hi'),settledAnswer(ANSWER,null)],
{streamId:'st1',lastAssistantText:'Hey! What can I help'},'st1');
assert.strictEqual(_settledTranscriptOwnsLiveTurn('s1',turnWith(seg('Hey! What can I help'))),false,
'a still-growing stream must not be dropped against a longer settled answer');
setState([user('Hi'),settledAnswer(ANSWER,null)],{streamId:'st1'},'st1');
assert.strictEqual(_settledTranscriptOwnsLiveTurn('s1',turnWith(seg(ANSWER))),false,
'no identity and no streamed source text → fail closed');
console.log('OK');
"""
run_node(script)
@pytest.mark.skipif(NODE is None, reason="node not on PATH")
def test_append_decision_renders_the_answer_exactly_once():
"""Observable outcome of the two append branches, driven by the real
predicate: one copy for a settled leftover, and the live turn is still
attached in every case where the turn may still be running."""
script = DOM_STUB + js(
"msgContent",
"_settledAssistantStreamId",
"_messageHasLiveAssistantProjection",
"_liveTurnCarriesUnsettledContent",
"_settledTranscriptOwnsLiveTurn",
) + """
// Mirrors the two call sites: the branch appends unless the settled transcript
// provably owns the node (static/ui.js renderMessages re-attach and
// restoreLiveTurnHtmlForSession).
function visibleAnswers(messages,inflight,activeStreamId,turn){
setState(messages,inflight,activeStreamId);
const settledCopies=messages.filter(m=>m.role==='assistant'&&!m._live).length;
const appended=!_settledTranscriptOwnsLiveTurn('s1',turn)?1:0;
return {settledCopies,appended,total:settledCopies+appended};
}
const ANSWER='Hey! What can I help you with today?';
// #2051: exactly one copy of the answer.
assert.deepStrictEqual(
visibleAnswers([user('Hi'),settledAnswer(ANSWER,'st1')],{streamId:'st1'},'st1',
turnWith(seg(ANSWER),emptyWorklogShell())),
{settledCopies:1,appended:0,total:1},
'the settled answer must render exactly once');
// A live turn is still attached while the turn is running (#3877).
assert.deepStrictEqual(
visibleAnswers([user('Hi'),settledAnswer(ANSWER,'st1'),user('Hi again')],
{streamId:'st2'},'st2',turnWith(seg(ANSWER))),
{settledCopies:1,appended:1,total:2},
'a running turn must stay on screen even when it repeats an earlier answer');
// A leftover carrying unpersisted structure is kept whole (#3714).
assert.deepStrictEqual(
visibleAnswers([user('Hi'),settledAnswer(ANSWER,'st1')],{streamId:'st1'},'st1',
turnWith(toolCard(),seg(ANSWER))).appended,1,
'never discard live-only blocks the rebuild did not produce');
console.log('OK');
"""
run_node(script)
@pytest.mark.skipif(NODE is None, reason="node not on PATH")
def test_settled_scene_drops_the_prose_row_that_is_the_final_answer():
script = DOM_STUB + js(
"_anchorSceneIsSettledSuccessfulCompression",
"_anchorSceneProseMatchesFinalAnswer",
"_anchorSceneProseDuplicatesFinalAnswer",
"_anchorSceneRowsForRendering",
) + """
function _anchorSceneToolRowLogicalKey(row){ return (row&&row.row_id)||''; }
function _anchorSceneMergeToolRows(_prev,row){ return row; }
const FINAL='Hey! What can I help you with today?';
const scene={
final_answer:FINAL,
activity_rows:[
{role:'thinking',row_id:'t1',text:'thinking out loud'},
{role:'prose',row_id:'p1',text:'Let me check the config first.'},
// the reported shape: the answer minus its last streamed token (35 vs 36)
{role:'prose',row_id:'p2',text:'Hey! What can I help you with today'},
{role:'terminal',row_id:'d1',source_event_type:'done'},
],
};
const texts=(opts)=>_anchorSceneRowsForRendering(scene,opts).map(r=>r.text);
assert.deepStrictEqual(texts({settled:true}),['thinking out loud','Let me check the config first.'],
'the settled scene must not rebuild the final answer as a second segment (#2051)');
assert.deepStrictEqual(texts({settled:false}),
['thinking out loud','Let me check the config first.','Hey! What can I help you with today'],
'LIVE rendering must keep the prose row — it IS the visible answer while streaming');
// Exact repeats and multi-block answers are dropped too; source text is compared,
// so the markdown pipeline never matters here.
const code={final_answer:'Here you go:\\n\\n```python\\nprint(1)\\n```',activity_rows:[
{role:'prose',row_id:'p1',text:'Here you go:\\n\\n```python\\nprint(1)\\n```'},
{role:'prose',row_id:'p2',text:'Working on it.'}]};
assert.deepStrictEqual(_anchorSceneRowsForRendering(code,{settled:true}).map(r=>r.text),
['Working on it.'],'an exact fenced-code duplicate must be dropped, intermediates kept');
// Codex #4568: a SHORT distinct intermediate that merely prefixes a LONG answer
// must survive — the >=0.9 length ratio, not the removed absolute floor, is what
// protects it.
const long_={final_answer:'Hey! What can I help you with today? I can read files, run commands and edit code.',
activity_rows:[{role:'prose',row_id:'p1',text:'Hey! What can'}]};
assert.deepStrictEqual(_anchorSceneRowsForRendering(long_,{settled:true}).map(r=>r.text),
['Hey! What can'],'a distinct short intermediate must not be swallowed by a long answer');
// The pinned per-row matcher is untouched: its absolute floor still rejects the
// short near-miss that the new helper accepts.
assert.strictEqual(_anchorSceneProseMatchesFinalAnswer('Hey! What can I help you with today',FINAL),false,
'_anchorSceneProseMatchesFinalAnswer must keep its >=80 char floor');
assert.strictEqual(_anchorSceneProseDuplicatesFinalAnswer('Hey! What can I help you with today',FINAL),true,
'the new helper must catch the short near-miss');
assert.strictEqual(_anchorSceneProseDuplicatesFinalAnswer('Hey! What can',FINAL),false,
'the new helper must keep the >=0.9 ratio guard');
console.log('OK');
"""
run_node(script)
def test_both_append_branches_are_gated_by_the_ownership_check():
"""Wiring: the predicate only helps if it guards the two branches that ADD a
turn (the replace branches cannot duplicate and must stay unguarded)."""
restore = UI_JS[UI_JS.index("function restoreLiveTurnHtmlForSession"):]
restore = restore[:restore.index("\nfunction markInflight")]
assert "if(existing) existing.replaceWith(restored);" in restore
assert "_settledTranscriptOwnsLiveTurn(sid, restored)" in restore, (
"the restore append branch must be gated by the ownership check"
)
assert "inflight.liveTurnHtml=null;" in restore, (
"a refused snapshot must be released, or it is re-refused on every restore"
)
reattach = UI_JS[UI_JS.index(" const _rebuilt=document.getElementById('liveAssistantTurn');"):]
reattach = reattach[:reattach.index(" // Only force-scroll when not actively streaming")]
assert "_settledTranscriptOwnsLiveTurn(sid,_preservedLiveTurn)" in reattach, (
"the renderMessages append branch must be gated by the ownership check"
)
assert reattach.count("_settledTranscriptOwnsLiveTurn(") == 1, (
"the segment-swap and whole-turn replace branches must stay unguarded — "
"they replace a node instead of adding one and cannot duplicate"
)
def test_segment_block_insertion_moves_nodes_instead_of_reparsing_html():
"""Wiring: every message-block insertion in renderMessages() goes through the
helper, and the helper MOVES parsed nodes instead of re-parsing a string.
Appending a DocumentFragment moves its children and empties it, so a DOM-API
wrapper that executes the call twice appends nothing the second time; an
insertAdjacentHTML wrapper that does the same parses and appends a second copy.
That difference is the whole fix, so it is pinned here rather than left to the
three call sites.
"""
helper = UI_JS[UI_JS.index("function _insertSegmentBlock("):]
helper = helper[:helper.index("\nfunction renderMessages")]
assert "createElement('template')" in helper, (
"the block must be parsed into a <template>, not inserted as a string"
)
assert "seg.appendChild(tpl.content)" in helper, (
"the template's content must be MOVED into the segment — that is what makes "
"a repeated call a no-op"
)
assert helper.count("insertAdjacentHTML") == 1, (
"insertAdjacentHTML may survive only as the fallback for an environment "
"without <template>"
)
render = UI_JS[UI_JS.index("function renderMessages(options){"):]
assert render.count("_insertSegmentBlock(") == 3, (
"all three assistant-segment block insertions must go through the helper"
)
assert "seg.insertAdjacentHTML('beforeend', `${filesHtml}" not in render, (
"an assistant segment's block must not be built from an HTML string again"
)
tests/test_issue2051_duplicate_interrupt_notice.py
+234
View File
@@ -0,0 +1,234 @@
"""Regression: an interrupted turn's notice — and any turn-final answer — is
displayed twice (#6948 / #2051).
A ``hermes chat`` run killed mid tool-call persists a single assistant message
``Operation interrupted.``; the WebUI renders that notice twice. The same
doubling was reported for a long tool run's final answer and for an image reply
carrying a ``MEDIA:`` line. The stored data is clean in every case — one message
in ``state.db``, one in the session sidecar, one in ``/api/session`` — so both
copies are produced client-side.
``renderMessages()`` folds intermediate assistant segments into the collapsed
Worklog card (#3401): the inline segment is classed
``assistant-segment-worklog-source`` (``display:none``) and its prose is
re-rendered as a ``.wl-reason`` row inside the Worklog. Two copies, one hidden —
correct, as long as the fold decision is correct.
``_assistantMessageBelongsInWorklog()`` folded on ``m._live`` before it could
reach the rule that protects the answer (``hasVisibleText && isTurnFinalAssistant
-> false``). Its caller only consults the predicate once the turn has settled (it
gates on ``!S.busy``), so an ``_live`` marker seen there is a leftover of the
live-snapshot projection rather than an ongoing stream. Folding on it hides the
answer inline and echoes it into the Worklog: the turn then holds two copies and
no visible content, which the #3875 blank-turn fail-safe reveals — by expanding
the Worklog, or by un-hiding the source segments — so the user sees the notice or
answer twice.
Three contracts are pinned here:
1. ``_assistantMessageBelongsInWorklog``: a live marker may no longer outrank the
turn-final visible answer. Every other ``_live`` message folds as before, and
the change can only keep a segment visible, never hide or drop one.
2. ``renderMessages``: the Worklog may echo an anchor's prose as a ``.wl-reason``
row only when that anchor was folded into it —
``assistant-segment-worklog-source`` is that proof, and its ``display:none``
is the only reason the echo is not a second visible copy. This also covers
anchors that escape the fold for other reasons, such as ``_error`` messages,
which are never folded.
3. The #3875 fail-safe must materialize a deferred settled Worklog (#5839) before
judging it empty; otherwise it falls through to the last-resort un-hide,
strips the fold off every source segment, and the deferred rows then render
the same prose beside them.
Each contract is a structural/ownership check rather than a rendered-text
comparison, and each fails towards keeping content: the first only un-hides, the
second only suppresses an echo whose original is provably visible, the third only
builds rows earlier.
"""
import shutil
import subprocess
from pathlib import Path
import pytest
from tests.js_source_extract import extract_function
ROOT = Path(__file__).resolve().parents[1]
UI_JS = (ROOT / "static" / "ui.js").read_text(encoding="utf-8")
NODE = shutil.which("node")
def js(*names):
"""Real source of the named ui.js functions — never a re-implementation."""
return "\n".join(extract_function(UI_JS, name) for name in names)
def _function_body(src: str, name: str) -> str:
marker = f"function {name}("
start = src.find(marker)
assert start != -1, f"{name} not found"
brace = src.find("{", start)
assert brace != -1, f"{name} body not found"
depth = 0
for idx in range(brace, len(src)):
ch = src[idx]
if ch == "{":
depth += 1
elif ch == "}":
depth -= 1
if depth == 0:
return src[brace + 1 : idx]
raise AssertionError(f"{name} body not closed")
def run_node(script):
out = subprocess.run([NODE, "-e", script], capture_output=True, text=True)
assert out.returncode == 0, f"node harness failed:\n{out.stderr}\n{out.stdout}"
assert "OK" in out.stdout, out.stdout
return out.stdout
HARNESS = """
const assert=require('assert');
var window={};
function _messageHasReasoningPayload(m){ return !!(m&&(m.reasoning||m.reasoning_content)); }
const assistant=(content,extra)=>Object.assign({role:'assistant',content},extra||{});
"""
def test_live_marker_guard_precedes_the_live_fold_rule():
"""Contract 1, structurally: the guard sits before the live fold rule.
Written as an added guard rather than an edit to `if(m._live) return true;`
so the live rule keeps the exact shape and ordering pinned by
tests/test_live_to_final_anchor_visible_order.py.
"""
belongs = _function_body(UI_JS, "_assistantMessageBelongsInWorklog")
guard = "if(m._live&&hasVisibleText&&isTurnFinalAssistant) return false;"
assert guard in belongs, (
"a settled-render live remnant must not fold the turn's final visible answer"
)
assert belongs.index(guard) < belongs.index("if(m._live) return true;"), (
"the guard must be evaluated before the unconditional live fold"
)
@pytest.mark.skipif(NODE is None, reason="node not on PATH")
def test_stale_live_marker_never_folds_the_turn_final_answer():
"""The fold must not swallow the answer on a settled-render live remnant."""
script = HARNESS + js(
"msgContent",
"_isAssistantEmptyPlaceholderContent",
"_assistantMessageBelongsInWorklog",
) + """
const FINAL={isTurnFinalAssistant:true};
const MID={isTurnFinalAssistant:false};
const owners=new Set([0]);
// The bug: the settled turn-final answer folds on a stale live marker.
assert.strictEqual(
_assistantMessageBelongsInWorklog(assistant('Operation interrupted.',{_live:true}),0,owners,undefined,FINAL),
false,
'a settled turn-final notice must never be folded into the Worklog (#2051)');
assert.strictEqual(
_assistantMessageBelongsInWorklog(
assistant('Here is the image you asked for.\\n\\nMEDIA:/tmp/example.png',{_live:true}),0,owners,undefined,FINAL),
false,
'a MEDIA-bearing turn-final answer must never be folded either');
// Renderer-independent: the decision must not depend on what the text looks like.
assert.strictEqual(
_assistantMessageBelongsInWorklog(
assistant('Here you go:\\n\\n```python\\nprint(1)\\n```',{_live:true}),0,owners,undefined,FINAL),
false,
'a fenced-code turn-final answer must never be folded');
// Everything else about the fold is unchanged.
// A live intermediate segment still folds — that is the Worklog's content.
assert.strictEqual(
_assistantMessageBelongsInWorklog(assistant('Reading the files now.',{_live:true}),0,owners,undefined,MID),
true,
'a live intermediate segment must still fold into the Worklog');
// A live turn-final message with no visible text still folds (an empty tool-call
// anchor is Worklog material, not an answer).
assert.strictEqual(
_assistantMessageBelongsInWorklog(assistant('',{_live:true}),0,owners,undefined,FINAL),
true,
'an empty live anchor must still fold');
// `(empty)` placeholder + reasoning payload is not visible text either.
assert.strictEqual(
_assistantMessageBelongsInWorklog(
assistant('(empty)',{_live:true,reasoning:'thought'}),0,owners,undefined,FINAL),
true,
'an (empty) placeholder anchor must still fold');
// Settled (no marker at all) is untouched in both directions.
assert.strictEqual(
_assistantMessageBelongsInWorklog(assistant('Operation interrupted.'),0,owners,undefined,FINAL),
false, 'settled turn-final answer: unchanged');
// (A settled message with visible text and no live/burst marker never folds —
// `if(hasVisibleText) return false;` precedes the tool-metadata rule. Pinned so a
// future change to the live-marker line cannot quietly alter this ordering.)
assert.strictEqual(
_assistantMessageBelongsInWorklog(assistant('Reading the files now.'),0,owners,undefined,MID),
false, 'settled visible intermediate without a live/burst marker: unchanged');
assert.strictEqual(
_assistantMessageBelongsInWorklog(assistant('',{}),0,owners,undefined,MID),
true, 'settled empty tool-call anchor still folds: unchanged');
assert.strictEqual(
_assistantMessageBelongsInWorklog(assistant('Done.'),0,new Set(),undefined,MID),
false, 'settled intermediate with no tool metadata: unchanged');
// An error message is still never folded.
assert.strictEqual(
_assistantMessageBelongsInWorklog(assistant('**Error:** boom',{_error:true,_live:true}),0,owners,undefined,MID),
false, 'an error message is never folded');
// Live burst/segment markers on a non-final message still fold.
assert.strictEqual(
_assistantMessageBelongsInWorklog(assistant('mid',{_activityBurstId:2}),0,new Set(),undefined,MID),
true, 'activity-burst anchors still fold');
console.log('OK');
"""
run_node(script)
def test_worklog_reason_echo_requires_a_folded_anchor():
"""Contract 2: only a folded anchor may be echoed as a .wl-reason row."""
body = _function_body(UI_JS, "renderMessages")
assert "includeAnchorReason:!!includeAnchorReason&&!!anchorReasonHtml&&!!anchorIsWorklogSource," in body, (
"the settled Worklog step may only re-render an anchor's prose when that "
"anchor was folded into it (assistant-segment-worklog-source); without "
"that proof the row is a second visible copy of a visible segment"
)
# The ownership fact must be computed for every entry, not only for the first
# entry of a turn (where the group is constructed) — otherwise the append path
# would read a stale/undefined value for later anchors in the same turn.
hoisted = body.find(
"const anchorIsWorklogSource=anchorRow.classList"
"&&anchorRow.classList.contains('assistant-segment-worklog-source');"
)
state_lookup = body.find("let state=activityByTurn.get(anchorTurn);")
assert hoisted != -1 and state_lookup != -1, "settled worklog append path not found"
assert hoisted < state_lookup, (
"anchorIsWorklogSource must be computed per activity entry (before the "
"per-turn `state` lookup), not only inside the group-construction branch"
)
def test_blank_turn_failsafe_materializes_deferred_rows_before_judging_empty():
"""Contract 3: a deferred (#5839) Worklog is not empty."""
body = _function_body(UI_JS, "renderMessages")
failsafe = body[body.find("Fail-safe invariant (#3875)"):]
assert failsafe, "the #3875 fail-safe is missing"
materialize = failsafe.find("data-worklog-rows-deferred")
empty_check = failsafe.find("continue; // empty group can't help")
assert materialize != -1, (
"the fail-safe must materialize a deferred settled Worklog before judging "
"it empty — otherwise it un-hides every folded segment and the deferred "
"rows later render the same prose beside them"
)
assert materialize < empty_check, (
"deferred rows must be materialized before the empty-group check"
)
# The last-resort un-hide must still exist: content is never lost.
assert "seg.classList.remove('assistant-segment-worklog-source');" in failsafe, (
"the last-resort un-hide must remain — a duplicate is preferable to a "
"blank turn"
)
tests/test_issue2051_duplicate_model_picker_entry.py
+156
View File
@@ -0,0 +1,156 @@
"""Regression: the composer model picker lists one configured model twice (#2051).
The picker showed two rows for a single configured model, ``acme/example-model``
and ``custom/acme/example-model``, while ``GET /api/models`` returned a ``Custom``
group holding exactly one model and ``#modelSelect`` ended up with exactly one
matching ``<option>``. Both rows are produced client-side.
``configured_model_badges`` carries every routing spelling of a configured model.
For one custom-provider model that is three keys::
acme/example-model (bare/canonical)
custom/acme/example-model (provider-qualified)
@custom:acme/example-model (routable)
``renderModelDropdown()`` walks that map and pushes a synthetic ``_modelData``
row for every key that ``_isEquivalentConfiguredModelEntry()`` does not recognise
as an alias of a row the catalog already produced. That predicate recognised two
of the three spellings; it missed ``<provider>/<model>`` because
``_normalizeConfiguredModelKey()`` strips only one leading slash segment —
deliberately, since #3360 requires ``vendor_a/x`` and ``vendor_b/y/x`` to stay
distinct. So::
acme/example-model -> example.model
custom/acme/example-model -> acme/example.model (different key)
No match, so the alias was pushed as a second row: one model, two picker entries.
``_deduplicateModelPickerOptions()`` cannot catch this — it dedupes ``<option>``
elements inside ``<select>``, and this row never becomes an ``<option>``.
The contract pinned here: ``<provider>/<model>`` is recognised as an alias of
``<model>`` under the same proof the ``@<provider>:<model>`` rule already uses.
The badge declares a provider, the key begins with that provider's prefix, and an
existing picker row from that same provider normalises equal to the remainder.
Two genuinely different models never satisfy the last clause, so the change can
only drop a duplicate — never hide a distinct model.
"""
import shutil
import subprocess
from pathlib import Path
import pytest
from tests.js_source_extract import extract_function
ROOT = Path(__file__).resolve().parents[1]
UI_JS = (ROOT / "static" / "ui.js").read_text(encoding="utf-8")
NODE = shutil.which("node")
def js(*names):
"""Real source of the named ui.js functions — never a re-implementation."""
return "\n".join(extract_function(UI_JS, name) for name in names)
def run_node(script):
out = subprocess.run([NODE, "-e", script], capture_output=True, text=True)
assert out.returncode == 0, f"node harness failed:\n{out.stderr}\n{out.stdout}"
assert "OK" in out.stdout, out.stdout
return out.stdout
HARNESS = """
const assert=require('assert');
var window={};
"""
# One Custom group holding one model, with three badge spellings of it.
CATALOG_ROW = "{value:'acme/example-model',providerId:'custom'}"
BADGE = "{role:'primary',label:'Primary',provider:'custom'}"
@pytest.mark.skipif(NODE is None, reason="node not on PATH")
def test_provider_qualified_badge_alias_is_not_a_second_picker_row():
"""All three spellings of one configured model collapse to one row (#2051)."""
script = HARNESS + js(
"_normalizeConfiguredModelKey",
"_isEquivalentConfiguredModelEntry",
) + f"""
const entries=[{CATALOG_ROW}];
const badge={BADGE};
// All three routing spellings name the same model the catalog already rendered.
for(const alias of ['acme/example-model',
'custom/acme/example-model',
'@custom:acme/example-model']){{
assert.strictEqual(_isEquivalentConfiguredModelEntry(alias,badge,entries), true,
'badge alias '+alias+' must not become a second picker row (#2051)');
}}
console.log('OK');
"""
run_node(script)
@pytest.mark.skipif(NODE is None, reason="node not on PATH")
def test_a_genuinely_different_model_is_never_suppressed():
"""The new rule must not hide a model the catalog does not have."""
script = HARNESS + js(
"_normalizeConfiguredModelKey",
"_isEquivalentConfiguredModelEntry",
) + f"""
const entries=[{CATALOG_ROW}];
const badge={BADGE};
// Same provider prefix, different model -> must still render its own row.
assert.strictEqual(
_isEquivalentConfiguredModelEntry('custom/acme/other-model',badge,entries), false,
'a different model under the same provider must keep its own row');
// Right model name, but no catalog row owned by that provider -> not proven
// equivalent, so keep it.
assert.strictEqual(
_isEquivalentConfiguredModelEntry(
'custom/acme/example-model',
{{role:'primary',label:'Primary',provider:'otherprovider'}},
[{{value:'acme/example-model',providerId:'custom'}}]), false,
'a same-named model owned by a different provider must keep its own row');
// A badge with no provider cannot use the prefix proof at all.
assert.strictEqual(
_isEquivalentConfiguredModelEntry('custom/acme/example-model',
{{role:'primary',label:'Primary'}},entries), false,
'a provider-less badge must not be collapsed by the prefix rule');
// The prefix must be a real path segment, not a substring of the first segment.
assert.strictEqual(
_isEquivalentConfiguredModelEntry('customer/acme/example-model',badge,entries), false,
'only a literal `<provider>/` prefix counts');
console.log('OK');
"""
run_node(script)
def test_prefix_rule_mirrors_the_at_provider_rule_structurally():
"""The `<provider>/` proof must require the same three clauses as `@<provider>:`.
Structural, not behavioural: it pins that the new branch is gated on the
badge-declared provider, on a literal prefix match, and on an existing entry
of that same provider — so it cannot grow into a loose label comparison.
"""
src = extract_function(UI_JS, "_isEquivalentConfiguredModelEntry")
assert "const slashPrefix=provider?`${provider}/`:'';" in src, (
"the provider-qualified alias rule must derive its prefix from the "
"badge-declared provider"
)
assert "rawId.toLowerCase().startsWith(slashPrefix)" in src, (
"the rule must require a literal `<provider>/` prefix on the badge key"
)
assert "String(entry.providerId||'').toLowerCase()===provider" in src, (
"the rule must require an existing picker row owned by the same provider"
)
# It must run before the `@provider:` early return, which would otherwise
# short-circuit every non-`@` alias to false.
assert src.index("slashPrefix") < src.index(
"if(!prefix||!rawId.toLowerCase().startsWith(prefix)) return false;"
), "the `<provider>/` rule must be reached before the `@<provider>:` early return"
tests/test_issue2454_active_session_spinner.py
+2
View File
@@ -183,6 +183,7 @@ def test_optimistic_merge_preserves_server_running_state_when_local_cache_is_sta
optimistic_body = _function_body(SESSIONS_SRC, "function _isOptimisticFirstTurnSessionRow(")
keep_body = _function_body(SESSIONS_SRC, "function _shouldKeepLocalOnlyOptimisticSessionRow(")
drop_body = _function_body(SESSIONS_SRC, "function _dropStaleOptimisticSessionRow(")
owner_body = _function_body(SESSIONS_SRC, "function _hasOwnedOpenLiveStream(")
merge_body = _function_body(SESSIONS_SRC, "function _mergeOptimisticFirstTurnSessions(")
script = f"""
@@ -206,6 +207,7 @@ function _isServerIdleSessionRow(s) {{{helper_body}}}
function _isSessionLocallyStreaming(s) {{{local_body}}}
function _isOptimisticFirstTurnSessionRow(s) {{{optimistic_body}}}
function _shouldKeepLocalOnlyOptimisticSessionRow(local) {{{keep_body}}}
function _hasOwnedOpenLiveStream(sid) {{{owner_body}}}
function _dropStaleOptimisticSessionRow(sid) {{{drop_body}}}
function _mergeOptimisticFirstTurnSessions(fetchedSessions) {{{merge_body}}}
const merged = _mergeOptimisticFirstTurnSessions([{{
tests/test_issue2628_cli_sessions_perf.py
+66 -14
View File
@@ -1,15 +1,14 @@
"""Regression coverage for capped CLI/agent session sidebar scans (#2628)."""
import pathlib
import sqlite3
import time
from contextlib import contextmanager
import pytest
import api.agent_sessions as agent_sessions
_REAL_SQLITE_CONNECT = sqlite3.connect
REPO_ROOT = pathlib.Path(__file__).resolve().parents[1]
def _make_state_db(path, *, sessions=80, messages_per_session=3, create_messages_index=True, source="cli", session_source="cli"):
@@ -169,6 +168,24 @@ def _make_connect_with_progress_budget(*, budget_ops, interval=1):
return _connect
@contextmanager
def _capture_listing_sql():
"""Record the SQL statements the listing runs (trace callback, read-only)."""
statements = []
def _tracing_connect(*args, **kwargs):
conn = _REAL_SQLITE_CONNECT(*args, **kwargs)
conn.set_trace_callback(statements.append)
return conn
original_connect = agent_sessions.sqlite3.connect
agent_sessions.sqlite3.connect = _tracing_connect
try:
yield statements
finally:
agent_sessions.sqlite3.connect = original_connect
def _make_connect_with_progress_counter(*, interval=1):
steps = {"count": 0}
@@ -198,24 +215,59 @@ def test_importable_agent_rows_push_sidebar_limit_into_sql(tmp_path):
db = tmp_path / "state.db"
_make_state_db(db, sessions=120, messages_per_session=5)
with _capture_listing_sql() as statements:
rows = agent_sessions.read_importable_agent_session_rows(db, limit=20, exclude_sources=("webui",))
assert len(rows) == 20
assert [row["id"] for row in rows][:3] == ["cli_perf_0119", "cli_perf_0118", "cli_perf_0117"]
assert {row["actual_message_count"] for row in rows} == {5}
src = (REPO_ROOT / "api" / "agent_sessions.py").read_text()
assert "WITH candidates AS" in src
assert "JOIN candidates c ON c.id = s.id" in src
assert "latest_messages AS" in src
assert "LEFT JOIN latest_messages lm ON lm.session_id = s.id" in src
assert 'included == ("cron",)' in src
assert "not messages_index_present" in src
assert "PRAGMA index_list(messages)" in src
assert "CREATE INDEX IF NOT EXISTS idx_messages_session" in src
assert "_CRON_PREAGGREGATE_CANDIDATE_ORDER_MIN_MESSAGES" not in src
assert "MAX(mx.timestamp) FROM messages mx WHERE mx.session_id = s.id" in src
assert "candidate_limit = max(result_limit * 8, result_limit)" in src
candidates_sql = next(
statement
for statement in statements
if "LIMIT" in statement and "FROM sessions s" in statement
)
# The cap is applied inside the candidate subquery, not in Python after an
# unbounded aggregate: the oversampled window (limit * 8 = 160 here) is
# part of the statement SQLite actually runs, and it bounds the candidate
# subquery that the outer projection joins.
assert candidates_sql.count("LIMIT") == 1, candidates_sql
window_end = candidates_sql.index("LIMIT")
window = candidates_sql[:window_end]
assert "JOIN candidates" not in window, candidates_sql
cap = int(candidates_sql[window_end:].split(None, 2)[1])
assert cap == 160, candidates_sql
def test_importable_agent_rows_candidate_window_overfilters_then_fills(tmp_path):
"""Rows dropped after projection must not shrink the returned page.
The SQL candidate window is oversampled (limit * 8) precisely so sessions
filtered client-side of the SQL (invisible CLI rows here) only consume
candidates, not result slots: a page that would be short if the cap were
the plain ``limit`` still fills to ``limit`` visible rows.
"""
db = tmp_path / "state.db"
_make_state_db(db, sessions=120, messages_per_session=5)
# 30 newest sessions are CLI rows that the projection hides (ended, default
# title, no user turn visible): they must fall out of the window without
# starving the visible page below them.
conn = sqlite3.connect(str(db))
for i in range(90, 120):
conn.execute(
"UPDATE sessions SET title = 'cli session', ended_at = 1.0, end_reason = 'timeout' WHERE id = ?",
(f"cli_perf_{i:04d}",),
)
conn.commit()
conn.close()
rows = agent_sessions.read_importable_agent_session_rows(db, limit=20, exclude_sources=("webui",))
assert len(rows) == 20
assert rows[0]["id"] == "cli_perf_0089"
assert rows[-1]["id"] == "cli_perf_0070"
assert {row["actual_message_count"] for row in rows} == {5}
def test_importable_agent_rows_candidate_ordering_stays_under_progress_budget(tmp_path, monkeypatch):
tests/test_issue3293_title_language_drift.py
+29
View File
@@ -71,6 +71,35 @@ def test_cjk_start_english_title_is_rejected():
assert _title_language_mismatch("如何修复这个错误问题", "Fixing the Bug") is True
def test_cjk_start_mixed_cjk_latin_title_is_allowed():
"""CJK conversations frequently embed English product/technical terms in
titles (e.g. 'WeChat Pay 回调失败排查'). As long as the title also
contains CJK characters, Latin borrowed terms should not trigger rejection.
Regression test for #7693."""
from api.streaming import _title_language_mismatch
# Pure CJK user, CJK title with English product name
assert _title_language_mismatch(
"微信支付回调一直失败怎么办", "WeChat Pay 回调失败排查"
) is False
# CJK user, CJK title with English tech term
assert _title_language_mismatch(
"如何修复这个错误问题", "Python 代码修复"
) is False
# Mixed CJK+Latin user, mixed title
assert _title_language_mismatch(
"prores raw是否能选择压缩？", "ProRes RAW 压缩选项与 BRAW 对比"
) is False
def test_cjk_start_pure_latin_title_still_rejected():
"""A purely Latin title for a CJK conversation is still a genuine drift."""
from api.streaming import _title_language_mismatch
assert _title_language_mismatch("如何修复这个错误问题", "Fixing the Bug") is True
assert _title_language_mismatch("如何修复这个错误问题", "Python Error Guide") is True
# ── regression guards: legitimate same-script titles must NOT be rejected ───
def test_english_start_english_title_allowed():
tests/test_issue3506_memory_and_watcher.py
+1
View File
@@ -534,6 +534,7 @@ def test_poll_loop_skips_projection_when_unchanged(tmp_path, monkeypatch):
monkeypatch.setattr(gw, "_get_agent_sessions_from_db", fake_projection)
w = gw.GatewayWatcher(state_db_path=db)
w.subscribe() # Idle watchers deliberately do not poll without observers.
assert w._poll_once(now=1.0) is True
assert projected == [True]
tests/test_issue3718_live_models_custom_probe.py
+380 -175
View File
@@ -1,193 +1,398 @@
"""Regression test for #3718 -- /api/models/live skips probe for custom providers.
"""Handler-level regression tests for /api/models/live custom-provider probing.
When a custom provider (``custom_providers`` entry in config.yaml) has a
``model:`` field but no ``models:`` allowlist, the live endpoint previously
populated ``ids`` from the config entry and then skipped the live ``/v1/models``
probe (guarded by ``if not ids``). The fix collects config-specified model IDs
in a separate ``_config_ids`` list so the live fetch always runs, and merges
config entries as a fallback after the fetch.
These tests invoke the real ``routes._handle_live_models`` and inspect the
emitted ``models`` payload. Earlier revisions of this file re-implemented the
filtering loop locally, which meant they passed unchanged even with the
production filter deleted (a vacuous oracle). Every case here must fail if the
allowlist gating in ``api/routes.py`` regresses.
Covered shapes:
* native ``models`` list → live catalog filtered
* JSON-array-string ``models`` → live catalog filtered (#7120 class)
* Python-literal-string ``models`` → live catalog filtered
* singular ``model`` only, no ``models`` → live catalog NOT gated
* malformed serialized ``models`` → safe, treated as scalar name
* allowlisted model absent from live catalog → appended
* live probe failure → falls back to config ids
* dict-shaped ``models`` (Agent setup) → live catalog NOT gated (per-model
metadata, not an allowlist)
* dict-shaped ``models`` + ``discover_models:
false`` → pinned to dict keys (explicit opt-out)
* ``models_discovered: true`` catalog → live catalog NOT gated (discovery
metadata is not an allowlist)
* discovered + ``discover_models: false`` → still pinned (explicit opt-out)
"""
import io
import json
import pathlib
import unittest
from unittest import mock
import sys
import types
from urllib.parse import urlparse
REPO = pathlib.Path(__file__).parent.parent
ROUTES_PY = REPO / "api" / "routes.py"
import pytest
class TestLiveModelsCustomProviderProbe(unittest.TestCase):
"""Live endpoint must probe the upstream /v1/models even when config has entries."""
def _install_provider_model_ids(monkeypatch, fn):
"""Stub hermes_cli.models.provider_model_ids (returns [] for custom:*)."""
hermes_cli = types.ModuleType("hermes_cli")
hermes_cli.__path__ = []
models = types.ModuleType("hermes_cli.models")
models.provider_model_ids = fn
monkeypatch.setitem(sys.modules, "hermes_cli", hermes_cli)
monkeypatch.setitem(sys.modules, "hermes_cli.models", models)
def test_custom_provider_with_model_field_probes_upstream(self):
"""When a custom provider has model: in config, live fetch must still run.
Before the fix, _custom_provider_model_ids() populated ids=["assistant"],
and the `if not ids` guard prevented the upstream /v1/models probe from
running. The endpoint returned only ["assistant"] instead of the full
upstream catalog.
class _FakeResponse(io.BytesIO):
def __enter__(self):
return self
We test the fix by simulating the logic directly: config provides
model IDs, the upstream probe returns a different set, and the result
must contain BOTH the live models and the config fallback.
"""
# Simulate config-specified model IDs
_config_ids = ["assistant"]
def __exit__(self, *_exc):
self.close()
return False
# Simulate upstream /v1/models response
live_models = [
"assistant",
"assistant-pro",
"assistant-zdr",
"gpt-5.5:pt",
"claude-sonnet-4.6:pt",
]
# Simulate the merge logic from the fix:
# if ids: merge config entries not in live set
# else: fall back to config-only list
ids = list(live_models) # live fetch succeeded
_live_set = set(ids)
for _cid in _config_ids:
if _cid not in _live_set:
ids.append(_cid)
# "assistant" from config should not duplicate the live result
self.assertEqual(ids, live_models)
# All live models must be present
for m in live_models:
self.assertIn(m, ids)
def test_custom_provider_falls_back_to_config_when_probe_fails(self):
"""When the upstream probe fails, config entries must be used as fallback."""
_config_ids = ["assistant", "custom-only-model"]
# Simulate failed live fetch: ids stays empty
ids = []
# Fallback logic from the fix
if not ids:
ids = list(_config_ids)
self.assertEqual(ids, ["assistant", "custom-only-model"])
def test_config_only_model_appended_when_not_in_live_results(self):
"""Config-specified models not in the live results must be appended."""
_config_ids = ["assistant", "local-finetune"]
live_models = ["assistant", "assistant-pro", "gpt-5.5:pt"]
ids = list(live_models)
_live_set = set(ids)
for _cid in _config_ids:
if _cid not in _live_set:
ids.append(_cid)
# "assistant" is in both, "local-finetune" is config-only
self.assertIn("local-finetune", ids)
self.assertIn("assistant-pro", ids)
self.assertEqual(ids.count("assistant"), 1, "assistant must not be duplicated")
def test_live_fetch_guard_no_longer_checks_ids(self):
"""The live fetch block must not be guarded by `if not ids`.
This is a structural regression guard: the fix replaced
`if not ids and (provider == "custom" ...)` with
`if provider == "custom" ...` so the probe always runs regardless
of whether config entries populated ids.
"""
source = ROUTES_PY.read_text(encoding="utf-8")
# Find the "Always try live fetch" comment (added by the fix)
marker = "Always try live fetch for custom providers"
self.assertIn(marker, source, (
"routes.py must contain the 'Always try live fetch' comment (#3718)"
))
# Extract the block between the marker and the next major section
marker_pos = source.find(marker)
next_section = source.find("OpenAI-compat live fetch fallback", marker_pos)
self.assertNotEqual(next_section, -1, "could not find next section marker")
block = source[marker_pos:next_section]
# The old `if not ids and` guard must NOT appear in this block
self.assertNotIn("if not ids and", block, (
"Live fetch for custom providers must not be guarded by 'if not ids' (#3718)"
))
def test_mocked_live_fetch_returns_full_catalog_plus_config_entry(self):
"""Integration test: mock urlopen and exercise the real fetch/parse/merge path.
This test patches urllib.request.urlopen to return a fake /v1/models
response, then runs the same fetch+merge logic that the live handler
uses, verifying that config entries are merged and live models are
included.
"""
fake_models = {"data": [
{"id": "assistant"},
{"id": "assistant-pro"},
{"id": "gpt-5.5:pt"},
]}
fake_body = json.dumps(fake_models).encode("utf-8")
fake_resp = mock.MagicMock()
fake_resp.read.return_value = fake_body
fake_resp.__enter__ = mock.MagicMock(return_value=fake_resp)
fake_resp.__exit__ = mock.MagicMock(return_value=False)
_config_ids = ["assistant", "local-only"]
with mock.patch("urllib.request.urlopen", return_value=fake_resp):
# Replicate the fetch+parse logic from routes.py
def _install_live_catalog(monkeypatch, routes, catalog, *, fail=False):
"""Stub the upstream /v1/models probe used by the custom-provider branch."""
import urllib.request
ids = []
_req = urllib.request.Request(
"http://localhost:4000/v1/models",
headers={"Authorization": "Bearer test-key"},
requested: list[str] = []
def _fake_urlopen(req, timeout=None):
requested.append(getattr(req, "full_url", str(req)))
if fail:
raise OSError("probe failed")
body = {"data": [{"id": mid} for mid in catalog]}
return _FakeResponse(json.dumps(body).encode())
monkeypatch.setattr(urllib.request, "urlopen", _fake_urlopen)
return requested
def _run_live_models(monkeypatch, routes, custom_provider, *, catalog, fail=False):
"""Invoke the real handler for provider=custom:test-gateway."""
import api.config as config
import api.profiles as profiles
routes._clear_live_models_cache()
monkeypatch.setattr(
routes, "j", lambda _handler, payload, status=200, extra_headers=None: payload
)
with urllib.request.urlopen(_req, timeout=5) as _resp:
_body = json.loads(_resp.read())
monkeypatch.setattr(config, "_resolve_provider_alias", lambda provider: provider)
monkeypatch.setattr(profiles, "get_active_profile_name", lambda: "default")
if isinstance(_body, dict):
_data = _body.get("data", [])
if isinstance(_data, list):
ids = [m.get("id", "") for m in _data if m.get("id")]
config_data = {
"model": {"provider": "custom:test-gateway"},
"custom_providers": [custom_provider],
}
monkeypatch.setattr(config, "get_config", lambda: config_data)
monkeypatch.setattr(routes, "cfg", config_data, raising=False)
# Apply the merge logic from the fix
if ids:
_live_set = set(ids)
for _cid in _config_ids:
if _cid not in _live_set:
ids.append(_cid)
else:
ids = list(_config_ids)
# provider_model_ids() returns [] for custom:* → drives the custom branch.
_install_provider_model_ids(monkeypatch, lambda provider: [])
requested = _install_live_catalog(monkeypatch, routes, catalog, fail=fail)
# Live models must all be present
self.assertIn("assistant", ids)
self.assertIn("assistant-pro", ids)
self.assertIn("gpt-5.5:pt", ids)
# Config-only entry must be appended
self.assertIn("local-only", ids)
# No duplicates
self.assertEqual(ids.count("assistant"), 1)
def test_timeout_uses_config_constant(self):
"""The live fetch must use CUSTOM_MODELS_ENDPOINT_TIMEOUT_SECONDS, not a hardcoded value."""
source = ROUTES_PY.read_text(encoding="utf-8")
# Find the custom-provider live fetch block
marker = "Always try live fetch for custom providers"
marker_pos = source.find(marker)
next_section = source.find("OpenAI-compat live fetch fallback", marker_pos)
block = source[marker_pos:next_section]
# Must use the constant, not a bare number
self.assertIn("CUSTOM_MODELS_ENDPOINT_TIMEOUT_SECONDS", block, (
"Live fetch timeout must use CUSTOM_MODELS_ENDPOINT_TIMEOUT_SECONDS "
"instead of a hardcoded value (#3718 review feedback)"
))
self.assertNotIn("timeout=8", block, (
"Live fetch must not use hardcoded timeout=8 (#3718 review feedback)"
))
parsed = urlparse("/api/models/live?provider=custom:test-gateway")
payload = routes._handle_live_models(object(), parsed)
return payload, requested
if __name__ == "__main__":
unittest.main()
def _ids(payload):
return [m["id"] for m in payload.get("models", [])]
_LIVE_CATALOG = [
"chat-a",
"chat-b",
"image-gen-1", # non-chat noise the picker must not surface
"audio-tts-1",
]
_BASE_PROVIDER = {
"name": "Test Gateway",
"base_url": "https://gateway.example/v1",
"api_key": "sk-test",
}
@pytest.mark.parametrize(
"models_value",
[
pytest.param(["chat-a", "chat-b"], id="native-list"),
pytest.param('["chat-a","chat-b"]', id="json-array-string"),
pytest.param("['chat-a', 'chat-b']", id="python-literal-string"),
pytest.param(
[{"id": "chat-a"}, {"model": "chat-b"}], id="list-of-dicts"
),
],
)
def test_allowlist_filters_live_catalog(monkeypatch, models_value):
"""An explicit plural allowlist must trim the live catalog to itself.
The JSON-array-string and Python-literal cases are the regression: those
shapes are what ``hermes config set`` / JSON-mode editor saves persist, and
a dict/list-only parser drops them to ``[]`` → read as "no allowlist" → the
full upstream catalog floods the picker.
"""
import api.routes as routes
provider = dict(_BASE_PROVIDER, models=models_value)
payload, requested = _run_live_models(
monkeypatch, routes, provider, catalog=_LIVE_CATALOG
)
assert requested, "live /v1/models probe was never attempted"
assert _ids(payload) == ["chat-a", "chat-b"]
assert "image-gen-1" not in _ids(payload)
assert "audio-tts-1" not in _ids(payload)
def test_dict_models_without_discover_false_does_not_gate(monkeypatch):
"""A dict-shaped ``models`` is per-model metadata — NOT an allowlist (#7165).
Hermes Agent's setup flow and ``hermes_cli/model_switch.py::_save_custom_provider``
persist per-model metadata as a mapping, e.g. ``{chat-a: {context_length: 128000}}``.
Treating its keys as an allowlist would collapse the live picker to the saved
default while the CLI live-probe shows the full catalog. Without an explicit
``discover_models: false`` opt-out the dict must NOT gate — full live catalog wins.
"""
import api.routes as routes
provider = dict(
_BASE_PROVIDER,
models={"chat-a": {"context_length": 128000}, "chat-b": {"context_length": 32000}},
)
payload, requested = _run_live_models(
monkeypatch, routes, provider, catalog=_LIVE_CATALOG
)
assert requested, "live /v1/models probe was never attempted"
assert _ids(payload) == _LIVE_CATALOG, (
"dict-shaped models is per-model metadata, not an allowlist — the "
"full live catalog must be surfaced"
)
def test_dict_models_with_discover_false_pins_to_keys(monkeypatch):
"""``discover_models: false`` opts a dict-shaped ``models`` into an allowlist.
The explicit opt-out (bool ``False`` or the string forms ``"false"``/``"no"``/``"0"``)
means the dict keys are a hand-pinned allowlist and must filter the live catalog.
"""
import api.routes as routes
for discover in (False, "false", "no", "0"):
provider = dict(
_BASE_PROVIDER,
models={"chat-a": {"context_length": 128000}, "chat-b": {"context_length": 32000}},
discover_models=discover,
)
payload, requested = _run_live_models(
monkeypatch, routes, provider, catalog=_LIVE_CATALOG
)
assert requested, "live /v1/models probe was never attempted"
assert _ids(payload) == ["chat-a", "chat-b"], (
f"discover_models={discover!r} must pin the dict keys as the allowlist"
)
assert "image-gen-1" not in _ids(payload)
assert "audio-tts-1" not in _ids(payload)
def test_singular_model_only_does_not_gate_live_catalog(monkeypatch):
"""``model:`` is default metadata, not an allowlist — keep full discovery."""
import api.routes as routes
provider = dict(_BASE_PROVIDER, model="chat-a")
payload, _ = _run_live_models(
monkeypatch, routes, provider, catalog=_LIVE_CATALOG
)
assert _ids(payload) == _LIVE_CATALOG
def test_malformed_serialized_models_is_safe(monkeypatch):
"""A broken JSON-array string must not crash or silently drop the gate.
``_parse_config_string_list()`` treats an undecodable value as one literal
name, so the allowlist stays non-empty and the filter still runs.
"""
import api.routes as routes
provider = dict(_BASE_PROVIDER, models='["chat-a", "chat-b"') # missing ]
payload, _ = _run_live_models(
monkeypatch, routes, provider, catalog=_LIVE_CATALOG
)
# Treated as a single (unmatched) name → appended, catalog not flooded.
assert "image-gen-1" not in _ids(payload)
assert "audio-tts-1" not in _ids(payload)
def test_allowlisted_model_missing_from_live_is_appended(monkeypatch):
"""A declared model the upstream didn't return must still be selectable."""
import api.routes as routes
provider = dict(_BASE_PROVIDER, models=["chat-a", "chat-offline"])
payload, _ = _run_live_models(
monkeypatch, routes, provider, catalog=_LIVE_CATALOG
)
assert _ids(payload) == ["chat-a", "chat-offline"]
def test_contradictory_singular_model_outside_plural_allowlist_is_ignored(monkeypatch):
"""A singular default outside the plural allowlist must not leak into the picker.
Paired test: whether the live probe succeeds or fails, only the explicit
plural allowlist is exposed.
"""
import api.routes as routes
provider = dict(_BASE_PROVIDER, model="chat-default", models=["chat-a"])
# 1. Successful live fetch -> only plural allowlist
payload_success, _ = _run_live_models(
monkeypatch, routes, provider, catalog=_LIVE_CATALOG, fail=False
)
assert _ids(payload_success) == ["chat-a"]
# 2. Failed live fetch -> still only plural allowlist (not chat-default)
payload_fail, _ = _run_live_models(
monkeypatch, routes, provider, catalog=_LIVE_CATALOG, fail=True
)
assert _ids(payload_fail) == ["chat-a"]
def test_probe_failure_without_allowlist_falls_back_to_singular_model(monkeypatch):
"""When no plural allowlist is configured and probe fails, singular model is used."""
import api.routes as routes
provider = dict(_BASE_PROVIDER, model="chat-default")
payload, _ = _run_live_models(
monkeypatch, routes, provider, catalog=_LIVE_CATALOG, fail=True
)
assert _ids(payload) == ["chat-default"]
@pytest.mark.parametrize(
"models_value",
[
pytest.param([], id="native-empty-list"),
pytest.param("[]", id="serialized-empty-list"),
pytest.param({}, id="empty-mapping"),
pytest.param(" ", id="blank-scalar"),
],
)
def test_empty_allowlist_is_treated_as_not_configured(monkeypatch, models_value):
"""An empty ``models`` value must NOT be read as "allow nothing".
Pins a deliberate semantic decision. Gating the filter on *declared-ness*
rather than on a non-empty allowlist would make ``models: []`` emit an
EMPTY picker, leaving the provider unselectable — a harder failure than
surfacing a few extra models, and unrecoverable from the UI because
``custom_providers`` has no WebUI write path (hand-edited config.yaml only).
There is also no deny-all use case: a provider the user wants hidden is
removed from ``custom_providers`` outright.
If someone later "fixes" the empty case into a deny-all gate, this test
fails and forces the trade-off to be re-argued rather than silently shipped.
"""
import api.routes as routes
provider = dict(_BASE_PROVIDER, models=models_value)
payload, _ = _run_live_models(
monkeypatch, routes, provider, catalog=_LIVE_CATALOG
)
assert _ids(payload) == _LIVE_CATALOG, (
"empty allowlist must fall through to full live discovery, "
"never collapse the picker to zero models"
)
assert _ids(payload), "picker must never be emptied by an empty allowlist"
def test_models_discovered_catalog_is_not_an_allowlist(monkeypatch):
"""``models_discovered: true`` marks ``models:`` as discovery metadata (#7165).
Hermes persists *discovery results* back into config as ``models: {...}``
plus ``models_discovered: true``. That mapping is a snapshot of what the
gateway exposed at discovery time — NOT a hand-curated allowlist. Treating
it as one permanently pins the live catalog to the first-discovery set, so
a model the user later pulls into LM Studio / Ollama is silently dropped
from ``/api/models/live`` (the exact regression the gate reproduced: merge
base returned ``saved-a``, ``saved-b`` AND ``new-live-c``; the pinned head
dropped ``new-live-c``).
"""
import api.routes as routes
provider = dict(
_BASE_PROVIDER,
models={
"saved-a": {"context_length": 128000},
"saved-b": {"context_length": 32000},
},
models_discovered=True,
)
payload, requested = _run_live_models(
monkeypatch,
routes,
provider,
catalog=["saved-a", "saved-b", "new-live-c"],
)
assert requested, "live /v1/models probe was never attempted"
assert _ids(payload) == ["saved-a", "saved-b", "new-live-c"], (
"a discovered catalog must not gate the live probe — models that "
"appear upstream after discovery (new-live-c) must reach the picker"
)
def test_discovered_catalog_with_discover_models_false_still_pins(monkeypatch):
"""``discover_models: false`` re-pins a discovered catalog (control).
The explicit opt-out wins over ``models_discovered: true`` — same rule as
the ``/api/models`` catalog path (`api.config._provider_discover_allowed`).
Filtering to the configured set must survive the discovered-catalog fix.
"""
import api.routes as routes
provider = dict(
_BASE_PROVIDER,
models={
"saved-a": {"context_length": 128000},
"saved-b": {"context_length": 32000},
},
models_discovered=True,
discover_models=False,
)
payload, _ = _run_live_models(
monkeypatch,
routes,
provider,
catalog=["saved-a", "saved-b", "new-live-c"],
)
assert _ids(payload) == ["saved-a", "saved-b"]
def test_discovered_catalog_probe_failure_falls_back_to_saved_models(monkeypatch):
"""Discovered + failed probe → saved models, not an empty picker (control).
Releasing the allowlist must not empty the fallback: when the live probe
fails, the previously discovered ``models:`` mapping is still the best
answer and must be served from ``_config_ids``.
"""
import api.routes as routes
provider = dict(
_BASE_PROVIDER,
models={
"saved-a": {"context_length": 128000},
"saved-b": {"context_length": 32000},
},
models_discovered=True,
)
payload, _ = _run_live_models(
monkeypatch, routes, provider, catalog=[], fail=True
)
assert _ids(payload) == ["saved-a", "saved-b"]
tests/test_issue3887_index_prime.py
+22 -12
View File
@@ -74,28 +74,38 @@ def _messages_indexes(path):
return {r[0] for r in rows}
def test_prime_creates_missing_index(tmp_path):
"""A db missing idx_messages_session gets it primed on the first scan."""
def test_listing_does_not_create_missing_index(tmp_path, monkeypatch):
"""A missing index must never turn a sidebar read into a schema write.
CREATE INDEX on a multi-GiB ``messages`` table holds the SQLite writer lock
for minutes; index maintenance belongs to the explicit drained tool
(``scripts/ensure_state_db_read_indexes.py``), not to a listing.
"""
db = tmp_path / "state.db"
_full_schema_db(db)
assert "idx_messages_session" not in _messages_indexes(db)
connect_calls = []
real_connect = agent_sessions.sqlite3.connect
def recording_connect(*args, **kwargs):
connect_calls.append((args, kwargs))
return real_connect(*args, **kwargs)
monkeypatch.setattr(agent_sessions.sqlite3, "connect", recording_connect)
rows = agent_sessions.read_importable_agent_session_rows(
db, limit=20, exclude_sources=None
)
# Snapshot before the verifier below opens its own (non-listing) handle.
listing_connect_calls = list(connect_calls)
# Listing still returns the sessions ...
assert {r["id"] for r in rows} == {"sess0", "sess1", "sess2"}
# ... and the index now exists on (session_id, timestamp).
assert "idx_messages_session" in _messages_indexes(db)
conn = sqlite3.connect(str(db))
try:
sql = conn.execute(
"SELECT sql FROM sqlite_master WHERE name='idx_messages_session'"
).fetchone()[0]
finally:
conn.close()
assert "session_id" in sql and "timestamp" in sql
# ... without mutating the schema or opening a write-capable connection.
assert "idx_messages_session" not in _messages_indexes(db)
assert len(listing_connect_calls) == 1
assert "mode=ro" in str(listing_connect_calls[0][0][0])
assert listing_connect_calls[0][1].get("uri") is True
def test_prime_is_noop_when_index_exists(tmp_path):
tests/test_issue4067_import_cli_cross_profile_guard.py
+23 -4
View File
@@ -41,7 +41,17 @@ class _FakeHandler:
def _capture(monkeypatch):
cap = {}
monkeypatch.setattr(routes, "j", lambda h, o: (cap.__setitem__("ok", o), True)[1])
# #7710: the real ``j`` signature is ``j(handler, payload, status=200,
# extra_headers=None, *, pretty=True)``; the mock has to accept the
# status keyword so the helper's 409 ``session_profile_mismatch``
# payload can be captured here. Capture the STATUS too — swallowing it
# into ``*_`` would let a wrong status (200/404) pass this test.
def _fake_j(h, o, status=200, *_, **__):
cap["ok"] = o
cap["ok_status"] = status
return True
monkeypatch.setattr(routes, "j", _fake_j)
monkeypatch.setattr(
routes,
"bad",
@@ -72,7 +82,10 @@ class _FakeSession:
def test_import_cli_existing_foreign_profile_unqualified_request_404(monkeypatch):
"""Unqualified request (active profile=default) for a session stored under a
foreign profile must 404 — not read or refresh the foreign session."""
foreign profile must yield 409 ``session_profile_mismatch`` (so the client
can offer to switch to it per #5419) — not 404, which would mask a
legitimate owned-by-other-profile session as missing and trigger a
destructive self-heal in the frontend (#7710)."""
foreign = _FakeSession("foreign_existing_001", "other")
monkeypatch.setattr(routes.Session, "load", staticmethod(lambda sid: foreign))
monkeypatch.setattr(routes, "_get_active_profile_name", lambda: "default")
@@ -88,8 +101,14 @@ def test_import_cli_existing_foreign_profile_unqualified_request_404(monkeypatch
{"session_id": "foreign_existing_001"},
)
assert "bad" in cap, f"expected 404, got {cap}"
assert cap["bad"][1] == 404
assert "ok" in cap, f"expected 409 session_profile_mismatch, got {cap}"
assert cap["ok_status"] == 409, f"expected status 409, got {cap.get('ok_status')}"
assert cap["ok"] == {
"error": "Session belongs to a different profile",
"code": "session_profile_mismatch",
"session_id": "foreign_existing_001",
"profile": "other",
}
def test_import_cli_existing_same_profile_still_refreshes(monkeypatch):
tests/test_issue4766_sidebar_source_pushdown.py
+2 -2
View File
@@ -536,7 +536,7 @@ def test_source_filtered_cache_preserves_hidden_bucket_runtime_state():
remember_source_fn = _extract_function(src, "_rememberSessionListSource")
remember_streaming_fn = _extract_function(src, "_rememberRenderedStreamingState")
remember_snapshot_fn = _extract_function(src, "_rememberRenderedSessionSnapshot")
purge_fn = _extract_function(src, "_purgeStaleInflightEntries")
purge_fn = _extract_function(src, "_hasOwnedOpenLiveStream") + "\n" + _extract_function(src, "_purgeStaleInflightEntries")
mark_fn = _extract_function(src, "_markPollingCompletionUnreadTransitions")
script = f"""
global._allSessions = [{{
@@ -654,7 +654,7 @@ def test_session_list_response_omits_bucket_counts_when_missing(monkeypatch):
@pytest.mark.skipif(NODE is None, reason="node not on PATH")
def test_scope_mismatch_error_path_respects_sidebar_source():
src = SESSIONS_JS.read_text(encoding="utf-8")
purge_fn = _extract_function(src, "_purgeStaleInflightEntries")
purge_fn = _extract_function(src, "_hasOwnedOpenLiveStream") + "\n" + _extract_function(src, "_purgeStaleInflightEntries")
clear_fn = _extract_function(src, "_clearSessionSourceTabCounts")
requested_source_fn = _extract_function(src, "_requestedSessionSidebarSource")
exclude_hidden_fn = _extract_function(src, "_sessionListExcludeHiddenEnabled")
tests/test_issue4812_session_sse_stream.py
+70 -1
View File
@@ -507,7 +507,18 @@ def test_session_route_blocks_hidden_sessions_before_replay_or_live_attach(monke
handler = _FakeHandler()
routes.handle_get(handler, urlparse("/api/sessions/hidden_session/events"))
assert cap["bad"] == ("Session not found", 404)
# #7710: the generic request-guard now mirrors the detail-load
# endpoint's contract — a session owned by a KNOWN other profile
# yields 409 ``session_profile_mismatch`` so the client can offer
# to switch to it (#5419). The 404 self-heal path is preserved for
# the None-profile (unknown/legacy) case.
assert cap.get("status") == 409, cap
assert cap.get("ok") == {
"error": "Session belongs to a different profile",
"code": "session_profile_mismatch",
"session_id": "hidden_session",
"profile": "other",
}
def test_session_route_live_delivery_skips_replayed_active_run_items(monkeypatch):
@@ -583,6 +594,64 @@ def test_session_route_live_delivery_skips_replayed_active_run_items(monkeypatch
assert stream.unsubscribed is True
def test_session_route_replay_skips_metering_rows(monkeypatch):
import api.routes as routes
class _FakeStream:
def __init__(self):
self.q = queue.Queue()
self.q.put_nowait(("stream_end", {"status": "done"}, "run_active:2"))
self.unsubscribed = False
def subscribe_with_snapshot(self):
return self.q, {"last_event_id": "run_active:1", "offline_buffered_events": 1}
def unsubscribe(self, q):
self.unsubscribed = q is self.q
stream = _FakeStream()
monkeypatch.setattr(routes, "_session_id_visible_to_request_profile", lambda *_args, **_kwargs: True)
monkeypatch.setattr(
routes,
"get_session",
lambda sid, metadata_only=False: SimpleNamespace(
session_id=sid,
compact=lambda **_kwargs: {"session_id": sid, "title": "Session"},
),
)
monkeypatch.setattr(routes, "_active_run_stream_for_session", lambda *_args, **_kwargs: "run_active")
monkeypatch.setattr(routes, "STREAMS", {"run_active": stream})
import api.config as config
monkeypatch.setattr(config, "STREAMS", {"run_active": stream})
# Both the pre-attach replay and the post-attach reconciliation draw from the
# same journal; a metering row among them must never reach the SSE body.
monkeypatch.setattr(
routes,
"read_session_run_events",
lambda *_args, **_kwargs: {
"status": "ok",
"events": [
{"run_id": "run_prev", "seq": 1, "event": "token", "payload": {"text": "replayed"}, "event_id": "run_prev:1"},
{"run_id": "run_prev", "seq": 2, "event": "metering", "payload": {"tps": 47.3}, "event_id": "run_prev:2"},
],
},
)
handler = _FakeHandler()
routes._handle_session_sse_stream_for_session(
handler,
urlparse("/api/sessions/session_1/events?after_event_id=run_prev:0"),
"session_1",
)
body = handler.wfile.getvalue().decode("utf-8")
assert "id: run_prev:1\n" in body
assert "event: token\n" in body
assert "event: metering\n" not in body
assert "id: run_prev:2\n" not in body
assert stream.unsubscribed is True
def test_session_route_breaks_on_terminal_even_when_reconciliation_replayed_it(monkeypatch):
"""CORE regression (#5677 gate r3): if reconciliation replays the active run's
TERMINAL event at the subscribe cutoff, the live queue's copy is deduped — but
tests/test_issue5311_opencode_go_live_models.py
+302
View File
@@ -0,0 +1,302 @@
"""#5311 follow-up: OpenCode Go uses the live Hermes CLI catalog again.
Background: #5611 made WebUI skip the live ``/v1/models`` probe for OpenCode Go
because the probe returned public-catalog models not enabled on the Go tier.
That premise no longer holds: Hermes core v0.20.5 (commit fcbd1076, tag
v2026.8.19) ships a Go-specific provider profile that probes
``https://opencode.ai/zen/go/v1/models`` directly and merges results
live-first with its curated Go list (hermes_cli/models.py, #49129). With the
correct Go-tier endpoint in core, WebUI should delegate like every other
provider (#1240): live catalog first, the curated static ``_PROVIDER_MODELS``
list as offline fallback only.
"""
from __future__ import annotations
import sys
import types
import api.config as config
import api.profiles as profiles
_PROVIDER_ENV_VARS = (
"OPENCODE_ZEN_API_KEY",
"OPENCODE_GO_API_KEY",
"OPENCODE_API_KEY",
)
def _scrub_provider_env(monkeypatch):
for name in _PROVIDER_ENV_VARS:
monkeypatch.delenv(name, raising=False)
def _install_fake_hermes_cli(
monkeypatch,
*,
provider_id: str,
live_ids,
core_version: str = "0.20.5",
raise_on_lookup: bool = False,
):
"""Install a hermes_cli stub that reports one authenticated provider."""
fake_pkg = types.ModuleType("hermes_cli")
fake_pkg.__path__ = []
fake_pkg.__version__ = core_version
fake_models = types.ModuleType("hermes_cli.models")
fake_models.list_available_providers = lambda: [
{"id": provider_id, "authenticated": True}
]
calls: list[str] = []
def provider_model_ids(pid):
calls.append(pid)
if raise_on_lookup:
raise RuntimeError("simulated provider_model_ids failure")
return list(live_ids) if pid == provider_id else []
fake_models.provider_model_ids = provider_model_ids
fake_auth = types.ModuleType("hermes_cli.auth")
def get_auth_status(pid):
if pid == provider_id:
return {"logged_in": True, "key_source": ""}
return {"logged_in": False, "key_source": ""}
fake_auth.get_auth_status = get_auth_status
monkeypatch.setitem(sys.modules, "hermes_cli", fake_pkg)
monkeypatch.setitem(sys.modules, "hermes_cli.models", fake_models)
monkeypatch.setitem(sys.modules, "hermes_cli.auth", fake_auth)
monkeypatch.delitem(sys.modules, "agent.credential_pool", raising=False)
monkeypatch.delitem(sys.modules, "agent", raising=False)
config.invalidate_models_cache()
return calls
def _configure(monkeypatch, tmp_path, *, provider: str, providers: dict | None = None):
hermes_home = tmp_path / "hermes-home"
hermes_home.mkdir()
monkeypatch.setattr(profiles, "get_active_hermes_home", lambda: hermes_home)
monkeypatch.setattr(config, "_get_config_path", lambda: tmp_path / "missing-config.yaml")
monkeypatch.setattr(config, "_models_cache_path", tmp_path / "models_cache.json")
monkeypatch.setattr(
config,
"cfg",
{
"model": {"provider": provider, "default": ""},
"providers": providers if providers is not None else {},
"fallback_providers": [],
},
)
monkeypatch.setattr(config, "_cfg_mtime", 0.0)
monkeypatch.setattr(config, "_cfg_path", config._get_config_path(), raising=False)
config.invalidate_models_cache()
def _provider_group(result: dict, provider_id: str) -> dict:
return next(g for g in result["groups"] if g.get("provider_id") == provider_id)
def _ids(group: dict) -> list[str]:
return [m.get("id") for m in group.get("models", [])]
def test_opencode_go_probes_live_catalog(monkeypatch, tmp_path):
"""The Go picker group must come from the live Hermes CLI catalog.
``sentinel-go-model`` is intentionally absent from every static list. If
WebUI special-cases OpenCode Go away from the live probe again (the #5611
stopgap), the sentinel disappears and this test fails.
"""
_scrub_provider_env(monkeypatch)
calls = _install_fake_hermes_cli(
monkeypatch,
provider_id="opencode-go",
live_ids=["sentinel-go-model", "kimi-k3"],
)
_configure(monkeypatch, tmp_path, provider="opencode-go")
result = config.get_available_models()
group = _provider_group(result, "opencode-go")
assert calls == ["opencode-go"]
ids = _ids(group)
assert "sentinel-go-model" in ids
assert "kimi-k3" in ids
assert all(m.get("label") for m in group["models"])
def test_opencode_go_old_core_ignores_nonempty_generic_catalog(monkeypatch, tmp_path):
"""Pre-v0.20.5 cores must not leak their generic public catalog into Go."""
_scrub_provider_env(monkeypatch)
calls = _install_fake_hermes_cli(
monkeypatch,
provider_id="opencode-go",
live_ids=["go-ineligible-old-core-sentinel"],
core_version="0.20.4",
)
_configure(monkeypatch, tmp_path, provider="opencode-go")
result = config.get_available_models()
group = _provider_group(result, "opencode-go")
assert calls == []
ids = _ids(group)
assert "go-ineligible-old-core-sentinel" not in ids
assert "kimi-k3" in ids
def test_opencode_go_static_fallback_when_probe_fails(monkeypatch, tmp_path):
"""Offline / CLI failure must fall back to the curated static list."""
_scrub_provider_env(monkeypatch)
calls = _install_fake_hermes_cli(
monkeypatch,
provider_id="opencode-go",
live_ids=[],
raise_on_lookup=True,
)
_configure(monkeypatch, tmp_path, provider="opencode-go")
result = config.get_available_models()
group = _provider_group(result, "opencode-go")
assert calls == ["opencode-go"]
ids = _ids(group)
assert "sentinel-go-model" not in ids
assert "glm-5" in ids
assert "kimi-k2.5" in ids
def test_opencode_go_config_allowlist_still_wins(monkeypatch, tmp_path):
"""An explicit ``providers.opencode-go.models`` allowlist remains the
local source of truth (#644) and is not bypassed by the live probe."""
_scrub_provider_env(monkeypatch)
calls = _install_fake_hermes_cli(
monkeypatch,
provider_id="opencode-go",
live_ids=["sentinel-go-model"],
)
_configure(
monkeypatch,
tmp_path,
provider="opencode-go",
providers={"opencode-go": {"models": ["glm-5"]}},
)
result = config.get_available_models()
group = _provider_group(result, "opencode-go")
assert calls == [] # probe never consulted
assert _ids(group) == ["glm-5"]
# ── Static fallback list contract ─────────────────────────────────────
# The remaining tests pin the offline fallback list itself to Hermes core's
# current curated contract (hermes_cli/models_catalog_static.py, core main
# 2026-09-10; the list originated in v0.20.5, commit fcbd1076). Core owns
# the sync duty against the live
# https://opencode.ai/zen/go/v1/models endpoint and
# https://opencode.ai/docs/go/, and WebUI only mirrors. This catches drift
# in either direction: ids core added and ids core dropped (e.g.
# ox-alpha-free, delisted from the Go relay on 2026-09-09).
import ast
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
CONFIG_PATH = ROOT / "api" / "config.py"
CONFIG = CONFIG_PATH.read_text(encoding="utf-8")
EXPECTED_OPENCODE_GO_MODEL_IDS = [
"kimi-k3",
"kimi-k2.7-code",
"kimi-k2.6",
"kimi-k2.5",
"gpt-5.6-luna",
"grok-4.5",
"glm-5.3",
"glm-5.3-flash",
"glm-5.2",
"glm-5.1",
"glm-5",
"mimo-v2.5-pro",
"mimo-v2.5",
"mimo-v2-pro",
"mimo-v2-omni",
"minimax-m3",
"minimax-m2.7",
"minimax-m2.5",
"deepseek-v4-pro",
"deepseek-v4-flash",
"qwen3.8-max",
"qwen3.7-max",
"qwen3.7-plus",
"qwen3.6-plus",
"qwen3.5-plus",
"hy3",
"hy3-preview",
"muse-spark-1.2-contributor",
"muse-spark-1.3-contributor",
]
def _opencode_go_static_models():
tree = ast.parse(CONFIG, filename=str(CONFIG_PATH))
for node in tree.body:
if not isinstance(node, ast.Assign):
continue
if not any(isinstance(target, ast.Name) and target.id == "_PROVIDER_MODELS" for target in node.targets):
continue
provider_models = ast.literal_eval(node.value)
return provider_models["opencode-go"]
raise AssertionError("_PROVIDER_MODELS assignment not found")
def test_opencode_go_static_models_match_core_catalog_snapshot():
models = _opencode_go_static_models()
assert [model["id"] for model in models] == EXPECTED_OPENCODE_GO_MODEL_IDS
def test_opencode_go_static_models_match_installed_core_curated_list():
"""Exercise the actual Agent contract for the supported release pair.
The snapshot mirrors core main's curated catalog. A released (installed)
core can lag that sync by a few days — e.g. v2026.9.7 still lists
``ox-alpha-free``, which core removed on 2026-09-09 after the Go relay
delisted it. Equality binds only once the installed core carries the
mirrored sync; the marker for "lags the sync" is the delisted id still
being present. Anything else that differs is real drift and must fail.
"""
import pytest
hermes_cli = pytest.importorskip(
"hermes_cli", reason="hermes-agent is not a WebUI test dependency"
)
if not config._hermes_cli_supports_opencode_go_live_catalog():
pytest.skip(f"installed hermes-agent {hermes_cli.__version__} predates v0.20.5")
models_mod = pytest.importorskip("hermes_cli.models")
installed = list(models_mod._PROVIDER_MODELS["opencode-go"])
if "ox-alpha-free" in installed:
pytest.skip(
"installed hermes-agent catalog predates core's 2026-09-09 sync "
"(still lists relay-delisted 'ox-alpha-free'); equality binds "
"once a release carrying that sync is installed"
)
assert [model["id"] for model in _opencode_go_static_models()] == installed
def test_opencode_go_recent_additions_have_human_labels():
labels = {model["id"]: model["label"] for model in _opencode_go_static_models()}
assert labels["gpt-5.6-luna"] == "GPT 5.6 Luna"
assert labels["kimi-k3"] == "Kimi K3"
assert labels["glm-5.3"] == "GLM-5.3"
assert labels["grok-4.5"] == "Grok 4.5"
assert labels["qwen3.8-max"] == "Qwen3.8 Max"
assert labels["muse-spark-1.2-contributor"] == "Muse Spark 1.2 Contributor"
assert labels["muse-spark-1.3-contributor"] == "Muse Spark 1.3 Contributor"
assert labels["glm-5.3-flash"] == "GLM-5.3 Flash"
tests/test_issue5311_opencode_go_static_models.py
-72
View File
@@ -1,72 +0,0 @@
"""#5311 / #5611: OpenCode Go's generic live probe used to return public-catalog
models not enabled on the Go tier, so the picker intentionally falls back to a curated
static _PROVIDER_MODELS list. Keep that curated list in sync with the public Go docs
and documented Go models endpoint while excluding preview/free-only Zen models.
"""
import ast
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
CONFIG_PATH = ROOT / "api" / "config.py"
CONFIG = CONFIG_PATH.read_text(encoding="utf-8")
EXPECTED_OPENCODE_GO_MODEL_IDS = [
"minimax-m3",
"minimax-m2.7",
"minimax-m2.5",
"kimi-k2.7-code",
"kimi-k2.6",
"kimi-k2.5",
"glm-5.2",
"glm-5.1",
"glm-5",
"deepseek-v4-pro",
"deepseek-v4-flash",
"qwen3.7-max",
"qwen3.7-plus",
"qwen3.6-plus",
"qwen3.5-plus",
"mimo-v2-pro",
"mimo-v2-omni",
"mimo-v2.5-pro",
"mimo-v2.5",
]
def _opencode_go_static_models():
tree = ast.parse(CONFIG, filename=str(CONFIG_PATH))
for node in tree.body:
if not isinstance(node, ast.Assign):
continue
if not any(isinstance(target, ast.Name) and target.id == "_PROVIDER_MODELS" for target in node.targets):
continue
provider_models = ast.literal_eval(node.value)
return provider_models["opencode-go"]
raise AssertionError("_PROVIDER_MODELS assignment not found")
def test_opencode_go_skips_live_models_probe():
# The provider-loop must special-case opencode-go to skip the old generic
# live probe and fall through to this curated static list.
body = CONFIG[CONFIG.index("def get_available_models"):]
body = body[: body.index("\ndef ", 1)]
assert 'elif pid == "opencode-go":' in body
idx = body.index('elif pid == "opencode-go":')
branch = body[idx: idx + 400]
assert "_models_from_live_provider_ids" not in branch.split("else:")[0]
def test_opencode_go_static_models_match_documented_endpoint_snapshot():
# Snapshot from https://opencode.ai/zen/go/v1/models on 2026-07-07.
# This catches stale-list regressions such as missing GLM-5.2 / MiniMax M3.
models = _opencode_go_static_models()
assert [model["id"] for model in models] == EXPECTED_OPENCODE_GO_MODEL_IDS
def test_opencode_go_recent_additions_have_human_labels():
labels = {model["id"]: model["label"] for model in _opencode_go_static_models()}
assert labels["glm-5.2"] == "GLM-5.2"
assert labels["minimax-m3"] == "MiniMax M3"
assert labels["kimi-k2.7-code"] == "Kimi K2.7 Code"
assert labels["qwen3.7-max"] == "Qwen3.7 Max"
tests/test_issue5455_lineage_readonly_reads.py
+12 -13
View File
@@ -9,7 +9,6 @@ lineage-metadata reads, and the gateway-watcher fingerprint projection (a 5s
poll), were still opening a read-WRITE connection. This shared them onto the
same ``open_state_db_readonly`` helper.
"""
import logging
import sqlite3
from contextlib import closing
@@ -98,25 +97,25 @@ def test_helper_encodes_special_path_chars(tmp_path, monkeypatch):
assert calls[0]["target"].endswith("?mode=ro")
def test_helper_falls_back_to_writable_and_logs(tmp_path, monkeypatch, caplog):
def test_helper_never_falls_back_to_writable(tmp_path, monkeypatch):
"""A read-only open failure propagates; the helper never retries with a
write-capable handle on the live WAL database."""
db = tmp_path / "state.db"
_make_lineage_db(db)
real_connect = sqlite3.connect
calls = []
def fail_read_only(target, *args, **kwargs):
if kwargs.get("uri"):
calls.append({"target": str(target), "uri": bool(kwargs.get("uri"))})
raise sqlite3.OperationalError("synthetic read-only URI failure")
return real_connect(target, *args, **kwargs)
monkeypatch.setattr(agent_sessions.sqlite3, "connect", fail_read_only)
with caplog.at_level(logging.WARNING, logger="api.agent_sessions"):
conn = open_state_db_readonly(db)
try:
assert conn.execute("SELECT COUNT(*) FROM sessions").fetchone()[0] == 2
finally:
conn.close()
assert "read-only open failed" in caplog.text
assert "synthetic read-only URI failure" in caplog.text
with pytest.raises(sqlite3.OperationalError, match="synthetic read-only URI failure"):
open_state_db_readonly(db)
# Exactly one attempt, and it was the read-only URI form.
assert len(calls) == 1
assert calls[0]["uri"] is True
assert calls[0]["target"].endswith("?mode=ro")
def test_helper_raises_on_missing_db_instead_of_creating_a_ghost(tmp_path):
tests/test_issue5455_listing_readonly_connection.py
+94 -20
View File
@@ -6,13 +6,12 @@ re-run a defensive ``CREATE INDEX`` self-heal on every sidebar build. Holding a
write-capable handle while the agent streams into the same DB adds needless
checkpoint/lock surface.
The listing path now opens the DB read-only (``file:...?mode=ro``) and only
self-heals a missing ``idx_messages_session`` through a SEPARATE short-lived
writable connection. With the index present (the normal case) the read path
performs zero writes; when the index is missing the self-heal still runs and
rows still come back.
The listing path opens the DB read-only (``file:...?mode=ro``) and never
upgrades to a writable handle: a read-only open failure propagates, and a
missing ``idx_messages_session`` degrades to the pre-aggregated path instead
of being self-healed. Index maintenance is an explicit drained operation
(``scripts/ensure_state_db_read_indexes.py``).
"""
import logging
import sqlite3
import api.agent_sessions as agent_sessions
@@ -74,6 +73,77 @@ def test_listing_opens_read_only_and_returns_rows(tmp_path, monkeypatch):
assert "mode=ro" in calls[0]["target"]
def test_listing_keeps_model_config_branch_visible_and_never_leaks_it(tmp_path, monkeypatch):
"""#7021 re-gate: a CLI/agent branch whose _branched_from marker lives in
model_config must stay visible even when it starts inside the 2s
compression tolerance window, and model_config must not leak into the
returned rows."""
db = tmp_path / "state.db"
conn = sqlite3.connect(str(db))
conn.execute(
"""
CREATE TABLE sessions (
id TEXT PRIMARY KEY, title TEXT, model TEXT, message_count INTEGER,
started_at REAL, source TEXT, session_source TEXT,
parent_session_id TEXT, ended_at REAL, end_reason TEXT,
model_config TEXT
)
"""
)
conn.execute(
"""
INSERT INTO sessions
(id, title, model, message_count, started_at, source, session_source,
parent_session_id, ended_at, end_reason, model_config)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
"cli-parent", "Parent conversation", "gpt", 0, 1000.0, "cli", "cli",
None, 1100.0, "compression", None,
),
)
# An explicit Agent branch starting 1.5s BEFORE the parent's compression
# ended_at. No session_source — the fork identity lives in model_config.
conn.execute(
"""
INSERT INTO sessions
(id, title, model, message_count, started_at, source, session_source,
parent_session_id, ended_at, end_reason, model_config)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
"cli-branch", "Branch conversation", "gpt", 0, 1098.5, "cli", "cli",
"cli-parent", None, None, '{"_branched_from": "cli-parent"}',
),
)
conn.execute(
"CREATE TABLE messages (id INTEGER PRIMARY KEY, session_id TEXT, role TEXT, timestamp REAL)"
)
conn.executemany(
"INSERT INTO messages (session_id, role, timestamp) VALUES (?,?,?)",
[
("cli-parent", "user", 1001.0),
("cli-parent", "assistant", 1002.0),
("cli-branch", "user", 1099.0),
("cli-branch", "assistant", 1099.5),
],
)
conn.commit()
conn.close()
_record_connects(monkeypatch)
out = read_importable_agent_session_rows(db, exclude_sources=None)
out_ids = {r["id"] for r in out}
# The branch survives the tolerance window: it is NOT collapsed into the
# parent lineage.
assert "cli-branch" in out_ids
assert "cli-parent" in out_ids
# model_config is internal agent state — never exposed on returned rows.
for row in out:
assert "model_config" not in row
def test_listing_read_only_uri_encodes_special_path_chars(tmp_path, monkeypatch):
db_dir = tmp_path / "state dir #1"
db_dir.mkdir()
@@ -92,24 +162,27 @@ def test_listing_read_only_uri_encodes_special_path_chars(tmp_path, monkeypatch)
assert calls[0]["target"].endswith("?mode=ro")
def test_read_only_open_fallback_is_logged(tmp_path, monkeypatch, caplog):
def test_read_only_open_failure_never_retries_with_writable_connection(tmp_path, monkeypatch):
db = tmp_path / "state.db"
_make_db(db, with_index=True)
real_connect = sqlite3.connect
calls = []
def fail_read_only(target, *args, **kwargs):
if kwargs.get("uri"):
calls.append({"target": str(target), "uri": bool(kwargs.get("uri"))})
raise sqlite3.OperationalError("synthetic read-only URI failure")
return real_connect(target, *args, **kwargs)
monkeypatch.setattr(agent_sessions.sqlite3, "connect", fail_read_only)
with caplog.at_level(logging.WARNING, logger="api.agent_sessions"):
out = read_importable_agent_session_rows(db, exclude_sources=None)
try:
read_importable_agent_session_rows(db, exclude_sources=None)
raise AssertionError("read-only open failure unexpectedly recovered")
except sqlite3.OperationalError as exc:
assert "synthetic read-only URI failure" in str(exc)
assert "cli-1" in {r["id"] for r in out}
assert "read-only open failed" in caplog.text
assert "synthetic read-only URI failure" in caplog.text
# Exactly one attempt, and it was the read-only URI form.
assert len(calls) == 1
assert calls[0]["uri"] is True
assert "mode=ro" in calls[0]["target"]
def test_index_present_performs_no_writable_connection(tmp_path, monkeypatch):
@@ -124,7 +197,7 @@ def test_index_present_performs_no_writable_connection(tmp_path, monkeypatch):
assert all(c["uri"] and "mode=ro" in c["target"] for c in calls), calls
def test_missing_index_self_heals_via_separate_connection(tmp_path, monkeypatch):
def test_missing_index_degrades_without_writable_connection(tmp_path, monkeypatch):
db = tmp_path / "state.db"
_make_db(db, with_index=False)
calls = _record_connects(monkeypatch)
@@ -133,12 +206,13 @@ def test_missing_index_self_heals_via_separate_connection(tmp_path, monkeypatch)
# Rows still come back...
assert "cli-1" in {r["id"] for r in out}
# ...and the self-heal ran through a separate writable (non-ro) connection.
assert any((not c["uri"]) and "mode=ro" not in c["target"] for c in calls), calls
# The index now exists on disk.
# ...without any implicit schema-maintenance writer.
assert calls
assert all(c["uri"] and "mode=ro" in c["target"] for c in calls), calls
# A missing index stays a maintenance concern, not a listing side effect.
verify = sqlite3.connect(str(db))
try:
names = {row[1] for row in verify.execute("PRAGMA index_list(messages)")}
finally:
verify.close()
assert "idx_messages_session" in names
assert "idx_messages_session" not in names
tests/test_issue6869_gateway_launch_failure.py
+217
View File
@@ -0,0 +1,217 @@
"""Regression tests for #6869: failed Gateway worker launch cleanup."""
from types import SimpleNamespace
import api.config as config
import api.routes as routes
from api import turn_journal
def _make_session(session_id, *, active_stream_id=None, pending_user_message=None, save=None):
return SimpleNamespace(
session_id=session_id,
title="Existing",
active_stream_id=active_stream_id,
pending_user_message=pending_user_message,
pending_attachments=[],
pending_started_at=None,
pending_user_source=None,
messages=[{"role": "user", "content": "old"}],
workspace="/tmp",
model="old-model",
model_provider=None,
worktree_path=None,
profile=None,
save=save or (lambda: None),
)
def _register_failed_stream(session_id, stream_id):
"""Mirror the registry state _prepare_chat_start_session_for_stream installs."""
config.register_session_writeback_owner(session_id, stream_id)
config.register_stream_owner(stream_id, session_id)
config.STREAMS[stream_id] = object()
def test_gateway_thread_start_failure_releases_writeback_owner_and_stream_state(monkeypatch):
"""A thread-start exception must not leave one owner per failed Gateway launch."""
class FailingThread:
def __init__(self, *args, **kwargs):
pass
def start(self):
raise RuntimeError("thread launch failed")
canonical_by_id = {}
registered_stream_ids = []
def fake_prepare(session, *, stream_id, **kwargs):
session.active_stream_id = stream_id
session.pending_user_message = kwargs["msg"]
session.pending_started_at = 1.0
_register_failed_stream(session.session_id, stream_id)
canonical_by_id[session.session_id] = session
registered_stream_ids.append(stream_id)
monkeypatch.setattr(routes.threading, "Thread", FailingThread)
monkeypatch.setattr(routes, "_prepare_chat_start_session_for_stream", fake_prepare)
monkeypatch.setattr(routes, "set_last_workspace", lambda _workspace, **_kw: None)
monkeypatch.setattr(routes, "_is_hidden_empty_session", lambda _session: False)
monkeypatch.setattr(
routes,
"get_session",
lambda sid, metadata_only=False: canonical_by_id[sid],
)
monkeypatch.setattr(
turn_journal,
"append_turn_journal_event",
lambda *_args, **_kwargs: {},
)
for index in range(3):
session = _make_session(f"session-launch-failure-{index}")
try:
routes._start_chat_stream_for_session(
session,
msg="start gateway turn",
attachments=[],
workspace="/tmp",
model="test-model",
external_runtime_owned=True,
)
except RuntimeError as exc:
assert str(exc) == "thread launch failed"
else:
raise AssertionError("thread-start failure must propagate")
assert session.active_stream_id is None
assert session.pending_user_message is None
assert session.pending_started_at is None
# Scope these to the streams this test registered. scripts/test.sh runs the
# whole selection in one process, so asserting the global registries are
# empty makes this test fail on unrelated files that leave entries behind.
assert registered_stream_ids, "the launch path must register state before failing"
for stream_id in registered_stream_ids:
assert stream_id not in config.STREAM_SESSION_OWNERS
assert stream_id not in config.STREAMS
for session_id in canonical_by_id:
assert config.session_writeback_owner(session_id) is None
def test_gateway_launch_failure_cleanup_does_not_clear_successor_owner(monkeypatch):
"""A successor fully installed before cleanup keeps its registry ownership."""
canonical = _make_session(
"session-launch-successor",
active_stream_id="new-stream",
pending_user_message="new prompt",
save=lambda: None,
)
monkeypatch.setattr(
routes,
"get_session",
lambda sid, metadata_only=False: canonical,
)
stale = _make_session("session-launch-successor")
_register_failed_stream(stale.session_id, "old-stream")
config.register_session_writeback_owner(stale.session_id, "new-stream")
routes._cleanup_chat_start_launch_failure(stale, "old-stream")
assert config.session_writeback_owner(stale.session_id) == "new-stream"
assert canonical.active_stream_id == "new-stream"
assert canonical.pending_user_message == "new prompt"
assert "old-stream" not in config.STREAMS
def test_gateway_launch_failure_cleanup_does_not_wipe_successor_admitted_during_cleanup(monkeypatch):
"""A successor admitted mid-cleanup survives: the guard runs on the re-resolved
canonical session, not on the stale passed-in object that still shows the
failed stream."""
saved = []
canonical = _make_session(
"session-launch-race",
active_stream_id="new-stream",
pending_user_message="successor prompt",
save=lambda: saved.append("canonical"),
)
monkeypatch.setattr(
routes,
"get_session",
lambda sid, metadata_only=False: canonical,
)
stale = _make_session(
"session-launch-race",
active_stream_id="old-stream",
pending_user_message="failed-stream prompt",
save=lambda: saved.append("stale"),
)
_register_failed_stream(stale.session_id, "old-stream")
config.register_session_writeback_owner(stale.session_id, "new-stream")
routes._cleanup_chat_start_launch_failure(stale, "old-stream")
assert config.session_writeback_owner(stale.session_id) == "new-stream"
assert canonical.active_stream_id == "new-stream"
assert canonical.pending_user_message == "successor prompt"
assert "old-stream" not in config.STREAMS
assert saved == [], "cleanup must not save a session it early-returns on"
def test_gateway_launch_failure_cleanup_does_not_resurrect_deleted_session(monkeypatch):
"""A session deleted while the launch was failing must not be recreated:
the resolver raises KeyError and cleanup returns without saving anything."""
saved = []
def deleted_resolver(sid, metadata_only=False):
raise KeyError(sid)
monkeypatch.setattr(routes, "get_session", deleted_resolver)
stale = _make_session(
"session-launch-deleted",
active_stream_id="old-stream",
pending_user_message="pending prompt",
save=lambda: saved.append("stale"),
)
_register_failed_stream(stale.session_id, "old-stream")
routes._cleanup_chat_start_launch_failure(stale, "old-stream")
assert config.session_writeback_owner(stale.session_id) is None
assert "old-stream" not in config.STREAM_SESSION_OWNERS
assert "old-stream" not in config.STREAMS
assert stale.active_stream_id == "old-stream"
assert stale.pending_user_message == "pending prompt"
assert saved == [], "cleanup must not save a deleted session"
def test_gateway_launch_failure_cleanup_never_masks_the_launch_error(monkeypatch):
"""A resolution failure during cleanup must not replace the launch error.
Cleanup runs inside the launch-failure handler, so an error it fails to
contain is reported instead of the real launch failure. Resolution can fail
with something other than KeyError when the session store is unreadable or
cannot be deserialized.
"""
saved = []
def failing_resolver(sid, metadata_only=False):
raise OSError("session store unreadable")
monkeypatch.setattr(routes, "get_session", failing_resolver)
stale = _make_session(
"session-launch-resolution-error",
active_stream_id="old-stream",
pending_user_message="pending prompt",
save=lambda: saved.append("stale"),
)
_register_failed_stream(stale.session_id, "old-stream")
# Must not raise: the caller is already handling the launch failure.
routes._cleanup_chat_start_launch_failure(stale, "old-stream")
# The registry half still completed, and nothing was saved.
assert config.session_writeback_owner(stale.session_id) is None
assert "old-stream" not in config.STREAM_SESSION_OWNERS
assert "old-stream" not in config.STREAMS
assert saved == []
tests/test_issue7056_login_hides_password_form.py
+223
View File
@@ -0,0 +1,223 @@
"""Regression tests for issue #7056 — login page must hide the password form
when password auth is disabled (e.g. native OIDC configured and
``HERMES_WEBUI_PASSWORD`` unset).
The original bug: ``/api/auth/status`` correctly reports
``password_auth_enabled: false`` when password auth is disabled, but
``/login`` still rendered the password input, the "Sign in" button,
and the "Continue with SSO" entry point — and silently 401-ed every
submit. The fix routes the password input / submit / passkey controls
through a single ``{{PASSWORD_FORM_HTML}}`` placeholder in the
template, which the handler populates with the live markup only when
``is_password_auth_enabled()`` is True.
The OIDC SSO button stays the sole path when password auth is disabled.
"""
from __future__ import annotations
import io
from unittest import mock
import pytest
import api.routes as routes
from api.routes import _LOGIN_PAGE_HTML
# ---------------------------------------------------------------------------
# Source-shape: the template + handler must agree on a single placeholder
# for the password controls, and the handler must call the live predicate
# ---------------------------------------------------------------------------
def test_login_template_uses_password_form_placeholder() -> None:
"""The login template MUST carry a single ``{{PASSWORD_FORM_HTML}}`` placeholder.
A silent reversion to the previous multi-line ``<input>`` +
``<button>`` + ``<button>`` shape fails the suite.
"""
assert "{{PASSWORD_FORM_HTML}}" in _LOGIN_PAGE_HTML, (
"the login template must use a {{PASSWORD_FORM_HTML}} placeholder "
"so the handler can route the password controls through the "
"``is_password_auth_enabled()`` predicate"
)
def test_login_template_does_not_render_password_input_inline() -> None:
"""The login template MUST NOT carry a literal ``<input type="password"``.
The fix replaced the inline input with the placeholder; a silent
reversion that re-adds the inline input fails the suite.
"""
assert '<input type="password"' not in _LOGIN_PAGE_HTML, (
"the login template must not carry an inline <input type=\"password\">; "
"use the {{PASSWORD_FORM_HTML}} placeholder so the handler can "
"gate the password form on is_password_auth_enabled()"
)
def test_login_handler_calls_is_password_auth_enabled() -> None:
"""The /login branch in handle_get MUST call ``is_password_auth_enabled``.
Pins the contract that the live predicate — not a stale cached
flag, not a config-only check — gates the password controls. A
silent reversion to a hard-coded `True` (or any pre-baked
assumption about the environment) fails the suite.
"""
src = (routes.__file__).read_text(encoding="utf-8") if hasattr(routes.__file__, "read_text") else open(routes.__file__).read()
# Find the /login branch
login_idx = src.index('if parsed.path == "/login":')
# Find the next return statement that closes the /login branch
end_idx = src.index("content_type=\"text/html; charset=utf-8\")", login_idx)
branch = src[login_idx:end_idx]
assert "is_password_auth_enabled" in branch, (
"the /login branch must call is_password_auth_enabled() so the "
"password form visibility tracks /api/auth/status at request time"
)
assert "PASSWORD_FORM_HTML" in branch, (
"the /login branch must populate the {{PASSWORD_FORM_HTML}} "
"placeholder with the live password form markup"
)
# ---------------------------------------------------------------------------
# Behavioural: walk the /login handler with a real ``ParsedResult`` and
# a minimal fake handler. We mock the side-effecting helpers (settings
# + OIDC + auth predicate) so the test only covers the placeholder
# logic, not the whole login stack.
# ---------------------------------------------------------------------------
class _FakeHandler:
def __init__(self) -> None:
self.status: int | None = None
self.headers: dict[str, str] = {}
self.wfile = io.BytesIO()
self.command = "GET"
self.path = "/login"
self.client_address = ("127.0.0.1", 12345)
self.request_version = "HTTP/1.1"
def send_response(self, status):
self.status = status
def send_header(self, key, value):
self.headers[key] = value
def end_headers(self):
pass
def _stub_oidc(enabled: bool):
"""Mock the OIDC predicate the login branch uses to decide SSO button visibility.
``_oidc_login_html(parsed)`` is a module-level function the login
branch calls; the mock has to accept a single positional ``parsed``
argument and return a stable string. With OIDC enabled the
fixture returns a marker substring (``oidc-login``) so the
behavioural tests can assert its presence; with OIDC disabled it
returns the empty string the real implementation returns.
"""
def _stub(parsed):
del parsed
if enabled:
return (
'<a id="oidc-login" class="oidc-login" '
'href="/api/auth/oidc/start">Continue with SSO</a>'
)
return ""
return mock.patch.object(routes, "_oidc_login_html", _stub)
@pytest.fixture
def patched_login(monkeypatch):
"""Set up the login branch's external dependencies for behavioural tests.
Yields a (routes, parsed) tuple. Each test patches
``is_password_auth_enabled`` and ``_oidc_login_html`` itself so the
scenario is explicit.
"""
monkeypatch.setattr(
routes, "_LOGIN_LOCALE",
{
"en": {
"lang": "en",
"title": "Sign in",
"subtitle": "Sign in to continue",
"placeholder": "Password",
"btn": "Sign in",
"invalid_pw": "Invalid password",
"conn_failed": "Connection failed",
}
},
)
monkeypatch.setattr(routes, "_resolve_login_locale_key", lambda _lang: "en")
monkeypatch.setattr(routes, "load_settings", lambda: {"bot_name": "Hermes"})
from api import updates as _updates
monkeypatch.setattr(_updates, "WEBUI_VERSION", "test-0.0.0")
import html as _html
monkeypatch.setattr(routes, "_html", _html)
from urllib.parse import urlparse
yield routes, urlparse("/login?next=/x")
def test_login_renders_password_form_when_password_auth_enabled(patched_login, monkeypatch) -> None:
routes, parsed = patched_login
monkeypatch.setattr(
"api.auth.is_password_auth_enabled", lambda: True, raising=False
)
with _stub_oidc(enabled=False):
handler = _FakeHandler()
routes.handle_get(handler, parsed)
html = handler.wfile.getvalue().decode("utf-8", "replace")
assert '<input type="password"' in html, html
assert 'id="pw"' in html, html
assert '<button type="submit"' in html, html
assert 'id="passkey-login"' in html, html
def test_login_hides_password_form_when_password_auth_disabled(patched_login, monkeypatch) -> None:
"""With password auth disabled (e.g. OIDC-only), the form must NOT render
the password input / submit / passkey controls. The OIDC SSO entry
point (when OIDC is enabled) stays the sole path.
"""
routes, parsed = patched_login
monkeypatch.setattr(
"api.auth.is_password_auth_enabled", lambda: False, raising=False
)
with _stub_oidc(enabled=True):
handler = _FakeHandler()
routes.handle_get(handler, parsed)
html = handler.wfile.getvalue().decode("utf-8", "replace")
# The password controls are gone.
assert '<input type="password"' not in html, html
assert 'id="pw"' not in html, html
assert '<button type="submit"' not in html, html
assert 'id="passkey-login"' not in html, html
# The OIDC SSO entry point is still present (the user must still
# be able to log in — just via SSO, not via password). We assert
# the <a id="oidc-login"> element rather than the bare substring
# because the template's CSS class ``.oidc-login`` would match
# the page even when OIDC is disabled.
assert 'id="oidc-login"' in html, html
def test_login_hides_oidc_button_when_oidc_disabled(patched_login, monkeypatch) -> None:
"""Sanity: with password auth disabled AND OIDC disabled, the page is
effectively empty — no controls. This pre-existing behaviour is
unchanged by #7056 (the fix only adds the password-disabled path).
"""
routes, parsed = patched_login
monkeypatch.setattr(
"api.auth.is_password_auth_enabled", lambda: False, raising=False
)
with _stub_oidc(enabled=False):
handler = _FakeHandler()
routes.handle_get(handler, parsed)
html = handler.wfile.getvalue().decode("utf-8", "replace")
assert '<input type="password"' not in html, html
assert 'id="oidc-login"' not in html, html
tests/test_issue7290_no_slash_prefixed_alias.py
+259
View File
@@ -0,0 +1,259 @@
"""Regression tests for issue #7290 — configured-model badge must not leak a
``{provider}/{model}``-prefixed alias when the native model id itself
contains a slash.
When the active provider is a built-in plugin provider (e.g. ``commandcode``)
and the configured default model id contains a slash (e.g.
``deepseek/deepseek-v4-flash``), the configured-model badge builder used
to emit three key variants for the same configured identity:
- the bare native id (``deepseek/deepseek-v4-flash``)
- a slash-prefixed alias (``commandcode/deepseek/deepseek-v4-flash``)
- the ``@provider:model``-prefixed form (``@commandcode:deepseek/deepseek-v4-flash``)
The slash-prefixed alias was a synthetic key the catalog never actually
exposed — the WebUI dropdown rendered it as a real option, selecting it
sent ``model=commandcode/deepseek/deepseek-v4-flash`` to the agent, and
the provider rejected it with ``HTTP 400: Model ... is not supported on
this endpoint``. The session also persisted the wrong id, so reload
replayed the failure.
The fix removes the ``{provider}/{model}`` synthetic alias. Bare native
id (which is what the catalog actually exposes) and ``@provider:model``
(which is the documented prefix form for named custom providers) remain.
Multi-slash, URI-scheme, host:port, and ``@provider:model``-prefixed
model ids all keep working — only the synthesised slash-prefixed alias
disappears.
The pre-fix test below pins both failure modes reported in the issue:
1. the badge map MUST NOT contain the slash-prefixed alias
2. selecting the bare native id from the dropdown MUST NOT produce a
request whose ``model`` field is the slash-prefixed alias
"""
from __future__ import annotations
import re
import textwrap
from pathlib import Path
import pytest
REPO_ROOT = Path(__file__).resolve().parents[1]
def _build_badges(active_provider, default_model, groups, fallback_providers=None):
"""Inline-exec the live ``_build_configured_model_badges`` and return its result.
``_norm_model_id`` is inlined as a stand-in (its full body is part of
the production source and is exercised by ``test_norm_model_id_*``
siblings). Mirrors the existing pattern in
``test_duplicate_slash_id_primary_badge_sticks_to_matching_provider_only``.
"""
src = (REPO_ROOT / "api" / "config.py").read_text(encoding="utf-8")
start = src.index("def _build_configured_model_badges() -> dict[str, dict[str, str]]:")
end = src.index(" return badges", start) + len(" return badges")
fn_src = textwrap.dedent(src[start:end])
scope = {
"active_provider": active_provider,
"default_model": default_model,
"cfg": {"fallback_providers": fallback_providers or []},
"groups": groups,
"_resolve_provider_alias": lambda provider: provider,
}
# The live source uses a different `_norm_model_id` than the test's
# inline; either is fine because the badge key is the **raw** candidate,
# not the normalized key, so the raw_candidates list is the surface this
# test pins.
exec(
"def _norm_model_id(model_id):\n"
" s=str(model_id or '').strip().lower()\n"
" if s.startswith('@') and ':' in s: s=s.split(':',1)[1]\n"
" if '/' in s: s=s.split('/',1)[1]\n"
" return s.replace('-', '.')\n",
scope,
)
exec(fn_src, scope)
return scope["_build_configured_model_badges"]()
# ---------------------------------------------------------------------------
# Repro: commandcode + deepseek/deepseek-v4-flash — the literal bug report
# ---------------------------------------------------------------------------
def test_commandcode_deepseek_does_not_emit_slash_prefixed_alias() -> None:
"""The badge map MUST NOT contain ``commandcode/deepseek/deepseek-v4-flash``."""
badges = _build_badges(
active_provider="commandcode",
default_model="deepseek/deepseek-v4-flash",
groups=[
# The catalog exposes the bare native id under the commandcode
# provider — that's the only selectable form.
{
"provider": "CommandCode",
"provider_id": "commandcode",
"models": [{"id": "deepseek/deepseek-v4-flash"}],
}
],
)
# The slash-prefixed alias MUST NOT appear in the badge map.
assert "commandcode/deepseek/deepseek-v4-flash" not in badges, (
f"slash-prefixed alias leaked into the badge map: {badges!r}"
)
# The bare native id SHOULD be the primary badge (it is the only
# catalog-exposed form and is the correct value to send to the agent).
assert badges.get("deepseek/deepseek-v4-flash", {}).get("role") == "primary"
def test_commandcode_deepseek_at_prefixed_alias_still_present() -> None:
"""The ``@provider:model`` prefix form MUST still resolve to the primary badge."""
badges = _build_badges(
active_provider="commandcode",
default_model="deepseek/deepseek-v4-flash",
groups=[
{
"provider": "CommandCode",
"provider_id": "commandcode",
"models": [{"id": "deepseek/deepseek-v4-flash"}],
}
],
)
assert badges.get("@commandcode:deepseek/deepseek-v4-flash", {}).get("role") == "primary", (
f"@commandcode:deepseek/deepseek-v4-flash alias missing from badges: {badges!r}"
)
# ---------------------------------------------------------------------------
# Dedupe across providers: a slash-bearing id shared by two providers must
# keep the badge only on the matching provider's row, never the other way.
# ---------------------------------------------------------------------------
def test_duplicate_slash_id_does_not_pollute_other_provider() -> None:
"""Per #7290 contract: deduplicate only when provider+full native model match."""
badges = _build_badges(
active_provider="custom:alpha",
default_model="vendor/abc-model",
groups=[
{
"provider": "Alpha",
"provider_id": "custom:alpha",
"models": [{"id": "vendor/abc-model"}],
},
{
"provider": "Beta",
"provider_id": "custom:beta",
"models": [{"id": "vendor/abc-model"}],
},
],
)
# The bare native id is a real option for BOTH providers; only the
# matching provider's row gets the PRIMARY badge.
payload = badges.get("vendor/abc-model", {})
assert payload.get("role") == "primary"
assert payload.get("provider") == "custom:alpha"
# The OTHER provider's @-prefixed form is not produced (entry is for alpha
# only); but the alpha @-prefixed form IS produced.
assert "vendor/abc-model" in badges
# No synthetic slash-prefixed alias for the other provider.
assert "custom:beta/vendor/abc-model" not in badges
# ---------------------------------------------------------------------------
# Sanity: other forms of model id (URI scheme, host:port, @provider:model)
# must keep working — only the synthetic slash-prefixed alias is removed.
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"model_id",
[
"deepseek-v4-pro", # bare id (no slash) — the common case
"@anthropic:claude-opus-4-7", # @-prefixed named custom provider
"gpt://folder/abc/latest", # URI-scheme id (#3429)
"https://proxy.internal/models/gpt4", # URL id (#3429)
],
)
def test_non_slash_models_unchanged(model_id: str) -> None:
"""No regression: the bare / @-prefixed / URI-scheme id forms are unchanged."""
badges = _build_badges(
active_provider="commandcode",
default_model=model_id,
groups=[
{
"provider": "CommandCode",
"provider_id": "commandcode",
"models": [{"id": model_id}],
}
],
)
# The bare native id (or its @-prefixed alias) is the primary badge.
assert badges.get(model_id, {}).get("role") == "primary", (
f"bare id missing primary for {model_id!r}: {badges!r}"
)
# No synthesised slash-prefixed alias for ANY form.
slash_aliases = [k for k in badges if k.startswith("commandcode/") and k != model_id]
assert not slash_aliases, (
f"synthesised slash-prefixed aliases leaked: {slash_aliases!r} in {badges!r}"
)
# ---------------------------------------------------------------------------
# Source-shape: the ``{provider}/{model}`` synthetic alias must not appear
# in either configured-badge builder. A silent reversion to the bug
# leaves this regex matchable and the test fails.
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"needle",
[
# _build_configured_model_badges (line ~8618) — 12-space indent
'f"{provider}/{model}"',
# _configured_model_badges_from_static_catalog (line ~6943) — 8-space
'f"{provider}/{model}"',
],
)
def test_synthetic_slash_prefix_alias_is_not_in_source(needle: str) -> None:
"""The synthetic ``{provider}/{model}`` alias MUST NOT appear in either builder."""
src = (REPO_ROOT / "api" / "config.py").read_text(encoding="utf-8")
assert needle not in src, (
f"the synthetic slash-prefixed alias leaked back into the source: {needle!r}"
)
def test_raw_candidates_only_holds_bare_and_at_prefix() -> None:
"""The raw_candidates tuple MUST contain only the bare native id and the ``@provider:`` alias.
Catches both the old bug and a future regression that adds back
*any* synthetic prefix form (e.g. ``{provider}::{model}``).
"""
src = (REPO_ROOT / "api" / "config.py").read_text(encoding="utf-8")
# Find every `for candidate in (` block and verify the contents.
# The 12-space variant is the main builder; the 8-space variant is the
# static-catalog fallback; both must contain only bare + @-prefix.
pattern = re.compile(
r"for candidate in \(([\s\S]*?)\):\s*\n\s+if candidate and candidate not in raw_candidates:",
)
matches = list(pattern.finditer(src))
assert matches, "raw_candidates tuple not found in api/config.py"
for m in matches:
body = m.group(1)
# The body may be a single line `model, f"@{provider}:{model}"` or
# a multi-line stack; both are accepted. We only need to verify the
# two safe entries are present and the synthesised one is gone.
bare = re.search(r"\bmodel\b", body)
at_prefixed = re.search(
r'f["\']@\{provider\}:\{model\}["\']', body
)
assert bare, f"raw_candidates missing the bare `model` entry: {body!r}"
assert at_prefixed, (
f"raw_candidates missing the @-prefixed entry, or contains a "
f"synthesised prefix form: {body!r}"
)
# No synthesised prefix form like `{provider}/{model}` survives.
assert "{provider}/{model}" not in body, (
f"raw_candidates still emits a synthesised slash-prefixed alias: {body!r}"
)
tests/test_issue7299_hidden_poll_stops_on_404.py
+152
View File
@@ -0,0 +1,152 @@
"""Regression tests for issue #7299 — hidden-tab active-stream poll
keeps retrying deleted sessions forever, generating an invisible stream
of permanent 404 requests from stale background tabs.
When a tab is hidden, Hermes WebUI replaces the persistent per-session
SSE connection with a lightweight ``GET /api/session/status`` poll
every six seconds. If the tracked session has been deleted or no
longer exists in the active state directory, the endpoint returns
``404``, but the client previously converted every non-2xx response to
``null`` and kept the interval alive forever — one tab generates up to
ten requests per minute whenever browser timer throttling allows it.
The fix: in ``_startHiddenActiveStreamPoll``'s fetch chain, treat
``404 Not Found`` and ``410 Gone`` as **terminal** for the
session-owned poll — call ``_stopHiddenActiveStreamPoll()`` so the
interval and the bound ``_sessionStreamHiddenPollSid`` are cleared
together. Transient failures (5xx, rate-limit, network error) keep
polling because they do not necessarily mean the session is gone; a
later tick may catch a server-initiated turn starting while the tab
remains hidden.
"""
from __future__ import annotations
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
MESSAGES_JS = (REPO_ROOT / "static" / "messages.js").read_text(encoding="utf-8")
# ---------------------------------------------------------------------------
# Source-shape: the fetch chain in ``_startHiddenActiveStreamPoll`` must
# gate the terminal 404/410 path through ``_stopHiddenActiveStreamPoll``
# and the bound sid check. A silent reversion to the pre-fix
# "convert all non-2xx to null and keep polling" form fails the suite.
# ---------------------------------------------------------------------------
def test_hidden_poll_stops_on_404_response() -> None:
"""The fetch chain must call ``_stopHiddenActiveStreamPoll`` on a 404.
A 404 from ``/api/session/status`` means the session has been
deleted or no longer exists in the active state directory. The
pre-fix code converted every non-2xx to ``null`` and never
stopped the interval — so the same 404 fired every six seconds
for the lifetime of the hidden tab. Pin the contract that a 404
now tears down the poll.
"""
# The check is anchored on the hidden-poll function so a future
# refactor cannot satisfy it with an unrelated status-404
# handler in a different code path.
start = MESSAGES_JS.find("function _startHiddenActiveStreamPoll(sid)")
assert start != -1
# 3.6 KB is enough to cover the fetch → first .then → second .then
# block, even after the #7299 fix added the 404/410 terminal
# branch. The hidden poll is one of the larger functions in the
# file, so we take a wide window to be resilient to a small
# reformat.
body = MESSAGES_JS[start:start + 8000]
assert "r.status === 404" in body, (
"the hidden poll must check r.status === 404 in the fetch "
"response so a deleted session no longer triggers an "
"infinite 404 loop on a stale background tab"
)
assert "_stopHiddenActiveStreamPoll" in body, (
"the hidden poll must call _stopHiddenActiveStreamPoll when "
"the response is terminal (404/410) so the interval is "
"cleared instead of left running"
)
def test_hidden_poll_stops_on_410_response() -> None:
"""The fetch chain must also call ``_stopHiddenActiveStreamPoll`` on 410.
Some session-state endpoints use ``410 Gone`` (rather than
``404 Not Found``) for "this session id is permanently gone".
Both are terminal for the hidden poll — the pre-fix code treated
them identically, so the fix must handle both.
"""
start = MESSAGES_JS.find("function _startHiddenActiveStreamPoll(sid)")
body = MESSAGES_JS[start:start + 8000]
assert "r.status === 410" in body, (
"the hidden poll must check r.status === 410 in the fetch "
"response so a 'gone' session also tears down the poll, "
"matching the 404 contract"
)
def test_hidden_poll_404_stop_guards_on_bound_sid() -> None:
"""The 404/410 stop path must check the bound ``_sessionStreamHiddenPollSid``.
The same fetch chain is invoked on every tick of a 6-second
interval. A race can let a stale fetch's response land after
the user switched sessions — in that case ``_sessionStreamHiddenPollSid``
no longer matches the fetched ``sid``, and the stop must NOT
tear down the *new* session's poll. Pin the contract.
"""
start = MESSAGES_JS.find("function _startHiddenActiveStreamPoll(sid)")
body = MESSAGES_JS[start:start + 8000]
# The 404/410 branch must check the bound sid before stopping
# — pinned as a string match because the JS module is large
# and we want a regression to show up at a glance in CI.
assert "_sessionStreamHiddenPollSid === sid" in body, (
"the 404/410 stop path must guard on "
"_sessionStreamHiddenPollSid === sid so a stale fetch's "
"response cannot tear down a different session's poll"
)
def test_hidden_poll_preserves_transient_failure_retry() -> None:
"""5xx / rate-limit / network errors MUST keep polling (not stop).
The fix only narrows the stop path to terminal 404/410. Transient
failures (5xx, 429, network error) still produce ``null`` and
the next tick re-fires. A regression that broadens the stop
path to all non-2xx would re-introduce the original bug in
the opposite direction — the poll would die on a single
503 and never recover.
"""
start = MESSAGES_JS.find("function _startHiddenActiveStreamPoll(sid)")
body = MESSAGES_JS[start:start + 8000]
# The pre-fix ``r.ok ? r.json() : null`` fallback must still be
# present for any non-ok / non-terminal response.
assert "r.ok ? r.json() : null" in body, (
"the 5xx / rate-limit / network-error path must still "
"produce null and let the next tick re-fire; only 404/410 "
"should tear down the poll"
)
def test_hidden_poll_starts_interval_unchanged() -> None:
"""The interval is unchanged (still 6000 ms).
The fix is purely about the response handling, not the
cadence. A regression that doubles the interval (or removes
the immediate-fire) breaks the original contract — a turn
already running when the tab goes hidden must be caught
without waiting a full interval.
"""
start = MESSAGES_JS.find("function _startHiddenActiveStreamPoll(sid)")
body = MESSAGES_JS[start:start + 8000]
assert "setInterval(tick, 6000)" in body, (
"the hidden poll must keep the 6000ms interval; the fix is "
"in the response handling, not the cadence"
)
# The immediate-fire call (no full interval wait) must also
# still be present so a turn already running when the tab goes
# hidden is caught without waiting a full interval.
assert "tick();" in body, (
"the immediate-fire tick() call must be preserved so a "
"turn already running when the tab goes hidden is caught "
"without waiting a full interval"
)
tests/test_issue7305_profiles_fallback.py
+236
View File
@@ -0,0 +1,236 @@
"""#7305: GET /api/profiles must never 500 when the hermes-agent source is not
importable (two-container Docker / HERMES_WEBUI_CHAT_BACKEND=gateway).
Contract pinned here: when ``agent.skill_utils`` is missing, the WebUI reports
the skill stats as *unknown* — a stable ``(0, 0)`` — instead of counting skills
with a local partial re-implementation of the index walk. The profile picker
still renders, and it never claims a count it cannot verify (the UI omits the
skills line when the total is 0).
The fixture tree deliberately reproduces the shapes that a partial local scan
gets wrong: a SKILL.md whose declared ``name`` differs from its directory, two
directories declaring the same name, a declared name disabled in the profile
config, a platform-incompatible skill, and the paths the real index walk prunes
(support dirs of a skill package, dependency/VCS/metadata dirs, an inactive
org-mirror). ``test_agent_backed_path_counts_these_shapes_accurately`` is the
positive control: it runs the same tree through the real agent-backed path, so
the fixture cannot silently degrade into "a tree that never had a count to
lose".
"""
from __future__ import annotations
import builtins
import pathlib
import pytest
import api.profiles as profiles
# How a fixture entry relates to the reported counts.
COUNTED = "counted" # yields a logical skill; contributes to compatible/enabled
WALK_PRUNED = "walk-pruned" # never yielded by agent.skill_utils' index walk
PLATFORM_FILTERED = "platform-filtered" # walked, but incompatible with this host
# path -> (declared frontmatter name, platform restriction or None, outcome)
SKILLS_TREE = {
# Declared name differs from the directory name; that declared name is
# disabled for the webui platform in config.yaml below.
"alpha-dir/SKILL.md": ("my-skill", None, COUNTED),
# A second directory declaring the same name — one logical skill.
"beta-dir/SKILL.md": ("my-skill", None, COUNTED),
# Platform-incompatible: must not be offered (or counted) on this host.
"gamma-dir/SKILL.md": ("gamma-skill", "some-other-platform", PLATFORM_FILTERED),
# A plain, enabled skill.
"delta-dir/SKILL.md": ("delta-skill", None, COUNTED),
# Support dir of a skill package: disclosure-only, not a standalone skill.
"delta-dir/references/notes/SKILL.md": ("support-note", None, WALK_PRUNED),
# Dependency / virtualenv / VCS / metadata dirs are excluded wholesale, and
# an org mirror without its .active_org marker is token-gated shut.
"node_modules/dep/SKILL.md": ("vendored-skill", None, WALK_PRUNED),
".venv/lib/pkg/SKILL.md": ("venv-skill", None, WALK_PRUNED),
".git/hooks/SKILL.md": ("hook-skill", None, WALK_PRUNED),
"_org/other-org/org-skill/SKILL.md": ("org-skill", None, WALK_PRUNED),
}
# The one declared name the profile config disables for the webui platform.
DISABLED_NAMES = {"my-skill"}
CONFIG_YAML = (
"profile:\n"
" name: default\n"
"skills:\n"
" platform_disabled:\n"
" webui:\n"
" - my-skill\n"
)
CONFIG_WITHOUT_DISABLE = "profile:\n name: default\n"
def _expected_walked() -> set[str]:
return {
rel
for rel, (_, _, outcome) in SKILLS_TREE.items()
if outcome != WALK_PRUNED
}
@pytest.fixture()
def agent_absent(monkeypatch):
"""Simulate the agent-less deployment: importing anything under ``agent.``
raises ``ImportError``, even when the source tree is present locally."""
real_import = builtins.__import__
def _no_agent(name, *args, **kwargs):
if name == "agent" or str(name).startswith("agent."):
raise ImportError(f"{name} not mounted (simulated two-container Docker)")
return real_import(name, *args, **kwargs)
monkeypatch.setattr(builtins, "__import__", _no_agent)
@pytest.fixture()
def hermes_cli_absent(monkeypatch):
"""Simulate the same deployment's other half: no ``hermes_cli`` either."""
real_import = builtins.__import__
def _no_cli(name, *args, **kwargs):
if name == "hermes_cli" or str(name).startswith("hermes_cli."):
raise ImportError(f"{name} not mounted (simulated two-container Docker)")
return real_import(name, *args, **kwargs)
monkeypatch.setattr(builtins, "__import__", _no_cli)
def _write_profile(tmp_path: pathlib.Path, dir_name: str = "home") -> pathlib.Path:
"""Materialise SKILLS_TREE + config.yaml as a profile home."""
home = tmp_path / dir_name
skills = home / "skills"
for rel, (name, platform, _outcome) in SKILLS_TREE.items():
skill_md = skills / rel
skill_md.parent.mkdir(parents=True, exist_ok=True)
lines = ["---", f"name: {name}", "description: issue 7305 fixture"]
if platform:
lines += ["platforms:", f" - {platform}"]
lines += ["---", "body", ""]
skill_md.write_text("\n".join(lines), encoding="utf-8")
(home / "config.yaml").write_text(CONFIG_YAML, encoding="utf-8")
return home
def test_skills_stats_are_unknown_without_agent_skill_utils(tmp_path, agent_absent):
"""Without agent.skill_utils the stats are (0, 0) — never a guessed count,
no matter which fixtures a partial local scan would have counted."""
home = _write_profile(tmp_path)
assert profiles._get_profile_skills_stats(home) == (0, 0)
def test_default_profile_dict_never_raises_without_agent(tmp_path, agent_absent, monkeypatch):
"""_default_profile_dict() (the fallback row for GET /api/profiles) must
return a usable row with unknown skill stats instead of raising."""
home = _write_profile(tmp_path)
monkeypatch.setattr(profiles, "_DEFAULT_HERMES_HOME", home)
row = profiles._default_profile_dict()
assert row["name"] == "default"
assert row["path"] == str(home)
assert row["is_default"] is True
assert row["skill_count"] == 0
assert row["enabled_skills"] == 0
assert row["total_skills"] == 0
def test_list_profiles_api_isolated_mode_without_agent_or_cli(
tmp_path, agent_absent, hermes_cli_absent, monkeypatch
):
"""The #7305 path end to end: isolated profile mode with neither
agent.skill_utils nor hermes_cli importable must still return the
default-only row instead of propagating ImportError out of the route."""
home = _write_profile(tmp_path, dir_name="default")
monkeypatch.setattr(profiles, "_is_isolated_profile_mode", lambda: True)
monkeypatch.setattr(profiles, "_INITIAL_HERMES_HOME", str(home))
rows = profiles.list_profiles_api()
assert [r["name"] for r in rows] == ["default"]
assert rows[0]["is_active"] is True
assert rows[0]["skill_count"] == 0
assert rows[0]["enabled_skills"] == 0
assert rows[0]["total_skills"] == 0
def test_agent_backed_path_counts_these_shapes_accurately(tmp_path):
"""Positive control (agent source present): the fixture above is a real
semantic trap, not a tree with nothing to count.
The fixture's expectations are derived from the paths the agent's index
walk actually yields, because that walk (and its exclusion list) belongs to
the agent package and differs between deployments. What this asserts:
- the compatible total stays strictly below the walked file count — the
support/dependency/VCS/org-mirror files a naive local ``os.walk`` picks
up are either pruned by the walk or, when the walk keeps them, still not
enough to reach the walk's file count;
- the platform-incompatible and duplicate-declared-name files do not add to
the compatible total;
- identity comes from frontmatter ``name``: the disabled ``my-skill`` (a
declared name, not a directory) is compatible but not enabled;
- dropping that config disable flips it to enabled without changing the
compatible total.
"""
skill_utils = pytest.importorskip("agent.skill_utils")
# Other test files install a bare ``types.ModuleType`` stub for
# ``agent.skill_utils`` in ``sys.modules`` (no ``__file__``, rglob walk,
# MagicMock frontmatter). This positive control is only meaningful against
# the real agent package, so a stub must skip it, not satisfy it.
if not getattr(skill_utils, "__file__", None):
pytest.skip("agent.skill_utils is a test stub, not the real agent package")
from agent.skill_utils import iter_skill_index_files
home = _write_profile(tmp_path)
skills = home / "skills"
walked = {
p.relative_to(skills).as_posix() for p in iter_skill_index_files(skills, "SKILL.md")
}
on_disk = {p.relative_to(skills).as_posix() for p in skills.rglob("SKILL.md")}
# The index walk itself belongs to the agent package and its exclusion list
# is version-dependent, so this test does not freeze it: every expectation
# below is derived from the paths the walk actually yielded. (Removing the
# local mirror of that walk is the point of #7305 — the fallback reported
# (0, 0) instead of guessing.)
assert _expected_walked() <= walked, "the walk must keep the non-pruned skills"
assert walked <= on_disk
expected_compatible = {
name
for rel, (name, platform, _outcome) in SKILLS_TREE.items()
if rel in walked and platform is None
}
expected_enabled = expected_compatible - DISABLED_NAMES
# The fixture is a real semantic trap: the compatible total must stay below
# the walked file count. If the walk pruned the support/dependency/VCS/org
# files the count drops because those files are gone; if it did not, the
# count still drops because the platform-incompatible file is filtered out
# and the two directories declaring `my-skill` collapse to one logical
# skill.
assert len(expected_compatible) < len(walked), (
"fixture must make the compatible total fall below the walked file count"
)
enabled, compatible = profiles._compute_profile_skills_stats(home)
assert compatible == len(expected_compatible), (compatible, expected_compatible)
assert compatible < len(on_disk)
assert enabled == len(expected_enabled)
(home / "config.yaml").write_text(CONFIG_WITHOUT_DISABLE, encoding="utf-8")
enabled_all, compatible_all = profiles._compute_profile_skills_stats(home)
assert compatible_all == compatible
assert enabled_all == len(expected_compatible)
tests/test_issue7359_media_backtick_strip.py
+313
View File
@@ -0,0 +1,313 @@
"""Regression tests for issue #7359 — MEDIA: token regex must not capture trailing backtick.
Two-pass scan (#7680 re-gate, 9/22):
- `` `MEDIA:path` `` (backtick-wrapped, inline-code form): the wrapping
backticks are stripped by a pre-pass before the bare class scans the
text. This makes the closing backtick disappear as a wrapper instead
of being captured as part of the path.
- ``MEDIA:/path...`` (bare, no surrounding backticks): the capture class
no longer lists the backtick in the exclusion set, so a filename
that legally contains a backtick (``report`final.png``) is captured
in full instead of being truncated at the first backtick.
The pre-fix (round-1) PR added the backtick to the bare exclusion set,
which traded one bug for another: it dropped the closing wrapper on
the inline-code form but truncated any bare path that happened to
contain a backtick in the filename. The two-pass scan keeps the
original permissive bare class and adds an explicit pre-pass for the
wrapped form, so both cases round-trip correctly.
"""
import re
import subprocess
from pathlib import Path
import pytest
REPO_ROOT = Path(__file__).resolve().parents[1]
# Source-shape: each site's expected bare-token capture literal. The
# bare class no longer lists the backtick in the exclusion set; a
# backtick inside the filename (legal on POSIX and Windows) is
# captured in full. The wrapped-form pre-pass below strips the
# inline-code backticks before the bare scan.
JS_SITES: dict[str, list[str]] = {
"static/ui.js": [
r"/MEDIA:([^\s\)\]]+)/g", # linkifier
r"/MEDIA:[^\s]+/g", # TTS flattener (no paren/bracket exclusion)
],
"static/messages.js": [
r"/^MEDIA:([^\s\)\]]+)$/", # streaming chunk finalize
r"/MEDIA:([^\s\)\]]+)/g", # streaming combined walk
r"/MEDIA:[^\s\)\]]*$/", # streaming partial tail
],
}
PY_SITES: dict[str, str] = {
"api/routes.py": r"MEDIA:([^\s\)\]]+)",
"api/media_snapshots.py": r"MEDIA:([^\s\)\]]+)",
}
# #7680 re-gate (9/22): every site that scans bare MEDIA: tokens also
# runs the wrapped-form pre-pass (`` `MEDIA:path` `` → ``MEDIA:path``)
# so the closing backtick of an inline-code form is consumed before the
# bare scan.
JS_BACKTICK_PREPASS: dict[str, str] = {
"static/ui.js": r"/`MEDIA:([^`\s]+)`/g",
"static/messages.js": r"/`MEDIA:([^`\s]+)`/g",
}
PY_BACKTICK_PREPASS: dict[str, str] = {
"api/routes.py": r"`MEDIA:([^`\s]+)`",
"api/media_snapshots.py": r"`MEDIA:([^`\s]+)`",
}
def _read(rel: str) -> str:
return (REPO_ROOT / rel).read_text(encoding="utf-8")
# ---------------------------------------------------------------------------
# Source-shape: every site must carry the bare-token class + the pre-pass
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
("relpath", "needle"),
[
(relpath, needle)
for relpath, needles in JS_SITES.items()
for needle in needles
],
)
def test_js_site_carries_bare_token_class(relpath: str, needle: str) -> None:
"""Every JS capture site carries the bare-token class literal."""
src = _read(relpath)
assert needle in src, (
f"{relpath} is missing the bare-token capture literal: {needle!r}"
)
@pytest.mark.parametrize(
("relpath", "needle"),
[
(relpath, JS_BACKTICK_PREPASS[relpath])
for relpath in JS_BACKTICK_PREPASS
],
)
def test_js_site_carries_wrapped_prepass(relpath: str, needle: str) -> None:
"""#7680 re-gate: every JS site that scans bare MEDIA: tokens also
runs the wrapped-form pre-pass so the inline-code closing backtick
is eaten before the bare scan."""
src = _read(relpath)
assert needle in src, (
f"{relpath} is missing the wrapped-form pre-pass literal: {needle!r}"
)
@pytest.mark.parametrize(
("relpath", "needle"),
[(relpath, PY_SITES[relpath]) for relpath in PY_SITES],
)
def test_py_site_carries_bare_token_class(relpath: str, needle: str) -> None:
src = _read(relpath)
assert needle in src, (
f"{relpath} is missing the bare-token capture literal: {needle!r}"
)
@pytest.mark.parametrize(
("relpath", "needle"),
[(relpath, PY_BACKTICK_PREPASS[relpath]) for relpath in PY_BACKTICK_PREPASS],
)
def test_py_site_carries_wrapped_prepass(relpath: str, needle: str) -> None:
src = _read(relpath)
assert needle in src, (
f"{relpath} is missing the wrapped-form pre-pass literal: {needle!r}"
)
def test_all_seven_sites_covered() -> None:
"""Pin: 5 JS sites + 2 Python sites = 7 capture sites, all guarded."""
js_total = sum(len(v) for v in JS_SITES.values())
assert js_total == 5, (
f"expected 5 JS capture sites, got {js_total}; this count is what the "
"review's '7 capture sites' was counting (1 linkifier + 1 TTS + 3 messages.js)"
)
assert len(PY_SITES) == 2
@pytest.fixture(scope="module")
def _media_token_re() -> re.Pattern[str]:
"""Compile the live ``_MEDIA_TOKEN_RE`` from api/routes.py (plus the
pre-pass) so behaviour tests exercise the real source."""
routes_text = _read("api/routes.py")
m = re.search(
r'_MEDIA_TOKEN_RE = re\.compile\(r"([^"]+)"\)', routes_text, re.M
)
assert m, "_MEDIA_TOKEN_RE compile() not found in api/routes.py"
bare = re.compile(m.group(1))
pre = re.compile(r"`MEDIA:([^`\s]+)`")
return _TwoPass(bare, pre)
@pytest.fixture(scope="module")
def _media_snap_re() -> re.Pattern[str]:
"""Compile the live ``media_re`` from api/media_snapshots.py (plus the
pre-pass) so behaviour tests exercise the real source."""
snap_text = _read("api/media_snapshots.py")
m = re.search(
r'media_re = _re\.compile\(r"([^"]+)"\)', snap_text, re.M
)
assert m, "media_re compile() not found in api/media_snapshots.py"
bare = re.compile(m.group(1))
pre = re.compile(r"`MEDIA:([^`\s]+)`")
return _TwoPass(bare, pre)
class _TwoPass:
"""Compose the wrapped-form pre-pass with the bare-token class so the
behaviour tests below exercise the live source exactly as it runs."""
def __init__(self, bare: re.Pattern[str], prepass: re.Pattern[str]):
self._bare = bare
self._pre = prepass
def findall(self, text: str) -> list[str]:
text = self._pre.sub(lambda m: f"MEDIA:{m.group(1)}", text)
return self._bare.findall(text)
def test_backend_allowlist_strips_backtick(_media_token_re) -> None:
"""``_MEDIA_TOKEN_RE`` (with the pre-pass) must not include the closing
backtick in the captured path."""
captured = _media_token_re.findall("get `MEDIA:/home/kim/skills.zip` now")
assert captured == ["/home/kim/skills.zip"], captured
assert all(not c.endswith("`") for c in captured)
def test_backend_allowlist_bare_token_still_captures(_media_token_re) -> None:
"""Backward-compat: bare ``MEDIA:/path`` (no backtick) still matches."""
captured = _media_token_re.findall(
"MEDIA:/home/kim/a.zip and MEDIA:/home/kim/b.tar"
)
assert captured == ["/home/kim/a.zip", "/home/kim/b.tar"], captured
def test_backend_allowlist_bare_path_with_backtick_in_filename(
_media_token_re,
) -> None:
"""#7680 finding 1: a bare ``MEDIA:/tmp/report`final.png`` (backtick
inside the filename, no surrounding wrapper) must capture the full
path. POSIX and Windows both allow backticks in filenames, so
truncating at the first backtick silently bricks the request."""
captured = _media_token_re.findall("see MEDIA:/tmp/report`final.png now")
assert captured == ["/tmp/report`final.png"], captured
assert not any(c.endswith("`") and c != "/tmp/report`final.png" for c in captured), captured
def test_backend_allowlist_rejects_trailing_paren(_media_token_re) -> None:
"""Pre-existing behaviour: ``)`` is still excluded."""
captured = _media_token_re.findall("see [file](MEDIA:/home/kim/a.zip) here")
assert all(not c.endswith(")") for c in captured), captured
def test_snapshot_capture_strips_backtick(_media_snap_re) -> None:
"""``media_re`` (with the pre-pass) must not include the closing
backtick in the captured path."""
captured = _media_snap_re.findall(
"assistant: `MEDIA:/home/kim/shot.png` and MEDIA:/home/kim/a.zip"
)
assert captured == ["/home/kim/shot.png", "/home/kim/a.zip"], captured
assert all(not c.endswith("`") for c in captured)
def test_snapshot_capture_bare_path_with_backtick_in_filename(
_media_snap_re,
) -> None:
"""#7680 finding 1: same bare-with-backtick case for the snapshot
pipeline. Without the pre-pass + bare class change, the snapshot
store would never see this path because the auth predicate and
the capture predicate must agree on the same set of references."""
captured = _media_snap_re.findall("MEDIA:/home/kim/shot`final.png tail")
assert captured == ["/home/kim/shot`final.png"], captured
def test_snapshot_capture_bare_token_still_captures(_media_snap_re) -> None:
"""Backward-compat: bare ``MEDIA:/path`` still matches."""
captured = _media_snap_re.findall(
"MEDIA:/home/kim/a.zip and MEDIA:/home/kim/b.zip"
)
assert captured == ["/home/kim/a.zip", "/home/kim/b.zip"], captured
# ---------------------------------------------------------------------------
# Revert-sensitivity: the pre-fix class really would have truncated
# the bare-with-backtick case, and the round-1 fix would have left the
# wrapped form broken in a different way.
# ---------------------------------------------------------------------------
def test_round1_buggy_class_truncated_bare_path_with_backtick() -> None:
"""The buggy (round-1) class greedily truncated at the first backtick
in the path, so ``MEDIA:/tmp/report`final.png`` was captured as
``/tmp/report``. This is the regression we are guarding against."""
buggy = re.compile(r"MEDIA:([^\s\)\]\`]+)")
assert buggy.findall("MEDIA:/tmp/report`final.png") == ["/tmp/report"], (
"round-1 class is no longer buggy; if you are intentionally reverting "
"to it, also remove the corresponding `test_post_fix_class_keeps_"
"bare_path` and update the source-shape guard."
)
def test_round1_buggy_class_dropped_wrapped_closing_backtick() -> None:
"""The round-1 class was actually correct for the wrapped form: it
dropped the closing backtick that the round-0 class greedily
included. Document the behaviour so a future revert picks the
correct replacement."""
round1 = re.compile(r"MEDIA:([^\s\)\]\`]+)")
assert round1.findall("`MEDIA:/home/kim/a.zip`") == ["/home/kim/a.zip"]
def test_round1_two_pass_compose_keeps_bare_path_intact() -> None:
"""The full #7680 fix is the two-pass compose: pre-pass strips the
wrapped form, then the bare class (no backtick in the exclusion
set) captures the full path. Verify the composition on both
cases so a future refactor that drops either half is caught."""
pre = re.compile(r"`MEDIA:([^`\s]+)`")
bare = re.compile(r"MEDIA:([^\s\)\]]+)")
def findall_composed(text: str) -> list[str]:
text = pre.sub(lambda m: f"MEDIA:{m.group(1)}", text)
return bare.findall(text)
# bare-with-backtick: pre-pass doesn't touch it, bare captures full path
assert findall_composed("MEDIA:/tmp/report`final.png") == [
"/tmp/report`final.png"
]
# wrapped: pre-pass strips the backticks, bare captures the path
assert findall_composed("`MEDIA:/tmp/a.zip`") == ["/tmp/a.zip"]
# mixed
assert findall_composed(
"see `MEDIA:/home/kim/shot.png` and MEDIA:/tmp/r`f.png tail"
) == ["/home/kim/shot.png", "/tmp/r`f.png"]
# plain bare: unchanged
assert findall_composed("MEDIA:/home/kim/a.zip") == ["/home/kim/a.zip"]
# ---------------------------------------------------------------------------
# node --check (so a JS syntax error in the modified source fails the suite)
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("relpath", sorted(JS_SITES.keys()))
def test_js_file_parses(relpath: str) -> None:
"""``node --check`` on every modified JS file."""
proc = subprocess.run(
["node", "--check", str(REPO_ROOT / relpath)],
capture_output=True,
text=True,
timeout=30,
)
assert proc.returncode == 0, (
f"{relpath} failed `node --check`:\n{proc.stderr}"
)
Loaded 100 of 148 files, more files were not shown because too many files have changed in this diff. Show more