Commit Graph
1148 Commits
Author SHA1 Message Date
webtecnica 301ed54ab7 fix(context-length): restore base_url for two guarded configs (#7743)
The #7535 ownership guard skipped the global model.base_url backfill for
two shapes that master handled:

- A `model` section with `base_url` but no `provider` declares no owner,
  so `_providers_match_for_context("", <session provider>)` was false and
  the endpoint was dropped, silently falling back to 256,000 instead of
  querying the configured /v1/models (the profile-setup path writes this
  shape).
- `opencode_go` and `opencode-go` are two spellings of the same built-in
  provider, but `_canonical_context_provider` does not fold underscores,
  so the owner session lost its URL.

Allow the backfill when no owner is declared, and compare both sides
through api.config._canonicalise_provider_id, which folds `_` to `-` while
keeping distinct `custom:*` slugs distinct. The #7535 case (a custom:ark
owner vs an opencode-go session) still resolves to an empty base_url.
2026-09-23 09:26:24 -03:00
webtecnica a44e329997 Merge origin/master into fix/7535-provider-base-url-ownership 2026-09-23 09:20:41 -03:00
nesquena-hermes dc554cbd1c Merge PR #7165 into stage/7165-custom-allowlist 2026-09-23 10:22:41 +00:00
happy5318andClaude Opus 4.8 0fcd584274 fix(models): dict-shaped models is per-model metadata, not an allowlist (re-gate 9/23)
Re-review #7165 (nesquena-hermes 2026-09-23T06:21:57Z, CHANGES_REQUESTED).

CORE: a dict-shaped custom_providers[].models mapping (what Hermes Agent's
setup flow writes, e.g. {chat-a: {context_length: 128000}}) was treated as an
allowlist by iterating its keys, collapsing the live /v1/models catalog to a
single saved default while the CLI live-probed the full list. The Agent defines
exactly this contract in hermes_cli/model_switch.py::_models_config_is_allowlist:
a mapping is per-model metadata, NOT a catalog narrow.

Fix (all 3 spec points):

1. api/routes.py::_custom_provider_allowlist_ids: added
   _custom_provider_models_discover_is_false(_cp) (mirrors
   api.config._provider_discover_allowed / Agent
   model_switch_providers.py:557: bool False or string false/no/0,
   case-insensitive). In the dict branch, when discovery is NOT explicitly
   false, return no allowlist so the live probe returns the full catalog.
   Only discover_models: false opts into treating dict keys as a pinned
   allowlist. List / JSON-array-string / Python-literal shapes stay plain
   allowlists.

2. tests/test_issue3718_live_models_custom_probe.py: removed the
   mapping-metadata param that asserted the regression; added two
   real-handler tests: test_dict_models_without_discover_false_does_not_gate
   (dict -> full _LIVE_CATALOG) and test_dict_models_with_discover_false_pins_to_keys
   (bool False / "false" / "no" / "0" -> pinned to keys).

3. docs/architecture/live-models-allowlist.md: documented that list and
   string shapes filter, and a dict filters only with discover_models: false.

Companion test update (blocker reported by the fix itself):

tests/test_byok_model_dropdown.py rightcode-codex entry encoded the removed
dict-as-allowlist behavior (models: {"gpt-5.5-mini": {}} asserting the response
collapsed to just that key). Added discover_models: False to preserve the
provider-scoping intent (no sibling leak) under the new contract.

Red/green verified: removing the dict-branch guard turns
test_dict_models_without_discover_false_does_not_gate red (collapsed catalog);
restoring it turns it green.

Verification: 3718 18 passed; byok 24 passed; neighbour sweep (46 files)
520 passed (2 pre-existing order-dependent flakes reproduce identically on
clean HEAD, verified via git stash); ruff curated gate OK.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-23 17:48:34 +08:00
happy5318 18842cc707 fix(models): discovered catalogs are discovery metadata, not allowlists
Hermes persists discovery results as models: {...} + models_discovered: true.
_custom_provider_allowlist_ids() treated any plural models value as an
explicit allowlist, so after one discovery /api/models/live was permanently
pinned to the first-discovery set and newly available upstream models were
silently dropped (gate repro: new-live-c missing vs merge-base).

Gate the helper on api.config._provider_models_are_discovered_catalog() —
the same predicate the /api/models path uses — and return no allowlist when
it holds. discover_models: false still pins via the predicate; a hand-written
models: list without the discovered marker still filters; probe-failure
fallback still serves saved ids.

Real-handler tests: discovered catalog preserves full live catalog (RED on
the previous head), discover_models: false control stays pinned, failed-probe
control still returns saved models.
2026-09-23 16:58:09 +08:00
happy5318 4b689ecbce fix(models): enforce plural allowlist contract on failed live probes 2026-09-23 16:58:09 +08:00
happy5318 88236dbc26 docs(models): pin empty-allowlist semantics with regression coverage
Greptile flagged that `models: []` / `models: "[]"` falls through the
non-empty allowlist gate and leaves the full live catalog selectable. The
symptom is real, but implementing the ask (gate on declared-ness instead)
measurably makes things worse: it emits an EMPTY picker and leaves the
provider unselectable.

Measured both branches against the real handler before deciding:

  models key absent   declared=False -> [chat-a, chat-b, image-gen-1, audio-tts-1]
  models: []          declared=True  -> []   EMPTY PICKER
  models: "[]"        declared=True  -> []   EMPTY PICKER
  models: [chat-a]    declared=True  -> [chat-a]

An unselectable provider is a harder failure than a few extra visible
models, and it is unrecoverable from the UI: `custom_providers` has no
WebUI write path, so it is hand-edited in config.yaml. There is also no
deny-all use case — a provider the user wants hidden is removed from
`custom_providers` outright, not left behind with an empty allowlist.

No production behavior change. Records the trade-off in the filter block
and adds `test_empty_allowlist_is_treated_as_not_configured` (native empty
list, serialized "[]", empty mapping, blank scalar) so a future "fix" of
this edge case fails loudly instead of silently emptying the picker.

Verified non-vacuous: applying the declared-ness gate fails all 4 new
cases with `assert [] == [...]`.
2026-09-23 16:58:09 +08:00
happy5318 4a1fb47ce5 fix(models): respect config model allowlist in live custom-provider catalog
Custom providers (esp. New-API-style gateways) expose their ENTIRE catalog
via /v1/models, including image/audio models that aren't chat models. The
live-catalog enrichment surfaced all of them in the model picker regardless
of the `models` allowlist declared in config.yaml.

Filter the live catalog down to the explicit plural `models` allowlist, then
append any allowlisted models the upstream didn't return. Only the plural
`models` key gates the filter — a provider configured with just a singular
`model` (sticky/default metadata) keeps the unfiltered live catalog so
discovery still works.

Decode the plural value through `_parse_config_string_list()` so serialized
shapes are honored: `hermes config set` and JSON-mode editor saves persist
lists as quoted JSON-array strings ('["chat-a","chat-b"]') or Python
literals ("['chat-a']"). A dict/list-only parser dropped those to [], which
the caller reads as "no allowlist configured" → full catalog floods the
picker (same class as the skills.disabled regression, #7120/#7134). Native
dict/list values are still walked directly so dict entries keep their
id|model|name metadata, which the decoder would str()-ify.

Tests invoke the real `_handle_live_models` and assert on the emitted
`models` payload, covering: native list, JSON-array string, Python-literal
string, mapping metadata, list-of-dicts, singular-model-only (not gated),
malformed serialized input (safe), allowlisted model missing from live
(appended), and probe failure (config fallback).

Verified non-vacuous: removing the production filter block fails 7 of 9
cases; reverting only the string decode fails exactly the 3 serialized
cases with the documented flood symptom (image/audio models reappear).
2026-09-23 16:58:09 +08:00
nesquena-hermes a028961133 Merge PR #7708 (contributor head 6455aa9b2b) 2026-09-23 06:22:22 +00:00
happy5318 6455aa9b2b fix(media): two-pass MEDIA: scan (pre-pass for wrapped, bare class without backtick exclusion) (#7708 round 2)
#7708 re-gate (9/22 23:47), both findings:

**Finding 1 (SILENT)**: A bare path that contains a backtick
(MEDIA:/tmp/report`final.png) is truncated at the first backtick,
so the server stops authorizing it. The round-1 fix traded one bug
for another: adding the backtick to the bare-token exclusion set
dropped the inline-code closing backtick (good) but truncated any
bare path with a backtick in the filename (bad — POSIX and Windows
both allow backticks).

**Finding 2 (SILENT)**: TTS now reads filename fragments aloud
(MEDIA:/tmp/report(1).png -> 'a file).png') because the
parenthesis-exclusion was added to a site that should not need it.
Inline code is already stripped at the top of the TTS flattener.

**Fix shape** (maintainer-suggested two-pass scan):

1. Pre-pass: `MEDIA:([^`\s]+)` -> MEDIA:$1 — strips the
   inline-code backticks off the wrapped form so the bare class
   below sees a plain MEDIA:path.
2. Bare class: MEDIA:([^\s\)\]]) (original, backtick
   removed from the exclusion set) — captures the full filename
   even when the path itself contains a backtick.
3. TTS: restore the original MEDIA:[^\s]+ (no paren/bracket
   exclusion either). Inline code is stripped at the top of the
   flattener so the closing backtick of an inline-code wrapper has
   already been eaten by the time the TTS regex scans the text.

Applied to all 7 capture sites:
- api/routes.py:20453 (_MEDIA_TOKEN_RE, findall site 20513)
- api/media_snapshots.py:507 (media_re, findall site 533)
- static/ui.js:7695 (linkifier, replace site)
- static/ui.js:9142 (TTS flattener — separate shape)
- static/messages.js:4848 (chunk finalize, exec)
- static/messages.js:4896 (combined walk, exec loop)
- static/messages.js:4922 (partial tail, exec)

**Tests** (24 total, all green):
- 11 source-shape guards: each JS/PY site must carry the bare-token
  class literal AND the wrapped-form pre-pass literal
- 4 behaviour tests against the live source: wrapped form strips
  closing backtick, bare form captures plain path, bare form with
  backtick in filename captures the full path (the new regression),
  trailing paren is still excluded
- 2 round-1 regression guards documenting the trade-off that drove
  the two-pass compose
- 1 composed two-pass test (pre-pass + bare class on 4 inputs)
- 2 node --check syntax guards
- 1 site-count guard: 5 JS + 2 PY = 7 capture sites, all guarded

**Verification**:
- pytest tests/test_issue7359_media_backtick_strip.py -v -> 24/24 pass
- python3 scripts/ruff_lint.py --diff origin/master -> 'no new
  violations on added/modified lines. OK.'
- node --check on the two modified JS files -> both OK
2026-09-23 12:03:33 +08:00
webtecnica a8180bc51f fix(context-length): guard provider ownership on global model.base_url backfill (#7535)
A session on a built-in registry provider (e.g. opencode-go, endpoint in
PROVIDER_REGISTRY, empty base_url by design) had its empty base_url slot
backfilled with the global model.base_url owned by a *different* provider
(custom:ark). The context-length resolver then probed the wrong endpoint's
/models catalog and fell back to the generic 128K catalog entry for a
1M-context model, while the on-disk session and SSE stream carried 1,000,000.

Only fill the global model.base_url slot when the session provider IS the
configured model.provider owner, mirroring the ownership predicate used for
model_cfg's API key in _context_length_config_api_key_for_provider.
2026-09-22 22:54:12 -03:00
nesquena-hermes d79640f6b1 Merge PR #7631 into stage/rm-343-batch 2026-09-23 00:43:13 +00:00
nesquena-hermes c8b1f87583 Merge PR #7445 from ruizanthony: fix(sqlite): keep state.db readers strictly read-only and stop idle polling 2026-09-22 22:56:02 +00:00
nesquena-hermes 7109a9aed5 Merge PR #7569 from happy5318: perf(run-journal): rebuild live snapshots without quadratic work 2026-09-22 22:27:01 +00:00
nesquena-hermes 47f8e028cd Merge PR #6937 (contributor head 047dfa40a5) 2026-09-22 21:14:09 +00:00
carlotestor 5c24458dff fix(sidebar): derive the route session cap from the shared limit
The env override never reached /api/sessions because the route kept a
separate hard-coded cap of 20. Resolve the window in api.config before
profile init, clamp it at 200, and refuse a profile .env override.
2026-09-22 17:50:56 +00:00
Anthony Ruiz 7a30ea5f39 fix(api): fence chronological state merge by provenance 2026-09-22 16:37:14 +00:00
Rodrigo Gomes da Silva 7862927ef5 fix(http): keep a bodyless rejection's keep-alive, at every reject site
The mirror image of the sidecar over-close already fixed here: the sibling
reject-before-read sites still armed `arm_connection_close()` unconditionally,
so a request with NO declared body was answered with `Connection: close` and
the client's pipelined follow-up was dropped -- a healthy pooled connection
killed for no framing reason. Reproduced on the wire, pipelined down one
socket against the production handler (`responses seen: 1` at each site):

  POST /api/session/new     off-session auth  -> 401 + Connection: close
  POST /api/session/new     bad Origin        -> 403 + Connection: close
  POST /api/session/new     no CSRF token     -> 403 + Connection: close
  POST /api/csp-report      limiter tripped   -> 204 + Connection: close
  POST /api/health/restart  success           -> 200 + Connection: close
  POST /api/process-complete-ack              -> 410 + Connection: close
  POST /api/upload          no boundary       -> 400 + Connection: close

Each now routes through `arm_connection_close_if_body_pending()` (the helper
the sidecar path already used), so the framing decides in both directions:
body-bearing or unreadable framing still closes, an absent or agreeing-zero
Content-Length keeps keep-alive. `check_auth_or_close()` cannot swap the helper
in directly -- it has to arm before `check_auth()` writes its 401/302 and undo
that on success -- so it gates the arming on `request_declares_body()` instead
and keeps the success-path restore.

`/api/health/restart` is armed once, ahead of the outcome, so all four results
(completed / in_progress / busy / error) follow the framing rather than the
result; the success path was closing on every call the WebUI makes.

Only the boundary check in the upload preflight can reject a bodyless request:
every other raise there (any Transfer-Encoding, an unreadable or conflicting
length, an over-cap length) already declares a body and still closes.

Five tests pinned the unconditional close on stubs that declared no body --
i.e. they pinned the bug. Each is re-grounded on real body-bearing framing
(Content-Length, chunked, and an unreadable length) so the close half stays
pinned, with a bodyless sibling asserting keep-alive. New real-socket
regressions cover all seven sites in both directions.
2026-09-22 12:06:09 +00:00
Rodrigo Gomes da Silva b3a031a19d fix(http): decide reject-before-read closes from the framing, not the method
Two framing gaps the re-gate reproduced on the production handlers, both the
same class: something other than the wire was used to decide whether unread
body bytes are queued.

1. api/routes.py — the sidecar provenance 403 gated its close on
   `read_request_body`, a static per-method flag. That is wrong in both
   directions: a GET carrying `Content-Length`/`Transfer-Encoding` was answered
   with keep-alive and its body then parsed as the next request line
   (`501 Unsupported method ('{}GET')`), while a body-less DELETE/PUT/PATCH had
   a healthy pooled connection closed for nothing.

2. api/upload.py — all four multipart handlers read framing from
   `Content-Length` alone, so a chunked upload (which has none) was treated as
   length zero, matched no parts and was rejected 400 "No file field in
   request" with keep-alive intact. Wire trace before the fix:

       HTTP/1.1 400 Bad Request                  <- the upload rejection
       HTTP/1.1 400 Bad request syntax ('54')    <- our own chunk-size line

New primitive in api/helpers.py: `request_declares_body()` (Content-Length
present and non-zero, or any Transfer-Encoding other than identity; an
unparseable length counts as present, since it cannot prove absence) plus
`arm_connection_close_if_body_pending()` and `unsupported_transfer_encoding()`.

api/upload.py now routes all four handlers through one preflight,
`_read_multipart_or_reject()`, which refuses an undecodable transfer coding
with 411 before touching rfile — http.server never decodes chunked framing, so
that request can only be rejected — and keeps every previous status and message
(413 oversize, 400 invalid Content-Length/boundary) with its close.

Regressions: chunked reject that closes and a body-less reject that does not,
at all four upload handlers and at the sidecar 403, plus two pipelined
real-socket tests against the running server proving the follow-up request is
never answered off a poisoned socket.
2026-09-22 12:06:09 +00:00
Rodrigo Gomes da Silva a93e8ba890 refactor(http): route every reject-before-read through one arming helper
Replaces the inline `handler.close_connection = True` at each reject-before-read
site with api.helpers.arm_connection_close(), the shared mechanism the review
asked for. Behaviour is unchanged at every real handler; the helper adds the
same stub tolerance the read side just got, so a handler that refuses the
attribute can never turn a 4xx into a 500.

Sites: _check_csrf (origin + token), the sidecar provenance 403 (body-reading
calls only), the rate-limited CSP report, /api/health/restart, the deprecated
process-complete-ack 410, and all four upload handlers' pre-read
length/boundary rejections.
2026-09-22 12:06:09 +00:00
Rodrigo Gomes da Silva 3c5569a30e fix(http): advertise Connection: close on every reject-before-read path
Deep-review gate (2026-08-14) findings, all reproduced:

1. close_connection was never serialized: BaseHTTPRequestHandler does not
   emit Connection: close from the flag, so pooled clients kept reusing the
   socket and died with BrokenPipeError. Handler.end_headers() now emits
   the header (deduped) whenever close_connection is armed.
2. check_auth_or_close armed the flag too late (after the 401/302 was
   already flushed); it now arms before check_auth() and restores the
   prior flag on success.
3. The sidecar provenance rejection closed body-less GETs too; the close
   is now gated on read_request_body=True.
4. Remaining reject-before-read sites now close-and-advertise through the
   same mechanism: rate-limited CSP reports (204), the deprecated
   /api/process-complete-ack (410), /api/health/restart (all outcomes),
   and the four upload handlers (413 + pre-read Content-Length/boundary
   validation via _reject_before_read).

Tests: pooled-client regression against the production handler (reject
with a body, then follow-up on the same pooled connection succeeds and the
rejection carries Connection: close); unit coverage for the sidecar GET
keep-alive, CSP rate-limit, health/restart and upload oversize paths. The
server.py line guard is bumped to 760 with rationale: the deep-review
requires the emission in Handler.end_headers(), which cannot live outside
server.py.
2026-09-22 12:06:09 +00:00
Rodrigo Gomes da Silva 1bd3c72e17 fix: close connection on extension-sidecar provenance rejection
Same class as the auth/CSRF close-on-reject: the sidecar proxy's
same-origin provenance check runs before read_body(), so an HTTP/1.1
connection reused after a 403 would parse the unread body as the next
request line. Flag close_connection on this path too.

Found by Greptile review of #6658; covered by a new unit test.
2026-09-22 12:06:08 +00:00
Rodrigo Gomes da Silva 20f6a68626 fix: close rejected HTTP writes before reuse 2026-09-22 12:06:08 +00:00
happy5318 d698fb4772 fix(routes): mirror the detail-load endpoint's 409 contract in every profile-mismatch check (#7710)
## Thinking Path

The detail-load endpoint (and the foreign-session synthesizer) already
distinguish "session owned by a KNOWN other profile" (return ``409
session_profile_mismatch`` so the client can offer to switch to it
per #5419) from "session missing/legacy with no profile stamped"
(return ``404 Session not found`` so the frontend's self-heal
clears the stale URL + localStorage).

The generic request-guard
``_session_id_visible_to_request_profile`` flattened both cases to a
``404 Session not found``, so any cross-profile POST/PATCH/DELETE
(archive, rename, pin, delete, session-attach, anchor-scene, …) and
the stream-related status/cancel/replay paths failed with a
misleading "Session not found" instead of the actionable 409 the
detail endpoint emits. The reporter's wire evidence:

```
POST /api/session/archive  {"session_id":"<id>","archived":true}
-> 404 {"error": "Session not found"}

GET /api/session?session_id=<same id>
-> 409 {"error": "Session belongs to a different profile",
        "code": "session_profile_mismatch", "session_id": "<id>", "profile": "def"}
```

Same session id, same moment: the detail endpoint reports a profile
mismatch, archive masks it as 404.

The fix mirrors the detail endpoint's contract in the generic guard
and in every other inline check that previously went to
``bad(handler, "Session not found", 404)`` for a known-other-profile
session.

## What Changed

1. ``api/routes.py:595`` — ``_session_id_visible_to_request_profile``
   now emits ``409 session_profile_mismatch`` for a session owned by
   a known other profile, and keeps ``404 Session not found`` only
   for the None-profile (unknown/legacy) case so the frontend
   self-heal still fires for actually-missing sids.

2. ``api/routes.py:5348`` (anchor-scene persist), ``:23994``
   (compression-recovery start), ``:24385`` (chat/start with
   the inline placeholder-retag rules preserved), ``:28621``
   (session import_cli's already-imported branch) — the same
   409-vs-404 split, with the existing branches kept intact (the
   chat/start retag path, the import_cli ``allow_all_profiles`` short
   circuit).

3. The ``_stream_id_visible_to_request_profile`` wrapper already
   delegates to ``_session_id_visible_to_request_profile`` for the
   owner-session check, so the stream status / cancel / replay
   endpoints get the new contract for free.

## Why It Matters

A user picking the configured default model in the dropdown sees two
visually identical entries for the same model id, picks the wrong
"prefixed" one (the one matching the dropdown's order), the agent
returns 400, and the session state is corrupted with a non-functional
model id until the user manually clears it. The dropdown's own
deduplication already handles `@provider:model` correctly via the
documented prefix form; only the synthesised ``{provider}/{model}``
form was wrong.

The cross-profile write paths (anchor-scene persist,
compression-recovery start, chat/start, session import) all had the
same gap: a successful auth under profile A could persist or recover
state against a session owned by profile B, surfacing a 404 to
profile A's user that reads like "the session is gone" — but the
session is in fact intact, just owned by another profile. The 409
contract is what closes the gap: the client gets a structured signal
it can act on, and the session state stays consistent.

## Verification

- ``tests/test_issue7710_cross_profile_404_to_409.py`` (new, 6 tests):
  - 5 behavioural: profile-mismatch path returns 409 with the
    ``session_profile_mismatch`` payload, the None-profile path
    keeps 404 for the frontend self-heal, visible session passes
    silently, missing session passes silently, ``emit_error=False``
    suppresses the response.
  - 1 source-shape: the helper body MUST contain a 409 branch with
    the ``session_profile_mismatch`` code; a silent reversion to
    the single-line ``bad(..., 404)`` fails the suite.
- ``tests/test_issue4812_session_sse_stream.py`` — 1 assertion
  updated to the new 409 payload (the SSE route walks through the
  generic guard).
- ``tests/test_session_active_profile_authorization.py`` — 7
  assertions updated to the 409 payload (duplicate, file, chat/start
  foreign, chat/stream/status, chat/stream/cancel, chat/stream
  replay). Stream tests use ``foreign_session`` (the owner session
  id), NOT the stream id, because ``_stream_id_visible_to_request_profile``
  delegates to the owner-session check.
- ``tests/test_anchor_scene_persistence.py`` — 1 assertion updated
  to the 409 payload. The 3 ``Assistant message not found`` 404
  cases (lines 454/561/611) are kept as-is — they're not profile
  mismatches, they're missing-message 404s.
- ``tests/test_issue4067_import_cli_cross_profile_guard.py`` —
  1 assertion updated to the 409 payload, plus the ``_capture`` mock
  signature updated to accept the ``status`` keyword that
  ``j(handler, payload, status=200, ...)`` already passes.

Other tests in the existing suite: **161 passed across 8 related
test files** (session active-profile authorization, anchor scene,
issue 4067, issue 4812 SSE, issue 7710 new, chat start claim,
workspace stale recovery, issue 5420 profile switch).

## Contract Change

The following routes now return ``409 session_profile_mismatch``
(``{error, code, session_id, profile}``) for a session owned by a
known other profile, where they previously returned
``404 Session not found``:

- ``POST /api/session/duplicate`` (now also goes through
  ``_guard_request_session_visibility`` for the body session id)
- ``GET  /api/file?session_id=…``
- ``POST /api/chat/start`` (the inline profile check after
  ``_get_or_materialize_session``)
- ``GET  /api/chat/stream/status?stream_id=…``
- ``GET  /api/chat/stream/cancel?stream_id=…``
- ``GET  /api/chat/stream?stream_id=…`` (the dead-stream replay
  branch)
- ``POST /api/session/anchor-scene``
- ``POST /api/session/compression-recovery/start``
- ``POST /api/session/import_cli`` (the already-imported branch
  when ``allow_all_profiles`` is False)

The ``404 Session not found`` self-heal path is preserved for the
None-profile (unknown/legacy) case across all of these routes, so
the frontend's existing stale-URL/localStorage cleanup behaviour is
unchanged for actually-missing sessions.

## Risks

- A consumer that previously relied on the 404 to detect "this
  session belongs to another profile" will now see a 409 with a
  ``code`` field. The detail-load endpoint has emitted this exact
  contract since #5419, and the frontend already handles it
  (offers to switch to the owning profile). The cross-profile write
  paths now match the detail-load contract; the frontend's
  already-shipped switcher UI handles both surfaces consistently.
- The anchor-scene / compression-recovery / chat-start tests that
  were updated are a small set (5 total); the existing
  ``Assistant message not found`` 404 paths in
  ``test_anchor_scene_persistence`` are kept untouched (those are
  not profile mismatches).

## Related

- #7710 — original issue (elkpvl-commits, 2026-09-22) with the
  full repro and the suggested fix shape.
- #7549 — the underlying cross-profile archive bug, already
  tracked by @webtecnica in #7559 (which is the "business-level"
  fix: row-profile carry + frontend sends profile + materialize
  into the row's own profile). This PR is the orthogonal
  error-code-level fix: the request-guard's 404-vs-409 split.
- #5419 — the original auto-switch-profiles-for-session-links
  feature, whose frontend switcher UI consumes the
  ``session_profile_mismatch`` code on the 409.
2026-09-22 10:21:22 +00:00
nesquena-hermes e6592f41af fix(login): restore i18n placeholders and keep passkey-only logins working
Gate follow-up for #7056. Two defects in the conditional password form:

1. The {{LOGIN_INVALID_PW}} / {{LOGIN_CONN_FAILED}} .replace() calls were
   deleted with the moved block, but those placeholders live on the <form>
   tag which ALWAYS renders. login.js reads data-invalid-pw back, and the
   literal is non-empty so its || fallback never fired: a wrong password
   displayed the raw text {{LOGIN_INVALID_PW}} in all 14 locales, even with
   password auth ENABLED. Proven by a clean-master control (test_login_locale
   3/3 on master, 2/3 on the PR).

2. is_password_auth_enabled() also gated the PASSKEY button, but passkeys are
   an independent auth method (auth.py is_auth_enabled, and settings expose
   passwordless_enabled = passkeys AND not password). A passwordless-passkey
   instance lost its only login control. login.js additionally bailed at
   'if (!form || !input) return' before ever wiring the passkey handler.
2026-09-22 09:57:37 +00:00
happy5318 c53226dcc4 fix(login): hide the password form when `is_password_auth_enabled()` is False (#7056)
## Thinking Path

With native OIDC configured and ``HERMES_WEBUI_PASSWORD`` unset, the
``/api/auth/status`` endpoint correctly reports
``password_auth_enabled: false``, but ``/login`` still rendered the
password input, the "Sign in" button, and the "Continue with SSO"
entry point — and silently 401-ed every submit. The reporter's wire
evidence:

```json
GET /api/auth/status  -> {
  "auth_enabled": true,
  "oidc_enabled": true,
  "password_auth_enabled": false
}

GET /login            -> still shows:
  "Enter your password to continue"
  + password <input>
  + "Sign in" button
  + "Continue with SSO" link

POST /api/auth/login   -> 401 {"error":"Invalid password"}
```

The fix routes the password input / submit / passkey controls through
a single ``{{PASSWORD_FORM_HTML}}`` placeholder in the template, which
the handler populates with the live markup only when
``is_password_auth_enabled()`` is True. The OIDC SSO entry point stays
the sole path when password auth is disabled.

## What Changed

1. ``api/routes.py`` template literal (``_LOGIN_PAGE_HTML``): the
   three inline elements (password input, submit button, passkey
   button) are replaced by a single ``{{PASSWORD_FORM_HTML}}``
   placeholder. The ``{{OIDC_LOGIN_HTML}}`` placeholder is kept
   untouched.

2. ``api/routes.py`` login branch: the handler imports
   ``is_password_auth_enabled`` from ``api.auth``, computes the
   password form markup when the predicate returns True, and an
   empty string otherwise. The ``.replace("{{PASSWORD_FORM_HTML}}", _password_form_html)``
   call slots the result into the template. The
   ``{{LOGIN_PLACEHOLDER}}``, ``{{LOGIN_BTN}}``, ``{{LOGIN_INVALID_PW}}``,
   ``{{LOGIN_CONN_FAILED}}`` placeholders are no longer referenced
   from the template (the password input is built with its own
   ``placeholder`` attribute), so the four corresponding
   ``.replace(...)`` calls are removed.

## Why It Matters

A user with OIDC-only auth clicking "Sign in" on the password form
gets a 401 with no UI signal — the OIDC entry point is right there,
but the password prompt was so prominent it read like the primary
path. Hiding the form when password auth is off makes OIDC the only
visible option, so the user is funneled to the working path.

## Verification

- ``tests/test_issue7056_login_hides_password_form.py`` (new, 6 tests):
  - **Source-shape (2 tests):** the template uses the
    ``{{PASSWORD_FORM_HTML}}`` placeholder AND does not carry an
    inline ``<input type="password">``; a silent reversion to the
    pre-fix multi-line shape fails the suite. The login branch
    imports ``is_password_auth_enabled`` and populates the
    placeholder.
  - **Behavioural (3 tests):** the rendered ``/login`` HTML contains
    the password input / submit / passkey controls when
    ``is_password_auth_enabled()`` is True; does NOT contain them
    when False; and keeps the ``<a id="oidc-login">`` SSO entry
    point when OIDC is enabled (so the user can still log in).
  - **Sanity (1 test):** with both password and OIDC disabled, the
    page renders neither control.
- 51 tests pass across the related auth/login files
  (``test_issue7056_login_hides_password_form``, ``test_auth_settings_safety``,
  ``test_issue3825_oidc_auth``) — no pre-existing test is broken.

## Contract Change

None. The ``/login`` HTML body changes shape only when
``is_password_auth_enabled()`` is False; the password input + submit
+ passkey controls are gone, but the OIDC SSO entry point stays
present. The ``/api/auth/status`` contract is unchanged — it already
reported the right ``password_auth_enabled`` flag; the fix is purely
about the page matching the status.

## Risks

- A consumer that scraped the ``/login`` HTML for the password
  input (e.g. a browser extension) will see the input disappear
  when ``HERMES_WEBUI_PASSWORD`` is unset. The OIDC entry point
  remains the only path, which is the intended behaviour when
  password auth is disabled.
- The handler now imports ``is_password_auth_enabled`` at request
  time (same module as the other auth predicates already imported);
  if the auth module is unavailable the import fails fast and the
  ``/login`` route surfaces the underlying exception, matching the
  behaviour of the other auth imports in the same branch.

## Related

- #7056 — original issue (segator, 2026-08-15) with the wire evidence
  and the expected-vs-actual contract.
- The OIDC SSO entry point lives in ``_oidc_login_html`` (line 11160)
  and is unaffected by this PR — the fix only gates the password
  controls, not the SSO button.

## Model Used

Hermes Agent (step-5-preview) with pytest, ruff. Tests use
``_FakeHandler`` with an ``io.BytesIO`` ``wfile`` (matching
``_safe_write``'s ``handler.wfile.write(body)`` expectation) and
mock the OIDC predicate through ``_stub_oidc`` so the
behavioural assertions don't depend on real OIDC env config.
2026-09-22 09:42:04 +00:00
happy5318 2330faa0e4 fix(media): MEDIA: token regex strips markdown backtick on 7 capture sites (#7359)
## Thinking Path

When an assistant emits `` `MEDIA:/path` `` wrapped in inline-code markdown, the
shared character class ``[^\s\)\]]+`` across all seven MEDIA capture sites
greedy-matched the closing backtick into the path. The resulting
``/api/media?path=...%60`` URL 404'd on the legitimate file (issue #7359).
The same drift between frontend linkification and backend allowlist means
even if the URL is constructed with the trailing backtick, the session
allowlist rejects the backtick-polluted path — the user gets a broken
download and a 404 body saved as ``.json``.

Verified the issue's repro: 6 sites share ``[^\s\)\]]+`` (the
preview-flattener at ui.js:9142 is ``[^\s]+`` and is a 7th site, also
covered). One additional site, ``messages.js:4896`` (combined walk),
was missing from the original 6-site list in the issue body — the
suggested fix shape applies to all seven.

The fix is a one-character-set widening: add `` ` `` to the excluded
class on every site so the capture stops at the markdown delimiter
instead of including it.

## What Changed

All seven sites: ``[^\s\)\]]+`` → ``[^\s\)\]\`]+`` (JS) and the
Python equivalent ``MEDIA:([^\s\)\]\`]+)``:

1. ``static/ui.js:7695`` — linkifier regex (the front-door capture).
2. ``static/ui.js:9142`` — preview-text flattener (also drops ``)`` and
   ``]`` for symmetric handling; was ``[^\s]+`` which would still pull
   the backtick into the preview).
3. ``static/messages.js:4848`` — streaming chunk finalize.
4. ``static/messages.js:4896`` — streaming combined walk (the
   issue-missing 7th site).
5. ``static/messages.js:4922`` — streaming partial tail.
6. ``api/routes.py:20453`` — ``_MEDIA_TOKEN_RE`` (the session allowlist).
7. ``api/media_snapshots.py:507`` — ``media_re`` (settle-time snapshot
   capture).

## Why It Matters

A user clicking an inline-code-wrapped MEDIA token gets a 404 body saved
as ``.json`` instead of the file they asked for. On Android Chrome (and
desktop Chrome), the 404 silently downloads as ``skills.zip.json`` with
no UI signal that the original was missing. The backend allowlist and
the frontend linkifier must agree, so the fix widens the character
class on both sides.

## Verification

- ``tests/test_issue7359_media_backtick_strip.py``: 17 passed (new).
  - 7 site-shape assertions (parametrized over the seven sites).
  - ``_MEDIA_TOKEN_RE`` behaviour: strips trailing backtick from inline-code
    payload, preserves bare-token capture, preserves trailing-``)`` exclusion.
  - ``media_re`` behaviour: same three.
  - pre-fix class round-trip: ``\`MEDIA:/a.zip\``` → ``["/a.zip\`"]``
    (the reported bug); post-fix → ``["/a.zip"]``.
  - ``node --check`` on the two modified JS files.
- ``pytest tests/ -k 'media and not 7359'``: 291 passed, 1 skipped,
  0 failed — no existing media test regresses.
- ``ruff check tests/test_issue7359_media_backtick_strip.py``: clean.
- ``scripts/ruff_lint.py --diff origin/master`` curated gate: 0 new
  findings on added/modified lines.

## Contract Change

No API contract change. UI rendering may now display
``somefile.zip`` (a file) where it previously displayed
``somefile.zip\``` (literal backtick in the preview); both already
displayed correctly in the bare-token case, and the inline-code path
was the broken one.

## Risks

- Sites that depended on the backtick being included in the path would
  regress, but no consumer in this repo treats a trailing `` ` `` as
  part of a MEDIA path — the original six-site fix shape was
  maintainer-recommended and the 7th site matches the same pattern.
- The preview flattener at ``ui.js:9142`` previously used a wider class
  (``[^\s]+`` only, no ``)``/``]``) so it would have matched a path
  containing a literal ``)``. The fix tightens that to match the
  other six sites; if anyone relied on the looser class the rendering
  would now use ``a file`` for the surrounding paragraph. Verified no
  test in the suite exercises a MEDIA token containing ``)``.

## Model Used

Hermes Agent (step-5-preview) with pytest, ruff, and node. Tests are
executed against the live compiled regexes (for the two Python sites)
and against the literal source text (for the five JS sites), so a
revert on any of the seven sites surfaces as a source-shape failure.

## Related

- #7359 — original issue (KimSpecht, 2026-08-29) with the repro and
  six-site list.
- #7334 — pre-existing PR (angelusbr, 2026-08-27) that covers three
  sites; this PR supersedes it by covering all seven and following the
  maintainer-recommended shape from the 9/20 comment.
- Maintainer's 9/20 comment on #7359 named #7334 as "the correct
  existing implementation lane"; if #7334 lands first, this PR is
  closeable as duplicate and the maintainer's ``sprint-candidate``
  label can move. Credit goes to angelusbr for the early character-class
  idea regardless of which PR merges.
2026-09-22 17:24:29 +08:00
Michael Snoyman fdd72e815a Fix gateway approval settlement lifecycle
Settle removed approval producers before fallible projection work, and fail-close every producer owned by a terminal gateway run across completion, failure, cancellation, and teardown while preserving unrelated runs.
2026-09-22 06:29:27 +00:00
Michael Snoyman 8c92d88225 Fix exact gateway approval producer resolution 2026-09-22 06:29:27 +00:00
laitekin 5cbc5f7106 fix(run-journal): preserve cursor ids for unjournaled metering 2026-09-21 22:27:23 +08:00
laitekin bcafbd6965 fix(run-journal): skip metering telemetry on write and replay (#7272)
Metering telemetry is journaled at ~10 Hz during active turns, so marathon
reasoning runs balloon the durable run journal (12.4 MB of a single 18 MB run
was metering). On a mid-run refresh the reconnect path replays the entire
journal as one unpaced SSE burst, OOM-crashing the browser tab in a
crash-refresh loop while the server stays healthy.

- Write time: RunJournalWriter.append_sse_event() skips metering rows via
  REPLAY_SKIPPED_SSE_EVENTS, returning None so caller journal-id plumbing is
  untouched and remaining journaled seqs stay contiguous.
- Replay time: _replay_run_journal() and emit_replay() filter metering rows
  through journal_replay_visible(), so legacy journals never stream the
  backlog either.
- Readers (read_run_events / read_session_run_events) deliberately keep every
  row: cursor math and the offline-gap coverage check count seqs.

Refs #7272
2026-09-21 22:27:23 +08:00
happy5318 5ac3632d58 chore(run-journal): drop now-unused echo helper imports from routes
The journal replay matches through _CompactEchoIndex, so
_find_compact_echo_suffix_start and _strip_compact_echo_suffix are no
longer referenced in api/routes.py. The helpers stay in api/streaming.py
as the equivalence oracle the tests compare against.
2026-09-21 12:34:30 +08:00
happy5318 dcac71c74e fix(run-journal): match echo suffixes through an incremental folded index
Review follow-up for #7569: change 4 of the live-snapshot perf PR replaced
one quadratic with another. The window-free backward walk is correct, but it
re-walks the raw reasoning transcript's whitespace once per interim event —
on a whitespace-heavy transcript that span is megabytes, and the replay went
~55x slower than master on the production shape the gate measured
(1MB / 20-interim journal: 0.035s master → 1.922s branch).

## Thinking Path

The gate's finding was that the cost is structural, not a tweak: the matcher
consults the *whole* accumulated transcript on every interim event, so any
raw-text rescan of it is O(interim_count x whitespace_span). The fix is the
data-structure change the review asked for — maintain the folded view and the
raw cut offsets incrementally, and match/strip from that indexed state
instead of rescanning the raw buffer.

## What Changed

- `api/streaming.py`: new `_CompactEchoIndex` — an append-only folded index
  that records, per non-whitespace character, its raw index. `append` folds a
  chunk once (cost proportional to the chunk, never the buffer),
  `matches_tail`/`cut_to` compare only the candidate against the folded tail
  (O(len(candidate)), no raw walk), `reset` drops the state after the raw
  text is truncated. The old raw helpers (`_find_compact_echo_suffix_start`,
  `_strip_compact_echo_suffix`) stay as the equivalence oracle.
- `api/routes.py::_run_journal_live_snapshot`: reasoning chunks feed the
  index as they are appended; `reasoning_echo_tail_matches` and
  `strip_reasoning_echo_tail` consult the index. The strip re-indexes the
  truncated transcript so later probes match against it.
- `api/streaming.py` live path: the same swap for the live-stream echo strip.
  The three live reasoning buffers (STREAM_REASONING_TEXT, the coalescing
  buffer, the per-message segments) each carry a mirrored index, fed on every
  append and re-indexed after a strip. The buffer flush resets the buffer's
  index with the text it drops, so the index never describes text that is no
  longer there.
- `docs/rfcs/webui-run-state-consistency-contract.md`: the echo-match clause
  now names the indexed representation and the scaling regression that pins
  it.

## Why It Matters

Removing one quadratic by introducing another in the same hot path is the
exact risk the gate flagged — the headline benchmark used a journal shape
where the window-free walk's cost does not dominate, so the aggregate number
still looked like a win. The index keeps the original 8.9x and adds the
scaling guarantee on the shape that was regressing.

## Verification

- Measured on the reproduced shape (1MB whitespace span, 20 interim events):
  0.972s → 0.049s. Doubling both span and interim count scales ~2x per step
  (x1 0.041s / x2 0.090s / x4 0.175s) — linear, not quadratic.
- Behaviour equivalence: 300 randomized journals (whitespace-heavy,
  Unicode, chunked, flagged/unflagged interims) rebuilt with the indexed
  implementation and with the raw walk swapped in — sha256 of the canonical
  snapshot JSON matched on all 300.
- `tests/test_compact_echo_index.py` (new, 6 tests): oracle equivalence over
  2000+ randomized cases, chunked-append equivalence, matches_tail/cut_to
  agreement, reset-after-truncation, and two cost probes that fail the raw
  walk.
- `tests/test_live_snapshot_echo_scan_scaling.py` (new, 4 tests): the
  proportional-scaling regression the review asked for — verified to FAIL on
  the pre-fix matcher (14.6x time for 4x rows, the quadratic signature) and
  pass on the indexed one.
- Existing echo/journal suites unchanged and green: 19 items across
  `test_compact_echo_suffix_linear_scan` / `test_live_snapshot_echo_dedup` /
  `test_live_snapshot_single_parse`, plus 66 neighbouring items
  (`test_issue1361_cancel_data_loss`, `test_issue2565_reasoning_accumulation`,
  `test_issue2713_streaming_segment_flush`, `test_cancel_interrupt`,
  `test_run_journal`).

## Contract Change

None. The matcher's observable behaviour is unchanged (same cut points, same
whitespace-insensitive semantics); only its cost profile changes. No config
knobs, no API surface.

## Risks

- Index drift: if a future writer mutates one of the reasoning buffers
  without feeding (or resetting) its index, the matcher would silently miss
  echoes. Mitigated by keeping the index feed adjacent to every existing
  write site and by the reset-on-flush/reset-on-strip symmetry; the
  equivalence test would catch a dropped feed as a missed echo.
- Memory: the index stores one char plus one int per non-whitespace
  character. On a whitespace-heavy transcript this is bounded by the folded
  length, which is what the transcript would cost folded anyway; the raw
  text is still the larger object.

## Model Used

Hermes Agent (step-5-preview) with direct file edits and pytest verification.
Cost properties were measured with a purpose-built scaling probe (proportional
span x interim count), not a fixed-size fixture; the probe is kept as a
regression test rather than a script.

## Related

#7569 (this PR), #7310 (slow /api/session under load), #7534 (incremental
journal read cache — complementary), #7273/#7291 (metering write-side skip).
2026-09-21 12:34:30 +08:00
happy5318 3369bc6a6d fix(run-journal): match echo suffixes with a window-free backward walk
Review follow-up: the bounded echo probe folded at most len(text)*3 raw characters, so a compact-equivalent suffix whose raw span was stretched by interior whitespace was missed - the echo strip was dropped and the interim text was duplicated in the restored transcript. The same retired window existed in the reverse helper (_strip_compact_echo_suffix), pinned by its own test.

Both now share _find_compact_echo_suffix_start: a backward walk over value/suffix that skips whitespace in value, inspects only the echo span, and allocates nothing. It restores the previous output semantics (origin/master folded the whole reasoning text) and is cheaper than the windowed version (no per-call tail fold).

Tests: new journal-reconstruction regression (a 5000-space-stretched echo must appear exactly once; genuine interim text must not be treated as an echo), the suffix-helper oracle updated to window-free semantics, and the window-contract test now pins that a stretched echo IS stripped.
2026-09-21 12:34:30 +08:00
happy5318 c799716461 perf(run-journal): rebuild live snapshots without quadratic work
`_run_journal_live_snapshot` rebuilds the live UI state for an active run
by replaying the whole durable journal. On a long run (22.7k events /
15MB) that rebuild took ~4.4s per session load and blocked the HTTP
thread - the dominant cost of "switching to a session with a task
running takes forever". Four output-preserving fixes:

1. Locate the journal without parsing it (`find_run_file`) and derive the
   durable summary from the single `read_run_events` parse instead of
   parsing the file twice (`find_run_summary` + `read_run_events`). The
   historical `find_run_summary` lookup is kept as the fallback when no
   journal file exists on disk.
2. Skip `metering` rows in the projection loop (~60% of a long live
   journal's bytes; nothing in the snapshot is built from them), while
   preserving the last_ts watermark they carry.
3. Collect reasoning deltas into a list and materialize lazily instead of
   `reasoning_text += chunk` (quadratic inside these closures: ~2.3s of
   the replay on a 9.7k-chunk run).
4. Fold only a bounded tail of the reasoning transcript for the
   interim-echo suffix check instead of `re.sub` over the whole growing
   text (~1.5s over 100+ probes).

Measured on a 15MB / 22.7k-row live journal: 4.34s -> 0.49s (8.9x) with
byte-identical snapshot output (sha256 of the canonical JSON unchanged).

Related work: #7310 (slow /api/session under load), #7534 (incremental
journal read cache - complementary: this removes one of the two parses
entirely and fixes the replay-loop hotspots), #7273/#7291 (metering
write-side skip - the read-side loop skip here complements them for
legacy journals that still carry metering rows).
2026-09-21 12:34:30 +08:00
Anthony Ruiz 1ee4fc1601 fix: resolve durable compressed sessions before WebUI resume 2026-09-20 17:07:58 +00:00
happy5318 8e31ae9f93 # Session MEDIA auth and snapshot capture inspect Codex commentary sidecars (#7565)
## Thinking Path

The Agent's OpenAI Codex Responses adapter persists user-visible
assistant progress in `codex_message_items` with `phase:
"commentary"` while the outer `content` field is empty. The Agent
already stamps these items with `role: assistant` and a normalized
phase (`agent/codex_responses_adapter.py::_message_item` and
`_OutputScan._message`), and the rendering lane is owned by #6828.

The WebUI's two consumers of those persisted rows, however, only
read the outer `content` field:

- `api/routes.py::_session_media_token_allows_path` flattens
  `message.get("content")` through `_message_content_text()` and
  scans it for `MEDIA:` tokens.
- `api/media_snapshots.py::annotate_media_snapshots` does the
  same when annotating the settled transcript so previews can
  survive an in-place overwrite of the source file.

For a safe image emitted only inside commentary, the browser
request hits the real `_handle_media` route, owns the correct
session_id and exact path, but is denied 403 by the auth predicate
because it cannot see the assistant's persisted `MEDIA:` token.
Symmetrically, the snapshot annotator never captures commentary
artifacts, so a subsequent in-place overwrite rewrites history
without a recoverable snapshot.

The fix is the maintainer's narrow recommendation: one shared
fail-closed public-assistant-text iterator in
`api/media_snapshots.py` (`codex_commentary_text`), consumed by
both the auth predicate and the snapshot annotator. Keeping the
extraction in one neutral helper means the auth and capture paths
cannot drift apart on a future change. The fail-closed predicate
is a strict chain — outer role is `assistant`; item type is
`message`; item role is `assistant`; item phase normalizes to
`commentary`; content is a list; the part is textual `output_text`
with a string `text` value. Every step that fails returns the safe
empty string, so the helper is safe to concatenate with the
existing top-level extraction. User rows, user sidecars, reasoning
items, analysis/final phases, malformed parts, non-string
`text` values, and arbitrary nested fields are all skipped.

## What Changed

- **`api/media_snapshots.py::codex_commentary_text`** — new
  module-level helper that returns the concatenation of all
  public-text `output_text` parts inside assistant-owned
  `codex_message_items` with `phase: "commentary"`. The outer
  row role, item type, item role, item phase, content shape, and
  part type are all gated, so the helper is fail-closed and
  safe to concatenate with the existing top-level text extraction.
- **`api/routes.py::_session_media_token_allows_path`** — inside
  the existing per-message loop, the per-row `text` is now the
  concatenation of `_message_content_text(message.get("content"))`
  and the new `codex_commentary_text(message)`. The exact-path,
  owning-session, safe-MIME, URL-rejection, hard-denied, and
  symlink guards are unchanged. The commentary helper is imported
  lazily inside the loop so this file does not pick up a new
  top-level dependency.
- **`api/media_snapshots.py::annotate_media_snapshots`** — the
  per-row `text` is now the concatenation of the existing
  top-level string `content` and the new commentary helper
  output. The existing `media_re.findall()` and resolve/allow
  paths are unchanged, so otherwise-capture-eligible commentary
  previews are now snapshotted at settle time. The string
  pre-filter on `"MEDIA:" in text` is kept so the helper
  short-circuits before regex extraction when neither source
  contains a token.
- **`tests/test_issue7565_codex_commentary_media.py`** — new
  regression module with eleven tests: a positive
  valid-shape test, six fail-closed tests (user outer role,
  wrong item type, wrong item role, wrong phase, non-output_text
  parts, non-string output_text text), a case-insensitive
  commentary-phase test, a malformed-input test, and two
  integration static tests pinning that the auth predicate and
  the snapshot annotator both consume the new helper.

## Why

- A user-visible artifact that the agent emitted only inside
  commentary was returning 403 at the real `_handle_media` route
  boundary even with the correct owning session_id, and the
  artifact was never snapshotted at settle time. The fix restores
  parity with top-level public assistant content for the
  commentary phase specifically.
- Symmetry between auth and capture matters: a token that auth
  admits must be one capture can stamp, otherwise the served
  bytes can desynchronize from the snapshot pointer and the
  in-place-overwrite survivor feature (#6922) silently breaks.
  Sharing the helper makes drift impossible.

## Verification

```
$ .venv/bin/pytest \
    tests/test_issue7565_codex_commentary_media.py \
    tests/test_media_inline.py \
    tests/test_media_message_snapshots.py \
    -q --no-header
…
============================= 115 passed in 6.16s ==============================
```

Eleven new tests cover the helper contract and the integration
points. The 104 surrounding media_inline / media_message_snapshots
tests stay green.

## Contract

| Outer role | Item shape | Helper result | Auth result | Snapshot result |
| --- | --- | --- | --- | --- |
| `assistant` | `{type:"message", role:"assistant", phase:"commentary", content:[{type:"output_text", text:"…MEDIA:…"}]}` | text surfaced | `MEDIA:` token found; existing exact-path / safe-MIME / owning-session guards apply | `MEDIA:` token snapshotted at settle time |
| `assistant` | `{type:"message", role:"assistant", phase:"analysis", content:[{type:"output_text", text:"…"}]}` | empty | unchanged from existing top-level-only behavior | unchanged |
| `assistant` | `{type:"message", role:"assistant", phase:"final", content:[{type:"output_text", text:"…"}]}` | empty | unchanged | unchanged |
| `assistant` | `{type:"reasoning", …}` | empty | unchanged | unchanged |
| `assistant` | item role `user` | empty | unchanged | unchanged |
| `assistant` | item content not a list | empty | unchanged | unchanged |
| `assistant` | part type `tool_call` / `image` / `input_text` (even with text containing `MEDIA:…`) | empty | unchanged | unchanged |
| `assistant` | `output_text` with non-string `text` (dict / list / number) | empty | unchanged | unchanged |
| `user` (outer role) | any sidecar | empty | unchanged (user rows already skipped) | unchanged |
| `assistant` | no `codex_message_items` | empty | unchanged | unchanged |
| malformed (`None`, `""`, list, string, missing role) | n/a | empty | unchanged | unchanged |

The fix only widens the inspection surface for the one
public-text sidecar shape; everything else stays exactly as it
was, so existing safe-MIME, hard-denied, exact-path-equality,
owning-session, URL-rejection, and symlink guards remain intact.

## Risks

- **Commentary widening the inspection surface**: yes, by design.
  The widening is narrowly scoped to the outer-assistant
  / item-message / item-role-assistant / phase-commentary /
  part-output_text / string-text chain. The new tests pin every
  step in that chain and every out-of-chain shape that must
  remain rejected.
- **Existing safe-MIME / hard-denied / exact-path / symlink
  guards weakened?** No. The `media_capture_allowed` predicate,
  the `MIME_MAP` lookup, the `Path(ref).expanduser().resolve() ==
  target_resolved` equality check, the `if "://" in ref` URL
  rejection, the hard-denied state/secret paths, and the symlink
  protections are all unchanged. The only change is what text
  the regex sees.
- **Agent producer change**: the helper reads the same shape
  the agent already stamps (`role: assistant`, `phase:
  "commentary"`, `content: [{type: "output_text", text: "..."}]`).
  A future producer change that adds a new phase or part type
  requires an explicit follow-up PR; the helper's comments
  document the precise chain.
- **Helper duplication drift risk avoided**: by placing the
  helper in `api/media_snapshots.py` (the storage half) and
  having `api/routes.py` import it lazily inside the loop, both
  consumers go through one definition. A follow-up change to
  the helper's contract automatically applies to both call sites.

## Model Used

- Generated with `MiniMax-M3` via `happy5318/hermes-webui` worktree
  `/tmp/webui-worktrees/pr7565-codex-media`, branch
  `fix/codex-commentary-media` (off `origin/master` via `ghfast`
  mirror — `origin` was still TLS recv-failing at PR creation
  time; the mirror was used for `git fetch` and `git worktree add`
  only; push goes through the `fork` remote as usual).

## Related

- Closes #7565
- Surface code:
  - `api/media_snapshots.py` (new) `codex_commentary_text` helper
  - `api/routes.py::_session_media_token_allows_path` (modified
    to concatenate the helper output)
  - `api/media_snapshots.py::annotate_media_snapshots` (modified
    to concatenate the helper output)
- Adjacent lanes (intentionally not folded in):
  - #7294 / #7297 (safe-MIME, session_id, snapshot query
    parameters) — independent contract
  - #6828 (persisted-commentary rendering / recovery) — frontend
    UI; the WebUI auth predicate does not assume or expand its
    rendering scope
  - Open PRs #7367 and #6923 — overlap the token-extraction sites
    but neither implements this commentary ownership rule, per
    the maintainer's open-PR search
- Test surface: new `tests/test_issue7565_codex_commentary_media.py`
2026-09-20 08:49:36 +00:00
happy5318 3163a91d1a # Cron edit/duplicate must round-trip a parseable schedule; bad input returns 400 (#7352)
## Thinking Path

The WebUI populates the editable schedule field from `schedule_display`,
which is the human-readable label `once at 2026-08-28 16:00` for one-shot
jobs. `saveCronForm()` then submits that exact string back to
`POST /api/crons/update`. The Agent's `cron.jobs.parse_schedule()`
rejects the display form with `ValueError` (it only accepts the canonical
ISO timestamp). The route's earlier `try/except ValueError` block
covered input normalization but not the `update_job()` call, so the
parser exception escaped as HTTP 500.

There are two independent failures to fix:

1. **Frontend** ships the unsafe display label as editable input. Both
   `openCronEdit()` (`static/panels.js:1593`) and `duplicateCurrentCron()`
   (`static/panels.js:1532`) use the same `schedule_display ||
   schedule.expression` fallback. The cleanest fix is a shared helper
   that returns the canonical Agent-parseable value by schedule kind,
   matching the Agent's own schema:
   - `once` → `schedule.run_at`
   - recurring cron → `schedule.expr` (current schema; `expression`
     accepted for legacy payloads)
   - interval / legacy → the stored value, with `schedule_display`
     kept as a final fallback only when it isn't the `once at …` label.
2. **Backend** converts `ValueError` from `update_job()` into HTTP 400
   with the parser message, so the WebUI can show a real validation
   toast instead of an opaque Internal Server Error. The 404 path
   (job not found) is preserved.

## What Changed

- **`static/panels.js`** — add `_cronScheduleForEdit(job)` helper and
  use it in both `openCronEdit()` and `duplicateCurrentCron()`. The
  helper returns `schedule.run_at` for one-shot jobs,
  `schedule.expr` (or legacy `schedule.expression`) for recurring
  cron, and falls back to `schedule_display` only when it is not the
  `once at …` presentation form. `saveCronForm()` is unchanged — it
  still submits `schEl.value`, which is now the canonical field
  instead of the display label.
- **`api/routes.py::_handle_cron_update`** — wrap the `update_job()`
  call in `try/except ValueError` and return HTTP 400 with the parser
  message. The earlier normalization try block stays untouched; the
  404 check (`if not job`) sits outside the new try block so a missing
  job still returns 404.
- **`tests/test_cron_edit_round_trip.py`** — new regression module
  with seven tests covering the helper contract (helper exists, both
  call sites use it, `saveCronForm` still submits the field by name),
  the route's 400-on-`ValueError` behavior, the preserved 404 path,
  and the happy-path 200.

## Why

- Editing a one-shot cron job's scheduled time has been returning
  HTTP 500 since the WebUI's initial public release (per #7352
  historical note), because the editable field round-trips the
  `once at …` display label straight to the Agent parser.
- The backend regression is independent: even with a correct
  frontend, any user-typed invalid schedule text (or a programmatic
  caller) used to produce a 500 instead of a validation response.
  Both halves are required to deliver a "no more 500" user
  experience.

## Verification

```
$ .venv/bin/pytest \
    tests/test_cron_edit_round_trip.py \
    tests/test_cron_no_agent_edit.py \
    tests/test_cron_model_override.py \
    tests/test_cron_model_provider_picker.py \
    tests/test_issue617_cron_profile_selector.py \
    -v --no-header
…
============================== 27 passed in 4.84s ==============================
```

Seven new tests cover both layers; twenty surrounding cron tests
(no-agent edit, model override, model/provider picker, profile
selector) stay green.

## Contract

| Scenario | Before | After |
| --- | --- | --- |
| Open one-shot job for edit, change time, save | 500 (display label round-trips to parser) | 200, persisted with new `run_at` |
| Open recurring cron job for edit | 200 (legacy `schedule.expression` chain) | 200, unchanged canonical field (`schedule.expr` first, `expression` as legacy fallback) |
| Duplicate a one-shot job | 500 on first save | 200, populated with `schedule.run_at` from the source job |
| Submit a `once at …` schedule value (programmatic / pre-fix frontend) | 500 | 400 with the parser message |
| Submit any other unparseable schedule | 500 | 400 with the parser message |
| Update a job that does not exist | 404 | 404 (preserved) |
| Update a job with a valid canonical schedule | 200 | 200 (preserved) |

## Risks

- **Frontend helper change**: I switched the editable field source
  for both edit and duplicate in one commit. `saveCronForm()` is
  untouched, so anything already rendered with the legacy chain
  (rare in practice — the helper is only called from these two
  functions) keeps working. The regression `test_save_cron_form_still_submits_schedule_value`
  pins the submit field name.
- **New `try/except ValueError` in `_handle_cron_update`**: the
  earlier normalization try block (lines 24986–24998) is left
  exactly as-is. The new try block sits directly around
  `update_job()` only, so a `ValueError` from input parsing still
  surfaces before any persistence side effect. The 404 check is
  outside the new try, so a missing-job path cannot be misreported
  as 400. Both `test_cron_update_preserves_404_when_job_missing` and
  `test_cron_update_valid_schedule_still_succeeds` pin those
  contracts.
- **Legacy `schedule.expression` payloads**: accepted as a
  back-compat fallback in `_cronScheduleForEdit` and in the route
  (the Agent schema is `expr`, but the parser still handles
  `expression` per the existing contract). If you prefer to drop
  the legacy fallback, the helper can be trimmed to `sched.expr`
  only.

## Model Used

- Generated with `MiniMax-M3` via `happy5318/hermes-webui` worktree
  `/tmp/webui-worktrees/pr7352-cron-edit`, branch
  `fix/cron-edit-round-trip` (off `origin/master`).

## Related

- Closes #7352
- Surface code:
  - `static/panels.js:1510` `duplicateCurrentCron` (uses helper)
  - `static/panels.js:1585` `openCronEdit` (uses helper)
  - `static/panels.js:1905` `saveCronForm` (submit path, unchanged)
  - `api/routes.py:24979` `_handle_cron_update` (new try/except)
  - `cron/jobs.py:785` `parse_schedule` (rejects the display form;
    Agent-side, not modified in this PR)
  - `cron/jobs.py:2199` `update_job` (the call that raises)
- Test surface: new `tests/test_cron_edit_round_trip.py`
2026-09-20 03:05:14 +00:00
n eb7f5df284 Merge PR #7394: fix(tts): honor the persisted TTS engine when requests omit it 2026-09-20 00:02:05 +00:00
James Drake 047dfa40a5 fix(routes): never mask the launch error during cleanup
_cleanup_chat_start_launch_failure runs inside the launch-failure handler,
but the session re-resolve only caught KeyError. An unreadable or
undeserializable session store raises something else, and that escaped the
helper, replacing the worker-launch error the caller is about to report and
aborting the reset after the registries were already cleared.

Contain the resolution-and-reset so the helper cannot raise. The KeyError
branch keeps its meaning (deleted session, return without saving); anything
else is logged at debug like the existing save() failure path.
2026-09-19 00:29:07 -05:00
James Drake b5c0136545 fix: run launch-failure session reset under session lock
The session-field half of _cleanup_chat_start_launch_failure mutated and
saved the passed-in session object without holding the per-session lock and
without re-resolving the canonical session. A concurrent successor turn
(same session_id, new stream_id) or a delete could be clobbered: pending
fields wiped or a deleted session resurrected via save().

Mirror the established #1533 pattern (routes.py:3077) and anchor-scene
write guard (routes.py:5140): hold _get_session_agent_lock, re-resolve with
get_session(), early-return on KeyError (session deleted) or when the
canonical no longer points at the failed stream, then reset pending fields
on the canonical object only.

Add deterministic race coverage: successor admitted during the
check-then-clear window keeps its pending fields and is not saved away, and
a deleted session is not resurrected by save(). Both tests fail against the
previous implementation.
2026-09-18 08:22:11 -05:00
James Drake c748049bf7 fix: clean up failed chat worker launches 2026-09-18 08:22:11 -05:00
Anthony Ruiz ef9341b2ab fix(sqlite): keep state.db readers strictly read-only and stop idle polling
Readers never upgrade to writers. open_state_db_readonly() opens
file:...?mode=ro only; a read-only open failure propagates instead of
silently retrying with a write-capable handle on the multi-GiB WAL
database the agent is streaming into. The session listing no longer
self-heals idx_messages_session with CREATE INDEX (a writer lock held for
minutes on a large messages table) — a missing index degrades to the
pre-aggregated path. Cron sidebar info, insights and deep health checks in
api/routes.py use the same read-only opener.

Watcher (api/gateway_watcher.py): _poll_once returns without touching the
database when nobody is subscribed, and forces a fresh full projection on
the next subscription.

Maintenance (scripts/ensure_state_db_read_indexes.py): the covering read
indexes (idx_messages_session, idx_messages_session_role with NOCASE,
idx_sessions_webui_fingerprint) are created only under --confirm-drained,
optionally holding an exclusive flock on --lock-file for deployments that
serialise agent turns on a lock file; existing indexes are verified for
shape/collation and coverage (EXPLAIN QUERY PLAN), never silently
replaced.

Tests: tests/test_state_db_readonly_readers.py (read-only failure never
retries writable; no poll without subscribers; listing creates no index;
index tool requires drain, is idempotent, works without a lock file and
refuses a held lock; cron sidebar read is mode=ro). Neighbouring tests
that asserted the retired writable fallback / self-heal writer
(test_issue3887_index_prime, test_issue5455_listing_readonly_connection,
test_issue5455_lineage_readonly_reads, test_issue2628_cli_sessions_perf,
test_issue1494_state_db_fd_leak) now assert the read-only contract;
test_issue3506 subscribes before polling.
2026-09-18 00:59:17 +00:00
nesquena-hermesandmikemchargue 818c392ed4 fix: resolve Hermes Agent names moved by the Sep 2026 decomposition (#7574)
The Agent's September 2026 decomposition moved several names the WebUI uses into
sibling modules. The old import paths resolved only through PEP 562 __getattr__
compat pointers whose removal date (2026-09-14) has already passed. Every affected
WebUI call site is wrapped in a broad except, so once those pointers go away nothing
crashes -- features silently degrade: MCP tools vanish from turns, the per-turn
approval session key never binds, Claude Code credential linking stops, and the
kanban bridge raises AttributeError.

Adds api/agent_compat.py::agent_attr(owner, name, home, default), which resolves a
moved name via the original module's own namespace (pre-split Agents and test stubs),
then the new home module (split Agents, with or without the pointers), then plain
attribute access -- and routes the 8 affected call sites through it.

Gate: Codex SAFE TO SHIP. All 12 mappings verified against the installed Agent's
authoritative compat_manifest.json; owner-wins precedence is correct because PEP 562
pointers do not populate vars(owner), so current pointers are bypassed without
warnings; the only explicit defaults are the pre-existing optional kanban fallbacks
and remain semantically equivalent; turn-approval identity, MCP discovery/reload/status,
OAuth credential reading, LM Studio probing, and all three kanban paths preserve their
previous values and control flow; cached import_module costs ~1.36us and the hottest
site is a 0.3s kanban poll, not per-token streaming (caching the callable would break
supported module replacement/monkeypatching). Tests are non-vacuous across pre-split,
pointer-present, and pointer-removed shapes. Full suite 15,597 pass with only the known
box-baseline singleton. Maintainer proof: agent_attr resolves the moved name with zero
HermesPluginCompatWarning while a clean-master control still resolves via the pointer
and warns.

Co-authored-by: mikemchargue <mikemchargue@users.noreply.github.com>
2026-09-17 21:57:25 +00:00
Mike McHargueandClaude Opus 5 1f7e5e78da fix: resolve Hermes Agent names moved by the Sep 2026 decomposition
Hermes Agent moved several names the WebUI uses into sibling modules
(tools.approval_context, tools.mcp_tool_discovery/_lifecycle,
hermes_cli.kanban_db_connect/_dispatch, hermes_cli.models_local,
agent.anthropic_credentials). The old paths only resolved through
PLUGIN-COMPAT __getattr__ pointers that warn on every use and are removed
on schedule by reverting one commit.

With the pointers removed, every affected WebUI call site is wrapped in a
broad except, so nothing crashes; features silently degrade instead:
per-turn approval session-key binding is skipped, MCP discovery/status
return nothing, /reload-mcp reports "MCP runtime unavailable", Claude Code
credential linking returns None, every kanban connection raises
AttributeError, and the dispatcher reports "unavailable".

Add api.agent_compat.agent_attr, which resolves a moved name from the
original module's own namespace first (pre-split Agents and existing test
stubs), then from its new home (split Agents, with or without pointers),
then plain attribute access (non-module doubles). Importing only the new
modules would have broken pre-split Agents and bypassed the sys.modules
stubs the test suite installs on the original modules.

Names still native to their original module (_servers, _lock,
has_blocking_approval, is_session_yolo_enabled, register/unregister
gateway notify, the rest of kanban_db) are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-17 21:26:50 +00:00
carlotestor 8c5ee95dab fix(media): serve .zip archives through /api/media
MEDIA:/path/to/file.zip links always failed to open in chat. .zip was
absent from MIME_MAP, so it resolved to application/octet-stream, which
_SESSION_MEDIA_TOKEN_TYPES rejects — the browser rendered a download
link that the server then refused.

Map .zip to application/zip and add an _ARCHIVE_TYPES set to the session
MEDIA: token allow-list. Archives are deliberately kept out of
_INLINE_PREVIEW_TYPES and _INLINE_IMAGE_TYPES so they are always sent as
Content-Disposition: attachment and never rendered inline.
2026-09-17 20:56:30 +00:00
Matt Van Horn a375aa9356 fix: honor the persisted TTS engine when requests omit it
Fixes #7391
2026-09-16 09:04:24 -07:00
Skawsk59andSkawsk59 723146cbfe fix: serve SPA for /sessions route (404 after login with auth enabled) (#7598)
Serve the SPA shell for /sessions so the post-login redirect (when password auth is enabled) no longer 404s.

Co-authored-by: Skawsk59 <Skawsk59@users.noreply.github.com>
2026-09-16 00:24:09 -07:00
b3nw fb2e8b0088 fix(custom-providers): route @custom:<slug> models to their own base_url (#7319)
Route @custom:<slug> models to their owning custom provider's base_url so a model picked from a second custom provider group resolves correctly instead of 400ing.

Co-authored-by: b3nw <b3nw@users.noreply.github.com>
2026-09-15 20:05:40 -07:00
alfred-rootson b71c88f292 fix(workspace): preserve remote POSIX paths across Session & streaming callsites (#7131) (#7168)
Closes the remote POSIX workspace path preservation gap across Session and streaming. Profile-aware runtime path resolution.

Co-authored-by: alfred-rootson <alfred.wojtasordzik@gmail.com>
2026-09-15 15:37:11 -07:00