Serve the SPA shell for /sessions so the post-login redirect (when password auth is enabled) no longer 404s.
Co-authored-by: Skawsk59 <Skawsk59@users.noreply.github.com>
Route @custom:<slug> models to their owning custom provider's base_url so a model picked from a second custom provider group resolves correctly instead of 400ing.
Co-authored-by: b3nw <b3nw@users.noreply.github.com>
Closes the remote POSIX workspace path preservation gap across Session and streaming. Profile-aware runtime path resolution.
Co-authored-by: alfred-rootson <alfred.wojtasordzik@gmail.com>
* fix(cron): restore messaging platforms in the delivery picker
The Agent moved _KNOWN_DELIVERY_PLATFORMS from cron.scheduler to
cron.scheduler_delivery. _handle_cron_delivery_options imported the old
path inside a bare except and fell back to an empty frozenset, so the
import error was swallowed and GET /api/crons/delivery-options silently
returned only local and origin — telegram, discord, slack, feishu and
every other messaging platform disappeared from the cron delivery picker
with no error surfaced anywhere.
Resolve the authority across both module paths (current first, legacy
fallback) so the picker keeps working across Agent versions, and add a
regression test that pins the resolution order and cross-checks the
endpoint output against the resolved authority.
* test(cron): skip the authority-resolution test where the Agent is absent
CI runs agent-free (tests/browser_smoke.py), so neither cron.scheduler_delivery
nor cron.scheduler is importable there and the authority resolves empty. The
new regression test asserted non-empty and therefore failed shard 3 on both
Python versions. Skip instead: with no cron package there is no authority to
resolve and nothing meaningful to assert. The test still fails loudly wherever
the Agent IS installed, which is where a future relocation would bite.
* test(cron): register the new test in _AGENT_DEPENDENT_TESTS
Use the project's existing skip mechanism (tests/conftest.py) rather than an
ad-hoc inline pytest.skip, matching the five sibling delivery-options tests.
CI runs agent-free, so there is no cron package and no authority to resolve
there; with the Agent installed the test still asserts loudly, which is the
environment where a future relocation would actually bite.
Verified both directions: 6/6 pass with the Agent present, and forcing
AGENT_MODULES_AVAILABLE=False skips all 6 (was 5) as CI will.
---------
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
* refactor(routes): extract the 585-line GET /api/session handler to _handle_session_get
Pure move out of handle_get's if/elif chain into a named module-level handler, matching the pattern the other ~200 routes already use. Body is verbatim (one-level dedent only); AST identity of the moved statements verified; no behavior change — the GET /api/session test slice passes identically before and after.
* test: repoint four source-window tests to _handle_session_get
The moved block's source-window extractors (chat_start synthesiser/read-only
pins, regressions runtime-streaming-state window, issue1436 load-path
markers, security-redaction inspect) now anchor on the extracted handler;
assertions unchanged.
* test: repoint messaging-session loader pin to _handle_session_get
---------
Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
* Fix geometric duplication of recovered reasoning-only rows at turn settle
_restore_display_reasoning_metadata() re-inserted every display-only
reasoning row at its API-safe position in the new result. After context
compaction that position is past the end of result_messages, so the row
was appended after the new reply; _message_identity() is None for an
empty assistant row, so the display merge could not dedup it. Each
settled turn therefore re-added every existing empty
_recovered_from_run_journal anchor once more (1,1,2,4,8,... blocks).
Only restore a reasoning-only row in front of its own API-safe
successor; a compacted result with no aligned slot is skipped.
Refs #7388, #7416 (independent: that PR fixes linear journal replay).
* tests: keep docstrings within the 5-line block budget
* Prefer stable ids when aligning a reasoning row's successor at restore
The successor-alignment guard compared the historical successor with the
aligned current row via _message_identity(), which ignores stable ids and
returns equal keys for repeated prompts ("continue"). A distinct current
turn with the same text false-matched the historical successor, every
recovered anchor before it was re-inserted, and 64 anchors became 128
after one restore->merge settle (nesquena-hermes gate on ed2076eda).
Alignment is now stable-id-first: when either row carries an id, both
must carry the same id; _message_identity() is the fallback only when
neither row has one. Adds a production-order regression through
_settle_result_messages() with 64 anchors, stable ids and repeated
"continue" prompts (RED on ed2076eda and origin/master: 128 != 64), plus
a unit test pinning all four id-presence branches.
* Fail closed on stable-id identity: positive-int only, unique, no content fallback
_restore_display_reasoning_metadata() compared ids with raw equality, so a
persisted historical successor with id=True (or 1.0) matched a distinct
current row minted as id=1 and restored the reasoning card before the
wrong turn. Ids are now accepted only when type(value) is int and > 0,
both rows must carry a valid equal id whenever either has an id field,
and the id must be the sole owner in both API-safe projections.
_assign_stable_message_ids() re-mints invalid ids so they cannot survive
into a collision with minted integers.
* tests: explicit zip strict= for the ruff B905 gate
* Resolve the active-turn boundary before restoration; never carry historical ids/rows into it
A historical assistant successor with the same content as the distinct
current assistant ("Done." / "Done.") let _restore_reasoning_metadata copy
the historical stable id onto the current row; the uniqueness guard then
accepted that forged identity and 64 recovered cards settled to 128.
_active_turn_boundary() is decided once in _prepare_marker_clean_writeback
before any restore (token > strict context prefix > last matching user row
> 0) and threaded through _settle_result_messages into both restores.
Rows at/after it receive no historical id/metadata and no historical
reasoning-only row. Assistant-only results are boundary zero.
* Keep the pinned _restore_reasoning_metadata signature; boundary goes through _restore_reasoning_metadata_before_boundary
tests/test_sprint49.py pins the two-arg signature as source text, so the
boundary-aware core lives in _restore_reasoning_metadata_before_boundary
and the original name stays a thin no-boundary wrapper.
* Content-only prefix is not turn ownership: fail closed to boundary 0; sync route passes result/Agent turn authority
- _active_turn_boundary: a previous-context prefix matched by _message_identity
(ids/timestamps ignored) only narrows where the current user may be; the
boundary is the prompt-matching user AT OR AFTER that prefix, else 0.
- _handle_chat_sync: resolve authority via _resolve_active_turn_authority
(result, agent) instead of hard-coding None.
- Regressions: three-row current-only collision (boundary 0, one 64 block),
exact-authority + append-only controls, ID-less-successor settle/save/load
pinned at the persisted-display boundary, sync-route authority pass-through.
---------
Co-authored-by: carlotestor <carlotestor@users.noreply.github.com>
Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
* rebase #7160 onto master (report unverified Agent updates, no auto-restart)
Co-authored-by: hrdwdmrbl <hrdwdmrbl@users.noreply.github.com>
* harden agent-update marker read: no-follow, fstat regular-file, bounded (fixes gate CORE)
The Agent update marker is attacker-adjacent shared state. _read_live_agent_update()
used Path.read_text(), which follows symlinks, blocks forever on a FIFO, and reads an
unbounded regular file — a rejected/stale-runtime request could hang or OOM. Now opens
O_RDONLY|O_NONBLOCK|O_NOFOLLOW, fstat-verifies a small regular file, and reads a bounded
max, classifying anything else as 'unknown' (fail-closed). Adds FIFO/oversized/symlink/
happy-path regression tests.
Co-authored-by: hrdwdmrbl <hrdwdmrbl@users.noreply.github.com>
* fix marker-read portability: gate os.open fast-path on O_NONBLOCK+O_NOFOLLOW
os.O_NONBLOCK is Unix-only; accessing it unconditionally raised AttributeError
on native Windows, breaking every stale-revision barrier (chat/compression/etc).
Now both flags resolve via getattr and the os.open() fast path is taken only
when BOTH are available (_MARKER_SAFE_OPEN_AVAILABLE); otherwise fall back to an
lstat-only classification (absent vs unknown) so a fallback platform can never
symlink-traverse or crash. Adds a Windows-fallback regression test.
Co-authored-by: hrdwdmrbl <hrdwdmrbl@users.noreply.github.com>
* docs(changelog): stamp #7160 stale-runtime report + hardened marker read
---------
Co-authored-by: n <a@n>
Co-authored-by: hrdwdmrbl <hrdwdmrbl@users.noreply.github.com>
Preserve nested request isolation, make the response allowlist single-source,
and strip heavy fields from existing index rows after reconciliation.
Co-authored-by: Martin Dell <martindell@users.noreply.github.com>
* fix(streaming): lock-disciplined STREAMS reads via peek_stream()
Writers pop STREAMS under STREAMS_LOCK, but four hot reads used bare
STREAMS.get(): the streaming worker start, the gateway run start, and the two
SSE attach paths in routes.py. Each is individually saved today by GIL-atomic
dict.get plus a None-guard, but the check-then-act windows (get → register,
get → subscribe) race teardown pops, and the bare pattern invites unprotected
copies. peek_stream() centralizes the locked read; all four sites migrate with
no behavioral change.
* test: dual-patch config.STREAMS alongside routes.STREAMS in SSE seam tests
peek_stream() reads the canonical api.config.STREAMS registry; tests that
inject a fake registry by patching only the routes-level re-export left the
helper reading the real (empty) registry and hung the SSE loop to the
pytest-timeout ceiling. Mirror the test_issue6623 pattern: patch both
bindings.
* docs(changelog): stamp #7339 lock-disciplined STREAMS reads
---------
Co-authored-by: Sebastian B <sb@wmc.sh>
Co-authored-by: n <a@n>
The not-found reply capped available_skills at 20 with no total and no
truncation marker, so a caller with more skills than the cap was handed a
partial list that reads as the complete set. A skill sorting past the cut
looked uninstalled.
Keep the bound; stop it lying. The payload now carries total_skills and
available_skills_truncated, and the hint names how many of how many were
shown, matching the (list, truncated) shape already used for capped commit
subject lists in api/updates.py.
hermes-agent v0.21 removed agent.anthropic_adapter.normalize_anthropic_response and the AIAgent._normalize_codex_response method; the sanctioned out-of-band contract is now agent._get_transport(<mode>).normalize_response(...) returning NormalizedResponse (.content). Port the anthropic_messages and codex_responses branches of generate_title_raw_via_agent (api/streaming.py) and _agent_text_completion (api/routes.py); on agent v0.21 those branches currently die on import/attribute errors and silently fall back, degrading title generation and handoff summaries. build_anthropic_kwargs, _anthropic_messages_create, and _run_codex_stream are unchanged in v0.21 and stay as-is.
read_body() silently turned malformed JSON into {} (clients then got a
misleading 'Missing field' 400) and passed non-dict JSON through to handlers
where body.get() raised AttributeError and surfaced as a generic 500.
Raise ValueError instead — handle_post already maps it to a clean 400 — and
give the PATCH/DELETE/PUT dispatchers the same mapping they lacked. Empty
bodies keep the documented {} default.
* perf(session-load): memoize the compression-lineage display stitch
_webui_sidecar_lineage_messages_for_display re-merged multi-thousand-row
parent+child transcripts on EVERY GET /api/session (measured 5.1s per
request on a real 5.2k-message lineage, py-spy: merge_session_messages_
append_only + _matching_visible_duplicate dominate). The merge result is
now cached per session id, validity keyed by the exact stat signature of
the child sidecar AND every snapshot parent involved, bounded LRU(16).
Any write to any involved sidecar invalidates naturally; cache hits and
the populating call hand out shallow-copied rows so caller-side display
metadata cannot corrupt cached state.
Measured: 5.17s cold -> 0.002s warm on the real session; invalidation
after touch re-merges. Tests: tests/test_lineage_display_cache.py
(correctness, copy isolation, child/parent invalidation, no pollution
for lineage-less sessions).
* perf(api): memoize inactive-session display merges
* perf(api): skip state load on cached transcript reopen
* test(api): remove unused lineage-cache import
* fix(api): reject streaming key without state database
* test(api): isolate display cache fixture
* fix(api): bind cache publication to read signature
* fix(api): harden display cache invalidation
* fix(api): bind cache hits to lineage provenance
* fix(api): close remaining display cache scope races
* fix(api): scope display cache revision to target
* fix(api): evict unverifiable lineage provenance
* fix(api): keep incomplete lineage provenance uncached
---------
Co-authored-by: Anthony Ruiz <ruizanthony@users.noreply.github.com>
* fix(workspace): new chat inherits the current conversation's workspace
`newSession()` resolved the workspace of a new chat as:
switchWs || S._profileDefaultWorkspace || S.session.workspace
`S._profileDefaultWorkspace` is global to the profile, hydrated from
`/api/profile/active`. Preferring it over the current conversation meant a new
chat opened from a MES conversation landed on the profile-global value instead
of MES. With several conversations running in parallel on different workspaces,
the context the user was actually in was systematically lost.
The precedence is worse than it looks, because that "profile default" is not a
stable configured value: `get_profile_default_workspace()` reads
`last_workspace.txt` BEFORE the configured key (api/workspace.py ~419-427), and
for the `default` profile that file is the GLOBAL one. `set_last_workspace()`
rewrites it on every `/api/chat/start` and `/api/session/update` of ANY
conversation. So "profile default wins over the current session" degrades in
practice to "whatever workspace another conversation last used wins over the
conversation I am in" -- observed live: the pointer flipped mid-operation
because a concurrent turn wrote to it.
New precedence:
switchWs || S.session.workspace || S._profileDefaultWorkspace
Preserved invariants:
- the one-shot profile-switch flag still wins first and is still consumed (#823);
- a blank page (no loaded session) still falls back to the profile default,
which remains required for the empty-composer workspace chip (#804/#5169);
- `_profileDefaultWorkspace` is never nulled by `newSession()` (#823).
This deliberately diverges from upstream #4755, which made the profile default
win. That decision assumed a stable configured default; on a shared/global
`last_workspace.txt` it inverts its own intent. The upstream test is rewritten
with the rationale inline so a future rebase does not silently restore the bug,
plus a new test that evaluates the real expression under node and proves the
blank-page fallback still holds.
Pre-existing failures in tests/test_workspace_stale_recovery.py (3) were
verified identical with and without this change.
* fix(workspace): recover stale inherited workspace
---------
Co-authored-by: Anthony Ruiz <ruizanthony@users.noreply.github.com>
* fix: route overlapping custom_providers models by active endpoint
When two custom_providers[] entries list the same bare model id (e.g.
dogapi and packyapi both advertising 'claude-sonnet-5'),
resolve_model_provider scanned them in config write order and returned
the first match, silently ignoring the active provider's base_url. A
user whose model.base_url pointed at packyapi got routed to dogapi
because dogapi appeared earlier in config.
Now, when the active provider resolves to a named custom provider
(custom:<slug>, including via model.base_url -> named-slug matching),
that provider claims models it owns before the ordered scan runs. Falls
through to the legacy first-match scan when the active provider is bare
'custom' with no base_url to disambiguate, so existing single-endpoint
setups are unaffected.
Adds 3 regression tests covering: shared-model active-endpoint wins,
provider-unique model still routes by ownership, and bare-custom
no-base_url preserves legacy order.
* fix: extend active-endpoint routing to aux, handoff, and slug-collision paths
Review follow-up: the overlapping-id fix must reach three sibling
resolution paths that still routed on the wrong endpoint.
1. Auxiliary slot base_url (config.py set_auxiliary_model): a bare
resolve_model_provider(model) ignored the SELECTED aux provider, so a
custom:A slot was persisted with the active custom:B endpoint when both
listed the model. Look up the selected provider's own custom entry via
resolve_custom_provider_connection. (NOT model_with_provider_context
here: the @custom:<slug>:model form resolves base_url to None, which
would drop the base_url entirely.)
2. Handoff summary (routes.py): read s_obj.model_provider and pass
model_with_provider_context(model, model_provider) so a session pinned
to custom:A is not rerouted to the active custom:B. base_url is
backfilled from the provider's own custom entry downstream.
3. Normalized-name slug collision (config.py resolve_model_provider): two
distinct provider names can normalize to the same slug (e.g. "Foo Bar"
and "foo-bar"). The active-slug guard now matches the EXACT entry by
base_url instead of re-deriving it from the non-unique slug, and fails
closed (falls through to the ordered scan) on genuine ambiguity.
Adds 4 regression tests: normalized-slug base_url disambiguation,
slug-collision fail-closed, aux-slot selected-provider base_url, and
session-provider-context handoff routing.
* fix: fail closed on custom-provider slug collisions and break aux deadlock
resolve_model_provider() returns a provider slug (custom:<slug>) and the
credential lookup (resolve_custom_provider_connection) later resolves the
API key from the FIRST same-slug entry regardless of base_url. So when two
distinct provider names normalize to the same slug (e.g. "Foo Bar" and
"foo-bar") and both list the requested model, returning the slug silently
pairs one entry's endpoint with another entry's credential. Now raise a
clear AmbiguousCustomProviderError instead of guessing, on both the
active-slug guard and the ordered-scan fall-through paths.
Also: an explicit, unambiguous named custom provider now stays authoritative
even when model.base_url is stale/absent and points at neither entry — a
stale URL is no longer evidence to demote it to config order.
And fix a self-deadlock in set_auxiliary_model(): it held the non-reentrant
_cfg_lock while calling resolve_custom_provider_connection() -> get_config()
-> reload_config_if_stale(), which re-acquires the same lock. The selected
custom provider's base_url is now resolved inline from the config_data
already loaded under the lock.
Tests: collision cases now assert the ambiguity error; added a stale-URL
explicit-provider regression and a watchdog-guarded aux no-deadlock
regression (both fail before the fix).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: fail closed on custom-provider slug collisions across all four paths
The previous cut only guarded the symmetric bare-model case, and it built
collision membership from entries that OWN the requested model. But credential
resolution scans ALL same-slug entries and first-matches regardless of
ownership, so several slug-only boundaries could still pair one entry's
endpoint with another entry's credential — including the asymmetric case where
only one colliding entry lists the model.
Introduce one shared, pure/lock-safe helper (_custom_provider_slug_key +
_unique_custom_provider_entry) that groups ALL named custom_providers entries by
a single canonical slug normalization and raises AmbiguousCustomProviderError
whenever a consumed slug maps to >=2 entries. Apply it at every slug-only
boundary so endpoint and credential can never come from different entries:
- bare resolution (resolve_model_provider): all-entry membership for both the
active-slug guard and the ordered ownership scan (was owner-only).
- provider-qualified @custom:<slug>:model return (the session/send/handoff
model_with_provider_context shape), which previously bypassed the check.
- credential lookup (resolve_custom_provider_connection), replacing its private
first-match-by-slug scan and its duplicate _slugify.
- auxiliary persistence (set_auxiliary_model), replacing its own inline
first-match-by-slug; the ambiguity now propagates (fails the save) instead of
being swallowed, while other best-effort errors stay non-fatal. Still resolves
from the in-scope config_data so it remains lock-safe (no deadlock regression).
Tests: add fail-closed regressions for the asymmetric A-first-non-owner/B-owner
bare case (active-endpoint and bare-custom paths), the model_with_provider_context
qualified handoff shape, the credential boundary called directly, and the
auxiliary-save ambiguity (asserts nothing is persisted). All five fail before
this change and pass after.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: one canonical custom-provider slug identity, point-of-return ambiguity, profile-scoped credentials
Deep re-gate follow-up (5 findings):
1. Collision detection now derives from the single slug PRODUCER
(_custom_provider_slug_from_name) instead of a looser private key, so
producer-level collisions on punctuated/parenthesized names (e.g.
'Foo (Bar)' vs 'foo-bar', both -> custom:foo-bar) are detected. Previously
the looser key normalized 'Foo (Bar)' to 'foo-(bar)' and missed the
collision, reintroducing the endpoint-A/credential-B bug for those names.
2. The all-entry ambiguity check now runs ONLY at the point a custom:<slug> is
about to be returned (via a _finalize() guard covering every custom-return in
resolve_model_provider, including the final active-provider fallback), not up
front. An unrelated collision on the active slug no longer blocks lanes that
resolve to a different provider (@openrouter, a non-colliding @custom:other).
3. streaming.py now resolves endpoint AND credential inside ONE profile scope:
the main path extends the scope over runtime-key + custom-provider resolution,
and both self-heal retry paths bind the session profile. This stops a detached
worker from pairing a named profile's endpoint with the default profile's key.
4. Handoff summary: AmbiguousCustomProviderError now returns HTTP 400 with the
actionable rename message instead of degrading to a 200 local fallback the UI
treats as success.
5. UI: aux + default model saves surface the server's message and abort (retain
dirty state) instead of swallowing it into a generic failure / "settings
saved"; the handoff catch shows the 400 rename message verbatim.
AmbiguousCustomProviderError gains a .message attribute for verbatim forwarding.
Tests: parenthesized-name collision fails closed on all direct paths; unrelated
lanes are not blocked by an active collision; custom overrides resolve inside the
passed profile scope; handoff returns 400 (not a persisted fallback); JS
error-surfacing source guards. Verified the finding #1/#2 cases fail before and
pass after (old key misses 'Foo (Bar)'/'foo-bar'; old up-front check over-blocks).
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
* fix(sse): scope gateway SSE probe and honor Last-Event-ID on /api/chat/stream
Two narrowly-scoped cross-client SSE fixes from the hermes-android#58
follow-up:
1. The /api/sessions/gateway/stream?probe=1 response only describes the
optional gateway/agent-sessions stream, but its payload gave clients no
way to tell that. A 404 'agent sessions not enabled' result could be
misread as SSE being unsupported server-wide, even though the persistent
per-session stream (/api/session/stream) is always on and independent of
the show_cli_sessions setting. Add additive scope markers to the probe
payload (scope, session_stream_available, session_stream_path) and
clarify the handler docstring.
2. /api/chat/stream emitted id: stream_id:seq on every journaled event
but ignored the Last-Event-ID request header, so any spec-compliant SSE
client that auto-reconnects (browser EventSource, Android/CLI clients)
replayed the journal from seq 0 and double-rendered the transcript.
_chat_stream_resume_after_seq() now resolves the resume cursor with
explicit after_event_id/after_seq query params first and the
Last-Event-ID header as fallback — the same resolution chain
api/kanban_bridge.py already uses. Both the dead-stream journal replay
and the live-subscribe gap check consume it; direct callers of the gap
check that only pass query params keep historical behavior.
Also documents the cross-client SSE endpoint contract in
docs/sse-streams.md.
Related hermes-webui/hermes-android#58
* fix(sse): split cursor presence from validity on /api/chat/stream resume
Address PR #6886 review (gate-fail): the resume logic conflated "no
cursor" with "invalid cursor" into a single None, which broke replay on
three edges. Root cause and fix:
Cursor presence vs validity — _chat_stream_resume_cursor() now returns
(after_seq, resume_requested) instead of a bare seq. A client that
SUPPLIED any cursor (query params, replay=1, or Last-Event-ID) asked to
resume; an invalid / foreign-run / unparseable cursor resolves to
after_seq=None but keeps resume_requested=True, and is normalized to
replay-from-start so no journal events are silently skipped. A genuinely
cursor-less request stays a fresh subscribe.
1. Explicit-query precedence is now decided by parameter PRESENCE, not
successful parsing: a malformed/foreign after_event_id/after_seq never
falls through to let the Last-Event-ID header override it and skip
events.
2. Invalid/foreign cursors no longer bypass replay + gap detection and
silently drain a truncated buffer; an ahead-of-stream cursor is no
longer clamped onto the snapshot's terminal fence (which emptied the
dead-stream replay by filtering the terminal frame).
3. The runner-local observe path now receives the resolved header cursor
(retaining explicit-query precedence) so header-only runner reconnects
resume instead of duplicating events from the start.
Regression tests cover invalid-cursor replay-from-start, ahead-of-stream
terminal survival, header-only runner reconnect, and explicit-query
precedence; the stale clamp test is updated to the corrected contract.
Related hermes-webui/hermes-android#58
* fix(sse): consume resolved resume cursor at every consumer on /api/chat/stream
Address PR #6886 r2 review (gate-fail). The r1 presence/validity split
fixed the entry resolution, but the individual consumers still re-derived
ahead/equality/re-parse decisions, so the invariant leaked at the edges.
This makes every consumer honor the resolved (after_seq, resume_requested)
pair and normalizes ahead-of-stream in one place.
1. Equality off-by-one: a valid cursor EQUAL to the snapshot cutoff now
enters the drain dedup bound (the event at the cursor was already
delivered), instead of being treated as "ahead" and double-sending the
retained buffer. Ahead is now strictly after_seq > snapshot_cutoff_seq.
2. Numeric ahead-of-stream cursors are normalized to replay-from-start:
on the active path, after_seq > snapshot_cutoff_seq resets to 0 before
gap-coverage/replay so a truncated buffer can no longer falsely declare
coverage and lose the pre-tail events; on the dead-stream path the
cursor is normalized against the journal summary's authoritative
last_seq before _replay_run_journal, so an ahead cursor returns the
full replay instead of an empty SSE body.
3. Runner-local now consumes the resolved cursor instead of re-parsing
after_event_id or re-reading Last-Event-ID. The resolver also returns
the opaque raw cursor so the runner uses the already-precedence-checked
value; explicit opaque cursor= keeps precedence, invalid/foreign
cursors pass no cursor (full replay), and the header is honored only
when no explicit cursor was supplied.
Regression tests cover the equality dedup, active+dead ahead-of-stream
normalization (with after_seq-filtering mocks that assert the real replay
body), and the three runner precedence probes (malformed-explicit blocks
header, foreign passes no cursor, opaque cursor= wins).
Related hermes-webui/hermes-android#58
* fix(sse): carry runner-cursor provenance through /api/chat/stream resume
Address PR #6886 r3 review (gate-fail). The (after_seq, resume_requested,
raw_cursor) tuple carried the cursor VALUE but not its PROVENANCE, so the
runner handoff couldn't distinguish a valid opaque runner cursor from a
journal-shaped one from a malformed one — and gated the runner handoff on
journal-specific validity, which is wrong for the runner adapter (its
cursors are opaque).
The resolver now returns a distinct, centrally-resolved runner cursor:
- a valid after_seq PAIRS with whatever after_event_id shape was supplied
— including an opaque runner id like event:2 that the journal parser
reads as a foreign run — so the paired runner cursor resumes at the seq
(never None / full replay, which would duplicate tokens and tool
events). after_seq is read directly (via _parse_run_journal_after_seq_value)
rather than through the after_event_id-first journal parser, which would
swallow a paired opaque runner id.
- a header-only opaque runner id (Last-Event-ID: event:2) is preserved
as-is so the runner resumes from it.
- a malformed or foreign explicit cursor WITHOUT a valid paired after_seq
still yields no runner cursor, blocking Last-Event-ID and replaying from
start — the r2 malformed-blocks-header guarantee is unchanged.
Regression tests pin the paired opaque id, header-only opaque id, and
malformed-with-valid-seq runner resumes, plus the unchanged
malformed-without-pair replay-from-start behavior.
Related hermes-webui/hermes-android#58
* fix(sse): normalize ahead cursor when the snapshot cutoff is unknown
Address PR #6886 r4 review (gate-fail). The ahead-cursor normalization
was guarded by `snapshot_cutoff_seq is not None`, so when the subscribe
snapshot had no parseable last_event_id (channel has not seen an
id-bearing frame yet), a positive ahead cursor (e.g. Last-Event-ID
run_1:999) was never normalized — it was installed verbatim as the live
dedup bound, filtering EVERY queued frame including the terminal
stream_end fence. The reconnect stalled on heartbeats with an empty SSE
body. origin/master ignored the header on this path and delivered the
events, so this was a new regression.
Fix: treat an unknown, non-truncated snapshot cutoff as fence 0 before
the ahead-cursor normalization, so a positive client cursor can't become
the live dedup bound with no cutoff to bound it — it fails closed to
replay-from-start, delivering the buffered events rather than silently
losing them. Frames dropped while the cutoff is unknown still cannot
prove gap coverage (cutoff None → not covered), so the recovery_control
fail-closed path is preserved.
Regression test: Last-Event-ID run_1:999 + snapshot_cutoff_seq=None
delivers queued events run_1:1 and run_1:2 exactly once, emits the
terminal stream_end, and releases the subscriber. Without the fix the
test stalls on heartbeats (pytest-timeout kills it), proving the stall.
Related hermes-webui/hermes-android#58
* Release — SSE Last-Event-ID resumption fix (#6886)
---------
Co-authored-by: Paladin173 <35980893+Paladin173@users.noreply.github.com>
Co-authored-by: n <a@n>
* fix(approval): stamp mirror token when creating no-run legacy mirror (#6008 follow-up)
In in-process (legacy) mode, a guarded command parks an _ApprovalEntry in
_gateway_queues whose .data has no run_id. submit_gateway_pending_mirror()
mirrors it into _pending for the UI in the `elif not exact_local_entry`
branch, but that branch never stamped _GATEWAY_MIRROR_TOKEN on the mirror.
_resolve_approval_legacy() binds a no-run mirror back to the parked
_ApprovalEntry via a token match (pending_token == gateway_token). Without
the mirror token the match fails, resolve_gateway_pending_local() is never
called and entry.event.set() never fires, so the agent thread stays blocked
even though the endpoint returns ok:true and the UI closes the card. Only a
~1.5s reconcile fallback re-creates the card with a token -> the FIRST click
on "Allow once" silently does nothing.
Stamp the token from the live gateway head at mirror-creation time, matching
the exact-local branch, so the first click unblocks the agent.
Adds a regression test reproducing the scenario.
* fix(approval): resolve non-head no-run mirrors by scanning all live producers
The first-click fix stamps the mirror token from the mirror's own producer.
But _resolve_approval_legacy() previously matched that token only against the
queue head (`_gateway_queues[sid][0]`), so a non-head mirror (multiple parked
producers, #7093) could not be resolved: the head's token differs from the
selected producer's, the match failed, and the agent thread stayed blocked.
Scan every live producer in _gateway_queues for a matching
_webui_mirror_token or approval_id, and stamp the resolved producer's
approval_id before delegating to resolve_gateway_pending_local(). Registered
producers keep head/tail ordering, so exact-producer isolation is preserved.
Adds tests/test_issue_legacy_first_click_repro.py::test_legacy_non_head_
producer_unblocks_only_its_producer, which reproduces the non-head scenario.
* fix(approval): fail closed on no-run mirror token binding (818fd2fd follow-up)
submit_gateway_pending_mirror() previously fell back to "first unclaimed
token" or the queue head when a no-run mirror's identity (request_id/
approval_id) didn't match any live producer. That let approving a
stale/foreign mirror silently resolve a different, unrelated live
producer than the one the user actually saw.
Bind the token only on an explicit identity match; otherwise leave the
mirror tokenless so it can't borrow another producer's slot. A tokenless
orphan is still created automatically for the legitimate #7093 case
(no live producer at all) via reconcile_gateway_pending_mirror_locked.
Adds two adversarial regression tests (stale request_id vs. a live
sibling, and an identity-less mirror with multiple producers parked) and
fixes an unrelated test that omitted request_id, unlike every real
_ApprovalEntry.
* fix(approval): don't let an unmatched no-run mirror mask a live producer
Follow-up to 63087a40. Failing closed on token binding stopped stale
mirror A from executing live producer B, but left A parked in _pending:
reconcile then deferred to the pre-existing no-run mirror and skipped
tokenizing B, so B never surfaced as the head. The real pending approval
became unactionable (409 in gateway mode, ok:true resolving nothing in
local mode) while an unresolvable card sat in front of it.
reconcile_gateway_pending_mirror_locked() now always tokenizes every live
no-run producer and derives live_token from the authoritative head
whenever _gateway_queues holds producers, so an unmatched tokenless
mirror is discarded instead of suppressing the real one. Retained
missing-producer mirrors survive only while no producer is live, keeping
the #7093 tokenless-orphan boundary intact.
Tests assert the displayed == resolved invariant: the authoritative
producer is the head, responding with the stale/identity-less id fails
without waking anything, and approving the displayed entry wakes exactly
its own producer. Three test_gateway_approval_runs_api.py cases that
encoded the old masking behavior are updated to the new invariant.
* docs(approval): correct reconcile comment on which mirrors survive
The comment added in 9c2ca104 claimed only the authoritative head's mirror
may survive while a producer is live. That contradicts the code it
documents and the non-head regression test: the `entry_token in
live_local_tokens` branch keeps a mirror bound to ANY live no-run
producer, which is exactly what makes a non-head mirror resolvable
(test_legacy_non_head_producer_unblocks_only_its_producer).
State the actual rule instead — a mirror survives only if bound to some
live producer's own token, the head's via live_token or a non-head
producer's via live_local_tokens — so unmatched and tokenless copies are
discarded without implying non-head mirrors are dropped too. Comment
only; no behavior change.
* Release — approval no-run mirror masking fix (#7086)
---------
Co-authored-by: VHSgunzo <wosganzo@gmail.com>
Co-authored-by: n <a@n>
* fix(session): sync provider change mid-session (#6703)
* fix(session): full chat-start parity for /goal explicit-pick (#6703)
Addresses review parity details on #6703:
- stamp model_explicit_pick_signature after explicit goal resolution
(both kickoff paths), matching /api/chat/start #5979, so a first /goal
launch after a deliberate custom-provider pick survives a cold
streaming catalog instead of reverting to the profile default
- consume the one-shot pending session-model marker in cmdGoal,
matching the chat/start path (messages.js), so a later /goal with an
unchanged dropdown isn't treated as an explicit pick
* fix(session): consume pending model-pick marker only after successful kickoff (#6705)
* fix(session): node-driven behavioral tests for /goal explicit-pick marker (#6703)
* changelog: #6705 — carry explicit_model_pick through /goal
* ci: re-trigger tests on fresh runners (prior run's shards hung on GitHub-hosted runners)
* ci: re-trigger (playwright browser-install hung on GH runners again; warm cache should restore)
---------
Co-authored-by: webtecnica <webtecnica@gmail.com>
Co-authored-by: n <a@n>
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
* fix(tests): make mtime_invalidation and glm_5_3 tests order-independent (#7100)
* fix: expose max reasoning for GPT-5.6 models
* fix(recovery): do not reattach cancelling runs
ACTIVE_RUNS tracks worker lifecycle, which is deliberately broader than
"a turn a browser may attach to". cancel_stream() keeps the row as
phase="cancelling" while the worker unwinds so a successor turn cannot
start on top of it, but the client has already reached a terminal state
for that stream: its run journal ends in a terminal event.
The recovery lookups treated every same-session ACTIVE_RUNS row as
attachable. An idle session holding a cancelling row therefore received a
recovered server_turn_started on every /api/session/stream subscription:
the client attached, replayed the terminal event, tore the renderer down,
resubscribed, and the server replayed the same frame again. The result is
an endless attach/replay loop that rebuilds the transcript repeatedly.
Separate the two meanings instead of narrowing one call site:
- api/config.py gains active_run_is_attachable() and
active_run_cancel_is_stale() as the shared predicates.
- active_stream_id_for_session() (browser recovery) returns attachable
rows only; _session_has_active_turn() (busy check) keeps counting a
fresh cancellation so a successor cannot overlap the unwinding worker.
- _live_active_stream_id() applies the same rule to the hidden-tab
status poller, on both the STREAMS and ACTIVE_RUNS paths.
- routes._cancelled_run_is_stale() now delegates to the shared predicate
rather than keeping a parallel copy of the staleness rule.
- A cancelling row past a bounded unwind window with no live STREAMS
channel is reclaimed from ACTIVE_RUNS and its stream owner released, so
a wedged worker cannot suppress background wakeups forever. Age alone
does not reclaim a row that still owns a live channel.
Staleness anchors on cancelled_at, falling back to started_at, so a
long-running turn cancelled moments ago is never treated as an orphan.
tests/test_cancelling_run_not_attachable.py covers both directions of
each rule. The six behavioral tests fail on the unpatched tree and pass
with the fix; two targeted mutations (forcing the attachability predicate
true, and disabling the staleness reaper) each turn the suite red.
* docs(rfc): document cancellation attach/admission split
Records the runtime contract introduced by the recovery fix in the WebUI
run-state consistency RFC, so the distinction is discoverable instead of
living only in code comments.
- Adds ACTIVE_RUNS to the State Layers table as the worker-lifecycle
registry, explicitly not the set of runs a browser may attach to.
- Adds invariant 9: lifecycle-busy is not client-attachable. Cancellation
splits the two meanings, recovery paths must exclude cancelling rows,
and admission checks must keep counting them.
- Documents the bounded cancellation-unwind window: reclamation needs
both age and the absence of a live STREAMS channel, and staleness is
anchored on the cancellation timestamp.
- Extends the review checklist with the admission-vs-attachment question
and the evidence required when changing a reclamation window.
* docs(changelog): note #7096 cancelling-run reattach, #7083 GPT-5.6 max reasoning, #7101 test order-independence
---------
Co-authored-by: webtecnica <webtecnica@gmail.com>
Co-authored-by: Abdulrahman Elkenany <boudy.elkenany123@gmail.com>
Co-authored-by: allenliang2022 <allenliang2022@users.noreply.github.com>
Co-authored-by: n <a@n>
* fix(goal): suppress reserved SILENT sentinel at /api/goal ingress
A wake relay POSTing the exact [SILENT] suppression sentinel to
/api/goal lets it reach _start_chat_stream_for_session, which persists
it as pending_user_message. If 8701 restarts while the turn is pending,
session recovery materializes it as a visible _recovered user turn —
the same phantom-recovered-turn exposure #7018 closed for chat ingress.
Apply the shared _is_silent_control_message() guard immediately after
/api/goal session-ID validation and before session lookup or goal-state
mutation, returning the same 200 no-op. Matching stays exact and
case-sensitive; ordinary kickoff text is unaffected.
Add tests mirroring test_silent_control_suppression.py for the goal
path (args and text fields, before-lookup suppression, exact-match
semantics).
Closes#7019
* fix(sessions): keep evicted subagent parents in the import window
The sidebar nests a subagent row under its parent only when the parent row
is present in the same payload. The visible-window limit was applied as a
flat per-row recency slice, so a frozen orchestrator (which stops writing
while its leaves keep streaming) lost the recency race against its own
leaves and fell outside the window -- promoting those leaves to top-level
sidebar rows.
Re-add subagent parents that the oversampled candidate set already
projected, after the slice. No extra queries, no change to
CLI_VISIBLE_SESSION_LIMIT. webui ancestors are deliberately not imported
because that sidebar bucket already owns them.
Supersedes #7031.
* fix(sessions): document and pin the parent-recovery bound (greptile review)
Greptile flagged (P1) that a selected subagent child whose parent ranks below
the limit * 8 oversample is still promoted to a top-level row. That is real and
measured (the parent drops out at candidate #25 of 24), but it is the bound of
the design, not a regression: the walk reuses rows the projection already
fetched and never issues an extra query. Resolving arbitrarily old ancestors
needs an unbounded per-row lookup on the hot sidebar path -- the approach
rejected in #7031 -- so the bound is documented and pinned instead.
- Docstring: state that `limit` bounds the recency slice, not the row count,
so callers must iterate rather than assume len(rows) <= limit; state that
recovery is bounded by the oversampled candidate set.
- Inline comment: mark the bound at the exact lines Greptile flagged and point
at `candidate_limit` as the knob if the window proves too tight.
- Tests: cover the candidate-window exhaustion Greptile said was untested --
parent inside the oversample is recovered, parent beyond it stays unresolved
-- plus the over-limit return contract and a parent-cycle guard.
No behaviour change; 7 tests pass.
* fix(share): add _hadAppearance guard to prevent fabricated appearance choice
Summary:
The inline appearance bootstrap in static/share.html writes hermes-theme and
hermes-skin to localStorage unconditionally on every page load, even when the
browser had no prior appearance state. This fabricates an explicit user choice
on first access via a shared link, making the server-side SETTINGS_DEFAULTS
unreachable for deployments that customise the default theme or skin.
Root Cause:
share.html:9 — the boot IIFE resolves a theme+skin and calls
localStorage.setItem() without guarding on whether the user had previously
chosen an appearance. The same bug was fixed in index.html by PR #6808
(commit tomtong2015) but share.html was left unchanged.
Change:
1. Added _hadAppearance guard before the two localStorage.setItem() calls:
var _hadAppearance = localStorage.getItem('hermes-theme') !== null ||
localStorage.getItem('hermes-skin') !== null;
if (_hadAppearance) { setItem('hermes-theme', t); setItem('hermes-skin', s); }
2. The first-paint DOM mutations (classList.add('dark'), dataset.skin) remain
outside the guard — only persistence is protected.
3. Synced the skin allowlist with index.html: added neon-soft and neon-paint
(zeus and verdigris were already present).
Verification:
- test_6808_appearance_bootstrap_no_fabricated_choice.py: 11/11 passed
covering fresh-browser (no writes), pre-paint fallback, explicit state
normalisation, and legacy migration survival.
Closes#7030
* fix(tests): pin rebuild budget in issue2513 custom-provider catalog test
test (3.13, 4) failed on this PR with:
assert "@custom:alpha-proxy:alpha/remote" in alpha_ids
E AssertionError: assert '@custom:alpha-proxy:alpha/remote' in {'alpha/sticky'}
WARNING api.config:config.py:8355 live provider-catalog rebuild exceeded
4.0s budget - serving fallback, refreshing catalog out-of-band
Pre-existing wall-clock flake, not a regression from this PR: this branch
touches only api/agent_sessions.py and tests/test_subagent_parent_in_import_
window.py, and both api/config.py and this test file are byte-identical to
origin/master. The same shard passed on 3.11 and 3.12.
The test never pinned _LIVE_REBUILD_BUDGET_SECONDS, so it raced the global
4s budget in get_available_models(). On a starved runner the cold rebuild
overruns, the degraded fallback catalog is served, and the monkeypatched-
urlopen model alpha/remote is dropped - leaving only the config-declared
sticky model, exactly as CI observed.
Force the synchronous (unbounded) rebuild path, matching the existing
precedent in tests/test_issue2540_models_endpoint_error.py:20-24.
Verified: with the budget forced to 0.001s the unpatched test reproduces
the CI assertion verbatim; patched it passes at 0.001s, 0, and default.
* fix(#7013): preserve media deny coverage across platforms
* fix(docker): keep repository agents out of runtime context
* fix(docker): gate image runtime proof behind integration job
* test(models): stabilize custom provider catalog regression
* Release batch A: 6 low-risk gate-passed fixes (experimental)
Batched contributor fixes, each individually Codex-gated during the overnight
certifier cycles and re-verified clean-to-ship as a combined stage (Codex SAFE
TO SHIP on the combined diff; full suite green except 8 pre-existing approval
tests that fail identically on clean origin/master — CI green on same commit).
- #7019 (@webtecnica) suppress reserved [SILENT] sentinel at /api/goal
- #7031 (@carlotestor) keep evicted subagent parents in the sidebar import window
- #7030 (@webtecnica) share.html first-visit appearance guard + skin-id sync
- #6853 (@rodboev) exclude repo-root AGENTS.md from the Docker runtime image
- #7013 (@webtecnica) test-only: platform-neutral test portability (playwright
import guard + media tests served from allowed roots)
- test-only (@carlotestor) pin custom-provider catalog rebuild budget (#7054)
Co-authored-by: webtecnica <webtecnica@users.noreply.github.com>
Co-authored-by: carlotestor <carlotestor@users.noreply.github.com>
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
---------
Co-authored-by: webtecnica <webtecnica@gmail.com>
Co-authored-by: carlotestor <carlotestor@users.noreply.github.com>
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Co-authored-by: n <a@n>
Co-authored-by: webtecnica <webtecnica@users.noreply.github.com>
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
Cron agents use the exact final response `[SILENT]` as a delivery-suppression
sentinel. If a wake relay accidentally POSTs that sentinel to `/api/chat/start`
and 8701 restarts while the turn is pending, session repair materializes it as
a visible `{role: user, _recovered: true}` message. The pending value can then
be recovered again on later restarts, creating repeated `[SILENT]` turns.
Treat the exact normalized sentinel as a successful no-op at both server-side
turn entry points: the HTTP `/api/chat/start` handler and
`start_session_turn`. Both checks run before session lookup, runtime barriers,
or pending-state mutation. Matching is deliberately exact and case-sensitive,
so `[silent]`, prose containing `[SILENT]`, and ordinary user text are not
suppressed.
Add regression tests proving both paths return HTTP/status 200 without session
lookup, plus negative cases for non-exact text. The new tests fail 3/3 before
the fix. Targeted and neighbouring chat-start suites pass 21/21.
Co-authored-by: allenliang2022 <allenliang2022@users.noreply.github.com>
Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
Addresses nesquena-hermes Round-2 review on #6979. All three MUST-FIX
items reproduced end-to-end are fixed; three SHOULD-FIX items folded in.
MUST-FIX 1 (data exfiltration) — capture/serve deny parity + source binding:
- The #3234 state/profile deny model is extracted into a single shared
predicate (routes._media_deny_reason) used by BOTH /api/media serving and
snapshot capture, so anything the serve path refuses is never captured in
the first place (STATE_DIR subdirs, webui_state subdirs, named-profile
roots, secret basenames, the snapshot store itself).
- Every captured digest now carries a server-owned canonical source-path
association (<digest>.src.json sidecar, tmp+rename atomic). /api/media
serves a digest only for the exact path it was captured from — replaying
a valid digest through a different allowed path falls back to the live
file instead of leaking stored bytes.
- Snapshot blobs are opened ANCHORED inside the store root (no-follow).
- Committed end-to-end regression: capture a denied state file and prove it
cannot be served through any path.
MUST-FIX 2 (data exfiltration) — configured store root denied regardless of name:
- get_snapshot_dir() is resolved and any bare path= target at or below it is
rejected BEFORE the allow/serve decision, so a custom
HERMES_WEBUI_MEDIA_SNAPSHOT_DIR (e.g. /tmp/custom-store-name) can no
longer be read via a plain path= request.
MUST-FIX 3 (silent corruption) — prefix-safe URL stamping:
- _stampMediaSnapshots now rewrites each COMPLETE path= query value
atomically (boundary-aware regex, decode whole value, exact map lookup)
instead of substring split/join. One path being a prefix of another
(/tmp/a.png vs /tmp/a.png.backup) can no longer corrupt the longer URL or
drop its digest. data-path placeholders use the same value-level rewrite
with HTML-entity unescaping. Prefix-path regression test committed.
SHOULD-FIX folded in:
- A recorded digest is FINAL: quota eviction no longer causes a later settle
to re-capture the current live bytes and rebind the historical message
(immutability restored + no quota-thrash I/O).
- is_valid_digest uses whole-string fullmatch (\Z) — a trailing newline can
no longer pass the gate.
- PDF/HTML fallback, download and open-full-page links now carry snap so
they open/download the frozen bytes.
Deferred (noted in review thread): emission-time capture (streaming.py) and
worklog scene-prose stamping — scene rows do not carry parent-message
snapshot metadata; threading it through the scene reconciler is a separate
data-pipeline change. Main transcript + transparent ordered segments remain
the old/new comparison surface.
Tests: +11 backend/frontend regressions (36 total in
test_media_message_snapshots.py), incl. the exact Round-2 repros.
Reverse-verified: the new tests fail on the round-1 head (8 failed).
PR #6922 made /api/media revalidate on every use, so overwriting a file in
place (same filename) also rewrote every historical chat preview that
referenced it — the old/new comparison was lost.
At settle time, snapshot each local-file MEDIA: reference into a
content-addressed store (<STATE_DIR>/media_snapshots/<sha256>.snap) and stamp
the message with _media_snapshots {path: digest}. The frontend appends
&snap=<digest> to historical preview URLs; /api/media serves the frozen bytes
with immutable caching when the digest is valid, and falls back to the live
file otherwise (eviction/never-captured). The existing path allow/deny checks
still gate the request — snap only selects which bytes to serve, it never
widens access.
- api/media_snapshots.py: content-addressed store (sha256 key, dedup, 50MB
per-file / 2GB total caps with FIFO eviction), MEDIA ref resolver, capture
allow-list mirroring the /api/media roots + #3234 deny list
- streaming.py: annotate settled assistant messages (idempotent, best-effort)
- routes.py: snap= serving branch + download_name override + deny direct
store-path access
- models.py: _media_snapshots registered in display-metadata merge keys
- ui.js: _stampMediaSnapshots post-cache stamping (main transcript,
transparent ordered segments, worklog scene prose) + data-snap threading
through the five lazy preview loaders
25 new tests incl. node-executed frontend behavior tests; reverse-verified.
The post-commit body-transmission block only caught
_CLIENT_DISCONNECT_ERRORS. A non-disconnect error during body write —
a generic OSError/EIO from f.read(), or a PermissionError — escaped
_serve_file_bytes entirely, and Handler.do_GET (server.py:388-395)
caught it and emitted a second status line after the already-committed
200/206, corrupting the HTTP stream.
Broaden the post-commit except to swallow every Exception (keeping the
disconnect branch for its debug message). Regression tests drive both
the snapshot and streaming paths with generic OSError/EIO and
PermissionError mid-body, asserting the status sequence is exactly
[200] with no trailing 500.
Move ETag/snapshot computation into a pre-commit try/except OSError so
read errors still convert to a clean 500 before any status line is sent.
Remove the outer OSError catch around body transmission: once
end_headers() returns the response is committed, and a client disconnect
(BrokenPipeError et al.) during the snapshot write or fdopen stream would
previously have triggered bad() -> send_response(500) as a second status
line on a dead stream. Body paths now catch _CLIENT_DISCONNECT_ERRORS
(same tuple as _safe_write()), log at debug, and return True.
Add regression tests with a fake writer that raises BrokenPipeError
after end_headers() for both body paths, asserting exactly one
send_response call.
When a file is truncated between fstat and the snapshot read, we now return
(None, data, actual_size) instead of (None, None, actual_size). This preserves
the captured bytes so the body path serves the immutable snapshot instead of
re-reading the fd after headers are committed.
This closes the second-order TOCTOU window: if the file shrinks AGAIN between
the first read and body transmission, Content-Length (from actual_size) and
body length (from snapshot) can never diverge.
Changes:
- api/routes.py:18494: return None, data, actual_size (preserve snapshot)
- tests: remove unused original_read (ruff F841)
- test_shrink_again_during_body_transmission_200: 200 advertised 6 / wrote 6
- test_shrink_again_during_body_transmission_range_206: 206 advertised 4 / wrote 4
Round 5 fix: when a file is truncated between fstat and the snapshot read,
Content-Length/Range headers must match the actual bytes read, not the stale
fstat size.
Changes:
1. Replace os.pread with os.lseek + looped os.read (cross-platform + short-read safe)
2. _etag_and_snapshot now returns 3 values: (etag, snapshot, actual_size)
3. Caller uses actual_size for Content-Length/Range calculations
4. If actual_size != file_size (truncation), etag=None → streaming fallback
Regressions:
- test_short_read_truncation_uses_actual_size: Content-Length == body length
- test_short_read_no_etag_on_truncation: no ETag on truncated snapshot
- Updated test_etag_no_body_on_pread_failure for os.read (was os.pread)
Native Windows opens files in text mode by default, which corrupts
binary media files via CRLF translation. O_BINARY prevents this.
This completes the portability story — the PR was already correct on
all supported platforms (Linux/macOS/WSL2), but the round-3 change was
explicitly about portability, so we close the Windows path too.
- _file_etag streams a SHA-256 digest through the already-authorized fd
instead of using mtime+size metadata, so same-size in-place replacements
on coarse-mtime filesystems (2s granularity) produce a new ETag — the
exact stale-cache bug this feature sets out to fix. fd is rewound before
the normal/Range send.
- close fd on the 304 early-return path (was leaking a descriptor per
revalidated hit)
- drop Content-Length: 0 from 304 responses (RFC 9110 §15.4.5)
- non-HTML /api/media cache policy is now "private, no-cache" so shared
intermediary caches cannot store per-user/per-session media
- regression test: equal-size + equal-mtime (ns-pinned) replacement must
change the ETag and return 200, not a stale 304
/api/media served every file type with Cache-Control: private, max-age=3600,
so the browser reused a cached response for 1 hour without revalidating. When
an agent overwrites a file in place (same name, new content — e.g. a
regenerated image), inline previews kept showing the stale version until a
manual full-page refresh.
Changes:
- _serve_file_bytes now computes a weak ETag from file metadata
(mtime_ns + size) and honors If-None-Match with a 304 short-circuit
(RFC 7232 weak comparison, W/ prefixes ignored, '*' support), processed
before the Range block since a matched conditional request skips the
entity entirely. 200/206 responses carry the ETag.
- _handle_media splits the cache strategy by MIME: text/html keeps no-store
(inline previews change constantly, PR #6706); all other media (images,
audio, video, PDF) switch from max-age=3600 to no-cache so the browser
MAY cache but MUST revalidate on every use — in-place updates are picked
up immediately while unchanged files still short-circuit with 304.
Why weak ETag: metadata-based equality is sufficient for the overwrite
case and avoids hashing large files per request.
Why no-cache over must-revalidate: no-cache forces revalidation before
every use regardless of freshness; must-revalidate only fires after
max-age elapses, which does not help when the browser believes its cached
copy is still fresh.
Tests: tests/test_media_etag_revalidation.py (15 tests) covering ETag
emission, 304 short-circuit (match/star/mismatch/list/weak-comparison),
ETag change on in-place replacement, Range+ETag interaction, 304-before-
Range precedence, and the _handle_media end-to-end path (no-cache for
images/PDF, no-store preserved for HTML).
Ships PR #6757 with a maintainer hardening pass folded in (authorship preserved).
Preserves the exact provider-facing reply text (api_content) through the internal
session store for byte-accurate agent replay, while guaranteeing it never reaches
a browser: every client-facing surface strips it via the public projection.
Maintainer fixes (2 defects my adversarial re-gate reproduced):
- Selective session redaction restored: only transcript fields are scrubbed;
operational fields (e.g. workspace path) pass through unchanged, so a path
containing credential-shaped text is no longer masked into an invalid path.
- Runner-backed SSE relay now projects payloads before emit, closing an
api_content leak through _stream_runner_run_events.
+5 regression tests. Codex adversarial re-gate: SAFE TO SHIP. Full suite clean.
Co-authored-by: franksong2702 <franksong2702@users.noreply.github.com>
* fix(cron): mark running cron sessions as active in sidebar
* fix(cron): include cron_running in effective-streaming predicate + test updates (#6734)
* fix(cron): longest-prefix-first cron job matching in session list cache (#6728)
* fix(cron): require exact run-timestamp suffix in cron_running prefix match
Gate re-review (Codex) found the longest-prefix-first match could misattribute:
cron_job_prefixes is built ONLY from RUNNING jobs, so when the true longer owner
(backup_full) is NOT running, a running shorter job (backup) swallowed a completed
cron_backup_full_YYYYMMDD_HHMMSS session (its prefix cron_backup_ still matched),
deferring that row's completion/unread transition. Now the text after a matched
cron_{jid}_ prefix must be EXACTLY a run timestamp (\d{8}_\d{6}), so a shorter job
id can't claim a longer job id's session regardless of which jobs are running.
Regression test added (fails without the guard).
Co-authored-by: webtecnica <webtecnica@users.noreply.github.com>
---------
Co-authored-by: webtecnica <webtecnica@users.noreply.github.com>
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
* feat(sidebar): add show_kanban_sessions toggle to hide kanban worker sessions
* fix(sidebar): kanban source-filter override + filtered-view cap (cron/webhook parity)
Gate re-review (Codex) found the kanban-hide addition missed cron/webhook
parity for the source-filter case:
- An explicit kanban source_filter now overrides the default hide (threaded
source_filter into _dedupe_cli_sidebar_sessions_for_api, normalized with
strip().lower() to match the loader), so a deliberate kanban view shows
those rows even with the toggle off. Scoped to the matching source only.
- A dedicated KANBAN_PROJECT_CHIP_LIMIT bounds a filtered kanban-only view,
mirroring CRON/WEBHOOK_PROJECT_CHIP_LIMIT.
Deliberately NOT changing the general exclude_sources: the original PR lets
kanban load on the source_filter=None path so the toggle-ON case works (the
hide is applied by _hide_from_default_sidebar only when the toggle is off).
Excluding kanban there would save the 20-row budget but break toggle-on
unless a bounded kanban merge is added (cron/webhook don't do that either) —
that query-budget optimization is deferred as a separate cross-cutting change.
The pinned-survives-hide finding is also out of scope (cron/webhook hide pinned
identically, confirmed by Codex). Adds bite-tested regressions.
Co-authored-by: vduke <vduke@users.noreply.github.com>
---------
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
Prevents a delayed cancelled streaming worker from writing back stale output over a completed successor's result, and clears the Gateway writeback-owner entry on teardown/pre-start-cancel to avoid a slow SESSION_WRITEBACK_OWNERS leak.
Co-authored-by: webtecnica <webtecnica@users.noreply.github.com>
* fix(branch): slice fork transcript in GET /api/session merged-view coordinates
keep_count sent by forkFromMessage is an index into the merged display
view (sidecar lineage stitched across compression snapshots, append-only
merged with state.db rows, parent backfill). The branch handler sliced the
raw sidecar instead, which diverges from that view in both length and
content after compactions: forks from the final conclusion landed rows
too early, ending the fork mid tool-run and dropping the conclusion.
Build source_messages through the exact same pipeline as the
GET /api/session load_messages path for non-messaging sessions.
Regression test: test_branch_route_slices_merged_display_view_not_raw_sidecar
(fails on the old code with the fork ending on a tool row).
* fix(branch): align fork source with display bounds
---------
Co-authored-by: Anthony Ruiz <ruizanthony@users.noreply.github.com>
Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
Project active run-journal recovery snapshots into a non-mutating compact transport form before returning them from /api/session, removing duplicate representations while preserving the live message timestamp. Thanks @allenliang2022.
Co-authored-by: allenliang2022 <allenliang2022@users.noreply.github.com>
The previous revision stripped the '@provider:' qualifier from model_value
in _session_model_state_from_request BEFORE _resolve_compatible_session_model_state
ran. That defeated two behaviors that rely on the qualified form:
1. provider-switch repair - a session pinned to a removed/unconfigured
provider (@removed:mistral-large) no longer reverted to the active
default; it returned (bare_model, removed) and targeted an unavailable
provider.
2. PR #6718's custom-provider normalizer, which needs the still-qualified
value to map @custom:<slug>:<model> to the canonical slug.
Keep the qualifier through session-state resolution (matching master) and
strip it to the bare model ONLY at the gateway request boundary via
_gateway_model_field(), which already runs on both gateway request sites.
Red-before-green: the rewritten test asserts @removed:mistral-large repairs
to the active default; it failed on the prior head and passes now.
The gateway-body assertions (TestGatewayRequestBodiesCarryBareModel) still
prove only the outbound payload is bare.
Verified: 275 tests pass across the issue-6722, provider-mismatch,
model-resolution-fast-path, gateway-approval (legacy+runs), gateway-routing,
custom-provider, session-model-switch and model-not-found suites.
A provider-qualified model string (@provider:model) is deliberately kept
intact for internal routing (#1253), but three places on the way to the
Hermes gateway mishandled it:
* _clean_session_model_provider returned the whole "provider:model"
value as the provider, so the gateway saw a provider named
"ollama-cloud:deepseek-v4-flash" -> "Unknown provider".
* _session_model_state_from_request extracted the provider correctly
but left the full "@provider:model" string as the model value.
* both gateway request builders sent that string verbatim as
body["model"], which the upstream provider API 404'd on.
Rather than add positional colon-split heuristics, route all three
through config._parse_provider_qualified_model_id() -- the existing
shared grammar that already distinguishes a multi-segment custom
provider ID (custom:<slug>, custom:<host>:<port>) from a model segment
carrying its own colon tag (:free, :0731). A positional split cannot
tell those two shapes apart, so any single split direction breaks the
other; the shared parser is the only thing that gets both right.
_split_provider_qualified_model now delegates to that parser too, which
fixes the same class of truncation at all of its call sites in routes.py
instead of only the two reported ones, and guarantees the provider used
for the routing decision matches the model sent in the request.
Fixes#6722
