Greptile's P2 was right: 'upgrade past the sync' had no installable
target. Version audit across core tags: every RELEASED Agent (0.20.5 ->
0.21.1, tag v2026.9.7) still carries ox-alpha-free in the curated floor;
the removal exists only on core main, which self-reports the same
0.21.1 version string as the stale tag — a version number alone cannot
distinguish fixed from unfixed. State that plainly and present the
config allowlist as the only verified workaround until a release notes
the 2026-09-09 catalog sync.
Verified against a real install (core v0.21.0 + live key + endpoint):
- on Agent releases between v0.20.5 and the 2026-09-09 catalog sync,
the delisted ox-alpha-free can STILL be offered — via core's own
live/curated merge AND via _seed_provider_models_from_core() injecting
the installed core's stale curated ids into WebUI's static list.
'Upgrade the Agent past the sync' is the correct guidance, not
'upgrade the WebUI'.
- allowlist precedence re-verified with a real config.yaml on disk
(group == exactly the allowlisted ids).
- note the 4s foreground rebuild budget: first picker open after a cold
start can serve last-known catalog while the live rebuild finishes
out-of-band.
User-visible runtime behavior changed in this PR (live catalog first for
Agent >= 0.20.5, resynced static fallback): document both failure
directions users can now hit — listed-but-failing ids (tier/region gates
on the live endpoint, the #7507 follow-up) and missing new ids on
older-AGENT installs — with the diagnostic (hermes --version + authed
curl of /zen/go/v1/models) and fixes (allowlist precedence, Agent
upgrade). Follows the file's Symptom/Why/Diagnostic/Fix/File-a-bug
format.
AGENTS.md asks for the documentation to move with a runtime-behaviour change, and docs/CONTRACTS.md points at the stable-assistant-turn-anchors RFC for anything that reconciles a live turn with the settled transcript. That is what this change adds, so the rule belongs there.
A new part of the Settlement section says why a preserved live node is not evidence of a running turn, the four facts that have to hold before one is dropped, that ownership is proved from state rather than by comparing two rendered trees, and that only the branches which ADD a turn are gated. It also records the settled-scene half: a prose row that duplicates the final answer, exactly or as a prefix within ten percent of its length, is dropped where both settled renderers take their rows, and live rendering is untouched.
Review follow-up for #7569: change 4 of the live-snapshot perf PR replaced
one quadratic with another. The window-free backward walk is correct, but it
re-walks the raw reasoning transcript's whitespace once per interim event —
on a whitespace-heavy transcript that span is megabytes, and the replay went
~55x slower than master on the production shape the gate measured
(1MB / 20-interim journal: 0.035s master → 1.922s branch).
## Thinking Path
The gate's finding was that the cost is structural, not a tweak: the matcher
consults the *whole* accumulated transcript on every interim event, so any
raw-text rescan of it is O(interim_count x whitespace_span). The fix is the
data-structure change the review asked for — maintain the folded view and the
raw cut offsets incrementally, and match/strip from that indexed state
instead of rescanning the raw buffer.
## What Changed
- `api/streaming.py`: new `_CompactEchoIndex` — an append-only folded index
that records, per non-whitespace character, its raw index. `append` folds a
chunk once (cost proportional to the chunk, never the buffer),
`matches_tail`/`cut_to` compare only the candidate against the folded tail
(O(len(candidate)), no raw walk), `reset` drops the state after the raw
text is truncated. The old raw helpers (`_find_compact_echo_suffix_start`,
`_strip_compact_echo_suffix`) stay as the equivalence oracle.
- `api/routes.py::_run_journal_live_snapshot`: reasoning chunks feed the
index as they are appended; `reasoning_echo_tail_matches` and
`strip_reasoning_echo_tail` consult the index. The strip re-indexes the
truncated transcript so later probes match against it.
- `api/streaming.py` live path: the same swap for the live-stream echo strip.
The three live reasoning buffers (STREAM_REASONING_TEXT, the coalescing
buffer, the per-message segments) each carry a mirrored index, fed on every
append and re-indexed after a strip. The buffer flush resets the buffer's
index with the text it drops, so the index never describes text that is no
longer there.
- `docs/rfcs/webui-run-state-consistency-contract.md`: the echo-match clause
now names the indexed representation and the scaling regression that pins
it.
## Why It Matters
Removing one quadratic by introducing another in the same hot path is the
exact risk the gate flagged — the headline benchmark used a journal shape
where the window-free walk's cost does not dominate, so the aggregate number
still looked like a win. The index keeps the original 8.9x and adds the
scaling guarantee on the shape that was regressing.
## Verification
- Measured on the reproduced shape (1MB whitespace span, 20 interim events):
0.972s → 0.049s. Doubling both span and interim count scales ~2x per step
(x1 0.041s / x2 0.090s / x4 0.175s) — linear, not quadratic.
- Behaviour equivalence: 300 randomized journals (whitespace-heavy,
Unicode, chunked, flagged/unflagged interims) rebuilt with the indexed
implementation and with the raw walk swapped in — sha256 of the canonical
snapshot JSON matched on all 300.
- `tests/test_compact_echo_index.py` (new, 6 tests): oracle equivalence over
2000+ randomized cases, chunked-append equivalence, matches_tail/cut_to
agreement, reset-after-truncation, and two cost probes that fail the raw
walk.
- `tests/test_live_snapshot_echo_scan_scaling.py` (new, 4 tests): the
proportional-scaling regression the review asked for — verified to FAIL on
the pre-fix matcher (14.6x time for 4x rows, the quadratic signature) and
pass on the indexed one.
- Existing echo/journal suites unchanged and green: 19 items across
`test_compact_echo_suffix_linear_scan` / `test_live_snapshot_echo_dedup` /
`test_live_snapshot_single_parse`, plus 66 neighbouring items
(`test_issue1361_cancel_data_loss`, `test_issue2565_reasoning_accumulation`,
`test_issue2713_streaming_segment_flush`, `test_cancel_interrupt`,
`test_run_journal`).
## Contract Change
None. The matcher's observable behaviour is unchanged (same cut points, same
whitespace-insensitive semantics); only its cost profile changes. No config
knobs, no API surface.
## Risks
- Index drift: if a future writer mutates one of the reasoning buffers
without feeding (or resetting) its index, the matcher would silently miss
echoes. Mitigated by keeping the index feed adjacent to every existing
write site and by the reset-on-flush/reset-on-strip symmetry; the
equivalence test would catch a dropped feed as a missed echo.
- Memory: the index stores one char plus one int per non-whitespace
character. On a whitespace-heavy transcript this is bounded by the folded
length, which is what the transcript would cost folded anyway; the raw
text is still the larger object.
## Model Used
Hermes Agent (step-5-preview) with direct file edits and pytest verification.
Cost properties were measured with a purpose-built scaling probe (proportional
span x interim count), not a fixed-size fixture; the probe is kept as a
regression test rather than a script.
## Related
#7569 (this PR), #7310 (slow /api/session under load), #7534 (incremental
journal read cache — complementary), #7273/#7291 (metering write-side skip).
Own a private stdout descriptor before agent imports and serialize request log writes. Keep logging failures isolated from HTTP responses. Add successful chat/start subprocess delivery regressions and document header-latency semantics and cron ownership.
PR #7636 review (greptile-apps) flagged the profile-home resolve cache
scoping (profile_home_resolve_cache_scope() / _PROFILE_HOME_RESOLVE_SCOPE
in api/workspace.py) as an undocumented runtime contract per AGENTS.md.
Record what is cached, the call-scoped (not process-lifetime) lifecycle,
which single call site owns it while every other caller resolves fresh,
and the freshness guarantee this gives against symlink retargets and
transient resolve fallbacks; index the document from docs/CONTRACTS.md,
following the same pattern used for the /api/models cache invalidation
contract. Docs only.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
On the msvcrt branch of scripts/ensure_state_db_read_indexes.py the tool
locked byte 0 of --lock-file right after open(..., "a+"). When that open
had just created the file (or the file was empty) the lock range lay beyond
EOF, which Windows refuses, so the documented locked-maintenance workflow
raised instead of acquiring the lock (greptile P1).
Write one byte when the lock file is empty, then seek(0) and lock as
before. Only the Windows branch changes; POSIX flock is untouched and an
existing (deployment-owned) lock file is never rewritten.
Tests (tests/test_pr7445_readonly_review_followups.py): a strict msvcrt
stand-in that refuses to lock past EOF proves the lock is taken on byte 0
of a fresh empty file, that an existing lock file keeps its content, and
that a held lock still propagates before the database is touched.
Review follow-ups on #7445 (nesquena-hermes CHANGES_REQUESTED + Greptile).
api/agent_sessions.py
- The read-only pre-aggregated `latest_messages` candidate ordering now
applies to every missing-index projection (default sidebar, gateway
watcher, cron/webhook/kanban views), not only `include_sources ==
("cron",)`. Since the listing no longer self-heals the index, the
correlated `MAX(mx.timestamp)` scan re-ran on every load: 9.86 s on a
5,000-session / 20,000-message DB vs 0.044 s after this change, with
identical selected rows and order. The indexed seek is unchanged.
- `state_db_file_uri()` / `state_db_readonly_uri()` replace
`Path.as_uri()`: a UNC `HERMES_HOME` produced `file://server/share/...`
and CPython's bundled SQLite (3.11-3.13 Windows builds, no
SQLITE_ALLOW_URI_AUTHORITY) rejected the authority, so every agent-backed
session vanished. The builder emits the empty-authority
`file:////server/share/state.db?mode=ro`, keeps `file:///C:/...` for
drive letters, strips `\\?\` / `\\?\UNC\` prefixes, quotes `?`/`#`, and
leaves POSIX output byte-identical. Strict `mode=ro`, no writable
fallback.
scripts/ensure_state_db_read_indexes.py
- `fcntl` / `msvcrt` are imported conditionally; the module imports and
the no-`--lock-file` path runs on native Windows. `--lock-file` uses
`flock` on POSIX, `msvcrt.locking(LK_NBLCK)` on Windows, and fails with
a clear RuntimeError when neither exists. The rw open reuses the
UNC-aware URI builder.
docs/troubleshooting.md
- Operator section: read-only contract, degraded missing-index listing,
drained maintenance prerequisites, lock behaviour, failure modes, UNC.
Tests: tests/test_pr7445_readonly_review_followups.py (19 tests, red
before this change: correlated-subquery plan, missing URI builder,
ImportError on fcntl). Query-plan contract via EXPLAIN QUERY PLAN — no
CORRELATED SCALAR SUBQUERY on missing-index listings, identical rows to
the indexed listing; platform-parametrised URI cases; maintenance module
imported with `sys.modules["fcntl"] = None`. test_issue2628 no longer
pins the cron-only gate; the flock-held test skips without fcntl.
Records the shipped identity and invalidation contract for the /api/models
catalog cache in api/config.py — what is cached (in-memory snapshot, per-profile
models_cache.json, cold vs hot path), the three source axes and why each is
fingerprinted the way it is, and the one-directional deny-list invariant for the
volatile-key sets. Changes no runtime behavior; added after #7556 shipped, whose
review flagged the contract as undocumented.
Verified before merge: all 14 symbols the doc names exist in api/config.py
(_available_models_cache, _AVAILABLE_MODELS_CACHE_TTL, _get_models_cache_path,
_models_cache_provenance, _sync_models_cache_provenance,
_models_cache_source_fingerprint, _is_loadable_disk_cache,
_models_cache_file_fingerprint, _auth_store_semantic_fingerprint,
_AUTH_FINGERPRINT_VOLATILE_KEYS, _codex_models_cache_fingerprint,
_CODEX_CACHE_FINGERPRINT_VOLATILE_KEYS, _endpoint_advertised_model_ids), and the
cited test_unknown_codex_field_stays_in_fingerprint is real
(tests/test_issue7540_codex_catalog_fingerprint.py:287). docs/architecture/ is
allowlisted in .gitignore with both the dir and /** lines, so the new file is
genuinely trackable.
Co-authored-by: webtecnica <webtecnica@users.noreply.github.com>
* perf(composer): make the single-row autoResize fast path reachable
autoResize() skips the height:'auto' -> scrollHeight -> restore round trip only
when the composer is already at its one-row height, but the guard compared
el.offsetHeight against the CSS min-height (44px) while the composer's natural
one-row border box is line-height + vertical padding + borders (~48px at the
stock font). 48 <= ceil(44)+1 is false, so the skip was unreachable in a real
browser: every append keystroke wrote style.height twice, and the scrollHeight
read that follows forces a synchronous layout of the whole document, transcript
included. The cost therefore grows with the rendered transcript, which is the
"typing gets laggy in long sessions" report.
Accept the natural one-row height as the skip ceiling, computed from the same
already-read getComputedStyle. The strict finite-pixel gate is kept for every
value involved, so a percentage / calc / normal line-height, padding or
min-height still fails closed to the full resize (the #6349 re-gate property is
preserved, and test_issue5514_composer_grow_scroll_pin.py still pins it).
Measured on a 758-message / 5 MB session (31k DOM nodes rendered) with the
browser Event Timing API (input -> next paint) at 6x CPU throttle: a keystroke
went from 168ms to ~136ms median echo latency (72ms with the transcript
detached), 96ms to ~72ms in the windowed state, and the style writes per 8
keystrokes from 16 to 2.
The #5514 invariant holds: an oversized composer still takes the full resize
because its offsetHeight is far above the one-row ceiling.
Tests:
- new tests/test_long_session_composer_typing_latency.py runs the real
autoResize() body from messages.js in a node sandbox whose getComputedStyle
returns the composer's computed styles; it is red before this change
(2 failed, 27 passed) and green after (29 passed).
- tests/test_issue5514_composer_grow_scroll_pin.py source pins updated to the
new guard shape.
* test(docs): address #7604 review - behavior-only tests + composer sizing docs
Review feedback on the PR flagged two repo-rule items: tests that pin source
strings, and an undocumented runtime behavior change. Both are addressed here.
- tests/test_long_session_composer_typing_latency.py: delete
test_fix_is_pinned_in_source. It asserted exact JS declarations, which the
repo testing guidance rejects and which the harness in this module already
covers by executing the REAL autoResize() body. In its place add
test_natural_row_skip_survives_every_append_of_a_typed_word, which drives the
actual typing loop ('' -> 'h' -> ... -> 'hello') and asserts zero height
writes on every keystroke. Red on the pre-fix tree, green with the fix.
- tests/test_issue5514_composer_grow_scroll_pin.py: convert
test_single_line_growth_skips_the_height_round_trip from source-string pins to
a behavior fixture whose getComputedStyle returns the composer's real computed
styles. It asserts the observable outcome: a one-row append at the natural
48px row takes no height write, while an oversized 176px composer still
remeasures back to one row. Red before the fix (2 writes), green after.
- docs/UIUX-GUIDE.md: document the composer sizing invariants in the composer
section - the single-row skip is a hot path and must stay reachable, its
ceiling is the natural one-row height or the CSS min-height (whichever is
larger), everything else still remeasures, and non-px computed values fail
closed.
./scripts/test.sh on the two touched modules: 29 passed (3 failed on the
pre-fix tree). Neighbour sweep over the 51 composer/scroll/typing modules:
386 passed. ruff check clean.
* fix(tests,docs): derive composer test dimensions from style.css (#7604 review)
Greptile's re-review is right: the fixtures hardcoded the 18px
`data-font-size=large` composer (line-height 29.7px -> a 47.7px one-row box) as
if it were the stylesheet's 16px default (44.4px). At the default the pre-fix
ceiling (ceil(44)+1 = 45px) DID clear the one-row box, so the regression is
font-size dependent - the 16px default reaches the skip, the 18px and 20px
appearance settings do not - and not a stock-configuration bug. The runtime
change is unaffected and unchanged.
- new `tests/_composer_metrics.py`: parses `static/style.css` (the base
`textarea#msg` rule, the `data-font-size` overrides, the skin overrides) and
exposes each configuration's box metrics - natural one-row height, the integer
offsetHeight a browser reports, and both the pre-fix and post-fix ceilings -
so a fixture cannot encode the wrong font size again.
- `tests/test_long_session_composer_typing_latency.py`: parameterized over every
configuration the stylesheet can produce, with
`test_the_regression_was_font_size_dependent_not_stock` pinning the grounding.
- `tests/test_issue5514_composer_grow_scroll_pin.py`: the converted single-row
test uses the same derived metrics.
- `docs/UIUX-GUIDE.md`: the sizing section now states that the natural row
tracks the appearance font size (44 / 48 / 51px) while `min-height` stays
44px, which is why a min-height-only ceiling silently disables the skip.
Pre-fix tree: `4 failed, 31 passed` - exactly the large-18px and xlarge-20px
parameterizations plus the two tests that assert the reported configuration.
Post-fix tree: `35 passed`. Neighbour sweep over the 51 composer/scroll/typing
modules: `392 passed`. ruff clean.
Serve the SPA shell for /sessions so the post-login redirect (when password auth is enabled) no longer 404s.
Co-authored-by: Skawsk59 <Skawsk59@users.noreply.github.com>
Decode the state.db content sentinel in the WebUI message projection so structured/multimodal content renders instead of a placeholder, with an out-of-band tuple identity that no scalar can forge and a per-call ContextVar memo for the canonical serialisation.
Co-authored-by: totalitarian <totalitarian@users.noreply.github.com>
Closes the remote POSIX workspace path preservation gap across Session and streaming. Profile-aware runtime path resolution.
Co-authored-by: alfred-rootson <alfred.wojtasordzik@gmail.com>
The /api/models cache identity shipped in #7556 (release #7558) changed from a
stat identity to content fingerprints with volatile-key deny-lists, and the #7556
review flagged that this runtime contract had no project-level documentation.
Record the three source-fingerprint axes and their rationale, the one-directional
deny-list invariant, the stat-identity fallbacks (including the deep-tree
RecursionError path), the independent schema/version stamps, and the change
protocol; index the document from docs/CONTRACTS.md. Docs only.
* rebase #7160 onto master (report unverified Agent updates, no auto-restart)
Co-authored-by: hrdwdmrbl <hrdwdmrbl@users.noreply.github.com>
* harden agent-update marker read: no-follow, fstat regular-file, bounded (fixes gate CORE)
The Agent update marker is attacker-adjacent shared state. _read_live_agent_update()
used Path.read_text(), which follows symlinks, blocks forever on a FIFO, and reads an
unbounded regular file — a rejected/stale-runtime request could hang or OOM. Now opens
O_RDONLY|O_NONBLOCK|O_NOFOLLOW, fstat-verifies a small regular file, and reads a bounded
max, classifying anything else as 'unknown' (fail-closed). Adds FIFO/oversized/symlink/
happy-path regression tests.
Co-authored-by: hrdwdmrbl <hrdwdmrbl@users.noreply.github.com>
* fix marker-read portability: gate os.open fast-path on O_NONBLOCK+O_NOFOLLOW
os.O_NONBLOCK is Unix-only; accessing it unconditionally raised AttributeError
on native Windows, breaking every stale-revision barrier (chat/compression/etc).
Now both flags resolve via getattr and the os.open() fast path is taken only
when BOTH are available (_MARKER_SAFE_OPEN_AVAILABLE); otherwise fall back to an
lstat-only classification (absent vs unknown) so a fallback platform can never
symlink-traverse or crash. Adds a Windows-fallback regression test.
Co-authored-by: hrdwdmrbl <hrdwdmrbl@users.noreply.github.com>
* docs(changelog): stamp #7160 stale-runtime report + hardened marker read
---------
Co-authored-by: n <a@n>
Co-authored-by: hrdwdmrbl <hrdwdmrbl@users.noreply.github.com>
* fix(sse): scope gateway SSE probe and honor Last-Event-ID on /api/chat/stream
Two narrowly-scoped cross-client SSE fixes from the hermes-android#58
follow-up:
1. The /api/sessions/gateway/stream?probe=1 response only describes the
optional gateway/agent-sessions stream, but its payload gave clients no
way to tell that. A 404 'agent sessions not enabled' result could be
misread as SSE being unsupported server-wide, even though the persistent
per-session stream (/api/session/stream) is always on and independent of
the show_cli_sessions setting. Add additive scope markers to the probe
payload (scope, session_stream_available, session_stream_path) and
clarify the handler docstring.
2. /api/chat/stream emitted id: stream_id:seq on every journaled event
but ignored the Last-Event-ID request header, so any spec-compliant SSE
client that auto-reconnects (browser EventSource, Android/CLI clients)
replayed the journal from seq 0 and double-rendered the transcript.
_chat_stream_resume_after_seq() now resolves the resume cursor with
explicit after_event_id/after_seq query params first and the
Last-Event-ID header as fallback — the same resolution chain
api/kanban_bridge.py already uses. Both the dead-stream journal replay
and the live-subscribe gap check consume it; direct callers of the gap
check that only pass query params keep historical behavior.
Also documents the cross-client SSE endpoint contract in
docs/sse-streams.md.
Related hermes-webui/hermes-android#58
* fix(sse): split cursor presence from validity on /api/chat/stream resume
Address PR #6886 review (gate-fail): the resume logic conflated "no
cursor" with "invalid cursor" into a single None, which broke replay on
three edges. Root cause and fix:
Cursor presence vs validity — _chat_stream_resume_cursor() now returns
(after_seq, resume_requested) instead of a bare seq. A client that
SUPPLIED any cursor (query params, replay=1, or Last-Event-ID) asked to
resume; an invalid / foreign-run / unparseable cursor resolves to
after_seq=None but keeps resume_requested=True, and is normalized to
replay-from-start so no journal events are silently skipped. A genuinely
cursor-less request stays a fresh subscribe.
1. Explicit-query precedence is now decided by parameter PRESENCE, not
successful parsing: a malformed/foreign after_event_id/after_seq never
falls through to let the Last-Event-ID header override it and skip
events.
2. Invalid/foreign cursors no longer bypass replay + gap detection and
silently drain a truncated buffer; an ahead-of-stream cursor is no
longer clamped onto the snapshot's terminal fence (which emptied the
dead-stream replay by filtering the terminal frame).
3. The runner-local observe path now receives the resolved header cursor
(retaining explicit-query precedence) so header-only runner reconnects
resume instead of duplicating events from the start.
Regression tests cover invalid-cursor replay-from-start, ahead-of-stream
terminal survival, header-only runner reconnect, and explicit-query
precedence; the stale clamp test is updated to the corrected contract.
Related hermes-webui/hermes-android#58
* fix(sse): consume resolved resume cursor at every consumer on /api/chat/stream
Address PR #6886 r2 review (gate-fail). The r1 presence/validity split
fixed the entry resolution, but the individual consumers still re-derived
ahead/equality/re-parse decisions, so the invariant leaked at the edges.
This makes every consumer honor the resolved (after_seq, resume_requested)
pair and normalizes ahead-of-stream in one place.
1. Equality off-by-one: a valid cursor EQUAL to the snapshot cutoff now
enters the drain dedup bound (the event at the cursor was already
delivered), instead of being treated as "ahead" and double-sending the
retained buffer. Ahead is now strictly after_seq > snapshot_cutoff_seq.
2. Numeric ahead-of-stream cursors are normalized to replay-from-start:
on the active path, after_seq > snapshot_cutoff_seq resets to 0 before
gap-coverage/replay so a truncated buffer can no longer falsely declare
coverage and lose the pre-tail events; on the dead-stream path the
cursor is normalized against the journal summary's authoritative
last_seq before _replay_run_journal, so an ahead cursor returns the
full replay instead of an empty SSE body.
3. Runner-local now consumes the resolved cursor instead of re-parsing
after_event_id or re-reading Last-Event-ID. The resolver also returns
the opaque raw cursor so the runner uses the already-precedence-checked
value; explicit opaque cursor= keeps precedence, invalid/foreign
cursors pass no cursor (full replay), and the header is honored only
when no explicit cursor was supplied.
Regression tests cover the equality dedup, active+dead ahead-of-stream
normalization (with after_seq-filtering mocks that assert the real replay
body), and the three runner precedence probes (malformed-explicit blocks
header, foreign passes no cursor, opaque cursor= wins).
Related hermes-webui/hermes-android#58
* fix(sse): carry runner-cursor provenance through /api/chat/stream resume
Address PR #6886 r3 review (gate-fail). The (after_seq, resume_requested,
raw_cursor) tuple carried the cursor VALUE but not its PROVENANCE, so the
runner handoff couldn't distinguish a valid opaque runner cursor from a
journal-shaped one from a malformed one — and gated the runner handoff on
journal-specific validity, which is wrong for the runner adapter (its
cursors are opaque).
The resolver now returns a distinct, centrally-resolved runner cursor:
- a valid after_seq PAIRS with whatever after_event_id shape was supplied
— including an opaque runner id like event:2 that the journal parser
reads as a foreign run — so the paired runner cursor resumes at the seq
(never None / full replay, which would duplicate tokens and tool
events). after_seq is read directly (via _parse_run_journal_after_seq_value)
rather than through the after_event_id-first journal parser, which would
swallow a paired opaque runner id.
- a header-only opaque runner id (Last-Event-ID: event:2) is preserved
as-is so the runner resumes from it.
- a malformed or foreign explicit cursor WITHOUT a valid paired after_seq
still yields no runner cursor, blocking Last-Event-ID and replaying from
start — the r2 malformed-blocks-header guarantee is unchanged.
Regression tests pin the paired opaque id, header-only opaque id, and
malformed-with-valid-seq runner resumes, plus the unchanged
malformed-without-pair replay-from-start behavior.
Related hermes-webui/hermes-android#58
* fix(sse): normalize ahead cursor when the snapshot cutoff is unknown
Address PR #6886 r4 review (gate-fail). The ahead-cursor normalization
was guarded by `snapshot_cutoff_seq is not None`, so when the subscribe
snapshot had no parseable last_event_id (channel has not seen an
id-bearing frame yet), a positive ahead cursor (e.g. Last-Event-ID
run_1:999) was never normalized — it was installed verbatim as the live
dedup bound, filtering EVERY queued frame including the terminal
stream_end fence. The reconnect stalled on heartbeats with an empty SSE
body. origin/master ignored the header on this path and delivered the
events, so this was a new regression.
Fix: treat an unknown, non-truncated snapshot cutoff as fence 0 before
the ahead-cursor normalization, so a positive client cursor can't become
the live dedup bound with no cutoff to bound it — it fails closed to
replay-from-start, delivering the buffered events rather than silently
losing them. Frames dropped while the cutoff is unknown still cannot
prove gap coverage (cutoff None → not covered), so the recovery_control
fail-closed path is preserved.
Regression test: Last-Event-ID run_1:999 + snapshot_cutoff_seq=None
delivers queued events run_1:1 and run_1:2 exactly once, emits the
terminal stream_end, and releases the subscriber. Without the fix the
test stalls on heartbeats (pytest-timeout kills it), proving the stall.
Related hermes-webui/hermes-android#58
* Release — SSE Last-Event-ID resumption fix (#6886)
---------
Co-authored-by: Paladin173 <35980893+Paladin173@users.noreply.github.com>
Co-authored-by: n <a@n>
* Add scoped extension Configure hook
* docs: fail-closed optional chaining in Configure consumer example + old-Core regression
The documented consumer used ext?.settings.registerConfigure(...), which throws
TypeError on an older E0 Core where settings exists but registerConfigure does not.
Use full optional chaining (ext?.settings?.registerConfigure?.()) and add a Node
regression proving the consumer survives a Core lacking the capability.
Co-authored-by: franksong2702 <franksong2702@users.noreply.github.com>
---------
Co-authored-by: Frank Song <franksong2702@gmail.com>
Co-authored-by: n <a@n>
Co-authored-by: franksong2702 <franksong2702@users.noreply.github.com>
* fix(tests): make mtime_invalidation and glm_5_3 tests order-independent (#7100)
* fix: expose max reasoning for GPT-5.6 models
* fix(recovery): do not reattach cancelling runs
ACTIVE_RUNS tracks worker lifecycle, which is deliberately broader than
"a turn a browser may attach to". cancel_stream() keeps the row as
phase="cancelling" while the worker unwinds so a successor turn cannot
start on top of it, but the client has already reached a terminal state
for that stream: its run journal ends in a terminal event.
The recovery lookups treated every same-session ACTIVE_RUNS row as
attachable. An idle session holding a cancelling row therefore received a
recovered server_turn_started on every /api/session/stream subscription:
the client attached, replayed the terminal event, tore the renderer down,
resubscribed, and the server replayed the same frame again. The result is
an endless attach/replay loop that rebuilds the transcript repeatedly.
Separate the two meanings instead of narrowing one call site:
- api/config.py gains active_run_is_attachable() and
active_run_cancel_is_stale() as the shared predicates.
- active_stream_id_for_session() (browser recovery) returns attachable
rows only; _session_has_active_turn() (busy check) keeps counting a
fresh cancellation so a successor cannot overlap the unwinding worker.
- _live_active_stream_id() applies the same rule to the hidden-tab
status poller, on both the STREAMS and ACTIVE_RUNS paths.
- routes._cancelled_run_is_stale() now delegates to the shared predicate
rather than keeping a parallel copy of the staleness rule.
- A cancelling row past a bounded unwind window with no live STREAMS
channel is reclaimed from ACTIVE_RUNS and its stream owner released, so
a wedged worker cannot suppress background wakeups forever. Age alone
does not reclaim a row that still owns a live channel.
Staleness anchors on cancelled_at, falling back to started_at, so a
long-running turn cancelled moments ago is never treated as an orphan.
tests/test_cancelling_run_not_attachable.py covers both directions of
each rule. The six behavioral tests fail on the unpatched tree and pass
with the fix; two targeted mutations (forcing the attachability predicate
true, and disabling the staleness reaper) each turn the suite red.
* docs(rfc): document cancellation attach/admission split
Records the runtime contract introduced by the recovery fix in the WebUI
run-state consistency RFC, so the distinction is discoverable instead of
living only in code comments.
- Adds ACTIVE_RUNS to the State Layers table as the worker-lifecycle
registry, explicitly not the set of runs a browser may attach to.
- Adds invariant 9: lifecycle-busy is not client-attachable. Cancellation
splits the two meanings, recovery paths must exclude cancelling rows,
and admission checks must keep counting them.
- Documents the bounded cancellation-unwind window: reclamation needs
both age and the absence of a live STREAMS channel, and staleness is
anchored on the cancellation timestamp.
- Extends the review checklist with the admission-vs-attachment question
and the evidence required when changing a reclamation window.
* docs(changelog): note #7096 cancelling-run reattach, #7083 GPT-5.6 max reasoning, #7101 test order-independence
---------
Co-authored-by: webtecnica <webtecnica@gmail.com>
Co-authored-by: Abdulrahman Elkenany <boudy.elkenany123@gmail.com>
Co-authored-by: allenliang2022 <allenliang2022@users.noreply.github.com>
Co-authored-by: n <a@n>
* feat: add GLM-5.3 to Z.AI model list
Add glm-5.3 as the newest zai entry in _PROVIDER_MODELS and a matching
zai/glm-5.3 entry in _FALLBACK_MODELS, and bump the Z.AI onboarding
default_model from glm-5.1 to glm-5.3 (Z.ai's current flagship; legacy
GLM-5.2/5.1 requests are routed to GLM-5.3 per docs.z.ai).
Reasoning gating needs no change: _zai_glm_classification() treats
GLM >= 5.2 as effort-ladder capable, so glm-5.3 is already covered and
pinned by tests/test_zai_reasoning_effort_gating.py.
New regression coverage in tests/test_glm_5_3_catalog.py: catalog
presence, newest-first ordering, fallback entry, onboarding default,
full reasoning_effort ladder, and the get_available_models() payload.
* test: restore _cfg_fingerprint in catalog test fixture
Review follow-up (#7017): the isolation fixture snapshot restored cfg,
_cfg_mtime, and _cfg_path but left _cfg_fingerprint pointing at the
temporary config loaded by the payload test. api/config.py uses that
fingerprint to distinguish in-memory overrides from changed files
(config.py:371), so a stale value could make later same-process tests
skip reloading a changed config. Snapshot and restore it like the rest.
* fix: keep Z.AI onboarding default at glm-5.1 until direct API serves GLM-5.3
Review follow-up (#7017): GLM-5.3 is live on Z.ai's Coding Plan
endpoint only; the direct api.z.ai endpoint the zai provider uses
still lists the GLM-5.3 API as coming soon. Defaulting new direct-API
users onto glm-5.3 would fail their first message, so the catalog
addition stays (opt-in) and the default stays glm-5.1. Bump the
default in a follow-up once the direct endpoint serves GLM-5.3.
* fix(docker): probe the configured state dir before /workspace for UID/GID (#7027)
UID/GID auto-detection probed /workspace before the configured
HERMES_WEBUI_STATE_DIR. In a stock single-container image none of the
priority-1 candidates exist, but /workspace does — owned by the image's
build-time 1024:1024. Detection therefore returned the image's own owner,
which carries no information about the host, while the one directory whose
owner *is* the host UID by definition — the state-dir bind mount — was never
probed. With a host-owned state mount and no explicit WANTED_UID the
container remapped to 1024, failed its own state-dir writability check, and
restart-looped.
The log line made this expensive to debug: 1024 is also the fallback default,
so "Auto-detected workspace UID: 1024" read as if detection had found nothing.
- probe ${HERMES_WEBUI_STATE_DIR:-/app/data} first, for both UID and GID
- keep /workspace as a lower-priority signal (unchanged for setups that
actually bind-mount it), and keep the hermes-home probes from #668
- stop treating an explicitly supplied 1024 as "unset": the sentinel and a
valid UID were the same number, so an operator who deliberately ran as 1024
got it overwritten by detection. The explicit/detected origin is persisted
next to the value because `su` drops the environment when the script
re-enters as the runtime user, so the second pass would otherwise see an
explicit choice as a detected one.
Tests: tests/test_7027_state_dir_uid_probe.py runs the real resolution block
under bash with `stat` stubbed, covering the non-1024 state mount, the
explicit 1024 override (including across the privilege drop), and the
pre-existing /workspace + hermes-home fallback paths. A new state-dir-uid job
in the Docker smoke workflow boots a real container on a host-owned state
mount and gates on /health.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Release batch B: Docker UID fix + GLM-5.3 catalog (experimental)
Two independently gate-passed contributor PRs, rebased fresh onto master and
re-gated as a combined stage (Codex SAFE TO SHIP; full suite green except the 8
pre-existing approval-test failures that fail identically on clean origin/master
— CI green on the same commit; tracked separately for a fix).
- #7027 (@jorgejiro) probe configured state-dir before /workspace for Docker
UID/GID; persist explicit marker so a supplied 1024 survives root->su re-entry
(fixes the single-container restart loop) (#7034)
- #7017 (@rh-id) add GLM-5.3 to the Z.AI model list (onboarding default stays glm-5.1)
Co-authored-by: jorgejiro <jorgejiro@users.noreply.github.com>
Co-authored-by: rh-id <rh-id@users.noreply.github.com>
---------
Co-authored-by: Ruby Hartono <58564005+rh-id@users.noreply.github.com>
Co-authored-by: Jorge <jorgejiro@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: n <a@n>
Co-authored-by: jorgejiro <jorgejiro@users.noreply.github.com>
Co-authored-by: rh-id <rh-id@users.noreply.github.com>
* feat(docker): enable gateway API and password forwarding in compose files
- Added API_SERVER_KEY forwarding to enable the gateway API listener in multi-container setups.
- Configured HERMES_WEBUI_PASSWORD to ensure password authentication works as expected.
- Updated documentation to clarify API_SERVER_KEY requirements and forwarding behavior.
- Added regression tests to cover the new functionality and ensure proper configuration.
* build(docker): force LF line endings for shell scripts via .gitattributes
A CRLF shebang (#!/bin/bash\r) breaks locally-built Docker images: the
kernel can't resolve the interpreter /bin/bash\r, so 'docker compose up
-d --build' on Windows with core.autocrlf=true fails with
'exec /hermeswebui_init.bash: no such file or directory'. The repo blobs
and CI/GHCR builds are LF and unaffected; only Windows local builds
baked the CRLF checkout into the image.
* fixup(docker): dedup API_SERVER_HOST after rebase onto #6968 (which already added it)
* docs(changelog): stamp #6917 (multi-container gateway API forwarding)
---------
Co-authored-by: Yang <yang_qwq@qq.com>
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
* fix(title): honor auxiliary.title_generation.enabled=false in WebUI title paths (#6814)
Hermes Agent gates title generation on auxiliary.title_generation.enabled
(default true), but the WebUI orchestration only used the aux config to pick
a model route, so enabled=false still triggered LLM title calls via the
active chat model or the aux route, and could persist a fallback title.
Add a profile-scoped _aux_title_generation_enabled() helper beside
_get_aux_title_config() with the same default/truthy semantics as the Agent
(boolean and string forms). Gate three automatic paths on it:
- _run_background_title_update(): skip with status skipped /
title_generation_disabled, provisional title untouched, no fall-through
to the active agent model.
- _run_background_title_refresh(): skip with refresh_skipped /
title_generation_disabled.
- generate_session_title_for_session(): explicit (None,
'title_generation_disabled', '') so manual regenerate returns an
explicit disabled response and imported-session auto-titling stays quiet.
Checks run inside the profile-scoped worker context so a background job
cannot read another profile after a switch.
Tests (tests/test_issue6814_title_generation_enabled_gate.py, 20 cases):
default-on when omitted, disabled with/without aux route (no LLM call,
title untouched, skipped status, stream_end still emitted), string/boolean
false forms, adaptive refresh path, manual regenerate explicit response,
helper parsing matrix. fail-before: 19 failed on unpatched code.
Regression: title family 154 passed (incl. test_2235 routing cases).
* docs: document auxiliary.title_generation.enabled behavior (#6814)
Addresses the Greptile #6877 review note: document the newly enforced
setting and the manual-regenerate disabled response in
docs/advanced-chat-setup.md.
* fix(title): mirror Hermes agent's canonical is_truthy_value contract exactly
Gate review (Codex) found the enabled-parser diverged from the agent's canonical
contract: it used a 'disable on false/0/no/off' blocklist, but the agent uses
is_truthy_value(value, default=True) — an allowlist (utils.TRUTHY_STRINGS =
{1,true,yes,on}). So None wrongly routed through the string branch and empty/
unrecognized strings wrongly stayed ENABLED. Now: None->True (default), bool
unchanged, str truthy ONLY for {1,true,yes,on}, else bool(value) — byte-for-byte
with agent/title_generator._auto_title_enabled. Test matrix corrected + expanded.
Co-authored-by: X-Shihai <X-Shihai@users.noreply.github.com>
---------
Co-authored-by: X-Shihai <48356232+xushihai@users.noreply.github.com>
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
Co-authored-by: X-Shihai <X-Shihai@users.noreply.github.com>
Anthony Ruiz