Codex CORE findings on the fingerprint: (1) default busy timeout could stall
/api/sessions ~10s on a locked state.db; (2) COUNT(*) forced a full messages
table SCAN (~16-22ms/refresh on 200k rows). Fix: open file:...?mode=ro with
timeout=0.05 + PRAGMA busy_timeout=50 (returns fast, falls back to stat stamp on
lock), and use index-only MAX(rowid) per table (no COUNT, no scan). Verified:
EXPLAIN shows SEARCH not SCAN, 0.36ms on 200k msgs, 0.16ms while EXCLUSIVE-locked.
Fingerprint shape is now (sessions_max_rowid, messages_max_rowid); regression
tests updated.
Codex root-cause consult confirmed: the CLI-session cache (_CLI_SESSIONS_CACHE,
5s TTL) and session-list cache were keyed on (st_mtime_ns, st_size) of state.db
+ WAL sidecars. Under WAL-mode writes those stamps collide (same mtime-ns bucket
+ WAL frame size after checkpoint), serving a stale cache -> freshly-committed
gateway/CLI session intermittently absent from /api/sessions (the recurring
test_gateway_sync flake AND a real up-to-5s sidebar delay for gateway writes).
PRAGMA data_version is NOT usable here (fresh per-request connection always
reports its own initial value, never advances — verified). Fix = a cheap
content fingerprint (COUNT/MAX(rowid) over sessions+messages) read on a fresh
connection, which DOES advance on every commit and is immune to mtime
granularity. Wired into _sqlite_file_stat_cache_key (models) + the routes-layer
source stamp. + 3 regression tests + corrected the earlier false root-cause
comments.
Codex completeness finding: show_previous_messaging_sessions is a third
visibility setting in the /api/sessions cache key (bypasses messaging dedupe)
with the same stale-cache bug. Added it to the invalidation tuple.
ROOT FIX (product bug): POST /api/settings never invalidated the session-list
cache, so toggling show_cli_sessions/show_cron_sessions could serve a stale
sidebar for up to the cache TTL when the settings-file mtime stamp collided with
a cached entry. The handler now clears the cache directly on those toggles. This
is also the root cause of the recurring test_gateway_sync CI flake (session
absent from /api/sessions right after the visibility toggle).
DEFENSE-IN-DEPTH: cleanup_test_sessions fixture now resets show_cli_sessions to
default BEFORE each test too, not only in teardown.
FLAKE FIX (test infra): bump 31 node-subprocess correctness-harness timeouts from
10s -> 30s across 23 test files. Node spawn + V8 warmup under parallel CI shard
load can exceed 10s, causing TimeoutExpired flakes (e.g. test_1325 on #4445).
Only real node subprocess.run() sites changed — mock signatures, HTTP urlopen,
and deliberate timeout-behavior tests (test_api_timeout, timeout=0/1/2) untouched.
Display-only sidebar fix (does not mutate archived state; rows reappear with
show-archived). Chosen as the interim #4293 fix since the backend-cascade
alternative #4423 was bounced for a fork-archival mutation bug.
Co-authored-by: santastabber <santastabber@users.noreply.github.com>
Closes#4414Closes#4293
Codex re-gate found two issues in the uncapped-read fix: (1) open_anchored_fd
opens blocking O_RDONLY, which HANGS on a workspace FIFO/special file swapped in
at a regular-file path; (2) the non-regular fstat branch returned without
closing the fd (leak). Fix: lstat the symlink-resolved target BEFORE any open to
skip non-regular leaves (no blocking open on a FIFO), and close the fd on the
non-regular fstat branch. Adds a threaded FIFO-no-hang regression test.
Codex regression gate found that routing workspace reads through
read_file_content() introduced a 400KB MAX_FILE_BYTES cap not present on master
(Path.read_text), so a large but legitimate workspace file raised ValueError ->
caught -> None -> the rollback diff falsely reported it as DELETED. Replace with
an uncapped anchored O_NOFOLLOW read (safe_resolve_ws + open_anchored_fd +
S_ISREG fstat check) that keeps the symlink-escape protection but has no size
cap. Adds a >400KB regression test (RED on the capped path, GREEN with the fix).
Co-authored-by: Hinotoi-agent <Hinotoi-agent@users.noreply.github.com>
Absorbs #4412 (tomas-forgac) rebased onto fresh master, plus two
Codex-regression-gate fixes: (1) use the scheme that actually answered the
probe for the ready/already-up URL + browser-open, (2) honor an explicit
HERMES_WEBUI_HEALTH_URL override in the WSL autostart probe.
Co-authored-by: tomas-forgac <tomas-forgac@users.noreply.github.com>
Closes#4412
Codex regression gate found that _session_model_identity_matches normalized
only the @provider:model form, not provider/model (slash). A session stored as
'deepseek/deepseek-v4-1m' resolving to bare 'deepseek-v4-1m' was treated as a
model change -> the 256k fallback bypassed the accept-guard and clobbered the
persisted 1M window (the same #4248 bug on slash-qualified IDs). Mirror the
slash-aware split already used by _model_matches_configured_default. Adds a
route-level regression + unit test for the slash shape.
Co-authored-by: franksong2702 <franksong2702@users.noreply.github.com>
Restore previously used pathname-based shutil.copy2() which follows an in-tree
symlink whose target escapes the workspace. Route restore writes through the
workspace-anchored file helpers so a restored file always lands inside the
workspace tree. +regression test (escaping symlink refused, normal restore ok).
Co-authored-by: hinotoi-agent <paperlantern.agent@gmail.com>
Pastes over 4000 chars or 100 lines attach as a timestamped pasted-text-*.md in
the upload tray instead of flooding the composer. Image-paste path unchanged; an
oversize paste that exceeds MAX_UPLOAD_BYTES falls through to a normal inline
paste so nothing is lost.
Co-authored-by: santastabber <204424800+santastabber@users.noreply.github.com>
The outline FAB was position:fixed at z-index 9999 and could stack on top of the
scroll-to-bottom button. Anchor it inside the messages shell (position:absolute,
z-index 12) so the two controls don't overlap.
Co-authored-by: Mohamed Habib <Habib1001-m@users.noreply.github.com>
A user prompt that exists only in Hermes state.db (no sidecar copy) and falls
chronologically BEFORE a newer sidecar message tail was being appended after the
tail (or skipped) by merge_session_messages_append_only — losing or mis-ordering
it on load. The merge now inserts such a state.db user message at its correct
chronological position via _insert_state_message_chronologically, with a
role-aware tie-break for equal timestamps. Append-only invariant + dedup
(seen_content_keys) preserved.
Deterministic-ordering guard (maintainer add, addresses Codex review): on an
equal-timestamp collision the insert no longer lands at a slot whose left
neighbour shares the message's role — it advances past the same-role
equal-timestamp run, so an already-answered earlier turn can't be re-ordered
into user,user,assistant. (The agent core merges adjacent users before send, so
this was benign in practice, but the guard keeps the merged transcript correctly
ordered regardless.) +1 same-second multi-turn regression test, toothless-checked.
Co-authored-by: Dennis Soong <dso2ng@gmail.com>
Multi-container Docker deploys died at startup with 'could not create
hermes_agent.egg-info: Permission denied'. docker_init.bash stages the :ro
hermes-agent-src into /tmp/hermes-agent-build to avoid EROFS, but rsync -a/cp -a
preserve mode bits — a source mounted 555 left the staged copy 555, so
setuptools couldn't write egg-info next to the package. chmod -R u+w the staged
tree after the copy (error_exit on failure) so the build dir is genuinely
writable, not just owned by hermeswebui.
Co-authored-by: enihcam <enihcam@users.noreply.github.com>
Follow-up to #4385/#4388: that fix stopped stale cron sidecars from being
treated as CLI rows, but get_cli_sessions() still re-projected the same cron
run from Hermes state.db with a hardcoded archived: false after all_sessions()
omitted the hidden sidecar — so an archived run reappeared on refresh.
- New _state_projection_sidecar_metadata(sid) helper returns UI-owned title +
archived flag from the WebUI sidecar (load_metadata_only), degrading
gracefully to {title:None, archived:False} on any miss.
- Both state.db-projection paths (standard CLI + cron fallback) now use it and
carry the archived flag onto the projected row (was hardcoded False), so an
archived cron/tool/API run stays hidden regardless of which path surfaces it.
Title-preference behavior (#1486 rename persistence) is preserved.
Co-authored-by: TomBanksAU <246584738+TomBanksAU@users.noreply.github.com>
- Hamburger button: 32x32 -> 44x44 tap target (iOS HIG minimum), SVG icon
20 -> 24px. flex-shrink:0 preserved; spacer keeps the title centered.
- Edge-swipe-to-open-sidebar: edge zone 28 -> 80px, trigger 72 -> 64px, max
vertical 48 -> 56px; removes the _isPwaStandalone() gate in
_onPwaSidebarSwipeStart so the gesture works in a normal mobile browser, not
only the installed PWA. Still _isDesktopWidth()-blocked, touch/pen-only,
interactive-target-excluded, and requires horizontal intent (dx>=64 &&
dx>|dy|*1.5), so scroll/tap won't trigger it.
Known tradeoff (Nathan-approved ship-as-is): in mobile BROWSER the 80px left
edge zone overlaps iOS Safari's back-swipe; degrades gracefully (PWA, the
primary mobile target, has no back gesture).
Verified live at 390x844: hamburger renders 44x44 + tap opens sidebar; a
left-edge swipe opens the sidebar in a non-PWA context. 66 existing mobile/
swipe/titlebar tests pass.
Co-authored-by: kaishi00 <kaishi00@users.noreply.github.com>
Paused/disabled cron jobs are partitioned out of the active list into a
collapsible 'Paused (N)' <details> section so they don't drown the active
jobs. _cronList stays the single source of truth — only the render is split,
so openCronDetail / _cronNewJobIds / detail-refresh are untouched. Reuses the
already-translated cron_status_paused label (zero new i18n keys). Collapsed
state persisted in localStorage.
Maintainer polish on absorb: wrapped both localStorage read/write in try/catch
(repo convention — a storage throw in private mode must not dump the cron
sidebar into the error banner) + added the CHANGELOG entry.
Co-authored-by: Sanjay Santhanam <51058514+Sanjays2402@users.noreply.github.com>
The A3 flake (test_sessions_response_backfills_imported_messaging_source_metadata
failing with StopIteration on a sharded CI run while passing in isolation +
sequentially) had two compounding causes, both fixed:
1. ROOT CAUSE — show_cli_sessions setting leak across shard-ordered tests.
33 gateway_sync tests flip show_cli_sessions=True but only ~30 reset it,
and cleanup_test_sessions didn't reset it. Under pytest-shard ordering a
test could run with the setting in an unexpected state, so its session was
occasionally absent from /api/sessions. cleanup_test_sessions now resets
show_cli_sessions to its default (off) after every test, the same way it
already resets last_workspace.
2. SYMPTOM — bare next(generator) with no default raised an opaque
StopIteration (read as a mystery flake) instead of a diagnosable failure
when the session was absent. The 5 bare next() call sites over /api/sessions
results now use next(..., None) + an explicit assertion naming the missing
session, so any future isolation regression fails clearly instead of
masquerading as a flake.
No production code change; tests + fixture only.
Two coherent fixes for the #4346 transcript-virtualization scroll-jump
cluster, shipped together (contributor-approved) because both touch
static/ui.js + the same virtualization test file and interact (the
cache-reset path uses the per-role default from #4367). Both only run when
the default-OFF experimental "Virtualize long transcripts" setting is on.
#4367 — per-role virtual scroll height estimates:
Replaces the flat 140px unmeasured-row estimate with per-role fallbacks
(user 120 / assistant 160 / tool_call 400 / default 140) via
_messageVirtualDefaultHeightForRole + _messageVirtualRoleForEntry, so
tool-call rows (which measure 500-2000px) aren't under-counted before
measurement. Additive + backward-compat (opts.roleForIdx optional → flat
default when absent). The two sibling offset calculators route through the
per-role defaults too.
#4368 — anchor-based scroll compensation during measurement:
Adds _compensateScrollForMeasurementDelta (capture anchor + scrollTop →
render → re-pin scrollTop by the anchor row's measured delta; null/missing/
<2px guards; programmatic-scroll flag managed) wrapping
renderMessages({preserveScroll:true}) in _scheduleMessageVirtualizedRender,
plus a 150ms scroll-active guard deferring measurement refresh while the user
scrolls, and session-switch cleanup of the three scroll-settle globals.
Maintainer note: combining the two required the test harness for
test_clear_height_cache_resets_scroll_settle_globals (#4368) to also eval
_messageVirtualDefaultHeightForRole + define MESSAGE_VIRTUAL_DEFAULT_ROW_HEIGHTS
(#4367), since the combined _clearMessageVirtualHeightCache resets the estimate
via the per-role helper. Both PRs' test sets retained.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
A stale sidecar with is_cli_session=true could make an archived
cron/tool/API session resurface in the Sessions tab on refresh. The
session loader now treats explicit cron/tool/api source metadata as
non-CLI even when the sidecar claims is_cli_session, and preserves cron
identity when the archive/materialization fallback creates a session.
Co-authored-by: TomBanksAU <246584738+TomBanksAU@users.noreply.github.com>
Adds WebUI controls to manage the messaging gateway lifecycle without a
terminal. _handle_gateway_lifecycle routes start/stop/restart to the
existing platform-aware `hermes gateway {action}` CLI via a list-argv
subprocess (no shell), scoped to the active profile, with bounded
timeout/error handling and a status-card refresh. Auth + CSRF gated;
profile name charset-validated; stdout/stderr never leaked to the client.
Maintainer hardening (warm-up SHOULD-FIX): added _GATEWAY_ACTION_LOCK, a
server-side single-flight guard. The client disables its button in-flight,
but a scripted authed client could otherwise fire overlapping
start/stop/restart calls and spawn concurrent gateway subprocesses. A
non-blocking acquire now returns 409 on contention (mirrors the self-update
_apply_lock pattern), released in finally. Added a regression test
(toothless-checked: without the lock the 2nd action spawns a subprocess +
returns 200).
Cross-platform: delegates to the mature hermes_cli gateway subcommand
(systemctl/launchctl), pathlib paths, PYTHONUTF8=1 + BROWSER=echo in env.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
The inline PDF preview rendered only page 1, so multi-page documents
looked clipped. loadPdfInline now renders every page (capped at MAX_PAGES=20
for memory) sequentially into a scrollable .pdf-preview-body, shows the page
count in the header, and appends a "Showing the first N of M pages" notice
(pdf_truncated, 13 locales) when the PDF exceeds the cap.
Robustness: renderPage(i) attaches each canvas only AFTER its render
promise resolves, and a per-page failure is caught and skipped
(`.then(()=>renderPage(i+1)).catch(()=>renderPage(i+1))`) so one corrupt
page can't halt the preview or surface an unhandled rejection. XSS-safe
(esc(fname), t() with numeric args).
Co-authored-by: Gouranga satapathy <29521030+gourangasatapathyvit@users.noreply.github.com>
"Reply with selection" no longer appends the selected text straight into
the composer. It now adds a named chip ("Context N") to a row above the
message box (composerSelectionChips). Chips can be renamed (double-click,
esc()-escaped) or removed (x); multiple stack. On send(),
_flushSelectionBlocksToComposer() materializes pending blocks into the
composer as **Name:** + markdown-quoted text, then the normal send path
runs unchanged on composer.value.
Regression-safety:
- _flushSelectionBlocksToComposer early-returns when no blocks are pending,
so send() is byte-identical for anyone who never uses the feature.
- loadSession() AND newSession() both clear pending blocks before replacing
S.session, so chips never leak across a sidebar switch OR a New Chat
(the New Chat clear + its regression test were added per Codex review).
- chip name/text rendered via esc(); XSS-safe.
Verified live (headless CDP drive): add/render/remove/rename, flush
produces correct markdown, empty-flush no-op, session-switch clears,
injected <img onerror=...> chip-name escaped (not executed).
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Surfaces the Hermes agent's LLM Wiki as a read-only knowledge-base
observability section in Insights (status counts + browsable read-only
pages). Unconfigured shows a clear "Unavailable" empty state.
Strictly read-only, auth-gated. The wiki file-serving path is hardened
against the full traversal/symlink/TOCTOU class (Codex + Opus review, each
vector regression-tested + toothless-checked):
- _llm_wiki_page_files resolves the wiki root once as the trust base;
skips a section whose resolved path escapes the root; requires each *.md
target's RESOLVED path under both real root and real section; rejects
dot-prefixed segments on lexical AND resolved-relative paths (closes
symlink-to-hidden/out-of-tree + symlinked-section escapes). Browse + read
share this allowlist.
- /api/wiki/page rejects abs paths + a real '..' SEGMENT (legit v1..v2.md
still opens); requires the resolved target in the allowlist; reads the
resolved path with O_NOFOLLOW; and fstat-verifies the open fd's
(st_dev, st_ino) against the identity captured at allowlist time — closing
final-component AND parent-directory symlink-swap TOCTOU races. Any
mismatch/vanish -> clean 404, never a 500.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
The WebUI gateway chat backend routed every browser chat turn through the
gateway's /v1/runs API whenever the gateway advertised approval/runs
support (gateway_chat.py capability gate). The runs API creates an isolated
Api_Server session per turn, so mid-conversation context was lost between
messages. Routing through /v1/runs is now gated behind an explicit opt-in
(_gateway_use_runs_api_enabled: HERMES_WEBUI_GATEWAY_USE_RUNS_API env var or
webui_gateway_use_runs_api config key, env-over-config precedence, default
off). The capability gate now requires BOTH the opt-in flag AND gateway
approval support before using /v1/runs; otherwise normal chat stays on
/v1/chat/completions, preserving the session via X-Hermes-Session-Id.
Operators who want the in-gateway-chat approval-events flow re-enable it
with the flag.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Ships @dso2ng's fix: /api/session now reconciles stale CLI source flags
against the state.db WebUI-origin row, and _isExternalSession() refuses to
classify an explicit WebUI-source session as external — so a WebUI-native
transcript with a stale sidecar isn't force-reloaded like an imported one.
Rebased onto master (contributor's net code-diff applied; byte-identical to
PR head). CHANGELOG stamped v0.51.474.
Co-authored-by: dso2ng <dso2ng@users.noreply.github.com>
Opus review note: the substring match in _stale_prefix_matches_prior_user_context
is intentionally loose, but correctness depends on the caller's
suffix-equals-entire-submitted-turn invariant (cleaning can only ever rewrite a
flagged row to the user's actual current turn — never drop legitimate content).
Comment-only; no behavior change.
test_fade_text_effect_uses_dynamic_window_check used PANELS_JS.index(needle,
300000) to skip the payload-builder occurrence and land on the listener block.
The #3850 searchable-settings additions shifted byte offsets so the magic 300000
now lands on the payload-builder occurrence instead → false failure (the actual
listener code is unchanged + correct). Anchor on the listener-specific
'fadeTextCb=...\n if(fadeTextCb){' instead. No production code change.
Codex+Opus review fixes:
1. Index query missed the common toggle shape <label><input><span data-i18n>>
— only label[data-i18n] was indexed, so most checkbox settings (Virtualize
long transcripts, Browser notifications, etc.) were silently unsearchable.
Broaden to 'label[data-i18n], label [data-i18n], label' and make
_resolveSettingsField resolve any [data-i18n] node to its .settings-field.
2. Stale async-render race: _beginSettingsPanelSession now bumps
_settingsSearchSeq + clears the input/results, and the outside-click dismiss
handler also bumps the seq, so an in-flight index build from a prior session
can't paint into the dropdown. + 4 regression tests.
Ships @rodboev's searchable-settings feature (#3850/#3936): a search box in
Settings that finds any field across all 5 sections with breadcrumbs +
click-to-navigate+highlight+clear. Rebased fresh onto master (post v0.51.470),
5 feature files byte-identical to PR head, re-stamped v0.51.471 (was v0.51.469,
collided with the shipped transcript-virtualization release).
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Virtualization (#500/#4214) caused a scroll-up flicker on long sessions with
tall tool-call rows (variable-height anchor oscillation), distinct from the
stream-completion jump fixed in #4325. Per maintainer decision, flip the
'Virtualize long transcripts' preference to experimental/opt-IN (default OFF)
for everyone until the scroll behavior is reliable.
- api/config.py: virtualize_transcript default True->False; add
virtualize_transcript_optin marker (both bool-allowlisted); load_settings
force-off migration — a stored true WITHOUT the post-flip opt-in marker is
reset to off, so 100% of existing users land on off; an explicit opt-in
(marker present) is honored.
- boot.js / panels.js: default-off semantics (===true), persist the opt-in
marker on enable, checkbox not pre-checked.
- index.html / i18n.js (en): relabel experimental + flicker warning.
- Tests: rewrite the #4325 toggle assertions for the opt-in contract + 4 new
migration tests (unset/stale-true/explicit-optin/marker-without-true).
Phase B (root-cause scroll-stability) tracked separately. Thanks @b3nw.
The opt-out toggle's change handler wrapped renderMessages in a silent
empty catch. Keep the defensive try (a toggle must never wedge the
settings handler) but console.warn on failure so a broken render is
diagnosable instead of swallowed.
Render symlinks in the workspace file tree with a distinct chain-link
icon (vs regular file/folder icons), and show "Symlink → {target}" on
hover with long targets middle-elided. Directory and file symlinks are
both handled; regular entries keep their normal icons.
Rebased onto master (v0.51.466) and code-diff verified byte-identical to
the contributor's PR head before staging.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Two side-by-side PWA standalone windows saturate the browser's per-origin
HTTP/1.1 connection limit, causing 'Request timed out' toasts to cycle every
~30s. The idle-SSE close added in #3992/#3996 keys off the Page Visibility API
(visibilitychange / document.hidden), but a PWA standalone window does not
reliably fire visibilitychange when it loses focus to another window of the
same app -- document.hidden only flips on minimize. So both windows stay
'visible', each holds its two global sidebar SSE streams (session-events +
gateway) open, 2x3 = 6 hits the connection cap, and subsequent fetch() calls
(the 30s background polls) queue behind the saturated pool and time out.
The two global sidebar streams now also close on a sustained window blur (gated
on document.hasFocus(), the signal visibilitychange misses) and reopen --
catching up the session list -- on focus, via a shared _sidebarSseBackgrounded()
predicate and a debounced _installSidebarSseFocusHook().
The per-session live stream (messages.js startSessionStream) is intentionally
left visibility-only: it carries live bg_task_complete toasts and
server_turn_started live-view that an unfocused-but-visible window must still
receive (the OS-notification path is gated on document.hidden, so the in-app
toast is the only completion signal a visible-unfocused window gets). Closing
it on blur would regress the multi-window live-view UX.
Reported by @ddiall with a thorough root-cause analysis.
Co-authored-by: ddiall <ddiall@users.noreply.github.com>
nesquena-hermes