* fix(sse): scope gateway SSE probe and honor Last-Event-ID on /api/chat/stream
Two narrowly-scoped cross-client SSE fixes from the hermes-android#58
follow-up:
1. The /api/sessions/gateway/stream?probe=1 response only describes the
optional gateway/agent-sessions stream, but its payload gave clients no
way to tell that. A 404 'agent sessions not enabled' result could be
misread as SSE being unsupported server-wide, even though the persistent
per-session stream (/api/session/stream) is always on and independent of
the show_cli_sessions setting. Add additive scope markers to the probe
payload (scope, session_stream_available, session_stream_path) and
clarify the handler docstring.
2. /api/chat/stream emitted id: stream_id:seq on every journaled event
but ignored the Last-Event-ID request header, so any spec-compliant SSE
client that auto-reconnects (browser EventSource, Android/CLI clients)
replayed the journal from seq 0 and double-rendered the transcript.
_chat_stream_resume_after_seq() now resolves the resume cursor with
explicit after_event_id/after_seq query params first and the
Last-Event-ID header as fallback — the same resolution chain
api/kanban_bridge.py already uses. Both the dead-stream journal replay
and the live-subscribe gap check consume it; direct callers of the gap
check that only pass query params keep historical behavior.
Also documents the cross-client SSE endpoint contract in
docs/sse-streams.md.
Related hermes-webui/hermes-android#58
* fix(sse): split cursor presence from validity on /api/chat/stream resume
Address PR #6886 review (gate-fail): the resume logic conflated "no
cursor" with "invalid cursor" into a single None, which broke replay on
three edges. Root cause and fix:
Cursor presence vs validity — _chat_stream_resume_cursor() now returns
(after_seq, resume_requested) instead of a bare seq. A client that
SUPPLIED any cursor (query params, replay=1, or Last-Event-ID) asked to
resume; an invalid / foreign-run / unparseable cursor resolves to
after_seq=None but keeps resume_requested=True, and is normalized to
replay-from-start so no journal events are silently skipped. A genuinely
cursor-less request stays a fresh subscribe.
1. Explicit-query precedence is now decided by parameter PRESENCE, not
successful parsing: a malformed/foreign after_event_id/after_seq never
falls through to let the Last-Event-ID header override it and skip
events.
2. Invalid/foreign cursors no longer bypass replay + gap detection and
silently drain a truncated buffer; an ahead-of-stream cursor is no
longer clamped onto the snapshot's terminal fence (which emptied the
dead-stream replay by filtering the terminal frame).
3. The runner-local observe path now receives the resolved header cursor
(retaining explicit-query precedence) so header-only runner reconnects
resume instead of duplicating events from the start.
Regression tests cover invalid-cursor replay-from-start, ahead-of-stream
terminal survival, header-only runner reconnect, and explicit-query
precedence; the stale clamp test is updated to the corrected contract.
Related hermes-webui/hermes-android#58
* fix(sse): consume resolved resume cursor at every consumer on /api/chat/stream
Address PR #6886 r2 review (gate-fail). The r1 presence/validity split
fixed the entry resolution, but the individual consumers still re-derived
ahead/equality/re-parse decisions, so the invariant leaked at the edges.
This makes every consumer honor the resolved (after_seq, resume_requested)
pair and normalizes ahead-of-stream in one place.
1. Equality off-by-one: a valid cursor EQUAL to the snapshot cutoff now
enters the drain dedup bound (the event at the cursor was already
delivered), instead of being treated as "ahead" and double-sending the
retained buffer. Ahead is now strictly after_seq > snapshot_cutoff_seq.
2. Numeric ahead-of-stream cursors are normalized to replay-from-start:
on the active path, after_seq > snapshot_cutoff_seq resets to 0 before
gap-coverage/replay so a truncated buffer can no longer falsely declare
coverage and lose the pre-tail events; on the dead-stream path the
cursor is normalized against the journal summary's authoritative
last_seq before _replay_run_journal, so an ahead cursor returns the
full replay instead of an empty SSE body.
3. Runner-local now consumes the resolved cursor instead of re-parsing
after_event_id or re-reading Last-Event-ID. The resolver also returns
the opaque raw cursor so the runner uses the already-precedence-checked
value; explicit opaque cursor= keeps precedence, invalid/foreign
cursors pass no cursor (full replay), and the header is honored only
when no explicit cursor was supplied.
Regression tests cover the equality dedup, active+dead ahead-of-stream
normalization (with after_seq-filtering mocks that assert the real replay
body), and the three runner precedence probes (malformed-explicit blocks
header, foreign passes no cursor, opaque cursor= wins).
Related hermes-webui/hermes-android#58
* fix(sse): carry runner-cursor provenance through /api/chat/stream resume
Address PR #6886 r3 review (gate-fail). The (after_seq, resume_requested,
raw_cursor) tuple carried the cursor VALUE but not its PROVENANCE, so the
runner handoff couldn't distinguish a valid opaque runner cursor from a
journal-shaped one from a malformed one — and gated the runner handoff on
journal-specific validity, which is wrong for the runner adapter (its
cursors are opaque).
The resolver now returns a distinct, centrally-resolved runner cursor:
- a valid after_seq PAIRS with whatever after_event_id shape was supplied
— including an opaque runner id like event:2 that the journal parser
reads as a foreign run — so the paired runner cursor resumes at the seq
(never None / full replay, which would duplicate tokens and tool
events). after_seq is read directly (via _parse_run_journal_after_seq_value)
rather than through the after_event_id-first journal parser, which would
swallow a paired opaque runner id.
- a header-only opaque runner id (Last-Event-ID: event:2) is preserved
as-is so the runner resumes from it.
- a malformed or foreign explicit cursor WITHOUT a valid paired after_seq
still yields no runner cursor, blocking Last-Event-ID and replaying from
start — the r2 malformed-blocks-header guarantee is unchanged.
Regression tests pin the paired opaque id, header-only opaque id, and
malformed-with-valid-seq runner resumes, plus the unchanged
malformed-without-pair replay-from-start behavior.
Related hermes-webui/hermes-android#58
* fix(sse): normalize ahead cursor when the snapshot cutoff is unknown
Address PR #6886 r4 review (gate-fail). The ahead-cursor normalization
was guarded by `snapshot_cutoff_seq is not None`, so when the subscribe
snapshot had no parseable last_event_id (channel has not seen an
id-bearing frame yet), a positive ahead cursor (e.g. Last-Event-ID
run_1:999) was never normalized — it was installed verbatim as the live
dedup bound, filtering EVERY queued frame including the terminal
stream_end fence. The reconnect stalled on heartbeats with an empty SSE
body. origin/master ignored the header on this path and delivered the
events, so this was a new regression.
Fix: treat an unknown, non-truncated snapshot cutoff as fence 0 before
the ahead-cursor normalization, so a positive client cursor can't become
the live dedup bound with no cutoff to bound it — it fails closed to
replay-from-start, delivering the buffered events rather than silently
losing them. Frames dropped while the cutoff is unknown still cannot
prove gap coverage (cutoff None → not covered), so the recovery_control
fail-closed path is preserved.
Regression test: Last-Event-ID run_1:999 + snapshot_cutoff_seq=None
delivers queued events run_1:1 and run_1:2 exactly once, emits the
terminal stream_end, and releases the subscriber. Without the fix the
test stalls on heartbeats (pytest-timeout kills it), proving the stall.
Related hermes-webui/hermes-android#58
* Release — SSE Last-Event-ID resumption fix (#6886)
---------
Co-authored-by: Paladin173 <35980893+Paladin173@users.noreply.github.com>
Co-authored-by: n <a@n>
Rebased 363 commits forward (branch-point 2026-07-05). Applied the PR's code
diff cleanly across 16 files; resolved api/models.py additively (both master's
process_wakeup_pause/anchor_scene_index fields AND the PR's share_token/
share_created_at fields kept in the Session constructor, metadata list, and
to_dict). CI workflow (.github/workflows/tests.yml) intentionally NOT taken from
the stale branch — master owns it (the branch's apparent removal of Playwright/
Office steps was a PAT-workflow-scope revert; rebase restores master's CI).
Co-authored-by: MicoRobot. Concept approved by maintainer (opt-in public share,
2026-07-11). Note: 4 dynamic share tests fail identically on the original
unrebased head (pre-existing test-env Session.load fragility, not rebase-caused);
to be resolved during the security re-gate in a proper sandbox.
Root cause of the silent-crash-after-hours cluster (#4765/#2233/#4633): the
global in-memory SESSIONS LRU evicted with a blind popitem(last=False), which
could drop an actively streaming or not-yet-persisted session (data loss) and
was capped only via an env var. On long-running installs the effective result
was unbounded RAM growth until segfault.
- Add _session_is_evictable(): a session is evictable ONLY when it is not
streaming (no active_stream_id), has no in-flight turn (no pending_user_message
/ pending_started_at), and its full state is proven on disk (sidecar
message_count >= in-memory count; metadata-only stubs and zero-message shells
are trivially safe).
- Add _evict_sessions_over_cap(): replaces all 8 blind popitem loops across
models.py, routes.py, streaming.py. Walks the LRU oldest-first and removes only
provably-safe entries; never acquires LOCK/stream locks itself (caller holds
LOCK) so no lock-ordering deadlock. May briefly exceed the cap rather than ever
evict an active/unsaved session.
- Make the cap configurable via config.yaml webui.sessions_cache_max
(get_sessions_cache_max()); precedence config.yaml -> HERMES_WEBUI_SESSIONS_MAX
(legacy) -> DEFAULT_SESSIONS_CACHE_MAX=300. No new HERMES_* env var. Invalid or
<1 values fall back so a typo can never disable the bound.
- Evicted sessions lazily reload from their JSON sidecar via the existing
get_session() accessor; no call sites changed. _index.json sidebar behavior
unchanged (the sidebar reads the index, not SESSIONS).
- Add tests/test_issue4765_sessions_lru_eviction.py (8 tests): eviction past
cap, active/streaming never evicted, unsaved/stale-tail never evicted, lazy
reload with identical content, and no-data-loss under heavy churn.
- README: document the config.yaml key + safety semantics.
Fixes#4765.
The previous wording implied external gateway routing was not yet possible.
HERMES_WEBUI_CHAT_BACKEND=gateway already ships; updated wording distinguishes
it from full agent-loop delegation, which is not yet shipped and tracked in #1925.
HERMES_API_URL is only read by the Tasks/cron health probe (api/agent_health.py)
and does not route chat. Adds a plain-language note to Quick Start with the two
supported options for users who run an external endpoint. Closes#3873.
Opus flagged a real upgrade gap on the #4449 state-dir contract folded into
#4454: a deployment that relocated HERMES_HOME to a non-default base WITHOUT
setting HERMES_WEBUI_STATE_DIR sees WebUI state move to $HERMES_HOME/webui on
upgrade (empty app at the new address). Add (1) README note documenting the
move + how to keep the old location, and (2) two backward-compat regression
tests pinning the invariants: HERMES_HOME-unset stays at the platform default
(normal installs don't relocate), and HERMES_WEBUI_STATE_DIR takes precedence.
* docs: weekly contributor tally refresh (20260619)
Regenerated by scripts/regen_contributors.py --apply. Includes the source (e)/(f)
ship-signal accuracy fix that recovers systematically-undercounted cherry-picked
contributions (~77 contributors). 280 contributors / 1559 credits (was 263/1232).
Markers spliced, prose untouched, --verify clean, 0 double-counts.
* docs: align CONTRIBUTORS header prose with the 5-source credit algorithm
Addresses greptile P2: the header still described the old three-source union
(merged-PR list + CHANGELOG + Co-authored-by) but regen_contributors.py now adds
two more — ship-comment scans and absorb-commit scans — for five total. Updates
the prose to match what the script actually runs.
---------
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
start.sh exec's bootstrap.py without --foreground, which double-forks into a
detached daemon, so it belongs in the 'find PID + kill' row, not the Ctrl-C
foreground row. Docs-only, no version stamp.
Co-authored-by: tomas-forgac
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
* docs: refresh contributor tally for v0.51.390 (248 contributors)
Regenerated via scripts/regen_contributors.py after fixing three real
over-counting bugs in the generator (release-vehicle PR# leak, reporter
false-positives, prose-token leakage — see workspace commit). Net:
- 248 contributors / 1092 credits (corrected; the prior generator inflated
this with ~106 spurious release-vehicle co-author credits + reporter mentions)
- CONTRIBUTORS.md now recognizes 3+ PR contributors in ranked tables and lists
everyone with 1-2 PRs in a combined roll, so nobody already credited is dropped
- README top-10 snapshot + notable-contribution counts updated to match
(franksong 180, Michaelyklam 118, rodboev 83 [new], ai-ag2026 75, bergeouss 70)
Every name in the tally is authorship-verified (merged PR, CHANGELOG attribution,
or a Co-authored-by trailer); bug reporters / relayers / confirmers are no longer
miscredited as authors.
* docs: add generated-region markers so the contributor tally regenerates cleanly
Wraps the machine-generated tables in CONTRIBUTORS.md + README.md with
<!-- BEGIN/END GENERATED CONTRIBUTORS --> markers. scripts/regen_contributors.py
--apply now splices fresh tables between them on each run while leaving the
hand-written prose (How credit is tracked, Special thanks, README Notable
contributions) untouched. Verified idempotent: a second --apply is a no-op.
This is what the weekly refresh cron runs.
Tally re-synced to current master: 248 contributors / 1093 credits.
---------
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
Co-authored-by: Nathan Esquenazi <nesquena@gmail.com>
* docs(#3794): clarify how to stop each launch method in Quick start
* docs(#3794): use lsof/ss for PID discovery instead of implying bootstrap prints it
---------
Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
## Release v0.51.256 — Release HX (stage-r4)
Performance — bound WebUI memory growth & idle CPU on large installs.
### Fixed
| Issue | Author | Fix |
|-------|--------|-----|
| #3506 | @nesquena-hermes (reported w/ profiling by @djenttleman) | On a large install (~615 sessions / 40k messages / 454 MB state.db) the WebUI process climbed ~100 MB → ~1.5 GB RSS over days and held high idle CPU. Three root causes fixed: (1) `session_lifecycle._sessions` grew unbounded → new `discard_session()` drops the entry at agent-eviction boundaries, only when no in-flight commit / no uncommitted memory work (retry invariant preserved); (2) cache caps now operator-tunable (`HERMES_WEBUI_AGENT_CACHE_MAX` default 50→25, `HERMES_WEBUI_SESSIONS_MAX`); (3) GatewayWatcher computes a cheap fingerprint before the expensive per-session `MAX(messages.timestamp)` projection and only re-projects on change. |
### Rebase + review notes
- Rebased onto current master; the code diff was verified **byte-identical to the nesquena-APPROVED head** at rebase time (only CHANGELOG re-resolved).
- The Codex regression gate then surfaced **two correctness gaps** the approval didn't catch, both fixed here with regression tests:
1. **Watcher fingerprint missed same-count transcript rewrites.** `/retry`,`/undo`,`/compress` (`SessionDB.replace_messages`) rewrite messages with new timestamps but can leave `message_count` unchanged → stale sidebar `last_activity`. Fixed with a **per-session** grouped message aggregate (`id, count, user_count, MAX(timestamp)`) over the same non-excluded sessions (a global MAX would miss a rewrite of an older, non-newest session); cron/webui stay excluded so idle churn still doesn't re-project.
2. **LRU agent-cache eviction could close a live worker's agent** (`popitem(last=False)`, liveness-blind — pre-existing, but the lower 50→25 cap made it more likely). Eviction now snapshots `ACTIVE_RUNS` session_ids (before the cache lock — no nested lock) and skips live sessions, deferring (temporarily exceeding cap) rather than closing a live agent.
### Gate
- Full pytest suite: **7612 passed, 0 failed** (one boot-cascade flake re-run; clean on re-run)
- ruff: CLEAN · Codex (regression): 4 rounds → both gaps + a stale test fixed → **SAFE TO SHIP**
Co-authored-by: nesquena <nesquena@users.noreply.github.com>
## Release v0.51.252 — Release HT (stage-q24)
Two trivially-safe @rodboev changes (independent).
### Fixed
| Issue | Author | Fix |
|-------|--------|-----|
| #2481 | @rodboev | The floating "selected-text reply" button now has `user-select:none`, so its own label can't get caught in a text selection (no bleed-through). CSS one-liner. |
### Docs
- README **Compatibility** section: upgrade WebUI + hermes-agent together until the stable agent API (#2491) lands. (@rodboev)
### Dropped from this batch
- **#2977 `/use` skill command** was staged here but **dropped** — the Codex regression gate found an async stale-directive race (`cmdUse()` awaits `/api/skills` but `send()` doesn't await the handler → a fast next send can miss it, or a stale directive leaks to a later message) plus an over-eager `finally` clear that silently discards the directive on a local slash-command early-return. Held with `changes-requested` + a detailed rework note (tracked pending promise + clear-on-consume). Concept approved; needs lifecycle hardening.
### Gate
- Full pytest suite: **7570 passed, 0 failed**
- ESLint: CLEAN · ruff: CLEAN · browser-smoke: CLEAN
- Codex (regression): **SAFE TO SHIP** — `user-select:none` scoped to the button only, README docs-only, no `/use` code remains
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
Information-architecture pass on the 840-line README so the most important
things come first and secondary/niche content is linked rather than inline:
- Reorder: Why -> Quick start -> FEATURES (was at line 502, now right after
Quick start) -> Configuration & access -> Docker -> Running tests ->
Architecture -> Docs -> Contributors. Readers see what it does before the
deployment minutiae.
- Consolidate the scattered access sections (start.sh discovery, overrides,
remote/SSH, Tailscale, manual launch) under one '## Configuration & access'
H2 with H3 subsections.
- Extract two genuinely-niche blocks to new linked docs (nothing deleted):
- docs/advanced-chat-setup.md — dynamic recall-prefill + Gateway-backed chat
- docs/remote-access.md — SSH tunnel + Tailscale + ARM64-Android field report
Quick start keeps a one-line pointer to each.
- Update Contents TOC + Docs index for the new order and new files.
README 840 -> 705 lines; content preserved (verified moved-not-dropped); all
internal links + new docs verified to resolve; docs/*.md gitignore-allowlisted.
- ROADMAP.md: 8-themes -> 11-skins in status snapshot; replace stale per-file
LOC architecture table with role descriptions + CI gate list (ruff/shard/
browser/docker); note LOC drifts.
- ARCHITECTURE.md: refresh File Inventory — drop chronically-stale per-file LOC,
add missing modules (state_sync.py, updates.py, i18n/login/icons/sw.js,
pyproject.toml, ctl.sh, CONTRIBUTORS.md), refresh test count to ~7,150.
- README.md: update the 8 'Notable contributions' blurb headers (PR counts +
release ranges) to match the regenerated tally.
Regenerate CONTRIBUTORS.md + README contributors section from a verified
3-source union: GitHub merged-PR list, CHANGELOG.md attribution lines, and
Co-authored-by trailers on master commits (the canonical signal for a CLOSED
contributor PR whose commits were cherry-picked/absorbed and attributed).
- New tally: 194 contributors / 843 PR credits (was a stale 137 / 646).
- The increase: ~135 releases since the v0.51.58 pin PLUS newly-detected
absorbed-CLOSED PRs the prior hand-count missed (e.g. franksong2702 148 =
129 merged + 19 cherry-picked-and-attributed).
- UNION with the existing hand-curated file as a floor: 27 old contributors
had no machine-readable signal (very old closed PRs) — preserved, ZERO dropped.
- Refreshed special-thanks PR counts to match.
- Generator committed to the maintainer workspace as scripts/regen_contributors.py
(--merge-existing keeps it safe for all future refreshes).
Verified: every one of the original 137 logins still present (+57 new).
README:
- Add a Contents table of contents for navigability (800-line doc).
- Freshen stale snapshots: test count 5303/488 files -> ~7,150/~700 files;
reframe contributor counts to point at CONTRIBUTORS.md as the live source.
- Rebuild the Architecture section: drop per-file exact LOC (drifts every
release -> chronic staleness) in favor of a stable backend/frontend role map;
add pyproject.toml + the ruff/browser/docker CI gates.
- Reorganize the Docs index by purpose (Start here / Using / Deploying /
Contributing & design / Release history); add missing CONTRIBUTING.md,
DESIGN.md, docs/workspace-git.md; convert bare paths to working links.
ARCHITECTURE.md: header v0.51.54/5303 -> v0.51.192/~7,150 + note that the
numbers are a periodic snapshot (authoritative source = git tag + collect-only).
TESTING.md: header + footer test counts refreshed; drop the stale
'through v0.50.21' framing; note the ruff/browser/docker gates.
ROADMAP.md: refresh the 'Last updated' stamp (v0.51.31/5028 -> v0.51.192/~7,150)
with recent themes.
Markdown-only; all internal links verified to resolve.
CharlesMcquade