dcb515611d
* fix(sse): scope gateway SSE probe and honor Last-Event-ID on /api/chat/stream Two narrowly-scoped cross-client SSE fixes from the hermes-android#58 follow-up: 1. The /api/sessions/gateway/stream?probe=1 response only describes the optional gateway/agent-sessions stream, but its payload gave clients no way to tell that. A 404 'agent sessions not enabled' result could be misread as SSE being unsupported server-wide, even though the persistent per-session stream (/api/session/stream) is always on and independent of the show_cli_sessions setting. Add additive scope markers to the probe payload (scope, session_stream_available, session_stream_path) and clarify the handler docstring. 2. /api/chat/stream emitted id: stream_id:seq on every journaled event but ignored the Last-Event-ID request header, so any spec-compliant SSE client that auto-reconnects (browser EventSource, Android/CLI clients) replayed the journal from seq 0 and double-rendered the transcript. _chat_stream_resume_after_seq() now resolves the resume cursor with explicit after_event_id/after_seq query params first and the Last-Event-ID header as fallback — the same resolution chain api/kanban_bridge.py already uses. Both the dead-stream journal replay and the live-subscribe gap check consume it; direct callers of the gap check that only pass query params keep historical behavior. Also documents the cross-client SSE endpoint contract in docs/sse-streams.md. Related hermes-webui/hermes-android#58 * fix(sse): split cursor presence from validity on /api/chat/stream resume Address PR #6886 review (gate-fail): the resume logic conflated "no cursor" with "invalid cursor" into a single None, which broke replay on three edges. Root cause and fix: Cursor presence vs validity — _chat_stream_resume_cursor() now returns (after_seq, resume_requested) instead of a bare seq. A client that SUPPLIED any cursor (query params, replay=1, or Last-Event-ID) asked to resume; an invalid / foreign-run / unparseable cursor resolves to after_seq=None but keeps resume_requested=True, and is normalized to replay-from-start so no journal events are silently skipped. A genuinely cursor-less request stays a fresh subscribe. 1. Explicit-query precedence is now decided by parameter PRESENCE, not successful parsing: a malformed/foreign after_event_id/after_seq never falls through to let the Last-Event-ID header override it and skip events. 2. Invalid/foreign cursors no longer bypass replay + gap detection and silently drain a truncated buffer; an ahead-of-stream cursor is no longer clamped onto the snapshot's terminal fence (which emptied the dead-stream replay by filtering the terminal frame). 3. The runner-local observe path now receives the resolved header cursor (retaining explicit-query precedence) so header-only runner reconnects resume instead of duplicating events from the start. Regression tests cover invalid-cursor replay-from-start, ahead-of-stream terminal survival, header-only runner reconnect, and explicit-query precedence; the stale clamp test is updated to the corrected contract. Related hermes-webui/hermes-android#58 * fix(sse): consume resolved resume cursor at every consumer on /api/chat/stream Address PR #6886 r2 review (gate-fail). The r1 presence/validity split fixed the entry resolution, but the individual consumers still re-derived ahead/equality/re-parse decisions, so the invariant leaked at the edges. This makes every consumer honor the resolved (after_seq, resume_requested) pair and normalizes ahead-of-stream in one place. 1. Equality off-by-one: a valid cursor EQUAL to the snapshot cutoff now enters the drain dedup bound (the event at the cursor was already delivered), instead of being treated as "ahead" and double-sending the retained buffer. Ahead is now strictly after_seq > snapshot_cutoff_seq. 2. Numeric ahead-of-stream cursors are normalized to replay-from-start: on the active path, after_seq > snapshot_cutoff_seq resets to 0 before gap-coverage/replay so a truncated buffer can no longer falsely declare coverage and lose the pre-tail events; on the dead-stream path the cursor is normalized against the journal summary's authoritative last_seq before _replay_run_journal, so an ahead cursor returns the full replay instead of an empty SSE body. 3. Runner-local now consumes the resolved cursor instead of re-parsing after_event_id or re-reading Last-Event-ID. The resolver also returns the opaque raw cursor so the runner uses the already-precedence-checked value; explicit opaque cursor= keeps precedence, invalid/foreign cursors pass no cursor (full replay), and the header is honored only when no explicit cursor was supplied. Regression tests cover the equality dedup, active+dead ahead-of-stream normalization (with after_seq-filtering mocks that assert the real replay body), and the three runner precedence probes (malformed-explicit blocks header, foreign passes no cursor, opaque cursor= wins). Related hermes-webui/hermes-android#58 * fix(sse): carry runner-cursor provenance through /api/chat/stream resume Address PR #6886 r3 review (gate-fail). The (after_seq, resume_requested, raw_cursor) tuple carried the cursor VALUE but not its PROVENANCE, so the runner handoff couldn't distinguish a valid opaque runner cursor from a journal-shaped one from a malformed one — and gated the runner handoff on journal-specific validity, which is wrong for the runner adapter (its cursors are opaque). The resolver now returns a distinct, centrally-resolved runner cursor: - a valid after_seq PAIRS with whatever after_event_id shape was supplied — including an opaque runner id like event:2 that the journal parser reads as a foreign run — so the paired runner cursor resumes at the seq (never None / full replay, which would duplicate tokens and tool events). after_seq is read directly (via _parse_run_journal_after_seq_value) rather than through the after_event_id-first journal parser, which would swallow a paired opaque runner id. - a header-only opaque runner id (Last-Event-ID: event:2) is preserved as-is so the runner resumes from it. - a malformed or foreign explicit cursor WITHOUT a valid paired after_seq still yields no runner cursor, blocking Last-Event-ID and replaying from start — the r2 malformed-blocks-header guarantee is unchanged. Regression tests pin the paired opaque id, header-only opaque id, and malformed-with-valid-seq runner resumes, plus the unchanged malformed-without-pair replay-from-start behavior. Related hermes-webui/hermes-android#58 * fix(sse): normalize ahead cursor when the snapshot cutoff is unknown Address PR #6886 r4 review (gate-fail). The ahead-cursor normalization was guarded by `snapshot_cutoff_seq is not None`, so when the subscribe snapshot had no parseable last_event_id (channel has not seen an id-bearing frame yet), a positive ahead cursor (e.g. Last-Event-ID run_1:999) was never normalized — it was installed verbatim as the live dedup bound, filtering EVERY queued frame including the terminal stream_end fence. The reconnect stalled on heartbeats with an empty SSE body. origin/master ignored the header on this path and delivered the events, so this was a new regression. Fix: treat an unknown, non-truncated snapshot cutoff as fence 0 before the ahead-cursor normalization, so a positive client cursor can't become the live dedup bound with no cutoff to bound it — it fails closed to replay-from-start, delivering the buffered events rather than silently losing them. Frames dropped while the cutoff is unknown still cannot prove gap coverage (cutoff None → not covered), so the recovery_control fail-closed path is preserved. Regression test: Last-Event-ID run_1:999 + snapshot_cutoff_seq=None delivers queued events run_1:1 and run_1:2 exactly once, emits the terminal stream_end, and releases the subscriber. Without the fix the test stalls on heartbeats (pytest-timeout kills it), proving the stall. Related hermes-webui/hermes-android#58 * Release — SSE Last-Event-ID resumption fix (#6886) --------- Co-authored-by: Paladin173 <35980893+Paladin173@users.noreply.github.com> Co-authored-by: n <a@n>
4.1 KiB
4.1 KiB
SSE streams and capability signaling
Cross-client reference for the server-sent events (SSE) endpoints Hermes WebUI exposes. Browser and non-browser clients (Android wrapper, CLI observers) should integrate against this page so every client describes the same behavior.
All endpoints below are served by the WebUI origin and sit behind the same
authentication as every other /api/* route: when a WebUI password or OIDC
is configured, clients must authenticate before opening any stream.
Endpoint inventory
| Endpoint | Availability | Purpose |
|---|---|---|
GET /api/chat/stream?stream_id=<id> |
Always on | Live agent-turn relay (tokens, tool calls, approvals, done, stream_end). Falls back to run-journal replay when the in-memory stream is gone. Resume cursors: after_event_id / after_seq query params, with the standard Last-Event-ID header as fallback (events carry id: <stream_id>:<seq>). An invalid/foreign/ahead-of-stream cursor is honored as replay-from-start rather than silently skipping events. |
GET /api/session/stream?session_id=<id> |
Always on | Persistent per-session channel that survives across agent turns (initial, server_turn_started, session-updated, bg_task_complete). This is the stream the WebUI frontend keeps open per session and the stream non-browser clients should prefer for background session updates. |
GET /api/sessions/events |
Always on | Global session-list invalidation (sessions_changed + keepalives). A signal to re-read /api/sessions, not a per-session lifecycle feed. |
GET /api/sessions/{session_id}/events |
Always on | Per-session run-journal relay with Last-Event-ID / after_event_id resume and snapshot fallback. See docs/rfcs/session-sse-contract-v1.md for the contract and its proof gates. |
GET /api/sessions/gateway/stream |
Optional | Real-time updates for CLI/TUI/messaging (agent) sessions merged into the sidebar. Only streams when the Agent sessions setting (show_cli_sessions) is enabled and the gateway watcher thread is running. |
The authoritative event: names on /api/chat/stream are listed in the
Authoritative emitted events table of
docs/rfcs/session-sse-contract-v1.md.
Gateway probe scope (important for non-browser clients)
GET /api/sessions/gateway/stream?probe=1 returns a JSON capability payload
for the optional gateway stream only instead of holding an SSE connection:
{
"enabled": false,
"ok": false,
"watcher_running": false,
"fallback_poll_ms": 30000,
"error": "agent sessions not enabled",
"scope": "gateway_sessions",
"session_stream_available": true,
"session_stream_path": "/api/session/stream"
}
404+error: "agent sessions not enabled"means only that the optional gateway/agent-sessions stream is disabled on this server.503+error: "watcher not started"means the setting is on but the gateway watcher thread is not running.200withok: truemeans gateway SSE is usable.
A negative gateway probe result must not be treated as "SSE unavailable".
The persistent per-session stream (/api/session/stream) and the chat-turn
relay (/api/chat/stream) are always on and are not gated by the Agent
sessions setting. The scope, session_stream_available, and
session_stream_path fields make this explicit so clients do not need to
infer it from the status code. Clients that only need session updates should
use /api/session/stream directly; the gateway probe is only relevant for
clients that display CLI/TUI/messaging sessions.
Heartbeats and proxy behavior
- All long-lived streams emit SSE keepalive comment lines on the
_SSE_HEARTBEAT_INTERVAL_SECONDScadence (currently 5 seconds), which is short enough to survive typical reverse-proxy idle timeouts. - Handlers send
X-Accel-Buffering: noso nginx-style proxies pass events through unbuffered. - Deployments behind buffering proxies that read-until-close (notably
Tornado-based
jupyter-server-proxy) can setHERMES_WEBUI_SSE_CHUNKED=1to frame each event as an HTTP/1.1 chunk. The default wire format is unchanged when the flag is unset.