* fix(sse): scope gateway SSE probe and honor Last-Event-ID on /api/chat/stream Two narrowly-scoped cross-client SSE fixes from the hermes-android#58 follow-up: 1. The /api/sessions/gateway/stream?probe=1 response only describes the optional gateway/agent-sessions stream, but its payload gave clients no way to tell that. A 404 'agent sessions not enabled' result could be misread as SSE being unsupported server-wide, even though the persistent per-session stream (/api/session/stream) is always on and independent of the show_cli_sessions setting. Add additive scope markers to the probe payload (scope, session_stream_available, session_stream_path) and clarify the handler docstring. 2. /api/chat/stream emitted id: stream_id:seq on every journaled event but ignored the Last-Event-ID request header, so any spec-compliant SSE client that auto-reconnects (browser EventSource, Android/CLI clients) replayed the journal from seq 0 and double-rendered the transcript. _chat_stream_resume_after_seq() now resolves the resume cursor with explicit after_event_id/after_seq query params first and the Last-Event-ID header as fallback — the same resolution chain api/kanban_bridge.py already uses. Both the dead-stream journal replay and the live-subscribe gap check consume it; direct callers of the gap check that only pass query params keep historical behavior. Also documents the cross-client SSE endpoint contract in docs/sse-streams.md. Related hermes-webui/hermes-android#58 * fix(sse): split cursor presence from validity on /api/chat/stream resume Address PR #6886 review (gate-fail): the resume logic conflated "no cursor" with "invalid cursor" into a single None, which broke replay on three edges. Root cause and fix: Cursor presence vs validity — _chat_stream_resume_cursor() now returns (after_seq, resume_requested) instead of a bare seq. A client that SUPPLIED any cursor (query params, replay=1, or Last-Event-ID) asked to resume; an invalid / foreign-run / unparseable cursor resolves to after_seq=None but keeps resume_requested=True, and is normalized to replay-from-start so no journal events are silently skipped. A genuinely cursor-less request stays a fresh subscribe. 1. Explicit-query precedence is now decided by parameter PRESENCE, not successful parsing: a malformed/foreign after_event_id/after_seq never falls through to let the Last-Event-ID header override it and skip events. 2. Invalid/foreign cursors no longer bypass replay + gap detection and silently drain a truncated buffer; an ahead-of-stream cursor is no longer clamped onto the snapshot's terminal fence (which emptied the dead-stream replay by filtering the terminal frame). 3. The runner-local observe path now receives the resolved header cursor (retaining explicit-query precedence) so header-only runner reconnects resume instead of duplicating events from the start. Regression tests cover invalid-cursor replay-from-start, ahead-of-stream terminal survival, header-only runner reconnect, and explicit-query precedence; the stale clamp test is updated to the corrected contract. Related hermes-webui/hermes-android#58 * fix(sse): consume resolved resume cursor at every consumer on /api/chat/stream Address PR #6886 r2 review (gate-fail). The r1 presence/validity split fixed the entry resolution, but the individual consumers still re-derived ahead/equality/re-parse decisions, so the invariant leaked at the edges. This makes every consumer honor the resolved (after_seq, resume_requested) pair and normalizes ahead-of-stream in one place. 1. Equality off-by-one: a valid cursor EQUAL to the snapshot cutoff now enters the drain dedup bound (the event at the cursor was already delivered), instead of being treated as "ahead" and double-sending the retained buffer. Ahead is now strictly after_seq > snapshot_cutoff_seq. 2. Numeric ahead-of-stream cursors are normalized to replay-from-start: on the active path, after_seq > snapshot_cutoff_seq resets to 0 before gap-coverage/replay so a truncated buffer can no longer falsely declare coverage and lose the pre-tail events; on the dead-stream path the cursor is normalized against the journal summary's authoritative last_seq before _replay_run_journal, so an ahead cursor returns the full replay instead of an empty SSE body. 3. Runner-local now consumes the resolved cursor instead of re-parsing after_event_id or re-reading Last-Event-ID. The resolver also returns the opaque raw cursor so the runner uses the already-precedence-checked value; explicit opaque cursor= keeps precedence, invalid/foreign cursors pass no cursor (full replay), and the header is honored only when no explicit cursor was supplied. Regression tests cover the equality dedup, active+dead ahead-of-stream normalization (with after_seq-filtering mocks that assert the real replay body), and the three runner precedence probes (malformed-explicit blocks header, foreign passes no cursor, opaque cursor= wins). Related hermes-webui/hermes-android#58 * fix(sse): carry runner-cursor provenance through /api/chat/stream resume Address PR #6886 r3 review (gate-fail). The (after_seq, resume_requested, raw_cursor) tuple carried the cursor VALUE but not its PROVENANCE, so the runner handoff couldn't distinguish a valid opaque runner cursor from a journal-shaped one from a malformed one — and gated the runner handoff on journal-specific validity, which is wrong for the runner adapter (its cursors are opaque). The resolver now returns a distinct, centrally-resolved runner cursor: - a valid after_seq PAIRS with whatever after_event_id shape was supplied — including an opaque runner id like event:2 that the journal parser reads as a foreign run — so the paired runner cursor resumes at the seq (never None / full replay, which would duplicate tokens and tool events). after_seq is read directly (via _parse_run_journal_after_seq_value) rather than through the after_event_id-first journal parser, which would swallow a paired opaque runner id. - a header-only opaque runner id (Last-Event-ID: event:2) is preserved as-is so the runner resumes from it. - a malformed or foreign explicit cursor WITHOUT a valid paired after_seq still yields no runner cursor, blocking Last-Event-ID and replaying from start — the r2 malformed-blocks-header guarantee is unchanged. Regression tests pin the paired opaque id, header-only opaque id, and malformed-with-valid-seq runner resumes, plus the unchanged malformed-without-pair replay-from-start behavior. Related hermes-webui/hermes-android#58 * fix(sse): normalize ahead cursor when the snapshot cutoff is unknown Address PR #6886 r4 review (gate-fail). The ahead-cursor normalization was guarded by `snapshot_cutoff_seq is not None`, so when the subscribe snapshot had no parseable last_event_id (channel has not seen an id-bearing frame yet), a positive ahead cursor (e.g. Last-Event-ID run_1:999) was never normalized — it was installed verbatim as the live dedup bound, filtering EVERY queued frame including the terminal stream_end fence. The reconnect stalled on heartbeats with an empty SSE body. origin/master ignored the header on this path and delivered the events, so this was a new regression. Fix: treat an unknown, non-truncated snapshot cutoff as fence 0 before the ahead-cursor normalization, so a positive client cursor can't become the live dedup bound with no cutoff to bound it — it fails closed to replay-from-start, delivering the buffered events rather than silently losing them. Frames dropped while the cutoff is unknown still cannot prove gap coverage (cutoff None → not covered), so the recovery_control fail-closed path is preserved. Regression test: Last-Event-ID run_1:999 + snapshot_cutoff_seq=None delivers queued events run_1:1 and run_1:2 exactly once, emits the terminal stream_end, and releases the subscriber. Without the fix the test stalls on heartbeats (pytest-timeout kills it), proving the stall. Related hermes-webui/hermes-android#58 * Release — SSE Last-Event-ID resumption fix (#6886) --------- Co-authored-by: Paladin173 <35980893+Paladin173@users.noreply.github.com> Co-authored-by: n <a@n>
This commit is contained in:
8 files changed
+999
-38
No files matched your search
@@ -0,0 +1,69 @@
|
||||
# SSE streams and capability signaling
|
||||
|
||||
Cross-client reference for the server-sent events (SSE) endpoints Hermes WebUI
|
||||
exposes. Browser and non-browser clients (Android wrapper, CLI observers)
|
||||
should integrate against this page so every client describes the same
|
||||
behavior.
|
||||
|
||||
All endpoints below are served by the WebUI origin and sit behind the same
|
||||
authentication as every other `/api/*` route: when a WebUI password or OIDC
|
||||
is configured, clients must authenticate before opening any stream.
|
||||
|
||||
## Endpoint inventory
|
||||
|
||||
| Endpoint | Availability | Purpose |
|
||||
|---|---|---|
|
||||
| `GET /api/chat/stream?stream_id=<id>` | Always on | Live agent-turn relay (tokens, tool calls, approvals, `done`, `stream_end`). Falls back to run-journal replay when the in-memory stream is gone. Resume cursors: `after_event_id` / `after_seq` query params, with the standard `Last-Event-ID` header as fallback (events carry `id: <stream_id>:<seq>`). An invalid/foreign/ahead-of-stream cursor is honored as replay-from-start rather than silently skipping events. |
|
||||
| `GET /api/session/stream?session_id=<id>` | Always on | Persistent per-session channel that survives across agent turns (`initial`, `server_turn_started`, `session-updated`, `bg_task_complete`). This is the stream the WebUI frontend keeps open per session and the stream non-browser clients should prefer for background session updates. |
|
||||
| `GET /api/sessions/events` | Always on | Global session-list invalidation (`sessions_changed` + keepalives). A signal to re-read `/api/sessions`, not a per-session lifecycle feed. |
|
||||
| `GET /api/sessions/{session_id}/events` | Always on | Per-session run-journal relay with `Last-Event-ID` / `after_event_id` resume and snapshot fallback. See `docs/rfcs/session-sse-contract-v1.md` for the contract and its proof gates. |
|
||||
| `GET /api/sessions/gateway/stream` | Optional | Real-time updates for CLI/TUI/messaging (agent) sessions merged into the sidebar. Only streams when the **Agent sessions** setting (`show_cli_sessions`) is enabled and the gateway watcher thread is running. |
|
||||
|
||||
The authoritative `event:` names on `/api/chat/stream` are listed in the
|
||||
**Authoritative emitted events** table of
|
||||
[`docs/rfcs/session-sse-contract-v1.md`](rfcs/session-sse-contract-v1.md).
|
||||
|
||||
## Gateway probe scope (important for non-browser clients)
|
||||
|
||||
`GET /api/sessions/gateway/stream?probe=1` returns a JSON capability payload
|
||||
for the **optional gateway stream only** instead of holding an SSE connection:
|
||||
|
||||
```json
|
||||
{
|
||||
"enabled": false,
|
||||
"ok": false,
|
||||
"watcher_running": false,
|
||||
"fallback_poll_ms": 30000,
|
||||
"error": "agent sessions not enabled",
|
||||
"scope": "gateway_sessions",
|
||||
"session_stream_available": true,
|
||||
"session_stream_path": "/api/session/stream"
|
||||
}
|
||||
```
|
||||
|
||||
- `404` + `error: "agent sessions not enabled"` means only that the optional
|
||||
gateway/agent-sessions stream is disabled on this server.
|
||||
- `503` + `error: "watcher not started"` means the setting is on but the
|
||||
gateway watcher thread is not running.
|
||||
- `200` with `ok: true` means gateway SSE is usable.
|
||||
|
||||
**A negative gateway probe result must not be treated as "SSE unavailable".**
|
||||
The persistent per-session stream (`/api/session/stream`) and the chat-turn
|
||||
relay (`/api/chat/stream`) are always on and are not gated by the Agent
|
||||
sessions setting. The `scope`, `session_stream_available`, and
|
||||
`session_stream_path` fields make this explicit so clients do not need to
|
||||
infer it from the status code. Clients that only need session updates should
|
||||
use `/api/session/stream` directly; the gateway probe is only relevant for
|
||||
clients that display CLI/TUI/messaging sessions.
|
||||
|
||||
## Heartbeats and proxy behavior
|
||||
|
||||
- All long-lived streams emit SSE keepalive comment lines on the
|
||||
`_SSE_HEARTBEAT_INTERVAL_SECONDS` cadence (currently 5 seconds), which is
|
||||
short enough to survive typical reverse-proxy idle timeouts.
|
||||
- Handlers send `X-Accel-Buffering: no` so nginx-style proxies pass events
|
||||
through unbuffered.
|
||||
- Deployments behind buffering proxies that read-until-close (notably
|
||||
Tornado-based `jupyter-server-proxy`) can set `HERMES_WEBUI_SSE_CHUNKED=1`
|
||||
to frame each event as an HTTP/1.1 chunk. The default wire format is
|
||||
unchanged when the flag is unset.
|
||||
Reference in new issue
Block a user