Commit Graph
73 Commits
Author SHA1 Message Date
starship-s 2262444806 fix(chat): keep image-turn context out of user bubbles
Preserve the submitted image turn as the display owner while retaining rich Agent context for model replay and suppressing its durable scalar mirror from recovered WebUI transcripts.

Assisted-by: Hermes Agent:gpt-6-sol
Assisted-by: Codex:gpt-6-luna
Assisted-by: Claude Code:claude-opus-5-5
2026-09-23 02:50:40 -06:00
nesquena-hermesandrodrigogs 5982a56698 fix(server): close the do_GET auth residual + recover comma-combined Content-Length
Gate fixes on top of @rodrigogs's #6658, both independently reproduced first.

do_GET called bare check_auth while every write path used check_auth_or_close,
so a body-bearing GET failing top-level auth left its bytes queued on a
keep-alive socket. check_auth_or_close is already body-pending-aware, so the
bodyless login-redirect flow keeps its keep-alive. Removing the now-unused
check_auth import (ruff F401) broke 9 tests that monkeypatch that symbol
through a module alias; rewritten to the new name.

_declared_content_lengths ran isdigit() across the whole raw header value, so
a comma-combined 'Content-Length: 0, 0' -- the same bodyless request as two
repeated headers under RFC 9110 5.3 -- read as unreadable and over-closed a
healthy connection. Now splits on commas before validating each member;
disagreeing lists still fail closed.

Verified: 16/16 adversarial CL vectors still fail closed (no fail-open
reopened), 13/13 over-close cases fixed, live-socket drive shows close armed
on a body-bearing reject and keep-alive preserved on a bodyless one, and a
chunked request behaves byte-identically to master (master never supported
chunked either, so no proxy deployment regresses).

Co-authored-by: rodrigogs <rodrigogs@users.noreply.github.com>
2026-09-22 12:53:29 +00:00
Rodrigo Gomes da Silva b022a79eef fix(http): reject every Transfer-Encoding, not just the ones we can't name
`identity` was exempt as a coding that "frames nothing", so a request
carrying it read back as body-less however many payload bytes followed.
It frames nothing either: RFC 9112 6.3 says a request whose final
transfer coding is not `chunked` has a body length no recipient can
determine, RFC 7230 removed `identity` from the transfer codings
outright, and `http.server` decodes none of them (`rfile` is the raw
socket). Every present value has to reject-and-close.

Reproduced on 8c326003b, pipelined down one socket:

  GET /api/extensions/probe/sidecar/ping  `Transfer-Encoding: identity`
                                          {"stale": true}
    -> HTTP/1.1 403 Forbidden        (no Connection: close)
    -> HTTP/1.1 400 Bad request syntax
           ('{"stale": true}GET /api/health/agent HTTP/1.1')
  POST /api/upload  (and /upload/extract, /workspace/upload, /transcribe)
    -> HTTP/1.1 400 Bad Request     ("No file field in request", no close)
    -> HTTP/1.1 400 Bad request syntax ('--x')

Deletes the `_UNFRAMED_TRANSFER_CODINGS` allowlist and the fold-safety
test that existed only to guard its membership check; `_BLANK_TRANSFER_CODING`
stays, since a present-but-empty header still needs a named, non-falsy
report. The identity rows of the framing tables and of
`test_bodyless_framing_keeps_the_pooled_socket_alive` move to the
closes-the-socket side, with real-socket coverage on the sidecar path and
all four upload endpoints. The keep-alive half stays pinned by the cases
that genuinely have no body: no framing header at all, `Content-Length: 0`,
an OWS-padded zero, two agreeing zeroes.

Swept the rest of the coding surface for the same class: every case
spelling, OWS and non-OWS padding, `identity, chunked`, `chunked, identity`,
`identity,identity`, a lone comma, an unknown coding, duplicated headers,
and a coding beside both a real and a zero Content-Length.
2026-09-22 12:06:09 +00:00
Rodrigo Gomes da Silva 01f2446459 fix(http): read a length only when it is 1*DIGIT, not whenever int() copes
`_declared_content_lengths()` parsed with `int()`, which is far looser than RFC
9110's `Content-Length = 1*DIGIT`: it accepts a leading sign, PEP 515
underscores and non-ASCII digits, and `str.strip()` ahead of it also erased
non-ASCII whitespace. Every such spelling of ZERO therefore parsed to an honest
`0`, took the "every declared length agrees on zero" keep-alive branch, and left
the payload in rfile for the server to parse as the next request line -- the
same defect the blank value had, one layer down. Reproduced live at the previous
head, pipelined down one socket:

  GET /api/extensions/probe/sidecar/ping  `Content-Length: +0`  {"stale": true}
    -> HTTP/1.1 403 Forbidden       (no Connection: close)
    -> HTTP/1.1 400 Bad request syntax
           ('{"stale": true}GET /api/health/agent HTTP/1.1')

byte-for-byte the trace greptile filed the P1 for, and identical for `-0`,
`0_0`, `+00`, `0_0_0`, `+0_0`, `\xa00`, `0\xa0` and `\x850`. On /api/upload the
same values slipped `unreadable_content_length()` into `int('0_0')` == 0 and
answered 400 keep-alive then `400 Bad request syntax ('--x')`.

A sign was reported safe last round because it was probed as `+15`, where a
wrong parse closes anyway; only the zero-valued spellings expose it.

The parse is now the grammar: an unsigned run of ASCII digits, or the framing is
unreadable and a body is assumed pending. The `int()` guard stays -- a digit run
longer than `sys.get_int_max_str_digits()` (4300) raises ValueError too, and a
raise on a rejection path becomes a spurious 500.

`Transfer-Encoding` gets the same audit. Trimming is now SP/HTAB only, RFC 9110
OWS and nothing else, because `str.strip()` erased U+00A0/U+0085 there too:
`Transfer-Encoding: \xa0identity` -- a spelling no token grammar allows -- read
back as the `identity` token, so it counted as unframed and poisoned the socket
with the same 403-then-400 trace (411 and close on uploads). Both characters
reach the wire, since a request line decodes as latin-1. Membership in
`_UNFRAMED_TRANSFER_CODINGS` is tested through `str.lower()`, which maps some
non-ASCII characters ONTO ASCII ones (U+212A -> `k`), so the set is now
documented as ASCII-token-only and a test scans every codepoint to prove no
non-ASCII spelling can reach it.

Non-ASCII digits are unreadable by the same rule and pinned at the helper level:
U+0660/U+FF10 cannot cross an HTTP/1.1 header line (latin-1 leaves only the
U+00B2/B3/B9 superscripts, which `int()` already rejects), but the helpers take
any headers mapping and enforce the grammar rather than trusting the transport.

The no-body contract is unchanged and pinned in both directions on the wire and
at the helper: no length header, `0`, `00`, an OWS-padded zero, two AGREEING
zeroes, and `identity` in any ASCII case still keep the pooled connection alive.
2026-09-22 12:06:09 +00:00
Rodrigo Gomes da Silva 79f5b25f5f fix(http): treat a blank framing value as a declared body, not an absent one
`Content-Length:` with nothing after the colon reads back as '' from
`Message.get_all()`, and `_declared_content_lengths()` SKIPPED the empty value.
The result was "no length declared", so `request_declares_body()` reported no
pending body and the sidecar provenance rejection answered 403 with keep-alive
intact -- leaving the payload in rfile for the server to parse as the next
request line. Reproduced live, pipelined down one socket:

  GET /api/extensions/probe/sidecar/ping  `Content-Length:`  {"stale": true}
    -> HTTP/1.1 403 Forbidden       (no Connection: close)
    -> HTTP/1.1 400 Bad request syntax
           ('{"stale": true}GET /api/health/agent HTTP/1.1')

A blank value cannot prove a body's absence, so it now joins `banana` and two
disagreeing lengths as unreadable framing: bytes are assumed pending and the
connection closes. Absence is something only the wire can say, and an absent
header already yields no values at all.

Three adjacent holes of the same class were found by probing and are fixed by
the same rule (each reproduced live with the same bad-syntax 400):

* whitespace-only `Content-Length:   ` -- `Message` collapses it to the same '',
  as it does a duplicated blank or a blank paired with a real length.
* the four multipart upload handlers: a blank length slipped past
  `unreadable_content_length()`, reached `int('' or 0)` == 0, and answered 400
  "No file field in request" with the payload still queued. Now 400 "Invalid
  Content-Length" with the connection closed.
* blank `Transfer-Encoding:` -- '' sat in `_UNFRAMED_TRANSFER_CODINGS` beside
  `identity`, so a header naming no coding read as "nothing to frame". A blank
  coding is undecodable, not unframed; uploads answer 411 and close.

The no-body contract is unchanged and pinned in both directions: no length
header, a padded zero, two AGREEING zeroes and `identity` all still keep the
pooled connection alive.
2026-09-22 12:06:09 +00:00
Rodrigo Gomes da Silva fb7664bc40 fix(http): read framing from every header value, not just the first
Adversarially probing the re-gated head found the same poisoning reachable by
DUPLICATING a framing header instead of omitting one. `Message.get()` returns
only the first occurrence, so `Content-Length: 0` sent ahead of the real length
made the body invisible to every reader: nothing was drained and the real bytes
were parsed as the next request line. Reproduced live on the otherwise-fixed
head, over one pooled socket:

    HTTP/1.1 403 Forbidden                    <- sidecar provenance reject
    HTTP/1.1 400 Bad request syntax ('{"stale": true}GET /api/health/agent ...')

    HTTP/1.1 400 Bad Request                  <- /api/upload reject
    HTTP/1.1 400 Bad request syntax ('--x')

api/helpers.py now reads framing through `_framing_header_values()`, which
returns every value the wire carried. `request_declares_body()` treats
disagreeing or unparseable lengths as a declared body (they cannot prove the
body's absence), and `unsupported_transfer_encoding()` inspects every declared
coding so `identity` cannot mask a `chunked` behind it. Two agreeing zeroes are
still no body -- repetition alone is not a conflict.

New `unreadable_content_length()` gates the multipart preflight: with no honest
byte count there is nothing to drain, so all four handlers refuse with 400
"Invalid Content-Length" before touching rfile, connection armed for close.

13 regressions, each verified red without this commit: the helper predicates on
a real email.message.Message, the production sidecar 403, all four multipart
handlers, and five pipelined real-socket tests proving the follow-up request is
never answered off a poisoned socket.
2026-09-22 12:06:09 +00:00
Rodrigo Gomes da Silva b3a031a19d fix(http): decide reject-before-read closes from the framing, not the method
Two framing gaps the re-gate reproduced on the production handlers, both the
same class: something other than the wire was used to decide whether unread
body bytes are queued.

1. api/routes.py — the sidecar provenance 403 gated its close on
   `read_request_body`, a static per-method flag. That is wrong in both
   directions: a GET carrying `Content-Length`/`Transfer-Encoding` was answered
   with keep-alive and its body then parsed as the next request line
   (`501 Unsupported method ('{}GET')`), while a body-less DELETE/PUT/PATCH had
   a healthy pooled connection closed for nothing.

2. api/upload.py — all four multipart handlers read framing from
   `Content-Length` alone, so a chunked upload (which has none) was treated as
   length zero, matched no parts and was rejected 400 "No file field in
   request" with keep-alive intact. Wire trace before the fix:

       HTTP/1.1 400 Bad Request                  <- the upload rejection
       HTTP/1.1 400 Bad request syntax ('54')    <- our own chunk-size line

New primitive in api/helpers.py: `request_declares_body()` (Content-Length
present and non-zero, or any Transfer-Encoding other than identity; an
unparseable length counts as present, since it cannot prove absence) plus
`arm_connection_close_if_body_pending()` and `unsupported_transfer_encoding()`.

api/upload.py now routes all four handlers through one preflight,
`_read_multipart_or_reject()`, which refuses an undecodable transfer coding
with 411 before touching rfile — http.server never decodes chunked framing, so
that request can only be rejected — and keeps every previous status and message
(413 oversize, 400 invalid Content-Length/boundary) with its close.

Regressions: chunked reject that closes and a body-less reject that does not,
at all four upload handlers and at the sidecar 403, plus two pipelined
real-socket tests against the running server proving the follow-up request is
never answered off a poisoned socket.
2026-09-22 12:06:09 +00:00
Rodrigo Gomes da Silva 304815228a fix(http): don't let a non-iterable header buffer crash end_headers
advertise_connection_close() guarded its two attribute reads but not the
dedup scan over _headers_buffer. A stub whose buffer is truthy and
non-iterable (a bare Mock) raised TypeError from inside end_headers(),
which _handle_write's generic except would swallow into a spurious 500 --
the same failure class as the missing close_connection guard.
2026-09-22 12:06:09 +00:00
Rodrigo Gomes da Silva 9a181c0026 fix(http): emit Connection: close exactly once and never crash on stubs
The first cut of the close advertisement had two defects, both caught by CI
(9 red jobs) and by the deep-review gate's own MUST-FIX 1 wording.

1. Dedup was keyed off a private `_close_advertised` flag that end_headers()
   set itself. Every SSE endpoint already sends its own `Connection: close`,
   and BaseHTTPRequestHandler.send_header flips close_connection when it does
   — so end_headers() appended a SECOND copy to every stream response.
   Dedup now derives from the pending header buffer, which sees those
   site-level headers too.

2. `self.close_connection` was read unguarded. The attribute only exists once
   a request line has been parsed, so the many suite handlers built with
   `Handler.__new__(Handler)` raised AttributeError instead of emitting
   headers — and in _handle_write that AttributeError became a spurious 500.
   Both reads are now defensive, and check_auth_or_close() no longer invents
   the attribute on a handler that never had it (which would have left a bogus
   keep-alive verdict behind).

The mechanism moves to api/helpers.advertise_connection_close(), which also
puts server.py back under the original 750-line guard, so the test's ceiling
bump to 760 is reverted rather than shipped.
2026-09-22 12:06:09 +00:00
n 3da9a0975c Merge PR #7641: fix(redaction): validate MPF JPEG attachments before the native-image exemption 2026-09-20 02:25:35 +00:00
George Byers II 34bc4b9cd0 fix: validate MPF JPEG attachments before redaction 2026-09-18 20:35:31 -04:00
Anthony Ruiz fc7ae2ddd1 fix(redaction): fail closed on large strings, byte-bound the prefilter cache
Gate found two blocking defects in the new caches:

1. The large-string LRU was keyed on input text only. The installed
   agent.redact registry can extend the secret matcher at runtime via
   register_redaction_patterns() and exposes no policy generation or
   invalidation hook, so a >16k string primed before a registration was
   served with the new secret intact (reproduced against the installed
   registry). Drop the large-string cache: strings above 16,384 chars run
   the redactor uncached again. Regression primes a large string, registers
   a runtime pattern on the real registry and proves the next
   redact_session_data() response is redacted.

2. The prefilter LRU was count-bounded (8192) but its per-entry gate was a
   character cap, which is not a byte bound: 16k four-byte code points is
   ~64 KiB per key, ~512 MiB at capacity, and the workload died with
   MemoryError under 96 MiB of headroom. Gate each entry on
   sys.getsizeof(text) <= 2048 so the retained-key ceiling is a hard
   16 MiB. Regressions cover max-width Unicode, sensitive verdict pairs,
   eviction past the cap, and a low-memory RLIMIT_AS subprocess.
2026-09-17 23:35:54 +00:00
Anthony Ruiz 86fe1a5746 perf(webui): make redaction containers copy-on-write 2026-09-17 23:21:46 +00:00
Hermes Agent b7f30c6365 perf(webui): memoize the sensitive-text prefilter
Once large strings stopped being re-redacted on every response, cProfile
showed the cheap prefilter had become the dominant cost of the redaction
pass: 2.98s of 3.7s, spent testing 71 literal markers against each of the
~74k strings (~19MB) of a large session, i.e. ~3.1M interpreted iterations
per request — repeated by every tab and every poll for a constant verdict.

Memoize the verdict behind a bounded LRU, mirroring the existing redaction
caches: entry count and per-entry size are both capped, so retained bytes
stay bounded and oversized blobs simply run the scan uncached. Non-strings
never reach the cache, so an unhashable value cannot raise.

Measured on a real 22MB session, warm: full redaction 2.07s -> 1.07s (-49%),
output byte-identical. Combined with the large-string cache, the recurring
redaction cost drops from ~8.4s to ~1.1s.

Rejected alternative, kept in a comment: collapsing the 71 `in` scans into
one compiled alternation is the obvious-looking fix but measured 38% SLOWER
(2.60s vs 1.88s) — CPython's substring search is already a tuned C routine.

Security: the prefilter gates the entire redaction pass, so equivalence is
pinned by a differential test suite against a copy of the previous
implementation — every marker individually, case variants, embeddings,
near-misses, unicode lowercase traps, 4000 fuzz cases, plus oversized and
unhashable inputs. Verdicts are identical on all 73,658 strings of a real
session (1935 flagged before and after).

Broad selection: 22 failures before and after, identical set — all
pre-existing.
2026-09-17 23:21:46 +00:00
Hermes Agent 6f05ff9657 perf(webui): stop re-redacting large strings on every response
`_redact_fn_cached` skipped memoization for any string over 16KB, so those
strings re-ran the full ~15-pass redactor on every single request. On a real
22MB session that is 59 large strings (29 unique, 0.76MB) costing 1.68s per
request — 99.9% of the recurring redaction cost once the small-string cache is
warm — repaid by every tab and every poll for a byte-identical result.

Large strings now get their own bounded cache, separate from the 4096-entry
small-string one so a large blob can never evict the thousands of small
recurring strings. Entry count and per-entry size are both capped; anything
above the ceiling stays uncached, keeping retained bytes bounded.

`_redact_value` additionally stops rebuilding containers whose descendants
were unchanged. Most of a transcript carries no credential marker, so the
unconditional dict/list comprehension deep-copied the whole payload to
reproduce an identical structure — pure CPU under the GIL, which serializes
concurrent tab loads on a threaded server. Callers treat redacted output as
read-only (`_public_message_projection` and `redact_session_data` both build
fresh containers), so sharing an unmodified subtree cannot leak a later
mutation back into session state. As soon as one string is masked, every
container on the path to it is rebuilt and the caller's original is untouched.

Measured on the live 22.6MB session, steady state: redact_session_data 3.78s
-> 1.96s single tab (-48%), 7.93s -> 4.06s with two concurrent tabs (-49%),
byte-identical output.
2026-09-17 23:21:46 +00:00
nandzicochaos 1e17127f49 fix: treat whitespace-only body as empty ({}) not a 400
Gate finding on #7336: read_body() rejected whitespace-only bodies that
worked on master (a body-optional DELETE/PATCH/PUT can send b' \n'), returning
400 Invalid JSON body. Normalize a fully-whitespace body to {} before json.loads,
matching the pre-#7336 lenient behavior for that shape. Adds a parametrized
regression test.

Co-authored-by: zicochaos <zicochaos@users.noreply.github.com>
2026-09-09 04:21:46 +00:00
Sebastian B 743b757a77 fix(api): reject malformed and non-object JSON bodies with clean 400s
read_body() silently turned malformed JSON into {} (clients then got a
misleading 'Missing field' 400) and passed non-dict JSON through to handlers
where body.get() raised AttributeError and surfaced as a generic 500.
Raise ValueError instead — handle_post already maps it to a clean 400 — and
give the PATCH/DELETE/PUT dispatchers the same mapping they lacked. Empty
bodies keep the documented {} default.
2026-08-28 00:49:50 +01:00
43bcdcf97f fix(transcript): recognize Agent _row_id during replay merge (#7133)
* fix(transcript): recognize agent row ids in replay merge

* Strip imported _row_id so replay merge cannot trust caller provenance.

Session import already dropped _state_db_row_id aliases. After treating
Agent _row_id as identity, the same public/import scrubber must drop that
field too, or a crafted import can collide with a later state.db turn.

---------

Co-authored-by: Anthony Ruiz <ruizanthony@users.noreply.github.com>
Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
2026-08-20 01:01:10 +00:00
4d03578192 fix(#6611): make regeneration start atomic (#6677)
* fix(#6611): make regeneration transaction canonical

* fix(#6611): close cross-provider regeneration blockers

* fix(#6611): harden regeneration transaction boundaries

* fix(#6611): bind canonical context and fork lineage

* fix(#6611): restore active turn token stamping

* test(#6611): make regeneration regressions CI portable

* test(#6611): preserve the tracked regeneration error fixture

* fix(#6611): preserve extracted session harness contracts

* fix(regeneration): preserve concurrent and imported turn ownership (#6611)

* fix(regeneration): honor imported turn ownership safely (#6611)

* test(regeneration): cover both session lock outcomes (#6611)

* test(regeneration): clear unused transaction fixtures (#6611)

* fix(#6611): consume public active-turn marker

* ci: re-trigger (Playwright install hung ~2.5h on GH runners, warm-cache retry)

---------

Co-authored-by: nesquena-hermes <nesquena+hermes@gmail.com>
Co-authored-by: n <a@n>
2026-08-19 11:11:53 +00:00
nesquena-hermesandfranksong2702 cf7eb48e56 Release exp-v0.52.197: preserve api_content for agent replay (#6757, @franksong2702)
Ships PR #6757 with a maintainer hardening pass folded in (authorship preserved).

Preserves the exact provider-facing reply text (api_content) through the internal
session store for byte-accurate agent replay, while guaranteeing it never reaches
a browser: every client-facing surface strips it via the public projection.

Maintainer fixes (2 defects my adversarial re-gate reproduced):
- Selective session redaction restored: only transcript fields are scrubbed;
  operational fields (e.g. workspace path) pass through unchanged, so a path
  containing credential-shaped text is no longer masked into an invalid path.
- Runner-backed SSE relay now projects payloads before emit, closing an
  api_content leak through _stream_runner_run_events.
+5 regression tests. Codex adversarial re-gate: SAFE TO SHIP. Full suite clean.

Co-authored-by: franksong2702 <franksong2702@users.noreply.github.com>
2026-08-11 02:10:21 +00:00
33d55ec897 Release: raster-data-URI redaction fast-path (#6311, @inch772) (#6367)
* fix(redaction): skip native raster data URIs

* fix(redaction): accept mixed-case raster MIME types

* fix(redaction): validate complete raster payloads

* docs(changelog): stamp #6311 raster-data-URI redaction fast-path

---------

Co-authored-by: Su Ahn Lee <11433303+inch772@users.noreply.github.com>
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
2026-07-20 16:30:04 +00:00
b7b6b3a4c8 Release exp-v0.52.43: trusted-header auth / reverse-proxy SSO (#5568, #3351) (#5956)
* feat(#3351): add trusted-header auth core

* fix(#3351): reconcile trusted sessions per request

* fix(#3351): preserve valid trusted group mappings

* fix(#3351): defer trusted logout reconciliation until after csrf

* fix(auth): clear _pending_set_cookies in reset_trusted_auth_request_state (keep-alive cookie cross-request leak)

Codex gate CORE finding: a Set-Cookie queued in request N but not flushed
survived onto the reused HTTP/1.1 handler and could be emitted by request N+1
— after trusted-identity rotation on logout this could overwrite a valid login
cookie and 401 the user. Reset the queue at the per-request boundary + add a
keep-alive regression test.

* docs(.env.example): document trusted-header auth / reverse-proxy SSO env vars

HERMES_WEBUI_TRUSTED_AUTH_HEADER + TRUSTED_PROXY_CIDRS (with exact-proxy-IP
security warning) + optional groups-header/JSON group-profile-map/logout-url.

* Release exp-v0.52.43: trusted-header auth / reverse-proxy SSO (#5568, #3351) + keep-alive cookie fix + docs

---------

Co-authored-by: Rod Boev <rod.boev@gmail.com>
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
2026-07-11 16:34:14 -07:00
nesquena-hermes e061e8534c Rebase #5337 (public read-only conversation sharing) onto master
Rebased 363 commits forward (branch-point 2026-07-05). Applied the PR's code
diff cleanly across 16 files; resolved api/models.py additively (both master's
process_wakeup_pause/anchor_scene_index fields AND the PR's share_token/
share_created_at fields kept in the Session constructor, metadata list, and
to_dict). CI workflow (.github/workflows/tests.yml) intentionally NOT taken from
the stale branch — master owns it (the branch's apparent removal of Playwright/
Office steps was a PAT-workflow-scope revert; rebase restores master's CI).

Co-authored-by: MicoRobot. Concept approved by maintainer (opt-in public share,
2026-07-11). Note: 4 dynamic share tests fail identically on the original
unrebased head (pre-existing test-env Session.load fragility, not rebase-caused);
to be resolved during the security re-gate in a proper sandbox.
2026-07-11 01:39:42 +00:00
AnhNVT 9ad1fa0372 fix(redaction): memoize deterministic API response redactor
The combined credential redactor (~15 regex passes per string) runs on the
full message history of every /api/session response. Under concurrent
dashboard polling of unchanged sessions this became the dominant CPU cost
and, behind the GIL in the single-process server, serialized requests to
10-44s each — surfacing as repeated 'connection lost' in the browser as
the proxy/browser timed out and the frontend retried, piling on threads.

The redactor is pure and deterministic (force=True, fixed masking), so
identical strings always map to identical output and are safe to memoize
without invalidation. Wrap it in an lru_cache(maxsize=4096), gated to
strings <=16KB so a few giant tool-output dumps can't evict the many small
recurring strings or balloon RSS. Repeated polls become O(1) cache hits
(measured 3.7ms -> 0.002ms per 3KB marker string).

_redact_fn_cached is preserved as the call-site name so the existing
monkeypatch-based redaction tests bypass the cache unchanged (69/69 pass).
2026-06-29 18:22:18 +00:00
nesquena-hermes 58ba991c0d feat(extensions): opt-in CSP frame-src allowlist (HERMES_WEBUI_CSP_FRAME_EXTRA)
Adds a configurable frame-src directive so an extension can embed a
self-hosted web app in an iframe (e.g. an 'external app tab' pinning
Grafana / Vaultwarden / a dashboard). Mirrors the existing
HERMES_WEBUI_CSP_CONNECT_EXTRA knob: space-separated http(s) origins with
optional *.subdomain wildcard + port, validated, ignored if malformed
(directive-injection / paths / ws scheme / bad ports all rejected).

- Default frame-src is 'self' only, so existing same-origin iframes are
  unchanged; default-off when the env var is unset.
- frame-ancestors stays 'none' -- this only governs what THIS page may embed,
  never who may embed the WebUI.
- Threaded through both the enforced and report-only CSP builders + the
  server.py Handler wrapper, so both headers stay identical.

Docs in docs/EXTENSIONS.md. 8 new regression tests
(tests/test_csp_frame_src_extra.py) cover default, valid widening, enforced
parity, directive-injection rejection, path/ws/port rejection, and
connect/frame independence. Existing CSP test suite still green.

Note: full suite shows 2 pre-existing order-dependent flakes
(test_issue1574 spawn-context, test_issue3283 import-order) that pass in
isolation and are untouched by this CSP-only diff.
2026-06-28 04:08:04 +00:00
Frank Song 6079b7ba24 Address compaction marker and redaction review 2026-06-26 00:12:09 +00:00
Frank Song 9f0864ea82 Fix compaction marker writeback and code redaction 2026-06-26 00:12:09 +00:00
0fd9bf3c37 Release v0.51.564 — loopback sidecar diagnostics (#4612) (#4632)
* stage #4612 (santastabber): surface loopback sidecar diagnostics + CHANGELOG

Settings -> Extensions surfaces manifest-declared loopback sidecar companion services
(type:loopback, http(s) on 127.0.0.1/localhost/::1 only, sanitized origin+health_path)
as read-only health cards w/ a browser-side credentials-omitted health probe. Backend
hard-whitelists scheme + loopback host, rejects userinfo/path/query/traversal, rebuilds
output from parsed components (no echo of rejected input); frontend esc()'s every field,
whitelists badge status, fetch never reads body. Purely additive to GET /api/extensions/status.
Code applied byte-faithful from PR head 7080929f3727. 33 own tests pass.
Nathan concept-approved (loopback-companion direction).

Co-authored-by: santastabber <santastabber@users.noreply.github.com>

* fix #4612 Codex gate findings: reject encoded query/fragment in health_path + align CSP loopback allowlist

Codex SHIP-WITH-FIXES (2 reproduced SILENT findings):
1. _normalize_sidecar_health_path percent-decoded AFTER the raw query/fragment ban,
   so /health%3Ftoken=abc -> ?token=abc survived into the probed URL. Now re-rejects
   decoded ? and # + regression test (%3F/%23 skipped with sidecar_health_path_rejected).
2. The origin validator accepts https:// and [::1], but CSP connect-src only listed
   http 127.0.0.1/localhost -> accepted https/IPv6 loopback sidecars would be CSP-blocked.
   Aligned _CSP_CONNECT_BASE (added https v4/localhost + http(s) [::1]) + updated the 4
   exact-string CSP tests + a regression test asserting all loopback forms are allowed.

Co-authored-by: santastabber <santastabber@users.noreply.github.com>

* Release v0.51.564 — Release TW (loopback sidecar diagnostics; #4612)

* fix(#4612): drop browser-invalid [::1]:* CSP source (browser-smoke fail)

The CSP-loopback-alignment fix added http(s)://[::1]:* to connect-src, but CSP
host-source grammar can't express a port wildcard on a bracketed IPv6 literal —
Chromium rejects '[::1]:*' as an invalid source, which browser-smoke flags as a
console error (CI fail). Keep the valid https://127.0.0.1:*/localhost:* additions;
drop the IPv6 wildcard entries. IPv6 [::1] sidecars are still accepted by the
validator (displayed in the card); their browser health probe surfaces as
'blocked' under CSP unless an operator adds a specific-port [::1]:<port> via
HERMES_WEBUI_CSP_CONNECT_EXTRA. Tests updated to match + assert [::1]:* absent.

---------

Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
Co-authored-by: santastabber <santastabber@users.noreply.github.com>
2026-06-21 13:07:32 -07:00
nesquena-hermesandai-ag2026 0b57a7101f fix: keep session sidebar visible on slow loads (#4167)
Sidebar vanished during resilient/slow session loads. Preserve sidebar
fields on resilient loads + add an opt-in api() retry so a slow /api/sessions
shows a retryable error state instead of an empty sidebar.

Maintainer fixes for the 3 review-blocking SILENT regressions:
- Restore read_only / is_read_only / gateway_routing to the sidebar response
  allowlist, so read-only (imported CLI + Claude Code) sessions keep their
  rename/action-menu/swipe suppression and the gateway model label renders.
  (gateway_routing_history intentionally NOT sent — the label reader prefers
  the latest gateway_routing, and the full history would bloat each row.)
- Scope-tag the session cache (active profile + all-profiles flag). When a
  /api/sessions refresh fails right after a profile switch, the catch path now
  clears the stale rows and shows the error state instead of re-rendering the
  PRIOR profile's sessions.

Co-authored-by: ai-ag2026 <ai-ag2026@users.noreply.github.com>
2026-06-16 18:42:01 +00:00
nesquena-hermes 6e6931a8c3 Merge #4028 (align profile cookie env var with HERMES_WEBUI_ prefix) onto master 2026-06-13 07:20:58 +00:00
Rod Boev ddcaf6091c fix(#4040): allow Tauri IPC host in connect-src 2026-06-12 07:50:43 -04:00
gaku 6a13feaf8b refactor(profiles): align profile cookie env var with HERMES_WEBUI_* naming
The profile cookie has been configurable since #1756 via
WEBUI_PROFILE_COOKIE_NAME, the lone WebUI env var missing the HERMES_WEBUI_
prefix shared by every other setting (e.g. HERMES_WEBUI_COOKIE_NAME from #3981).

- Read HERMES_WEBUI_PROFILE_COOKIE_NAME first (canonical name)
- Keep WEBUI_PROFILE_COOKIE_NAME as a deprecated alias so existing deployments
  are unaffected; behavior is unchanged, only the name is aligned
- Warn once per process for the legacy name (this resolver runs on every
  request, so the deprecation log must not fire per-request)
- Add resolution tests covering canonical, legacy, precedence, blank, and
  warn-once paths
2026-06-12 09:55:49 +00:00
nesquena-hermes 03799f8e4a harden(#4023): apply Opus security findings — verify-side name-pattern gate + require handler when auth enabled
Opus independent security review concurred SAFE and surfaced 2 LOW defense-in-depth
items, both applied: (1) verify_profile_cookie_value now validates the profile name
against _PROFILE_ID_RE itself (not only in get_profile_cookie) so a future second
caller can't return an unvalidated name; (2) build_profile_cookie raises when auth is
enabled and no handler is passed, so a future call site can't silently emit an
unsigned (session-unbound) profile cookie. +3 regression tests.
2026-06-12 07:37:48 +00:00
hinotoi-agent 4dca506858 Bind profile cookie to auth session 2026-06-12 10:52:56 +08:00
nesquena-hermesandt3chn0pr13st 6bac70d298 fix: preserve live stream output across session switches (cross-client)
Adds a server-side run-journal live snapshot (_run_journal_live_snapshot) returned
in GET /api/session as runtime_journal_snapshot, so a FRESH client (another device,
or a tab with no in-memory snapshot) opening an in-progress session immediately sees
the already-streamed assistant text + tool cards rebuilt from the server. Composes
with the existing _replay_run_journal cursor path (seeds lastRunJournalSeq so replay
resumes from the snapshot cutoff, not duplicating it) and keys tool cards by the same
5 id aliases (tid/id/tool_call_id/tool_use_id/call_id) as #3763 so SSE replay replaces
rather than duplicates snapshot cards. Payload values truncated; redaction test added.

Co-authored-by: t3chn0pr13st <technopriest@live.ru>
2026-06-10 20:12:05 +00:00
nesquena-hermes bc6949219d Release v0.51.317 — Release KG (Phase 3 light: align CSP enforcement with report policy, #3727) (#3786)
Phase-3-LOW backend security fix. #3727 (@rodboev, #1909): enforced CSP now honors the same connect-src as report-only (built from one shared template). Full suite 8205 passed, Codex SAFE (live-verified headers), Opus SHIP. Co-authored-by: rodboev <rodboev@users.noreply.github.com>
2026-06-07 10:52:07 -07:00
8f89b4f825 Release v0.51.278 — Release IT (stage-p3g — repair inline PDF preview #3652) (#3684)
* fix(ui): repair inline PDF preview (blob module loader + CSP worker-src) (#3652, #3649)

Co-authored-by: sky <example@email.com>

* docs(changelog): v0.51.278 — Release IT (stage-p3g, #3652 only); widen CSP test window

---------

Co-authored-by: nesquena-hermes <[email protected]>
Co-authored-by: sky <example@email.com>
2026-06-05 14:27:38 -07:00
Qi Zhou 5c595c08d6 fix(todos): hydrate cold-load state from session snapshot 2026-06-02 17:33:49 +00:00
Ed 79f526b1b2 fix(helpers): use ssl.SSLError instead of broad OSError in disconnect tuple
OSError is too broad — it masks real errors like file-not-found.
ssl.SSLError specifically catches SSL-level disconnects without
swallowing unrelated OSError subtypes.

Closes the test_excludes_broad_oserror CI failure.
2026-05-31 01:56:39 +00:00
Ed 78652ecdea fix: consolidate _CLIENT_DISCONNECT_ERRORS, remove scratch file
Address review feedback from @nesquena-hermes on PR #3210:

1. Deduplicate _CLIENT_DISCONNECT_ERRORS:
   - Single authoritative definition in api/helpers.py
   - api/routes.py now imports from api.helpers instead of defining
     its own copy with different membership
   - Unified tuple uses OSError (covers ssl.SSLError since it
     subclasses OSError) — broad socket-level disconnect coverage

2. Remove github-search-report.md:
   - Research scratch output that doesn't belong in the repo root
   - Content belongs in PR description or a gist

3. Docstring improvement:
   - Added comment explaining why OSError covers ssl.SSLError
   - Documents the errno-level socket errors caught by OSError
2026-05-31 01:56:39 +00:00
Ed 59b7a3ac94 fix: eliminate silent failures in client disconnect handling
- api/helpers.py: _safe_write() now logs disconnects at debug level
  instead of silently passing. No more invisible errors.

- server.py: Restructure exception handlers to catch
  _CLIENT_DISCONNECT_ERRORS first, then Exception. Remove the
  isinstance() filter inside except Exception (LBYL anti-pattern).
  The 500-response fallback now catches _CLIENT_DISCONNECT_ERRORS
  separately (expected) and logs unexpected failures via
  traceback.print_exc() instead of bare except Exception: pass.

- tests/test_broken_pipe_cascade.py: Add coverage for SSL/Timeout
  disconnect routing and 500-response safety (both disconnect
  survival and unexpected error logging).
2026-05-31 01:56:39 +00:00
someaka 2cee44b1b2 fix(helpers): catch client disconnect errors in response write path
Extract _safe_write() helper that wraps end_headers() + wfile.write()
in try/except (BrokenPipeError, ConnectionResetError, ConnectionAbortedError,
TimeoutError, ssl.SSLError).  Both j() and t() now use _safe_write()
instead of raw wfile calls.

Fixes cascading BrokenPipeError + SSL BAD_LENGTH crash when a client
disconnects mid-response and the error handler tries to write a 500
status through the same broken socket.
2026-05-31 01:56:39 +00:00
AJV20 60d4b2d990 fix: harden WebUI bugfix sweep 2026-05-28 13:38:50 -04:00
ai-ag2026 815df9de76 fix: vendor math and yaml assets for CSP 2026-05-26 23:49:02 +02:00
Hermes Agent 7c2b2785e7 stage-348: apply Opus SHOULD-FIX-pre-merge — add '://' to _SENSITIVE_LOWER_MARKERS
Opus advisor flagged that PR #2171's credential prefilter only listed
specific DB scheme prefixes and form keys, letting OAuth callback URLs,
URL userinfo, signed-URL query params bypass the hard agent redactor.

Adding the generic '://' marker restores the WebUI-as-hard-safety-boundary
contract. Plain URLs without sensitive substrings still pass through
unchanged because the redactor itself only mutates sensitive substrings.

Regression-pinned with 5 new parametric cases in test_security_redaction.py
plus 1 negative-case companion. Verified test FAILS without the fix and
PASSES with it.
2026-05-13 16:54:36 +00:00
Frank Song da73c00f06 Harden session tail redaction prefilter 2026-05-13 18:58:49 +08:00
Frank Song b7ac5a8b88 Trim session tail response overhead 2026-05-13 15:57:29 +08:00
ChaseFlorell d8612ba323 fix: add cdn.jsdelivr.net to CSP connect-src to allow xterm source map fetches
Closes #1850

Co-authored-by: Chase Florell <ChaseFlorell@users.noreply.github.com>
2026-05-07 20:42:55 +00:00
Nick d5a31a0f4d fix: isolate profile cookie per webui instance 2026-05-06 22:01:20 +00:00
nesquena-hermes 5ce516ed38 v0.50.255: Opus follow-ups (4 fixes) + CHANGELOG
Opus pre-release advisor caught 4 issues in stage-255 (#1390 + #1405):

1. MUST-FIX: api/rollback.py path-traversal — _checkpoint_root() / ws_hash /
   checkpoint did NOT normalize Path() / "../escape", so an authenticated
   caller could read or restore from another allowlisted workspace via
   ../<other-ws-hash>/<sha>. New _validate_checkpoint_id() regex-guards
   with ^[A-Za-z0-9_-][A-Za-z0-9_.-]{0,63}$ and rejects . and .. literals.
   Both get_checkpoint_diff and restore_checkpoint validate.

2. SHOULD-FIX: redact_session_data perf cliff — the new api_redact_enabled
   toggle in #1405 called uncached load_settings() per string, recursed
   across messages[] and tool_calls[]. For a 50-message session: hundreds
   of disk reads per /api/session response. Now read once at the top and
   thread _enabled through via private kwarg.

3. SHOULD-FIX: voice-mode wrong-session TTS — the patched autoReadLastAssistant
   fires globally; if the user navigated to a different session between
   sending and stream completion, TTS would speak the wrong session\\s reply.
   New _voiceModeThinkingSid closure captures S.session.session_id at
   thinking-time; _speakResponse bails to _startListening() on mismatch.

4. NIT: rollback._inspect_checkpoint had bare Exception in the except tuple
   alongside specific catches, swallowing everything. Now (TimeoutExpired,
   OSError) only.

6 regression tests in test_v050255_opus_followups.py. Full suite: 3587 passed,
2 skipped, 3 xpassed.
2026-05-01 17:19:53 +00:00