00afdf7fb05ddec0da532fedf025a20dfa4229ec
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Rodrigo Gomes da Silva
|
7862927ef5 |
fix(http): keep a bodyless rejection's keep-alive, at every reject site
The mirror image of the sidecar over-close already fixed here: the sibling reject-before-read sites still armed `arm_connection_close()` unconditionally, so a request with NO declared body was answered with `Connection: close` and the client's pipelined follow-up was dropped -- a healthy pooled connection killed for no framing reason. Reproduced on the wire, pipelined down one socket against the production handler (`responses seen: 1` at each site): POST /api/session/new off-session auth -> 401 + Connection: close POST /api/session/new bad Origin -> 403 + Connection: close POST /api/session/new no CSRF token -> 403 + Connection: close POST /api/csp-report limiter tripped -> 204 + Connection: close POST /api/health/restart success -> 200 + Connection: close POST /api/process-complete-ack -> 410 + Connection: close POST /api/upload no boundary -> 400 + Connection: close Each now routes through `arm_connection_close_if_body_pending()` (the helper the sidecar path already used), so the framing decides in both directions: body-bearing or unreadable framing still closes, an absent or agreeing-zero Content-Length keeps keep-alive. `check_auth_or_close()` cannot swap the helper in directly -- it has to arm before `check_auth()` writes its 401/302 and undo that on success -- so it gates the arming on `request_declares_body()` instead and keeps the success-path restore. `/api/health/restart` is armed once, ahead of the outcome, so all four results (completed / in_progress / busy / error) follow the framing rather than the result; the success path was closing on every call the WebUI makes. Only the boundary check in the upload preflight can reject a bodyless request: every other raise there (any Transfer-Encoding, an unreadable or conflicting length, an over-cap length) already declares a body and still closes. Five tests pinned the unconditional close on stubs that declared no body -- i.e. they pinned the bug. Each is re-grounded on real body-bearing framing (Content-Length, chunked, and an unreadable length) so the close half stays pinned, with a bodyless sibling asserting keep-alive. New real-socket regressions cover all seven sites in both directions. |
||
|
Rodrigo Gomes da Silva
|
b022a79eef |
fix(http): reject every Transfer-Encoding, not just the ones we can't name
`identity` was exempt as a coding that "frames nothing", so a request
carrying it read back as body-less however many payload bytes followed.
It frames nothing either: RFC 9112 6.3 says a request whose final
transfer coding is not `chunked` has a body length no recipient can
determine, RFC 7230 removed `identity` from the transfer codings
outright, and `http.server` decodes none of them (`rfile` is the raw
socket). Every present value has to reject-and-close.
Reproduced on 8c326003b, pipelined down one socket:
GET /api/extensions/probe/sidecar/ping `Transfer-Encoding: identity`
{"stale": true}
-> HTTP/1.1 403 Forbidden (no Connection: close)
-> HTTP/1.1 400 Bad request syntax
('{"stale": true}GET /api/health/agent HTTP/1.1')
POST /api/upload (and /upload/extract, /workspace/upload, /transcribe)
-> HTTP/1.1 400 Bad Request ("No file field in request", no close)
-> HTTP/1.1 400 Bad request syntax ('--x')
Deletes the `_UNFRAMED_TRANSFER_CODINGS` allowlist and the fold-safety
test that existed only to guard its membership check; `_BLANK_TRANSFER_CODING`
stays, since a present-but-empty header still needs a named, non-falsy
report. The identity rows of the framing tables and of
`test_bodyless_framing_keeps_the_pooled_socket_alive` move to the
closes-the-socket side, with real-socket coverage on the sidecar path and
all four upload endpoints. The keep-alive half stays pinned by the cases
that genuinely have no body: no framing header at all, `Content-Length: 0`,
an OWS-padded zero, two agreeing zeroes.
Swept the rest of the coding surface for the same class: every case
spelling, OWS and non-OWS padding, `identity, chunked`, `chunked, identity`,
`identity,identity`, a lone comma, an unknown coding, duplicated headers,
and a coding beside both a real and a zero Content-Length.
|
||
|
Rodrigo Gomes da Silva
|
01f2446459 |
fix(http): read a length only when it is 1*DIGIT, not whenever int() copes
`_declared_content_lengths()` parsed with `int()`, which is far looser than RFC
9110's `Content-Length = 1*DIGIT`: it accepts a leading sign, PEP 515
underscores and non-ASCII digits, and `str.strip()` ahead of it also erased
non-ASCII whitespace. Every such spelling of ZERO therefore parsed to an honest
`0`, took the "every declared length agrees on zero" keep-alive branch, and left
the payload in rfile for the server to parse as the next request line -- the
same defect the blank value had, one layer down. Reproduced live at the previous
head, pipelined down one socket:
GET /api/extensions/probe/sidecar/ping `Content-Length: +0` {"stale": true}
-> HTTP/1.1 403 Forbidden (no Connection: close)
-> HTTP/1.1 400 Bad request syntax
('{"stale": true}GET /api/health/agent HTTP/1.1')
byte-for-byte the trace greptile filed the P1 for, and identical for `-0`,
`0_0`, `+00`, `0_0_0`, `+0_0`, `\xa00`, `0\xa0` and `\x850`. On /api/upload the
same values slipped `unreadable_content_length()` into `int('0_0')` == 0 and
answered 400 keep-alive then `400 Bad request syntax ('--x')`.
A sign was reported safe last round because it was probed as `+15`, where a
wrong parse closes anyway; only the zero-valued spellings expose it.
The parse is now the grammar: an unsigned run of ASCII digits, or the framing is
unreadable and a body is assumed pending. The `int()` guard stays -- a digit run
longer than `sys.get_int_max_str_digits()` (4300) raises ValueError too, and a
raise on a rejection path becomes a spurious 500.
`Transfer-Encoding` gets the same audit. Trimming is now SP/HTAB only, RFC 9110
OWS and nothing else, because `str.strip()` erased U+00A0/U+0085 there too:
`Transfer-Encoding: \xa0identity` -- a spelling no token grammar allows -- read
back as the `identity` token, so it counted as unframed and poisoned the socket
with the same 403-then-400 trace (411 and close on uploads). Both characters
reach the wire, since a request line decodes as latin-1. Membership in
`_UNFRAMED_TRANSFER_CODINGS` is tested through `str.lower()`, which maps some
non-ASCII characters ONTO ASCII ones (U+212A -> `k`), so the set is now
documented as ASCII-token-only and a test scans every codepoint to prove no
non-ASCII spelling can reach it.
Non-ASCII digits are unreadable by the same rule and pinned at the helper level:
U+0660/U+FF10 cannot cross an HTTP/1.1 header line (latin-1 leaves only the
U+00B2/B3/B9 superscripts, which `int()` already rejects), but the helpers take
any headers mapping and enforce the grammar rather than trusting the transport.
The no-body contract is unchanged and pinned in both directions on the wire and
at the helper: no length header, `0`, `00`, an OWS-padded zero, two AGREEING
zeroes, and `identity` in any ASCII case still keep the pooled connection alive.
|
||
|
Rodrigo Gomes da Silva
|
79f5b25f5f |
fix(http): treat a blank framing value as a declared body, not an absent one
`Content-Length:` with nothing after the colon reads back as '' from
`Message.get_all()`, and `_declared_content_lengths()` SKIPPED the empty value.
The result was "no length declared", so `request_declares_body()` reported no
pending body and the sidecar provenance rejection answered 403 with keep-alive
intact -- leaving the payload in rfile for the server to parse as the next
request line. Reproduced live, pipelined down one socket:
GET /api/extensions/probe/sidecar/ping `Content-Length:` {"stale": true}
-> HTTP/1.1 403 Forbidden (no Connection: close)
-> HTTP/1.1 400 Bad request syntax
('{"stale": true}GET /api/health/agent HTTP/1.1')
A blank value cannot prove a body's absence, so it now joins `banana` and two
disagreeing lengths as unreadable framing: bytes are assumed pending and the
connection closes. Absence is something only the wire can say, and an absent
header already yields no values at all.
Three adjacent holes of the same class were found by probing and are fixed by
the same rule (each reproduced live with the same bad-syntax 400):
* whitespace-only `Content-Length: ` -- `Message` collapses it to the same '',
as it does a duplicated blank or a blank paired with a real length.
* the four multipart upload handlers: a blank length slipped past
`unreadable_content_length()`, reached `int('' or 0)` == 0, and answered 400
"No file field in request" with the payload still queued. Now 400 "Invalid
Content-Length" with the connection closed.
* blank `Transfer-Encoding:` -- '' sat in `_UNFRAMED_TRANSFER_CODINGS` beside
`identity`, so a header naming no coding read as "nothing to frame". A blank
coding is undecodable, not unframed; uploads answer 411 and close.
The no-body contract is unchanged and pinned in both directions: no length
header, a padded zero, two AGREEING zeroes and `identity` all still keep the
pooled connection alive.
|
||
|
Rodrigo Gomes da Silva
|
fb7664bc40 |
fix(http): read framing from every header value, not just the first
Adversarially probing the re-gated head found the same poisoning reachable by
DUPLICATING a framing header instead of omitting one. `Message.get()` returns
only the first occurrence, so `Content-Length: 0` sent ahead of the real length
made the body invisible to every reader: nothing was drained and the real bytes
were parsed as the next request line. Reproduced live on the otherwise-fixed
head, over one pooled socket:
HTTP/1.1 403 Forbidden <- sidecar provenance reject
HTTP/1.1 400 Bad request syntax ('{"stale": true}GET /api/health/agent ...')
HTTP/1.1 400 Bad Request <- /api/upload reject
HTTP/1.1 400 Bad request syntax ('--x')
api/helpers.py now reads framing through `_framing_header_values()`, which
returns every value the wire carried. `request_declares_body()` treats
disagreeing or unparseable lengths as a declared body (they cannot prove the
body's absence), and `unsupported_transfer_encoding()` inspects every declared
coding so `identity` cannot mask a `chunked` behind it. Two agreeing zeroes are
still no body -- repetition alone is not a conflict.
New `unreadable_content_length()` gates the multipart preflight: with no honest
byte count there is nothing to drain, so all four handlers refuse with 400
"Invalid Content-Length" before touching rfile, connection armed for close.
13 regressions, each verified red without this commit: the helper predicates on
a real email.message.Message, the production sidecar 403, all four multipart
handlers, and five pipelined real-socket tests proving the follow-up request is
never answered off a poisoned socket.
|
||
|
Rodrigo Gomes da Silva
|
b3a031a19d |
fix(http): decide reject-before-read closes from the framing, not the method
Two framing gaps the re-gate reproduced on the production handlers, both the
same class: something other than the wire was used to decide whether unread
body bytes are queued.
1. api/routes.py — the sidecar provenance 403 gated its close on
`read_request_body`, a static per-method flag. That is wrong in both
directions: a GET carrying `Content-Length`/`Transfer-Encoding` was answered
with keep-alive and its body then parsed as the next request line
(`501 Unsupported method ('{}GET')`), while a body-less DELETE/PUT/PATCH had
a healthy pooled connection closed for nothing.
2. api/upload.py — all four multipart handlers read framing from
`Content-Length` alone, so a chunked upload (which has none) was treated as
length zero, matched no parts and was rejected 400 "No file field in
request" with keep-alive intact. Wire trace before the fix:
HTTP/1.1 400 Bad Request <- the upload rejection
HTTP/1.1 400 Bad request syntax ('54') <- our own chunk-size line
New primitive in api/helpers.py: `request_declares_body()` (Content-Length
present and non-zero, or any Transfer-Encoding other than identity; an
unparseable length counts as present, since it cannot prove absence) plus
`arm_connection_close_if_body_pending()` and `unsupported_transfer_encoding()`.
api/upload.py now routes all four handlers through one preflight,
`_read_multipart_or_reject()`, which refuses an undecodable transfer coding
with 411 before touching rfile — http.server never decodes chunked framing, so
that request can only be rejected — and keeps every previous status and message
(413 oversize, 400 invalid Content-Length/boundary) with its close.
Regressions: chunked reject that closes and a body-less reject that does not,
at all four upload handlers and at the sidecar 403, plus two pipelined
real-socket tests against the running server proving the follow-up request is
never answered off a poisoned socket.
|
||
|
Rodrigo Gomes da Silva
|
3c5569a30e |
fix(http): advertise Connection: close on every reject-before-read path
Deep-review gate (2026-08-14) findings, all reproduced: 1. close_connection was never serialized: BaseHTTPRequestHandler does not emit Connection: close from the flag, so pooled clients kept reusing the socket and died with BrokenPipeError. Handler.end_headers() now emits the header (deduped) whenever close_connection is armed. 2. check_auth_or_close armed the flag too late (after the 401/302 was already flushed); it now arms before check_auth() and restores the prior flag on success. 3. The sidecar provenance rejection closed body-less GETs too; the close is now gated on read_request_body=True. 4. Remaining reject-before-read sites now close-and-advertise through the same mechanism: rate-limited CSP reports (204), the deprecated /api/process-complete-ack (410), /api/health/restart (all outcomes), and the four upload handlers (413 + pre-read Content-Length/boundary validation via _reject_before_read). Tests: pooled-client regression against the production handler (reject with a body, then follow-up on the same pooled connection succeeds and the rejection carries Connection: close); unit coverage for the sidecar GET keep-alive, CSP rate-limit, health/restart and upload oversize paths. The server.py line guard is bumped to 760 with rationale: the deep-review requires the emission in Handler.end_headers(), which cannot live outside server.py. |