Gate found two blocking defects in the new caches:
1. The large-string LRU was keyed on input text only. The installed
agent.redact registry can extend the secret matcher at runtime via
register_redaction_patterns() and exposes no policy generation or
invalidation hook, so a >16k string primed before a registration was
served with the new secret intact (reproduced against the installed
registry). Drop the large-string cache: strings above 16,384 chars run
the redactor uncached again. Regression primes a large string, registers
a runtime pattern on the real registry and proves the next
redact_session_data() response is redacted.
2. The prefilter LRU was count-bounded (8192) but its per-entry gate was a
character cap, which is not a byte bound: 16k four-byte code points is
~64 KiB per key, ~512 MiB at capacity, and the workload died with
MemoryError under 96 MiB of headroom. Gate each entry on
sys.getsizeof(text) <= 2048 so the retained-key ceiling is a hard
16 MiB. Regressions cover max-width Unicode, sensitive verdict pairs,
eviction past the cap, and a low-memory RLIMIT_AS subprocess.
`_redact_fn_cached` skipped memoization for any string over 16KB, so those
strings re-ran the full ~15-pass redactor on every single request. On a real
22MB session that is 59 large strings (29 unique, 0.76MB) costing 1.68s per
request — 99.9% of the recurring redaction cost once the small-string cache is
warm — repaid by every tab and every poll for a byte-identical result.
Large strings now get their own bounded cache, separate from the 4096-entry
small-string one so a large blob can never evict the thousands of small
recurring strings. Entry count and per-entry size are both capped; anything
above the ceiling stays uncached, keeping retained bytes bounded.
`_redact_value` additionally stops rebuilding containers whose descendants
were unchanged. Most of a transcript carries no credential marker, so the
unconditional dict/list comprehension deep-copied the whole payload to
reproduce an identical structure — pure CPU under the GIL, which serializes
concurrent tab loads on a threaded server. Callers treat redacted output as
read-only (`_public_message_projection` and `redact_session_data` both build
fresh containers), so sharing an unmodified subtree cannot leak a later
mutation back into session state. As soon as one string is masked, every
container on the path to it is rebuilt and the caller's original is untouched.
Measured on the live 22.6MB session, steady state: redact_session_data 3.78s
-> 1.96s single tab (-48%), 7.93s -> 4.06s with two concurrent tabs (-49%),
byte-identical output.
Anthony Ruiz