fix(redaction): memoize deterministic API response redactor

The combined credential redactor (~15 regex passes per string) runs on the
full message history of every /api/session response. Under concurrent
dashboard polling of unchanged sessions this became the dominant CPU cost
and, behind the GIL in the single-process server, serialized requests to
10-44s each — surfacing as repeated 'connection lost' in the browser as
the proxy/browser timed out and the frontend retried, piling on threads.

The redactor is pure and deterministic (force=True, fixed masking), so
identical strings always map to identical output and are safe to memoize
without invalidation. Wrap it in an lru_cache(maxsize=4096), gated to
strings <=16KB so a few giant tool-output dumps can't evict the many small
recurring strings or balloon RSS. Repeated polls become O(1) cache hits
(measured 3.7ms -> 0.002ms per 3KB marker string).

_redact_fn_cached is preserved as the call-site name so the existing
monkeypatch-based redaction tests bypass the cache unchanged (69/69 pass).
This commit is contained in:
AnhNVT authored and nesquena-hermes committed 2026-06-29 18:22:18 +00:00
1 parent 791744ac29
commit 9ad1fa0372
1 file changed
+20 -1
+20 -1
View File
@@ -1,6 +1,7 @@
"""
Hermes Web UI -- HTTP helper functions.
"""
import functools
import json as _json
import logging
import os
@@ -416,7 +417,25 @@ def _build_redact_fn():
return _combined_redact
_redact_fn_cached = _build_redact_fn()
_redact_fn_uncached = _build_redact_fn()
# Repeated dashboard polls re-request the same unchanged session payloads, so
# the combined redactor (~15 regex passes per string) was the dominant CPU cost
# under concurrent polling — enough to wedge the single-process server behind
# the GIL and surface as "Mất kết nối" in the browser. The redactor is pure and
# deterministic (force=True, fixed masking), so identical strings always map to
# identical output and are safe to memoize without invalidation.
_redact_fn_lru = functools.lru_cache(maxsize=4096)(_redact_fn_uncached)
# Cap per-entry size so a handful of giant tool-output dumps can't evict the
# thousands of small recurring strings that actually benefit, or balloon RSS.
_REDACT_CACHE_MAX_TEXT_LEN = 16384
def _redact_fn_cached(text):
if len(text) > _REDACT_CACHE_MAX_TEXT_LEN:
return _redact_fn_uncached(text)
return _redact_fn_lru(text)
_SENSITIVE_CASE_MARKERS = (