fix(models): cap _CLI_SESSIONS_CACHE with LRU eviction (drop-oldest)
Memory: the CLI/cron session projection cache was a plain dict with TTL storage but NO size cap and only lazy, same-key eviction. Each value is a copy.deepcopy() of the full CLI/cron session list (the expensive projection behind #4842/#4672). The cache key folds in a state.db content fingerprint that advances on every streamed message, so the next poll builds a NEW key; the previous key's heavy deepcopy was never looked up again, so its lazy-expiry path never ran. A long-lived process under churn could accumulate orphaned heavy deepcopies until the next structural clear_cli_sessions_cache(). Make it an OrderedDict capped at _CLI_SESSIONS_CACHE_MAX_ENTRIES (8) with drop-oldest on write (while len > MAX: popitem(last=False)) and move_to_end on fresh hits, mirroring the _CLAUDE_CODE_PARSE_CACHE / _SIDECAR_METADATA_CACHE LRU pattern in the same file. TTL remains the primary freshness control; the cap is the backstop the plain dict lacked. clear_cli_sessions_cache() keeps .clear(). Test: tests/test_cli_sessions_cache_cap.py — cache stays at len == MAX after MAX+5 distinct keys (oldest 5 evicted), a fresh hit marks the entry most-recently-used so it survives over colder entries, and the cache is an OrderedDict (the regression: it was an unbounded plain dict). Existing streaming-cache-TTL/cron-session tests unchanged. Contract Routing - Task type: memory-bound fix (unbounded projection cache). - Touched areas: CLI/cron sidebar projection cache (api/models.py _CLI_SESSIONS_CACHE). - Relevant public docs: AGENTS.md, CONTRIBUTING.md, docs/CONTRACTS.md, docs/rfcs/webui-run-state-consistency-contract.md (Sidebar/session metadata layer). - State layer mutated: Sidebar/session metadata layer — specifically the in-memory projection cache that backs the CLI/cron sidebar rows. The cache is a read-derivation of state.db content; it is NOT a source of truth. Dropped entries are rebuilt on the next poll (TTL/invalidation already drive rebuilds). - Invariant preserved — Invariant 4 (Maintenance is not activity): the cache is a derived projection; evicting an entry cannot move a session in the sidebar or change a row's content, because the next poll recomputes the projection from state.db. TTL remains the primary freshness control; the LRU cap only bounds how many STALE orphaned deepcopies linger between invalidations. Regression: test_cli_sessions_cache_cap.py plus the existing streaming-cache-TTL and cron-session tests (which exercise the real invalidate/rebuild path). - Scope boundaries: does not change sidebar ordering, unread state, updated_at, or the projection result. The cap is a memory backstop; correctness still flows from the existing TTL + clear-on-mutation invalidation. Release note: the CLI/cron sidebar projection cache is now LRU-bounded, capping worst-case memory for long-lived processes under heavy streaming churn. Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent). No AI-specific tool use mattered beyond the repo's standard read/edit/test workflow.
This commit is contained in:
1 parent
6ed474ca54
commit
015171eba3
2 files changed
+106
-1
No files matched your search
+17
-1
@@ -63,7 +63,16 @@ _CLI_SESSIONS_CACHE_STREAMING_TTL_SECONDS = 45.0
|
||||
_CLI_SESSIONS_CACHE_LOCK = threading.Lock()
|
||||
_CLI_SESSIONS_CACHE_INFLIGHT: "dict[tuple, threading.Event]" = {}
|
||||
_CLI_SESSIONS_CACHE_INVALIDATION_VERSION = 0
|
||||
_CLI_SESSIONS_CACHE = {}
|
||||
# LRU-bounded (drop-oldest) so a long-lived process under churn — where the
|
||||
# state.db fingerprint advances on every streamed message and the structural
|
||||
# clear-on-mutation listener doesn't fire for every fingerprint advance — can't
|
||||
# accumulate orphaned heavy deepcopies. Each value is a copy.deepcopy() of the
|
||||
# full CLI/cron session list (the expensive projection behind #4842/#4672), so
|
||||
# the cap is deliberately small. TTL is still the primary freshness control;
|
||||
# the cap is the backstop that the plain dict previously lacked. Mirrors the
|
||||
# _CLAUDE_CODE_PARSE_CACHE / _SIDECAR_METADATA_CACHE LRU pattern.
|
||||
_CLI_SESSIONS_CACHE: "collections.OrderedDict[tuple, tuple]" = collections.OrderedDict()
|
||||
_CLI_SESSIONS_CACHE_MAX_ENTRIES = 8
|
||||
_CLI_SESSIONS_CACHE_WAIT_SECONDS = 0.25
|
||||
# Event waits that keep stale rows visible while a rebuild is in flight.
|
||||
_CLI_SESSIONS_CACHE_STALE_WAIT_SECONDS = 0.10
|
||||
@@ -6405,6 +6414,9 @@ def _cache_cli_sessions_if_current(
|
||||
invalidation_stamp,
|
||||
_copy_cli_sessions(sessions),
|
||||
)
|
||||
_CLI_SESSIONS_CACHE.move_to_end(cache_key)
|
||||
while len(_CLI_SESSIONS_CACHE) > _CLI_SESSIONS_CACHE_MAX_ENTRIES:
|
||||
_CLI_SESSIONS_CACHE.popitem(last=False)
|
||||
return True
|
||||
|
||||
|
||||
@@ -6423,6 +6435,8 @@ def _copy_fresh_cli_sessions_cache_entry(cache_key: tuple):
|
||||
return None
|
||||
if cached_expires_at <= time.monotonic():
|
||||
return None
|
||||
# LRU: a fresh hit is the most-recently-used entry.
|
||||
_CLI_SESSIONS_CACHE.move_to_end(cache_key)
|
||||
return _copy_cli_sessions(cached_sessions)
|
||||
|
||||
|
||||
@@ -7272,6 +7286,8 @@ def get_cli_sessions(
|
||||
if cached_stamp != _CLI_SESSIONS_CACHE_INVALIDATION_VERSION:
|
||||
_CLI_SESSIONS_CACHE.pop(cache_key, None)
|
||||
elif cached_expires_at > now:
|
||||
# LRU: a fresh hit is the most-recently-used entry.
|
||||
_CLI_SESSIONS_CACHE.move_to_end(cache_key)
|
||||
return _copy_cli_sessions(cached_sessions)
|
||||
else:
|
||||
stale_sessions = _copy_cli_sessions(cached_sessions)
|
||||
|
||||
Reference in new issue
Block a user