fix(models): cap _CLI_SESSIONS_CACHE with LRU eviction (drop-oldest)

Memory: the CLI/cron session projection cache was a plain dict with TTL storage
but NO size cap and only lazy, same-key eviction. Each value is a copy.deepcopy()
of the full CLI/cron session list (the expensive projection behind #4842/#4672).
The cache key folds in a state.db content fingerprint that advances on every
streamed message, so the next poll builds a NEW key; the previous key's heavy
deepcopy was never looked up again, so its lazy-expiry path never ran. A
long-lived process under churn could accumulate orphaned heavy deepcopies until
the next structural clear_cli_sessions_cache().

Make it an OrderedDict capped at _CLI_SESSIONS_CACHE_MAX_ENTRIES (8) with
drop-oldest on write (while len > MAX: popitem(last=False)) and move_to_end on
fresh hits, mirroring the _CLAUDE_CODE_PARSE_CACHE / _SIDECAR_METADATA_CACHE LRU
pattern in the same file. TTL remains the primary freshness control; the cap is
the backstop the plain dict lacked. clear_cli_sessions_cache() keeps .clear().

Test: tests/test_cli_sessions_cache_cap.py — cache stays at len == MAX after
MAX+5 distinct keys (oldest 5 evicted), a fresh hit marks the entry
most-recently-used so it survives over colder entries, and the cache is an
OrderedDict (the regression: it was an unbounded plain dict). Existing
streaming-cache-TTL/cron-session tests unchanged.

Contract Routing
- Task type: memory-bound fix (unbounded projection cache).
- Touched areas: CLI/cron sidebar projection cache (api/models.py
  _CLI_SESSIONS_CACHE).
- Relevant public docs: AGENTS.md, CONTRIBUTING.md, docs/CONTRACTS.md,
  docs/rfcs/webui-run-state-consistency-contract.md (Sidebar/session metadata
  layer).
- State layer mutated: Sidebar/session metadata layer — specifically the
  in-memory projection cache that backs the CLI/cron sidebar rows. The cache is
  a read-derivation of state.db content; it is NOT a source of truth. Dropped
  entries are rebuilt on the next poll (TTL/invalidation already drive rebuilds).
- Invariant preserved — Invariant 4 (Maintenance is not activity): the cache is a
  derived projection; evicting an entry cannot move a session in the sidebar or
  change a row's content, because the next poll recomputes the projection from
  state.db. TTL remains the primary freshness control; the LRU cap only bounds
  how many STALE orphaned deepcopies linger between invalidations. Regression:
  test_cli_sessions_cache_cap.py plus the existing streaming-cache-TTL and
  cron-session tests (which exercise the real invalidate/rebuild path).
- Scope boundaries: does not change sidebar ordering, unread state, updated_at,
  or the projection result. The cap is a memory backstop; correctness still
  flows from the existing TTL + clear-on-mutation invalidation.

Release note: the CLI/cron sidebar projection cache is now LRU-bounded, capping
worst-case memory for long-lived processes under heavy streaming churn.

Model Used: builtin:zai-coding-plan/GLM-5.2 (ZCode agent). No AI-specific tool
use mattered beyond the repo's standard read/edit/test workflow.
This commit is contained in:
Ruby Hartono committed 2026-07-16 19:31:47 +07:00
1 parent 6ed474ca54
commit 015171eba3
2 files changed
+106 -1

No files matched your search

+17 -1
View File
@@ -63,7 +63,16 @@ _CLI_SESSIONS_CACHE_STREAMING_TTL_SECONDS = 45.0
_CLI_SESSIONS_CACHE_LOCK = threading.Lock()
_CLI_SESSIONS_CACHE_INFLIGHT: "dict[tuple, threading.Event]" = {}
_CLI_SESSIONS_CACHE_INVALIDATION_VERSION = 0
_CLI_SESSIONS_CACHE = {}
# LRU-bounded (drop-oldest) so a long-lived process under churn — where the
# state.db fingerprint advances on every streamed message and the structural
# clear-on-mutation listener doesn't fire for every fingerprint advance — can't
# accumulate orphaned heavy deepcopies. Each value is a copy.deepcopy() of the
# full CLI/cron session list (the expensive projection behind #4842/#4672), so
# the cap is deliberately small. TTL is still the primary freshness control;
# the cap is the backstop that the plain dict previously lacked. Mirrors the
# _CLAUDE_CODE_PARSE_CACHE / _SIDECAR_METADATA_CACHE LRU pattern.
_CLI_SESSIONS_CACHE: "collections.OrderedDict[tuple, tuple]" = collections.OrderedDict()
_CLI_SESSIONS_CACHE_MAX_ENTRIES = 8
_CLI_SESSIONS_CACHE_WAIT_SECONDS = 0.25
# Event waits that keep stale rows visible while a rebuild is in flight.
_CLI_SESSIONS_CACHE_STALE_WAIT_SECONDS = 0.10
@@ -6405,6 +6414,9 @@ def _cache_cli_sessions_if_current(
invalidation_stamp,
_copy_cli_sessions(sessions),
)
_CLI_SESSIONS_CACHE.move_to_end(cache_key)
while len(_CLI_SESSIONS_CACHE) > _CLI_SESSIONS_CACHE_MAX_ENTRIES:
_CLI_SESSIONS_CACHE.popitem(last=False)
return True
@@ -6423,6 +6435,8 @@ def _copy_fresh_cli_sessions_cache_entry(cache_key: tuple):
return None
if cached_expires_at <= time.monotonic():
return None
# LRU: a fresh hit is the most-recently-used entry.
_CLI_SESSIONS_CACHE.move_to_end(cache_key)
return _copy_cli_sessions(cached_sessions)
@@ -7272,6 +7286,8 @@ def get_cli_sessions(
if cached_stamp != _CLI_SESSIONS_CACHE_INVALIDATION_VERSION:
_CLI_SESSIONS_CACHE.pop(cache_key, None)
elif cached_expires_at > now:
# LRU: a fresh hit is the most-recently-used entry.
_CLI_SESSIONS_CACHE.move_to_end(cache_key)
return _copy_cli_sessions(cached_sessions)
else:
stale_sessions = _copy_cli_sessions(cached_sessions)