Release v0.51.256 — Release HX (stage-r4) (#3596)

## Release v0.51.256 — Release HX (stage-r4)

Performance — bound WebUI memory growth & idle CPU on large installs.

### Fixed
| Issue | Author | Fix |
|-------|--------|-----|
| #3506 | @nesquena-hermes (reported w/ profiling by @djenttleman) | On a large install (~615 sessions / 40k messages / 454 MB state.db) the WebUI process climbed ~100 MB → ~1.5 GB RSS over days and held high idle CPU. Three root causes fixed: (1) `session_lifecycle._sessions` grew unbounded → new `discard_session()` drops the entry at agent-eviction boundaries, only when no in-flight commit / no uncommitted memory work (retry invariant preserved); (2) cache caps now operator-tunable (`HERMES_WEBUI_AGENT_CACHE_MAX` default 50→25, `HERMES_WEBUI_SESSIONS_MAX`); (3) GatewayWatcher computes a cheap fingerprint before the expensive per-session `MAX(messages.timestamp)` projection and only re-projects on change. |

### Rebase + review notes
- Rebased onto current master; the code diff was verified **byte-identical to the nesquena-APPROVED head** at rebase time (only CHANGELOG re-resolved).
- The Codex regression gate then surfaced **two correctness gaps** the approval didn't catch, both fixed here with regression tests:
  1. **Watcher fingerprint missed same-count transcript rewrites.** `/retry`,`/undo`,`/compress` (`SessionDB.replace_messages`) rewrite messages with new timestamps but can leave `message_count` unchanged → stale sidebar `last_activity`. Fixed with a **per-session** grouped message aggregate (`id, count, user_count, MAX(timestamp)`) over the same non-excluded sessions (a global MAX would miss a rewrite of an older, non-newest session); cron/webui stay excluded so idle churn still doesn't re-project.
  2. **LRU agent-cache eviction could close a live worker's agent** (`popitem(last=False)`, liveness-blind — pre-existing, but the lower 50→25 cap made it more likely). Eviction now snapshots `ACTIVE_RUNS` session_ids (before the cache lock — no nested lock) and skips live sessions, deferring (temporarily exceeding cap) rather than closing a live agent.

### Gate
- Full pytest suite: **7612 passed, 0 failed** (one boot-cascade flake re-run; clean on re-run)
- ruff: CLEAN · Codex (regression): 4 rounds → both gaps + a stale test fixed → **SAFE TO SHIP**

Co-authored-by: nesquena <nesquena@users.noreply.github.com>

api/session_lifecycle.py

@@ -92,6 +92,39 @@ def unregister_agent(session_id: str) -> None:
         _condition.notify_all()
 
 
+def discard_session(session_id: str) -> bool:
+    """Permanently drop a session's lifecycle entry to bound memory growth.
+
+    The ``_sessions`` dict is process-global and historically only ever grew:
+    ``register_agent`` / ``mark_turn_completed`` insert keys but no runtime path
+    ever removed them, so every unique ``session_id`` the WebUI touched leaked a
+    permanent entry (issue #3506). Over days of use on a large install this is a
+    monotonic, unbounded climb.
+
+    This removes the entry, but only when it is provably safe to do so: no commit
+    is in flight and there is no uncommitted memory work that still needs the
+    retained agent handle. If the entry is busy or dirty it is left untouched so
+    failed batch-extraction memory work stays retryable -- exactly the invariant
+    ``unregister_agent`` and ``_evict_session_agent`` already preserve.
+
+    Returns True when the entry was removed (or was already absent), False when
+    it was retained because work is still pending.
+    """
+    if not session_id:
+        return False
+    with _condition:
+        entry = _sessions.get(session_id)
+        if entry is None:
+            return True
+        if entry["in_flight"]:
+            return False
+        if entry["generation"] > entry["committed_generation"]:
+            return False
+        del _sessions[session_id]
+        _condition.notify_all()
+        return True
+
+
 def mark_turn_completed(session_id: str, *, agent=None) -> int:
     if not session_id:
         return 0