Rebased 363 commits forward (branch-point 2026-07-05). Applied the PR's code
diff cleanly across 16 files; resolved api/models.py additively (both master's
process_wakeup_pause/anchor_scene_index fields AND the PR's share_token/
share_created_at fields kept in the Session constructor, metadata list, and
to_dict). CI workflow (.github/workflows/tests.yml) intentionally NOT taken from
the stale branch — master owns it (the branch's apparent removal of Playwright/
Office steps was a PAT-workflow-scope revert; rebase restores master's CI).
Co-authored-by: MicoRobot. Concept approved by maintainer (opt-in public share,
2026-07-11). Note: 4 dynamic share tests fail identically on the original
unrebased head (pre-existing test-env Session.load fragility, not rebase-caused);
to be resolved during the security re-gate in a proper sandbox.
Reasoning-heavy sessions accumulate 250-480KB of anchor_activity_scenes serialized
BEFORE the messages array, so the 64KB metadata-prefix read always overflowed ->
full-parse fallback on every read -> memory growth. Move scene data AFTER messages;
metadata prefix now ~2KB, modern reads skip the full parse. Old-layout files still
load (one authoritative parse then cached); round-trip preserves scenes exactly.
Independent review: nesquena APPROVED (end-to-end, on head). Gate-pass: Codex SAFE
(933KB legacy sidecar loads clean + round-trip verified), 17 own + 2945 regression green.
Co-authored-by: t <a@b>
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
Phase 1 low-risk backend batch (on top of Batch A). Both byte-verified vs PR head,
Codex SAFE, 64 own + 749 regression tests pass, coexists with Batch A's #5833.
- #5828 (ai-ag2026): frame CORS OPTIONS preflight with Content-Length: 0 (keep-alive hang fix)
- #5834 (ai-ag2026): parse session _index.json from bytes not decoded str (all 10 sites)
Co-authored-by: t <a@b>
Co-authored-by: ai-ag2026 <ai-ag2026@users.noreply.github.com>
When cached session has no anchor_activity_scenes and the prefix reader
returns None (e.g. sidecar too large for prefix), the empty-scenes path
did not mark _scene_check_inconclusive = True. This meant an inactive
session with matching message counts would return False (cache fresh)
even though disk may have gained its first scene record.
Fix: the empty-scenes else-branch now sets _scene_check_inconclusive
when disk_meta_quick is None, matching the existing behavior in the
cached-scenes branch (line 3173). The code then falls through to the
full metadata comparison instead of serving a stale cache.
discussion_r3548650345
Two Greptile P1 comments from review of b5c0447a (tier3c):
1. Inconclusive Prefix Stays Fresh (discussion_r3548477859):
When _persisted_session_meta_prefix() returns None and cached has scene
records, the fast-path skipped scene comparison entirely and returned
False (cache fresh) for inactive equal-count sessions. Added
_scene_check_inconclusive flag that forces fall-through to the full
Session.load_metadata_only() comparison when the cheap prefix read
fails, preventing stale anchor_activity_scenes.
2. Prefix Check Misses Edits (discussion_r3548477915):
The 128-char content-prefix backstop in _read_profile_config_cached
only detected edits within the first 128 bytes of config.yaml. Replaced
with a full-content comparison that reads the entire file (~1-10KB)
and compares it to a copy stored in the cache entry. This detects
edits anywhere in the file at a fraction of the cost of yaml.safe_load().
Updated cache type hint from tuple[float, str|None, dict] to
tuple[float, str, dict].
Tests: 108 passed (41 reconciliation + 39 anchor-scene + 28 profile-config)
The fast-path scene comparison in _cached_session_lags_disk had two bugs
flagged by nesquena-hermes review:
1. Direction: used symmetric != instead of directional subset check.
When the cache had scene keys that disk lacked (cache-ahead), the old
code force-reloaded from disk, silently dropping un-persisted scene
data. Now mirrors master: reload only when disk is strictly ahead
(disk_keys.issubset(cached_keys)). The 'not disk_scenes' early-return
is also removed — empty disk scenes means disk is not ahead, so keep
the cache.
2. Key filtering mismatch: used raw set(disk_scenes.keys()) instead of
_anchor_scene_record_keys() helper which filters to truthy keys with
dict values. Now filters disk keys the same way as master, and uses
_anchor_scene_record_keys(cached) for the cached session object.
Also updated the tier3b content-prefix docstring to clarify it is a
best-effort backstop limited to the first 128 chars, not a full
integrity check.
Three unaddressed Greptile review comments fixed:
P1 - Empty scenes skip check (api/models.py):
When the cached session had no anchor_activity_scenes, the scene-comparison
branch was skipped entirely by the 'if cached_scenes:' guard. An inactive
session would return False (cache fresh) even when disk had gained its first
scene record. Add an else clause that checks disk for scene records when the
cache has none.
P2 - Draft slow log sink (api/routes.py):
The /api/session/draft handler's slow-path log (>200ms) used logger.warning,
which is silently dropped in the WebUI process (root logger has no handler).
Route through handler._safe_webui_print() instead, the same fix applied to
the /api/session slow log in tier2c.
P2 - Config cache same-timestamp stale (api/routes.py):
The profile config.yaml cache key (profile_name, mtime, size) could collide
on coarse-mtime filesystems (FAT, some network FS) when a config edit within
the same timestamp tick happened to produce the same byte count. Add st_ino
(inode) to the cache key — most editors use atomic-rename writes which assign
a new inode, so the key changes even when mtime+size are identical.
Greptile review caught that the Tier 1a fast-path returns 'cache fresh' when
disk_count == cached_count AND the session is inactive, but api/routes.py
saves the sidecar after editing only anchor_activity_scenes (no message
change). That path leaves the cached scene records stale until the cache is
otherwise invalidated.
Fix: when the cached session has scene records, also compare them against
the disk's scenes via the same metadata-prefix read that _persisted_message_count
already does. anchor_activity_scenes is in METADATA_FIELDS and serializes
before messages, so it lands in the prefix.
New _persisted_session_meta_prefix() helper exposes the parsed prefix dict;
_cached_session_lags_disk reads it and compares scene key sets + max
updated_at against the cached session.
Cost: one extra dict parse per cache hit on sessions that have scene
records, no cost on sessions without. Inactive sessions still take the
fast-path exit when scenes match.
Prior to this change _cached_session_lags_disk() called
Session.load_metadata_only(sid) on every LRU cache hit, parsing the
full sidecar JSON (~15-20ms even for a 1.3MB sidecar on Celeron+ eMMC).
For draft auto-saves hitting get_session() on every keystroke debounce
(~400ms cadence while typing), that 15-20ms multiplied out to ~75% of
the request's wall time on the Chromebook.
Reorder the check: read only the JSON metadata prefix
(_persisted_message_count, already defined for the LRU-eviction path)
to compare message counts. For INACTIVE sessions
(no active_stream_id, no pending_user_message) with matching counts,
the cache is provably at parity with disk — anchor-scene records cannot
have advanced without the message count advancing too, and the full
Session.load_metadata_only() is skipped. For ACTIVE sessions, or when
counts differ, fall through to the original anchor-scene comparison
path so correctness is preserved.
Measured cost reduction on the request hot path: 15-20ms -> <1ms on
the typical cache-hit case (inactive session, count matches). The
slow path is unchanged for cases that genuinely need the full reload.
No public API change. No behavior change observable on user-facing
flows. The LRU eviction safety invariant (#4765) is preserved —
_session_is_evictable() still calls _persisted_message_count under
the same conditions, and Tier 1a adds NO new eviction gate.
Greptile rerun and the maintainer review both raised the same concern:
state.db and the sidecar self.messages can diverge by hundreds of
messages during recovery / mid-flight writes / pending_user_message.
On the test corpus:
0db167553ac7 db=11 sidecar=12
295592fd560e db=813 sidecar=889
7478cae31f01 db=2730 sidecar=2405
The previous patch's SQL path was reading state.db. The pre-patch
inline walk was reading self.messages (the sidecar). Sidebar stale-row
detection (_looks_like_stale_zero_message_row,
_row_may_need_sidecar_metadata_refresh) consumes this field as if the
sidecar were the source of truth, so swapping the source silently flips
the field's semantics.
Drop the SQL path entirely. The helper now does the same in-memory
walk the pre-patch code did, extracted into a named method for
visibility. The inline role check (dict.get('role') with default
empty string) is 1.7x faster than calling _message_role() per
iteration on the test corpus (2.57ms -> 1.49ms across 8 sidecars,
2405 msgs total).
Bench (2405-msg sidecar): 0.76ms per call, vs ~1.5s for the pre-patch
inline walk on the same sidecar (the original perf hotspot). The
1.5s figure included JSON parse, dict construction, and the SQLite
overhead of the now-removed SQL path; the bare walk is the
0.76ms shown here.
Correctness: 27/27 webui sidecars produce identical counts vs the
pre-patch inline walk. No behavior change for any caller.
Addresses Greptile findings (P1 silent-zero, P2 docstring, P2 dead
allowlist) by removing the SQL path entirely rather than patching
around its divergent-source issue.
Greptile flagged that _compute_user_message_count_lazy silently returns 0
when the SQLite query fails (DB locked, missing file, schema drift).
The pre-patch compact() walked self.messages in Python and returned the
correct count without ever touching the DB, so the silent zero
fallback is a regression that breaks _looks_like_stale_zero_message_row
under contention.
Fix: helper now takes an optional messages argument. On SQLite failure
it falls back to the same in-memory walk the original code used, so
callers see the same number they'd have seen before for any session
where Session.load() populated self.messages.
Also drop the dead ('GET', '/api/session') allowlist entry flagged by
Greptile — the /api/session handler already does its own per-stage
logging via the _t0.._t6 block (with auto-log on slow requests), and
adding RequestDiagnostics on top would just duplicate the slow-request
journal entries without adding information.
Session.compact() called _last_message_timestamp(self.messages) which
reversed-walked all 2,730 messages on every /api/session response.
The messages array is chronologically ordered, so the last non-tool
message sits at the very end. Scan only the last 8 messages; fall
back to a full scan only if no timestamp is found in the window
(unusual sessions with >8 trailing tool rows).
Reverts to identical behavior when the window hits, since the most
recent user/assistant message's timestamp is what compact() emits.
Bench (single-shot, live Chromebook eMMC):
before: get_session=1107ms compact=1725ms total=2880ms
after: get_session= 648ms compact= 288ms total=1313ms
(-80% on compact, -54% on total)
Revert: git checkout master && systemctl --user restart hermes-webui
get_state_db_session_messages() returned archived (active=0) rows that in-place
compaction had marked inactive. WebUI reconciliation feeds this reader straight
into the next model-facing context (reconciled_state_db_messages_for_session,
prefer_context=True), so those archived rows got pulled back in — resurrecting
pre-compaction history and making every later turn re-trigger compression. Now,
WHEN the messages table exposes an `active` column, inactive rows are excluded by
default (`AND (active IS NULL OR active != 0)` — NULL/legacy rows preserved), with
an `include_inactive=True` escape hatch for explicit recovery/audit callers.
Schemas without the column are unaffected (guarded by `'active' in available`).
Contributor stage; ungated on arrival. Co-authored-by: ai-ag2026 <ai-ag2026@users.noreply.github.com>
Closes the crash-window resurrection corner in the #5532 clear-conversation
data-loss fix. /api/session/clear writes a .json.bak before truncating, and
because the cleared sidecar's truncation_watermark is 0.0, startup recovery saw
bak_count > live_count and could restore the backup — resurrecting the cleared
transcript after a crash/restart in the clear window.
Fix: stamp a unique per-clear `clear_generation` UUID on the sidecar when
clearing a session that had messages (persisted via Session model). Recovery's
_session_records_clear_sentinel() returns no_action when the live sidecar carries
a clear_generation the backup doesn't share AND the live sidecar matches the
exact cleared-empty shape (messages/context empty, watermark/boundary 0.0,
pending fields cleared). Same-generation backups stay recoverable; unreadable or
partial matches FAIL OPEN (normal recovery), so a real crash-loss is never
suppressed.
Contributor stage; gate-pass. Co-authored-by: rodboev <rodboev@users.noreply.github.com>
Restore the compact() source-order expectation used by the branch metadata contract test while keeping focused compression recovery markers in the compact payload.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Make focused compression-recovery continuation creation idempotent by marking child sessions and reusing the existing child on repeated start requests. Also surface malformed successful recovery responses in the UI instead of silently re-enabling the action.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
@rodboev independently pushed fixes for messaging-tombstone (matches ours) and an
audit orphan-.bak skip — but the audit skip used _marks_deleted_webui_session
(index heuristic), which re-introduces the crash-suppression bug, and the other 4
resurrection/recovery paths were still open. Rebuilt the stage on rodboev's new
head (fc21c3f3, attribution preserved) and applied the complete superset:
- routes.py claim path: tombstone 404 gated on _state_db_session_source in
('','webui','fork') so foreign-source rows always materialize (finding B)
- models.py sidebar projection: skip tombstoned source='webui' rows w/ no live
sidecar so a deleted session can't resurface in /api/sessions (finding C)
- session_recovery.py orphan-.bak startup: skip restore when durable tombstone
present, durable-only (finding D)
- session_recovery.py audit orphan-.bak: durable-only classify as
deleted_session_skipped (corrects rodboev's index-heuristic version)
- session_recovery.py _read_state_db_missing_sidecar_rows: durable-tombstone-only
so a genuine crash (index intact, no durable tombstone) still repairs (finding F)
Regression tests (all verified fail-without-fix): sidebar-projection, orphan-.bak
startup skip, messaging-not-tombstoned, index-only-crash-repairable. Both advisors
(Codex SAFE TO SHIP + Opus ship-with-fixes) cleared the prior stage; this rebuild
carries the same fixes onto the contributor's latest work.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
_session_message_content_key (the state.db reconciliation key in
api/models.py) normalized whitespace only, while the streaming-side
identity _message_identity strips the workspace prefix for user turns.
WebUI sends the model a workspace-prefixed user_message
([Workspace::v1: /path]\n<text>) while the visible/optimistic bubble and
sidecar row carry the bare <text>. The mismatch made a prefixed state.db
row and a bare sidecar row key differently, so state_db_delta_after_context
failed to align them, treated the state.db copy as new, and appended a
duplicate user turn. The agent-side merge then concatenated the two
adjacent user rows into a permanent composite -- the post-restart
stale-user-prepend bug.
Fix: strip the workspace prefix for role=='user' in the reconciliation
key, reusing the same _strip_workspace_prefix helper the streaming side
uses (lazy import to avoid the api.streaming -> api.models cycle) so the
two dedup layers can't drift again. Assistant/tool keys are unchanged and
prefix-free user messages key identically (idempotent).
Fixes#5339.
Root cause of the silent-crash-after-hours cluster (#4765/#2233/#4633): the
global in-memory SESSIONS LRU evicted with a blind popitem(last=False), which
could drop an actively streaming or not-yet-persisted session (data loss) and
was capped only via an env var. On long-running installs the effective result
was unbounded RAM growth until segfault.
- Add _session_is_evictable(): a session is evictable ONLY when it is not
streaming (no active_stream_id), has no in-flight turn (no pending_user_message
/ pending_started_at), and its full state is proven on disk (sidecar
message_count >= in-memory count; metadata-only stubs and zero-message shells
are trivially safe).
- Add _evict_sessions_over_cap(): replaces all 8 blind popitem loops across
models.py, routes.py, streaming.py. Walks the LRU oldest-first and removes only
provably-safe entries; never acquires LOCK/stream locks itself (caller holds
LOCK) so no lock-ordering deadlock. May briefly exceed the cap rather than ever
evict an active/unsaved session.
- Make the cap configurable via config.yaml webui.sessions_cache_max
(get_sessions_cache_max()); precedence config.yaml -> HERMES_WEBUI_SESSIONS_MAX
(legacy) -> DEFAULT_SESSIONS_CACHE_MAX=300. No new HERMES_* env var. Invalid or
<1 values fall back so a typo can never disable the bound.
- Evicted sessions lazily reload from their JSON sidecar via the existing
get_session() accessor; no call sites changed. _index.json sidebar behavior
unchanged (the sidebar reads the index, not SESSIONS).
- Add tests/test_issue4765_sessions_lru_eviction.py (8 tests): eviction past
cap, active/streaming never evicted, unsaved/stale-tail never evicted, lazy
reload with identical content, and no-data-loss under heavy churn.
- README: document the config.yaml key + safety semantics.
Fixes#4765.
Regression seam behind #5308: a delegated subagent child's sidebar row is built
from a stale sidecar that reports message_count==0, and the state.db count
overlay in _apply_sidebar_state_db_override_metadata was gated on
`state_db_source == 'webui'`. A subagent child (state_db_source=='subagent')
therefore never received its true message count, so the front-end visibility
predicate (_sidebarRowHasVisibleMessages) dropped the row and the subagent
session disappeared entirely (not nested, not orphaned) after #5244+#5306.
Fix: widen the count/last-message overlay to `state_db_source in ('webui','subagent')`,
keeping the same conservative anti-resurrection guard. The source-tag/title
reassignment stays WebUI-only so a subagent child keeps its subagent
classification. Same state.db-blind-metadata root as the #5307 transcript
recovery, fixed server-side rather than by loosening the front-end predicate
(which would fight the #5306 active-parent scoping).
Tests: subagent child gets its count overlaid + classification preserved; a
non-webui/non-subagent foreign source (cron) still gets NO overlay.
Fixes#5308.
Windows locks session JSON files briefly during antivirus scans or
browser polling, causing os.replace() to raise WinError 5
(ERROR_ACCESS_DENIED). Add _safe_replace() helper that retries with
exponential backoff (50ms to 800ms, max 5 attempts) on PermissionError.
No-op on non-Windows platforms.
Replaces all 5 os.replace() call sites in models.py:
- Session.save() (main file + backup)
- _write_session_index() (full rebuild + fast path)
- prune_session_from_index()
nesquena-hermes