The rebase onto e88aff438 put upstream's subagent-parent recovery walk
(8686528b1, supersedes #7031) and this PR's CANDIDATE_WINDOW_MULTIPLIERS
re-widening loop in the same tail of read_importable_agent_session_rows.
The walk anchors a frozen orchestrator by re-reading the projection of the
candidate window, so it has to see the window that ACTUALLY executed, not
the first pass.
Nothing pinned that. Feeding the walk a stale first-window projection keeps
every test in tests/test_subagent_parent_in_import_window.py green, because
those dbs never trigger the second window.
Add the missing case: two 12-segment compression lineages own the 24 newest
raw rows and collapse to 2 logical conversations, so a limit=3 request
consumes the 8x window and under-delivers; the hot subagent leaf and its
quiet parent are only reachable once the window re-widens. Plus a negative
control that pins the 8x window really does stop at the two lineages, so the
recovery is attributable to the widened window.
Verified with teeth: removing the walk fails the new test and 4 upstream
ones; feeding the walk the stale first window fails the new test alone.
read_importable_agent_session_rows()'s `limit` is a bound on LOGICAL
conversations (the slice runs after lineage projection), but the bounded
candidate CTE spends its raw window on compression segments and rows the
post-projection visibility filters drop. A lineage-heavy profile therefore got
back far fewer conversations than it asked for, and silently: with twelve
ten-segment chains, limit=20 returned 8.
Retry once with a wider window when the first one was fully consumed AND still
under-delivered. The common case is unchanged - the first window is still 8x the
requested limit and the retry never runs when the db simply has nothing more to
give - so the #2628 capped-scan guarantee holds.
Replace the remaining source-text assertions in test_issue2628 (the Greptile
P2 thread): the pre-aggregation shape, index detection, correlated seek and
pure-read contracts are already pinned behaviourally by EXPLAIN QUERY PLAN,
result equivalence and PRAGMA index_list in test_pr7445_readonly_review_followups.py,
test_state_db_readonly_readers.py and test_issue3887_index_prime.py.
The one contract only source text covered — the SQL candidate window is
capped and oversampled (limit * 8) so post-projection filtering cannot
starve the visible page — is now asserted behaviourally instead:
* the traced statement SQLite actually runs carries a single LIMIT whose
bound is the oversampled cap (limit * 8 = 160), applied inside the
candidate subquery the outer projection joins;
* a page still fills to limit when the 30 newest candidates are invisible
CLI rows (ended, default title): they consume candidates, not result
slots. Verified to fail (0 rows) against the old non-oversampled shape.
On native Windows the text-mode ``handle.write("\n")`` that seeds an empty
``--lock-file`` is translated to ``\r\n``, so a fresh lock file is two bytes
and the msvcrt regression test fails on the platform it protects. Seed the
file with ``os.write(fd, b"\0")`` on the descriptor instead: exactly one byte
whatever the text-mode newline translation. The Windows-branch tests now run
under a ``\n`` -> ``\r\n`` translating ``open`` so this failure reproduces
on any host.
Remove the two ``... not in src`` assertions added for the previous review
from test_issue2628: the behaviours they hinted at (every missing-index
projection uses the pre-aggregated ordering; listing never creates an
index) are already asserted through query plans, result equivalence and
``PRAGMA index_list`` in test_pr7445_readonly_review_followups.py,
test_state_db_readonly_readers.py and test_issue3887_index_prime.py.
Review follow-ups on #7445 (nesquena-hermes CHANGES_REQUESTED + Greptile).
api/agent_sessions.py
- The read-only pre-aggregated `latest_messages` candidate ordering now
applies to every missing-index projection (default sidebar, gateway
watcher, cron/webhook/kanban views), not only `include_sources ==
("cron",)`. Since the listing no longer self-heals the index, the
correlated `MAX(mx.timestamp)` scan re-ran on every load: 9.86 s on a
5,000-session / 20,000-message DB vs 0.044 s after this change, with
identical selected rows and order. The indexed seek is unchanged.
- `state_db_file_uri()` / `state_db_readonly_uri()` replace
`Path.as_uri()`: a UNC `HERMES_HOME` produced `file://server/share/...`
and CPython's bundled SQLite (3.11-3.13 Windows builds, no
SQLITE_ALLOW_URI_AUTHORITY) rejected the authority, so every agent-backed
session vanished. The builder emits the empty-authority
`file:////server/share/state.db?mode=ro`, keeps `file:///C:/...` for
drive letters, strips `\\?\` / `\\?\UNC\` prefixes, quotes `?`/`#`, and
leaves POSIX output byte-identical. Strict `mode=ro`, no writable
fallback.
scripts/ensure_state_db_read_indexes.py
- `fcntl` / `msvcrt` are imported conditionally; the module imports and
the no-`--lock-file` path runs on native Windows. `--lock-file` uses
`flock` on POSIX, `msvcrt.locking(LK_NBLCK)` on Windows, and fails with
a clear RuntimeError when neither exists. The rw open reuses the
UNC-aware URI builder.
docs/troubleshooting.md
- Operator section: read-only contract, degraded missing-index listing,
drained maintenance prerequisites, lock behaviour, failure modes, UNC.
Tests: tests/test_pr7445_readonly_review_followups.py (19 tests, red
before this change: correlated-subquery plan, missing URI builder,
ImportError on fcntl). Query-plan contract via EXPLAIN QUERY PLAN — no
CORRELATED SCALAR SUBQUERY on missing-index listings, identical rows to
the indexed listing; platform-parametrised URI cases; maintenance module
imported with `sys.modules["fcntl"] = None`. test_issue2628 no longer
pins the cron-only gate; the flock-held test skips without fcntl.
Readers never upgrade to writers. open_state_db_readonly() opens
file:...?mode=ro only; a read-only open failure propagates instead of
silently retrying with a write-capable handle on the multi-GiB WAL
database the agent is streaming into. The session listing no longer
self-heals idx_messages_session with CREATE INDEX (a writer lock held for
minutes on a large messages table) — a missing index degrades to the
pre-aggregated path. Cron sidebar info, insights and deep health checks in
api/routes.py use the same read-only opener.
Watcher (api/gateway_watcher.py): _poll_once returns without touching the
database when nobody is subscribed, and forces a fresh full projection on
the next subscription.
Maintenance (scripts/ensure_state_db_read_indexes.py): the covering read
indexes (idx_messages_session, idx_messages_session_role with NOCASE,
idx_sessions_webui_fingerprint) are created only under --confirm-drained,
optionally holding an exclusive flock on --lock-file for deployments that
serialise agent turns on a lock file; existing indexes are verified for
shape/collation and coverage (EXPLAIN QUERY PLAN), never silently
replaced.
Tests: tests/test_state_db_readonly_readers.py (read-only failure never
retries writable; no poll without subscribers; listing creates no index;
index tool requires drain, is idempotent, works without a lock file and
refuses a held lock; cron sidebar read is mode=ro). Neighbouring tests
that asserted the retired writable fallback / self-heal writer
(test_issue3887_index_prime, test_issue5455_listing_readonly_connection,
test_issue5455_lineage_readonly_reads, test_issue2628_cli_sessions_perf,
test_issue1494_state_db_fd_leak) now assert the read-only contract;
test_issue3506 subscribes before polling.
Rodrigo Gomes da Silva