fix(sqlite): keep state.db readers strictly read-only and stop idle polling

Readers never upgrade to writers. open_state_db_readonly() opens
file:...?mode=ro only; a read-only open failure propagates instead of
silently retrying with a write-capable handle on the multi-GiB WAL
database the agent is streaming into. The session listing no longer
self-heals idx_messages_session with CREATE INDEX (a writer lock held for
minutes on a large messages table) — a missing index degrades to the
pre-aggregated path. Cron sidebar info, insights and deep health checks in
api/routes.py use the same read-only opener.

Watcher (api/gateway_watcher.py): _poll_once returns without touching the
database when nobody is subscribed, and forces a fresh full projection on
the next subscription.

Maintenance (scripts/ensure_state_db_read_indexes.py): the covering read
indexes (idx_messages_session, idx_messages_session_role with NOCASE,
idx_sessions_webui_fingerprint) are created only under --confirm-drained,
optionally holding an exclusive flock on --lock-file for deployments that
serialise agent turns on a lock file; existing indexes are verified for
shape/collation and coverage (EXPLAIN QUERY PLAN), never silently
replaced.

Tests: tests/test_state_db_readonly_readers.py (read-only failure never
retries writable; no poll without subscribers; listing creates no index;
index tool requires drain, is idempotent, works without a lock file and
refuses a held lock; cron sidebar read is mode=ro). Neighbouring tests
that asserted the retired writable fallback / self-heal writer
(test_issue3887_index_prime, test_issue5455_listing_readonly_connection,
test_issue5455_lineage_readonly_reads, test_issue2628_cli_sessions_perf,
test_issue1494_state_db_fd_leak) now assert the read-only contract;
test_issue3506 subscribes before polling.
This commit is contained in:
Anthony Ruiz committed 2026-09-18 00:59:17 +00:00
1 parent 2cf8e8a5ea
commit ef9341b2ab
11 files changed
+303 -105

No files matched your search

+13 -14
View File
@@ -9,7 +9,6 @@ lineage-metadata reads, and the gateway-watcher fingerprint projection (a 5s
poll), were still opening a read-WRITE connection. This shared them onto the
same ``open_state_db_readonly`` helper.
"""
import logging
import sqlite3
from contextlib import closing
@@ -98,25 +97,25 @@ def test_helper_encodes_special_path_chars(tmp_path, monkeypatch):
assert calls[0]["target"].endswith("?mode=ro")
def test_helper_falls_back_to_writable_and_logs(tmp_path, monkeypatch, caplog):
def test_helper_never_falls_back_to_writable(tmp_path, monkeypatch):
"""A read-only open failure propagates; the helper never retries with a
write-capable handle on the live WAL database."""
db = tmp_path / "state.db"
_make_lineage_db(db)
real_connect = sqlite3.connect
calls = []
def fail_read_only(target, *args, **kwargs):
if kwargs.get("uri"):
raise sqlite3.OperationalError("synthetic read-only URI failure")
return real_connect(target, *args, **kwargs)
calls.append({"target": str(target), "uri": bool(kwargs.get("uri"))})
raise sqlite3.OperationalError("synthetic read-only URI failure")
monkeypatch.setattr(agent_sessions.sqlite3, "connect", fail_read_only)
with caplog.at_level(logging.WARNING, logger="api.agent_sessions"):
conn = open_state_db_readonly(db)
try:
assert conn.execute("SELECT COUNT(*) FROM sessions").fetchone()[0] == 2
finally:
conn.close()
assert "read-only open failed" in caplog.text
assert "synthetic read-only URI failure" in caplog.text
with pytest.raises(sqlite3.OperationalError, match="synthetic read-only URI failure"):
open_state_db_readonly(db)
# Exactly one attempt, and it was the read-only URI form.
assert len(calls) == 1
assert calls[0]["uri"] is True
assert calls[0]["target"].endswith("?mode=ro")
def test_helper_raises_on_missing_db_instead_of_creating_a_ghost(tmp_path):