ci: run tests across 5 shards (up from 3) + fix 2 latent test-isolation leaks

Two cross-file isolation leaks that a finer shard partition exposes are fixed at
root so every shard passes regardless of execution order:
- test_auth_session_persistence.py: bind api.auth STATE_DIR/_SESSIONS_FILE/
  _LOGIN_ATTEMPTS_FILE in setUp + restore in tearDown (was relying on a sibling
  reload test to rebind them).
- test_request_diagnostics_cache.py: pin the session-list cache source stamp via
  an autouse fixture (matches test_session_sidebar_cache.py) so a leaked state.db
  WAL sidecar can't cause a spurious cache-miss.

Verified all shards green under 3-, 4-, and 5-way partitions. CI infra + test
harness only; no application behavior change.
This commit is contained in:
nesquena-hermes committed 2026-06-29 05:55:16 +00:00
1 parent f2ffeacc64
commit 72f7530c45
4 files changed
+56 -4

No files matched your search

+4 -4
View File
@@ -77,14 +77,14 @@ jobs:
fail-fast: false fail-fast: false
matrix: matrix:
python-version: ['3.11', '3.12', '3.13'] python-version: ['3.11', '3.12', '3.13']
# Split the suite across 3 parallel shards per Python version. pytest-shard # Split the suite across 5 parallel shards per Python version. pytest-shard
# partitions tests deterministically by test-id hash; the suite was made # partitions tests deterministically by test-id hash; the suite was made
# shard-safe (no cross-test state leakage) so every shard passes # shard-safe (no cross-test state leakage) so every shard passes
# independently. See docs/agent-memory note on test-suite shard-safety. # independently. See docs/agent-memory note on test-suite shard-safety.
# NOTE: pytest-shard is 0-indexed — shard ids must be 0..num_shards-1. # NOTE: pytest-shard is 0-indexed — shard ids must be 0..num_shards-1.
# Using 1-based ids would crash the out-of-range job AND silently skip # Using 1-based ids would crash the out-of-range job AND silently skip
# shard 0's tests. # shard 0's tests.
shard: [0, 1, 2] shard: [0, 1, 2, 3, 4]
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
@@ -116,5 +116,5 @@ jobs:
# importorskip and the matrix stays green. # importorskip and the matrix stays green.
pip install mcp || echo "mcp install failed — test_mcp_server.py will importorskip" pip install mcp || echo "mcp install failed — test_mcp_server.py will importorskip"
- name: Run tests (shard ${{ matrix.shard }} of 3) - name: Run tests (shard ${{ matrix.shard }} of 5)
run: pytest tests/ -v --timeout=60 --shard-id=${{ matrix.shard }} --num-shards=3 run: pytest tests/ -v --timeout=60 --shard-id=${{ matrix.shard }} --num-shards=5
+1
View File
@@ -23,6 +23,7 @@
### Changed ### Changed
- **Internal: added a runtime-journal ↔ settled-hydration parity regression test** for anchor activity scenes. The test drives a real `RunJournalWriter` through a full turn (token / reasoning / tool / tool_complete) and asserts that `_run_journal_live_snapshot(...).anchor_activity_scene` and the persisted-then-`_hydrate_anchor_activity_scenes(...)` settled scene expose identical visible semantics — locking in the invariant that a live-streaming turn and a reloaded settled turn render the same anchor activity. Test-only; no behavior change. Thanks @franksong2702. (#5183) - **Internal: added a runtime-journal ↔ settled-hydration parity regression test** for anchor activity scenes. The test drives a real `RunJournalWriter` through a full turn (token / reasoning / tool / tool_complete) and asserts that `_run_journal_live_snapshot(...).anchor_activity_scene` and the persisted-then-`_hydrate_anchor_activity_scenes(...)` settled scene expose identical visible semantics — locking in the invariant that a live-streaming turn and a reloaded settled turn render the same anchor activity. Test-only; no behavior change. Thanks @franksong2702. (#5183)
- **Internal: CI test suite now runs across 5 shards per Python version (up from 3) for faster feedback.** Two latent test-isolation leaks that a finer partition would have exposed were fixed at the root so every shard passes independently regardless of execution order: (1) `tests/test_auth_session_persistence.py` now binds `api.auth`'s `STATE_DIR`/`_SESSIONS_FILE`/`_LOGIN_ATTEMPTS_FILE` to its own temp state in `setUp` and restores them in `tearDown` (previously it relied on a sibling reload test having rebound them, so a shard without that test read the wrong key dir and the warning assertion failed); (2) `tests/test_request_diagnostics_cache.py` pins the session-list cache source stamp via an autouse fixture (matching `test_session_sidebar_cache.py`) so a leaked `state.db` WAL sidecar can't cause a spurious cache-miss. Verified all shards green under 3-, 4-, and 5-way partitions. CI infra + test-harness only; no application behavior change. (internal)
- **The new-conversation splash logo now breathes free instead of sitting in an app-icon tile.** On the empty-state hero, the caduceus mark dropped its glossy navy rounded-rect (which read like a misplaced macOS app-launcher tile — a metaphor mismatch, since nothing launches there) in favor of a bare, larger (~88px) mark over a soft radial "spirit" bloom, matching how Claude/ChatGPT/Gemini render their empty-state marks. The mark gradient and the bloom are tuned per theme: dark themes get the bright `#08EBF1 → #3889FD` mark with a luminous cyan bloom, light themes a deeper `#0AA6CC → #1E63D6` mark with a cooler, restrained bloom so it stays crisp on the pale background. A gentle entrance is gated behind `prefers-reduced-motion`. The titlebar icon and favicon family (which legitimately *are* app icons) keep the navy badge. Thanks @rodboev for the core call. - **The new-conversation splash logo now breathes free instead of sitting in an app-icon tile.** On the empty-state hero, the caduceus mark dropped its glossy navy rounded-rect (which read like a misplaced macOS app-launcher tile — a metaphor mismatch, since nothing launches there) in favor of a bare, larger (~88px) mark over a soft radial "spirit" bloom, matching how Claude/ChatGPT/Gemini render their empty-state marks. The mark gradient and the bloom are tuned per theme: dark themes get the bright `#08EBF1 → #3889FD` mark with a luminous cyan bloom, light themes a deeper `#0AA6CC → #1E63D6` mark with a cooler, restrained bloom so it stays crisp on the pale background. A gentle entrance is gated behind `prefers-reduced-motion`. The titlebar icon and favicon family (which legitimately *are* app icons) keep the navy badge. Thanks @rodboev for the core call.
+24
View File
@@ -28,6 +28,22 @@ class TestSessionPersistence(unittest.TestCase):
"""Sessions survive a simulated process restart (module reload).""" """Sessions survive a simulated process restart (module reload)."""
def setUp(self) -> None: def setUp(self) -> None:
# Bind api.auth's STATE_DIR-derived globals to THIS file's _TEST_STATE,
# regardless of import/reload order. pytest-shard distributes individual
# test items, so a shard may run a key/session test WITHOUT any of the
# _simulate_restart() tests that (as a reload side effect) rebind
# auth.STATE_DIR to _TEST_STATE. Without this, auth.STATE_DIR keeps
# conftest's TEST_STATE_DIR: _load_key() then reads the wrong dir (so the
# sentinel key is "missing", no OSError fires, and assertLogs sees no
# WARNING), and the warning's "STATE_DIR=..." never contains _TEST_STATE.
# Patching the actual bindings the code reads makes every test in this
# class self-contained. Saved values are restored in tearDown.
self._saved_state_dir = auth.STATE_DIR
self._saved_sessions_file = auth._SESSIONS_FILE
self._saved_login_attempts_file = auth._LOGIN_ATTEMPTS_FILE
auth.STATE_DIR = _TEST_STATE
auth._SESSIONS_FILE = _TEST_STATE / '.sessions.json'
auth._LOGIN_ATTEMPTS_FILE = _TEST_STATE / '.login_attempts.json'
auth._sessions.clear() auth._sessions.clear()
auth._PBKDF2_KEY_CACHE = None auth._PBKDF2_KEY_CACHE = None
auth._SIGNING_KEY_CACHE = None auth._SIGNING_KEY_CACHE = None
@@ -36,6 +52,14 @@ class TestSessionPersistence(unittest.TestCase):
if path.exists(): if path.exists():
path.unlink() path.unlink()
def tearDown(self) -> None:
auth.STATE_DIR = self._saved_state_dir
auth._SESSIONS_FILE = self._saved_sessions_file
auth._LOGIN_ATTEMPTS_FILE = self._saved_login_attempts_file
auth._sessions.clear()
auth._PBKDF2_KEY_CACHE = None
auth._SIGNING_KEY_CACHE = None
def _simulate_restart(self) -> None: def _simulate_restart(self) -> None:
"""Reload auth module to simulate a fresh process start. """Reload auth module to simulate a fresh process start.
+27
View File
@@ -1,7 +1,34 @@
import pytest
import api.routes as routes import api.routes as routes
from api.request_diagnostics import RequestDiagnostics from api.request_diagnostics import RequestDiagnostics
@pytest.fixture(autouse=True)
def _stable_session_list_cache_stamp(monkeypatch):
"""Pin the session-list cache source stamp so these stage-emission unit
tests are independent of leaked filesystem state.
``_get_cached_session_list_payload`` stores then re-reads the cache, and the
cache's freshness check compares the source stamp captured at store time
against the stamp recomputed at read time. That stamp folds in a content
fingerprint of the active ``state.db``: computing it opens the DB in WAL
mode, which creates a zero-byte ``-wal`` sidecar as a side effect. If an
earlier test in the same pytest shard left a ``state.db`` in the shared test
STATE_DIR, the FIRST stamp computation materialises the ``-wal`` file and the
immediately-following one sees a different sidecar stat — so the stamps
differ, the cache reports a (spurious) miss, the builder runs, and the
cache-hit assertion fails. In isolation no ``state.db`` exists, the stamp is
a stable all-zero tuple, and the test passes — masking the bug at the
current 3-shard split. These are unit tests of diagnostic STAGE emission, not
of stamp derivation, so we stabilise the stamp the same way
test_session_sidebar_cache.py does. (Patches the binding the cache's
resolved-stamp lookup actually reads; auto-restored by monkeypatch.)
"""
monkeypatch.setattr(routes, "_session_list_cache_source_stamp", lambda _key: ("stable",))
def test_request_diagnostics_maybe_start_still_covers_sessions_route(): def test_request_diagnostics_maybe_start_still_covers_sessions_route():
assert RequestDiagnostics.maybe_start("GET", "/api/sessions") is not None assert RequestDiagnostics.maybe_start("GET", "/api/sessions") is not None