diff --git a/docs/CONTRACTS.md b/docs/CONTRACTS.md index fcfc5d54..05ed701a 100644 --- a/docs/CONTRACTS.md +++ b/docs/CONTRACTS.md @@ -78,6 +78,13 @@ contributor guidance; it does not change runtime behavior or CI gates. proof gates. Prefer the RFC's **Authoritative emitted events** table (live `/api/chat/stream` wire names) over the aspirational semantic taxonomy when writing clients against current source. +- [`docs/architecture/models-cache-invalidation.md`](architecture/models-cache-invalidation.md): + current contract for the `/api/models` catalog cache identity: the + `config.yaml`, `auth.json`, and catalog source-fingerprint axes, the + one-directional volatile-key deny-lists (auth rotation and Codex's + `models_cache.json` refresh timestamps), the stat-identity fallbacks, and the + schema/version stamps. Start here before changing model-catalog caching, the + `/api/models` cache keys, or the Codex catalog dependency (#2443, #7540). When a change touches streaming, recovery, replay, compression, context reconstruction, cancellation, approval/clarify, session metadata, or run state, diff --git a/docs/architecture/models-cache-invalidation.md b/docs/architecture/models-cache-invalidation.md new file mode 100644 index 00000000..f50d14b0 --- /dev/null +++ b/docs/architecture/models-cache-invalidation.md @@ -0,0 +1,95 @@ +# Models catalog cache invalidation contract (`/api/models`) + +This document records the current identity and invalidation contract for the +`/api/models` catalog cache in `api/config.py`. It describes shipped behavior and +changes no runtime behavior. It was added after #7556 shipped in `exp-v0.52.303`, +which the #7556 review flagged as an undocumented runtime contract. + +## What is cached + +- **In memory:** `_available_models_cache` plus `_available_models_cache_ts`, + with `_AVAILABLE_MODELS_CACHE_TTL` set to 24 hours. +- **On disk:** one `models_cache.json` per profile under the WebUI state + directory (`_get_models_cache_path()`), stamped with `_schema_version` and + `_webui_version`. +- **Cold path:** `get_available_models(prefer_cache=...)`; the `prefer_cache` + branch never starts a live provider rebuild. +- **Hot path:** `_endpoint_advertised_model_ids()` reads only the published + in-memory snapshot through the lock-free `_models_cache_provenance` tuple and + validates it against the current source fingerprint before trusting it. + `_sync_models_cache_provenance()` must run at every site that publishes or + invalidates the snapshot, so the tuple can never tear. + +## The three source axes + +`_models_cache_source_fingerprint()` is the single chokepoint. A cache is served +only when every axis matches the value recorded in the cache — the disk reader +compares it in `_is_loadable_disk_cache()`, and the hot path compares it against +the fingerprint captured at publish time. + +| Axis | Identity | Why it is fingerprinted this way | +| --- | --- | --- | +| `config_yaml` | stat identity: `mtime_ns` + size (`_models_cache_file_fingerprint`) | The file is rewritten only on deliberate user edits, and any edit can change the provider/model set, so the cheap conservative identity wins. | +| `auth_json` | content hash with a volatile-key deny-list (`_auth_store_semantic_fingerprint`, `_AUTH_FINGERPRINT_VOLATILE_KEYS`) | The credential store is rewritten roughly every 14 minutes by credential-pool / OAuth refresh; none of those rotating fields feed `detected_providers` or the returned catalog, and stat identity made the 24h cache churn on every refresh (RCA `t_d127953d` / `t_16551f61`). | +| `catalog` | baked-in provider catalog sha256 (`_PROVIDER_MODELS` + `_PROVIDER_DISPLAY`) plus the Codex local catalog (`_codex_models_cache_fingerprint`, `_CODEX_CACHE_FINGERPRINT_VOLATILE_KEYS`) | A restart after a catalog change must not keep serving a persisted payload for up to 24h (#2443). Codex rewrites `~/.codex/models_cache.json` on its own timer, bumping `mtime_ns` and size while models, `etag`, and `client_version` stay identical, so the Codex axis hashes **content** with only the refresh timestamps (`fetched_at`, `updated_at`) removed (#7540, #7556). | + +## Invariant: deny-lists are one-directional + +- Both volatile-key sets are deny-lists, never allow-lists. They may remove only + fields that provably do not gate the provider/model set. +- Every other field — including fields that Codex or the auth store may add in + the future — stays in the fingerprint + (`test_unknown_codex_field_stays_in_fingerprint`). +- Consequence: excluding a volatile key can only make the fingerprint **more + stable**; it can never hide a genuine catalog or provider change. When in + doubt, keep the key in. + +## Invariant: fallbacks are never less safe than stat + +- Missing file → recorded as missing, and the fingerprint stays stable + (`test_missing_codex_cache_fingerprint_is_stable_and_marked`). +- Unreadable, corrupt, or mid-write JSON → stat identity, marked + `unparsed-fallback`. +- Transform failure — including `RecursionError` on a pathologically deep tree → + stat identity, marked `encode-fallback` + (`test_deeply_nested_codex_cache_degrades_to_stat_fallback_without_crashing`). + The fingerprint must never raise into `/api/models`; a real rewrite still + changes the stat identity, so the fallback is strictly no less safe than the + pre-#7556 behavior. + +## Version stamps (independent of the fingerprint) + +- `_schema_version` must equal `_MODELS_CACHE_SCHEMA_VERSION` (currently `3`). + Bump it when the cached payload shape changes incompatibly. +- `_webui_version` must equal the running version, which forces a rebuild after + every release so picker-shape fixes appear immediately instead of after the + TTL expires. When the runtime version cannot be resolved (early boot), that + check is skipped rather than wedging the boot. + +## Change protocol + +1. Add or change a source axis in `_models_cache_source_fingerprint()` only — + one chokepoint, so the disk reader and the published-snapshot reader always + agree. +2. When adding a volatile key, name the value it derives from and prove it + inert: one test that fails if the key is dropped from the deny-list, and one + that shows a genuine catalog change still invalidates the cache. +3. Keep the fingerprint cheap, deterministic, and safe to run synchronously — + it is recomputed on cache reads, including a per-turn hot path. +4. Do not add a second, parallel cache-identity mechanism. The stat-based + `_models_cache_file_fingerprint()` stays only as the conservative fallback for + the axes above. +5. This is a runtime contract: changes here update this document and are + described in the PR body. Release-note wording belongs in the PR body, not in + `CHANGELOG.md`, which release commits own. + +## Tests + +`tests/test_issue7540_codex_catalog_fingerprint.py` covers both invariant groups: +timestamp-only churn keeps the fingerprint identical (and a session visit after a +Codex refresh needs no live rebuild), while genuine changes — a new model, a +visibility change, any catalog field, any unknown field — still invalidate. + +## References + +Issues/PRs: #2443, #7540, #7556, #7558. RCAs: `t_d127953d`, `t_16551f61`.