webtecnica
80610c7b6c
The /api/models cache identity shipped in #7556 (release #7558) changed from a stat identity to content fingerprints with volatile-key deny-lists, and the #7556 review flagged that this runtime contract had no project-level documentation. Record the three source-fingerprint axes and their rationale, the one-directional deny-list invariant, the stat-identity fallbacks (including the deep-tree RecursionError path), the independent schema/version stamps, and the change protocol; index the document from docs/CONTRACTS.md. Docs only.
96 lines
5.7 KiB
Markdown
96 lines
5.7 KiB
Markdown
# Models catalog cache invalidation contract (`/api/models`)
|
|
|
|
This document records the current identity and invalidation contract for the
|
|
`/api/models` catalog cache in `api/config.py`. It describes shipped behavior and
|
|
changes no runtime behavior. It was added after #7556 shipped in `exp-v0.52.303`,
|
|
which the #7556 review flagged as an undocumented runtime contract.
|
|
|
|
## What is cached
|
|
|
|
- **In memory:** `_available_models_cache` plus `_available_models_cache_ts`,
|
|
with `_AVAILABLE_MODELS_CACHE_TTL` set to 24 hours.
|
|
- **On disk:** one `models_cache.json` per profile under the WebUI state
|
|
directory (`_get_models_cache_path()`), stamped with `_schema_version` and
|
|
`_webui_version`.
|
|
- **Cold path:** `get_available_models(prefer_cache=...)`; the `prefer_cache`
|
|
branch never starts a live provider rebuild.
|
|
- **Hot path:** `_endpoint_advertised_model_ids()` reads only the published
|
|
in-memory snapshot through the lock-free `_models_cache_provenance` tuple and
|
|
validates it against the current source fingerprint before trusting it.
|
|
`_sync_models_cache_provenance()` must run at every site that publishes or
|
|
invalidates the snapshot, so the tuple can never tear.
|
|
|
|
## The three source axes
|
|
|
|
`_models_cache_source_fingerprint()` is the single chokepoint. A cache is served
|
|
only when every axis matches the value recorded in the cache — the disk reader
|
|
compares it in `_is_loadable_disk_cache()`, and the hot path compares it against
|
|
the fingerprint captured at publish time.
|
|
|
|
| Axis | Identity | Why it is fingerprinted this way |
|
|
| --- | --- | --- |
|
|
| `config_yaml` | stat identity: `mtime_ns` + size (`_models_cache_file_fingerprint`) | The file is rewritten only on deliberate user edits, and any edit can change the provider/model set, so the cheap conservative identity wins. |
|
|
| `auth_json` | content hash with a volatile-key deny-list (`_auth_store_semantic_fingerprint`, `_AUTH_FINGERPRINT_VOLATILE_KEYS`) | The credential store is rewritten roughly every 14 minutes by credential-pool / OAuth refresh; none of those rotating fields feed `detected_providers` or the returned catalog, and stat identity made the 24h cache churn on every refresh (RCA `t_d127953d` / `t_16551f61`). |
|
|
| `catalog` | baked-in provider catalog sha256 (`_PROVIDER_MODELS` + `_PROVIDER_DISPLAY`) plus the Codex local catalog (`_codex_models_cache_fingerprint`, `_CODEX_CACHE_FINGERPRINT_VOLATILE_KEYS`) | A restart after a catalog change must not keep serving a persisted payload for up to 24h (#2443). Codex rewrites `~/.codex/models_cache.json` on its own timer, bumping `mtime_ns` and size while models, `etag`, and `client_version` stay identical, so the Codex axis hashes **content** with only the refresh timestamps (`fetched_at`, `updated_at`) removed (#7540, #7556). |
|
|
|
|
## Invariant: deny-lists are one-directional
|
|
|
|
- Both volatile-key sets are deny-lists, never allow-lists. They may remove only
|
|
fields that provably do not gate the provider/model set.
|
|
- Every other field — including fields that Codex or the auth store may add in
|
|
the future — stays in the fingerprint
|
|
(`test_unknown_codex_field_stays_in_fingerprint`).
|
|
- Consequence: excluding a volatile key can only make the fingerprint **more
|
|
stable**; it can never hide a genuine catalog or provider change. When in
|
|
doubt, keep the key in.
|
|
|
|
## Invariant: fallbacks are never less safe than stat
|
|
|
|
- Missing file → recorded as missing, and the fingerprint stays stable
|
|
(`test_missing_codex_cache_fingerprint_is_stable_and_marked`).
|
|
- Unreadable, corrupt, or mid-write JSON → stat identity, marked
|
|
`unparsed-fallback`.
|
|
- Transform failure — including `RecursionError` on a pathologically deep tree →
|
|
stat identity, marked `encode-fallback`
|
|
(`test_deeply_nested_codex_cache_degrades_to_stat_fallback_without_crashing`).
|
|
The fingerprint must never raise into `/api/models`; a real rewrite still
|
|
changes the stat identity, so the fallback is strictly no less safe than the
|
|
pre-#7556 behavior.
|
|
|
|
## Version stamps (independent of the fingerprint)
|
|
|
|
- `_schema_version` must equal `_MODELS_CACHE_SCHEMA_VERSION` (currently `3`).
|
|
Bump it when the cached payload shape changes incompatibly.
|
|
- `_webui_version` must equal the running version, which forces a rebuild after
|
|
every release so picker-shape fixes appear immediately instead of after the
|
|
TTL expires. When the runtime version cannot be resolved (early boot), that
|
|
check is skipped rather than wedging the boot.
|
|
|
|
## Change protocol
|
|
|
|
1. Add or change a source axis in `_models_cache_source_fingerprint()` only —
|
|
one chokepoint, so the disk reader and the published-snapshot reader always
|
|
agree.
|
|
2. When adding a volatile key, name the value it derives from and prove it
|
|
inert: one test that fails if the key is dropped from the deny-list, and one
|
|
that shows a genuine catalog change still invalidates the cache.
|
|
3. Keep the fingerprint cheap, deterministic, and safe to run synchronously —
|
|
it is recomputed on cache reads, including a per-turn hot path.
|
|
4. Do not add a second, parallel cache-identity mechanism. The stat-based
|
|
`_models_cache_file_fingerprint()` stays only as the conservative fallback for
|
|
the axes above.
|
|
5. This is a runtime contract: changes here update this document and are
|
|
described in the PR body. Release-note wording belongs in the PR body, not in
|
|
`CHANGELOG.md`, which release commits own.
|
|
|
|
## Tests
|
|
|
|
`tests/test_issue7540_codex_catalog_fingerprint.py` covers both invariant groups:
|
|
timestamp-only churn keeps the fingerprint identical (and a session visit after a
|
|
Codex refresh needs no live rebuild), while genuine changes — a new model, a
|
|
visibility change, any catalog field, any unknown field — still invalidate.
|
|
|
|
## References
|
|
|
|
Issues/PRs: #2443, #7540, #7556, #7558. RCAs: `t_d127953d`, `t_16551f61`.
|