fix: Nous Portal full live catalog + dropdown cache invalidation on provider remove

Closes #1538, #1539. Two related dropdown-staleness bugs reported by Deor
(Discord, May 03 2026).

#1538 — Nous Portal picker showed only 4 hardcoded models
=========================================================
The Settings → Default Model picker, the composer model dropdown, the
/model slash command, and the Settings → Providers card all showed only
four Nous models (Claude Opus 4.6, Claude Sonnet 4.6, GPT-5.4 Mini, Gemini
3.1 Pro Preview) because `_PROVIDER_MODELS["nous"]` had four hardcoded
entries and `_build_available_models_uncached()` fell through to the
generic `pid in _PROVIDER_MODELS` branch.

The actual Nous Portal catalog has 30 models live — Claude Opus 4.7, GPT-5.5,
Kimi K2.6, MiniMax M2.7, Gemini 3.1 Pro/Flash, several Xiaomi/Tencent/StepFun
entries, and more.

Fix:
- New `_format_nous_label()` helper in `api/config.py` — reuses the
  `_format_ollama_label()` token rules, drops the vendor namespace, and
  appends ` (via Nous)` so labels disambiguate from same-named direct-
  provider entries (e.g. "Claude Opus 4.7" via direct Anthropic).
- New `elif pid == "nous":` branch in `_build_available_models_uncached()`
  mirroring the Ollama Cloud pattern: live-fetch through
  `hermes_cli.models.provider_model_ids("nous")`, prefix every id with
  `@nous:` (matches the existing routing convention from PR-era #854 and
  pinned in tests/test_nous_portal_routing.py), fall back to the curated
  4-entry static list when hermes_cli is unavailable.
- Same fix applied to `api/providers.py:get_providers()` — that's the
  separate code path that builds Settings → Providers card models, and
  it had the identical bug shape.

#1539 — Removed provider lingered in dropdowns until restart
============================================================
After Settings → Providers → Remove, the provider's models still appeared
in every model dropdown until the page was reloaded. The server-side
TTL cache was correctly flushed (`set_provider_key()` calls
`invalidate_models_cache()` on both add and remove) but JS-side caches
were never dropped:

- `_slashModelCache` / `_slashModelCachePromise` (commands.js) — feeds
  the `/model` slash-command suggestions.
- `_dynamicModelLabels` / `window._configuredModelBadges` (ui.js) —
  populated by `populateModelDropdown()` on app boot and profile switch.

Pre-fix, `_removeProviderKey()` only called `loadProvidersPanel()`
which refreshed the providers card list but never asked any consumer
to re-fetch /api/models.

Fix:
- `static/commands.js`: new `_invalidateSlashModelCache()` helper that
  nulls both cache slots, exposed on `window` (typeof-guarded so the
  module remains importable in headless vm contexts — needed by the
  existing tests/test_cli_only_slash_commands.py harness).
- `static/panels.js`: new `_refreshModelDropdownsAfterProviderChange()`
  helper that calls the invalidator + `populateModelDropdown()`, wrapped
  in try/catch so the providers panel update never breaks if a
  downstream module hasn't loaded yet. Both `_saveProviderKey` and
  `_removeProviderKey` invoke it (defense-in-depth: same staleness shape
  applies to the add path too).

Tests
-----
- `tests/test_issue1538_nous_live_catalog.py` (12 tests): live-fetch
  surfaces ≥20 entries, every id starts with `@nous:`, every label ends
  with ` (via Nous)`, recent flagships (Opus 4.7, GPT-5.5, Kimi K2.6,
  Gemini 3.1 Pro, MiniMax M2.7) reach the dropdown, static fallback
  works when hermes_cli raises, label formatter unit tests (vendor
  namespace stripping, variant rendering, MiniMax mixed-case), the
  curated static list and its routing invariants are preserved.
- `tests/test_issue1539_provider_removal_dropdown_invalidation.py`
  (11 tests): invalidator helper exists and clears both cache slots,
  exposed on window with typeof guard, both save and remove paths
  invoke the dropdown flush, helper calls both invalidator and
  populateModelDropdown, helper is resilient to missing modules,
  helper does not block panel refresh, server-side
  `set_provider_key → invalidate_models_cache` invariant pinned.

Verified live on port 8789: `/api/models` Nous group returns 30
models (was 4); browser `document.getElementById('modelSelect')`
exposes 30 options under the "Nous Portal" group; the dropdown-flush
helper is callable from the browser and round-trip rebuild keeps the
dropdown at 30 options.

Test counts:
- Full pytest: 4013 passed, 2 skipped, 3 xpassed, 0 failures
  (was 3990 → 4013, +23 from this PR).
- QA harness pytest: 20 passed.
- Browser API sanity: 11/11 passed.
- Agent Browser CDP: 21/23 passed (the 2 SSE liveness failures
  reproduce on master and are unrelated to this PR).
This commit is contained in:
nesquena-hermes authored and Hermes Bot committed 2026-05-03 18:12:01 +00:00
1 parent 84e74407c9
commit bff8cb2b58
6 files changed
+674 -1

No files matched your search

+67
View File
@@ -860,6 +860,35 @@ def _format_ollama_label(mid: str) -> str:
return label
def _format_nous_label(mid: str) -> str:
"""Turn a Nous Portal model id into a readable display label.
Nous IDs are ``<vendor>/<model>[:<variant>]`` (e.g. ``anthropic/claude-opus-4.7``);
drop the vendor namespace, prettify the model name with the same token
rules as :func:`_format_ollama_label` (short acronyms uppercase, size
suffixes uppercase, capitalize the rest), then append ``" (via Nous)"``
so the entry is visually distinct from same-named models in other
provider groups (e.g. direct Anthropic).
Examples::
anthropic/claude-opus-4.7 -> Claude Opus 4.7 (via Nous)
openai/gpt-5.4-mini -> GPT 5.4 Mini (via Nous)
google/gemini-3.1-pro-preview -> Gemini 3.1 Pro Preview (via Nous)
moonshotai/kimi-k2.6 -> Kimi K2.6 (via Nous)
qwen/qwen3.5-plus-02-15 -> Qwen3.5 Plus 02 15 (via Nous)
nvidia/nemotron-3-super-120b-a12b -> Nemotron 3 Super 120B A12B (via Nous)
minimax/minimax-m2.5:free -> MiniMax M2.5 (Free) (via Nous)
"""
name_part = mid.split("/", 1)[-1] if "/" in mid else mid
# MiniMax-CN ids come back lowercase on the live wire (`minimax-m2.5`) but
# the curated label convention is mixed-case "MiniMax M2.5" — match that.
if name_part.lower().startswith("minimax"):
name_part = "MiniMax" + name_part[len("minimax"):]
base = _format_ollama_label(name_part)
return f"{base} (via Nous)"
def _apply_provider_prefix(
raw_models: list[dict],
provider_id: str,
@@ -2100,6 +2129,44 @@ def get_available_models() -> dict:
except Exception:
logger.warning("Failed to load Ollama Cloud models from hermes_cli")
if raw_models:
models = _apply_provider_prefix(raw_models, pid, active_provider)
groups.append(
{
"provider": provider_name,
"provider_id": pid,
"models": models,
}
)
elif pid == "nous":
# Nous Portal exposes a curated catalog (~30 models, currently)
# via inference-api.nousresearch.com. Like ollama-cloud, we
# live-fetch through hermes_cli.models.provider_model_ids()
# rather than relying on the static four-entry list, which
# chronically drifts out of date (#1538). Fall back to the
# static list when hermes_cli is unavailable (test envs,
# package mismatches) so the picker is never empty.
raw_models = []
try:
from hermes_cli.models import provider_model_ids as _provider_model_ids
live_ids = _provider_model_ids("nous") or []
raw_models = [
# Prefix every live id with "@nous:" so routing matches
# the explicit-provider-hint branch of resolve_model_provider
# (same convention as the curated static list — see
# tests/test_nous_portal_routing.py for the invariant).
{"id": f"@nous:{mid}", "label": _format_nous_label(mid)}
for mid in live_ids
]
except Exception:
logger.warning("Failed to load Nous Portal models from hermes_cli")
if not raw_models:
# Static fallback: deepcopy so dedup/prefix mutation
# below does not bleed into the module-level catalog.
raw_models = copy.deepcopy(_PROVIDER_MODELS.get("nous", []))
if raw_models:
models = _apply_provider_prefix(raw_models, pid, active_provider)
groups.append(
+20 -1
View File
@@ -391,7 +391,26 @@ def get_providers() -> dict[str, Any]:
except Exception:
pass
models = _PROVIDER_MODELS.get(pid, [])
models = list(_PROVIDER_MODELS.get(pid, []))
# Nous Portal: prefer the live catalog so the providers card matches
# the dropdown picker (#1538). Same fallback shape as the static-only
# case below — when hermes_cli is unavailable or its lookup raises,
# we keep the four-entry curated list.
if pid == "nous":
try:
from hermes_cli.models import provider_model_ids as _provider_model_ids
live_ids = _provider_model_ids("nous") or []
if live_ids:
# Lazy-import to avoid circular dep with api.config.
from api.config import _format_nous_label
models = [
{"id": f"@nous:{mid}", "label": _format_nous_label(mid)}
for mid in live_ids
]
except Exception:
logger.debug("Failed to load Nous Portal models from hermes_cli")
# Also include models from config.yaml providers section
if isinstance(providers_cfg, dict):
provider_cfg = providers_cfg.get(pid, {})
+16
View File
@@ -88,6 +88,22 @@ let _slashPersonalityCachePromise=null;
let _agentCommandCache=null;
let _agentCommandCachePromise=null;
// Invalidate the /api/models slash-suggestion cache. Called by panels.js
// after a provider is added or removed so the next /model autocomplete
// rebuilds from a fresh /api/models response (#1539). Returning a function
// rather than letting callers poke the module-local lets/promises directly
// keeps the cache shape encapsulated to this module.
function _invalidateSlashModelCache(){
_slashModelCache=null;
_slashModelCachePromise=null;
}
// Expose on window when available. Guarded by typeof so the module is
// importable in headless test contexts (vm.runInContext) that don't
// define a window global — see tests/test_cli_only_slash_commands.py.
if(typeof window!=='undefined'){
window._invalidateSlashModelCache=_invalidateSlashModelCache;
}
function _normalizeSlashSubArg(value){
return String(value||'').trim();
}
+33
View File
@@ -3364,6 +3364,11 @@ async function _saveProviderKey(providerId){
if(res.ok){
showToast(res.provider+' key '+res.action);
els.input.value='';
// Invalidate every dropdown surface that caches /api/models so the
// newly-configured provider's models show up without a server restart
// or page reload (#1539). Server-side invalidate_models_cache() is
// already called by api/providers.py:set_provider_key.
_refreshModelDropdownsAfterProviderChange();
await loadProvidersPanel(); // refresh list
}else{
showToast(res.error||'Failed to save key');
@@ -3385,6 +3390,12 @@ async function _removeProviderKey(providerId){
const res=await api('/api/providers/delete',{method:'POST',body:JSON.stringify({provider:providerId})});
if(res.ok){
showToast(res.provider+' key '+t('providers_key_removed').toLowerCase());
// Drop the removed provider from every cached dropdown surface so it
// disappears immediately — composer picker, /model slash command,
// Settings → Default Model, configured-model badges (#1539).
// Without this, a stale list from before the delete keeps offering
// the now-removed provider's models until the page is reloaded.
_refreshModelDropdownsAfterProviderChange();
await loadProvidersPanel(); // refresh list
}else{
showToast(res.error||'Failed to remove key');
@@ -3396,6 +3407,28 @@ async function _removeProviderKey(providerId){
}
}
// Shared dropdown-cache flush invoked after a provider add/remove. The
// server-side TTL cache is already invalidated by /api/providers and
// /api/providers/delete (via api/providers.py:set_provider_key); this
// flushes the JS-side caches so the next render rebuilds from a fresh
// /api/models response. Wrapped in a try/catch so a UI module that hasn't
// loaded yet (e.g. during early Settings open) cannot break the save flow.
function _refreshModelDropdownsAfterProviderChange(){
try{
if(typeof window._invalidateSlashModelCache==='function'){
window._invalidateSlashModelCache();
}
if(typeof populateModelDropdown==='function'){
// Fire-and-forget: don't block the providers panel refresh on a
// dropdown rebuild. The composer/Settings dropdowns will catch up
// on the very next paint frame.
Promise.resolve(populateModelDropdown()).catch(()=>{});
}
}catch(_e){
// Swallow — dropdown refresh is best-effort, providers panel must still update.
}
}
async function _refreshProviderModels(providerId, btn){
btn.disabled=true;
const orig=btn.innerHTML;
+313
View File
@@ -0,0 +1,313 @@
"""Regression tests for #1538 — Nous Portal model picker should live-fetch
the full catalog (~30 models) instead of returning the four-entry static list.
Background
----------
Settings → Default Model showed only four Nous models (Claude Opus 4.6, Claude
Sonnet 4.6, GPT-5.4 Mini, Gemini 3.1 Pro Preview) because
``_build_available_models_uncached()`` fell through to the generic
``pid in _PROVIDER_MODELS`` branch and returned ``copy.deepcopy(_PROVIDER_MODELS["nous"])``.
The actual Nous Portal catalog has ~30 models live — including the latest
Anthropic 4.7 family, GPT-5.5, Gemini 3.1 Pro/Flash, Kimi K2.6, MiniMax M2.7,
several Xiaomi/Tencent/StepFun entries.
Fix
---
A dedicated ``elif pid == "nous":`` branch in ``_build_available_models_uncached()``
mirroring the Ollama Cloud pattern: live-fetch via
``hermes_cli.models.provider_model_ids("nous")``, prefix every id with ``@nous:``
to match the existing routing convention, fall back to the curated static
list when ``hermes_cli`` is unavailable.
"""
from __future__ import annotations
import sys
import types
import api.config as config
import api.profiles as profiles
# Sample Nous catalog used in the live-fetch test. Mirrors the shape returned
# by hermes_cli.models.provider_model_ids("nous") (see #1538 issue body).
SAMPLE_NOUS_LIVE_IDS = [
"moonshotai/kimi-k2.6",
"xiaomi/mimo-v2.5-pro",
"anthropic/claude-opus-4.7",
"anthropic/claude-opus-4.6",
"anthropic/claude-sonnet-4.6",
"anthropic/claude-haiku-4.5",
"openai/gpt-5.5",
"openai/gpt-5.4-mini",
"openai/gpt-5.3-codex",
"google/gemini-3-pro-preview",
"google/gemini-3.1-pro-preview",
"google/gemini-3.1-flash-lite-preview",
"qwen/qwen3.5-plus-02-15",
"minimax/minimax-m2.7",
"z-ai/glm-5.1",
"x-ai/grok-4.20-beta",
"tencent/hy3-preview",
"stepfun/step-3.5-flash",
"nvidia/nemotron-3-super-120b-a12b",
"arcee-ai/trinity-large-thinking",
]
def _install_fake_hermes_cli(monkeypatch, *, nous_ids=None, raise_on_lookup=False):
"""Install fake ``hermes_cli`` modules so detection sees Nous as authenticated
and ``provider_model_ids("nous")`` returns the desired catalog.
Mirrors :func:`tests.test_issue1420_lmstudio_provider_env_var._install_fake_hermes_cli`
but specialised for Nous detection (Nous is OAuth so the env-var path
is not used — we drive detection via ``hermes_cli.auth.list_auth_providers``).
"""
fake_pkg = types.ModuleType("hermes_cli")
fake_pkg.__path__ = []
fake_models = types.ModuleType("hermes_cli.models")
fake_models.list_available_providers = lambda: []
if raise_on_lookup:
def _raise(_pid):
raise RuntimeError("simulated hermes_cli failure")
fake_models.provider_model_ids = _raise
else:
ids = list(nous_ids) if nous_ids is not None else []
fake_models.provider_model_ids = lambda pid: ids if pid == "nous" else []
fake_auth = types.ModuleType("hermes_cli.auth")
def _list_auth_providers():
return [{"id": "nous", "authenticated": True}]
def _get_auth_status(pid):
return {"logged_in": True, "key_source": ""} if pid == "nous" else {}
fake_auth.list_auth_providers = _list_auth_providers
fake_auth.get_auth_status = _get_auth_status
monkeypatch.setitem(sys.modules, "hermes_cli", fake_pkg)
monkeypatch.setitem(sys.modules, "hermes_cli.models", fake_models)
monkeypatch.setitem(sys.modules, "hermes_cli.auth", fake_auth)
monkeypatch.delitem(sys.modules, "agent.credential_pool", raising=False)
monkeypatch.delitem(sys.modules, "agent", raising=False)
config.invalidate_models_cache()
def _swap_in_test_config(extra_cfg):
"""Snapshot config.cfg, replace with a minimal test config; return restore-fn."""
old_cfg = dict(config.cfg)
old_mtime = config._cfg_mtime
config.cfg.clear()
config.cfg["model"] = {}
config.cfg.update(extra_cfg)
try:
config._cfg_mtime = config.Path(config._get_config_path()).stat().st_mtime
except Exception:
config._cfg_mtime = 0.0
def _restore():
config.cfg.clear()
config.cfg.update(old_cfg)
config._cfg_mtime = old_mtime
return _restore
def _scrub_provider_env(monkeypatch):
"""Drop every provider env var so detection only sees what we install
via the fake hermes_cli stubs (not unrelated keys leaked from the runner)."""
for var in (
"ANTHROPIC_API_KEY", "OPENAI_API_KEY", "GOOGLE_API_KEY", "GEMINI_API_KEY",
"DEEPSEEK_API_KEY", "XAI_API_KEY", "GROQ_API_KEY",
"MISTRAL_API_KEY", "OPENROUTER_API_KEY",
"OLLAMA_CLOUD_API_KEY", "OLLAMA_API_KEY",
"GLM_API_KEY", "KIMI_API_KEY", "MOONSHOT_API_KEY",
"MINIMAX_API_KEY", "MINIMAX_CN_API_KEY",
"OPENCODE_ZEN_API_KEY", "OPENCODE_GO_API_KEY",
"NOUS_API_KEY", "NVIDIA_API_KEY", "LM_API_KEY", "LMSTUDIO_API_KEY",
):
monkeypatch.delenv(var, raising=False)
class TestNousLiveCatalog:
"""When the Nous live catalog is available, the dropdown must surface it
in full (>=20 entries) — not the four-entry static fallback (#1538)."""
def test_nous_models_live_fetch_when_hermes_cli_available(self, monkeypatch, tmp_path):
_scrub_provider_env(monkeypatch)
_install_fake_hermes_cli(monkeypatch, nous_ids=SAMPLE_NOUS_LIVE_IDS)
monkeypatch.setattr(profiles, "get_active_hermes_home", lambda: tmp_path)
restore = _swap_in_test_config({"model": {"provider": "nous"}})
try:
data = config.get_available_models()
nous_groups = [g for g in data.get("groups", []) if g.get("provider_id") == "nous"]
assert len(nous_groups) == 1, (
f"Expected exactly one Nous group, got {len(nous_groups)}: "
f"{[g.get('provider_id') for g in data.get('groups', [])]}"
)
models = nous_groups[0]["models"]
assert len(models) >= 20, (
f"Live-fetched Nous catalog should expose >=20 entries, got "
f"{len(models)}. The dispatch branch fell through to the four-entry "
f"static list — pre-#1538 behaviour."
)
finally:
restore()
def test_nous_model_ids_carry_at_nous_prefix(self, monkeypatch, tmp_path):
_scrub_provider_env(monkeypatch)
_install_fake_hermes_cli(monkeypatch, nous_ids=SAMPLE_NOUS_LIVE_IDS)
monkeypatch.setattr(profiles, "get_active_hermes_home", lambda: tmp_path)
restore = _swap_in_test_config({"model": {"provider": "nous"}})
try:
data = config.get_available_models()
nous_group = next(g for g in data["groups"] if g["provider_id"] == "nous")
for m in nous_group["models"]:
assert m["id"].startswith("@nous:"), (
f"Every Nous model id must start with '@nous:' so "
f"resolve_model_provider routes through the explicit-provider-hint "
f"branch (matches the static-list invariant from "
f"tests/test_nous_portal_routing.py). Got: {m['id']!r}"
)
finally:
restore()
def test_nous_labels_carry_via_nous_suffix(self, monkeypatch, tmp_path):
_scrub_provider_env(monkeypatch)
_install_fake_hermes_cli(monkeypatch, nous_ids=SAMPLE_NOUS_LIVE_IDS)
monkeypatch.setattr(profiles, "get_active_hermes_home", lambda: tmp_path)
restore = _swap_in_test_config({"model": {"provider": "nous"}})
try:
data = config.get_available_models()
nous_group = next(g for g in data["groups"] if g["provider_id"] == "nous")
for m in nous_group["models"]:
assert m["label"].endswith(" (via Nous)"), (
f"Every Nous live-fetched label must end with ' (via Nous)' so "
f"the user can distinguish them from same-named direct-provider "
f"entries (e.g. 'Claude Opus 4.7' via direct Anthropic). "
f"Got: {m['label']!r}"
)
finally:
restore()
def test_nous_live_catalog_includes_recent_models(self, monkeypatch, tmp_path):
"""Sanity: the recent-flagship models from the user's bug report
(Claude Opus 4.7, GPT-5.5, Kimi K2.6) must reach the dropdown."""
_scrub_provider_env(monkeypatch)
_install_fake_hermes_cli(monkeypatch, nous_ids=SAMPLE_NOUS_LIVE_IDS)
monkeypatch.setattr(profiles, "get_active_hermes_home", lambda: tmp_path)
restore = _swap_in_test_config({"model": {"provider": "nous"}})
try:
data = config.get_available_models()
nous_group = next(g for g in data["groups"] if g["provider_id"] == "nous")
ids = {m["id"] for m in nous_group["models"]}
for required in (
"@nous:anthropic/claude-opus-4.7",
"@nous:openai/gpt-5.5",
"@nous:moonshotai/kimi-k2.6",
"@nous:google/gemini-3.1-pro-preview",
"@nous:minimax/minimax-m2.7",
):
assert required in ids, (
f"{required} missing from live-fetched Nous catalog. Either "
f"the hermes_cli dispatch is broken or the @nous: prefix is "
f"missing."
)
finally:
restore()
class TestNousStaticFallback:
"""When ``hermes_cli`` is not importable or its lookup raises, we fall back
to the curated four-entry static list — never empty."""
def test_static_fallback_when_hermes_cli_raises(self, monkeypatch, tmp_path):
_scrub_provider_env(monkeypatch)
_install_fake_hermes_cli(monkeypatch, raise_on_lookup=True)
monkeypatch.setattr(profiles, "get_active_hermes_home", lambda: tmp_path)
restore = _swap_in_test_config({"model": {"provider": "nous"}})
try:
data = config.get_available_models()
nous_groups = [g for g in data.get("groups", []) if g.get("provider_id") == "nous"]
assert nous_groups, (
"Nous group must still appear when hermes_cli fails — the "
"branch should fall back to the curated static list."
)
models = nous_groups[0]["models"]
assert len(models) == 4, (
f"Static fallback should expose exactly the four curated entries "
f"in _PROVIDER_MODELS['nous']. Got {len(models)}: "
f"{[m['id'] for m in models]}"
)
for m in models:
assert m["id"].startswith("@nous:"), m["id"]
finally:
restore()
class TestFormatNousLabel:
"""Unit tests for the label formatter helper."""
def test_strips_vendor_namespace(self):
from api.config import _format_nous_label
assert _format_nous_label("anthropic/claude-opus-4.7") == "Claude Opus 4.7 (via Nous)"
assert _format_nous_label("openai/gpt-5.4-mini") == "GPT 5.4 Mini (via Nous)"
def test_handles_missing_vendor(self):
from api.config import _format_nous_label
# Defensive: id without slash should still render a sane label.
assert _format_nous_label("kimi-k2.6") == "Kimi K2.6 (via Nous)"
def test_handles_variant_after_colon(self):
from api.config import _format_nous_label
# Variant rendered in parentheses, mirroring _format_ollama_label.
out = _format_nous_label("minimax/minimax-m2.5:free")
assert out.endswith(" (via Nous)")
assert "Free" in out
assert "MiniMax M2.5" in out
def test_minimax_renders_mixed_case(self):
from api.config import _format_nous_label
# Live wire returns lowercase 'minimax/minimax-...' but the curated
# convention is mixed-case 'MiniMax'.
assert _format_nous_label("minimax/minimax-m2.7").startswith("MiniMax M2.7")
def test_label_always_ends_with_via_nous_suffix(self):
from api.config import _format_nous_label
for sample in [
"anthropic/claude-opus-4.7",
"openai/gpt-5.5",
"google/gemini-3.1-pro-preview",
"moonshotai/kimi-k2.6",
"z-ai/glm-5.1",
"stepfun/step-3.5-flash",
]:
assert _format_nous_label(sample).endswith(" (via Nous)"), sample
class TestStaticListPreservedAsFallback:
"""The curated ``_PROVIDER_MODELS['nous']`` entry stays as the static
fallback; existing routing invariants from
:mod:`tests.test_nous_portal_routing` must remain valid."""
def test_static_list_present(self):
from api.config import _PROVIDER_MODELS
assert _PROVIDER_MODELS.get("nous"), (
"The curated static Nous list must remain in _PROVIDER_MODELS as "
"a fallback for environments where hermes_cli is unavailable."
)
def test_static_list_keeps_at_nous_prefix(self):
# Keep parity with tests/test_nous_portal_routing.py — ensures the
# static fallback path produces correctly-routable ids when used.
from api.config import _PROVIDER_MODELS
for m in _PROVIDER_MODELS["nous"]:
assert m["id"].startswith("@nous:"), m["id"]
@@ -0,0 +1,225 @@
"""Regression tests for #1539 — removing a provider in Settings must invalidate
every dropdown surface that caches /api/models, so the removed provider
disappears immediately without a server restart or page reload.
The bug
-------
Pre-fix, ``_removeProviderKey()`` in ``static/panels.js`` only called
``loadProvidersPanel()`` after deletion. That refreshed the providers card
list but left these JS-side caches stale:
* ``_slashModelCache`` / ``_slashModelCachePromise`` (``static/commands.js``) —
cache for the ``/model`` slash-command suggestions.
* ``_dynamicModelLabels`` / ``window._configuredModelBadges`` (``static/ui.js``) —
populated by ``populateModelDropdown()`` on boot and on profile switch.
Layered server-side cache via ``api/config.invalidate_models_cache`` was
already flushed (``set_provider_key`` calls it on both add + remove), so the
next ``/api/models`` request would return the correct list — but no consumer
was triggering one.
The fix
-------
``static/commands.js`` exposes an ``_invalidateSlashModelCache()`` helper on
``window``. ``static/panels.js`` calls it from a shared
``_refreshModelDropdownsAfterProviderChange()`` helper after both the save
and the remove paths, plus invokes ``populateModelDropdown()`` to rebuild
the composer / Settings dropdowns and ``_configuredModelBadges`` map.
"""
from __future__ import annotations
import re
from pathlib import Path
import pytest
REPO = Path(__file__).resolve().parent.parent
def _read_static(name: str) -> str:
return (REPO / "static" / name).read_text(encoding="utf-8")
def _extract_function_body(src: str, signature: str) -> str:
"""Return the source of a top-level ``async function NAME(...)`` /
``function NAME(...)`` declaration via brace-balance — robust to nested
blocks (try/catch/await) and not dependent on indentation.
"""
idx = src.find(signature)
if idx == -1:
raise AssertionError(f"signature {signature!r} not found in source")
open_idx = src.find("{", idx)
if open_idx == -1:
raise AssertionError(f"could not find opening brace after {signature!r}")
depth = 0
for i in range(open_idx, len(src)):
c = src[i]
if c == "{":
depth += 1
elif c == "}":
depth -= 1
if depth == 0:
return src[idx : i + 1]
raise AssertionError(f"unbalanced braces in {signature!r}")
class TestSlashModelCacheInvalidator:
"""``static/commands.js`` must export the helper to ``window`` so
``static/panels.js`` can drop the slash-command cache without poking
module-local lets across module boundaries."""
def test_invalidator_helper_defined(self):
src = _read_static("commands.js")
assert "function _invalidateSlashModelCache(" in src, (
"_invalidateSlashModelCache helper missing from static/commands.js. "
"Without it static/panels.js cannot drop the /model slash-command "
"cache when a provider is added/removed (#1539)."
)
def test_invalidator_clears_both_cache_slots(self):
src = _read_static("commands.js")
body = _extract_function_body(src, "function _invalidateSlashModelCache(")
# Cache slots from static/commands.js:84-85 — keep both null'd.
assert "_slashModelCache=null" in body, (
"_invalidateSlashModelCache must null _slashModelCache so the next "
"/model autocomplete refetches /api/models."
)
assert "_slashModelCachePromise=null" in body, (
"_invalidateSlashModelCache must null _slashModelCachePromise so an "
"in-flight load doesn't resolve into the stale cache slot after "
"invalidation."
)
def test_invalidator_exposed_on_window(self):
src = _read_static("commands.js")
# Exposed on window via a typeof-guarded assignment so the module is
# also importable in headless test contexts (vm.runInContext) that
# don't define a window global.
assert "window._invalidateSlashModelCache=_invalidateSlashModelCache" in src, (
"_invalidateSlashModelCache must be exposed on window so static/panels.js "
"can invoke it across module boundaries."
)
assert "typeof window!=='undefined'" in src, (
"The window-export assignment must be guarded by `typeof window!=='undefined'` "
"so static/commands.js stays importable in headless vm contexts (the "
"tests/test_cli_only_slash_commands.py harness has no window global)."
)
class TestProviderRemoveInvalidatesDropdowns:
"""The remove path in ``static/panels.js`` must trigger the dropdown-cache
flush and rebuild — otherwise the dropped provider lingers in every
/model dropdown until the page reloads (#1539)."""
def test_remove_path_invokes_dropdown_flush(self):
src = _read_static("panels.js")
body = _extract_function_body(src, "async function _removeProviderKey(")
assert "_refreshModelDropdownsAfterProviderChange()" in body, (
"_removeProviderKey must call _refreshModelDropdownsAfterProviderChange() "
"after a successful delete. Without this, the JS-side caches "
"(_slashModelCache, _dynamicModelLabels, _configuredModelBadges) "
"still offer the deleted provider's models until reload (#1539)."
)
def test_save_path_invokes_dropdown_flush(self):
"""Defense-in-depth: adding a key has the same staleness shape — the
new provider's models won't show up until reload without this call.
Bundled in #1539."""
src = _read_static("panels.js")
body = _extract_function_body(src, "async function _saveProviderKey(")
assert "_refreshModelDropdownsAfterProviderChange()" in body, (
"_saveProviderKey must also call _refreshModelDropdownsAfterProviderChange() "
"so a newly-configured provider's models appear in every dropdown "
"without a reload. Same staleness shape as the remove path (#1539)."
)
def test_dropdown_flush_helper_defined(self):
src = _read_static("panels.js")
assert "function _refreshModelDropdownsAfterProviderChange(" in src, (
"_refreshModelDropdownsAfterProviderChange must be defined in "
"static/panels.js (single helper used by both save + remove paths)."
)
def test_dropdown_flush_calls_slash_cache_invalidator(self):
src = _read_static("panels.js")
body = _extract_function_body(src, "function _refreshModelDropdownsAfterProviderChange(")
# Must invoke the commands.js helper — directly poking module-local
# lets across module boundaries is brittle.
assert "_invalidateSlashModelCache" in body, (
"_refreshModelDropdownsAfterProviderChange must call "
"window._invalidateSlashModelCache() so the /model slash-command "
"cache is dropped (covers the slash-command surface from #1539)."
)
def test_dropdown_flush_calls_populate_model_dropdown(self):
src = _read_static("panels.js")
body = _extract_function_body(src, "function _refreshModelDropdownsAfterProviderChange(")
assert "populateModelDropdown" in body, (
"_refreshModelDropdownsAfterProviderChange must call "
"populateModelDropdown() so the composer model picker, Settings → "
"Default Model dropdown, _dynamicModelLabels, and "
"_configuredModelBadges all rebuild from a fresh /api/models "
"response (covers the dropdown + badge surfaces from #1539)."
)
def test_dropdown_flush_is_resilient_to_missing_modules(self):
"""If commands.js or ui.js failed to load, the providers panel must
still update — the dropdown flush is best-effort (#1539)."""
src = _read_static("panels.js")
body = _extract_function_body(src, "function _refreshModelDropdownsAfterProviderChange(")
# Outer try/catch wraps the whole helper so a runtime error inside
# populateModelDropdown / cache flush cannot surface as an unhandled
# rejection that breaks the surrounding save/remove flow.
assert re.search(r"\btry\s*\{", body), (
"_refreshModelDropdownsAfterProviderChange must wrap its work in "
"try/catch — if commands.js or ui.js failed to load, a missing "
"function should not break the providers panel update (#1539)."
)
# And the populateModelDropdown call must be guarded by typeof — the
# dropdown rebuild is best-effort.
assert "typeof populateModelDropdown" in body, (
"populateModelDropdown lookup must use typeof so it gracefully "
"skips when ui.js hasn't loaded yet."
)
def test_dropdown_flush_does_not_block_panel_refresh(self):
"""populateModelDropdown is async; its result must not be awaited
synchronously inside the helper — otherwise a slow /api/models would
delay the providers panel re-render (#1539)."""
src = _read_static("panels.js")
body = _extract_function_body(src, "function _refreshModelDropdownsAfterProviderChange(")
# The helper itself is non-async (signature checked indirectly: the
# source begins with 'function _refresh...', not 'async function').
# Anything async is fired with Promise.resolve(...).catch(...) so the
# provider panel re-render is not blocked.
assert body.startswith("function _refreshModelDropdownsAfterProviderChange"), (
"_refreshModelDropdownsAfterProviderChange should be a sync helper "
"that fires-and-forgets populateModelDropdown — not an async one "
"the save/remove paths await."
)
class TestServerSideInvariantPreserved:
"""Server-side ``invalidate_models_cache()`` is the load-bearing invariant
that lets the next /api/models request return correct data; #1539 was a
pure frontend bug, but pin the server-side wiring so a refactor of
``set_provider_key`` cannot silently regress it."""
def test_set_provider_key_invalidates_cache(self):
src = (REPO / "api" / "providers.py").read_text(encoding="utf-8")
# set_provider_key is the canonical write path — both add and remove
# flow through it (remove_provider_key calls set_provider_key(pid, None)).
m = re.search(
r"def set_provider_key\([^)]*\).*?(?=\ndef |\Z)",
src,
re.DOTALL,
)
assert m, "set_provider_key not found in api/providers.py"
body = m.group(0)
assert "invalidate_models_cache()" in body, (
"set_provider_key must call invalidate_models_cache() so the "
"server-side TTL cache is flushed on every add/remove. Without "
"this, even a perfectly-cached frontend would receive stale data."
)