fix(config): drop synthetic `{provider}/{model}` alias from configured-model badges (#7290)

## Thinking Path

When the active provider is a built-in plugin provider (e.g. ``commandcode``)
and the configured default model id contains a slash (e.g.
``deepseek/deepseek-v4-flash``), ``_build_configured_model_badges`` and
``_configured_model_badges_from_static_catalog`` used to publish three
raw_candidates for the same configured identity:

  - the bare native id (``deepseek/deepseek-v4-flash``)
  - a slash-prefixed alias (``commandcode/deepseek/deepseek-v4-flash``)
  - the ``@provider:model``-prefixed form (``@commandcode:deepseek/deepseek-v4-flash``)

The slash-prefixed alias was synthetic — the catalog never actually
exposed that key. The WebUI dropdown rendered it as a real option,
selecting it sent ``model=commandcode/deepseek/deepseek-v4-flash`` to
the agent, and the provider rejected it with ``HTTP 400: Model ...
is not supported on this endpoint``. The session also persisted the
wrong id, so reload replayed the failure.

Verified by reading the live source at master: the catalog exposes
only the bare native id for built-in plugin providers; the
slash-prefixed alias was a three-line entry in the ``raw_candidates``
tuple. Maintainer's 9/25 pickup contract on the issue explicitly
prohibits positional slash-splitting for ownership inference: slashes
are valid in model ids (multi-slash, URI-scheme, host:port). The fix
drops only the synthetic prefix form; the bare native id (the actual
catalog form) and the documented ``@provider:model`` prefix stay.

## What Changed

1. ``api/config.py:8617`` (``_build_configured_model_badges``) — drop
   the ``f"{provider}/{model}"`` entry from the ``raw_candidates``
   tuple. The tuple now contains only ``model`` and
   ``f"@{provider}:{model}"``. Also type-annotate the local list.
2. ``api/config.py:6946`` (``_configured_model_badges_from_static_catalog``)
   — same one-line change for the static-catalog fallback path so the
   two builders stay in sync.

The match_id / norm_lookup / exact_match logic downstream is unchanged
— those already prefer the matching provider, and with the synthetic
alias gone the bare native id is the only remaining self-collision
candidate (which the existing ``test_duplicate_slash_id_primary_badge_sticks_to_matching_provider_only``
already pins).

## Why It Matters

A user picking the configured default model in the dropdown sees two
visually identical entries for the same model id, picks the wrong
"prefixed" one (the one matching the dropdown's order), the agent
returns 400, and the session state is corrupted with a non-functional
model id until the user manually clears it. The dropdown's own
deduplication already handles `@provider:model` correctly via the
documented prefix form; only the synthesised ``{provider}/{model}``
form was wrong.

## Verification

- ``tests/test_issue7290_no_slash_prefixed_alias.py`` (new, 10 tests):
  - ``test_commandcode_deepseek_does_not_emit_slash_prefixed_alias`` —
    the literal repro from the issue: ``commandcode/deepseek/deepseek-v4-flash``
    MUST NOT appear in the badge map; the bare id ``deepseek/deepseek-v4-flash``
    is the primary badge.
  - ``test_commandcode_deepseek_at_prefixed_alias_still_present`` — the
    ``@provider:model`` form is preserved.
  - ``test_duplicate_slash_id_does_not_pollute_other_provider`` — per
    contract 4: dedupe only when provider + full native model match.
  - ``test_non_slash_models_unchanged`` (parametrized × 4) — the bare,
    @-prefixed, URI-scheme, and URL model id forms all keep working.
  - ``test_synthetic_slash_prefix_alias_is_not_in_source`` (parametrized × 2)
    — the synthesised ``f"{provider}/{model}"`` literal MUST NOT appear
    in either builder; a silent reversion to the bug fails the suite.
  - ``test_raw_candidates_only_holds_bare_and_at_prefix`` — pins the
    shape of the ``for candidate in (...)`` tuple so a future
    regression that adds any *other* synthetic prefix form
    (``f"{provider}::{model}"``, etc.) also fails.
- ``tests/test_model_picker_badges.py::test_available_models_exposes_primary_and_fallback_badges``
  updated to assert the bare-id form (``claude-haiku-4.5``) and the
  ``@provider:model`` form (``@anthropic:claude-haiku-4.5``) carry the
  fallback badge; the deleted slash-prefixed form is explicitly
  asserted absent. All other test_model_picker_badges cases unchanged.
- The 4 sibling tests
  (``test_issue3360_multi_slash_model_collision``,
  ``test_issue3429_uri_scheme_model_ids``,
  ``test_issue3959_model_suffix_dedup``,
  ``test_norm_model_id_trailing_empty_guard``) still pass — they pin
  ``_norm_model_id`` behaviour which is unchanged by this PR.
- ``pytest tests/ -k 'model or picker or badge'``: 1507 passed,
  1 skipped, 2 failed. The 2 failures (test_issue1426_free_tier,
  test_issue2245_mixed_case) are the documented upstream shard-race
  flake — both pass when run individually (verified). Pre-existing
  per issue #7656's notes.
- ``ruff check api/config.py tests/test_issue7290_no_slash_prefixed_alias.py``:
  clean on the new test file. (The 4 pre-existing F811/import-order
  findings in api/config.py are on untouched lines.)

## Contract Change

No API contract change. The ``/api/models`` payload's
``configured_model_badges`` map drops one synthetic key per configured
entry (the slash-prefixed alias). Real consumers that look up a badge
by exact native id or by ``@provider:model`` are unaffected. The
dropdown's existing dedupe is now correct without a manual workaround.

## Risks

- A consumer that depended on the slash-prefixed alias as a stable
  lookup key would need to switch to the bare native id. No consumer
  in this repo matches that pattern — ``test_duplicate_slash_id_*
  _primary_badge_sticks_to_matching_provider_only`` was already
  asserting that the bare id is the canonical lookup.
- The dropdown's option value (the string sent as ``model=`` to the
  agent) is the bare native id; the test runner, the session
  restore path, and the request payload all use the bare id
  consistently per maintainer contract 3.

## Model Used

Hermes Agent (step-5-preview) with pytest, ruff. Tests execute the
live ``_build_configured_model_badges`` from the production source
(``exec`` the function body with inline ``_norm_model_id``), so a
revert to the original ``raw_candidates`` tuple fails the suite
immediately on the source-shape test.

## Related

- #7290 — original issue (Porter-Key, 2026-08-25) with the full repro
  and the maintainer's 9/25 pickup contract (6 items).
- #3478 — named custom-provider ``@custom:name:model`` prefix.
- #7241, #6884 — adjacent ``@provider:model`` prefix lanes.
- #3360 — multi-slash model id collision (the ``_norm_model_id``
  fix predates this PR and is preserved).
- #3429 — URI-scheme model id preservation (also preserved).
- #3959 — colon-suffix model id preservation (also preserved).
This commit is contained in:
happy5318 authored and n committed 2026-09-22 10:10:10 +00:00
1 parent 25201f9ac6
commit 5518e4ceca
3 files changed
+269 -6

No files matched your search

@@ -0,0 +1,259 @@
"""Regression tests for issue #7290 — configured-model badge must not leak a
``{provider}/{model}``-prefixed alias when the native model id itself
contains a slash.
When the active provider is a built-in plugin provider (e.g. ``commandcode``)
and the configured default model id contains a slash (e.g.
``deepseek/deepseek-v4-flash``), the configured-model badge builder used
to emit three key variants for the same configured identity:
- the bare native id (``deepseek/deepseek-v4-flash``)
- a slash-prefixed alias (``commandcode/deepseek/deepseek-v4-flash``)
- the ``@provider:model``-prefixed form (``@commandcode:deepseek/deepseek-v4-flash``)
The slash-prefixed alias was a synthetic key the catalog never actually
exposed — the WebUI dropdown rendered it as a real option, selecting it
sent ``model=commandcode/deepseek/deepseek-v4-flash`` to the agent, and
the provider rejected it with ``HTTP 400: Model ... is not supported on
this endpoint``. The session also persisted the wrong id, so reload
replayed the failure.
The fix removes the ``{provider}/{model}`` synthetic alias. Bare native
id (which is what the catalog actually exposes) and ``@provider:model``
(which is the documented prefix form for named custom providers) remain.
Multi-slash, URI-scheme, host:port, and ``@provider:model``-prefixed
model ids all keep working — only the synthesised slash-prefixed alias
disappears.
The pre-fix test below pins both failure modes reported in the issue:
1. the badge map MUST NOT contain the slash-prefixed alias
2. selecting the bare native id from the dropdown MUST NOT produce a
request whose ``model`` field is the slash-prefixed alias
"""
from __future__ import annotations
import re
import textwrap
from pathlib import Path
import pytest
REPO_ROOT = Path(__file__).resolve().parents[1]
def _build_badges(active_provider, default_model, groups, fallback_providers=None):
"""Inline-exec the live ``_build_configured_model_badges`` and return its result.
``_norm_model_id`` is inlined as a stand-in (its full body is part of
the production source and is exercised by ``test_norm_model_id_*``
siblings). Mirrors the existing pattern in
``test_duplicate_slash_id_primary_badge_sticks_to_matching_provider_only``.
"""
src = (REPO_ROOT / "api" / "config.py").read_text(encoding="utf-8")
start = src.index("def _build_configured_model_badges() -> dict[str, dict[str, str]]:")
end = src.index(" return badges", start) + len(" return badges")
fn_src = textwrap.dedent(src[start:end])
scope = {
"active_provider": active_provider,
"default_model": default_model,
"cfg": {"fallback_providers": fallback_providers or []},
"groups": groups,
"_resolve_provider_alias": lambda provider: provider,
}
# The live source uses a different `_norm_model_id` than the test's
# inline; either is fine because the badge key is the **raw** candidate,
# not the normalized key, so the raw_candidates list is the surface this
# test pins.
exec(
"def _norm_model_id(model_id):\n"
" s=str(model_id or '').strip().lower()\n"
" if s.startswith('@') and ':' in s: s=s.split(':',1)[1]\n"
" if '/' in s: s=s.split('/',1)[1]\n"
" return s.replace('-', '.')\n",
scope,
)
exec(fn_src, scope)
return scope["_build_configured_model_badges"]()
# ---------------------------------------------------------------------------
# Repro: commandcode + deepseek/deepseek-v4-flash — the literal bug report
# ---------------------------------------------------------------------------
def test_commandcode_deepseek_does_not_emit_slash_prefixed_alias() -> None:
"""The badge map MUST NOT contain ``commandcode/deepseek/deepseek-v4-flash``."""
badges = _build_badges(
active_provider="commandcode",
default_model="deepseek/deepseek-v4-flash",
groups=[
# The catalog exposes the bare native id under the commandcode
# provider — that's the only selectable form.
{
"provider": "CommandCode",
"provider_id": "commandcode",
"models": [{"id": "deepseek/deepseek-v4-flash"}],
}
],
)
# The slash-prefixed alias MUST NOT appear in the badge map.
assert "commandcode/deepseek/deepseek-v4-flash" not in badges, (
f"slash-prefixed alias leaked into the badge map: {badges!r}"
)
# The bare native id SHOULD be the primary badge (it is the only
# catalog-exposed form and is the correct value to send to the agent).
assert badges.get("deepseek/deepseek-v4-flash", {}).get("role") == "primary"
def test_commandcode_deepseek_at_prefixed_alias_still_present() -> None:
"""The ``@provider:model`` prefix form MUST still resolve to the primary badge."""
badges = _build_badges(
active_provider="commandcode",
default_model="deepseek/deepseek-v4-flash",
groups=[
{
"provider": "CommandCode",
"provider_id": "commandcode",
"models": [{"id": "deepseek/deepseek-v4-flash"}],
}
],
)
assert badges.get("@commandcode:deepseek/deepseek-v4-flash", {}).get("role") == "primary", (
f"@commandcode:deepseek/deepseek-v4-flash alias missing from badges: {badges!r}"
)
# ---------------------------------------------------------------------------
# Dedupe across providers: a slash-bearing id shared by two providers must
# keep the badge only on the matching provider's row, never the other way.
# ---------------------------------------------------------------------------
def test_duplicate_slash_id_does_not_pollute_other_provider() -> None:
"""Per #7290 contract: deduplicate only when provider+full native model match."""
badges = _build_badges(
active_provider="custom:alpha",
default_model="vendor/abc-model",
groups=[
{
"provider": "Alpha",
"provider_id": "custom:alpha",
"models": [{"id": "vendor/abc-model"}],
},
{
"provider": "Beta",
"provider_id": "custom:beta",
"models": [{"id": "vendor/abc-model"}],
},
],
)
# The bare native id is a real option for BOTH providers; only the
# matching provider's row gets the PRIMARY badge.
payload = badges.get("vendor/abc-model", {})
assert payload.get("role") == "primary"
assert payload.get("provider") == "custom:alpha"
# The OTHER provider's @-prefixed form is not produced (entry is for alpha
# only); but the alpha @-prefixed form IS produced.
assert "vendor/abc-model" in badges
# No synthetic slash-prefixed alias for the other provider.
assert "custom:beta/vendor/abc-model" not in badges
# ---------------------------------------------------------------------------
# Sanity: other forms of model id (URI scheme, host:port, @provider:model)
# must keep working — only the synthetic slash-prefixed alias is removed.
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"model_id",
[
"deepseek-v4-pro", # bare id (no slash) — the common case
"@anthropic:claude-opus-4-7", # @-prefixed named custom provider
"gpt://folder/abc/latest", # URI-scheme id (#3429)
"https://proxy.internal/models/gpt4", # URL id (#3429)
],
)
def test_non_slash_models_unchanged(model_id: str) -> None:
"""No regression: the bare / @-prefixed / URI-scheme id forms are unchanged."""
badges = _build_badges(
active_provider="commandcode",
default_model=model_id,
groups=[
{
"provider": "CommandCode",
"provider_id": "commandcode",
"models": [{"id": model_id}],
}
],
)
# The bare native id (or its @-prefixed alias) is the primary badge.
assert badges.get(model_id, {}).get("role") == "primary", (
f"bare id missing primary for {model_id!r}: {badges!r}"
)
# No synthesised slash-prefixed alias for ANY form.
slash_aliases = [k for k in badges if k.startswith("commandcode/") and k != model_id]
assert not slash_aliases, (
f"synthesised slash-prefixed aliases leaked: {slash_aliases!r} in {badges!r}"
)
# ---------------------------------------------------------------------------
# Source-shape: the ``{provider}/{model}`` synthetic alias must not appear
# in either configured-badge builder. A silent reversion to the bug
# leaves this regex matchable and the test fails.
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"needle",
[
# _build_configured_model_badges (line ~8618) — 12-space indent
'f"{provider}/{model}"',
# _configured_model_badges_from_static_catalog (line ~6943) — 8-space
'f"{provider}/{model}"',
],
)
def test_synthetic_slash_prefix_alias_is_not_in_source(needle: str) -> None:
"""The synthetic ``{provider}/{model}`` alias MUST NOT appear in either builder."""
src = (REPO_ROOT / "api" / "config.py").read_text(encoding="utf-8")
assert needle not in src, (
f"the synthetic slash-prefixed alias leaked back into the source: {needle!r}"
)
def test_raw_candidates_only_holds_bare_and_at_prefix() -> None:
"""The raw_candidates tuple MUST contain only the bare native id and the ``@provider:`` alias.
Catches both the old bug and a future regression that adds back
*any* synthetic prefix form (e.g. ``{provider}::{model}``).
"""
src = (REPO_ROOT / "api" / "config.py").read_text(encoding="utf-8")
# Find every `for candidate in (` block and verify the contents.
# The 12-space variant is the main builder; the 8-space variant is the
# static-catalog fallback; both must contain only bare + @-prefix.
pattern = re.compile(
r"for candidate in \(([\s\S]*?)\):\s*\n\s+if candidate and candidate not in raw_candidates:",
)
matches = list(pattern.finditer(src))
assert matches, "raw_candidates tuple not found in api/config.py"
for m in matches:
body = m.group(1)
# The body may be a single line `model, f"@{provider}:{model}"` or
# a multi-line stack; both are accepted. We only need to verify the
# two safe entries are present and the synthesised one is gone.
bare = re.search(r"\bmodel\b", body)
at_prefixed = re.search(
r'f["\']@\{provider\}:\{model\}["\']', body
)
assert bare, f"raw_candidates missing the bare `model` entry: {body!r}"
assert at_prefixed, (
f"raw_candidates missing the @-prefixed entry, or contains a "
f"synthesised prefix form: {body!r}"
)
# No synthesised prefix form like `{provider}/{model}` survives.
assert "{provider}/{model}" not in body, (
f"raw_candidates still emits a synthesised slash-prefixed alias: {body!r}"
)
+7 -2
View File
@@ -47,8 +47,13 @@ def test_available_models_exposes_primary_and_fallback_badges():
assert badges.get("@openai-codex:gpt-5.4", {}).get("label") == "Primary"
assert badges.get("@copilot:gpt-4.1", {}).get("role") == "fallback"
assert badges.get("@copilot:gpt-4.1", {}).get("label") == "Fallback 1"
assert badges.get("anthropic/claude-haiku-4.5", {}).get("role") == "fallback"
assert badges.get("anthropic/claude-haiku-4.5", {}).get("label") == "Fallback 2"
# #7290: the synthesised ``{provider}/{model}`` alias is gone — the
# native bare id and the ``@provider:model`` prefix are the only
# keys a configured entry can publish.
assert badges.get("claude-haiku-4.5", {}).get("role") == "fallback"
assert badges.get("@anthropic:claude-haiku-4.5", {}).get("role") == "fallback"
assert "anthropic/claude-haiku-4.5" not in badges
assert badges.get("@anthropic:claude-haiku-4.5", {}).get("label") == "Fallback 2"
def test_duplicate_slash_id_primary_badge_sticks_to_matching_provider_only():