Maintainer review found regression 2 on commit 575996a7: the new
_has_provider_route gate (api/config.py) required api/base_url/api_key/key_env
before admitting a configured provider into the picker, which dropped a
models-only custom provider config (the lmstudio-style shape from #1970,
tests/test_pr1970_lmstudio_base_url_fallback.py::test_provider_catalog_preserves_dict_shaped_raw_key_lookup).
A naive fix (admit any config with models) would re-break
test_unknown_duplicate_copilot_provider_config_is_not_rendered, since a
spurious alias like copilot-2: {name: copilot, models: {...}} must stay
rejected.
Fix: admit a models-only provider config as evidence only when its canonical
id matches the active/configured provider (threaded via active_provider,
already in scope). Non-active models-only configs (including duplicate
aliases of known providers) are still rejected.
Added a regression test:
tests/test_pr1970_lmstudio_base_url_fallback.py::test_provider_catalog_rejects_non_active_models_only_custom_provider
covering both the active-admits and non-active-rejects cases side by side.
Verification:
./scripts/test.sh tests/test_issue5057_moa_webui_route.py tests/test_copilot_provider_model_settings_not_allowlist.py tests/test_moa_model_picker_provider.py tests/test_pr1970_lmstudio_base_url_fallback.py -q
26 passed, 1 skipped in 5.58s
Broader sanity pass: ./scripts/test.sh tests/ -k "config or provider or model or copilot or moa or lmstudio" -q
1728 passed, 5 skipped, 9654 deselected (4 pre-existing failures confirmed unrelated: 2 reproduce identically on the pre-fix commit via git stash, 2 are order-dependent and pass in isolation).
CI on Python 3.13 (clean editable install, no hermes_cli package) was still
failing the 3 lmstudio tests after the first fix attempt. Root cause: the
outer try/except in the lmstudio branch was catching ImportError from
`from hermes_cli.models import provider_model_ids`, hijacking the whole
branch and silently skipping the urlopen fallback.
Restructured into two independent tiers:
1. hermes_cli lookup in its own try/except — ImportError logs at DEBUG
and continues with lm_ids=[].
2. urlopen fallback runs unconditionally when lm_ids is empty, including
after hermes_cli import failure.
New regression test `test_lmstudio_fallback_works_when_hermes_cli_unavailable`
explicitly blocks hermes_cli via sys.meta_path and verifies the lmstudio
group still populates from the urlopen fallback. Without this test, the
CI-vs-local divergence (local env had hermes_cli installed, CI didn't)
would keep slipping through.
All 12 lmstudio-related tests pass, including the 3 #1527 tests that
broke on stage-337.
PR #1970 added a dedicated `elif pid == "lmstudio":` branch in
`get_available_models()` that fetches the live /v1/models list when the
hermes_cli helper doesn't have ids cached. The fallback path inside that
branch only looked at `cfg["providers"]["lmstudio"]["base_url"]`, missing
the historical config shape where the URL lives under `cfg["model"]`:
model:
provider: lmstudio
base_url: http://192.168.1.22:1234/v1 ← here, not under providers.lmstudio
providers:
lmstudio:
api_key: local-key
3 pre-existing tests in tests/test_issue1527_lmstudio_base_url_classification
broke on stage-337 because of this — they passed on master, failed after
the PR #1970 merge.
The simpler fix is to enhance the already-introduced `_get_provider_base_url()`
helper so it falls back to `cfg["model"]["base_url"]` when
`cfg["model"]["provider"] == provider_id`, then use the helper inside the
lmstudio branch instead of a direct lookup. This keeps the previous
behaviour (where the generic configured-provider branch handled lmstudio
via the model block) while preserving PR #1970's live-discovery additions.
Belt-and-suspenders: `_get_provider_base_url()` explicitly does NOT inherit
model.base_url for providers other than the active one — if a user's config
says `model.provider: anthropic` and they have `providers.openai` configured
without a base_url, openai must still resolve to None (use SDK default),
not to the anthropic proxy URL.
6 new regression tests in tests/test_pr1970_lmstudio_base_url_fallback.py
lock the two-location lookup, the precedence rule (explicit providers entry
wins over model fallback), trailing-slash stripping, and the negative case
(model.base_url MUST NOT leak to non-active providers).
All 51 tests in the existing model-resolver + custom-provider banks still
pass.
Caught by maintainer review on stage-337 (full pytest with the new network
isolation in place surfaced the regression that the fork-CI mock-server path
would have hidden).
Rod Boev