Re-review #7165 (nesquena-hermes 2026-09-23T06:21:57Z, CHANGES_REQUESTED).
CORE: a dict-shaped custom_providers[].models mapping (what Hermes Agent's
setup flow writes, e.g. {chat-a: {context_length: 128000}}) was treated as an
allowlist by iterating its keys, collapsing the live /v1/models catalog to a
single saved default while the CLI live-probed the full list. The Agent defines
exactly this contract in hermes_cli/model_switch.py::_models_config_is_allowlist:
a mapping is per-model metadata, NOT a catalog narrow.
Fix (all 3 spec points):
1. api/routes.py::_custom_provider_allowlist_ids: added
_custom_provider_models_discover_is_false(_cp) (mirrors
api.config._provider_discover_allowed / Agent
model_switch_providers.py:557: bool False or string false/no/0,
case-insensitive). In the dict branch, when discovery is NOT explicitly
false, return no allowlist so the live probe returns the full catalog.
Only discover_models: false opts into treating dict keys as a pinned
allowlist. List / JSON-array-string / Python-literal shapes stay plain
allowlists.
2. tests/test_issue3718_live_models_custom_probe.py: removed the
mapping-metadata param that asserted the regression; added two
real-handler tests: test_dict_models_without_discover_false_does_not_gate
(dict -> full _LIVE_CATALOG) and test_dict_models_with_discover_false_pins_to_keys
(bool False / "false" / "no" / "0" -> pinned to keys).
3. docs/architecture/live-models-allowlist.md: documented that list and
string shapes filter, and a dict filters only with discover_models: false.
Companion test update (blocker reported by the fix itself):
tests/test_byok_model_dropdown.py rightcode-codex entry encoded the removed
dict-as-allowlist behavior (models: {"gpt-5.5-mini": {}} asserting the response
collapsed to just that key). Added discover_models: False to preserve the
provider-scoping intent (no sibling leak) under the new contract.
Red/green verified: removing the dict-branch guard turns
test_dict_models_without_discover_false_does_not_gate red (collapsed catalog);
restoring it turns it green.
Verification: 3718 18 passed; byok 24 passed; neighbour sweep (46 files)
520 passed (2 pre-existing order-dependent flakes reproduce identically on
clean HEAD, verified via git stash); ruff curated gate OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
