fix(#3750): preserve LM Studio auth on reasoning probe (self-rebased #3837 + credential hardening) (#4202)
* fix(#3750): preserve LM Studio auth on reasoning probe Rebased rodboev's #3837 onto current master (master reworked resolve_model_reasoning_efforts via #3431 with a combined hermes_cli import + _nested_route_reasoning_denied guard). Threaded the api_key resolution + the _lmstudio_model_reasoning_options wrapper + the HTTP-fallback probe into master's current structure; intent preserved byte-identically (+164/-24, same as the original PR diff). Co-authored-by: Rod Boev <rod.boev@gmail.com> * fix(#3837): gate LM Studio probe credential to configured endpoint + no-redirect opener Codex security review (CORE) on the rebased #3837: GET /api/reasoning accepts a caller-supplied base_url, and the new probe attached the configured LM Studio API key as a Bearer token; urllib also re-sends Authorization across redirects. Together that could exfiltrate the stored credential to an attacker-controlled host. (master only escalates here because master probed WITHOUT a key.) Fixes: 1. Only forward the LM Studio credential when the probe target normalizes to the configured LM Studio base URL (reuses _normalize_base_url_for_match); a caller-supplied non-matching base_url is probed keyless. 2. _lmstudio_reasoning_probe_options_fallback uses a no-redirect opener (_NoRedirectHandler) so a 3xx can't forward the Authorization header onward. 3. Two regression tests: caller-supplied base_url drops the key; redirect is not followed (credential never reaches the redirect target). Co-authored-by: Rod Boev <rod.boev@gmail.com> * fix(#3837): route ALL credentialed LM Studio probes through the no-redirect probe Codex re-gate caught that the first fix only hardened the built-in fallback; in production hermes_cli IS importable, so the preferred path called hermes_cli.models.lmstudio_model_reasoning_options(..., api_key=...) which uses plain urlopen and still follows redirects with the credential. Fix: _lmstudio_model_reasoning_options now short-circuits to the built-in no-redirect probe whenever api_key is set; the hermes_cli path is used ONLY for keyless probes (no credential to leak). Reworked tests accordingly: - test_credentialed_probe_never_calls_hermes_cli: credentialed probe bypasses CLI - test_keyless_probe_logs_signature_mismatch_before_fallback: keyless CLI degrade - test_credentialed_probe_does_not_follow_redirects_end_to_end: E2E through resolve_model_reasoning_efforts with real hermes_cli; redirect target never hit Verified both security tests fail without the guard and pass with it. Co-authored-by: Rod Boev <rod.boev@gmail.com> * chore(release): stamp v0.51.420 (Release OG) in CHANGELOG --------- Co-authored-by: nesquena-hermes <agent@nesquena-hermes> Co-authored-by: Rod Boev <rod.boev@gmail.com>
This commit is contained in:
3 files changed
+850
-25
No files matched your search
@@ -3,6 +3,12 @@
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [v0.51.420] — 2026-06-14 — Release OG (LM Studio reasoning-probe auth, #3750/#3837)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **LM Studio reasoning-effort probes now keep their auth and work without the bundled CLI.** When the active model is an LM Studio provider, the WebUI reasoning-effort probe now resolves an LM Studio API key (active model key → `providers.lmstudio.api_key` → `LM_API_KEY` → `LMSTUDIO_API_KEY`) and forwards it as a Bearer token, so authenticated LM Studio servers no longer 401 the probe and silently report "no reasoning support." If `hermes_cli` is unavailable (or its `lmstudio_model_reasoning_options` signature changes), the probe falls back to a built-in HTTP query of the LM Studio `/api/v1/models` capabilities endpoint instead of skipping reasoning detection entirely. The configured LM Studio credential is only ever sent to the configured LM Studio endpoint (a caller-supplied `base_url` that does not match is probed without the key), and the probe refuses HTTP redirects so the credential can never be forwarded to another host. (#3750, #3837)
|
||||
|
||||
## [v0.51.419] — 2026-06-14 — Release OF (settled-stream message-count consistency, #4192)
|
||||
|
||||
### Fixed
|
||||
|
||||
+218
-25
@@ -21,6 +21,8 @@ import sys
|
||||
import threading
|
||||
import time
|
||||
import traceback
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from urllib.parse import parse_qs, urlparse
|
||||
@@ -2623,6 +2625,184 @@ def _models_dev_reasoning_efforts(model_id: str, provider_id: str) -> list[str]
|
||||
return None
|
||||
|
||||
|
||||
class _NoRedirectHandler(urllib.request.HTTPRedirectHandler):
|
||||
"""urllib redirect handler that refuses to follow any redirect.
|
||||
|
||||
Used by the LM Studio reasoning probe so a 3xx from the probe URL can never
|
||||
forward the ``Authorization`` header (the configured LM Studio key) to a
|
||||
redirected, possibly attacker-controlled host. ``redirect_request``
|
||||
returning ``None`` makes urllib raise the original 3xx as an ``HTTPError``,
|
||||
which the probe swallows. (#3837 security review)
|
||||
"""
|
||||
|
||||
def redirect_request(self, req, fp, code, msg, headers, newurl):
|
||||
return None
|
||||
|
||||
|
||||
def _get_lmstudio_reasoning_probe_api_key() -> str | None:
|
||||
"""Resolve the LM Studio key for reasoning probes with WebUI precedence."""
|
||||
config_data = cfg
|
||||
model_cfg = config_data.get("model") or {}
|
||||
if isinstance(model_cfg, dict):
|
||||
active_provider = str(model_cfg.get("provider") or "").strip().lower()
|
||||
model_key = str(model_cfg.get("api_key") or "").strip()
|
||||
if active_provider == "lmstudio" and model_key:
|
||||
return model_key
|
||||
|
||||
providers_cfg = config_data.get("providers") or {}
|
||||
if isinstance(providers_cfg, dict):
|
||||
lmstudio_cfg = providers_cfg.get("lmstudio") or {}
|
||||
if isinstance(lmstudio_cfg, dict):
|
||||
config_key = str(lmstudio_cfg.get("api_key") or "").strip()
|
||||
if config_key:
|
||||
return config_key
|
||||
|
||||
env_key = str(os.getenv("LM_API_KEY") or "").strip()
|
||||
if env_key:
|
||||
return env_key
|
||||
|
||||
legacy_env_key = str(os.getenv("LMSTUDIO_API_KEY") or "").strip()
|
||||
if legacy_env_key:
|
||||
return legacy_env_key
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _lmstudio_reasoning_probe_options_fallback(
|
||||
model: str,
|
||||
base_url: str | None,
|
||||
*,
|
||||
api_key: str | None = None,
|
||||
timeout: float = 5.0,
|
||||
) -> list[str]:
|
||||
"""Query LM Studio reasoning options without relying on hermes_cli."""
|
||||
server_root = str(base_url or "").strip().rstrip("/")
|
||||
if server_root.endswith("/v1"):
|
||||
server_root = server_root[:-3].rstrip("/")
|
||||
if not server_root or not model:
|
||||
return []
|
||||
|
||||
headers = {
|
||||
"Accept": "application/json",
|
||||
"User-Agent": "hermes-webui-reasoning-probe",
|
||||
}
|
||||
if api_key:
|
||||
headers["Authorization"] = f"Bearer {api_key}"
|
||||
|
||||
request = urllib.request.Request(
|
||||
server_root + "/api/v1/models",
|
||||
headers=headers,
|
||||
method="GET",
|
||||
)
|
||||
# SECURITY: never follow redirects on this probe. urllib re-sends request
|
||||
# headers (including Authorization: Bearer <lmstudio key>) to the redirect
|
||||
# target, so a 3xx from the probe URL could exfiltrate the configured
|
||||
# LM Studio credential to an attacker-controlled host. A no-redirect opener
|
||||
# turns any 3xx into an HTTPError we swallow below. (#3837 security review)
|
||||
opener = urllib.request.build_opener(_NoRedirectHandler)
|
||||
try:
|
||||
with opener.open(request, timeout=timeout) as response: # nosec B310
|
||||
payload = json.loads(response.read().decode("utf-8", errors="replace"))
|
||||
except urllib.error.HTTPError as exc:
|
||||
logger.debug(
|
||||
"LM Studio reasoning probe at %s failed with HTTP %s",
|
||||
server_root,
|
||||
exc.code,
|
||||
)
|
||||
return []
|
||||
except Exception as exc:
|
||||
logger.debug("LM Studio reasoning probe at %s failed: %s", server_root, exc)
|
||||
return []
|
||||
|
||||
raw_models = payload.get("models") if isinstance(payload, dict) else None
|
||||
if not isinstance(raw_models, list):
|
||||
logger.debug(
|
||||
"LM Studio reasoning probe at %s returned malformed payload",
|
||||
server_root,
|
||||
)
|
||||
return []
|
||||
|
||||
for raw in raw_models:
|
||||
if not isinstance(raw, dict):
|
||||
continue
|
||||
if raw.get("key") != model and raw.get("id") != model:
|
||||
continue
|
||||
caps = raw.get("capabilities")
|
||||
reasoning = caps.get("reasoning") if isinstance(caps, dict) else None
|
||||
opts = reasoning.get("allowed_options") if isinstance(reasoning, dict) else None
|
||||
if isinstance(opts, list):
|
||||
return [str(opt).strip().lower() for opt in opts if isinstance(opt, str)]
|
||||
return []
|
||||
return []
|
||||
|
||||
|
||||
def _lmstudio_model_reasoning_options(
|
||||
model: str,
|
||||
base_url: str | None,
|
||||
*,
|
||||
api_key: str | None = None,
|
||||
timeout: float = 5.0,
|
||||
) -> list[str]:
|
||||
"""Prefer hermes_cli, but keep WebUI reasoning probes working without it.
|
||||
|
||||
SECURITY: when an ``api_key`` is being sent, always use the built-in
|
||||
no-redirect fallback probe rather than ``hermes_cli``. The bundled CLI probe
|
||||
uses a plain ``urllib.request.urlopen`` that follows redirects and re-sends
|
||||
the ``Authorization`` header to the redirect target, which could leak the
|
||||
configured LM Studio credential to another host. We can only guarantee
|
||||
redirect safety for code we control, so credentialed probes go through
|
||||
``_lmstudio_reasoning_probe_options_fallback`` (no-redirect opener). Keyless
|
||||
probes have no credential to leak, so they may use the richer CLI path.
|
||||
(#3837 security review)
|
||||
"""
|
||||
if api_key:
|
||||
return _lmstudio_reasoning_probe_options_fallback(
|
||||
model,
|
||||
base_url,
|
||||
api_key=api_key,
|
||||
timeout=timeout,
|
||||
)
|
||||
|
||||
try:
|
||||
from hermes_cli.models import (
|
||||
lmstudio_model_reasoning_options as _cli_lmstudio_model_reasoning_options,
|
||||
)
|
||||
except Exception:
|
||||
return _lmstudio_reasoning_probe_options_fallback(
|
||||
model,
|
||||
base_url,
|
||||
api_key=api_key,
|
||||
timeout=timeout,
|
||||
)
|
||||
|
||||
try:
|
||||
return _cli_lmstudio_model_reasoning_options(
|
||||
model,
|
||||
base_url,
|
||||
api_key=api_key,
|
||||
timeout=timeout,
|
||||
)
|
||||
except (TypeError, AttributeError):
|
||||
logger.warning(
|
||||
"hermes_cli.lmstudio_model_reasoning_options has an unexpected signature; "
|
||||
"falling back to the built-in LM Studio reasoning probe",
|
||||
exc_info=True,
|
||||
)
|
||||
return _lmstudio_reasoning_probe_options_fallback(
|
||||
model,
|
||||
base_url,
|
||||
api_key=api_key,
|
||||
timeout=timeout,
|
||||
)
|
||||
except Exception:
|
||||
return _lmstudio_reasoning_probe_options_fallback(
|
||||
model,
|
||||
base_url,
|
||||
api_key=api_key,
|
||||
timeout=timeout,
|
||||
)
|
||||
|
||||
|
||||
def resolve_model_reasoning_efforts(
|
||||
model_id: str | None = None,
|
||||
provider_id: str | None = None,
|
||||
@@ -2650,34 +2830,47 @@ def resolve_model_reasoning_efforts(
|
||||
if _nested_route_reasoning_denied(hinted_model):
|
||||
return []
|
||||
|
||||
try:
|
||||
from hermes_cli.models import (
|
||||
github_model_reasoning_efforts,
|
||||
lmstudio_model_reasoning_options,
|
||||
)
|
||||
except Exception:
|
||||
if provider in {"copilot", "github-copilot"}:
|
||||
if provider in {"copilot", "github-copilot"}:
|
||||
try:
|
||||
from hermes_cli.models import github_model_reasoning_efforts
|
||||
except Exception:
|
||||
return _heuristic_reasoning_efforts(hinted_model, provider)
|
||||
else:
|
||||
if provider in {"copilot", "github-copilot"}:
|
||||
return _filter_reasoning_efforts_for_provider(
|
||||
github_model_reasoning_efforts(hinted_model), hinted_model, provider
|
||||
)
|
||||
return _filter_reasoning_efforts_for_provider(
|
||||
github_model_reasoning_efforts(hinted_model), hinted_model, provider
|
||||
)
|
||||
|
||||
if provider == "lmstudio":
|
||||
probe_base = resolved_base_url or _get_provider_base_url(provider)
|
||||
opts = lmstudio_model_reasoning_options(hinted_model, probe_base)
|
||||
normalized = [str(opt).strip().lower() for opt in opts if str(opt).strip()]
|
||||
if not normalized or set(normalized).issubset({"off"}):
|
||||
return []
|
||||
level_opts = [opt for opt in normalized if opt in VALID_REASONING_EFFORTS]
|
||||
if level_opts:
|
||||
return _filter_reasoning_efforts_for_provider(
|
||||
level_opts, hinted_model, provider
|
||||
)
|
||||
if set(normalized).issubset({"off", "on"}):
|
||||
return []
|
||||
if provider == "lmstudio":
|
||||
configured_base = _get_provider_base_url(provider)
|
||||
probe_base = resolved_base_url or configured_base
|
||||
# SECURITY: only forward the configured LM Studio credential when the
|
||||
# probe target is the configured LM Studio endpoint. /api/reasoning
|
||||
# accepts a caller-supplied base_url, so a request could otherwise point
|
||||
# the probe at an arbitrary host and harvest the stored key. When the
|
||||
# caller supplies a base_url that does not normalize to the configured
|
||||
# one, probe it WITHOUT a key. (#3837 security review)
|
||||
probe_key: str | None = None
|
||||
if not resolved_base_url or (
|
||||
configured_base
|
||||
and _normalize_base_url_for_match(probe_base)
|
||||
== _normalize_base_url_for_match(configured_base)
|
||||
):
|
||||
probe_key = _get_lmstudio_reasoning_probe_api_key()
|
||||
opts = _lmstudio_model_reasoning_options(
|
||||
hinted_model,
|
||||
probe_base,
|
||||
api_key=probe_key,
|
||||
)
|
||||
normalized = [str(opt).strip().lower() for opt in opts if str(opt).strip()]
|
||||
if not normalized or set(normalized).issubset({"off"}):
|
||||
return []
|
||||
level_opts = [opt for opt in normalized if opt in VALID_REASONING_EFFORTS]
|
||||
if level_opts:
|
||||
return _filter_reasoning_efforts_for_provider(
|
||||
level_opts, hinted_model, provider
|
||||
)
|
||||
if set(normalized).issubset({"off", "on"}):
|
||||
return []
|
||||
return []
|
||||
|
||||
# _models_dev_reasoning_efforts already applies the provider/model filter
|
||||
# internally, so it is returned as-is here (filtering again would be
|
||||
|
||||
@@ -0,0 +1,626 @@
|
||||
"""Regression tests for #3750 LM Studio reasoning probes dropping auth."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
|
||||
import pytest
|
||||
|
||||
import api.config as config
|
||||
import api.onboarding as onboarding
|
||||
import api.profiles as profiles
|
||||
|
||||
|
||||
_API_KEY_ENV_VARS = (
|
||||
"LM_API_KEY",
|
||||
"LMSTUDIO_API_KEY",
|
||||
)
|
||||
|
||||
|
||||
class _LmStudioProbeServer:
|
||||
def __init__(self):
|
||||
self.requests: list[dict[str, str | None]] = []
|
||||
self._server = HTTPServer(("127.0.0.1", 0), self._handler())
|
||||
self._thread = threading.Thread(target=self._server.serve_forever, daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
@property
|
||||
def base_v1(self) -> str:
|
||||
return f"http://127.0.0.1:{self._server.server_address[1]}/v1"
|
||||
|
||||
def close(self) -> None:
|
||||
self._server.shutdown()
|
||||
self._server.server_close()
|
||||
self._thread.join(timeout=5)
|
||||
|
||||
def _handler(self):
|
||||
parent = self
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
auth = self.headers.get("Authorization")
|
||||
parent.requests.append(
|
||||
{
|
||||
"path": self.path,
|
||||
"authorization": auth,
|
||||
}
|
||||
)
|
||||
|
||||
if self.path == "/api/v1/models":
|
||||
if auth:
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(
|
||||
json.dumps(
|
||||
{
|
||||
"models": [
|
||||
{
|
||||
"key": "auth-model",
|
||||
"capabilities": {
|
||||
"reasoning": {
|
||||
"allowed_options": ["low", "medium", "high"]
|
||||
}
|
||||
},
|
||||
}
|
||||
]
|
||||
}
|
||||
).encode()
|
||||
)
|
||||
return
|
||||
|
||||
self.send_response(401)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(json.dumps({"error": "unauthorized"}).encode())
|
||||
return
|
||||
|
||||
if self.path == "/v1/models":
|
||||
if auth:
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(json.dumps({"data": [{"id": "auth-model"}]}).encode())
|
||||
return
|
||||
|
||||
self.send_response(401)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(
|
||||
json.dumps({"error": {"message": "unauthorized"}}).encode()
|
||||
)
|
||||
return
|
||||
|
||||
self.send_response(404)
|
||||
self.end_headers()
|
||||
|
||||
def log_message(self, fmt, *args):
|
||||
return None
|
||||
|
||||
return Handler
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _isolate_config(monkeypatch, tmp_path):
|
||||
old_cfg = dict(config.cfg)
|
||||
old_mtime = config._cfg_mtime
|
||||
old_path = config._cfg_path
|
||||
old_fp = config._cfg_fingerprint
|
||||
monkeypatch.setattr(profiles, "get_active_hermes_home", lambda: tmp_path)
|
||||
monkeypatch.setenv("BROWSER", "echo")
|
||||
for var in _API_KEY_ENV_VARS:
|
||||
monkeypatch.delenv(var, raising=False)
|
||||
config.invalidate_models_cache()
|
||||
yield
|
||||
config.cfg.clear()
|
||||
config.cfg.update(old_cfg)
|
||||
config._cfg_mtime = old_mtime
|
||||
config._cfg_path = old_path
|
||||
config._cfg_fingerprint = old_fp
|
||||
config.invalidate_models_cache()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def lmstudio_probe_server():
|
||||
server = _LmStudioProbeServer()
|
||||
try:
|
||||
yield server
|
||||
finally:
|
||||
server.close()
|
||||
|
||||
|
||||
def _write_config(tmp_path, monkeypatch, text: str) -> None:
|
||||
cfgfile = tmp_path / "config.yaml"
|
||||
cfgfile.write_text(text, encoding="utf-8")
|
||||
monkeypatch.setattr(config, "_get_config_path", lambda: cfgfile)
|
||||
config.reload_config()
|
||||
config.invalidate_models_cache()
|
||||
|
||||
|
||||
def test_reasoning_probe_uses_config_key_before_env_aliases(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
lmstudio_probe_server,
|
||||
):
|
||||
_write_config(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
f"""
|
||||
model:
|
||||
provider: lmstudio
|
||||
default: auth-model
|
||||
base_url: {lmstudio_probe_server.base_v1}
|
||||
providers:
|
||||
lmstudio:
|
||||
api_key: config-token
|
||||
agent:
|
||||
reasoning_effort: medium
|
||||
display:
|
||||
show_reasoning: true
|
||||
""",
|
||||
)
|
||||
monkeypatch.setenv("LM_API_KEY", "env-token")
|
||||
monkeypatch.setenv("LMSTUDIO_API_KEY", "legacy-token")
|
||||
|
||||
status = config.get_reasoning_status()
|
||||
|
||||
assert status["supports_reasoning_effort"] is True
|
||||
assert status["supported_efforts"] == ["low", "medium", "high"]
|
||||
assert lmstudio_probe_server.requests[0] == {
|
||||
"path": "/api/v1/models",
|
||||
"authorization": "Bearer config-token",
|
||||
}
|
||||
|
||||
|
||||
def test_reasoning_probe_honors_active_model_api_key(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
lmstudio_probe_server,
|
||||
):
|
||||
_write_config(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
f"""
|
||||
model:
|
||||
provider: lmstudio
|
||||
default: auth-model
|
||||
base_url: {lmstudio_probe_server.base_v1}
|
||||
api_key: model-token
|
||||
providers:
|
||||
lmstudio:
|
||||
api_key: provider-token
|
||||
agent:
|
||||
reasoning_effort: medium
|
||||
display:
|
||||
show_reasoning: true
|
||||
""",
|
||||
)
|
||||
|
||||
status = config.get_reasoning_status()
|
||||
|
||||
assert status["supports_reasoning_effort"] is True
|
||||
assert status["supported_efforts"] == ["low", "medium", "high"]
|
||||
assert lmstudio_probe_server.requests[0] == {
|
||||
"path": "/api/v1/models",
|
||||
"authorization": "Bearer model-token",
|
||||
}
|
||||
|
||||
|
||||
def test_reasoning_probe_falls_back_without_hermes_cli(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
lmstudio_probe_server,
|
||||
):
|
||||
blocked_modules = [
|
||||
name
|
||||
for name in list(sys.modules)
|
||||
if name == "hermes_cli" or name.startswith("hermes_cli.")
|
||||
]
|
||||
for name in blocked_modules:
|
||||
monkeypatch.delitem(sys.modules, name, raising=False)
|
||||
|
||||
class _Blocker:
|
||||
def find_module(self, name, path=None):
|
||||
if name == "hermes_cli" or name.startswith("hermes_cli."):
|
||||
return self
|
||||
return None
|
||||
|
||||
def load_module(self, name):
|
||||
raise ImportError(f"hermes_cli blocked for test: {name}")
|
||||
|
||||
blocker = _Blocker()
|
||||
sys.meta_path.insert(0, blocker)
|
||||
try:
|
||||
_write_config(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
f"""
|
||||
model:
|
||||
provider: lmstudio
|
||||
default: auth-model
|
||||
base_url: {lmstudio_probe_server.base_v1}
|
||||
providers:
|
||||
lmstudio:
|
||||
api_key: config-token
|
||||
agent:
|
||||
reasoning_effort: medium
|
||||
display:
|
||||
show_reasoning: true
|
||||
""",
|
||||
)
|
||||
|
||||
status = config.get_reasoning_status()
|
||||
|
||||
assert status["supports_reasoning_effort"] is True
|
||||
assert status["supported_efforts"] == ["low", "medium", "high"]
|
||||
assert lmstudio_probe_server.requests[0] == {
|
||||
"path": "/api/v1/models",
|
||||
"authorization": "Bearer config-token",
|
||||
}
|
||||
finally:
|
||||
try:
|
||||
sys.meta_path.remove(blocker)
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
|
||||
def test_reasoning_probe_stays_keyless_when_no_key_is_configured(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
lmstudio_probe_server,
|
||||
):
|
||||
_write_config(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
f"""
|
||||
model:
|
||||
provider: lmstudio
|
||||
default: auth-model
|
||||
base_url: {lmstudio_probe_server.base_v1}
|
||||
agent:
|
||||
reasoning_effort: medium
|
||||
display:
|
||||
show_reasoning: true
|
||||
""",
|
||||
)
|
||||
|
||||
status = config.get_reasoning_status()
|
||||
|
||||
assert status["supports_reasoning_effort"] is False
|
||||
assert status["supported_efforts"] == []
|
||||
assert lmstudio_probe_server.requests[0] == {
|
||||
"path": "/api/v1/models",
|
||||
"authorization": None,
|
||||
}
|
||||
|
||||
|
||||
def test_onboarding_probe_remains_authorized_control(
|
||||
lmstudio_probe_server,
|
||||
):
|
||||
result = onboarding.probe_provider_endpoint(
|
||||
"lmstudio",
|
||||
lmstudio_probe_server.base_v1,
|
||||
api_key="control-token",
|
||||
)
|
||||
|
||||
assert result["ok"] is True
|
||||
assert lmstudio_probe_server.requests[0] == {
|
||||
"path": "/v1/models",
|
||||
"authorization": "Bearer control-token",
|
||||
}
|
||||
|
||||
|
||||
def test_credentialed_probe_never_calls_hermes_cli(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
lmstudio_probe_server,
|
||||
):
|
||||
"""When a credential is configured, the probe must use the built-in
|
||||
no-redirect fallback and NEVER the bundled hermes_cli probe (which follows
|
||||
redirects and could forward the key). (#3837 security review)
|
||||
"""
|
||||
_write_config(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
f"""
|
||||
model:
|
||||
provider: lmstudio
|
||||
default: auth-model
|
||||
base_url: {lmstudio_probe_server.base_v1}
|
||||
providers:
|
||||
lmstudio:
|
||||
api_key: config-token
|
||||
agent:
|
||||
reasoning_effort: medium
|
||||
display:
|
||||
show_reasoning: true
|
||||
""",
|
||||
)
|
||||
|
||||
cli_calls: list[tuple] = []
|
||||
|
||||
class _CliModule:
|
||||
@staticmethod
|
||||
def lmstudio_model_reasoning_options(model, base_url, api_key=None, timeout=5.0):
|
||||
cli_calls.append((model, base_url, api_key))
|
||||
return ["low", "medium", "high"]
|
||||
|
||||
monkeypatch.setitem(sys.modules, "hermes_cli", type("HermesCli", (), {})())
|
||||
monkeypatch.setitem(sys.modules, "hermes_cli.models", _CliModule)
|
||||
|
||||
status = config.get_reasoning_status()
|
||||
|
||||
assert status["supports_reasoning_effort"] is True
|
||||
assert status["supported_efforts"] == ["low", "medium", "high"]
|
||||
# The credential reached the configured endpoint via the built-in probe …
|
||||
assert lmstudio_probe_server.requests[0] == {
|
||||
"path": "/api/v1/models",
|
||||
"authorization": "Bearer config-token",
|
||||
}
|
||||
# … and the redirect-following hermes_cli probe was never invoked.
|
||||
assert cli_calls == [], "credentialed probe must bypass hermes_cli"
|
||||
|
||||
|
||||
def test_keyless_probe_logs_signature_mismatch_before_fallback(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
lmstudio_probe_server,
|
||||
):
|
||||
"""Keyless probes still prefer hermes_cli; an incompatible CLI signature
|
||||
logs a warning and degrades to the built-in probe. (#3750)
|
||||
"""
|
||||
_write_config(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
f"""
|
||||
model:
|
||||
provider: lmstudio
|
||||
default: auth-model
|
||||
base_url: {lmstudio_probe_server.base_v1}
|
||||
agent:
|
||||
reasoning_effort: medium
|
||||
display:
|
||||
show_reasoning: true
|
||||
""",
|
||||
)
|
||||
|
||||
captured: list[tuple[str, bool]] = []
|
||||
|
||||
class _CliModule:
|
||||
@staticmethod
|
||||
def lmstudio_model_reasoning_options(model, base_url):
|
||||
raise TypeError("unexpected keyword argument: api_key")
|
||||
|
||||
monkeypatch.setitem(sys.modules, "hermes_cli", type("HermesCli", (), {})())
|
||||
monkeypatch.setitem(sys.modules, "hermes_cli.models", _CliModule)
|
||||
monkeypatch.setattr(
|
||||
config.logger,
|
||||
"warning",
|
||||
lambda msg, *args, **kwargs: captured.append((str(msg), bool(kwargs.get("exc_info")))),
|
||||
)
|
||||
|
||||
# No key configured -> CLI path is attempted, raises TypeError, warning is
|
||||
# logged, then the built-in (keyless) probe runs against the auth-requiring
|
||||
# test server, which 401s -> []. The point is the warning + degrade path.
|
||||
status = config.get_reasoning_status()
|
||||
|
||||
assert status["supports_reasoning_effort"] is False
|
||||
assert status["supported_efforts"] == []
|
||||
assert lmstudio_probe_server.requests[-1]["authorization"] is None
|
||||
assert captured, "TypeError fallback must log a warning before probing directly"
|
||||
assert "unexpected signature" in captured[0][0]
|
||||
assert captured[0][1] is True
|
||||
|
||||
|
||||
def test_reasoning_probe_drops_key_for_caller_supplied_base_url(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
lmstudio_probe_server,
|
||||
):
|
||||
"""A caller-supplied base_url that is NOT the configured LM Studio endpoint
|
||||
must be probed WITHOUT the stored credential. /api/reasoning takes a
|
||||
base_url query param, so attaching the key to an arbitrary host would leak
|
||||
it. (#3837 security review)
|
||||
"""
|
||||
_write_config(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
f"""
|
||||
model:
|
||||
provider: lmstudio
|
||||
default: auth-model
|
||||
base_url: {lmstudio_probe_server.base_v1}
|
||||
providers:
|
||||
lmstudio:
|
||||
api_key: config-token
|
||||
agent:
|
||||
reasoning_effort: medium
|
||||
display:
|
||||
show_reasoning: true
|
||||
""",
|
||||
)
|
||||
|
||||
# Point the probe at the SAME server but via an unconfigured base_url. The
|
||||
# configured base_url is the only one allowed to carry the key. The probe
|
||||
# server returns 401 without auth, so reasoning is reported unsupported and,
|
||||
# crucially, no Bearer token reaches the caller-supplied URL.
|
||||
efforts = config.resolve_model_reasoning_efforts(
|
||||
"auth-model",
|
||||
provider_id="lmstudio",
|
||||
base_url=lmstudio_probe_server.base_v1.replace("127.0.0.1", "localhost"),
|
||||
)
|
||||
|
||||
assert efforts == []
|
||||
assert lmstudio_probe_server.requests, "probe should still be attempted keyless"
|
||||
assert lmstudio_probe_server.requests[0]["authorization"] is None
|
||||
|
||||
|
||||
def test_reasoning_probe_does_not_follow_redirects_with_credential(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
):
|
||||
"""A 3xx from the configured probe URL must NOT forward the Authorization
|
||||
header to the redirect target. urllib re-sends request headers across
|
||||
redirects, so the probe uses a no-redirect opener. (#3837 security review)
|
||||
"""
|
||||
captured: list[dict[str, str | None]] = []
|
||||
|
||||
class _Target(BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
captured.append(
|
||||
{"host": "target", "authorization": self.headers.get("Authorization")}
|
||||
)
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(json.dumps({"models": []}).encode())
|
||||
|
||||
def log_message(self, fmt, *args):
|
||||
return None
|
||||
|
||||
class _Redirector(BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
captured.append(
|
||||
{"host": "redirector", "authorization": self.headers.get("Authorization")}
|
||||
)
|
||||
self.send_response(302)
|
||||
self.send_header(
|
||||
"Location", f"http://127.0.0.1:{target.server_address[1]}/api/v1/models"
|
||||
)
|
||||
self.end_headers()
|
||||
|
||||
def log_message(self, fmt, *args):
|
||||
return None
|
||||
|
||||
target = HTTPServer(("127.0.0.1", 0), _Target)
|
||||
target_thread = threading.Thread(target=target.serve_forever, daemon=True)
|
||||
target_thread.start()
|
||||
redir = HTTPServer(("127.0.0.1", 0), _Redirector)
|
||||
redir_thread = threading.Thread(target=redir.serve_forever, daemon=True)
|
||||
redir_thread.start()
|
||||
try:
|
||||
result = config._lmstudio_reasoning_probe_options_fallback(
|
||||
"auth-model",
|
||||
f"http://127.0.0.1:{redir.server_address[1]}/v1",
|
||||
api_key="secret-token",
|
||||
)
|
||||
finally:
|
||||
redir.shutdown()
|
||||
redir.server_close()
|
||||
redir_thread.join(timeout=5)
|
||||
target.shutdown()
|
||||
target.server_close()
|
||||
target_thread.join(timeout=5)
|
||||
|
||||
# The probe stops at the redirector (no-redirect opener) and never reaches
|
||||
# the redirect target, so the credential is never forwarded onward.
|
||||
assert result == []
|
||||
hosts_hit = {c["host"] for c in captured}
|
||||
assert "target" not in hosts_hit, "redirect must not be followed"
|
||||
assert all(c["host"] == "redirector" for c in captured)
|
||||
|
||||
|
||||
def test_credentialed_probe_does_not_follow_redirects_end_to_end(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
):
|
||||
"""End-to-end through resolve_model_reasoning_efforts() with the REAL
|
||||
hermes_cli importable: when the CONFIGURED LM Studio endpoint returns a 302,
|
||||
the credentialed probe must not forward the key to the redirect target.
|
||||
This is the production path (hermes_cli present) — credentialed probes route
|
||||
through the built-in no-redirect probe, so the redirect is refused.
|
||||
(#3837 security review)
|
||||
"""
|
||||
captured: list[dict[str, str | None]] = []
|
||||
|
||||
class _Target(BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
captured.append(
|
||||
{"host": "target", "authorization": self.headers.get("Authorization")}
|
||||
)
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(
|
||||
json.dumps(
|
||||
{
|
||||
"models": [
|
||||
{
|
||||
"key": "auth-model",
|
||||
"capabilities": {
|
||||
"reasoning": {"allowed_options": ["low", "high"]}
|
||||
},
|
||||
}
|
||||
]
|
||||
}
|
||||
).encode()
|
||||
)
|
||||
|
||||
def log_message(self, fmt, *args):
|
||||
return None
|
||||
|
||||
class _Redirector(BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
captured.append(
|
||||
{"host": "redirector", "authorization": self.headers.get("Authorization")}
|
||||
)
|
||||
self.send_response(302)
|
||||
self.send_header(
|
||||
"Location", f"http://127.0.0.1:{target.server_address[1]}/api/v1/models"
|
||||
)
|
||||
self.end_headers()
|
||||
|
||||
def log_message(self, fmt, *args):
|
||||
return None
|
||||
|
||||
target = HTTPServer(("127.0.0.1", 0), _Target)
|
||||
target_thread = threading.Thread(target=target.serve_forever, daemon=True)
|
||||
target_thread.start()
|
||||
redir = HTTPServer(("127.0.0.1", 0), _Redirector)
|
||||
redir_thread = threading.Thread(target=redir.serve_forever, daemon=True)
|
||||
redir_thread.start()
|
||||
|
||||
# Sanity: this test only proves the redirect guard if hermes_cli is
|
||||
# importable (the production path). If it isn't, the keyless/credentialed
|
||||
# split still routes credentialed probes through the no-redirect fallback,
|
||||
# so the assertion holds either way.
|
||||
redir_base = f"http://127.0.0.1:{redir.server_address[1]}/v1"
|
||||
_write_config(
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
f"""
|
||||
model:
|
||||
provider: lmstudio
|
||||
default: auth-model
|
||||
base_url: {redir_base}
|
||||
providers:
|
||||
lmstudio:
|
||||
api_key: secret-token
|
||||
agent:
|
||||
reasoning_effort: medium
|
||||
display:
|
||||
show_reasoning: true
|
||||
""",
|
||||
)
|
||||
try:
|
||||
efforts = config.resolve_model_reasoning_efforts(
|
||||
"auth-model",
|
||||
provider_id="lmstudio",
|
||||
base_url=redir_base,
|
||||
)
|
||||
finally:
|
||||
redir.shutdown()
|
||||
redir.server_close()
|
||||
redir_thread.join(timeout=5)
|
||||
target.shutdown()
|
||||
target.server_close()
|
||||
target_thread.join(timeout=5)
|
||||
|
||||
assert efforts == []
|
||||
hosts_hit = {c["host"] for c in captured}
|
||||
assert "target" not in hosts_hit, "credentialed redirect must not be followed"
|
||||
# The credential only ever touched the configured (redirector) endpoint.
|
||||
assert all(c["host"] == "redirector" for c in captured)
|
||||
Reference in new issue
Block a user