Files
hermes-webui/tests
nesquena-hermesandnesquena-hermes 1dd5cc43ce fix(tests): isolate model-selection env vars so the suite passes on any runner (#6395)
The suite failed 6 tests (test_sprint12 default-model, test_issue1538 Nous
catalog, test_issue1567 picker capacity/symmetry) whenever it ran on a box that
exports HERMES_MODEL (e.g. inside a live Hermes agent session where HERMES_MODEL
is the agent's runtime model). Root cause: get_effective_default_model() in
api/config.py treats HERMES_MODEL / OPENAI_MODEL / LLM_MODEL as the
highest-priority default-model source — above the isolated test config — so a
runner-exported value overrode the sandbox config and polluted the model
picker/catalog (e.g. injecting "opus-4.8" and breaking the @nous: prefix
invariant / default-model readback).

Fix: scrub the three model-selection env vars at conftest module level (before
any product module imports, covering in-process tests) and again in the
out-of-process test_server subprocess env, and add them to the per-test
_scrub_provider_env helpers in the two Nous test modules. The isolated config /
HERMES_WEBUI_DEFAULT_MODEL is now authoritative regardless of the runner env.

Verified: with HERMES_MODEL=opus-4.8 set, the full suite goes from 6 failed to
0 failed (13550 passed); non-vacuity confirmed (disabling the scrub reproduces
the failures).

Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
2026-07-21 00:00:52 -07:00
..