Files
hermes-webui/tests/test_issue1257_llm_wiki_status.py
T
nesquena-hermesandnesquena-hermes 2d20d336ea Release v0.51.230 — Release GX (stage-p14 — extract <think> to m.reasoning #3455 + LLM Wiki last-writer #1257) (#3466)
* Release v0.51.230 (stage-p14): extract <think> to m.reasoning #3455 + LLM Wiki last-writer (#1257)

Salvage of #3455 (@gsurenull): dropped the stale api/config.py bits (MiniMax-M3 +
SCHEMA_VERSION 3->4 — both already on master via #3374). Kept the two genuine fixes:
(1) _splitThinkFromContent persist-path extraction of inline <think> blocks into
m.reasoning (fixes 30-50% session bloat for reasoning-only providers like MiniMax-M3);
(2) LLM Wiki status Last-writer 3-tier fallback (was always 'Not available' since #1257).
Added 9 Node-driven think-split regression tests (data-loss guards: content-before/after
preserved, unclosed blocks intact, lookalike tags not extracted).

* fix(#3455): renderer-matching think extraction + wiki symlink/bounded-read guards (Codex review)

Codex review of stage-p14 found 3 SILENT bugs, all fixed:
(1) DATA-LOSS: _splitThinkFromContent's Pass-2 whole-body scan extracted a CLOSED literal
<think>...</think> from visible prose/code (e.g. inside a fenced code block) into m.reasoning,
emptying it — more aggressive than the renderer (which only strips LEADING blocks). Removed
Pass 2; extraction now matches _streamDisplay semantics (leading-only, loop captures
consecutive leading blocks). +fenced-code regression test.
(2) PRIVACY: _llm_wiki_last_writer followed symlinked .md pages resolving OUTSIDE the wiki
(is_file follows symlinks), leaking external frontmatter. Now requires resolved path under
wiki_root. +symlink-containment regression test.
(3) CONTRACT/PERF: replaced full read_text() with bounded line-by-line reads (frontmatter
block only / capped log-heading scan), never page bodies.

* fix(#3455): think-split is leading-single (renderer-matching) + fix 2 stale source-match tests

Codex re-review finding #2: looping consecutive leading blocks diverged from the renderer
(_streamDisplay/_parseStreamState strip ONE leading block). Now extracts exactly one leading
block. Also updated 2 tests that asserted pre-split implementation strings:
test_live_stream_tokens_persist (content:assistantText -> content:split.content, invariant
preserved) and the consecutive-blocks test. NOTE: Codex finding #1 (client-only split doesn't
persist server-side) is a separate architectural decision pending Nathan.

* feat(#3455): split inline <think> server-side before s.save() so persisted file is compacted (Codex #1)

Codex finding #1: the think-split was client-only, so the SAVED session file still
carried inline <think> blocks (bloat) — the fix only compacted the browser copy.
Added _split_thinking_from_content (api/streaming.py), a server-side twin of the JS
helper with identical leading-only/single-block semantics, applied to the final
assistant message before s.save() (extended the existing reasoning-persist block).
Merges with on_reasoning-stream reasoning. +8 backend-parity regression tests covering
the mid-body-code-block data-loss guard, unclosed-intact, single-leading, none-content.

* test: update 3 save-path source-assertion tests for #3455 server-side think-split

The backend think-split (api/streaming.py reasoning-persist block) changed the literal
code shape + grew the pre-save block, breaking 8 source-assertion tests that anchor on it:
- test_sprint42: assert _rm['reasoning']=_reasoning_text -> now _merged_reasoning/_existing_reasoning
  + _split_thinking_from_content present (intent preserved: reasoning persisted before save).
- test_pr1318 (6) + test_pr1341: re-anchored the locator from the changed 'if _reasoning_text
  and s.messages:' line to the stable 'Persist reasoning trace in the session' comment marker;
  bumped the 1341 byte-distance limit 15000->16000 (the test self-documents bumping on legit
  pre-save growth). All behavioral invariants (reasoning persisted + context fields before save)
  unchanged.

---------

Co-authored-by: nesquena-hermes <[email protected]>
2026-06-02 22:48:35 -07:00

140 lines
5.1 KiB
Python

from __future__ import annotations
from pathlib import Path
from types import SimpleNamespace
from urllib.parse import urlparse
from unittest.mock import patch
REPO = Path(__file__).resolve().parents[1]
def _write(path: Path, text: str = "# Synthetic\n") -> Path:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(text, encoding="utf-8")
return path
def test_llm_wiki_status_reads_synthetic_fixture_without_exposing_content(tmp_path, monkeypatch):
"""The wiki status API should summarize counts/mtime without leaking page text."""
import api.routes as routes
wiki = tmp_path / "wiki"
_write(wiki / "SCHEMA.md", "# Schema\n")
_write(wiki / "index.md", "# Index\n")
_write(wiki / "log.md", "# Log\n## [2026-05-04] update | Secret project name\n- Details stay private\n")
_write(
wiki / "entities" / "private-agent.md",
"---\ntitle: Private Agent\nupdated: 2026-05-04\n---\nSensitive body text must not ship.\n",
)
_write(wiki / "concepts" / "safe-summary.md", "---\ntitle: Safe Summary\n---\nMore private text\n")
_write(wiki / "raw" / "articles" / "source.md", "Raw source body should not count as wiki page\n")
monkeypatch.setenv("WIKI_PATH", str(wiki))
status = routes._build_llm_wiki_status()
assert status["available"] is True
assert status["enabled"] is True
assert status["entry_count"] == 2
assert status["page_count"] == 2
assert status["raw_source_count"] == 1
assert status["last_updated"] is not None
# log.md in the fixture has a "## [2026-05-04] update | ..." heading,
# so the new last-writer reader must surface that action verb.
assert status["last_writer"] == "ai-agent (update)"
assert status["toggle_available"] is False
assert status["docs_url"].endswith("/research-llm-wiki")
serialized = repr(status)
assert "Sensitive body text" not in serialized
assert "Secret project name" not in serialized
assert str(wiki) not in serialized
def test_llm_wiki_status_reports_unavailable_when_path_missing(tmp_path, monkeypatch):
import api.routes as routes
missing = tmp_path / "does-not-exist"
monkeypatch.setenv("WIKI_PATH", str(missing))
status = routes._build_llm_wiki_status()
assert status["available"] is False
assert status["enabled"] is False
assert status["entry_count"] == 0
assert status["page_count"] == 0
assert status["raw_source_count"] == 0
assert status["last_updated"] is None
assert status["status"] == "missing"
def test_api_wiki_status_route_is_registered(monkeypatch, tmp_path):
import api.routes as routes
wiki = tmp_path / "wiki"
_write(wiki / "entities" / "one.md")
monkeypatch.setenv("WIKI_PATH", str(wiki))
captured = {}
def fake_j(handler, payload, status=200, extra_headers=None):
captured["status"] = status
captured["payload"] = payload
with patch("api.routes.j", side_effect=fake_j):
handled = routes.handle_get(SimpleNamespace(), urlparse("/api/wiki/status"))
assert handled is True
assert captured["status"] == 200
assert captured["payload"]["entry_count"] == 1
def test_insights_panel_fetches_and_renders_llm_wiki_status_card():
panels_src = (REPO / "static" / "panels.js").read_text(encoding="utf-8")
index_src = (REPO / "static" / "index.html").read_text(encoding="utf-8")
style_src = (REPO / "static" / "style.css").read_text(encoding="utf-8")
assert "api('/api/wiki/status')" in panels_src
assert "function _renderLlmWikiStatus" in panels_src
assert "llmWikiStatusCard" in index_src
assert "wiki-status-card" in style_src
assert "raw/" in panels_src
assert "recent_entries" not in panels_src
def test_last_writer_reads_frontmatter(tmp_path):
"""#3455 part 2: the Last writer field reads page frontmatter updated_by/writer/author."""
import api.routes as routes
wiki = tmp_path / "wiki"
_write(wiki / "entities" / "a.md", "---\ntitle: A\nupdated_by: alice\n---\nbody\n")
pages = routes._llm_wiki_page_files(wiki)
assert routes._llm_wiki_last_writer(wiki, pages) == "alice"
def test_last_writer_rejects_symlink_outside_wiki(tmp_path):
"""#3455 review (Codex): a symlinked .md page resolving OUTSIDE the wiki must
not be read — its frontmatter must never leak into the status card."""
import api.routes as routes
outside = tmp_path / "outside"
outside.mkdir(parents=True, exist_ok=True)
secret = outside / "private.md"
secret.write_text("---\nauthor: outside-secret\n---\nprivate body\n", encoding="utf-8")
wiki = tmp_path / "wiki" / "entities"
wiki.mkdir(parents=True, exist_ok=True)
link = wiki / "linked.md"
try:
link.symlink_to(secret)
except (OSError, NotImplementedError):
import pytest
pytest.skip("symlinks not supported on this platform")
wiki_root = tmp_path / "wiki"
pages = routes._llm_wiki_page_files(wiki_root)
writer = routes._llm_wiki_last_writer(wiki_root, pages)
# The external symlink's frontmatter must NOT surface; falls back to ai-agent.
assert writer != "outside-secret"
assert "outside-secret" not in writer