Files
hermes-webui/tests/test_issue3401_deep_review_fixes.py
T
e3a7c93dc6 [HELD — independent review pending] Release v0.51.294 — stage-3401 (live-to-final redesign #3401 + 4 deep-review fixes) (#3741)
* Harden interrupted recovery control filtering

* Redesign live-to-final assistant replies

* Fix live activity anchor test fixture

* Fix CI lint issues for live reply tests

* Strengthen live progress prompt contract

* Recover PR #3401 refresh on origin/master

* Repair live-to-final refresh regressions

* Fix live worklog refresh regressions

* Show live footer timer on initial stream start

* Restore live stream shell after reload

* Preserve per-frame live SSE replay cursors

* Preserve reasoning as Worklog Thinking cards

* Quiet Worklog Thinking card styling

* Align Worklog Thinking card styling

* Scope live Worklog Thinking cards by segment

* Suppress exact duplicate settled Thinking

* Close #3401 merge review test gaps

* fix(#3401): resolve 4 deep-review regressions (inline-think, reconnect-dup, neon skin, busy-gate worklog)

Deep review (Codex diff-vs-master + live-browser drive) of the live-to-final refactor
surfaced 4 regressions vs master that the rewritten suite no longer guarded:

1. Inline <think>…</think>answer reasoning vanished — _assistantReasoningPayloadText
   used $-anchored regexes so a leading think block + visible answer extracted nothing
   and the Thinking card never rendered. Removed the 3 $ anchors to match the
   (non-anchored) display stripper. Live: inline-think thinking-only turn now renders.
2. (CORE) reconnect/reload duplicated the live reply — _rememberRunJournalCursor advanced
   a closure-local seq but never wrote INFLIGHT[activeSid].lastRunJournalSeq, so a reload
   replayed the journal from after_seq=0 over restored lastAssistantText. Now mirrors the
   cursor onto INFLIGHT + schedules a throttled persist.
3. Neon skin silently broke — PR deleted the :root[data-skin="neon"] CSS but left Neon in
   the picker. Restored the neon CSS block from master.
4. Settled tool-worklog rebuild gated purely on !S.busy — dropped every prior settled
   turn's worklog when renderMessages re-ran during an active stream (switch-back to an
   in-progress session). Restored master's !S.busy || (S.toolCalls && S.toolCalls.length).
   Live: busy re-render now preserves tool cards (4→4, was 4→0).

Live-verified all 4 + confirmed #3709/#3592 invariants still hold (1 thinking card, none
below footer; distinct siblings preserved). + tests/test_issue3401_deep_review_fixes.py (7).

* test(#3401): realign 3 stale source-shape assertions to the deep-review fixes

Fix commit changed two source literals that existing stage tests scanned for:
- test_live_activity_timeline.py (x2): split anchor 'if(!S.busy){' → the restored
  'if(!S.busy || (S.toolCalls&&S.toolCalls.length)){' guard (fix 4).
- test_run_journal_frontend_static.py: 'after_seq=0' not in source — fix 2's comment
  contained that literal; rephrased the comment to 'the zero floor (after_seq of 0)'.
Intent of all three assertions unchanged; only the matched string updated. No code
behavior change.

* docs(changelog): v0.51.294 — Release JJ (stage-3401, #3401 live-to-final redesign)

---------

Co-authored-by: Frank Song <franksong2702@gmail.com>
Co-authored-by: Nathan-Hermes <nesquena-hermes@users.noreply.github.com>
Co-authored-by: nesquena-hermes <[email protected]>
2026-06-06 12:12:37 -07:00

150 lines
7.5 KiB
Python

"""Regression coverage for the three #3401 deep-review fixes (release stage-3401).
#3401 (live-to-final assistant reply redesign) is a large refactor of the highest-traffic
render/stream surface. Its full test suite was green and Opus passed it, but a Codex
diff-vs-master regression read + a live-browser drive surfaced three regressions the
rewritten tests no longer guarded:
FIX 1 — inline ``<think>…</think>answer`` reasoning vanished. ``_assistantReasoningPayloadText``
used ``$``-anchored regexes, so a leading think block followed by a visible answer
extracted NO reasoning and the Thinking card never rendered (master used the
non-anchored form). The matching display-stripper is non-anchored, so the extractor
must be too.
FIX 2 — reconnect/reload duplicated the live reply. ``_rememberRunJournalCursor`` advanced a
closure-local seq but never wrote ``INFLIGHT[activeSid].lastRunJournalSeq`` — the value
``persistInflightState`` saves and a reload reads back as the ``after_seq`` replay
floor. So a hard reload restored ``lastAssistantText`` then replayed the journal from
``after_seq=0`` on top of it.
FIX 3 — the shipped Neon skin silently stopped working. The PR deleted the
``:root[data-skin="neon"]`` CSS while leaving Neon registered in the picker, so users
could select it and get default styling.
These are static source-structure / registration assertions plus a general
"every registered skin has CSS" guard so a dropped-skin regression fails fast.
"""
from __future__ import annotations
import re
from pathlib import Path
REPO = Path(__file__).resolve().parent.parent
UI_JS = (REPO / "static" / "ui.js").read_text(encoding="utf-8")
MESSAGES_JS = (REPO / "static" / "messages.js").read_text(encoding="utf-8")
CSS = (REPO / "static" / "style.css").read_text(encoding="utf-8")
BOOT_JS = (REPO / "static" / "boot.js").read_text(encoding="utf-8")
INDEX_HTML = (REPO / "static" / "index.html").read_text(encoding="utf-8")
def _function_body(src: str, name: str) -> str:
match = re.search(rf"function\s+{re.escape(name)}\s*\(", src)
assert match, f"{name}() not found"
brace = src.find("{", match.end())
depth = 1
i = brace + 1
while i < len(src) and depth:
if src[i] == "{":
depth += 1
elif src[i] == "}":
depth -= 1
i += 1
return src[brace + 1:i - 1]
# ── FIX 1: inline <think> reasoning extraction must be non-anchored ──
def test_reasoning_payload_extractor_is_not_dollar_anchored():
"""A leading <think> block followed by visible answer text must still yield the
reasoning. A trailing `$` anchor dropped it whenever the turn also had an answer."""
body = _function_body(UI_JS, "_assistantReasoningPayloadText")
# The think/thought/turn extraction regexes must NOT end with `$` after the close tag.
assert "<\\/think>\\s*$/" not in body and "</think>\\s*$/" not in body, (
"the <think> extraction regex must not be $-anchored (drops reasoning when a "
"visible answer follows the think block) — #3401 inline-think regression"
)
assert "<channel\\|>\\s*$/" not in body, "channel-thought extraction must not be $-anchored"
assert "<turn\\|>\\s*$/" not in body, "turn-thinking extraction must not be $-anchored"
# And the non-anchored leading-block form must be present.
assert "<think>([\\s\\S]*?)<\\/think>\\s*/" in body, (
"extractor must match a LEADING <think> block (non-anchored), mirroring "
"_stripLeadingAssistantThinkingMarkup"
)
def test_extractor_and_stripper_anchoring_agree():
"""The reasoning extractor and the display-content stripper must use the same
(non-anchored, leading-block) matching, or one shows a card the other can't strip."""
strip = _function_body(UI_JS, "_stripLeadingAssistantThinkingMarkup")
extract = _function_body(UI_JS, "_assistantReasoningPayloadText")
# Neither should be $-anchored on the think close tag.
assert "</think>\\s*$/" not in strip and "<\\/think>\\s*$/" not in strip
assert "</think>\\s*$/" not in extract and "<\\/think>\\s*$/" not in extract
# ── FIX 2: reconnect cursor must persist into INFLIGHT ──
def test_run_journal_cursor_persisted_into_inflight():
"""_rememberRunJournalCursor must write the advanced seq onto INFLIGHT so a reload
replays from the correct after_seq floor (not 0 over restored live text)."""
body = _function_body(MESSAGES_JS, "_rememberRunJournalCursor")
assert "INFLIGHT[activeSid]" in body, (
"the cursor must be mirrored onto the persisted INFLIGHT entry (#3401 reconnect dup)"
)
assert "lastRunJournalSeq=seq" in body.replace(" ", ""), (
"INFLIGHT[activeSid].lastRunJournalSeq must be set to the advanced seq"
)
# And a persist must be scheduled so the value survives a reload.
assert "_throttledPersist" in body or "persistInflightState" in body, (
"advancing the cursor must schedule an INFLIGHT persist"
)
def test_persist_inflight_saves_run_journal_seq():
"""persistInflightState must still save lastRunJournalSeq (the value reload reads back)."""
body = _function_body(MESSAGES_JS, "persistInflightState")
assert "lastRunJournalSeq" in body
# ── FIX 3: every registered skin must have CSS (general guard) ──
def test_neon_skin_css_restored():
assert ':root[data-skin="neon"]' in CSS, "Neon skin CSS block must be present"
assert ':root.dark[data-skin="neon"]' in CSS, "Neon dark variant CSS must be present"
def test_every_registered_skin_has_css():
"""Any skin offered in the picker (_SKINS in boot.js) must have a CSS block, so a
refactor cannot silently drop a shipped skin's styling while leaving it selectable."""
# Extract skin values registered in boot.js _SKINS = [ {name:'X', value:'y'}, ... ]
skins_block = re.search(r"_SKINS\s*=\s*\[(.*?)\]", BOOT_JS, re.DOTALL)
assert skins_block, "_SKINS registration not found in boot.js"
values = set(re.findall(r"value\s*:\s*'([a-z0-9-]+)'", skins_block.group(1)))
# 'default'/'system' style entries have no data-skin CSS; only check non-default skins.
css_skins = set(re.findall(r'data-skin="([a-z0-9-]+)"', CSS))
missing = sorted(s for s in values if s and s not in css_skins and s not in {"default", "system", ""})
assert not missing, f"registered skins with no CSS block (silent breakage): {missing}"
# ── FIX 4: settled tool-worklog rebuild must run while busy too (switch-back) ──
def test_settled_worklog_rebuild_not_gated_on_idle_only():
"""The settled tool/worklog/thinking rebuild must also run when busy if there are
tool calls. Gating purely on `!S.busy` dropped every prior settled turn's worklog
when renderMessages re-ran during an active stream (switch-back to in-progress
session) — the same content-loss-on-switch class as #3668. (#3401 regression)"""
body = UI_JS
# The rebuild guard must include the `|| (S.toolCalls && S.toolCalls.length)` arm.
assert re.search(
r"if\(!S\.busy\s*\|\|\s*\(S\.toolCalls\s*&&\s*S\.toolCalls\.length\)\)\{",
body,
), (
"the settled worklog rebuild must run while busy when tool calls exist "
"(if(!S.busy || (S.toolCalls && S.toolCalls.length))), not gate purely on !S.busy"
)
# And the bare `if(!S.busy){` immediately before that rebuild's worklog-wipe must be gone.
assert "if(!S.busy){\n inner.querySelectorAll('.tool-worklog-group" not in body, (
"the worklog-rebuild block must not be gated on the bare !S.busy form"
)