diff --git a/CHANGELOG.md b/CHANGELOG.md index 5bd84347..849b0a16 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ ### Added +- **Full-session resolve concurrency is configurable.** `HERMES_WEBUI_MAX_SESSION_RESOLVE` sets how + many full-transcript session resolves may run at once (default 2, a positive integer up to 64; + zero, negative, non-numeric or out-of-range values fall back to 2). It is process-wide, so a + profile's `.env` cannot override it. (#7421, #7656 by @happy5318) - **The sidebar's recent-session window is configurable.** `HERMES_WEBUI_VISIBLE_SESSION_LIMIT` sets how many recent sessions the sidebar lists (default 20). It also bounds how many delegated subagent children can nest at once, so raise it for wide fan-outs. Invalid or non-positive values fall back to @@ -24,6 +28,84 @@ ### Fixed +- **Delegated subagent sessions show inside their parent's project.** With a project selected in the + sidebar, subagent sessions disappeared because `state.db` never gives them a `project_id`. Only + subagent rows now inherit their parent's project, resolved once per lineage, so forks keep their + own "No project" assignment and deep lineages stay fast. (#7765 by @carlotestor) +- **Saving a cron job with the model picker at "Default" keeps its provider-only pin.** Editing a job + pinned to a provider with no model (the usual shape for a self-hosted OpenAI-compatible router) + sent `provider: null` and wiped the pin. The editor now preserves a provider-only pin when the + model picker is left at Default. (#7779 by @cushingw) +- **A consumed mid-turn `/steer` no longer leaves its out-of-band wrapper in the settled chat (#7600).** + A `/steer` reaches the agent as an `[OUT-OF-BAND USER MESSAGE …]` block appended to the turn's + last tool result. After the turn settled, that wrapper stayed visible in the chat transcript. The + settled-transcript writeback now scrubs the consumed wrapper from the rows it is built from. + (#7610 by @webtecnica) +- **Saving a very large session no longer reads and parses the whole file just to count messages.** + The #1558 backup safeguard in `Session.save()` needs the on-disk message count; it obtained it by + loading the entire sidecar. On a real 203 MB / 266,940-message session that made every save + expensive. The count is now taken without a full parse, and the shrink-backup behaviour is + unchanged. (#7578 by @rodrigogs) +- **A cancelled or recovered turn no longer duplicates an answer that was already saved (#6366).** + When a completed assistant turn had been persisted and a later cancel/recovery path ran for the + same pending turn, recovery could append a duplicate user turn plus a `_partial` clone of the + journal. Recovery now stops once the transcript has already advanced past the pending turn. + (#7682 by @happy5318) + +- **The live stream reports which model actually served the turn.** A new additive `runtime_model` + SSE event carries the model (and provider, when known) that the Agent reported while producing + output, separately from the model that was requested. It is journaled for replay, is exposed as + `runtime_journal_snapshot.runtime_model`, never falls back to the configured selection when + unknown, and never changes the model requested for the next turn. Existing clients ignore it. + (#7767 by @ruizanthony) +- **Reconnecting to a new turn no longer resumes from the previous turn's replay cursor.** A new + turn copies the full transcript into the in-flight state, so the previous turn's assistant reply + could seed the replay floor and the reattached stream skipped the current reply's early events. + Replay now seeds only from the current live assistant row, and falls back to a full replay that + rebuilds the assistant body when a cursor outlives its live state. (#7651 by @happy5318) +- **Context-length lookup keeps the configured base URL for ownerless and underscore-named + providers.** The #7535 ownership guard also dropped the global `model.base_url` for a `model:` + section with a `base_url` but no `provider`, and for provider IDs written with an underscore + (`opencode_go` vs `opencode-go`), so those sessions could resolve the wrong context window. Both + now keep the URL, while a different declared owner still does not leak its endpoint. + (#7743 by @webtecnica) +- **A session deleted during a restart no longer produces a spurious recovery warning.** When + WebUI startup recovery re-attached background processes, a session that had vanished between + enumeration and rebind raised a `KeyError` that was logged as a warning. It now follows the + existing skip path, confined to the session lookup, so the vanished owner is skipped, live owners + still rebind, and registry errors still warn. (#7753, #7774 by @happy5318) + +- **Waiting on the Agent's session lease is shown as a warning instead of looking stuck.** When + another Hermes process (gateway, CLI or cron) holds the session's turn lease, the Agent's + "another Hermes process is using this session" notices now reach the chat as a warning status + instead of being dropped, and the status clears when the run ends. Classification keys on the Agent + status kind (`lifecycle` / `warn`), so user-authored text can never be promoted to a warning. + (#7760 by @ruizanthony) +- **A stale in-flight projection can no longer reach a gateway watcher after its last subscriber + leaves.** Final unsubscribe and queue eviction now invalidate the cache and fence projections that + were already in flight, without holding the lock across database reads or SSE writes, so a client + that re-subscribes gets a fresh snapshot instead of a stale one. (#7761 by @ruizanthony) +- **A burst of "session busy" refusals no longer drops a background-task completion.** When + `start_session_turn()` refused a completion wake-up with a transient 409 (Agent runtime stale, + process wake-ups paused, or the session busy with another turn), the bridge released the durable + claim as a plain failure, so a few refusals in a row could terminally drop a completion whose + session was alive and waiting. Transient refusals now return the claim as retryable, while hard + failures still use up the attempt budget. (#7758 by @ruizanthony) +- **A first visit now uses the browser's language.** The server stores "no preference" as `null` + instead of defaulting to `"en"`, so a first-time visitor gets `navigator.language` while an + explicitly saved language (including English) still wins, and legacy `settings.json` files that + already hold `"en"` keep English. Reading the browser language is guarded, so an environment where + `navigator` throws falls back cleanly. (#7622, #7730 by @happy5318) +- **The live model list for a custom provider respects its `models:` allowlist, without treating + per-model metadata as one.** `/api/models/live` filters a custom provider's live catalog to its + configured `models:` when that is a list, or when it is a mapping with `discover_models: false`. + A mapping of per-model settings (the shape `hermes setup` writes) keeps the full live catalog, as + the Agent does. (#7165 by @happy5318) +- **`MEDIA:` links wrapped in inline code no longer 404.** Every `MEDIA:` capture site (renderer, + streaming parser, TTS stripper, session-media authorization and snapshot capture, seven in all) + swallowed the closing backtick of `` `MEDIA:/path` `` into the path, so the file lookup and the + session allowlist both missed. Backtick-wrapped refs are now normalized first, while bare paths that + genuinely contain a backtick keep their full name. (#7359, #7708 by @happy5318) - **A turn's Worklog no longer vanishes when the sidebar reports idle before the final frame.** `/api/sessions` could say a session was idle before the chat stream's terminal frame reached the page, and three sidebar paths (idle reconciliation, the INFLIGHT purge and optimistic-row diff --git a/README.md b/README.md index 76429aea..3cce0d14 100644 --- a/README.md +++ b/README.md @@ -220,7 +220,7 @@ If an AI assistant is helping with install, reinstall, bootstrap, provider setup - Session actions via `⋯` dropdown per session — pin, move to project, archive, duplicate, delete - Pin/star sessions to the top of the sidebar (gold indicator) - Archive sessions (hide without deleting, toggle to show) -- Session projects -- named groups with colors for organizing sessions +- Session projects -- named groups with colors for organizing sessions; delegated subagent sessions have no project of their own and follow their nearest ancestor's project in the project filter and the Unassigned chip; forks and other child sessions keep their own project, so a fork moved to "No project" stays Unassigned - Session tags -- add #tag to titles for colored chips and click-to-filter - Grouped by Today / Yesterday / Earlier in the sidebar (collapsible date groups) - Download as Markdown transcript, full JSON export, or import from JSON diff --git a/api/background_process.py b/api/background_process.py index 2a3c581f..eea15ebb 100644 --- a/api/background_process.py +++ b/api/background_process.py @@ -1020,6 +1020,29 @@ def _record_async_delegation_accepted( ) +def _wakeup_refusal_is_transient(resp: dict | None, status: int) -> bool: + """True when the target refused the wake-up for a reason it will recover from. + + Such refusals are not delivery failures of the completion itself, so they + must not consume its bounded durable delivery budget. Otherwise a burst of + refusals (e.g. ``agent_runtime_stale`` answers while the Agent runtime is + being replaced) terminally drops a completion whose origin session is alive: + + * any payload that explicitly says ``retryable: True`` (stale runtime); + * the paused-wakeup contract (``process_wakeup_paused``); + * a session busy with another turn (``active_stream_id`` on a 409). + """ + if not isinstance(resp, dict): + return False + if resp.get("retryable") is True: + return True + if status != 409: + return False + if resp.get("error") == "process_wakeup_paused": + return True + return bool(resp.get("active_stream_id")) + + def _start_async_delegation_wakeup_turn( session_id: str, wakeup_prompt: str, @@ -1059,7 +1082,11 @@ def _start_async_delegation_wakeup_turn( ) return - release_async_delegation_delivery(evt, claim) + transient = _wakeup_refusal_is_transient(resp, status) + if transient: + release_async_delegation_delivery(evt, claim, retryable=True) + else: + release_async_delegation_delivery(evt, claim) _retry_unclaimed_async_delegation_event( process_registry, evt, keep_legacy_retrying=True ) @@ -1068,6 +1095,14 @@ def _start_async_delegation_wakeup_turn( "async delegation wakeup paused for session %s; delivery remains retryable", session_id, ) + elif transient: + logger.info( + "async delegation wakeup refused transiently for session %s " + "(status=%s err=%r); attempt refunded, durable retry scheduled", + session_id, + status, + (resp or {}).get("error"), + ) else: logger.debug( "async delegation wakeup not accepted for session %s: " @@ -1848,7 +1883,22 @@ def recover_processes_for_webui(process_registry=None, get_session_fn=None) -> i try: proc_session = process_registry.get(process_id) session_key = str(getattr(proc_session, "session_key", "") or "") - if not session_key or get_session_fn(session_key, metadata_only=True) is None: + if not session_key: + continue + # The session resolver (``api.models._resolve_session_once``) + # raises ``KeyError(sid)`` for a missing session — that is its + # documented contract for the "owner gone" outcome, not a + # fault. Treat it identically to a ``None`` return: skip the + # process without logging, and let a neighbouring live + # process still rebind. The narrow ``except KeyError`` lives + # INSIDE the outer try so an unrelated ``KeyError`` from + # ``process_registry.get()`` (or any routing-index work) still + # surfaces in the existing warning path. + try: + resolved = get_session_fn(session_key, metadata_only=True) + except KeyError: + continue + if resolved is None: continue except Exception: logger.warning( diff --git a/api/config.py b/api/config.py index 5c38d375..27e10cf9 100644 --- a/api/config.py +++ b/api/config.py @@ -11483,7 +11483,13 @@ _SETTINGS_DEFAULTS = { "hidden_tabs": [], # sidebar tab panel names hidden by user (e.g. ["tasks","kanban"]); chat and settings are always visible "tab_order": [], # user-defined sidebar/rail tab order for reorderable tabs; chat/settings stay fixed "composer_control_order": [], # user-defined composer footer control order; invalid/duplicate keys are ignored - "language": "en", # UI locale code; must match a key in static/i18n.js LOCALES + # #7622 (round 3): language is intentionally absent from the defaults so + # `load_settings()` reports `None` for a fresh install. This lets the + # client distinguish "no preference" from an explicit saved choice, + # so a user who genuinely picked English (and has `language: "en"` + # persisted on disk) is no longer overridden by the browser hint on + # their first hydration. When a user picks a locale in the Settings + # modal the value is written here and the field is set explicitly. "bot_name": os.getenv( "HERMES_WEBUI_BOT_NAME", "Hermes" ), # display name for the assistant @@ -11723,6 +11729,16 @@ def load_settings() -> dict: settings["default_model_provider"] = str(model_cfg.get("provider")) except Exception: logger.debug("Failed to resolve default model provider for settings") + # #7622/#7730 (round 4): keep the tri-state signal explicit. `language` + # is intentionally absent from `_SETTINGS_DEFAULTS` (see above), so + # without this line a fresh install's returned dict would OMIT the key + # entirely and the API payload would carry no `language` field. Emit an + # explicit `None` (serialized as JSON `null`) so the client receives the + # three-way signal it trusts: `null` = no preference, "en" = explicitly + # saved English, any other code = explicitly chosen locale. Stored + # values (including a legacy English pick written before this change) + # win via the merge above and are never touched here. + settings.setdefault("language", None) return settings @@ -11730,6 +11746,15 @@ _SETTINGS_ALLOWED_KEYS = set(_SETTINGS_DEFAULTS.keys()) - { "password_hash", "default_model", "simplified_tool_calling", +} | { + # #7622 (round 3): `language` is intentionally absent from + # `_SETTINGS_DEFAULTS` so a fresh install returns `None` for + # `settings["language"]` and the client can distinguish "no + # preference" from an explicit saved choice. But the user + # MUST still be able to pick a locale in the Settings modal, + # so we add it back to the explicit allow-list here. The + # existing BCP-47 validation at save-time still applies. + "language", } _SETTINGS_ENUM_VALUES = { "send_key": {"enter", "ctrl+enter", "shift+enter"}, diff --git a/api/gateway_chat.py b/api/gateway_chat.py index 23833777..b4bd1731 100644 --- a/api/gateway_chat.py +++ b/api/gateway_chat.py @@ -38,6 +38,7 @@ from api.config import ( from api.helpers import _redact_text, redact_session_data from api.models import clear_process_wakeup_pause, get_session, merge_session_messages_append_only from api.run_journal import RunJournalWriter, bound_run_journal_snapshot_args +from api.turn_journal import append_turn_journal_event_for_stream logger = logging.getLogger(__name__) @@ -1419,6 +1420,21 @@ def _run_gateway_chat_streaming( if cancel_event.is_set(): _restore_cancelled_success_writeback() return + # #6366 re-gate: record the durable same-stream completion + # event in the crash-safe turn journal. The run journal's + # terminal state is only reached on its own ``stream_end`` + # write path, so a Gateway run whose terminal write is lost + # would otherwise leave no completion evidence at all and + # stale-cancel recovery would re-append a duplicate + # recovered row after the valid final answer. + try: + append_turn_journal_event_for_stream( + session_id, + stream_id, + {"event": "completed", "created_at": time.time()}, + ) + except Exception: + logger.debug("Failed to append completed turn journal event", exc_info=True) success_writeback_committed = True try: from api.goals import evaluate_goal_after_turn, has_active_goal diff --git a/api/gateway_watcher.py b/api/gateway_watcher.py index dc93ecb9..69b1bbb6 100644 --- a/api/gateway_watcher.py +++ b/api/gateway_watcher.py @@ -203,6 +203,8 @@ class GatewayWatcher: ): self._subscribers: list[queue.Queue] = [] self._sub_lock = threading.Lock() + # Final removal invalidates any projection admitted by an earlier cohort. + self._subscriber_epoch = 0 self._stop_event = threading.Event() # Wakes a poll loop parked because nobody is subscribed (subscribe/stop). self._idle_wake = threading.Event() @@ -286,21 +288,32 @@ class GatewayWatcher: self._idle_wake.set() return q - def unsubscribe(self, q: queue.Queue): - """Remove a subscriber queue.""" - with self._sub_lock: - try: - self._subscribers.remove(q) - except ValueError: - pass + def _remove_subscriber_locked(self, q: queue.Queue) -> bool: + """Remove a known queue under _sub_lock; fence polls on the last removal.""" + try: + self._subscribers.remove(q) + except ValueError: + return False + if not self._subscribers: + self._subscriber_epoch += 1 + self._last_cheap_fp = '' + self._last_full_projection_at = None + return True - def _notify_subscribers(self, sessions: list): + def unsubscribe(self, q: queue.Queue): + """Remove a subscriber queue, invalidating projection on the last removal.""" + with self._sub_lock: + self._remove_subscriber_locked(q) + + def _notify_subscribers(self, sessions: list, *, epoch: int | None = None): """Push change event to all subscribers.""" event = { 'type': 'sessions_changed', 'sessions': sessions, } with self._sub_lock: + if epoch is not None and epoch != self._subscriber_epoch: + return # An old poll must not notify a new subscriber cohort. dead = [] for q in self._subscribers: try: @@ -310,10 +323,7 @@ class GatewayWatcher: except Exception: dead.append(q) for q in dead: - try: - self._subscribers.remove(q) - except ValueError: - pass + self._remove_subscriber_locked(q) # Send a None sentinel so the SSE handler unblocks, closes, # and lets the browser's EventSource auto-reconnect. try: @@ -330,9 +340,9 @@ class GatewayWatcher: """ with self._sub_lock: has_subscribers = bool(self._subscribers) + admission_epoch = self._subscriber_epoch if not has_subscribers: - # Force fresh projection on the next subscription, without touching DB. - self._last_full_projection_at = None + # Final removal already invalidated the cache under the same lock. return False db_path = self._state_db_path # A watcher may start before the agent has created state.db. Publishing an @@ -361,14 +371,20 @@ class GatewayWatcher: if sessions is None: return False current_hash = _snapshot_hash(sessions) - if cheap_fp is not None: - self._last_cheap_fp = cheap_fp - self._last_full_projection_at = current_time - - if current_hash != self._last_hash: - self._last_hash = current_hash - self._last_sessions = sessions - self._notify_subscribers(sessions) + with self._sub_lock: + if admission_epoch != self._subscriber_epoch or not self._subscribers: + return False # Never restore a cache invalidated during the DB read. + if cheap_fp is not None: + self._last_cheap_fp = cheap_fp + self._last_full_projection_at = current_time + if current_hash != self._last_hash: + changed = True + self._last_hash = current_hash + self._last_sessions = sessions + else: + changed = False + if changed: + self._notify_subscribers(sessions, epoch=admission_epoch) return True def _poll_loop(self): diff --git a/api/media_snapshots.py b/api/media_snapshots.py index 929c1431..e54c34d8 100644 --- a/api/media_snapshots.py +++ b/api/media_snapshots.py @@ -505,6 +505,13 @@ def annotate_media_snapshots( if allowed_predicate is None: allowed_predicate = media_capture_allowed media_re = _re.compile(r"MEDIA:([^\s\)\]]+)") + # #7680 re-gate (9/22): two-pass scan. First strip backtick + # wrappers (`` `MEDIA:path` `` → ``MEDIA:path``) so the bare + # class below does not consume the closing backtick as part of + # the path. Then the bare class (no backtick in the exclusion + # set) captures the full filename even when the path itself + # contains a backtick (e.g. ``report`final.png``). + backtick_re = _re.compile(r"`MEDIA:([^`\s]+)`") captured = 0 for msg in messages or []: if not isinstance(msg, dict) or msg.get("role") != "assistant": @@ -530,6 +537,7 @@ def annotate_media_snapshots( if not text_parts: continue text = "\n".join(text_parts) + text = backtick_re.sub(lambda m: f"MEDIA:{m.group(1)}", text) refs = media_re.findall(text) if not refs: continue diff --git a/api/models.py b/api/models.py index b1b04471..2488c9df 100644 --- a/api/models.py +++ b/api/models.py @@ -1107,6 +1107,14 @@ _METADATA_PREFIX_MAX_BYTES = 1024 * 1024 #: even when the stop key sits at 2 KB. _METADATA_PREFIX_FIRST_STAGE_BYTES = 64 * 1024 +# Marks a sidecar as written by the CURRENT writer contract, where the +# persisted `message_count` equals len(messages) by construction (both keys +# land in the same atomic write). save()'s bounded-prefix shrink check trusts +# a persisted count ONLY when this marker sits next to it in the file; see +# _prefix_message_count(). Bump the value whenever the writer contract changes +# meaning, so counts from an older contract fall back to the full parse. +_MESSAGE_COUNT_MARKER = 1 + def _read_metadata_json_prefix(path, max_prefix_bytes=_METADATA_PREFIX_MAX_BYTES): """Read only the metadata portion before the large arrays. @@ -1546,6 +1554,16 @@ class Session: # legacy-format reader that stops at a scene key still finds the count. # The full anchor_activity_scenes bodies serialize AFTER messages. meta['message_count'] = len(self.messages or []) + # _mc_v marks this file as written by the current writer contract, + # where `message_count` equals len(messages) by construction and both + # keys land in the same atomic write. save()'s shrink guard takes the + # bounded-prefix shortcut ONLY for marked files; an unmarked count + # (a legacy pre-#5854 sidecar, a sidecar materialized by an older + # recovery writer, any foreign writer) gets the full parse, so a stale + # count from outside this contract can never read a real shrink as a + # growth and skip the #1558 backup. One save re-marks the file, so the + # fast path still covers steady state. + meta['_mc_v'] = _MESSAGE_COUNT_MARKER meta['anchor_scene_index'] = _anchor_scene_index_from_records(self.anchor_activity_scenes) # Keep the in-memory fingerprint aligned with what we just persisted, so a # later metadata-only reload of THIS object (or any fingerprint reader) @@ -1558,7 +1576,7 @@ class Session: meta['anchor_activity_scenes'] = self.anchor_activity_scenes if isinstance(self.anchor_activity_scenes, dict) else {} # Fields not in METADATA_FIELDS (e.g. last_usage) go at the end. Exclude # the keys we placed explicitly above so they aren't emitted twice. - _placed = {'message_count', 'anchor_scene_index', 'messages', 'tool_calls', 'anchor_activity_scenes'} + _placed = {'message_count', '_mc_v', 'anchor_scene_index', 'messages', 'tool_calls', 'anchor_activity_scenes'} extra = {k: v for k, v in self.__dict__.items() if k not in METADATA_FIELDS and k not in _placed and not k.startswith('_')} @@ -1578,12 +1596,46 @@ class Session: # their .bak get restored automatically. try: if self.path.exists(): - existing_text = self.path.read_text(encoding='utf-8') - try: - existing = json.loads(existing_text) - existing_msg_count = len(existing.get('messages') or []) - except (json.JSONDecodeError, ValueError): - existing_msg_count = -1 # corrupt → always back up + # The on-disk count, without reading the body. + # + # The decision below is a function of ONE integer -- how many + # messages the file on disk holds -- and save() already writes + # that integer into the metadata prefix, before `messages`, as + # `message_count` (see METADATA_FIELDS above; load_metadata_only + # and the sidebar freshness check read it the same way). So read + # THAT through a bounded 64 KiB prefix instead of the whole file. + # Measured before this: a 203,439,398-byte sidecar cost 20,377 ms + # (17,453 in read_text, 2,924 in json.loads) to yield one integer, + # on EVERY save -- including the grow-saves that never back + # anything up. The prefix read is O(64 KiB), and because the count + # is part of the bytes on disk it travels with any rewrite of them. + # + # An in-memory "I wrote this, stat says nothing changed" cache is + # NOT sufficient here, and was removed after review: (inode, size, + # mtime_ns) is not a content identity. A same-length in-place + # rewrite inside one mtime tick keeps all three fields -- ext4 + # stamps mtime from a coarse clock, so two writes in the same tick + # share one mtime_ns -- and a stale cached count then reads a real + # shrink as a growth and skips the #1558 backup. The prefix count + # cannot be fooled that way. + # + # Every unknown falls through to the full read + parse below: a + # legacy (pre-#5854) sidecar whose count is not in the prefix, a + # count written without the current writer's _mc_v marker (an + # older writer's count can be stale relative to the messages + # array next to it), a corrupt or truncated prefix, a file with + # no top-level `messages` key at all, or metadata alone that + # overflows the budget. + # Fail-open is the contract -- never "assume no shrink". + existing_text = None + existing_msg_count = _prefix_message_count(self.path) + if existing_msg_count is None: + existing_text = self.path.read_text(encoding='utf-8') + try: + existing = json.loads(existing_text) + existing_msg_count = len(existing.get('messages') or []) + except (json.JSONDecodeError, ValueError): + existing_msg_count = -1 # corrupt → always back up incoming_msg_count = len(self.messages or []) if ( existing_msg_count > 0 @@ -1601,6 +1653,10 @@ class Session: return if existing_msg_count > incoming_msg_count: bak_path = self.path.with_suffix('.json.bak') + if existing_text is None: + # The .bak body is the one thing that needs the full text, + # and a shrink is the one time it is needed. + existing_text = self.path.read_text(encoding='utf-8') # SHOULD-FIX #2 (Opus): atomic write via tmp+replace, # mirroring the main save() pattern below. Prevents a # torn .bak from a crash mid-write or a concurrent @@ -2483,6 +2539,191 @@ def _latest_user_matches_pending_text(messages, pending_text): return False +def _pending_active_turn_token(session): + """Return the exact WebUI active-turn token for the pending turn. + + The token is ``":"`` — the same value the + eager user-message checkpoint and the agent-result merge stamp onto the + materialized user row (``stamp_message_source``). It is unforgeable + per-turn identity: two distinct streams that submit the same prompt + inside the same second produce different tokens. + """ + from api.process_event_utils import build_active_turn_token + + return build_active_turn_token( + getattr(session, 'active_stream_id', None), + getattr(session, 'pending_started_at', None), + ) + + +def _transcript_user_row_is_pending_turn(message, session, pending_token) -> bool: + """Return True only when a transcript user row provably IS the pending turn. + + Identity is bound to the active-turn token, never to integer-second + timestamp equality: two different streams that send the same prompt + within one second truncate to the same ``int(timestamp)``, so the + checkpoint matcher alone would accept the *other* stream's row as the + pending turn. A token mismatch therefore always loses, and a row that + carries no token at all cannot be proven to be this turn — which means + the caller must recover, not suppress. + """ + if not isinstance(message, dict) or message.get('role') != 'user': + return False + row_token = message.get('_active_turn_token') + if pending_token: + if not row_token: + # Ambiguous legacy identity: the pending turn has a resolvable + # token but this row predates token stamping. We cannot prove it + # is the same turn, so recovery must run. + return False + return row_token == pending_token + if row_token: + # The row belongs to a different, token-bearing turn. Never let a + # same-second timestamp collision override that mismatch. + return False + # Neither side carries a token (legacy/dead-stream pending state): fall + # back to the strict content + timestamp + source + attachments + # checkpoint, which is the only identity signal available. + return _message_matches_pending_checkpoint( + message, + getattr(session, 'pending_user_message', None), + getattr(session, 'pending_started_at', None), + getattr(session, 'pending_user_source', None), + getattr(session, 'pending_attachments', None), + ) + + +def _pending_turn_has_final_assistant_answer(messages, start_idx: int) -> bool: + """Return True when the matched user turn ends in a genuine final answer. + + Reuses the established final-answer semantics + (``_assistant_message_has_final_visible_text``) instead of "an assistant + row exists". An assistant row only settles the turn when it carries real + visible answer text: empty rows, tool-call-only rows, interim + ``_partial`` progress rows, rows recovered from the run journal and + compaction reference cards are all rejected, so a turn that was + interrupted mid-tool-execution is never mistaken for a completed one. + + The scan stops at the next real user row — a final answer that belongs + to a *later* turn must not settle this pending turn. + """ + from api.streaming import ( + _assistant_message_has_final_visible_text, + _is_synthetic_control_message, + ) + + for later in messages[start_idx + 1:]: + if not isinstance(later, dict): + continue + if _is_synthetic_control_message(later): + continue + role = later.get('role') + if role == 'user': + if is_context_compression_marker(later): + # Synthetic compaction cards are not a user-turn boundary. + continue + return False + if role != 'assistant': + # Tool rows never prove a final answer. + continue + if ( + later.get('_partial') + or later.get('_error') + or later.get('_recovered') + or later.get('_recovered_from_run_journal') + ): + continue + if is_context_compression_marker(later): + continue + if _assistant_message_has_final_visible_text(later): + return True + return False + + +def _transcript_already_advanced_past_pending(session) -> bool: + """#6366: a stale pending user message whose own user row already appears + in the durable transcript, followed by a genuine final assistant answer + inside that same user-turn boundary, means the transcript has already + advanced past this pending turn. Returning True here lets the recovery + path skip the recovered user / ``_partial`` clone / journal replay / + generic no-response error that would otherwise append after the valid + final answer. The tail-only check at line ~3485 (``session.messages[-1]``) + misses this case: when the pending turn's user row is older than + the transcript tail, the tail is a *newer* assistant row that + can never match the pending **user** checkpoint, and the recovery + path falls through into the append branch. + + Identity is bound to the pending stream's exact active-turn token + (``_transcript_user_row_is_pending_turn``), so a repeated prompt from a + different stream — even one submitted in the same integer second — + can never be matched against this pending turn and silently dropped. + + Completion requires a genuine final visible assistant answer + (``_pending_turn_has_final_assistant_answer``): empty, tool-call-only, + interim ``_partial``, journal-recovered and compaction tails all keep + recovery active, because that is precisely the interrupted-turn shape + the recovery path exists for. + + The transcript heuristic alone is not durable: an unflagged interim + prose row ("Let me check the logs first.") followed by tool rows that + were never followed by a final answer in the transcript passes the + predicate as a "completed" turn, but the stream may have died mid- + tool and the run journal was never marked done. 9/23 re-gate: also + require durable same-stream terminal evidence. The run journal + cannot supply that evidence on this path — the caller in + ``_apply_core_sync_or_error_marker`` already returns early whenever + the run journal reports ``completed`` for the stream, so the branch + that reaches this predicate can never observe that state (which is + why the previous run-journal gate was unreachable dead code). The + evidence therefore has to come from the turn journal, which records + the exact-stream ``completed`` event for the pending turn + (``_turn_journal_records_completion``). Without it, fall through to + the normal journal-recovery path so the partial output is replayed + and the interruption is marked. + + The suppression is therefore deliberately conservative: whenever the + turn's identity, its completion, or its durable terminal evidence + cannot be positively proven, this helper returns False and the + prompt is recovered instead of discarded. A false negative leaves a + visible cosmetic duplicate; a false positive silently destroys a + prompt or a response. + + Idempotent: this is a pure read that does not mutate the session. + Running recovery twice therefore produces the same outcome. + """ + pending_text = getattr(session, 'pending_user_message', None) + if not pending_text: + return False + messages = getattr(session, 'messages', None) + if not isinstance(messages, list): + return False + pending_token = _pending_active_turn_token(session) + for idx, message in enumerate(messages): + if not _transcript_user_row_is_pending_turn(message, session, pending_token): + continue + # Found the pending turn's own user row at ``idx``. A genuine + # final answer inside that turn's boundary is necessary but not + # sufficient: durable same-stream terminal evidence must also + # exist, otherwise unflagged interim prose + # ("Let me check the logs first.") followed by tool rows can be + # mistaken for a finished turn. + if not _pending_turn_has_final_assistant_answer(messages, idx): + continue + stream_id = getattr(session, 'active_stream_id', None) + # The turn journal — not the run journal — carries the + # completion evidence that is still observable here: the caller + # (``_apply_core_sync_or_error_marker``) already consumed the + # run-journal ``completed`` state on its own early return, so a + # run-journal gate at this depth can never fire. + if not _turn_journal_records_completion(session, stream_id): + # No durable same-stream terminal evidence — the turn may + # have died mid-tool. Fall through to the journal-recovery + # path instead of suppressing it. + return False + return True + return False + + def _partial_message_signature(message: dict) -> tuple: """Return a stable identity for partial assistant markers recovered on load.""" if not isinstance(message, dict): @@ -2684,6 +2925,54 @@ def _run_journal_terminal_state(session, stream_id: str | None) -> str | None: return str(terminal.get('terminal_state') or '') or None +def _turn_journal_records_completion(session, stream_id: str | None) -> bool: + """Return True when the crash-safe turn journal proves the turn for + ``stream_id`` reached its terminal ``completed`` state. + + The turn journal is written per stream/turn id by the exact-stream + workers and by :mod:`api.routes` on submission, so a ``completed`` + event for the *pending stream id* is completion evidence that + survives a run-journal write failure: the run journal only reaches + ``done`` / ``stream_end`` on its own write path, and a process that + dies between the turn journal write and the run-journal terminal + append leaves the run journal without terminal state while the turn + journal still records that the turn actually finished. + + Bound strictly to ``stream_id`` — a ``completed`` turn belonging to + another stream of the same session never settles this one — and to + the latest turn recorded for that stream, so a turn that was + cancelled or crashed (``interrupted``) keeps recovery active even + when an earlier turn of the same stream completed normally. + """ + if not stream_id: + return False + try: + from api.turn_journal import ( + derive_turn_journal_states, + read_turn_journal, + ) + journal = read_turn_journal(session.session_id) + states, _ = derive_turn_journal_states(journal.get('events') or []) + except Exception: + return False + latest: tuple[float, str, dict] | None = None + for turn_id, event in states.items(): + if str(event.get('stream_id') or '') != str(stream_id): + continue + try: + created_at = float(event.get('created_at') or 0) + except (TypeError, ValueError): + created_at = 0.0 + # ``>=`` on both elements keeps the winner deterministic when two + # turns of the same stream share a ``created_at``. + candidate = (created_at, turn_id, event) + if latest is None or candidate[:2] >= latest[:2]: + latest = candidate + if latest is None: + return False + return latest[2].get('event') == 'completed' + + def _recoverable_unsaved_gateway_terminal_error( session, stream_id: str | None, @@ -3560,6 +3849,29 @@ def _apply_core_sync_or_error_marker( ) return True if not _tail_user_already_checkpointed: + # #6366 re-gate: when the durable transcript has already + # advanced past this pending turn into a newer settled + # user/assistant boundary, the recovery path must NOT + # append a recovered user row + ``_partial`` clone + + # journal replay + generic no-response error after the + # valid final answer. The tail-only check above misses + # that case (the tail is a newer assistant row that can + # never match the pending user checkpoint). Clear only + # the stale pending fields and return; the transcript is + # already correct and durable. + if _transcript_already_advanced_past_pending(session): + session.active_stream_id = None + session.pending_user_message = None + session.pending_attachments = [] + session.pending_started_at = None + session.pending_user_source = None + session.save(touch_updated_at=touch_updated_at) + logger.info( + "Session %s: cleared stale pending state for stream %s — transcript already advanced past this turn", + sid, + _stream_id, + ) + return True _append_recovered_pending_turn(session, timestamp=_recovered_ts) else: recovered = { @@ -4342,6 +4654,57 @@ def _sidecar_stat_signature(path): int(st.st_size), int(getattr(st, 'st_ctime_ns', int(st.st_ctime * 1_000_000_000)))) +_MESSAGE_COUNT_MARKER = 1 + + +def _prefix_message_count(path): + """The sidecar's own persisted ``message_count``, from a bounded prefix. + + The #1558 shrink check in save() needs exactly one integer -- how many + messages the file on disk holds -- and the writer puts that integer in the + metadata prefix, before ``messages``, precisely so readers can have it + without parsing the body (see METADATA_FIELDS). Reading the prefix costs + O(64 KiB) against O(file size) for ``read_text`` + ``json.loads`` (measured + 2026-09-15: 20,377 ms for one integer on a 203,439,398-byte sidecar). + + Content-derived on purpose. A stat identity cannot stand in for it: an + in-place rewrite of the same length inside one mtime tick keeps inode, size + *and* mtime_ns, so a cached count would read a real shrink as a growth and + skip the backup. The count is part of the bytes, so any rewrite of them + rewrites it too. + + Gated on the writer marker (``_mc_v``, review 2026-09-22): a count is only + trusted when the SAME write that produced the messages array also vouched + for the count. Anything unmarked -- a legacy pre-#5854 sidecar, a sidecar + materialized by an older recovery writer whose denormalized count could be + stale against its rows, any foreign writer -- returns None and the caller + falls back to the full read + parse. After one save by the current writer + the file carries the marker and the fast path resumes. + + Returns None -- meaning "the caller must fall back to the full read + + parse" -- for every shape that does not carry a usable MARKED count: an + unmarked or wrong-marker file, a legacy sidecar whose count is not in the + prefix, a file with no top-level ``messages`` key at all, a corrupt or + truncated prefix, an unreadable path, or metadata alone that overflows the + budget. + """ + try: + raw = _read_metadata_json_prefix(path) + except (OSError, ValueError): + return None + if not raw: + return None + try: + parsed = json.loads(raw) + except (json.JSONDecodeError, ValueError): + return None + if not isinstance(parsed, dict): + return None + if parsed.get('_mc_v') != _MESSAGE_COUNT_MARKER: + return None + return _parse_nonnegative_int(parsed.get('message_count')) + + def _legacy_sidecar_facts_get(sid): """Return cached authoritative facts for a LEGACY sidecar, or None (#5854). @@ -4891,7 +5254,41 @@ class _FullSessionResolveRequired(RuntimeError): """Internal signal that a full sidecar load must enter the bounded path.""" -_FULL_SESSION_RESOLVE_MAX_CONCURRENT = 2 +def _read_max_session_resolve_concurrent() -> int: + """#7421: env override for the heavy session-resolve cap. The + hardcoded literal 2 is too low for high-concurrency + deployments where several parallel active sessions all need + a full-transcript resolve; operators can set + ``HERMES_WEBUI_MAX_SESSION_RESOLVE`` to a positive int to + raise the cap without code changes. A bad or missing value + falls back to the previous default of 2, so an operator who + sets ``HERMES_WEBUI_MAX_SESSION_RESOLVE=0`` or a non-numeric + value does not regress to a blocked or unbounded state. + + The cap is a *safety bound* (per-resolve cost tracked in + #7310), not a user preference — a typo like ``=999999`` + must fall back to the safe default, never to the permissive + extreme. Every other bad input in this function falls back + to ``2``; out-of-range values do too. The previously + accepted upper bound (64) is reachable only via an explicit + in-range operator value; a profile's ``.env`` cannot reach + it at all (see ``_PROTECTED_ENV_KEYS``). + """ + raw = (os.getenv("HERMES_WEBUI_MAX_SESSION_RESOLVE") or "").strip() + if not raw: + return 2 + try: + n = int(raw) + except (TypeError, ValueError): + return 2 + if n < 1: + return 2 + if n > 64: + return 2 # #7656 round-3: typo-safe fallback, not clamp-up-to-64 + return n + + +_FULL_SESSION_RESOLVE_MAX_CONCURRENT = _read_max_session_resolve_concurrent() _FULL_SESSION_RESOLVE_SLOTS = threading.BoundedSemaphore( _FULL_SESSION_RESOLVE_MAX_CONCURRENT ) diff --git a/api/process_event_utils.py b/api/process_event_utils.py index 2d9872f6..f6970d6d 100644 --- a/api/process_event_utils.py +++ b/api/process_event_utils.py @@ -3,6 +3,7 @@ from __future__ import annotations from collections import OrderedDict from dataclasses import dataclass +import inspect import logging import math import re @@ -451,16 +452,46 @@ def complete_async_delegation_delivery( _mark_legacy_async_delivery_complete(claim.delegation_id) +def _release_accepts_retryable(release_fn: Any) -> bool: + """Whether the core ``release_event_delivery`` supports ``retryable=``. + + Older Agent cores predate the keyword; calling them with it would raise + ``TypeError`` and skip the release entirely. + """ + try: + params = inspect.signature(release_fn).parameters + except (TypeError, ValueError): + return False + if "retryable" in params: + return params["retryable"].kind in ( + inspect.Parameter.KEYWORD_ONLY, + inspect.Parameter.POSITIONAL_OR_KEYWORD, + ) + return any(p.kind is inspect.Parameter.VAR_KEYWORD for p in params.values()) + + def release_async_delegation_delivery( evt: Any, claim: AsyncDelegationDeliveryClaim, + *, + retryable: bool = False, ) -> None: - """Release a failed claim so a later WebUI consumer can retry it.""" + """Release a failed claim so a later WebUI consumer can retry it. + + ``retryable=True`` tells the core the refusal was transient, so the claimed + attempt is refunded instead of counting against the bounded delivery + budget (``tools.async_delegation.release_event_delivery(..., + retryable=True)``). A core that predates the keyword gets a plain release: + the attempt is then consumed as before, never an exception. + """ try: if claim.durable: from tools.async_delegation import release_event_delivery - release_event_delivery(evt, claim.claim_id) + if retryable and _release_accepts_retryable(release_event_delivery): + release_event_delivery(evt, claim.claim_id, retryable=True) + else: + release_event_delivery(evt, claim.claim_id) except Exception: logger.warning( "Failed to release durable async delegation delivery for %s", diff --git a/api/profiles.py b/api/profiles.py index a4b5d8f7..e56b32fd 100644 --- a/api/profiles.py +++ b/api/profiles.py @@ -202,6 +202,14 @@ _PROTECTED_ENV_KEYS = frozenset({ 'HERMES_WEBUI_ISOLATED_PROFILE', # Server-wide sidebar window; a profile .env must not widen the per-request query. 'HERMES_WEBUI_VISIBLE_SESSION_LIMIT', + # #7656 round-3: HERMES_WEBUI_MAX_SESSION_RESOLVE is a process-wide resource + # cap (controls the BoundedSemaphore that gates full-transcript resolves). + # A profile's .env is per-profile by design; allowing it to set a + # server-wide limit (and a first-load-wins asymmetry) would let a + # single profile drag the rest into a tighter or looser posture than + # the operator intended. Same shape as the isolated-profile key: only + # the operator/launcher env at startup can set it. + 'HERMES_WEBUI_MAX_SESSION_RESOLVE', }) @@ -939,6 +947,12 @@ _BLOCKED_RUNTIME_ENV_KEYS = { # profile's own env on any runtime/gateway-parity path. 'HERMES_WEBUI_ISOLATED_PROFILE', 'HERMES_WEBUI_VISIBLE_SESSION_LIMIT', + # #7656 round-3: process-wide resolve cap. Filter out a profile's + # .env override the same way we filter the isolated-profile key, so + # the cap stays at the operator/launcher value across profile + # switches rather than re-sizing to whatever the first-loaded + # profile env set. + 'HERMES_WEBUI_MAX_SESSION_RESOLVE', } diff --git a/api/routes.py b/api/routes.py index 71ef658e..755bbbd8 100644 --- a/api/routes.py +++ b/api/routes.py @@ -4067,6 +4067,7 @@ def _run_journal_live_snapshot(stream_id: str | None, *, handler=None) -> dict | "tool_calls": tool_calls, "last_assistant_text": assistant_text, "last_reasoning_text": reasoning_text, + "runtime_model": runtime_model_from_events(session_id, stream_id, events), "activity_burst_anchors": activity_burst_anchors, "current_activity_burst_id": current_activity_burst_id, "current_live_segment_seq": current_live_segment_seq, @@ -7183,7 +7184,35 @@ def _context_length_lookup_inputs_for_model( if not effective_provider: effective_provider = _canonical_context_provider(model_cfg.get("provider")) if not effective_base_url: - effective_base_url = str(model_cfg.get("base_url") or "").strip() + # #7535: the global model.base_url may only fill an empty slot when + # the session provider IS the configured model.provider owner + # (mirror the ownership predicate used for model_cfg's API key in + # _context_length_config_api_key_for_provider). A built-in registry + # provider (empty base_url by design) must keep the slot empty so + # the registry endpoint resolves instead of another provider's URL. + # + # Two shapes cannot own the slot and therefore cannot conflict, so + # they keep master's backfill: a config that declares no provider + # at all (the profile-setup path writes model.base_url without one) + # and the two spellings of the same built-in id (opencode_go == + # opencode-go), folded through api.config._canonicalise_provider_id + # so distinct custom:* slugs stay distinct. + _model_cfg_provider = _canonical_context_provider(model_cfg.get("provider")) + _owner_provider = _model_cfg_provider + _session_provider = effective_provider + try: + from api.config import _canonicalise_provider_id as _canon_provider_id + + _owner_provider = _canon_provider_id(_owner_provider) or _owner_provider + _session_provider = _canon_provider_id(_session_provider) or _session_provider + except Exception: + pass + if ( + not effective_provider + or not _model_cfg_provider + or _providers_match_for_context(_owner_provider, _session_provider) + ): + effective_base_url = str(model_cfg.get("base_url") or "").strip() custom_providers = cfg.get("custom_providers") if isinstance(cfg, dict) else None if not isinstance(custom_providers, list): @@ -10866,6 +10895,7 @@ from api.streaming import ( ) from api.gateway_chat import _run_gateway_chat_streaming, webui_gateway_chat_enabled from api.run_journal import ( + runtime_model_from_events, _parse_run_journal_event_id as _shared_parse_run_journal_event_id, _summary_from_events, bound_run_journal_snapshot_args, @@ -20678,6 +20708,16 @@ def _serve_inline_html_preview(handler, target: Path, cache_control: str, *, csp _MEDIA_TOKEN_RE = re.compile(r"MEDIA:([^\s\)\]]+)") +# #7680 re-gate (9/22): two-pass scan. +# 1. `` `MEDIA:path` `` (backtick-wrapped, inline-code form) → strip +# the wrapping backticks so the bare-token pass below sees a +# plain ``MEDIA:path`` and the closing backtick is not consumed +# as part of the path. +# 2. ``MEDIA:[^\s\)\]]+`` (bare, no backtick in the exclusion +# class) so a filename that legally contains a backtick +# (``report`final.png``) is captured in full instead of being +# truncated at the first backtick. +_BACKTICK_MEDIA_RE = re.compile(r"`MEDIA:([^`\s]+)`") def _message_content_text(content) -> str: @@ -20737,6 +20777,10 @@ def _session_media_token_allows_path(sid: str, target: Path, allowed_mimes: set[ ) if "MEDIA:" not in text: continue + # #7680 re-gate: strip backtick wrappers first so the bare + # class below captures the full path even when the filename + # itself contains a backtick. + text = _BACKTICK_MEDIA_RE.sub(lambda m: f"MEDIA:{m.group(1)}", text) for ref in _MEDIA_TOKEN_RE.findall(text): if "://" in ref: continue @@ -21929,6 +21973,99 @@ def _handle_live_models(handler, parsed): _env = str(_cp.get("key_env") or "").strip() return os.getenv(_env, "").strip() if _env else "" + def _custom_provider_models_discover_is_false(_cp): + """True when ``discover_models`` is an explicit ``false`` opt-out. + + Mirrors ``api.config._provider_discover_allowed`` / Hermes + Agent ``model_switch_providers._discover_flag``: ``discover_models`` + defaults to True, and the string forms ``"false"``/``"no"``/``"0"`` + (case-insensitive) mean False. Used to decide whether a + dict-shaped ``models`` mapping is a hand-pinned allowlist (only + when discovery is off) rather than per-model metadata. + """ + _discover = _cp.get("discover_models", True) if isinstance(_cp, dict) else True + if isinstance(_discover, str): + return _discover.strip().lower() in {"false", "no", "0"} + return not bool(_discover) + + def _custom_provider_allowlist_ids(_cp): + """Plural ``models`` list only — the explicit allowlist signal. + + The singular ``model`` field is sticky/default metadata, NOT an + allowlist: it must not gate live-catalog filtering, otherwise a + provider configured with only ``model: assistant`` (no ``models`` + list) would be collapsed to a single model. Only an explicit + ``models`` allowlist expresses "show exactly these models". + + An auto-discovered catalog is NOT an allowlist: when Hermes + persisted discovery results back into config (``models: {...}`` + plus ``models_discovered: true``), that mapping is a snapshot + of what the gateway exposed at discovery time. Gating on it + would permanently pin the live catalog to the first-discovery + set, silently dropping any model the user pulls in later + (LM Studio / Ollama). ``_provider_models_are_discovered_catalog()`` + is the shared predicate the ``/api/models`` path already uses; + when it says "discovered", return no allowlist and let the + live probe win. An explicit ``discover_models: false`` opt-out + re-pins the catalog (the predicate accounts for it), so a + hand-pinned discovered catalog still filters. + + A dict-shaped ``models`` mapping that is NOT marked discovered is + *per-model metadata* written by the Hermes Agent setup flow + (``hermes_cli/model_switch.py::_save_custom_provider`` and the + setup wizard) — e.g. ``{chat-a: {context_length: 128000}}``. It is + not a catalog narrow: treating its keys as an allowlist would + collapse the live picker to the single saved default (keyless + Ollama) while the CLI live-probe shows the full catalog. Only an + explicit ``discover_models: false`` opts into treating the dict + keys as a pinned allowlist. List/JSON-array-string/Python-literal + shapes remain plain allowlists. + + The plural value is decoded through ``_parse_config_string_list()`` + because ``hermes config set`` / JSON-mode editor saves persist + lists as quoted JSON-array strings (``'["chat-a","chat-b"]'``) or + Python literals (``"['chat-a']"``). Handling only ``dict``/``list`` + made a serialized allowlist fall through to ``[]``, which the + caller reads as "no allowlist configured" and floods the picker + with the full upstream catalog — the same class of bug as the + ``skills.disabled`` regression (#7120 / #7134). + """ + from api.config import _provider_models_are_discovered_catalog + + if _provider_models_are_discovered_catalog(_cp): + return [] + _ids = [] + _models = _cp.get("models") + # Serialized shapes only: ``hermes config set`` / JSON-mode + # editor saves persist lists as quoted JSON-array strings + # (``'["chat-a","chat-b"]'``) or Python literals + # (``"['chat-a']"``). Decode those through the shared helper so + # they are honored; native dict/list values are walked below so + # dict *entries* keep their id|model|name metadata (the decoder + # str()-ifies list members, which would mangle them). + if isinstance(_models, str): + _models = _parse_config_string_list(_models) + if isinstance(_models, dict): + # A dict-shaped ``models`` is per-model metadata written by + # the Hermes Agent setup flow, NOT a catalog narrow. Only a + # ``discover_models: false`` opt-out treats the dict keys as a + # pinned allowlist; otherwise return the empty allowlist so the + # live probe returns the full catalog. + if not _custom_provider_models_discover_is_false(_cp): + return [] + for _mid in _models: + if isinstance(_mid, str) and _mid.strip(): + _ids.append(_mid.strip()) + elif isinstance(_models, (list, tuple)): + for _item in _models: + if isinstance(_item, str) and _item.strip(): + _ids.append(_item.strip()) + elif isinstance(_item, dict): + _mid = _item.get("id") or _item.get("model") or _item.get("name") + if _mid and str(_mid).strip(): + _ids.append(str(_mid).strip()) + return _ids + # For 'custom' and 'custom:*' providers, provider_model_ids() # returns [] because they aren't real hermes_cli endpoints. # Fall back to the custom_providers entries from config.yaml so @@ -21937,11 +22074,13 @@ def _handle_live_models(handler, parsed): # Collect config-specified model IDs separately so they don't # prevent the live fetch below from running (#3718). _config_ids = [] + _allowlist_ids = [] if provider == "custom" or provider.startswith("custom:"): for _cp in _custom_provider_entries_for_request(): if custom_provider_entry is None: custom_provider_entry = _cp _config_ids.extend(_custom_provider_model_ids(_cp)) + _allowlist_ids.extend(_custom_provider_allowlist_ids(_cp)) # Always try live fetch for custom providers — config entries are a # fallback, not a replacement. The live endpoint should return ALL @@ -22012,15 +22151,39 @@ def _handle_live_models(handler, parsed): except Exception as _fetch_err: logger.debug("Live fetch from custom provider failed: %s", _fetch_err) - # If live fetch succeeded, merge with config entries (live takes - # priority). If live fetch failed, fall back to config-only list. + # If live fetch succeeded, filter the live catalog down to the + # config-declared models allowlist first, then append any + # allowlisted models the live endpoint didn't return. Custom + # providers (esp. New-API-style gateways) expose their ENTIRE + # catalog via /v1/models — including image/audio models that + # are not chat models. The picker must not surface models the + # user never declared in config.yaml. Only a NON-EMPTY explicit + # ``models`` allowlist gates the filter — a provider configured + # with just a singular ``model`` (no ``models`` list) keeps the + # unfiltered live catalog. When no allowlist is configured, + # return the live list as-is (preserves the pre-filter + # discovery behaviour). + # + # An empty allowlist (``models: []``, ``models: "[]"``, or a + # value that decodes to no usable ids) is deliberately treated + # as "not configured", NOT as "allow nothing". Gating on + # declared-ness instead would emit an EMPTY picker and make the + # provider unselectable — a harder failure than surfacing a few + # extra models, and unrecoverable from the UI because + # ``custom_providers`` is hand-edited in config.yaml (the WebUI + # has no write path for it). ``[]`` in practice means a + # leftover/placeholder key, not an intentional deny-all, and no + # deny-all use case exists: a provider the user wants hidden is + # removed from ``custom_providers`` outright. if ids: - _live_set = set(ids) - for _cid in _config_ids: - if _cid not in _live_set: - ids.append(_cid) + if _allowlist_ids: + _allowlist_set = set(_allowlist_ids) + ids = [m for m in ids if m in _allowlist_set] or [] + for _aid in _allowlist_ids: + if _aid not in ids: + ids.append(_aid) else: - ids = list(_config_ids) + ids = list(_allowlist_ids or _config_ids) # ── OpenAI-compat live fetch fallback ────────────────────────────────── # When provider_model_ids() is unavailable or returns [] for a provider diff --git a/api/run_journal.py b/api/run_journal.py index 48cd8ef7..231d6699 100644 --- a/api/run_journal.py +++ b/api/run_journal.py @@ -10,6 +10,7 @@ import os import re import threading import time +from copy import deepcopy from collections import OrderedDict from pathlib import Path from typing import Any, Iterable @@ -130,7 +131,7 @@ def _get_cached_summary(path: Path) -> dict | None: _SUMMARY_CACHE.pop(key, None) return None _SUMMARY_CACHE.move_to_end(key) - return dict(summary) + return deepcopy(summary) def _cache_summary( @@ -147,7 +148,7 @@ def _cache_summary( return key = str(path) with _SUMMARY_CACHE_LOCK: - _SUMMARY_CACHE[key] = (signature, dict(summary)) + _SUMMARY_CACHE[key] = (signature, deepcopy(summary)) _SUMMARY_CACHE.move_to_end(key) while len(_SUMMARY_CACHE) > _SUMMARY_CACHE_MAX_ENTRIES: _SUMMARY_CACHE.popitem(last=False) @@ -534,6 +535,37 @@ def select_authoritative_terminal_event(events: Iterable[dict]) -> dict | None: ) +def runtime_model_from_events(session_id: str, stream_id: str, events: Iterable[dict]) -> dict | None: + """Project only observed serving identity for this journal owner. + + A fallback warning or malformed later observation invalidates old evidence; + the configured selection and free-text status never supply serving identity. + """ + observed = None + for event in events: + if not isinstance(event, dict) or event.get("session_id") != session_id or event.get("run_id") != stream_id: + continue + payload = event.get("payload") + if not isinstance(payload, dict): + if event.get("event") == "runtime_model": + observed = None + continue + if event.get("event") == "warning" and payload.get("type") == "fallback": + observed = None + elif event.get("event") == "runtime_model": + observed = None + if (payload.get("session_id") != session_id or payload.get("stream_id") != stream_id + or not isinstance(payload.get("model"), str) or not payload["model"].strip() + or not isinstance(payload.get("fallback_active"), bool) + or payload.get("phase") not in ("observed_output", "route_observed")): + continue + observed = {key: payload[key] for key in ( + "session_id", "stream_id", "model", "fallback_active", "phase")} + if isinstance(payload.get("provider"), str) and payload["provider"].strip(): + observed["provider"] = payload["provider"] + return observed + + def _summary_from_events(session_id: str, run_id: str, events: Iterable[dict]) -> dict: ordered = [event for event in events if isinstance(event, dict)] last = ordered[-1] if ordered else None @@ -549,6 +581,7 @@ def _summary_from_events(session_id: str, run_id: str, events: Iterable[dict]) - "terminal": bool(terminal), "terminal_state": status, "last_event": (last or {}).get("event"), + "runtime_model": runtime_model_from_events(session_id, run_id, ordered), } diff --git a/api/session_recovery.py b/api/session_recovery.py index d2d46cb2..d9b09479 100644 --- a/api/session_recovery.py +++ b/api/session_recovery.py @@ -601,7 +601,14 @@ def _state_db_row_to_sidecar(row: dict) -> dict: 'session_id': row.get('id'), 'title': row.get('title') or 'Recovered WebUI Session', 'workspace': workspace_value if isinstance(workspace_value, str) else '', - 'message_count': row.get('message_count') if isinstance(row.get('message_count'), int) else len(messages), + # state.db is canonical for the recovered message rows. Its denormalized + # sessions.message_count is absent from older schemas (read above as 0) + # and can be stale after recovery, so never copy it into a sidecar whose + # `messages` array we just materialized. Session.save() uses this field + # to decide whether an overwrite shrinks the transcript; it must match + # the payload exactly or the #1558 backup / empty-snapshot guards can be + # bypassed. + 'message_count': len(messages if isinstance(messages, list) else []), 'worktree_path': row.get('worktree_path') or None, 'worktree_branch': row.get('worktree_branch') or None, 'worktree_repo_root': row.get('worktree_repo_root') or None, diff --git a/api/streaming.py b/api/streaming.py index 050ba1d0..ee1c8418 100644 --- a/api/streaming.py +++ b/api/streaming.py @@ -895,10 +895,39 @@ def _is_fallback_lifecycle_message(kind: str, message: str) -> bool: or 'falling back' in m or 'fallback activated' in m or 'trying fallback' in m + or 'model fallback:' in m + or 'switched to fallback' in m + or 'primary model restored:' in m ) ) +# Session turn-lease notices emitted by the Agent (agent/turn_facade_lease.py) +# while another Hermes process (gateway, CLI, cron) holds this session's turn +# lease. Emitted via ``_emit_status`` (kind ``lifecycle``) while waiting and on +# admission, and via ``_emit_warning`` (kind ``warn``) when the wait times out +# and the message was not processed. +_SESSION_LEASE_WAIT_MARKERS = ( + 'another hermes process is using this session', + 'still waiting for the other hermes process', + 'another hermes process kept this session busy', + 'session is free; loading the latest transcript', +) + + +def _is_session_lease_wait_message(kind: str, message: str) -> bool: + """Return True for Agent session turn-lease wait notices. + + Classification keys on the Agent status kind (``lifecycle`` / ``warn``) so + user-authored text can never be promoted to a warning. + """ + k = str(kind or '').strip().lower() + if k not in ('lifecycle', 'warn'): + return False + m = str(message or '').strip().lower() + return any(marker in m for marker in _SESSION_LEASE_WAIT_MARKERS) + + def _is_agent_compression_start_status(kind: str, message: str) -> bool: """Return True only for real Hermes context-compression start notices. @@ -2351,6 +2380,14 @@ def _prepare_marker_clean_writeback( cleaned, has_verification_nudge = _clean_synthetic_control_messages_with_provenance( result_messages ) + # Same internal-control class, second home: a consumed mid-turn /steer is + # appended to the turn's last tool result wrapped in + # [OUT-OF-BAND USER MESSAGE ...] ... [/OUT-OF-BAND USER MESSAGE]. Strip it + # here, on the rows both writebacks are built from, so neither + # session.messages (rendered verbatim) nor session.context_messages keeps + # the raw wrapper. Stripping the incoming rows too keeps them identity-equal + # to the marker-free rows persisted by earlier turns. (#7600) + cleaned = _strip_oob_markers_from_messages(cleaned) provenance = { 'verification_nudge_seen': has_verification_nudge, 'active_turn_identity': copy.deepcopy(active_turn_identity), @@ -2502,6 +2539,11 @@ def _settle_result_messages( ) ): context_user[_WEBUI_TRUSTED_AGENT_INPUT_FIELD] = trusted_agent_input + # The merge carries earlier display rows across turns verbatim, so a row + # settled before this guard existed would keep its raw wrapper forever. + # Scrub the persisted display copy too — after the merge, so identity + # matching above still saw the rows unchanged. (#7600) + session.messages = _strip_oob_markers_from_messages(session.messages) _annotate_media_snapshots_for_settled_messages(session.messages) _compact_session_image_parts_for_persistence(session) _advance_truncation_watermark_after_commit(session) # #3831 @@ -5719,6 +5761,94 @@ def _strip_oob_blocks(content): return content +_OOB_ANY_OPEN_RE = re.compile( + r'\[OUT-OF-BAND\s+USER\s+MESSAGE(?:\s*(?:—|-)\s*.*?)?\]', + re.IGNORECASE, +) +_OOB_ANY_CLOSE_RE = re.compile( + r'\[/OUT-OF-BAND\s+USER\s+MESSAGE\]', + re.IGNORECASE, +) + + +def _unwrap_single_oob_frame(content: str) -> str | None: + """Unwrap exactly one fully-validated [OUT-OF-BAND USER MESSAGE] frame. + + Returns the extracted inner user text if and only if ``content`` consists of + exactly one valid opening tag and one valid closing tag wrapping the user + content. If markers are multiple, nested, incomplete, or ambiguous, returns + None so caller preserves the row byte-for-byte. + """ + if not isinstance(content, str): + return None + stripped = content.strip() + if not stripped: + return None + + open_matches = list(_OOB_ANY_OPEN_RE.finditer(stripped)) + close_matches = list(_OOB_ANY_CLOSE_RE.finditer(stripped)) + + # Must have exactly one opening marker and one closing marker + if len(open_matches) != 1 or len(close_matches) != 1: + return None + + open_m = open_matches[0] + close_m = close_matches[0] + + # Opening marker must be at the very start of stripped content + if open_m.start() != 0: + return None + + # Closing marker must be at the very end of stripped content + if close_m.end() != len(stripped): + return None + + # Opening marker must end before closing marker starts + if open_m.end() > close_m.start(): + return None + + inner = stripped[open_m.end():close_m.start()] + # Strip surrounding whitespace/newlines from the extracted user text + return inner.strip('\r\n').strip() + + +def _unwrap_steer_row_oob_marker(message: dict) -> None: + """Extract inner steer text from a typed steer row in place (#7600). + + The Hermes Agent emits mid-turn steers as standalone typed user rows + (`role == 'user'`, `display_kind == 'steer'`). The control wrapper + `[OUT-OF-BAND USER MESSAGE ...] ... [/OUT-OF-BAND USER MESSAGE]` is + extracted to preserve only the user-authored instruction. + + Mutates caller-row in place to maintain object identity. If the marker + frame is malformed, nested, multiple, or legacy, preserves the row + byte-for-byte. + """ + if not isinstance(message, dict): + return + if message.get('role') != 'user' or message.get('display_kind') != 'steer': + return + content = message.get('content') + if isinstance(content, str): + unwrapped = _unwrap_single_oob_frame(content) + if unwrapped is not None: + message['content'] = unwrapped + elif isinstance(content, list): + if len(content) == 1 and isinstance(content[0], dict): + part = content[0] + if part.get('type') == 'text' and isinstance(part.get('text'), str): + unwrapped = _unwrap_single_oob_frame(part['text']) + if unwrapped is not None: + part['text'] = unwrapped + + +def _strip_oob_markers_from_messages(messages): + """Unwrap OOB steer markers from typed steer rows in place (#7600).""" + for message in messages or []: + _unwrap_steer_row_oob_marker(message) + return messages + + def _content_has_reasoning_only_parts(content) -> bool: if not isinstance(content, list) or not content: return False @@ -10027,6 +10157,37 @@ def _run_agent_streaming( except Exception: logger.debug("Failed to put event to queue") + _last_runtime_model_identity = None + _runtime_model_session_id = session_id + + def _observe_runtime_model(): + """Publish this turn's Agent identity at output, never at an attempted route.""" + nonlocal _last_runtime_model_identity + raw_model = getattr(agent, 'model', None) + if not isinstance(raw_model, str) or not raw_model.strip(): + return + raw_provider = getattr(agent, 'provider', None) + provider = str(raw_provider).strip().lstrip('@').lower() if isinstance(raw_provider, str) else '' + # The Agent may carry a provider-qualified routing hint in its model. + from api.config import _parse_provider_qualified_model_id + parsed = _parse_provider_qualified_model_id(raw_model.strip()) + model_id = (parsed[0] if parsed else raw_model).strip() + if not model_id: + return + fallback_active = getattr(agent, '_provider_fallback_active', None) is True + identity = (provider, model_id.lower(), fallback_active) + if identity == _last_runtime_model_identity: + return + payload = { + 'session_id': _runtime_model_session_id, 'stream_id': stream_id, + 'model': model_id, 'fallback_active': fallback_active, + 'phase': 'observed_output', + } + if provider: + payload['provider'] = provider + put('runtime_model', payload) + _last_runtime_model_identity = identity + # #5940: capture a terminal (non-retryable) provider error the Agent emits via # its lifecycle status_callback. The Agent aborts a non-retryable API error # (e.g. HTTP 400 "invalid model / no credentials") with @@ -10047,6 +10208,7 @@ def _run_agent_streaming( turn-completion classifier can report the real cause instead of the generic no_response fallback. All other lifecycle messages are dropped. """ + nonlocal _last_runtime_model_identity _message = str(message or '').strip() _kind = str(kind or '').strip().lower() if not _message: @@ -10069,8 +10231,14 @@ def _run_agent_streaming( return # Pass through rate-limit and fallback messages so the frontend can # show them as warnings via the existing messages.js 'warning' listener. + # Session turn-lease waits (another Hermes process owns this session) + # use the same channel so a delayed turn explains itself. + if _is_session_lease_wait_message(_kind, _message): + put('warning', {'type': 'session_lease_wait', 'message': _message}) + return _is_fallback_notice = _is_fallback_lifecycle_message(_kind, _message) if _is_fallback_notice: + _last_runtime_model_identity = None put('warning', {'type': 'fallback', 'message': _message}) # xsession wakeup misroute root fix (Option 1): pre-init so the outer @@ -10618,6 +10786,8 @@ def _run_agent_streaming( nonlocal _token_sent if text is None: return # end-of-stream sentinel + if text: + _observe_runtime_model() # #4729: visible output is starting — flush any buffered reasoning tail # first so the live Thinking stream is complete before/at the transition. _flush_reasoning_buffer() @@ -10661,6 +10831,8 @@ def _run_agent_streaming( # partial window is not lost when the reasoning phase ends. _flush_reasoning_buffer() return + if text: + _observe_runtime_model() _tool_boundary_advanced = False reasoning_delta = str(text) # Some runtimes mirror user-visible progress text through the @@ -11873,6 +12045,11 @@ def _run_agent_streaming( if isinstance(_m, dict) and _m.get('role') == 'assistant': _answer = str(_m.get('content', '')) break + if (_answer.strip() and not result.get('error') + and not _agent_result_terminal_failure(result) + and not getattr(agent, '_last_error', None) + and not _captured_terminal_error[0]): + _observe_runtime_model() # /btw is intentionally non-persistent, but its terminal SSE # payload is still public output. Project the ephemeral # session before enqueueing it so raw Agent ``api_content`` or @@ -12584,6 +12761,8 @@ def _run_agent_streaming( # the catch-all label, hint, and provider details. return # apperror already closes the stream on the client side + _observe_runtime_model() + # ── Handle context compression side effects ── # Also detect compression via the result dict or compressor state if not _compressed: @@ -12786,7 +12965,9 @@ def _run_agent_streaming( # mutates agent.model when a fallback fires, so the pre-run # resolved_model would mis-attribute exactly the turns where # attribution matters most. - _used_model = getattr(agent, 'model', None) or resolved_model or model + # The configured selection is not proof that it served this turn. + _observed_model = getattr(agent, 'model', None) + _used_model = _observed_model.strip() if isinstance(_observed_model, str) else None if _gateway_routing: s.gateway_routing = _gateway_routing _history = list(getattr(s, 'gateway_routing_history', None) or []) @@ -13672,6 +13853,9 @@ def _run_agent_streaming( if 'credential_pool' in _agent_params: _heal_kwargs['credential_pool'] = _runtime_bundle['credential_pool'] _heal_agent = _AIAgent(**_heal_kwargs) + # Delta callbacks close over `agent`; the replacement, not + # the failed original, owns any successful retry output. + agent = _heal_agent _agent_sig = _compute_agent_cache_signature( resolved_model, resolved_api_key, @@ -13808,6 +13992,7 @@ def _run_agent_streaming( s, tool_calls=s.tool_calls ) ) + _observe_runtime_model() if _done_session_payload is not None: put('done', { 'session': _done_session_payload, diff --git a/docs/CONTRACTS.md b/docs/CONTRACTS.md index 2989b587..0b81b9bb 100644 --- a/docs/CONTRACTS.md +++ b/docs/CONTRACTS.md @@ -89,6 +89,15 @@ contributor guidance; it does not change runtime behavior or CI gates. `models_cache.json` refresh timestamps), the stat-identity fallbacks, and the schema/version stamps. Start here before changing model-catalog caching, the `/api/models` cache keys, or the Codex catalog dependency (#2443, #7540). +- [`docs/architecture/live-models-allowlist.md`](architecture/live-models-allowlist.md): + current contract for how `/api/models/live` filters a custom provider's + upstream catalog: the four signals in evaluation order (discovered catalog + defers to the live probe, explicit plural `models:` allowlist filters, + singular `model:` never gates, no allowlist shows the full catalog), the + serialized-list shapes `hermes config set` persists, the deliberate + empty-allowlist-is-not-configured rule, and the probe-failure fallback. + Start here before changing custom-provider model filtering or + discovery-vs-allowlist semantics (#7165, #7404). - [`docs/architecture/profile-home-resolve-cache.md`](architecture/profile-home-resolve-cache.md): current contract for the call-scoped memoization around `_resolve_profile_home_param()` in `api/workspace.py`: what it caches, the diff --git a/docs/architecture/live-models-allowlist.md b/docs/architecture/live-models-allowlist.md new file mode 100644 index 00000000..83c7363a --- /dev/null +++ b/docs/architecture/live-models-allowlist.md @@ -0,0 +1,123 @@ +# Live model picker allowlist contract (`/api/models/live`) + +This document records the runtime contract for how `/api/models/live` decides +which models a `custom` / `custom:*` provider contributes to the chat model +picker. It describes shipped behavior and changes no runtime behavior. It was +added because the #7165 review flagged the contract as undocumented: the +filtering rules live in `_handle_live_models` (`api/routes.py`) and had only +code comments backing them. + +Start here before changing how the live catalog of a custom provider is +filtered, how discovery snapshots are distinguished from allowlists, or what +the endpoint returns when the upstream probe fails. + +## Why the endpoint filters at all + +OpenAI-compatible gateways (New-API-style aggregators, LM Studio, Ollama +proxies) expose their **entire** catalog from `/v1/models`, including +image-generation, audio, TTS, and embedding models that are not chat models. +The background enrichment path must not surface models the user never opted +into — but it also must not hide models the gateway gained since config was +last edited. The contract below is how those two goals are reconciled. + +## The four signals, in evaluation order + +For a custom provider the handler consults these signals: + +1. **Discovered catalog → no allowlist (live probe wins).** + When the provider entry carries `models_discovered: true` and discovery is + allowed (`discover_models` is not an explicit opt-out), the `models:` + mapping is a *snapshot Hermes persisted from a past discovery*, not a + hand-curated list. It must not gate the live probe: a model the user pulls + into LM Studio / Ollama after discovery has to appear in the picker. + Detected by the shared predicate + `api.config._provider_models_are_discovered_catalog()` — the same + predicate the `/api/models` catalog path uses, so both endpoints agree. + With this signal, no allowlist is derived and the full live catalog is + returned. + +2. **Explicit plural `models:` allowlist → filter.** + A non-empty plural `models:` value on an entry that is *not* a discovered + catalog expresses "show exactly these models". The live catalog is + intersected with it, then any allowlisted model missing from the live + response is appended (so an allowlisted-but-offline model stays + selectable). This is the #7165 filter. Only **list** and **string** + (serialized) shapes are plain allowlists — see "Dict-shaped `models:`" + below. + +3. **Singular `model:` → never gates.** + The singular field is sticky/default metadata (the model to preselect), + not an allowlist. A provider configured only with `model: assistant` keeps + the full unfiltered live catalog; gating on it would collapse discovery to + a single model. + +4. **No allowlist → full live catalog.** + Preserves the pre-#7165 discovery behavior for setups that rely on it. + +### Serialized shapes + +`hermes config set` and the JSON-mode editor persist lists as quoted +JSON-array strings (`'["chat-a","chat-b"]'`) or Python literals +(`"['chat-a']"`). The plural value is decoded through the shared +`_parse_config_string_list()` before id extraction — but only when the value +is a `str`; native `dict` / `list` values are walked directly so dict *entries* +keep their `id|model|name` metadata (the decoder stringifies members). A +dict-only parser drops serialized allowlists to "no allowlist" and floods the +picker — the same bug class as `skills.disabled` (#7120 / #7134). + +### Dict-shaped `models:` + +A dict-shaped `models:` mapping (e.g. `{chat-a: {context_length: 128000}}`) +is *per-model metadata* written by the Hermes Agent setup flow — +`hermes_cli/model_switch.py::_save_custom_provider` and the setup wizard — +**not** a catalog narrow. Treating its keys as an allowlist would collapse the +live picker to the single saved default (keyless Ollama) while the CLI +live-probe shows the full catalog. So a dict is **never** a plain allowlist: + +- without an explicit `discover_models: false`, a dict-shaped `models:` + contributes **no allowlist** and the full live catalog is returned; +- with `discover_models: false` (bool `False` or the string forms + `"false"`/`"no"`/`"0"`, case-insensitive — mirroring + `model_switch_providers.py:557`), the dict **keys** are honored as a pinned + allowlist. + +This complements the discovered-catalog rule: `models_discovered: true` +without the opt-out is already ignored (signal 1). The dict guard covers the +Agent-setup shape that is *not* flagged discovered. + +If someone later "fixes" the dict shape back into an always-allowlist, the +regression test `test_dict_models_without_discover_false_does_not_gate` fails +and forces the trade-off to be re-argued. + +### Empty allowlist semantics + +`models: []`, `models: "[]"`, an empty mapping, or a blank scalar are +deliberately treated as **not configured**, never as "allow nothing". The +filter runs only on a non-empty allowlist because gating on declared-ness +would emit an empty picker and make the provider unselectable — a harder +failure than showing a few extra models, and unrecoverable from the UI +(`custom_providers` has no WebUI write path; it is hand-edited in +config.yaml). There is no deny-all use case: a provider the user wants hidden +is removed from `custom_providers` outright. + +### Probe failure / empty probe + +When the upstream probe fails or returns no models, the endpoint falls back +to `allowlist or config ids` — the non-empty plural allowlist first, so a +singular `model:` outside the allowlist cannot leak in precisely when the +probe is unavailable; with no allowlist, the configured ids (singular + +plural) are served. A discovered catalog therefore still serves its saved +models on a transient probe failure instead of emptying the picker. + +## Tests + +- `tests/test_issue3718_live_models_custom_probe.py` — real-handler + (`_handle_live_models`) coverage of every signal above: plural list / + JSON-array-string / Python-literal / list-of-dicts filtering, dict-shaped + `models:` (not gated) and dict + `discover_models: false` (pinned), + singular passthrough, malformed safety, discovered-catalog passthrough, + `discover_models: false` pinning, and both fallback branches. +- `tests/test_issue7404_models_discovered_not_allowlist.py` — the + `/api/models` side of the discovered-catalog contract. +- `tests/test_byok_model_dropdown.py` — picker rendering, profile-scoped + caching, and current-session model preservation around this endpoint. diff --git a/docs/rfcs/session-sse-contract-v1.md b/docs/rfcs/session-sse-contract-v1.md index e6248e03..9abbc1a2 100644 --- a/docs/rfcs/session-sse-contract-v1.md +++ b/docs/rfcs/session-sse-contract-v1.md @@ -162,7 +162,7 @@ gate rather than inventing values without source support. ## Authoritative emitted events (`/api/chat/stream`) These are the **real wire `event:` names** emitted by `api/streaming.py` today -(23 names). Clients and docs must use this table — not the semantic draft above — +(24 names). Clients and docs must use this table — not the semantic draft above — when integrating with the live chat SSE relay. | Wire name | Role | @@ -179,6 +179,7 @@ when integrating with the live chat SSE relay. | `title` | Session title update (often after `done`) | | `title_status` | Title generation status / skip reason | | `warning` | Non-fatal provider/fallback warning | +| `runtime_model` | Local Agent serving identity observed at output or successful completion | | `apperror` | Terminal application error (no trailing `stream_end`) | | `cancel` | Run cancelled | | `done` | Turn finalized (session payload); title/`stream_end` may follow | @@ -199,6 +200,31 @@ The semantic taxonomy table remains a draft for the proposed per-session endpoint vocabulary and must be confirmed during maintainer review before that endpoint claims parity. +### Observed local runtime model + +The local worker emits `runtime_model` with +`{session_id, stream_id, model, provider?, fallback_active, phase}`. The IDs +refer to the original run-journal owner even if compression rotates the Agent's +session. `phase="observed_output"` means the Agent's own model was read at a +nonempty token/reasoning callback or after successful completion (including a +non-streaming reply or credential self-heal). It does not promise that a turn +which emitted partial output will finish successfully. Missing Agent model +sends no observation; the configured selection is not used as serving proof. +`fallback_active` is true only when the Agent explicitly reports its fallback +flag. Consecutive identical observations are deduplicated within the turn. + +Fallback lifecycle warnings invalidate older serving evidence and reset the +deduplication; status text alone never establishes the replacement model. A +fresh observation after the warning reestablishes it, including when a buffered +success notice arrives after output. Durable replay retains the event IDs. +The journal summary and HTTP `runtime_journal_snapshot.runtime_model` project +the latest valid observation from the same session/stream event window, or null +if a warning or malformed latest observation invalidated it. No previous run's +footer or selected route fills an unknown current run. This is a local-worker +producer; gateway attribution and frontend presentation are separate concerns +(see #6272 and #7181). `route_observed` remains reserved for route-only data, +not successful output. + ## Cursor and resume semantics `Last-Event-ID` is the standard SSE reconnect header. Clients send the last diff --git a/docs/rfcs/webui-pending-intent-controls.md b/docs/rfcs/webui-pending-intent-controls.md index d8c3e65a..d2195e74 100644 --- a/docs/rfcs/webui-pending-intent-controls.md +++ b/docs/rfcs/webui-pending-intent-controls.md @@ -215,6 +215,14 @@ settle. Steer must not exist only as a toast or transient DOM state. It must survive refresh, session switch, replay, and settled render. +**The transport wrapper is not the Steer row.** A Steer is emitted as a standalone +typed user row (`role: user`, `display_kind: "steer"`). The surrounding +`[OUT-OF-BAND USER MESSAGE …] … [/OUT-OF-BAND USER MESSAGE]` block is transport-level +control data: settlement unwraps the single validated frame in place, extracting +and preserving the user's authored steer instruction while dropping the wrapper. +Legacy tool rows and rows where markers are multiple, nested, incomplete, or +contain ambiguous delimiters are preserved byte-for-byte. + ### Relationship to system control events Steer is similar to system-delivered control events such as tool-iteration-limit diff --git a/docs/sse-streams.md b/docs/sse-streams.md index 07fca5b6..12eb3898 100644 --- a/docs/sse-streams.md +++ b/docs/sse-streams.md @@ -56,6 +56,21 @@ infer it from the status code. Clients that only need session updates should use `/api/session/stream` directly; the gateway probe is only relevant for clients that display CLI/TUI/messaging sessions. +## Gateway watcher projection lifecycle + +The optional gateway stream's watcher owns a cheap database fingerprint and a +timestamp for its last full session projection. While clients are subscribed, +an unchanged fingerprint can skip the expensive read until the bounded parity +interval. With no subscribers the poll loop parks instead of reading the DB. +When the final subscriber leaves (including eviction of a full or failed +consumer queue), the watcher invalidates both cache fields. A reconnect must +therefore trigger a fresh projection even when the database fingerprint has not +changed and the parity interval has not elapsed. Final removal also advances a +subscriber epoch: an older in-flight DB read cannot restore the invalidated +cache or publish its result to a later subscriber cohort. The subscriber lock +protects removal, epoch checks and cache commits, but is not held during DB +reads, so disconnect is not blocked by a slow projection. + ## Heartbeats and proxy behavior - All long-lived streams emit SSE keepalive comment lines on the diff --git a/static/boot.js b/static/boot.js index d6476b60..06c3888d 100644 --- a/static/boot.js +++ b/static/boot.js @@ -3431,8 +3431,17 @@ window._mirrorSpeechSettingsFromServer=_mirrorSpeechSettingsFromServer; localStorage.setItem('hermes-font-size',fontSize); _applyFontSize(fontSize); if(typeof setLocale==='function'){ + // #7622 (round 3): the settings payload's `s.language` is + // absent (None) for a fresh install, so an explicit non-empty + // value is the user's genuine saved choice. The browser + // navigator hint is now read via the guarded + // `_detectBrowserLanguageHint()` helper (round-3 finding 2) so + // a throwing `navigator` accessor in some embedded webviews + // can no longer abort this branch and reset loaded preferences. + // The fallback ternary preserves the pre-#7622 boot behaviour + // when neither helper is in scope (defence in depth). const _lang=typeof resolvePreferredLocale==='function' - ? resolvePreferredLocale(s.language, localStorage.getItem('hermes-lang')) + ? resolvePreferredLocale(s.language, localStorage.getItem('hermes-lang'), _detectBrowserLanguageHint()) : (s.language || localStorage.getItem('hermes-lang') || 'en'); setLocale(_lang); if(typeof applyLocaleToDOM==='function')applyLocaleToDOM(); diff --git a/static/i18n.js b/static/i18n.js index 0192fd5c..269ffe43 100644 --- a/static/i18n.js +++ b/static/i18n.js @@ -26983,15 +26983,68 @@ function resolveLocale(lang) { /** * Resolve locale with precedence: - * 1) primary (typically server setting) - * 2) fallback (typically localStorage) - * 3) English - * @param {string} primary - * @param {string} fallback + * 1) primary (typically server setting) + * 2) fallback (typically localStorage) + * 3) fallback2 (typically browser navigator hint — first-visit only) + * 4) 'en' (final fallback) + * + * Resolve a language preference to a known LOCALES key, applying a strict + * `primary -> fallback -> fallback2 -> 'en'` precedence chain. + * + * #7622 (round 3): the server no longer injects `"language": "en"` as a + * schema default — `load_settings()` reports `None` (the key is absent + * from the response) until the user explicitly picks a locale in the + * Settings modal. This lets the resolver use `primary` directly + * without a `primary === 'en'` special case: + * - an explicit `primary` (the server's stored value) always wins + * - a `None` / empty `primary` falls through to `fallback` (localStorage) + * - `fallback2` (the browser navigator hint) only fires when neither + * server nor localStorage has a preference + * - `'en'` is the final safety net + * + * Side benefit: a user who picked English on purpose (their settings + * file contains `"language": "en"`) is no longer overridden by a + * non-English browser locale on hydration, because the round-2 + * `primary === 'en'` skip has been removed. + * + * @param {string|null|undefined} primary server-stored language + * @param {string|null|undefined} fallback localStorage `hermes-lang` + * @param {string|null|undefined} fallback2 browser navigator hint * @returns {string} */ -function resolvePreferredLocale(primary, fallback) { - return resolveLocale(primary) || resolveLocale(fallback) || 'en'; +function resolvePreferredLocale(primary, fallback, fallback2) { + return resolveLocale(primary) + || resolveLocale(fallback) + || resolveLocale(fallback2) + || 'en'; +} + +/** + * Detect the browser's preferred language for first-visit locale + * selection. Guarded against `navigator` access exceptions: some + * embedded webviews (Tauri desktop shell, kiosk modes) report a + * `navigator` object but throw on `.languages` / `.language` reads. + * Callers (loadLocale, boot.js, panels.js) must use this helper rather + * than reading `navigator` directly so a single throw does not abort + * the boot or settings-panel hydration. + * + * @returns {string|null} the first entry of `navigator.languages`, or + * `navigator.language`, or `null` when neither is reachable. + */ +function _detectBrowserLanguageHint() { + try { + const nav = (typeof navigator !== 'undefined') ? navigator : null; + if (!nav) return null; + if (Array.isArray(nav.languages) && nav.languages.length) { + return nav.languages[0] || null; + } + if (typeof nav.language === 'string' && nav.language) { + return nav.language; + } + return null; + } catch (_) { + return null; + } } /** @@ -27029,11 +27082,26 @@ function setLocale(lang) { /** * Load locale from localStorage (called once at boot, before DOMContentLoaded). * Server-persisted preference is applied later in loadSettingsPanel(). + * + * #7622: on the FIRST visit (no stored preference), fall back to the + * browser's preferred language (`navigator.languages[0]` / + * `navigator.language`) so non-English speakers don't have to dig into + * Settings after every fresh install. This is strictly a first-visit + * hint — the moment the user picks a language in Settings (or the + * server-side preference is applied via `loadSettingsPanel`), the + * stored value takes over and the browser hint is never consulted + * again for that browser profile. */ function loadLocale() { let stored = null; try { stored = localStorage.getItem('hermes-lang'); } catch (_) {} - setLocale(resolvePreferredLocale(null, stored)); + // Only consult the browser hint when no stored preference exists. + // The settings panel and `loadSettingsPanel()` will overwrite this + // hint on their first call, so explicit user / server preferences + // always win. #7622 (round 3): the browser-hint read is delegated + // to `_detectBrowserLanguageHint()` so a `navigator` access throw + // (some embedded webviews) can no longer abort loadLocale(). + setLocale(resolvePreferredLocale(stored, _detectBrowserLanguageHint())); } /** diff --git a/static/messages.js b/static/messages.js index 45cc458f..dbd13588 100644 --- a/static/messages.js +++ b/static/messages.js @@ -2760,11 +2760,18 @@ function attachLiveStream(activeSid, streamId, uploaded=[], options={}){ let _currentActivityBurstId=Number((INFLIGHT[activeSid]&&INFLIGHT[activeSid].currentActivityBurstId)||0)||0; let _currentLiveSegmentSeq=Number((INFLIGHT[activeSid]&&INFLIGHT[activeSid].currentLiveSegmentSeq)||0)||0; let _assistantSegmentSeq=Number((INFLIGHT[activeSid]&&INFLIGHT[activeSid].currentLiveSegmentSeq)||0)||0; - let _lastRunJournalSeq=reconnecting - ? Number((INFLIGHT[activeSid]&&INFLIGHT[activeSid].lastRunJournalSeq)||0) + // #7640: the replay floor is only as trustworthy as the recovery state behind + // it. A cache that kept the cursor but lost the live assistant projection must + // not raise `after_seq`: the server would then replay only the tail (often just + // `stream_end`), and the missing journal range never gets a chance to rebuild + // the body — the settled footer paints over a blank message until a reload. + // Fall back to the zero floor whenever the state cannot be validated. + const _replayCursorInflight=reconnecting?INFLIGHT[activeSid]:null; + let _lastRunJournalSeq=(typeof _runJournalReplayFloorForInflight==='function') + ? _runJournalReplayFloorForInflight(_replayCursorInflight) : 0; - let _lastRunJournalEventId=reconnecting - ? String((INFLIGHT[activeSid]&&INFLIGHT[activeSid].lastRunJournalEventId)||'') + let _lastRunJournalEventId=(typeof _runJournalReplayEventIdForInflight==='function') + ? _runJournalReplayEventIdForInflight(_replayCursorInflight) : ''; const _STREAM_FADE_MS=620; const _STREAM_FADE_MAX_MS=900; @@ -4846,7 +4853,12 @@ function attachLiveStream(activeSid, streamId, uploaded=[], options={}){ function _smdMediaTailFlushEntry(entry){ const chunk=_smdMediaTailEntryChunk(entry); if(!chunk) return; - const m=/^MEDIA:([^\s\)\]]+)$/.exec(String(chunk)); + // #7680 re-gate (9/22): strip backtick wrappers so the bare-token + // match below sees a plain ``MEDIA:path`` and the bare class + // (no backtick in the exclusion set) captures the full filename + // even when the path itself contains a backtick. + const normalized = String(chunk).replace(/`MEDIA:([^`\s]+)`/g, 'MEDIA:$1'); + const m=/^MEDIA:([^\s\)\]]+)$/.exec(normalized); const emitted=!!(m && entry && entry.parent && _smdAppendMediaNode(entry.parent, m[1])); if(!emitted && entry) _smdMediaWriteText(entry.parent, entry.data, entry.baseAddText, entry.writeText, chunk); } @@ -4894,23 +4906,29 @@ function attachLiveStream(activeSid, streamId, uploaded=[], options={}){ // Prose runs go through the owning text writer. MEDIA tokens go through // the single-token DOMParser helper only after a delimiter or // reliable filename suffix proves the ref is complete. + // #7680 re-gate (9/22): strip backtick wrappers first so the bare + // class (no backtick in the exclusion set) captures the full + // filename even when the path itself contains a backtick. + // The pre-pass replaces `` `MEDIA:path` `` with ``MEDIA:path`` + // so the wrapped form is consumed before the bare scan. + const normalized = combined.replace(/`MEDIA:([^`\s]+)`/g, 'MEDIA:$1'); const re=/MEDIA:([^\s\)\]]+)/g; let last=0, m; let unmatchedTail=null; - while((m=re.exec(combined))){ + while((m=re.exec(normalized))){ const matchEnd = m.index + m[0].length; if(m.index>last){ - const slice = combined.slice(last, m.index); + const slice = normalized.slice(last, m.index); writeCurrent(slice); } - if(matchEnd===combined.length && !_smdMediaRefHasReliableBoundary(m[1])){ - const candidate = combined.slice(m.index); + if(matchEnd===normalized.length && !_smdMediaRefHasReliableBoundary(m[1])){ + const candidate = normalized.slice(m.index); if(candidate.length < _MEDIA_TAIL_MAX){ unmatchedTail = candidate; } else { writeCurrent(candidate); } - last = combined.length; + last = normalized.length; break; } if(!_smdAppendMediaNode(parent, m[1])) writeCurrent(m[0]); @@ -4918,7 +4936,7 @@ function attachLiveStream(activeSid, streamId, uploaded=[], options={}){ } // Tail buffer — hold trailing bytes that look like an unterminated // MEDIA prefix; flush any prose before the partial MEDIA suffix. - const rest = combined.slice(last); + const rest = normalized.slice(last); if(rest){ const tailMatch = /MEDIA:[^\s\)\]]*$/.exec(rest); const prefixTail = tailMatch ? '' : _smdMediaPrefixTail(rest); diff --git a/static/panels.js b/static/panels.js index dd5090c3..4f708aae 100644 --- a/static/panels.js +++ b/static/panels.js @@ -25,6 +25,7 @@ let _currentCronDetail = null; // full cron job object let _currentCronDetailKey = ''; let _cronMode = 'empty'; // 'empty' | 'read' | 'create' | 'edit' let _cronPreFormDetail = null; // snapshot of prior selection when entering a form +let _cronModelPickerTouched = false; // true once the user changes the model picker in the current form let _showAllCronProfiles = false; let _cronOtherProfileCount = 0; let _currentWorkspaceDetail = null; // { path, name, is_default } @@ -1550,6 +1551,7 @@ function duplicateCurrentCron(){ const job = _currentCronDetail; if (typeof switchPanel === 'function' && _currentPanel !== 'tasks') switchPanel('tasks'); _cronPreFormDetail = { ...job }; + _cronModelPickerTouched = false; _editingCronId = null; _cronMode = 'create'; _cronIsDuplicate = true; @@ -1606,6 +1608,7 @@ let _cronDeliveryOptionsCache=null; function openCronCreate(){ if (typeof switchPanel === 'function' && _currentPanel !== 'tasks') switchPanel('tasks'); _cronPreFormDetail = _currentCronDetail ? { ..._currentCronDetail } : null; + _cronModelPickerTouched = false; _editingCronId = null; _cronMode = 'create'; _cronIsDuplicate = false; @@ -1623,6 +1626,7 @@ function openCronCreate(){ function openCronEdit(job){ if (!job) return; _cronPreFormDetail = { ...job }; + _cronModelPickerTouched = false; _editingCronId = job.id; _cronMode = 'edit'; _cronSelectedSkills = Array.isArray(job.skills) ? [...job.skills] : []; @@ -1855,6 +1859,9 @@ async function _populateCronFormModelSelect(selectedModel, selectedProvider, dis sel.appendChild(opt); } sel.dataset.loaded = '1'; + // Track deliberate picker changes so an untouched "Default" can keep a + // hidden provider-only pin on save (see _cronProviderForClear). + sel.addEventListener('change', () => { _cronModelPickerTouched = true; }); } catch (e) { console.warn('Failed to load cron model picker:', e.message); // Load failed: dataset.loaded stays unset so saveCronForm omits model/provider @@ -1940,6 +1947,18 @@ function _cronModelBareName(model, provider) { return _modelBareNameForProvider(model, provider); } +function _cronProviderForClear(prevDetail, pickerTouched) { + // Provider to submit when the picker shows "Default" (no model) on save. + // An explicit return to Default honors "Default = no overrides" (#4030) and + // clears the provider. An untouched picker must not: provider-only jobs + // (a pinned provider with no model) render as "Default" in the combined + // picker, so saving any other field would silently erase the pin the user + // never touched. + const prev = prevDetail || {}; + if (pickerTouched) return null; + return (prev.model == null) ? prev.provider : null; +} + async function saveCronForm(){ const nameEl=$('cronFormName'); const schEl=$('cronFormSchedule'); @@ -1977,8 +1996,11 @@ async function saveCronForm(){ updates.model = _cronModelBareName(modelState.model, modelState.model_provider) || null; updates.provider = modelState.model_provider || null; } else if (modelLoaded) { + // "Default" selected (no model). An untouched picker preserves a + // hidden provider-only pin; a deliberate return to Default clears + // it (#4030 "Default = no overrides"). updates.model = null; - updates.provider = null; + updates.provider = _cronProviderForClear(_cronPreFormDetail, _cronModelPickerTouched); } // else: select not yet populated — omit model/provider to preserve saved value } @@ -9300,8 +9322,17 @@ async function loadSettingsPanel(){ _setHiddenTabs(hiddenTabs); _applyTabVisibility(hiddenTabs); _renderTabVisibilityChips(); + // #7622 (round 3): the settings payload's `settings.language` is + // absent (None) for a fresh install, so an explicit non-empty + // value is the user's genuine saved choice. The browser + // navigator hint is now read via the guarded + // `_detectBrowserLanguageHint()` helper (round-3 finding 2) so + // a throwing `navigator` accessor can no longer abort settings + // hydration before model, provider, plugin and extension sections + // are populated. The fallback ternary preserves the pre-#7622 + // settings-modal behaviour when neither helper is in scope. const resolvedLanguage=(typeof resolvePreferredLocale==='function') - ? resolvePreferredLocale(settings.language, localStorage.getItem('hermes-lang')) + ? resolvePreferredLocale(settings.language, localStorage.getItem('hermes-lang'), _detectBrowserLanguageHint()) : (settings.language || localStorage.getItem('hermes-lang') || 'en'); // Keep settings modal and current page strings in sync with the resolved locale. if(typeof setLocale==='function'){ diff --git a/static/sessions.js b/static/sessions.js index f748f45a..593d660a 100644 --- a/static/sessions.js +++ b/static/sessions.js @@ -1035,6 +1035,74 @@ function _inflightHasVisibleLiveState(inflight) { return false; } +// #7640: a run-journal replay cursor is only safe when the recovery object can +// still repaint the live assistant turn. `_inflightHasVisibleLiveState()` above +// deliberately counts a plain user row (the optimistic row has to survive a +// mid-turn reload), but such an object carries no assistant projection at all: +// seeding `after_seq` from it tells the server to skip every journal event that +// would rebuild the body, so the settled footer lands over a blank message until +// a manual reload replays the journal from the zero floor. Only recoverable live +// output may raise the replay floor. +function _inflightCanSeedJournalReplay(inflight){ + if(!inflight||typeof inflight!=='object') return false; + if(String(inflight.lastAssistantText||'').trim()) return true; + if(String(inflight.lastReasoningText||'').trim()) return true; + if(String(inflight.liveTurnHtml||'').trim()) return true; + if(Array.isArray(inflight.toolCalls)&&inflight.toolCalls.length) return true; + if(Array.isArray(inflight.activityBurstAnchors)&&inflight.activityBurstAnchors.length) return true; + const anchorScene=inflight.anchorActivityScene; + if(anchorScene&&Array.isArray(anchorScene.activity_rows)&&anchorScene.activity_rows.length) return true; + if(Array.isArray(inflight.messages)){ + // The transcript copy carries every earlier turn (#7651): in an established + // conversation the last assistant row is the PREVIOUS turn's reply, not the + // one the cursor belongs to, so it must never authorize a nonzero floor. + // Message evidence counts only from a current `_live` assistant standing + // after the latest user boundary; anything historical fails closed to zero. + const list=inflight.messages; + let latestUserIdx=-1; + for(let i=list.length-1;i>=0;i--){ + const row=list[i]; + if(row&&row.role==='user'){latestUserIdx=i;break;} + } + return list.some((msg,idx)=>{ + if(!msg||msg.role!=='assistant') return false; + if(!msg._live) return false; + if(idx<=latestUserIdx) return false; + const content=msg.content; + if(typeof content==='string') return Boolean(content.trim()); + if(Array.isArray(content)) return content.length>0; + return Boolean(content); + }); + } + return false; +} + +function _runJournalReplayFloorForInflight(inflight){ + if(!_inflightCanSeedJournalReplay(inflight)) return 0; + return Math.max(0,Number((inflight&&inflight.lastRunJournalSeq)||0)||0); +} + +function _runJournalReplayEventIdForInflight(inflight){ + if(!_runJournalReplayFloorForInflight(inflight)) return ''; + return String((inflight&&inflight.lastRunJournalEventId)||''); +} + +// #7640: re-validate the selected recovery object immediately before reattach +// instead of trusting its existence. A cursor that outlived its assistant +// projection is dropped here so it cannot reach the wire as a replay floor; the +// replay then restarts from zero, which is the path a manual reload already +// proves out. Normalising the object (not just the floor) also keeps a later +// persist from resurrecting the stale cursor on the next reattach. +function _normalizeInflightReplayCursorForReattach(inflight){ + if(!inflight||typeof inflight!=='object') return inflight; + const seq=Math.max(0,Number(inflight.lastRunJournalSeq||0)||0); + if(seq>0&&!_inflightCanSeedJournalReplay(inflight)){ + inflight.lastRunJournalSeq=0; + inflight.lastRunJournalEventId=''; + } + return inflight; +} + function _serverLiveSnapshotToolId(tc){ return String(tc&&(tc.tid||tc.id||tc.tool_call_id||tc.tool_use_id||tc.call_id||'')||'').trim(); } @@ -2208,6 +2276,11 @@ async function loadSession(sid){ if(INFLIGHT[sid].reattach&&activeStreamId&&typeof attachLiveStream==='function'){ INFLIGHT[sid].reattach=false; if (!_isCurrentLoad()) return; + // #7640: validate the selected recovery object at the moment of reattach + // rather than trusting its existence. A cache that kept `lastRunJournalSeq` + // but lost the live assistant projection would otherwise seed the replay + // floor and skip the journal range that rebuilds the message body. + _normalizeInflightReplayCursorForReattach(INFLIGHT[sid]); didReconnect=true; attachLiveStream(sid, activeStreamId, S.session.pending_attachments||[], {reconnecting:true}); } @@ -2337,6 +2410,8 @@ async function loadSession(sid){ if(activeStreamId){ S.busy=true; S.activeStreamId=activeStreamId; + // #7640: same reattach validation as the `reattach` branch above. + if(INFLIGHT[sid]) _normalizeInflightReplayCursorForReattach(INFLIGHT[sid]); if(typeof attachLiveStream==='function') attachLiveStream(sid, activeStreamId, S.session.pending_attachments||[], {reconnecting:true}); else if(typeof watchInflightSession==='function') watchInflightSession(sid, activeStreamId); updateSendBtn(); @@ -7596,7 +7671,50 @@ function _sidebarRowHasVisibleMessages(s, activeSidForSidebar){ (S.session&&s.session_id===S.session.session_id&&(S.session.message_count||0)>0); } +function _isDelegatedSubagentRow(s){ + if(!_isChildSession(s)) return false; + const role=[s.raw_source,s.source_tag,s.source].map(v=>String(v||'').trim().toLowerCase()).find(Boolean)||''; + return role==='subagent'; +} + +// Delegated subagents without a project_id take their nearest ancestor's project; +// every other row keeps its own. Memoized per render, so each lineage resolves once. +function _sidebarProjectResolver(rowsById){ + const memo=new Map(); + return function projectIdFor(s){ + if(!s) return null; + const path=[]; + const onPath=new Set(); + let cur=s, result=null; + while(cur){ + const sid=cur.session_id; + if(memo.has(sid)){ result=memo.get(sid); break; } + if(cur.project_id||!_isDelegatedSubagentRow(cur)){ result=cur.project_id||null; if(sid) path.push(sid); break; } + if(onPath.has(sid)) break; + onPath.add(sid); path.push(sid); + cur=rowsById?rowsById.get(cur.parent_session_id):null; + } + for(const sid of path) memo.set(sid,result); + return result; + }; +} + +function _sidebarRowsById(rows){ + const byId=new Map(); + for(const list of rows){ + if(!Array.isArray(list)) continue; + for(const s of list) if(s&&s.session_id&&!byId.has(s.session_id)) byId.set(s.session_id,s); + } + return byId; +} + +// True when any row resolves to no project, using the filter's own ancestor map. +function _sidebarHasUnprojectedRows(rows, projectIdFor){ + return rows.some(s=>!projectIdFor(s)); +} + function _partitionSidebarSessionRows(allMatched, activeSidForSidebar){ + const projectIdFor=_sidebarProjectResolver(_sidebarRowsById([allMatched, typeof _sidebarReferenceSessions!=='undefined'?_sidebarReferenceSessions:null])); let cliSessionCount=0; const webuiProfileFiltered=[]; const cliProfileFiltered=[]; @@ -7615,10 +7733,11 @@ function _partitionSidebarSessionRows(allMatched, activeSidForSidebar){ const referenceRaw=isCli ? cliReferenceRaw : webuiReferenceRaw; const sessionsRaw=isCli ? cliSessionsRaw : webuiSessionsRaw; profileFiltered.push(s); + const projectId=projectIdFor(s); if(_activeProject===NO_PROJECT_FILTER){ - if(s.project_id) continue; + if(projectId) continue; } else if(_activeProject){ - if(s.project_id!==_activeProject) continue; + if(projectId!==_activeProject) continue; } referenceRaw.push(s); if(s.archived){ @@ -7642,6 +7761,7 @@ function _partitionSidebarSessionRows(allMatched, activeSidForSidebar){ cliReferenceRaw, webuiSessionsRaw, cliSessionsRaw, + projectIdFor, }; } @@ -7653,15 +7773,18 @@ function _partitionSidebarSessionRows(allMatched, activeSidForSidebar){ // suppression context — silently hiding a visible child/fork whose archived // ancestor lives outside the current view. Scope the references to the same // project + source bucket as the render they feed before using them. -function _scopedSidebarReferenceRows(isCli){ +// projectIdFor: the render's shared resolver; standalone callers get one over references + _allSessions. +function _scopedSidebarReferenceRows(isCli, projectIdFor){ if(typeof _sidebarReferenceSessions==='undefined'||!Array.isArray(_sidebarReferenceSessions)||!_sidebarReferenceSessions.length) return []; + const resolve=projectIdFor||_sidebarProjectResolver(_sidebarRowsById([_sidebarReferenceSessions, typeof _allSessions!=='undefined'?_allSessions:null])); return _sidebarReferenceSessions.filter(s=>{ if(!s) return false; // Source scope: only references in the same webui/cli bucket as this render. if(_isCliSession(s)!==!!isCli) return false; // Project scope: mirror _partitionSidebarSessionRows exactly. - if(_activeProject===NO_PROJECT_FILTER){ if(s.project_id) return false; } - else if(_activeProject){ if(s.project_id!==_activeProject) return false; } + const projectId=resolve(s); + if(_activeProject===NO_PROJECT_FILTER){ if(projectId) return false; } + else if(_activeProject){ if(projectId!==_activeProject) return false; } return true; }); } @@ -7767,19 +7890,20 @@ function renderSessionListFromCache(){ cliReferenceRaw, webuiSessionsRaw, cliSessionsRaw, + projectIdFor, }=_partitionSidebarSessionRows(allMatched, activeSidForSidebar); const referenceRaw=_sessionSourceFilter==='cli'?cliReferenceRaw:webuiReferenceRaw; const isCliView=_sessionSourceFilter==='cli'; - const sessions=_renderSidebarRowsFromRawSessions(sessionsRaw, [...referenceRaw, ..._scopedSidebarReferenceRows(isCliView)]); + const sessions=_renderSidebarRowsFromRawSessions(sessionsRaw, [...referenceRaw, ..._scopedSidebarReferenceRows(isCliView, projectIdFor)]); // Server-provided source bucket counts are authoritative for the current // payload. When present, skip the expensive cross-bucket render/count pass; // null is a deliberate "not computed" sentinel consumed only by // _sessionSourceTabCount's fallback path below. const renderedWebuiSessionCount=_serverWebuiSessionCount===null - ? _renderSidebarRowsFromRawSessions(webuiSessionsRaw, [...webuiReferenceRaw, ..._scopedSidebarReferenceRows(false)]).length + ? _renderSidebarRowsFromRawSessions(webuiSessionsRaw, [...webuiReferenceRaw, ..._scopedSidebarReferenceRows(false, projectIdFor)]).length : null; const renderedCliSessionCount=_serverCliSessionCount===null - ? _renderSidebarRowsFromRawSessions(cliSessionsRaw, [...cliReferenceRaw, ..._scopedSidebarReferenceRows(true)]).length + ? _renderSidebarRowsFromRawSessions(cliSessionsRaw, [...cliReferenceRaw, ..._scopedSidebarReferenceRows(true, projectIdFor)]).length : null; const webuiSessionTabCount=_sessionSourceTabCount('webui', renderedWebuiSessionCount, renderedCliSessionCount); const cliSessionTabCount=_sessionSourceTabCount('cli', renderedWebuiSessionCount, renderedCliSessionCount); @@ -7841,7 +7965,7 @@ function renderSessionListFromCache(){ } // Project filter bar — show when there are real projects OR there are // unassigned sessions (so the Unassigned chip has something to filter to). - const hasUnprojected=profileFiltered.some(s=>!s.project_id); + const hasUnprojected=_sidebarHasUnprojectedRows(profileFiltered, projectIdFor); if(_allProjects.length>0||hasUnprojected){ const bar=document.createElement('div'); bar.className='project-bar'; diff --git a/static/ui.js b/static/ui.js index 610ad14c..201eb7fd 100644 --- a/static/ui.js +++ b/static/ui.js @@ -7723,6 +7723,16 @@ function renderMd(raw){ // generated images) and replace them with inline or download links. // Stashed so the path/URL is never processed as markdown. const media_stash=[]; + // #7680 re-gate (9/22): two-pass scan. + // 1. `` `MEDIA:path` `` (backtick-wrapped, inline-code form) → strip + // the wrapping backticks so the bare-token pass below sees a + // plain ``MEDIA:path`` and the closing backtick is not consumed + // as part of the path. + // 2. ``MEDIA:[^\s\)\]]+`` (bare, no backtick in the exclusion + // class) so a filename that legally contains a backtick + // (``report`final.png``) is captured in full instead of being + // truncated at the first backtick. + s=s.replace(/`MEDIA:([^`\s]+)`/g,'MEDIA:$1'); s=s.replace(/MEDIA:([^\s\)\]]+)/g,(_,raw_ref)=>{ media_stash.push(raw_ref); return '\x00D'+(media_stash.length-1)+'\x00'; @@ -9170,6 +9180,12 @@ function _stripForTTS(text){ // Strip links, keep text text=text.replace(/\[([^\]]+)\]\([^)]+\)/g,'$1'); // Replace MEDIA: paths with a simple label + // #7680 re-gate (9/22): TTS does not need the backtick-aware + // terminator. Inline code was already stripped at the top of this + // function (`:9137`), so a bare ``MEDIA:[^\s]+`` is correct and + // round-trips the original filename including a backtick in the + // path — the only thing the TTS ever does with the captured + // substring is throw it away. text=text.replace(/MEDIA:[^\s]+/g,'a file'); // Strip emoji and emoticons text=text.replace(/[\u{1F600}-\u{1F64F}\u{1F300}-\u{1F5FF}\u{1F680}-\u{1F6FF}\u{1F1E0}-\u{1F1FF}\u{2600}-\u{26FF}\u{2700}-\u{27BF}\u{FE00}-\u{FE0F}\u{1F900}-\u{1F9FF}\u{1FA00}-\u{1FA6F}\u{1FA70}-\u{1FAFF}\u{200D}]/gu,''); diff --git a/tests/test_5306_subagent_sidebar_flicker.py b/tests/test_5306_subagent_sidebar_flicker.py index 61bb8dbe..ca8c2c18 100644 --- a/tests/test_5306_subagent_sidebar_flicker.py +++ b/tests/test_5306_subagent_sidebar_flicker.py @@ -93,6 +93,9 @@ eval(extractFunc('_collapseSessionLineageForSidebar')); eval(extractFunc('_attachChildSessionsToSidebarRows')); eval(extractFunc('_sessionAttentionState')); eval(extractFunc('_sidebarRowHasVisibleMessages')); +eval(extractFunc('_isDelegatedSubagentRow')); +eval(extractFunc('_sidebarProjectResolver')); +eval(extractFunc('_sidebarRowsById')); eval(extractFunc('_partitionSidebarSessionRows')); eval(extractFunc('_scopedSidebarReferenceRows')); eval(extractFunc('_renderSidebarRowsFromRawSessions')); @@ -216,9 +219,9 @@ console.log(JSON.stringify({ })); """ out = json.loads(_run_node(source)) - # Child survives the visibility/project scope into sessionsRaw (parent does not)... - assert out["sessionsRaw"] == ["subagent_child"] - # ...but is NOT rendered as a top-level orphan. + # The child inherits its parent's project, so the "no project" filter drops both... + assert out["sessionsRaw"] == [] + # ...and it is NOT rendered as a top-level orphan. assert out["topLevel"] == [] assert out["orphans"] == [] diff --git a/tests/test_async_delegation_retryable_refusal.py b/tests/test_async_delegation_retryable_refusal.py new file mode 100644 index 00000000..17be2e52 --- /dev/null +++ b/tests/test_async_delegation_retryable_refusal.py @@ -0,0 +1,159 @@ +"""A wake-up refused for a transient reason the WebUI did not cause must not +consume the durable delivery budget. + +Scenario: while the Agent runtime is being replaced, the still-running WebUI +answers every server-side wake-up with ``409 agent_runtime_stale +retryable=True``. Releasing the claim as a plain failure each time exhausts the +bounded budget within seconds and terminally drops a completion whose origin +session is alive and waiting. + +Contract under test (``api/background_process._start_async_delegation_wakeup_turn``): + +* a refusal whose payload says ``retryable: True`` (stale runtime), a paused + wake-up or a session busy with another stream releases the claim with + ``retryable=True`` so the core refunds the attempt; +* every other rejection keeps the historical bounded behaviour; +* an older core whose ``release_event_delivery`` has no ``retryable`` keyword + still gets a plain release (compatibility, never an exception). +""" +from __future__ import annotations + +import sys +import time +import types + +import pytest + +from api import background_process as bp +from api import config as cfg +from api import process_event_utils as peu + + +def _wait_until(predicate, timeout=3.0): + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + if predicate(): + return True + time.sleep(0.01) + return bool(predicate()) + + +def _reset() -> None: + cfg.PROCESS_SESSION_INDEX.clear() + cfg.PENDING_BG_TASK_COMPLETIONS.clear() + cfg.BG_TASK_COMPLETE_EVENTS_SEEN.clear() + cfg.DEFERRED_PROCESS_WAKEUPS.clear() + reset = getattr(peu, "_reset_legacy_async_delivery_dedupe_for_tests", None) + if reset is not None: + reset() + + +def _event(): + return { + "type": "async_delegation", + "delegation_id": "deleg_retryable", + "session_key": "webui-session-1", + "status": "completed", + "is_batch": True, + "results": [{"task_index": 0, "status": "completed", "summary": "verdict"}], + "dispatched_at": time.time() - 2, + "completed_at": time.time(), + } + + +class _Registry: + def __init__(self): + import queue + + self.completion_queue = queue.Queue() + + +def _install_core(monkeypatch, *, supports_retryable: bool): + calls = {"release": [], "complete": []} + mod = types.ModuleType("tools.async_delegation") + mod.claim_event_delivery = lambda evt, consumer: f"claim:{consumer}" + mod.complete_event_delivery = lambda evt, claim_id: calls["complete"].append(claim_id) + mod.get_durable_delegation = lambda delegation_id: {"delivery_state": "pending", "delivery_attempts": 1} + mod.restore_undelivered_completions = lambda queue_: 0 + if supports_retryable: + def _release(evt, claim_id, *, retryable=False): + calls["release"].append((claim_id, retryable)) + else: + def _release(evt, claim_id): # pre-refund core: no keyword at all + calls["release"].append((claim_id, None)) + mod.release_event_delivery = _release + pkg = sys.modules.get("tools") or types.ModuleType("tools") + monkeypatch.setitem(sys.modules, "tools", pkg) + monkeypatch.setitem(sys.modules, "tools.async_delegation", mod) + monkeypatch.setattr(pkg, "async_delegation", mod, raising=False) + return calls + + +def _run(monkeypatch, response, *, supports_retryable=True): + _reset() + calls = _install_core(monkeypatch, supports_retryable=supports_retryable) + from api import routes + + monkeypatch.setattr(routes, "start_session_turn", lambda *a, **k: response) + monkeypatch.setattr(bp, "ASYNC_DELIVERY_ROUTING_RETRY_SECONDS", 10.0) + registry = _Registry() + evt = _event() + claim = peu.claim_async_delegation_delivery(evt, "webui-background") + assert claim is not None + try: + bp._start_async_delegation_wakeup_turn( + "webui-session-1", "verdict", delegation_id="deleg_retryable", + evt=evt, claim=claim, process_registry=registry, + ) + assert _wait_until(lambda: len(calls["release"]) == 1) + assert calls["complete"] == [] + return calls["release"][0] + finally: + _reset() + + +@pytest.mark.parametrize("response", [ + {"_status": 409, "error": "runtime changed", "type": "agent_runtime_stale", "retryable": True}, + {"_status": 503, "error": "temporarily unavailable", "retryable": True}, + {"_status": 409, "error": "process_wakeup_paused"}, + {"_status": 409, "error": "session already has an active stream", "active_stream_id": "s1"}, +]) +def test_transient_refusal_releases_with_refund(monkeypatch, response): + _claim, retryable = _run(monkeypatch, response) + assert retryable is True + + +@pytest.mark.parametrize("response", [ + {"_status": 500, "error": "boom"}, + {"_status": 404, "error": "Session not found"}, + {"_status": 400, "error": "message is required"}, + {"_status": 409, "error": "conflict", "retryable": False}, + None, +]) +def test_hard_rejection_keeps_bounded_budget(monkeypatch, response): + _claim, retryable = _run(monkeypatch, response) + assert retryable is False + + +def test_old_core_without_retryable_keyword_still_releases(monkeypatch): + _claim, retryable = _run( + monkeypatch, + {"_status": 409, "error": "x", "type": "agent_runtime_stale", "retryable": True}, + supports_retryable=False, + ) + assert retryable is None # plain release happened, no TypeError escaped + + +def test_real_core_signature_is_detected(): + def new_core(evt, claim_id, *, retryable=False): + return None + + def kwargs_core(evt, claim_id, **kwargs): + return None + + def old_core(evt, claim_id): + return None + + assert peu._release_accepts_retryable(new_core) is True + assert peu._release_accepts_retryable(kwargs_core) is True + assert peu._release_accepts_retryable(old_core) is False diff --git a/tests/test_background_process_restart_recovery.py b/tests/test_background_process_restart_recovery.py index 0ab76e36..45ee82ff 100644 --- a/tests/test_background_process_restart_recovery.py +++ b/tests/test_background_process_restart_recovery.py @@ -1,5 +1,6 @@ """Regression coverage for notify_on_complete across a WebUI restart.""" +import logging import threading import time from types import SimpleNamespace @@ -158,4 +159,103 @@ def test_recovery_is_fail_soft_without_agent(monkeypatch): monkeypatch.setattr("builtins.__import__", fake_import) assert bp.recover_processes_for_webui() == 0 - assert bp._PROCESS_RECOVERY_DONE is False \ No newline at end of file + assert bp._PROCESS_RECOVERY_DONE is False + + +def test_recovery_skips_process_whose_session_is_gone(monkeypatch, caplog): + """A checkpointed process whose owner has been pruned (the resolver's + ``KeyError(sid)`` contract) must be silently skipped, while a neighbouring + live process in the same recovery sweep still rebinds. Regression for + issue #7753: previously every vanished owner logged a full WARNING + traceback, indistinguishable from a real fault. + """ + calls = {"registered": []} + + class FakeRegistry: + def recover_from_checkpoint(self): + return 2 # two checkpointed processes survive restart + + def list_sessions(self): + return [ + {"session_id": "proc_vanished", "detached": True}, + {"session_id": "proc_live", "detached": True}, + ] + + def get(self, process_id): + if process_id == "proc_vanished": + return _FakeProcessSession("vanished-session") + if process_id == "proc_live": + return _FakeProcessSession("live-session") + raise AssertionError(process_id) + + fake_registry = FakeRegistry() + monkeypatch.setattr(bp, "_PROCESS_CHECKPOINT_RECOVERED", False) + monkeypatch.setattr(bp, "_PROCESS_RECOVERY_DONE", False) + monkeypatch.setattr( + bp, + "register_process_session", + lambda key, sid: calls["registered"].append((key, sid)), + ) + + def get_session(sid, metadata_only=False): + # Mirrors api.models._resolve_session_once's contract: missing + # sessions raise KeyError, NOT None. + if sid == "vanished-session": + raise KeyError(sid) + return SimpleNamespace(id=sid) + + with caplog.at_level(logging.WARNING, logger="api.background_process"): + result = bp.recover_processes_for_webui(fake_registry, get_session) + + assert result == 2 + # The live process is rebound; the vanished one is silently skipped. + assert calls["registered"] == [("live-session", "live-session")] + # No WARNING emitted for the vanished-owner case. + warning_records = [ + r for r in caplog.records if r.levelno >= logging.WARNING + ] + assert warning_records == [], ( + "vanished-session KeyError must not produce a WARNING, got: " + f"{[r.getMessage() for r in warning_records]}" + ) + assert "Could not resolve recovered WebUI process" not in caplog.text + + +def test_recovery_still_warns_on_unexpected_resolve_failure(monkeypatch, caplog): + """Non-KeyError failures from the resolver (e.g. ``OSError`` on an + unreadable sidecar) must keep entering the existing WARNING path. This + pins that the fix for #7753 is narrow and does not over-broaden the + silent-skip contract. + """ + class FakeRegistry: + def recover_from_checkpoint(self): + return 1 + + def list_sessions(self): + return [{"session_id": "proc_unexpected", "detached": True}] + + def get(self, process_id): + assert process_id == "proc_unexpected" + return _FakeProcessSession("webui-session") + + fake_registry = FakeRegistry() + monkeypatch.setattr(bp, "_PROCESS_CHECKPOINT_RECOVERED", False) + monkeypatch.setattr(bp, "_PROCESS_RECOVERY_DONE", False) + + def get_session(sid, metadata_only=False): + raise OSError("sidecar unreadable") + + with caplog.at_level(logging.WARNING, logger="api.background_process"): + result = bp.recover_processes_for_webui(fake_registry, get_session) + + assert result == 1 + warning_records = [ + r for r in caplog.records if r.levelno >= logging.WARNING + ] + assert any( + "Could not resolve recovered WebUI process 'proc_unexpected'" in r.getMessage() + for r in warning_records + ), ( + "OSError from the resolver must still emit a WARNING, got: " + f"{[r.getMessage() for r in warning_records]}" + ) \ No newline at end of file diff --git a/tests/test_byok_model_dropdown.py b/tests/test_byok_model_dropdown.py index 6a8329aa..976dfdd6 100644 --- a/tests/test_byok_model_dropdown.py +++ b/tests/test_byok_model_dropdown.py @@ -291,6 +291,13 @@ class TestLiveModelsCustomProviderFallback: "model": "gpt-5.5", "models": {"gpt-5.5-mini": {}}, "base_url": "https://right.codes/codex/v1", + # dict-shaped ``models`` is per-model metadata (Agent setup + # flow) and only narrows the live catalog when discovery is + # explicitly off — the new #7165 contract. This test still + # wants provider-scoping (no sibling leak), so pin the + # catalog via ``discover_models: false`` instead of relying + # on the removed dict-as-allowlist behavior. + "discover_models": False, }, { "name": "infini-ai", @@ -308,7 +315,7 @@ class TestLiveModelsCustomProviderFallback: resp = self._call_live_models(monkeypatch, cfg, "custom:rightcode-codex") assert resp["provider"] == "custom:rightcode-codex" - assert [m["id"] for m in resp["models"]] == ["gpt-5.5", "gpt-5.5-mini"] + assert [m["id"] for m in resp["models"]] == ["gpt-5.5-mini"] def test_bare_custom_fallback_ignores_named_custom_provider_models(self, monkeypatch): """Bare custom only represents unnamed custom entries, not named siblings.""" diff --git a/tests/test_cron_model_override.py b/tests/test_cron_model_override.py index bf3b4b89..6c9501c8 100644 --- a/tests/test_cron_model_override.py +++ b/tests/test_cron_model_override.py @@ -8,6 +8,8 @@ import sys import types from pathlib import Path +import pytest + REPO = Path(__file__).resolve().parents[1] PANELS_JS = (REPO / "static" / "panels.js").read_text(encoding="utf-8") @@ -170,3 +172,89 @@ def test_cron_model_picker_marks_loaded_only_after_successful_population(): catch_block = body.split("} catch (e)", 1)[1] assert "sel.dataset.loaded = '1'" in try_block assert "sel.dataset.loaded = '1'" not in catch_block + + +def test_cron_update_preserves_provider_only_pin_when_model_cleared(): + """Clearing the picker to "Default" must not wipe a provider-only pin. + + #4030 intentionally made "clearing the picker = no overrides", which is + correct for model+provider pins but silently destroyed provider-only + pins (jobs that pin a provider with no model, e.g. self-hosted LLM + endpoints). The combined picker cannot represent that state, so + saveCronForm must preserve prev.provider when the job carried no model + AND the user never changed the picker; a deliberate return to Default + must clear it. + """ + save_body = _function_body("saveCronForm") + helper_body = _function_body("_cronProviderForClear") + + # The clear branch delegates to the helper and records picker changes. + assert "_cronProviderForClear(_cronPreFormDetail, _cronModelPickerTouched)" in save_body + assert "preserveProvider" not in save_body + + # The helper distinguishes an explicit clear (pickerTouched) from an + # untouched provider-only pin. + assert "if (pickerTouched) return null" in helper_body + assert "prev.model == null" in helper_body + + # The form entry points reset the touched flag. + for fn in ("openCronCreate", "openCronEdit", "duplicateCurrentCron"): + assert "_cronModelPickerTouched = false" in _function_body(fn) + # The picker records deliberate changes. + pop_body = _function_body("_populateCronFormModelSelect") + assert "_cronModelPickerTouched = true" in pop_body + + +def test_cron_provider_for_clear_behavior(): + """Exercise _cronProviderForClear through node for both pin shapes.""" + import shutil + import subprocess + + node = shutil.which("node") + if node is None: + pytest.skip("node not on PATH") + + # Locate the full declaration (signature + body) of the helper. + src = PANELS_JS + start = src.find("function _cronProviderForClear(") + assert start != -1, "helper declaration not found" + brace = src.find("{", start) + depth = 0 + end = None + for i in range(brace, len(src)): + if src[i] == "{": + depth += 1 + elif src[i] == "}": + depth -= 1 + if depth == 0: + end = i + 1 + break + decl = src[start:end] + + driver = f""" +{decl} +const cases = [ + ['provider_only_untouched', {{model: null, provider: 'custom:llama-swap'}}, false], + ['provider_only_touched', {{model: null, provider: 'custom:llama-swap'}}, true], + ['model_pin_touched', {{model: 'gpt-5', provider: 'openai'}}, true], + ['no_override_untouched', {{model: null, provider: null}}, false], + ['no_prev', null, false], +]; +const out = {{}}; +for (const [name, detail, touched] of cases) out[name] = _cronProviderForClear(detail, touched) ?? null; +process.stdout.write(JSON.stringify(out)); +""" + proc = subprocess.run([node, "-e", driver], capture_output=True, text=True, timeout=30) + assert proc.returncode == 0, f"node driver failed: {proc.stderr}" + r = json.loads(proc.stdout) + + # Untouched picker on a provider-only job: the pin survives. + assert r["provider_only_untouched"] == "custom:llama-swap" + # Deliberate return to Default: "Default = no overrides" wins. + assert r["provider_only_touched"] is None + # Model-pinned job, explicit clear: provider removed too. + assert r["model_pin_touched"] is None + # No overrides stored, untouched picker: nothing to preserve. + assert r["no_override_untouched"] is None + # No pre-form snapshot (create form): no provider. + assert r["no_prev"] is None diff --git a/tests/test_gateway_watcher_resubscribe_fresh.py b/tests/test_gateway_watcher_resubscribe_fresh.py new file mode 100644 index 00000000..01e9baa8 --- /dev/null +++ b/tests/test_gateway_watcher_resubscribe_fresh.py @@ -0,0 +1,222 @@ +"""Regression: a re-subscribing SSE client gets a fresh watcher projection. + +``_poll_once`` resets the parity timestamp when it runs with no subscribers +("force fresh projection on the next subscription"), but since the idle-park +change (#7694) the poll loop parks *before* calling ``_poll_once`` once the +last subscriber leaves, so that reset is not reached on a normal disconnect. +A tab that reconnects inside the parity interval then resumes with the stale +cheap fingerprint and skips the projection. Final removal invalidates the +cache, and an epoch fences projections already in flight. +""" +from __future__ import annotations + +import importlib +import sqlite3 +import threading +import time +from pathlib import Path + +import pytest + + +def _make_db(tmp_path: Path) -> Path: + db = tmp_path / "state.db" + conn = sqlite3.connect(str(db)) + conn.executescript( + """ + CREATE TABLE sessions ( + id TEXT PRIMARY KEY, + source TEXT NOT NULL, + session_source TEXT, + model TEXT, + started_at REAL NOT NULL, + ended_at REAL, + end_reason TEXT, + parent_session_id TEXT, + message_count INTEGER DEFAULT 0, + title TEXT, + archived INTEGER DEFAULT 0 + ); + CREATE TABLE messages ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + session_id TEXT NOT NULL, + role TEXT NOT NULL, + content TEXT, + timestamp REAL NOT NULL + ); + """ + ) + conn.execute( + "INSERT INTO sessions (id, source, model, started_at, message_count, title) " + "VALUES ('tg1', 'telegram', 'm', ?, 1, 'Chat')", + (time.time(),), + ) + conn.execute( + "INSERT INTO messages (session_id, role, content, timestamp) VALUES ('tg1', 'user', 'x', ?)", + (time.time(),), + ) + conn.commit() + conn.close() + return db + + +def test_resubscribe_after_last_unsubscribe_forces_fresh_projection(tmp_path): + gw = importlib.import_module("api.gateway_watcher") + watcher = gw.GatewayWatcher(state_db_path=_make_db(tmp_path)) + q = watcher.subscribe() + assert watcher._poll_once(now=1.0) is True + + watcher.unsubscribe(q) + assert watcher._last_cheap_fp == "" + assert watcher._last_full_projection_at is None + + watcher.subscribe() + # Well inside PROJECTION_PARITY_INTERVAL and with an unchanged state.db. + assert watcher._poll_once(now=2.0) is True + + +def test_unsubscribe_keeps_fingerprint_while_other_subscribers_remain(tmp_path): + gw = importlib.import_module("api.gateway_watcher") + watcher = gw.GatewayWatcher(state_db_path=_make_db(tmp_path)) + first = watcher.subscribe() + watcher.subscribe() + assert watcher._poll_once(now=1.0) is True + fingerprint = watcher._last_cheap_fp + assert fingerprint + + watcher.unsubscribe(first) + assert watcher._last_cheap_fp == fingerprint + assert watcher._last_full_projection_at == 1.0 + assert watcher._poll_once(now=2.0) is False + + +def test_unsubscribe_of_unknown_queue_does_not_reset(tmp_path): + import queue as _queue + + gw = importlib.import_module("api.gateway_watcher") + watcher = gw.GatewayWatcher(state_db_path=_make_db(tmp_path)) + watcher.subscribe() + assert watcher._poll_once(now=1.0) is True + fingerprint = watcher._last_cheap_fp + + watcher.unsubscribe(_queue.Queue()) + assert watcher._last_cheap_fp == fingerprint + assert watcher._last_full_projection_at == 1.0 + + +def test_slow_consumer_eviction_invalidates_populated_cache(tmp_path): + gw = importlib.import_module("api.gateway_watcher") + watcher = gw.GatewayWatcher(state_db_path=_make_db(tmp_path)) + q = watcher.subscribe() + assert watcher._poll_once(now=1.0) is True + assert watcher._last_cheap_fp + assert watcher._last_full_projection_at == 1.0 + + for _ in range(q.maxsize - q.qsize()): + q.put_nowait({"type": "existing"}) + watcher._notify_subscribers([]) + assert not watcher._has_subscribers() + assert watcher._last_cheap_fp == "" + assert watcher._last_full_projection_at is None + watcher.unsubscribe(q) # The late SSE cleanup cannot undo the reset. + watcher.subscribe() + assert watcher._poll_once(now=2.0) is True + + +@pytest.mark.parametrize("removal", ["unsubscribe", "slow_consumer"]) +@pytest.mark.parametrize("reconnect_while_blocked", [False, True]) +def test_inflight_projection_cannot_restore_cache_after_last_removal( + tmp_path, monkeypatch, removal, reconnect_while_blocked, +): + """The DB read must not serialize final removal or commit across its epoch.""" + gw = importlib.import_module("api.gateway_watcher") + watcher = gw.GatewayWatcher(state_db_path=_make_db(tmp_path)) + q = watcher.subscribe() + entered = threading.Event() + release = threading.Event() + projections = [] + errors = [] + real_projection = gw._get_agent_sessions_from_db + + def blocked_projection(path): + projections.append(path) + if len(projections) == 1: + entered.set() + assert release.wait(5), "test never released the DB read" + return real_projection(path) + + def poll(): + try: + watcher._poll_once(now=1.0) + except BaseException as exc: + errors.append(exc) + + monkeypatch.setattr(gw, "_get_agent_sessions_from_db", blocked_projection) + worker = threading.Thread(target=poll, daemon=True) + worker.start() + new_q = None + try: + assert entered.wait(5), "poll never reached the DB read" + if removal == "unsubscribe": + watcher.unsubscribe(q) + else: + for _ in range(q.maxsize): + q.put_nowait({"type": "existing"}) + watcher._notify_subscribers([]) + # Handler teardown of an evicted queue must be harmless. + watcher.unsubscribe(q) + assert not watcher._has_subscribers() + assert watcher._last_cheap_fp == "" + assert watcher._last_full_projection_at is None + if reconnect_while_blocked: + new_q = watcher.subscribe() + finally: + release.set() + worker.join(timeout=5) + assert not worker.is_alive(), "poll did not finish" + assert not errors, errors + assert watcher._last_cheap_fp == "" + assert watcher._last_full_projection_at is None + if new_q is not None: + assert new_q.empty(), "old projection must not notify a new subscriber" + else: + watcher.subscribe() + assert watcher._poll_once(now=2.0) is True + assert len(projections) == 2 + assert watcher._last_cheap_fp + assert watcher._last_full_projection_at == 2.0 + + +def test_poll_loop_reprojects_for_reconnecting_subscriber(tmp_path, monkeypatch): + """Through the real loop: disconnect, park, reconnect -> projection runs again.""" + gw = importlib.import_module("api.gateway_watcher") + db = _make_db(tmp_path) + projections: list[float] = [] + real_projection = gw._get_agent_sessions_from_db + + def tracing_projection(path): + projections.append(time.monotonic()) + return real_projection(path) + + monkeypatch.setattr(gw, "_get_agent_sessions_from_db", tracing_projection) + watcher = gw.GatewayWatcher(state_db_path=db) + watcher.POLL_INTERVAL = 0.05 + watcher.start() + try: + q = watcher.subscribe() + deadline = time.monotonic() + 2.0 + while not projections and time.monotonic() < deadline: + time.sleep(0.01) + assert len(projections) == 1 + + watcher.unsubscribe(q) + time.sleep(watcher.POLL_INTERVAL * 4) # loop parks; no further projection + assert len(projections) == 1 + + watcher.subscribe() + deadline = time.monotonic() + 2.0 + while len(projections) < 2 and time.monotonic() < deadline: + time.sleep(0.01) + assert len(projections) == 2, "reconnecting subscriber must get a fresh projection" + finally: + watcher.stop() diff --git a/tests/test_inflight_stream_reuse.py b/tests/test_inflight_stream_reuse.py index 36b215f2..7f21b434 100644 --- a/tests/test_inflight_stream_reuse.py +++ b/tests/test_inflight_stream_reuse.py @@ -1060,6 +1060,305 @@ def test_load_session_discards_cursor_only_inflight_before_reattach(): assert 0 <= guard_pos < inflight_branch_pos +def test_user_only_inflight_must_not_seed_nonzero_replay_floor(): + """#7640: a plain user row must not authorize a nonzero replay floor. + + After a mid-stream SSE drop in transparent mode the client re-attaches with + `replay=1&after_seq=`. When the cached INFLIGHT entry kept the + optimistic user row plus `lastRunJournalSeq` but lost the live assistant + projection, that cursor told the server to skip every journal event that + rebuilds the message body: only the tail (often just `stream_end`) came back + and the settled footer painted over a blank message until a manual reload. + """ + assert NODE, "node not on PATH" + script = "\n".join( + [ + "const assert=require('assert');", + _function_decl(SESSIONS_JS, "_inflightCanSeedJournalReplay"), + _function_decl(SESSIONS_JS, "_runJournalReplayFloorForInflight"), + _function_decl(SESSIONS_JS, "_runJournalReplayEventIdForInflight"), + """ +const userOnly={ + messages:[{role:'user', content:'a very long ttft prompt'}], + lastRunJournalSeq:206, + lastRunJournalEventId:'stream-e75e:206', +}; +assert.strictEqual(_inflightCanSeedJournalReplay(userOnly), false); +assert.strictEqual(_runJournalReplayFloorForInflight(userOnly), 0); +assert.strictEqual(_runJournalReplayEventIdForInflight(userOnly), ''); +// The floor helper reports, it does not mutate the recovery object. +assert.strictEqual(userOnly.lastRunJournalSeq, 206); + +// A user row with a populated but assistant-less transcript is still user-only. +assert.strictEqual( + _runJournalReplayFloorForInflight({ + messages:[{role:'user', content:'hi'}, {role:'assistant', content:''}], + lastRunJournalSeq:206, + }), + 0 +); +assert.strictEqual( + _runJournalReplayFloorForInflight({ + messages:[{role:'user', content:'hi'}, {role:'assistant', content:' '}], + lastRunJournalSeq:206, + }), + 0 +); + +// Positive inverse: any recoverable live output keeps the cursor. +const recovering=[ + {lastAssistantText:'partial answer'}, + {lastReasoningText:'thinking...'}, + {liveTurnHtml:'
projected turn
'}, + {toolCalls:[{name:'read_file'}]}, + {activityBurstAnchors:[{seq:12}]}, + {anchorActivityScene:{activity_rows:[{kind:'tool'}]}}, + {messages:[{role:'user', content:'p'}, {role:'assistant', content:'answer', _live:true}]}, +]; +for (const extra of recovering) { + const state={messages:[{role:'user', content:'p'}], ...extra}; + state.lastRunJournalSeq=206; + state.lastRunJournalEventId='stream-e75e:206'; + assert.strictEqual( + _runJournalReplayFloorForInflight(state), + 206, + 'recoverable live output must keep the cursor: '+JSON.stringify(extra) + ); + assert.strictEqual(_runJournalReplayEventIdForInflight(state), 'stream-e75e:206'); +} + +// #7651: an ESTABLISHED conversation copies the whole transcript, so the last +// assistant row is the previous turn's reply. Historical assistant content must +// never authorize the stale cursor that belongs to the still-running turn. +const established={ + messages:[ + {role:'user', content:'first question'}, + {role:'assistant', content:'the previous turn answer'}, + {role:'user', content:'current question'}, + ], + lastRunJournalSeq:206, + lastRunJournalEventId:'stream-e75e:206', +}; +assert.strictEqual( + _inflightCanSeedJournalReplay(established), + false, + 'a historical assistant row must not seed the replay floor' +); +assert.strictEqual(_runJournalReplayFloorForInflight(established), 0); +assert.strictEqual(_runJournalReplayEventIdForInflight(established), ''); + +// Even after the latest user boundary, evidence must be LIVE output: a +// settled/non-live assistant row at the current position still yields zero. +assert.strictEqual( + _runJournalReplayFloorForInflight({ + messages:[ + {role:'user', content:'current question'}, + {role:'assistant', content:'settled reply without _live'}, + ], + lastRunJournalSeq:206, + }), + 0 +); + +// Degenerate inputs must stay on the zero floor. +assert.strictEqual(_runJournalReplayFloorForInflight(null), 0); +assert.strictEqual(_runJournalReplayFloorForInflight({}), 0); +assert.strictEqual(_runJournalReplayFloorForInflight({lastRunJournalSeq:206, messages:[]}), 0); +assert.strictEqual( + _runJournalReplayFloorForInflight({lastAssistantText:'partial', lastRunJournalSeq:-4}), + 0 +); +assert.strictEqual(_runJournalReplayEventIdForInflight({}), ''); +""", + ] + ) + result = subprocess.run([NODE, "-e", script], capture_output=True, text=True, check=False) + assert result.returncode == 0, result.stderr + + +def test_reattach_discards_replay_cursor_without_live_assistant_state(): + """#7640: validate the recovery object immediately before reattach. + + Trusting the object's existence (rather than its contents) is what let a + cursor-only cache reach the wire as a replay floor. Reattach now drops the + cursor so the journal replays from zero, which is the path a manual reload + already proves out, while a cache that still holds live assistant output + keeps its cursor and avoids re-rendering the whole journal. + """ + assert NODE, "node not on PATH" + script = "\n".join( + [ + "const assert=require('assert');", + _function_decl(SESSIONS_JS, "_inflightCanSeedJournalReplay"), + _function_decl(SESSIONS_JS, "_runJournalReplayFloorForInflight"), + _function_decl(SESSIONS_JS, "_normalizeInflightReplayCursorForReattach"), + """ +const cursorOnly={ + streamId:'stream-e75e', + reattach:true, + messages:[{role:'user', content:'a very long ttft prompt'}], + lastRunJournalSeq:206, + lastRunJournalEventId:'stream-e75e:206', +}; +assert.strictEqual( + _normalizeInflightReplayCursorForReattach(cursorOnly), + cursorOnly, + 'must return the same object so the caller keeps mutating the live cache' +); +assert.strictEqual(cursorOnly.lastRunJournalSeq, 0); +assert.strictEqual(cursorOnly.lastRunJournalEventId, ''); +assert.strictEqual(_runJournalReplayFloorForInflight(cursorOnly), 0); +assert.strictEqual( + cursorOnly.messages.length, + 1, + 'the optimistic user row must survive so pending dedupe still works' +); + +const recovering={ + streamId:'stream-e75e', + reattach:true, + messages:[{role:'user', content:'p'}], + lastAssistantText:'partial answer', + lastRunJournalSeq:206, + lastRunJournalEventId:'stream-e75e:206', +}; +_normalizeInflightReplayCursorForReattach(recovering); +assert.strictEqual(recovering.lastRunJournalSeq, 206); +assert.strictEqual(recovering.lastRunJournalEventId, 'stream-e75e:206'); + +// #7651: a historical assistant row must not protect the cursor either, or an +// established conversation settles with a blank body over a painted footer. +const established={ + streamId:'stream-e75e', + reattach:true, + messages:[ + {role:'user', content:'first question'}, + {role:'assistant', content:'the previous turn answer'}, + {role:'user', content:'current question'}, + ], + lastRunJournalSeq:206, + lastRunJournalEventId:'stream-e75e:206', +}; +_normalizeInflightReplayCursorForReattach(established); +assert.strictEqual(established.lastRunJournalSeq, 0); +assert.strictEqual(established.lastRunJournalEventId, ''); +assert.strictEqual(_runJournalReplayFloorForInflight(established), 0); +assert.strictEqual( + established.messages.length, + 3, + 'the transcript must survive; only the cursor halves are dropped' +); + +const noCursor={streamId:'stream-e75e', messages:[{role:'user', content:'p'}]}; +_normalizeInflightReplayCursorForReattach(noCursor); +assert.strictEqual(noCursor.lastRunJournalSeq, undefined); +assert.strictEqual(_runJournalReplayFloorForInflight(noCursor), 0); + +assert.strictEqual(_normalizeInflightReplayCursorForReattach(null), null); +assert.strictEqual(_normalizeInflightReplayCursorForReattach(undefined), undefined); +""", + ] + ) + result = subprocess.run([NODE, "-e", script], capture_output=True, text=True, check=False) + assert result.returncode == 0, result.stderr + + +def test_reattach_replay_params_from_a_cursor_only_cache_replay_from_zero(): + """#7640: end-to-end wire contract for a cursor-only reattach. + + Derives the replay params from the real floor helpers (node) and feeds them + to the real server cursor parser. Tying both halves together matters because + `api.routes._parse_run_journal_after_seq()` reads `after_event_id` BEFORE + `after_seq`: a fix that zeroed only the seq would still resume at the stale + event id and skip every journal event that rebuilds the message body. + """ + assert NODE, "node not on PATH" + script = "\n".join( + [ + "const assert=require('assert');", + _function_decl(SESSIONS_JS, "_inflightCanSeedJournalReplay"), + _function_decl(SESSIONS_JS, "_runJournalReplayFloorForInflight"), + _function_decl(SESSIONS_JS, "_runJournalReplayEventIdForInflight"), + """ +const runId='stream-7640'; +const userOnly={ + messages:[{role:'user', content:'a very long ttft prompt'}], + lastRunJournalSeq:206, + lastRunJournalEventId:runId+':205', +}; +console.log(JSON.stringify({ + after_seq:String(_runJournalReplayFloorForInflight(userOnly)), + after_event_id:String(_runJournalReplayEventIdForInflight(userOnly)||''), +})); +""", + ] + ) + result = subprocess.run([NODE, "-e", script], capture_output=True, text=True, check=False) + assert result.returncode == 0, result.stderr + params = json.loads(result.stdout.strip().splitlines()[-1]) + + # What the fixed client puts on the wire for a cache with no live projection. + assert params == {"after_seq": "0", "after_event_id": ""} + + from api.routes import _parse_run_journal_after_seq + + run_id = "stream-7640" + assert ( + _parse_run_journal_after_seq( + {"after_seq": [params["after_seq"]], "after_event_id": [params["after_event_id"]]}, + run_id, + ) + == 0 + ) + + # Precedence proof: dropping the seq alone would still resume at the stale id. + assert ( + _parse_run_journal_after_seq( + {"after_seq": ["0"], "after_event_id": [f"{run_id}:205"]}, run_id + ) + == 205 + ) + # ...and the pre-fix pair, for the record. + assert ( + _parse_run_journal_after_seq( + {"after_seq": ["206"], "after_event_id": [f"{run_id}:205"]}, run_id + ) + == 205 + ) + + +def test_attach_and_reattach_use_the_validated_replay_floor(): + """#7640 wiring guard: the validated floor is what reaches the wire. + + The behavioral coverage for the floor helpers lives in + `test_user_only_inflight_must_not_seed_nonzero_replay_floor` and + `test_reattach_discards_replay_cursor_without_live_assistant_state`; this + test pins that `attachLiveStream()` and both `loadSession()` reattach paths + actually go through them instead of reading the raw cursor field. + """ + body = _function_body(MESSAGES_JS, "attachLiveStream") + init_pos = body.index("const _replayCursorInflight=") + init_block = body[init_pos : body.index(";", body.index("let _lastRunJournalEventId=", init_pos))] + assert "INFLIGHT[activeSid]" in init_block, "must still read the live cache" + assert "_runJournalReplayFloorForInflight(_replayCursorInflight)" in init_block + assert "_runJournalReplayEventIdForInflight(_replayCursorInflight)" in init_block + assert "INFLIGHT[activeSid].lastRunJournalSeq" not in init_block, ( + "attachLiveStream must not read the raw cursor field anymore" + ) + assert "INFLIGHT[activeSid].lastRunJournalEventId" not in init_block + + load_body = re.sub(r"\s+", "", _function_body(SESSIONS_JS, "loadSession")) + assert load_body.count("_normalizeInflightReplayCursorForReattach(INFLIGHT[sid])") == 2, ( + "both reattach paths must re-validate the recovery object" + ) + first = load_body.index("_normalizeInflightReplayCursorForReattach(INFLIGHT[sid])") + first_attach = load_body.index("attachLiveStream(sid,activeStreamId", first) + assert first < first_attach, "validation must run before the stream is attached" + last = load_body.rindex("_normalizeInflightReplayCursorForReattach(INFLIGHT[sid])") + last_attach = load_body.rindex("attachLiveStream(sid,activeStreamId", 0, last) + assert last_attach < last + + def test_live_recovery_prefers_newer_durable_run_journal_snapshot(): """Recovery chooses by stream identity and durable journal progress.""" assert NODE, "node not on PATH" @@ -1174,7 +1473,10 @@ assert.strictEqual(inflight.lastRunJournalEventId, 'run-a:7'); assert "lastRunJournalEventId:String(stored.lastRunJournalEventId||'')" in load_body assert "lastRunJournalEventId:state.lastRunJournalEventId||''" in compact_body assert "inflight.lastRunJournalEventId||''" in attach_body - assert "INFLIGHT[activeSid]&&INFLIGHT[activeSid].lastRunJournalEventId" in attach_body + # #7640: the cursor's event id is gated by the same validation as the seq — + # reading the raw field here let a cursor outlive its assistant projection. + assert "_runJournalReplayEventIdForInflight(_replayCursorInflight)" in attach_body + assert "INFLIGHT[activeSid]&&INFLIGHT[activeSid].lastRunJournalEventId" not in attach_body assert "inflight.lastRunJournalEventId=raw" in attach_body assert "INFLIGHT[activeSid].streamId=streamId" in attach_body assert "INFLIGHT[activeSid].lastRunJournalEventId=''" in attach_body diff --git a/tests/test_issue3256_context_length_default_only_guard.py b/tests/test_issue3256_context_length_default_only_guard.py index ea9ac9da..b7bb3288 100644 --- a/tests/test_issue3256_context_length_default_only_guard.py +++ b/tests/test_issue3256_context_length_default_only_guard.py @@ -631,3 +631,165 @@ def test_sse_done_path_broadened_to_any_model_window_mismatch(): assert "_model_matches_configured_default as _mmcd_cc" not in _STREAMING_SRC, ( "the old default-only save matcher (_mmcd_cc) must be removed (#4618)" ) + + +# --- #7535: provider-ownership guard on the global model.base_url backfill --- +# Same family as #3256: a global value scoped to its owner must not leak onto +# sessions owned by a different provider. + + +def test_builtin_provider_lookup_does_not_inherit_other_providers_base_url(): + """#7535: a session on a built-in registry provider (whose base_url is empty + by design because the endpoint lives in PROVIDER_REGISTRY, not config.yaml) + must NOT inherit the global ``model.base_url`` owned by a different (custom) + provider. The unguarded backfill handed the custom endpoint to the + context-length resolver, which probed the wrong ``/models`` catalog and fell + back to the generic 128K catalog entry for a 1M-context model, while the + on-disk session and SSE stream both carried 1,000,000.""" + from api.routes import _context_length_lookup_inputs_for_model + + ark_base = "https://ark.cn-beijing.volces.com/api/coding/v3" + cfg = { + "model": { + "provider": "custom:ark", + "base_url": ark_base, + "default": "ark-code-latest", + "context_length": 1000000, + }, + "custom_providers": [ + { + "name": "ark", + "base_url": ark_base, + "api_key": "ark-key", + "model": "ark-code-latest", + } + ], + } + + lookup = _context_length_lookup_inputs_for_model( + "@opencode-go:deepseek-v4.1-flash", + "opencode-go", + cfg=cfg, + ) + + assert lookup.provider == "opencode-go" + assert lookup.base_url == "", ( + "the global model.base_url belongs to custom:ark; the opencode-go " + "registry slot must stay empty so PROVIDER_REGISTRY's endpoint " + "resolves the context window (#7535)" + ) + + +def test_same_provider_session_keeps_global_base_url_backfill(): + """#7535 control (compat): when the session provider IS the configured + owner of ``model.base_url``, the backfill behavior is preserved.""" + from api.routes import _context_length_lookup_inputs_for_model + + ark_base = "https://ark.cn-beijing.volces.com/api/coding/v3" + cfg = { + "model": { + "provider": "custom:ark", + "base_url": ark_base, + "default": "ark-code-latest", + }, + } + + lookup = _context_length_lookup_inputs_for_model( + "ark-code-latest", + "custom:ark", + cfg=cfg, + ) + + assert lookup.provider == "custom:ark" + assert lookup.base_url == ark_base, ( + "same-owner sessions must keep the model.base_url backfill (#7535 compat)" + ) + + +def test_base_url_only_config_without_provider_keeps_endpoint(): + """Regression (PR #7743 gate): a ``model`` section that declares + ``base_url`` but NO ``provider`` has no owner that could conflict, so any + session must still receive the URL — exactly as master did. The profile + setup path writes this shape (api/profiles.py), and dropping the URL made + the lookup skip the endpoint and silently fall back to 256,000.""" + from api.routes import _context_length_lookup_inputs_for_model + + only_base_url = "http://127.0.0.1:9/v1" + cfg = { + "model": { + "base_url": only_base_url, + "default": "some-model", + }, + } + + for session_provider in ("custom", "openrouter"): + lookup = _context_length_lookup_inputs_for_model( + "some-model", + session_provider, + cfg=cfg, + ) + + assert lookup.provider == session_provider + assert lookup.base_url == only_base_url, ( + f"a config with no declared provider owner cannot conflict with the " + f"{session_provider!r} session, so the base_url must survive the " + f"ownership guard" + ) + + +def test_underscore_provider_owner_matches_hyphen_session(): + """Regression (PR #7743 gate): ``opencode_go`` and ``opencode-go`` are two + spellings of the SAME built-in provider, so the owner session must still + receive ``model.base_url``. Comparing the ids through the repo's + canonicaliser folds the underscore without collapsing distinct + ``custom:*`` slugs.""" + from api.routes import _context_length_lookup_inputs_for_model + + owner_base = "http://127.0.0.1:9/v1" + cfg = { + "model": { + "provider": "opencode_go", + "base_url": owner_base, + "default": "some-model", + }, + } + + lookup = _context_length_lookup_inputs_for_model( + "some-model", + "opencode-go", + cfg=cfg, + ) + + assert lookup.provider == "opencode-go" + assert lookup.base_url == owner_base, ( + "opencode_go and opencode-go are the same built-in provider, so the " + "owner session must keep the base_url backfill" + ) + + +def test_underscore_provider_still_blocks_distinct_custom_owner(): + """Counterpart to test_underscore_provider_owner_matches_hyphen_session: + folding underscores must not open the #7535 guard. ``custom:my_box`` and a + session on the unrelated built-in ``opencode-go`` are still different + providers, so the custom owner's URL stays out of the registry slot.""" + from api.routes import _context_length_lookup_inputs_for_model + + cfg = { + "model": { + "provider": "custom:my_box", + "base_url": "http://127.0.0.1:9/v1", + "default": "some-model", + }, + } + + lookup = _context_length_lookup_inputs_for_model( + "some-model", + "opencode-go", + cfg=cfg, + ) + + assert lookup.provider == "opencode-go" + assert lookup.base_url == "", ( + "underscore folding must not let a distinct custom provider's " + "base_url leak onto another provider's session (#7535 guard)" + ) diff --git a/tests/test_issue3718_live_models_custom_probe.py b/tests/test_issue3718_live_models_custom_probe.py index 8939523b..ca6b31c1 100644 --- a/tests/test_issue3718_live_models_custom_probe.py +++ b/tests/test_issue3718_live_models_custom_probe.py @@ -1,193 +1,398 @@ -"""Regression test for #3718 -- /api/models/live skips probe for custom providers. +"""Handler-level regression tests for /api/models/live custom-provider probing. -When a custom provider (``custom_providers`` entry in config.yaml) has a -``model:`` field but no ``models:`` allowlist, the live endpoint previously -populated ``ids`` from the config entry and then skipped the live ``/v1/models`` -probe (guarded by ``if not ids``). The fix collects config-specified model IDs -in a separate ``_config_ids`` list so the live fetch always runs, and merges -config entries as a fallback after the fetch. +These tests invoke the real ``routes._handle_live_models`` and inspect the +emitted ``models`` payload. Earlier revisions of this file re-implemented the +filtering loop locally, which meant they passed unchanged even with the +production filter deleted (a vacuous oracle). Every case here must fail if the +allowlist gating in ``api/routes.py`` regresses. + +Covered shapes: + * native ``models`` list → live catalog filtered + * JSON-array-string ``models`` → live catalog filtered (#7120 class) + * Python-literal-string ``models`` → live catalog filtered + * singular ``model`` only, no ``models`` → live catalog NOT gated + * malformed serialized ``models`` → safe, treated as scalar name + * allowlisted model absent from live catalog → appended + * live probe failure → falls back to config ids + * dict-shaped ``models`` (Agent setup) → live catalog NOT gated (per-model + metadata, not an allowlist) + * dict-shaped ``models`` + ``discover_models: + false`` → pinned to dict keys (explicit opt-out) + * ``models_discovered: true`` catalog → live catalog NOT gated (discovery + metadata is not an allowlist) + * discovered + ``discover_models: false`` → still pinned (explicit opt-out) """ + +import io import json -import pathlib -import unittest -from unittest import mock +import sys +import types +from urllib.parse import urlparse -REPO = pathlib.Path(__file__).parent.parent -ROUTES_PY = REPO / "api" / "routes.py" +import pytest -class TestLiveModelsCustomProviderProbe(unittest.TestCase): - """Live endpoint must probe the upstream /v1/models even when config has entries.""" - - def test_custom_provider_with_model_field_probes_upstream(self): - """When a custom provider has model: in config, live fetch must still run. - - Before the fix, _custom_provider_model_ids() populated ids=["assistant"], - and the `if not ids` guard prevented the upstream /v1/models probe from - running. The endpoint returned only ["assistant"] instead of the full - upstream catalog. - - We test the fix by simulating the logic directly: config provides - model IDs, the upstream probe returns a different set, and the result - must contain BOTH the live models and the config fallback. - """ - # Simulate config-specified model IDs - _config_ids = ["assistant"] - - # Simulate upstream /v1/models response - live_models = [ - "assistant", - "assistant-pro", - "assistant-zdr", - "gpt-5.5:pt", - "claude-sonnet-4.6:pt", - ] - - # Simulate the merge logic from the fix: - # if ids: merge config entries not in live set - # else: fall back to config-only list - ids = list(live_models) # live fetch succeeded - _live_set = set(ids) - for _cid in _config_ids: - if _cid not in _live_set: - ids.append(_cid) - - # "assistant" from config should not duplicate the live result - self.assertEqual(ids, live_models) - # All live models must be present - for m in live_models: - self.assertIn(m, ids) - - def test_custom_provider_falls_back_to_config_when_probe_fails(self): - """When the upstream probe fails, config entries must be used as fallback.""" - _config_ids = ["assistant", "custom-only-model"] - - # Simulate failed live fetch: ids stays empty - ids = [] - - # Fallback logic from the fix - if not ids: - ids = list(_config_ids) - - self.assertEqual(ids, ["assistant", "custom-only-model"]) - - def test_config_only_model_appended_when_not_in_live_results(self): - """Config-specified models not in the live results must be appended.""" - _config_ids = ["assistant", "local-finetune"] - live_models = ["assistant", "assistant-pro", "gpt-5.5:pt"] - - ids = list(live_models) - _live_set = set(ids) - for _cid in _config_ids: - if _cid not in _live_set: - ids.append(_cid) - - # "assistant" is in both, "local-finetune" is config-only - self.assertIn("local-finetune", ids) - self.assertIn("assistant-pro", ids) - self.assertEqual(ids.count("assistant"), 1, "assistant must not be duplicated") - - def test_live_fetch_guard_no_longer_checks_ids(self): - """The live fetch block must not be guarded by `if not ids`. - - This is a structural regression guard: the fix replaced - `if not ids and (provider == "custom" ...)` with - `if provider == "custom" ...` so the probe always runs regardless - of whether config entries populated ids. - """ - source = ROUTES_PY.read_text(encoding="utf-8") - - # Find the "Always try live fetch" comment (added by the fix) - marker = "Always try live fetch for custom providers" - self.assertIn(marker, source, ( - "routes.py must contain the 'Always try live fetch' comment (#3718)" - )) - - # Extract the block between the marker and the next major section - marker_pos = source.find(marker) - next_section = source.find("OpenAI-compat live fetch fallback", marker_pos) - self.assertNotEqual(next_section, -1, "could not find next section marker") - block = source[marker_pos:next_section] - - # The old `if not ids and` guard must NOT appear in this block - self.assertNotIn("if not ids and", block, ( - "Live fetch for custom providers must not be guarded by 'if not ids' (#3718)" - )) - - def test_mocked_live_fetch_returns_full_catalog_plus_config_entry(self): - """Integration test: mock urlopen and exercise the real fetch/parse/merge path. - - This test patches urllib.request.urlopen to return a fake /v1/models - response, then runs the same fetch+merge logic that the live handler - uses, verifying that config entries are merged and live models are - included. - """ - fake_models = {"data": [ - {"id": "assistant"}, - {"id": "assistant-pro"}, - {"id": "gpt-5.5:pt"}, - ]} - fake_body = json.dumps(fake_models).encode("utf-8") - fake_resp = mock.MagicMock() - fake_resp.read.return_value = fake_body - fake_resp.__enter__ = mock.MagicMock(return_value=fake_resp) - fake_resp.__exit__ = mock.MagicMock(return_value=False) - - _config_ids = ["assistant", "local-only"] - - with mock.patch("urllib.request.urlopen", return_value=fake_resp): - # Replicate the fetch+parse logic from routes.py - import urllib.request - ids = [] - _req = urllib.request.Request( - "http://localhost:4000/v1/models", - headers={"Authorization": "Bearer test-key"}, - ) - with urllib.request.urlopen(_req, timeout=5) as _resp: - _body = json.loads(_resp.read()) - - if isinstance(_body, dict): - _data = _body.get("data", []) - if isinstance(_data, list): - ids = [m.get("id", "") for m in _data if m.get("id")] - - # Apply the merge logic from the fix - if ids: - _live_set = set(ids) - for _cid in _config_ids: - if _cid not in _live_set: - ids.append(_cid) - else: - ids = list(_config_ids) - - # Live models must all be present - self.assertIn("assistant", ids) - self.assertIn("assistant-pro", ids) - self.assertIn("gpt-5.5:pt", ids) - # Config-only entry must be appended - self.assertIn("local-only", ids) - # No duplicates - self.assertEqual(ids.count("assistant"), 1) - - def test_timeout_uses_config_constant(self): - """The live fetch must use CUSTOM_MODELS_ENDPOINT_TIMEOUT_SECONDS, not a hardcoded value.""" - source = ROUTES_PY.read_text(encoding="utf-8") - - # Find the custom-provider live fetch block - marker = "Always try live fetch for custom providers" - marker_pos = source.find(marker) - next_section = source.find("OpenAI-compat live fetch fallback", marker_pos) - block = source[marker_pos:next_section] - - # Must use the constant, not a bare number - self.assertIn("CUSTOM_MODELS_ENDPOINT_TIMEOUT_SECONDS", block, ( - "Live fetch timeout must use CUSTOM_MODELS_ENDPOINT_TIMEOUT_SECONDS " - "instead of a hardcoded value (#3718 review feedback)" - )) - self.assertNotIn("timeout=8", block, ( - "Live fetch must not use hardcoded timeout=8 (#3718 review feedback)" - )) +def _install_provider_model_ids(monkeypatch, fn): + """Stub hermes_cli.models.provider_model_ids (returns [] for custom:*).""" + hermes_cli = types.ModuleType("hermes_cli") + hermes_cli.__path__ = [] + models = types.ModuleType("hermes_cli.models") + models.provider_model_ids = fn + monkeypatch.setitem(sys.modules, "hermes_cli", hermes_cli) + monkeypatch.setitem(sys.modules, "hermes_cli.models", models) -if __name__ == "__main__": - unittest.main() \ No newline at end of file +class _FakeResponse(io.BytesIO): + def __enter__(self): + return self + + def __exit__(self, *_exc): + self.close() + return False + + +def _install_live_catalog(monkeypatch, routes, catalog, *, fail=False): + """Stub the upstream /v1/models probe used by the custom-provider branch.""" + import urllib.request + + requested: list[str] = [] + + def _fake_urlopen(req, timeout=None): + requested.append(getattr(req, "full_url", str(req))) + if fail: + raise OSError("probe failed") + body = {"data": [{"id": mid} for mid in catalog]} + return _FakeResponse(json.dumps(body).encode()) + + monkeypatch.setattr(urllib.request, "urlopen", _fake_urlopen) + return requested + + +def _run_live_models(monkeypatch, routes, custom_provider, *, catalog, fail=False): + """Invoke the real handler for provider=custom:test-gateway.""" + import api.config as config + import api.profiles as profiles + + routes._clear_live_models_cache() + monkeypatch.setattr( + routes, "j", lambda _handler, payload, status=200, extra_headers=None: payload + ) + monkeypatch.setattr(config, "_resolve_provider_alias", lambda provider: provider) + monkeypatch.setattr(profiles, "get_active_profile_name", lambda: "default") + + config_data = { + "model": {"provider": "custom:test-gateway"}, + "custom_providers": [custom_provider], + } + monkeypatch.setattr(config, "get_config", lambda: config_data) + monkeypatch.setattr(routes, "cfg", config_data, raising=False) + + # provider_model_ids() returns [] for custom:* → drives the custom branch. + _install_provider_model_ids(monkeypatch, lambda provider: []) + requested = _install_live_catalog(monkeypatch, routes, catalog, fail=fail) + + parsed = urlparse("/api/models/live?provider=custom:test-gateway") + payload = routes._handle_live_models(object(), parsed) + return payload, requested + + +def _ids(payload): + return [m["id"] for m in payload.get("models", [])] + + +_LIVE_CATALOG = [ + "chat-a", + "chat-b", + "image-gen-1", # non-chat noise the picker must not surface + "audio-tts-1", +] + +_BASE_PROVIDER = { + "name": "Test Gateway", + "base_url": "https://gateway.example/v1", + "api_key": "sk-test", +} + + +@pytest.mark.parametrize( + "models_value", + [ + pytest.param(["chat-a", "chat-b"], id="native-list"), + pytest.param('["chat-a","chat-b"]', id="json-array-string"), + pytest.param("['chat-a', 'chat-b']", id="python-literal-string"), + pytest.param( + [{"id": "chat-a"}, {"model": "chat-b"}], id="list-of-dicts" + ), + ], +) +def test_allowlist_filters_live_catalog(monkeypatch, models_value): + """An explicit plural allowlist must trim the live catalog to itself. + + The JSON-array-string and Python-literal cases are the regression: those + shapes are what ``hermes config set`` / JSON-mode editor saves persist, and + a dict/list-only parser drops them to ``[]`` → read as "no allowlist" → the + full upstream catalog floods the picker. + """ + import api.routes as routes + + provider = dict(_BASE_PROVIDER, models=models_value) + payload, requested = _run_live_models( + monkeypatch, routes, provider, catalog=_LIVE_CATALOG + ) + + assert requested, "live /v1/models probe was never attempted" + assert _ids(payload) == ["chat-a", "chat-b"] + assert "image-gen-1" not in _ids(payload) + assert "audio-tts-1" not in _ids(payload) + + +def test_dict_models_without_discover_false_does_not_gate(monkeypatch): + """A dict-shaped ``models`` is per-model metadata — NOT an allowlist (#7165). + + Hermes Agent's setup flow and ``hermes_cli/model_switch.py::_save_custom_provider`` + persist per-model metadata as a mapping, e.g. ``{chat-a: {context_length: 128000}}``. + Treating its keys as an allowlist would collapse the live picker to the saved + default while the CLI live-probe shows the full catalog. Without an explicit + ``discover_models: false`` opt-out the dict must NOT gate — full live catalog wins. + """ + import api.routes as routes + + provider = dict( + _BASE_PROVIDER, + models={"chat-a": {"context_length": 128000}, "chat-b": {"context_length": 32000}}, + ) + payload, requested = _run_live_models( + monkeypatch, routes, provider, catalog=_LIVE_CATALOG + ) + + assert requested, "live /v1/models probe was never attempted" + assert _ids(payload) == _LIVE_CATALOG, ( + "dict-shaped models is per-model metadata, not an allowlist — the " + "full live catalog must be surfaced" + ) + + +def test_dict_models_with_discover_false_pins_to_keys(monkeypatch): + """``discover_models: false`` opts a dict-shaped ``models`` into an allowlist. + + The explicit opt-out (bool ``False`` or the string forms ``"false"``/``"no"``/``"0"``) + means the dict keys are a hand-pinned allowlist and must filter the live catalog. + """ + import api.routes as routes + + for discover in (False, "false", "no", "0"): + provider = dict( + _BASE_PROVIDER, + models={"chat-a": {"context_length": 128000}, "chat-b": {"context_length": 32000}}, + discover_models=discover, + ) + payload, requested = _run_live_models( + monkeypatch, routes, provider, catalog=_LIVE_CATALOG + ) + assert requested, "live /v1/models probe was never attempted" + assert _ids(payload) == ["chat-a", "chat-b"], ( + f"discover_models={discover!r} must pin the dict keys as the allowlist" + ) + assert "image-gen-1" not in _ids(payload) + assert "audio-tts-1" not in _ids(payload) + + +def test_singular_model_only_does_not_gate_live_catalog(monkeypatch): + """``model:`` is default metadata, not an allowlist — keep full discovery.""" + import api.routes as routes + + provider = dict(_BASE_PROVIDER, model="chat-a") + payload, _ = _run_live_models( + monkeypatch, routes, provider, catalog=_LIVE_CATALOG + ) + + assert _ids(payload) == _LIVE_CATALOG + + +def test_malformed_serialized_models_is_safe(monkeypatch): + """A broken JSON-array string must not crash or silently drop the gate. + + ``_parse_config_string_list()`` treats an undecodable value as one literal + name, so the allowlist stays non-empty and the filter still runs. + """ + import api.routes as routes + + provider = dict(_BASE_PROVIDER, models='["chat-a", "chat-b"') # missing ] + payload, _ = _run_live_models( + monkeypatch, routes, provider, catalog=_LIVE_CATALOG + ) + + # Treated as a single (unmatched) name → appended, catalog not flooded. + assert "image-gen-1" not in _ids(payload) + assert "audio-tts-1" not in _ids(payload) + + +def test_allowlisted_model_missing_from_live_is_appended(monkeypatch): + """A declared model the upstream didn't return must still be selectable.""" + import api.routes as routes + + provider = dict(_BASE_PROVIDER, models=["chat-a", "chat-offline"]) + payload, _ = _run_live_models( + monkeypatch, routes, provider, catalog=_LIVE_CATALOG + ) + + assert _ids(payload) == ["chat-a", "chat-offline"] + + +def test_contradictory_singular_model_outside_plural_allowlist_is_ignored(monkeypatch): + """A singular default outside the plural allowlist must not leak into the picker. + + Paired test: whether the live probe succeeds or fails, only the explicit + plural allowlist is exposed. + """ + import api.routes as routes + + provider = dict(_BASE_PROVIDER, model="chat-default", models=["chat-a"]) + + # 1. Successful live fetch -> only plural allowlist + payload_success, _ = _run_live_models( + monkeypatch, routes, provider, catalog=_LIVE_CATALOG, fail=False + ) + assert _ids(payload_success) == ["chat-a"] + + # 2. Failed live fetch -> still only plural allowlist (not chat-default) + payload_fail, _ = _run_live_models( + monkeypatch, routes, provider, catalog=_LIVE_CATALOG, fail=True + ) + assert _ids(payload_fail) == ["chat-a"] + + +def test_probe_failure_without_allowlist_falls_back_to_singular_model(monkeypatch): + """When no plural allowlist is configured and probe fails, singular model is used.""" + import api.routes as routes + + provider = dict(_BASE_PROVIDER, model="chat-default") + payload, _ = _run_live_models( + monkeypatch, routes, provider, catalog=_LIVE_CATALOG, fail=True + ) + assert _ids(payload) == ["chat-default"] + + +@pytest.mark.parametrize( + "models_value", + [ + pytest.param([], id="native-empty-list"), + pytest.param("[]", id="serialized-empty-list"), + pytest.param({}, id="empty-mapping"), + pytest.param(" ", id="blank-scalar"), + ], +) +def test_empty_allowlist_is_treated_as_not_configured(monkeypatch, models_value): + """An empty ``models`` value must NOT be read as "allow nothing". + + Pins a deliberate semantic decision. Gating the filter on *declared-ness* + rather than on a non-empty allowlist would make ``models: []`` emit an + EMPTY picker, leaving the provider unselectable — a harder failure than + surfacing a few extra models, and unrecoverable from the UI because + ``custom_providers`` has no WebUI write path (hand-edited config.yaml only). + There is also no deny-all use case: a provider the user wants hidden is + removed from ``custom_providers`` outright. + + If someone later "fixes" the empty case into a deny-all gate, this test + fails and forces the trade-off to be re-argued rather than silently shipped. + """ + import api.routes as routes + + provider = dict(_BASE_PROVIDER, models=models_value) + payload, _ = _run_live_models( + monkeypatch, routes, provider, catalog=_LIVE_CATALOG + ) + + assert _ids(payload) == _LIVE_CATALOG, ( + "empty allowlist must fall through to full live discovery, " + "never collapse the picker to zero models" + ) + assert _ids(payload), "picker must never be emptied by an empty allowlist" + + +def test_models_discovered_catalog_is_not_an_allowlist(monkeypatch): + """``models_discovered: true`` marks ``models:`` as discovery metadata (#7165). + + Hermes persists *discovery results* back into config as ``models: {...}`` + plus ``models_discovered: true``. That mapping is a snapshot of what the + gateway exposed at discovery time — NOT a hand-curated allowlist. Treating + it as one permanently pins the live catalog to the first-discovery set, so + a model the user later pulls into LM Studio / Ollama is silently dropped + from ``/api/models/live`` (the exact regression the gate reproduced: merge + base returned ``saved-a``, ``saved-b`` AND ``new-live-c``; the pinned head + dropped ``new-live-c``). + """ + import api.routes as routes + + provider = dict( + _BASE_PROVIDER, + models={ + "saved-a": {"context_length": 128000}, + "saved-b": {"context_length": 32000}, + }, + models_discovered=True, + ) + payload, requested = _run_live_models( + monkeypatch, + routes, + provider, + catalog=["saved-a", "saved-b", "new-live-c"], + ) + + assert requested, "live /v1/models probe was never attempted" + assert _ids(payload) == ["saved-a", "saved-b", "new-live-c"], ( + "a discovered catalog must not gate the live probe — models that " + "appear upstream after discovery (new-live-c) must reach the picker" + ) + + +def test_discovered_catalog_with_discover_models_false_still_pins(monkeypatch): + """``discover_models: false`` re-pins a discovered catalog (control). + + The explicit opt-out wins over ``models_discovered: true`` — same rule as + the ``/api/models`` catalog path (`api.config._provider_discover_allowed`). + Filtering to the configured set must survive the discovered-catalog fix. + """ + import api.routes as routes + + provider = dict( + _BASE_PROVIDER, + models={ + "saved-a": {"context_length": 128000}, + "saved-b": {"context_length": 32000}, + }, + models_discovered=True, + discover_models=False, + ) + payload, _ = _run_live_models( + monkeypatch, + routes, + provider, + catalog=["saved-a", "saved-b", "new-live-c"], + ) + + assert _ids(payload) == ["saved-a", "saved-b"] + + +def test_discovered_catalog_probe_failure_falls_back_to_saved_models(monkeypatch): + """Discovered + failed probe → saved models, not an empty picker (control). + + Releasing the allowlist must not empty the fallback: when the live probe + fails, the previously discovered ``models:`` mapping is still the best + answer and must be served from ``_config_ids``. + """ + import api.routes as routes + + provider = dict( + _BASE_PROVIDER, + models={ + "saved-a": {"context_length": 128000}, + "saved-b": {"context_length": 32000}, + }, + models_discovered=True, + ) + payload, _ = _run_live_models( + monkeypatch, routes, provider, catalog=[], fail=True + ) + + assert _ids(payload) == ["saved-a", "saved-b"] diff --git a/tests/test_issue5121_provider_auth_terminal_error.py b/tests/test_issue5121_provider_auth_terminal_error.py index 19c9c817..09ada5b9 100644 --- a/tests/test_issue5121_provider_auth_terminal_error.py +++ b/tests/test_issue5121_provider_auth_terminal_error.py @@ -423,6 +423,72 @@ def test_auth_retry_success_does_not_append_error_turn(tmp_path, monkeypatch): assert not any(msg.get("_error") for msg in saved.messages) +def test_auth_retry_success_observes_replacement_agent_runtime(tmp_path, monkeypatch): + from api import run_journal + + session = _prepare_session("auth_runtime", "stream_auth_runtime", pending_user_message="Please retry") + Base = _build_auth_failure_agent(token_text="") + + class ReplacementAgent(Base): + def __init__(self, **kwargs): + super().__init__(**kwargs) + self.model = "original" if self.__class__.runs == 0 else "healed-model" + self.provider = "old-provider" if self.__class__.runs == 0 else "healed-provider" + + monkeypatch.setattr(run_journal, "_default_session_dir", lambda: tmp_path / "sessions") + q = queue.Queue() + streaming.STREAMS["stream_auth_runtime"] = q + config.STREAM_PARTIAL_TEXT["stream_auth_runtime"] = "" + heal_rt = {"provider": "test-provider", "api_key": "fresh-key", "base_url": None} + with mock.patch.object(streaming, "get_session", return_value=session), \ + mock.patch.object(streaming, "_get_ai_agent", return_value=ReplacementAgent), \ + mock.patch.object(streaming, "resolve_model_provider", return_value=("test-model", "test-provider", None)), \ + mock.patch("api.config.get_config", return_value={}), \ + mock.patch("api.config._resolve_cli_toolsets", return_value=[]), \ + mock.patch.object(streaming, "_attempt_credential_self_heal", return_value=heal_rt): + streaming._run_agent_streaming(session.session_id, "Please retry", "test-model", str(tmp_path), "stream_auth_runtime") + + events = _queue_events(q) + assert any(name == "done" for name, _ in events) + assert [(item["model"], item["provider"]) for name, item in events if name == "runtime_model"] == [ + ("healed-model", "healed-provider")] + summary = run_journal.latest_run_summary("auth_runtime", "stream_auth_runtime", session_dir=tmp_path / "sessions") + assert summary["runtime_model"]["model"] == "healed-model" + + +def test_auth_exception_retry_observes_replacement_agent(tmp_path, monkeypatch): + session = _prepare_session("auth_exception_runtime", "stream_auth_exception_runtime", pending_user_message="Retry") + + class RetryAgent(MockAgent): + runs = 0 + def __init__(self, **kwargs): + super().__init__(**kwargs) + self.model = "original" if type(self).runs == 0 else "retry-model" + self.provider = "original-provider" if type(self).runs == 0 else "retry-provider" + def run_conversation(self, **kwargs): + type(self).runs += 1 + if type(self).runs == 1: + raise RuntimeError("401 unauthorized") + return {"messages": list(kwargs.get("conversation_history") or []) + [ + {"role": "assistant", "content": "Recovered answer"}]} + + q = queue.Queue() + streaming.STREAMS["stream_auth_exception_runtime"] = q + config.STREAM_PARTIAL_TEXT["stream_auth_exception_runtime"] = "" + heal_rt = {"provider": "test-provider", "api_key": "fresh-key", "base_url": None} + with mock.patch.object(streaming, "get_session", return_value=session), \ + mock.patch.object(streaming, "_get_ai_agent", return_value=RetryAgent), \ + mock.patch.object(streaming, "resolve_model_provider", return_value=("test-model", "test-provider", None)), \ + mock.patch("api.config.get_config", return_value={}), \ + mock.patch("api.config._resolve_cli_toolsets", return_value=[]), \ + mock.patch.object(streaming, "_attempt_credential_self_heal", return_value=heal_rt): + streaming._run_agent_streaming(session.session_id, "Retry", "test-model", str(tmp_path), "stream_auth_exception_runtime") + events = _queue_events(q) + assert any(name == "done" for name, _ in events) + assert [(item["model"], item["provider"]) for name, item in events if name == "runtime_model"] == [ + ("retry-model", "retry-provider")] + + def test_auth_exception_retry_structured_failure_still_emits_error( tmp_path, monkeypatch, diff --git a/tests/test_issue6068_used_model_footer.py b/tests/test_issue6068_used_model_footer.py index 7ba04c53..ef0fd09b 100644 --- a/tests/test_issue6068_used_model_footer.py +++ b/tests/test_issue6068_used_model_footer.py @@ -140,10 +140,11 @@ console.log(JSON.stringify({{ def test_streaming_stamps_used_model_on_assistant_message_and_usage_payload(): assert "_dm['_usedModel'] = _used_model" in STREAMING_PY - # The served model must be read from the agent AFTER the run — the agent - # mutates agent.model when a fallback fires, so the pre-run resolved_model - # would mis-attribute fallback turns. - assert "_used_model = getattr(agent, 'model', None) or resolved_model or model" in STREAMING_PY + # Read the served model from the agent after the run (including fallback), + # without treating a requested/configured selection as proof of service. + assert "_observed_model = getattr(agent, 'model', None)" in STREAMING_PY + assert "_used_model = _observed_model.strip() if isinstance(_observed_model, str) else None" in STREAMING_PY + assert "_used_model = getattr(agent, 'model', None) or resolved_model or model" not in STREAMING_PY assert "usage['used_model'] = _used_model" in STREAMING_PY diff --git a/tests/test_issue6366_stale_cancel_recovery.py b/tests/test_issue6366_stale_cancel_recovery.py new file mode 100644 index 00000000..a76b3d73 --- /dev/null +++ b/tests/test_issue6366_stale_cancel_recovery.py @@ -0,0 +1,1424 @@ +"""Tests for #6366: stale-cancel recovery must not append duplicate +rows when the durable transcript has already advanced past the +pending turn — and must never silently discard a distinct repeated +prompt or an interrupted tool-running turn. + +Bug shape: the recovery path at ``api.models.py::_apply_core_sync_or_error_marker`` +checks only ``session.messages[-1]`` against the pending user +checkpoint. When a stale pending turn's user row is older than the +transcript tail, the tail is a *newer* settled assistant row that +can never match the pending **user** checkpoint, and the recovery +path falls through into the append branch. It then appends a +recovered user row, a ``_partial`` clone of the old journal, and a +generic ``No response from provider`` error — all after the valid +final answer. The duplicates survive reload. + +Fix: add a transcript-advance detector that scans the durable +messages for the pending turn's own user row at any index and, when +found, checks for a genuine final assistant answer inside that turn's +boundary. If one exists, the recovery path must clear the stale +pending fields without appending any rows. The helper is a pure read +so recovery is idempotent: running it twice produces the same +outcome. + +Two guard rails keep the suppression from destroying data: + +* **Identity** is bound to the pending stream's exact active-turn + token, never to integer-second timestamp equality. Two different + streams that send the same prompt within one second truncate to the + same second, so the checkpoint matcher alone would match the other + stream's row and the branch would then clear the only durable copy + of the prompt. A token mismatch — or a token-less legacy row — + always loses, so an unprovable identity recovers instead of + suppressing. + +* **Completion** reuses the established final-answer semantics + (``_assistant_message_has_final_visible_text``) rather than "an + assistant row exists". Empty, tool-call-only, interim ``_partial`` + and compaction tails all keep recovery active, so a turn that was + interrupted mid-tool-execution is replayed and marked instead of + being treated as already advanced. + +Because a false negative leaves a visible cosmetic duplicate while a +false positive silently destroys a prompt, the suppression defaults to +recovering whenever identity or completion cannot be proven. The +full-recovery cases below exercise ``_apply_core_sync_or_error_marker`` +itself, because neither guard rail is reachable through the predicate +in isolation — the suppression decision only takes effect on the real +recovery path. +""" + +import json + +import pytest + +import api.models as models +from api.models import ( + SESSIONS, + Session, + _apply_core_sync_or_error_marker, + _pending_turn_has_final_assistant_answer, + _run_journal_terminal_state, + _transcript_already_advanced_past_pending, + _transcript_user_row_is_pending_turn, + _turn_journal_records_completion, +) +from api.run_journal import append_run_event +from api.turn_journal import append_turn_journal_event_for_stream + + +@pytest.fixture(autouse=True) +def _isolate_session_state(tmp_path, monkeypatch): + session_dir = tmp_path / "sessions" + session_dir.mkdir() + monkeypatch.setattr(models, "SESSION_DIR", session_dir) + monkeypatch.setattr(models, "SESSION_INDEX_FILE", session_dir / "_index.json") + SESSIONS.clear() + yield + SESSIONS.clear() + + +class _FakeSession: + """Minimal stand-in for api.models.Session — only the fields the + new helpers read.""" + + def __init__( + self, + pending=None, + messages=None, + *, + stream_id=None, + started_at=None, + source=None, + attachments=None, + session_id="fake-session-6366", + ): + self.session_id = session_id + self.pending_user_message = pending + self.messages = list(messages or []) + self.active_stream_id = stream_id + self.pending_started_at = started_at + self.pending_user_source = source + self.pending_attachments = attachments + + +def _token(stream_id, started_at): + return f"{stream_id}:{float(started_at):.17g}" + + +# ── _transcript_user_row_is_pending_turn identity cases ───────────────── + + +def test_user_row_matches_only_with_identical_active_turn_token(): + """Identity is unforgeable: the row's ``_active_turn_token`` must equal + the token derived from the pending stream id + started_at.""" + s = _FakeSession( + pending="hello world", + stream_id="stream-a", + started_at=222.5, + ) + own_row = { + "role": "user", + "content": "hello world", + "timestamp": 222, + "_active_turn_token": "stream-a:222.5", + } + other_row = { + "role": "user", + "content": "hello world", + "timestamp": 222, + "_active_turn_token": "stream-b:222.5", + } + assert _transcript_user_row_is_pending_turn(own_row, s, _token("stream-a", 222.5)) is True + assert _transcript_user_row_is_pending_turn(other_row, s, _token("stream-a", 222.5)) is False + + +def test_user_row_without_token_never_matches_a_token_bearing_pending_turn(): + """A legacy row that predates token stamping cannot be proven to be the + pending turn, so the caller must recover rather than suppress.""" + s = _FakeSession( + pending="hello world", + stream_id="stream-a", + started_at=222.5, + ) + legacy_row = { + "role": "user", + "content": "hello world", + "timestamp": 222, + } + assert _transcript_user_row_is_pending_turn(legacy_row, s, _token("stream-a", 222.5)) is False + + +def test_token_bearing_row_never_matches_a_token_less_pending_turn(): + """The mirror case: a token-bearing row belongs to a real turn, so it + must not be claimed by a pending turn that has no resolvable token.""" + s = _FakeSession( + pending="hello world", + stream_id=None, + started_at=None, + ) + token_row = { + "role": "user", + "content": "hello world", + "timestamp": 222, + "_active_turn_token": "stream-a:222.5", + } + assert _transcript_user_row_is_pending_turn(token_row, s, None) is False + + +def test_legacy_pending_turn_falls_back_to_strict_checkpoint_match(): + """With no token on either side the strict content + timestamp + source + + attachments checkpoint is the only identity signal, and it still + distinguishes a same-second repeated prompt from the pending turn.""" + s = _FakeSession( + pending="hello world", + stream_id=None, + started_at=222, + source="webui", + attachments=[{"name": "a.png"}], + ) + same_turn_row = { + "role": "user", + "content": "hello world", + "timestamp": 222, + "_source": "webui", + "attachments": [{"name": "a.png"}], + } + other_turn_row = { + "role": "user", + "content": "hello world", + "timestamp": 222, + "_source": "webui", + "attachments": [{"name": "old.png"}], + } + assert _transcript_user_row_is_pending_turn(same_turn_row, s, None) is True + assert _transcript_user_row_is_pending_turn(other_turn_row, s, None) is False + + +# ── _pending_turn_has_final_assistant_answer completion cases ─────────── + + +def test_final_answer_requires_visible_answer_text(): + assert _pending_turn_has_final_assistant_answer( + [ + {"role": "user", "content": "q"}, + {"role": "assistant", "content": "Here is the final answer."}, + ], + 0, + ) is True + + +def test_final_answer_rejects_empty_assistant_row(): + assert _pending_turn_has_final_assistant_answer( + [ + {"role": "user", "content": "q"}, + {"role": "assistant", "content": ""}, + ], + 0, + ) is False + + +def test_final_answer_rejects_tool_call_only_row(): + assert _pending_turn_has_final_assistant_answer( + [ + {"role": "user", "content": "q"}, + { + "role": "assistant", + "content": "I found a source and will inspect it.", + "tool_calls": [{"id": "call_1"}], + }, + {"role": "tool", "tool_call_id": "call_1", "content": "result"}, + ], + 0, + ) is False + + +def test_final_answer_rejects_interim_partial_row(): + assert _pending_turn_has_final_assistant_answer( + [ + {"role": "user", "content": "q"}, + { + "role": "assistant", + "content": "Still working on it…", + "_partial": True, + }, + ], + 0, + ) is False + + +def test_final_answer_rejects_compaction_marker_row(): + assert _pending_turn_has_final_assistant_answer( + [ + {"role": "user", "content": "q"}, + { + "role": "assistant", + "content": "[CONTEXT COMPACTION — REFERENCE ONLY] summary", + }, + ], + 0, + ) is False + + +def test_final_answer_accepts_answer_after_compaction_marker(): + assert _pending_turn_has_final_assistant_answer( + [ + {"role": "user", "content": "q"}, + { + "role": "assistant", + "content": "[CONTEXT COMPACTION — REFERENCE ONLY] summary", + }, + {"role": "assistant", "content": "Here is the final answer."}, + ], + 0, + ) is True + + +def test_final_answer_does_not_leak_across_a_later_user_turn(): + """A final answer belonging to a *later* turn must not settle this one.""" + assert _pending_turn_has_final_assistant_answer( + [ + {"role": "user", "content": "q"}, + {"role": "user", "content": "later question"}, + {"role": "assistant", "content": "Here is the final answer."}, + ], + 0, + ) is False + + +def test_final_answer_tolerates_tool_rows_before_the_answer(): + assert _pending_turn_has_final_assistant_answer( + [ + {"role": "user", "content": "q"}, + {"role": "assistant", "content": "", "tool_calls": [{"id": "call_1"}]}, + {"role": "tool", "tool_call_id": "call_1", "content": "result"}, + {"role": "assistant", "content": "Here is the final answer."}, + ], + 0, + ) is True + + +# ── _transcript_already_advanced_past_pending unit cases ─────────────── + + +def test_advances_past_pending_when_user_matched_and_settled_assistant_follows(): + """Canonical bug shape: a stale pending turn's user row sits at + an older index, and a newer settled assistant row follows. + Recovery must not append duplicate rows after the assistant. + Identity comes from the pending stream's own active-turn token; + the journal is marked ``done`` (terminal_state ``completed``) so + the suppression gate is satisfied.""" + s = _FakeSession( + pending="hello world", + messages=[ + { + "role": "user", + "content": "hello world", + "timestamp": 222, + "_source": "webui", + "attachments": [{"name": "a.png"}], + "_active_turn_token": "stream-a:222.5", + }, + {"role": "assistant", "content": "answer", "timestamp": 223}, + ], + stream_id="stream-a", + started_at=222.5, + source="webui", + attachments=[{"name": "a.png"}], + ) + _write_journal(s.session_id, "stream-a", [("done", {})]) + _write_turn_journal( + s.session_id, + "stream-a", + [("submitted", {}), ("completed", {})], + ) + assert _transcript_already_advanced_past_pending(s) is True + + +def test_advances_past_pending_ignores_partial_clone_after_user(): + """A ``_partial`` clone after the pending turn's user row does NOT count + as a settled advance — the canonical bug shape is the partial + clone followed by a generic no-response error, both of which + the previous recovery path appended. Only a genuine final visible + answer represents a real advance.""" + s = _FakeSession( + pending="hello world", + messages=[ + { + "role": "user", + "content": "hello world", + "timestamp": 1, + "_active_turn_token": "stream-a:1.5", + }, + {"role": "assistant", "content": "answer", "timestamp": 2, "_partial": True}, + {"role": "assistant", "content": "no response", "timestamp": 3, "_error": True}, + ], + stream_id="stream-a", + started_at=1.5, + ) + assert _transcript_already_advanced_past_pending(s) is False + + +def test_advances_past_pending_ignores_recovered_assistant_row(): + """A previously-recovered assistant row is not a settled answer + either; the recovery path may have produced it on an earlier + pass. Skip it and keep looking forward.""" + s = _FakeSession( + pending="hello world", + messages=[ + { + "role": "user", + "content": "hello world", + "timestamp": 1, + "_active_turn_token": "stream-a:1.5", + }, + {"role": "assistant", "content": "answer", "timestamp": 2, "_recovered": True}, + ], + stream_id="stream-a", + started_at=1.5, + ) + assert _transcript_already_advanced_past_pending(s) is False + + +def test_does_not_advance_when_only_user_row_matches(): + """The pending turn's user row exists but no final assistant answer has + been settled. Recovery is the legitimate path; the helper + returns False.""" + s = _FakeSession( + pending="hello world", + messages=[ + { + "role": "user", + "content": "hello world", + "timestamp": 1, + "_active_turn_token": "stream-a:1.5", + }, + ], + stream_id="stream-a", + started_at=1.5, + ) + assert _transcript_already_advanced_past_pending(s) is False + + +def test_does_not_advance_when_no_user_row_matches(): + """No transcript row carries the pending turn's token. The bug shape + here is the older non-matching rows + a stale pending. The + recovery path is legitimate; the helper returns False.""" + s = _FakeSession( + pending="hello world", + messages=[ + { + "role": "user", + "content": "hello world", + "timestamp": 1, + "_active_turn_token": "stream-z:1.5", + }, + {"role": "assistant", "content": "answer", "timestamp": 2}, + ], + stream_id="stream-a", + started_at=1.5, + ) + assert _transcript_already_advanced_past_pending(s) is False + + +def test_does_not_advance_when_no_pending_text(): + """An empty pending text means there is nothing to compare + against. The helper returns False without scanning the + transcript.""" + s = _FakeSession( + pending=None, + messages=[ + {"role": "assistant", "content": "answer", "timestamp": 2}, + ], + ) + assert _transcript_already_advanced_past_pending(s) is False + + +def test_does_not_advance_when_no_messages(): + """An empty transcript is the cold-start case; recovery is the + legitimate path. The helper returns False.""" + s = _FakeSession(pending="hello world", messages=[]) + assert _transcript_already_advanced_past_pending(s) is False + + +def test_advances_past_pending_with_unrelated_intervening_rows(): + """The pending turn's user row followed by a tool row and THEN a final + answer counts as a real advance. The helper scans past non-assistant + roles correctly, and the journal is marked ``done`` so the durable + terminal evidence gate is satisfied.""" + s = _FakeSession( + pending="hello world", + messages=[ + { + "role": "user", + "content": "hello world", + "timestamp": 222, + "_source": "webui", + "attachments": [], + "_active_turn_token": "stream-a:222.5", + }, + {"role": "tool", "content": "tool output"}, + {"role": "assistant", "content": "answer", "timestamp": 223}, + ], + stream_id="stream-a", + started_at=222.5, + source="webui", + attachments=[], + ) + _write_journal(s.session_id, "stream-a", [("done", {})]) + _write_turn_journal( + s.session_id, + "stream-a", + [("submitted", {}), ("completed", {})], + ) + assert _transcript_already_advanced_past_pending(s) is True + + +def test_helper_is_pure_read_idempotent(): + """Idempotence: running the helper twice on the same session + state returns the same result. The helper does not mutate the + session, so recovery is automatically safe to run twice. Journal + is marked ``done`` so the durable terminal evidence gate passes + on both invocations.""" + s = _FakeSession( + pending="hello world", + messages=[ + { + "role": "user", + "content": "hello world", + "timestamp": 222, + "_source": "webui", + "attachments": [], + "_active_turn_token": "stream-a:222.5", + }, + {"role": "assistant", "content": "answer", "timestamp": 223}, + ], + stream_id="stream-a", + started_at=222.5, + source="webui", + attachments=[], + ) + _write_journal(s.session_id, "stream-a", [("done", {})]) + _write_turn_journal( + s.session_id, + "stream-a", + [("submitted", {}), ("completed", {})], + ) + first = _transcript_already_advanced_past_pending(s) + second = _transcript_already_advanced_past_pending(s) + assert first is True + assert second is True + # And the session state was not mutated. + assert s.pending_user_message == "hello world" + assert len(s.messages) == 2 + + +def test_does_not_advance_when_repeated_prompt_has_different_token(): + """#6366 regression guard for repeated-prompt identity: a *new* + user turn that happens to repeat the same prompt text as an + older transcript row is NOT a stale advance. The active-turn + token tells a stale duplicate from a legitimate re-send, even + when both truncate to the same integer second.""" + s = _FakeSession( + pending="repeat this", + messages=[ + # Older user row from a DIFFERENT stream, same text and the + # same integer second. + { + "role": "user", + "content": "repeat this", + "timestamp": 111, + "_source": "webui", + "attachments": [{"name": "old.png"}], + "_active_turn_token": "stream-old:111.5", + }, + {"role": "assistant", "content": "Earlier answer"}, + # The current turn's own row — pending, no answer yet. + { + "role": "user", + "content": "repeat this", + "timestamp": 111, + "_source": "webui", + "attachments": [{"name": "current.png"}], + "_active_turn_token": "stream-a:111.5", + }, + ], + stream_id="stream-a", + started_at=111.5, + source="webui", + attachments=[{"name": "current.png"}], + ) + assert _transcript_already_advanced_past_pending(s) is False + + +def test_advances_past_pending_with_exact_token_match(): + """The strict-match path: a transcript user row that carries the + pending stream's own active-turn token, followed by a genuine final + answer, is the canonical #6366 bug shape and must suppress the + recovery append.""" + s = _FakeSession( + pending="hello world", + messages=[ + { + "role": "user", + "content": "hello world", + "timestamp": 222, + "_source": "webui", + "attachments": [{"name": "a.png"}], + "_active_turn_token": "stream-a:222.5", + }, + {"role": "assistant", "content": "answer", "timestamp": 223}, + ], + stream_id="stream-a", + started_at=222.5, + source="webui", + attachments=[{"name": "a.png"}], + ) + _write_journal(s.session_id, "stream-a", [("done", {})]) + _write_turn_journal( + s.session_id, + "stream-a", + [("submitted", {}), ("completed", {})], + ) + assert _transcript_already_advanced_past_pending(s) is True + + +def test_does_not_advance_when_final_answer_belongs_to_a_later_turn(): + """A final answer that sits after a newer user row belongs to that + newer turn, so it cannot prove the pending turn advanced.""" + s = _FakeSession( + pending="hello world", + messages=[ + { + "role": "user", + "content": "hello world", + "timestamp": 222, + "_active_turn_token": "stream-a:222.5", + }, + {"role": "user", "content": "a different question", "timestamp": 224}, + {"role": "assistant", "content": "answer", "timestamp": 225}, + ], + stream_id="stream-a", + started_at=222.5, + ) + assert _transcript_already_advanced_past_pending(s) is False + + +def test_does_not_advance_when_tool_execution_was_interrupted(): + """A turn interrupted mid-tool-execution ends with a tool-call-only + assistant row and a tool result — not a final answer. Recovery must + still replay the journal and mark the interruption.""" + s = _FakeSession( + pending="run the migration", + messages=[ + { + "role": "user", + "content": "run the migration", + "timestamp": 222, + "_active_turn_token": "stream-a:222.5", + }, + { + "role": "assistant", + "content": "I will run the migration now.", + "tool_calls": [{"id": "call_1"}], + }, + {"role": "tool", "tool_call_id": "call_1", "content": "partial output"}, + ], + stream_id="stream-a", + started_at=222.5, + ) + assert _transcript_already_advanced_past_pending(s) is False + + +def test_does_not_advance_when_unflagged_interim_prose_precedes_tools(): + """9/22 re-gate regression: an unflagged interim prose row + ("Let me check the logs first.") followed by tool rows passes the + transcript predicate (``_pending_turn_has_final_assistant_answer`` + returns True at the first visible assistant row), but the run + journal was never marked ``done`` — the stream may have died mid- + tool. Without durable same-stream terminal evidence the suppression + gate must stay closed, so ``_transcript_already_advanced_past_pending`` + returns False and the recovery path is free to run.""" + s = _FakeSession( + pending="check the failing test", + messages=[ + { + "role": "user", + "content": "check the failing test", + "timestamp": 222, + "_active_turn_token": "stream-a:222.5", + }, + # The bug shape: an unflagged assistant row with visible + # text and no tool_calls. The transcript heuristic accepts + # this as a final answer (it carries visible text and has + # no _partial / _error / _recovered marker), but the row + # is actually pre-final interim prose that the repository + # journals as ``interim_assistant`` while the model + # decides what tool to call. The tool_calls that + # produced the following tool rows live on a *later* row + # in the original stream — the journal has not yet + # recorded them because the WebUI process died. + { + "role": "assistant", + "content": "Let me check the logs first.", + }, + # The stream then ran tools before dying. The transcript + # therefore has tool rows after the unflagged prose. + {"role": "tool", "tool_call_id": "call_1", "content": "log tail"}, + {"role": "tool", "tool_call_id": "call_2", "content": "log grep"}, + ], + stream_id="stream-a", + started_at=222.5, + ) + # The journal has tokens and an interim_assistant checkpoint, but + # no ``done`` / ``stream_end`` — the run is still mid-tool from + # the journal's point of view. + _write_journal( + s.session_id, "stream-a", + [ + ("token", {"text": "Let me check the logs first."}), + ("interim_assistant", {"text": "Let me check the logs first."}), + ("tool", {"name": "terminal", "tid": "call_1", "preview": "tail -n 50"}), + ("tool_complete", {"name": "terminal", "tid": "call_1", "preview": "log tail"}), + ("tool", {"name": "terminal", "tid": "call_2", "preview": "grep ERROR"}), + ], + ) + assert _transcript_already_advanced_past_pending(s) is False + + +def test_does_advance_when_journal_marked_done_after_interim_prose(): + """Companion positive control: when the run journal IS marked + ``done`` (terminal_state ``completed``) for the same stream, the + suppression gate opens even though the transcript carries the + unflagged interim prose + tool shape. Without the journal evidence + the predicate would still hold the gate closed, so this case pins + the boundary between the two new branches.""" + s = _FakeSession( + pending="check the failing test", + messages=[ + { + "role": "user", + "content": "check the failing test", + "timestamp": 222, + "_active_turn_token": "stream-a:222.5", + }, + { + "role": "assistant", + "content": "Let me check the logs first.", + }, + {"role": "tool", "tool_call_id": "call_1", "content": "log tail"}, + {"role": "assistant", "content": "The logs show a stale lock.", "timestamp": 223}, + ], + stream_id="stream-a", + started_at=222.5, + ) + _write_journal(s.session_id, "stream-a", [("done", {})]) + _write_turn_journal( + s.session_id, + "stream-a", + [("submitted", {}), ("completed", {})], + ) + assert _transcript_already_advanced_past_pending(s) is True + + +# ── turn-journal completion evidence (9/23 re-gate) ─────────────────── +# +# The suppression predicate used to require a *run-journal* terminal +# ``completed`` state. Its caller in ``_apply_core_sync_or_error_marker`` +# already returns early on that same condition, so by the time the +# predicate ran the run-journal branch could never be true and the +# predicate could never suppress anything. The completion evidence +# therefore has to come from the turn journal — the crash-safe journal +# that records the exact-stream ``completed`` event for the pending turn. + + +_TURN_JOURNAL_CLOCK = {"tick": 0} + + +def _write_turn_journal(session_id, stream_id, events): + """Append events to the turn journal for ``stream_id``, reusing the + stream's turn id exactly like ``append_turn_journal_event_for_stream`` + does in production. + + ``created_at`` is taken from a module-level monotonic clock so events + appended across several calls — and across turn ids — keep a strict + ordering; a per-call sequence would hand the same timestamp to events + of different turns, leaving the derived "latest turn per stream" + ambiguous. + """ + written = [] + for event_name, payload in events: + _TURN_JOURNAL_CLOCK["tick"] += 1 + written.append( + append_turn_journal_event_for_stream( + session_id, + stream_id, + { + "event": event_name, + "created_at": 1_700_000_000.0 + _TURN_JOURNAL_CLOCK["tick"], + **payload, + }, + ) + ) + return written + + +def test_turn_journal_helper_reports_completed_for_matching_stream(): + """The helper reads the turn journal and reports completion only for + the stream whose ``completed`` event was recorded.""" + session_id = "issue6366_turn_journal_helper" + _write_turn_journal( + session_id, + "stream-a", + [("submitted", {}), ("completed", {})], + ) + probe = _FakeSession( + pending="hello world", + messages=[], + stream_id="stream-a", + started_at=222.5, + session_id=session_id, + ) + assert _turn_journal_records_completion(probe, "stream-a") is True + assert _turn_journal_records_completion(probe, "stream-b") is False + + +def test_turn_journal_helper_rejects_interrupted_turn(): + """An ``interrupted`` turn is not a completed turn, even when the + journal also carries an earlier ``completed`` event for another + turn of the same stream.""" + session_id = "issue6366_turn_journal_interrupted" + _write_turn_journal( + session_id, + "stream-a", + [("submitted", {}), ("completed", {})], + ) + _write_turn_journal( + session_id, + "stream-a", + [("submitted", {}), ("interrupted", {"reason": "cancelled"})], + ) + probe = _FakeSession( + pending="hello world", + messages=[], + stream_id="stream-a", + started_at=222.5, + session_id=session_id, + ) + assert _turn_journal_records_completion(probe, "stream-a") is False + + +def test_advances_past_pending_from_turn_journal_without_run_journal_terminal(): + """9/23 re-gate root case: the run journal never wrote a terminal + event (the stream died before ``done``/``stream_end`` reached it), + but the turn journal recorded the exact-stream ``completed`` event. + + The transcript alone proves the turn produced a genuine final + answer, and the turn journal proves the turn reached its terminal + state, so the recovery path must suppress the duplicate recovered + user row + journal clone + interruption marker. + """ + session_id = "issue6366_turn_journal_completed" + s = _FakeSession( + pending="hello world", + messages=[ + { + "role": "user", + "content": "hello world", + "timestamp": 222, + "_source": "webui", + "attachments": [{"name": "a.png"}], + "_active_turn_token": "stream-a:222.5", + }, + {"role": "assistant", "content": "answer", "timestamp": 223}, + ], + stream_id="stream-a", + started_at=222.5, + source="webui", + attachments=[{"name": "a.png"}], + session_id=session_id, + ) + # The run journal holds visible progress but NO terminal event. + _write_journal( + session_id, + "stream-a", + [ + ("token", {"text": "answer"}), + ], + ) + # The turn journal has the exact-stream completion event. + _write_turn_journal( + session_id, + "stream-a", + [("submitted", {}), ("completed", {})], + ) + assert _run_journal_terminal_state(s, "stream-a") != "completed" + assert _turn_journal_records_completion(s, "stream-a") is True + assert _transcript_already_advanced_past_pending(s) is True + + +def test_full_recovery_suppresses_duplicates_on_turn_journal_completion(tmp_path): + """Full-path 9/23 re-gate regression: the run journal never wrote a + terminal state, but the turn journal recorded the exact-stream + ``completed`` event. Recovery must clear the stale pending state + without appending the recovered user row, the journal clone, or the + interruption marker after the valid final answer.""" + session_id = "issue6366_full_recovery_turn_journal_completed" + stream_id = "stream-turn-journal-completed" + started_at = 1_700_002_100.5 + + session = Session( + session_id=session_id, + title="Completed per turn journal, silent run journal", + messages=[ + { + "role": "user", + "content": "summarise the diff", + "timestamp": int(started_at), + "_source": "webui", + "attachments": [], + "_active_turn_token": f"{stream_id}:{started_at:.17g}", + }, + {"role": "assistant", "content": "The diff adds a guard."}, + ], + context_messages=[ + {"role": "user", "content": "summarise the diff"}, + {"role": "assistant", "content": "The diff adds a guard."}, + ], + pending_user_message="summarise the diff", + pending_started_at=started_at, + pending_user_source="webui", + pending_attachments=[], + active_stream_id=stream_id, + ) + # The run journal has visible progress but no terminal event, so the + # line-3671 early return cannot fire and the suppression predicate + # is now the only gate that can keep the transcript clean. + _write_journal( + session_id, + stream_id, + [("token", {"text": "The diff adds a guard."})], + ) + # The turn journal recorded the exact-stream terminal completion. + _write_turn_journal( + session_id, + stream_id, + [("submitted", {}), ("completed", {})], + ) + before = json.dumps(session.messages, ensure_ascii=False) + + assert _apply_core_sync_or_error_marker( + session, + tmp_path / "missing-core-transcript.json", + stream_id_for_recheck=stream_id, + ) is True + + # Nothing was appended after the valid final answer. + assert json.dumps(session.messages, ensure_ascii=False) == before + assert not any( + message.get("_recovered") or message.get("_recovered_from_run_journal") + for message in session.messages + ) + assert not any(message.get("_error") for message in session.messages) + # The stale pending state was cleared. + assert session.pending_user_message is None + assert session.active_stream_id is None + assert session.pending_started_at is None + assert session.pending_user_source is None + assert session.pending_attachments == [] + + +def test_full_recovery_appends_rows_on_turn_journal_completion_loss(tmp_path): + """Mirror control: when neither the run journal nor the turn journal + has terminal completion evidence, the same transcript shape must + still recover — the suppression must not widen to cover a turn whose + completion can no longer be proven.""" + session_id = "issue6366_full_recovery_no_turn_journal_completion" + stream_id = "stream-no-turn-journal-completion" + started_at = 1_700_002_200.5 + + session = Session( + session_id=session_id, + title="Silent run journal, no turn journal completion", + messages=[ + { + "role": "user", + "content": "summarise the diff", + "timestamp": int(started_at), + "_source": "webui", + "attachments": [], + "_active_turn_token": f"{stream_id}:{started_at:.17g}", + }, + {"role": "assistant", "content": "The diff adds a guard."}, + ], + context_messages=[ + {"role": "user", "content": "summarise the diff"}, + ], + pending_user_message="summarise the diff", + pending_started_at=started_at, + pending_user_source="webui", + pending_attachments=[], + active_stream_id=stream_id, + ) + # Neither journal carries terminal completion evidence. + _write_journal( + session_id, + stream_id, + [("token", {"text": "The diff adds a guard."})], + ) + before = json.dumps(session.messages, ensure_ascii=False) + + assert _apply_core_sync_or_error_marker( + session, + tmp_path / "missing-core-transcript.json", + stream_id_for_recheck=stream_id, + ) is True + + # Recovery ran: the journal replay is visible again and the turn is + # marked interrupted instead of being silently declared finished. + assert json.dumps(session.messages, ensure_ascii=False) != before + assert any( + message.get("_recovered_from_run_journal") + and "The diff adds a guard." in str(message.get("content", "")) + for message in session.messages + ) + assert any( + message.get("_error") and message.get("type") == "interrupted" + for message in session.messages + ) + + +# ── full _apply_core_sync_or_error_marker recovery cases ──────────────── +# +# Neither guard rail above is reachable through the predicate in +# isolation: the suppression decision only takes effect where +# ``_apply_core_sync_or_error_marker`` calls it. These probes drive the +# real repair path end to end. + + +def _write_journal(session_id, stream_id, events): + for event_name, payload in events: + append_run_event(session_id, stream_id, event_name, payload) + + +def test_full_recovery_keeps_distinct_same_second_repeated_prompt(tmp_path): + """Two distinct streams send the same prompt inside one integer second. + + The earlier stream's turn is fully settled in the durable transcript. + The later stream is the pending one and was cancelled before it ever + checkpointed a user row. The repair must recover the pending prompt + instead of matching it against the earlier stream's row — which the + integer-second-truncating checkpoint matcher happily accepts — and + silently clearing the only durable copy of it. + """ + session_id = "issue6366_same_second_repeat" + stale_stream = "stream-stale" + pending_stream = "stream-pending" + # Both turns started inside the same integer second. + shared_second = 1_700_000_000 + + session = Session( + session_id=session_id, + title="Same-second repeated prompt", + messages=[ + { + "role": "user", + "content": "run the migration", + "timestamp": shared_second + 0.1, + "_active_turn_token": f"{stale_stream}:{shared_second + 0.1:.17g}", + }, + {"role": "assistant", "content": "Migration finished."}, + ], + context_messages=[ + {"role": "user", "content": "run the migration"}, + {"role": "assistant", "content": "Migration finished."}, + ], + pending_user_message="run the migration", + pending_started_at=shared_second + 0.6, + active_stream_id=pending_stream, + ) + _write_journal( + session_id, + pending_stream, + [("token", {"text": "Checking the migration status."})], + ) + + assert _apply_core_sync_or_error_marker( + session, + tmp_path / "missing-core-transcript.json", + stream_id_for_recheck=pending_stream, + ) is True + + # The pending prompt survives — recovery materialized it as a recovered + # user row instead of dropping it against the other stream's checkpoint. + recovered_users = [ + message + for message in session.messages + if message.get("_recovered") and message.get("content") == "run the migration" + ] + assert len(recovered_users) == 1 + # The journal's visible partial work was replayed, not suppressed. + assert any( + message.get("_recovered_from_run_journal") + and message.get("content") == "Checking the migration status." + for message in session.messages + ) + # And the interruption is marked so the user can see what happened. + assert any( + message.get("_error") and message.get("type") == "interrupted" + for message in session.messages + ) + # The earlier settled turn was left untouched. + assert session.messages[1]["content"] == "Migration finished." + + +def test_full_recovery_keeps_legacy_same_second_repeated_prompt(tmp_path): + """Legacy transcript with no tokens on either side: two identical + prompts inside one second must still be told apart, otherwise the + second prompt is matched against the first's checkpoint and dropped.""" + session_id = "issue6366_legacy_same_second_repeat" + stream_id = "stream-legacy-pending" + shared_second = 1_700_000_100 + + session = Session( + session_id=session_id, + title="Legacy same-second repeated prompt", + messages=[ + # The other stream's row: same text, same integer second, and + # distinguishable only by its different attachments. + { + "role": "user", + "content": "retry the upload", + "timestamp": shared_second, + "_source": "webui", + "attachments": [{"name": "first.png"}], + }, + {"role": "assistant", "content": "Upload done."}, + ], + context_messages=[ + {"role": "user", "content": "retry the upload"}, + {"role": "assistant", "content": "Upload done."}, + ], + pending_user_message="retry the upload", + pending_started_at=shared_second, + pending_user_source="webui", + pending_attachments=[{"name": "second.png"}], + active_stream_id=stream_id, + ) + _write_journal(session_id, stream_id, [("token", {"text": "Re-running upload."})]) + + assert _apply_core_sync_or_error_marker( + session, + tmp_path / "missing-core-transcript.json", + stream_id_for_recheck=stream_id, + ) is True + + recovered_users = [ + message + for message in session.messages + if message.get("_recovered") and message.get("content") == "retry the upload" + ] + assert len(recovered_users) == 1 + assert recovered_users[0].get("attachments") == [{"name": "second.png"}] + assert any( + message.get("_recovered_from_run_journal") + and message.get("content") == "Re-running upload." + for message in session.messages + ) + + +@pytest.mark.parametrize( + "tail_rows", + [ + pytest.param( + [{"role": "assistant", "content": ""}], + id="empty-assistant-tail", + ), + pytest.param( + [ + { + "role": "assistant", + "content": "I will inspect the file now.", + "tool_calls": [{"id": "call_1"}], + }, + {"role": "tool", "tool_call_id": "call_1", "content": "partial output"}, + ], + id="tool-call-only-tail", + ), + pytest.param( + [ + { + "role": "assistant", + "content": "Still working on the first step…", + "_partial": True, + }, + ], + id="interim-progress-tail", + ), + pytest.param( + [ + { + "role": "assistant", + "content": "[CONTEXT COMPACTION — REFERENCE ONLY] summary", + }, + ], + id="compaction-tail", + ), + ], +) +def test_full_recovery_runs_journal_replay_for_non_final_tails(tmp_path, tail_rows): + """An interrupted turn whose tail is not a genuine final answer must NOT + be treated as completed: journal replay has to run and the turn has to + be marked as interrupted. + """ + session_id = f"issue6366_tail_{tail_rows[-1].get('_partial', 'x')}" + session_id = "issue6366_non_final_tail" + stream_id = "stream-interrupted-tools" + started_at = 1_700_000_500.5 + + session = Session( + session_id=session_id, + title="Interrupted tool-running turn", + messages=[ + { + "role": "user", + "content": "inspect the failing test", + "timestamp": int(started_at), + "_active_turn_token": f"{stream_id}:{started_at:.17g}", + }, + *tail_rows, + ], + context_messages=[ + {"role": "user", "content": "inspect the failing test"}, + ], + pending_user_message="inspect the failing test", + pending_started_at=started_at, + active_stream_id=stream_id, + ) + _write_journal( + session_id, + stream_id, + [ + ("reasoning", {"text": "Reading the assertion first."}), + ("token", {"text": "The assertion never ran."}), + ( + "tool", + { + "name": "terminal", + "preview": "pytest tests/test_x.py", + "args": {"command": "pytest tests/test_x.py"}, + }, + ), + ], + ) + before = json.dumps(session.messages, ensure_ascii=False) + + assert _apply_core_sync_or_error_marker( + session, + tmp_path / "missing-core-transcript.json", + stream_id_for_recheck=stream_id, + ) is True + + # Journal replay ran: the visible partial output is back in the transcript. + assert any( + message.get("_recovered_from_run_journal") + and message.get("content") == "The assertion never ran." + for message in session.messages + ) + # The interrupted turn is marked, so the user sees what happened. + assert any( + message.get("_error") + and message.get("type") == "interrupted" + and "partial output above was recovered" + in str(message.get("content", "")) + for message in session.messages + ) + # Recovery actually appended rows rather than clearing pending state only. + assert json.dumps(session.messages, ensure_ascii=False) != before + assert session.messages[0]["content"] == "inspect the failing test" + + +def test_full_recovery_suppresses_duplicates_only_for_genuine_final_answer(tmp_path): + """Positive control for the suppression branch: when the pending turn's + own user row is followed by a genuine final answer AND the run journal + has the same-stream ``done`` event (durable terminal evidence), the + repair clears the stale pending state and appends nothing. + """ + session_id = "issue6366_suppression_control" + stream_id = "stream-stale-cancelled" + started_at = 1_700_000_900.25 + + session = Session( + session_id=session_id, + title="Already-answered pending turn", + messages=[ + { + "role": "user", + "content": "summarise the diff", + "timestamp": int(started_at), + "_active_turn_token": f"{stream_id}:{started_at:.17g}", + }, + {"role": "assistant", "content": "The diff adds a guard."}, + ], + context_messages=[ + {"role": "user", "content": "summarise the diff"}, + {"role": "assistant", "content": "The diff adds a guard."}, + ], + pending_user_message="summarise the diff", + pending_started_at=started_at, + active_stream_id=stream_id, + ) + # Same-stream durable terminal evidence — the run journal was marked + # ``done`` (terminal_state ``completed``) so the suppression gate + # opens and the transcript-only advance is accepted as terminal. + # No visible token events are written so the line-3648 path's + # ``_append_journaled_partial_output`` has nothing to append. + _write_journal(session_id, stream_id, [("done", {})]) + before = json.dumps(session.messages, ensure_ascii=False) + + assert _apply_core_sync_or_error_marker( + session, + tmp_path / "missing-core-transcript.json", + stream_id_for_recheck=stream_id, + ) is True + + # Nothing was appended after the valid final answer. + assert json.dumps(session.messages, ensure_ascii=False) == before + assert not any( + message.get("_recovered") or message.get("_recovered_from_run_journal") + for message in session.messages + ) + assert not any(message.get("_error") for message in session.messages) + # The stale pending state was cleared. + assert session.pending_user_message is None + assert session.active_stream_id is None + assert session.pending_started_at is None + assert session.pending_user_source is None + assert session.pending_attachments == [] + + +def test_full_recovery_runs_journal_replay_for_unflagged_interim_prose(tmp_path): + """9/22 re-gate regression, end-to-end: the durable transcript + already advanced past the pending turn into an unflagged interim + prose row followed by tool rows, but the run journal was never + marked ``done``. Before the fix, + ``_transcript_already_advanced_past_pending`` returned True at the + first visible assistant row, ``_apply_core_sync_or_error_marker`` + cleared the pending state, and the user saw a half-finished turn + with the journal's partial output and the interruption marker + both suppressed. The fix requires durable same-stream terminal + evidence (``done`` / ``stream_end`` → ``completed``) before + suppressing recovery, so the run journal is replayed and the + ``interrupted`` marker is appended for this unflagged-prose + shape. + """ + session_id = "issue6366_unflagged_interim_prose" + stream_id = "stream-unflagged-interim" + started_at = 1_700_001_200.75 + + session = Session( + session_id=session_id, + title="Unflagged interim prose followed by tool rows", + messages=[ + { + "role": "user", + "content": "check the failing test", + "timestamp": int(started_at), + "_active_turn_token": f"{stream_id}:{started_at:.17g}", + }, + # The bug shape: an unflagged assistant row that the + # transcript heuristic accepts as a "final answer" because + # it carries visible text and no _partial / _error / + # _recovered / _recovered_from_run_journal hint. The + # stream had just started talking about what it was about + # to do, then ran tools. The tool_calls that produced the + # following tool rows live on a later row in the original + # stream — the WebUI process died before the tool-call + # assistant row was committed, so the transcript currently + # shows just the pre-final prose followed by tool result + # rows. + { + "role": "assistant", + "content": "Let me check the logs first.", + }, + {"role": "tool", "tool_call_id": "call_1", "content": "log tail"}, + {"role": "tool", "tool_call_id": "call_2", "content": "log grep"}, + ], + context_messages=[ + {"role": "user", "content": "check the failing test"}, + ], + pending_user_message="check the failing test", + pending_started_at=started_at, + active_stream_id=stream_id, + ) + # The run journal has visible progress — the unflagged prose was + # emitted as a token, the run was checkpointed as interim_assistant, + # and one of the tool rows finished. Crucially there is no + # ``done`` / ``stream_end`` event: the stream died mid-tool, so + # the journal has no durable terminal evidence. + _write_journal( + session_id, stream_id, + [ + ("reasoning", {"text": "Reading the assertion first."}), + ("token", {"text": "Let me check the logs first."}), + ("interim_assistant", {"text": "checkpoint"}), + ( + "tool", + { + "name": "terminal", + "tid": "call_1", + "preview": "tail -n 50", + "args": {"command": "tail -n 50"}, + }, + ), + ( + "tool_complete", + { + "name": "terminal", + "tid": "call_1", + "preview": "log tail", + "duration": 0.05, + }, + ), + ], + ) + before = json.dumps(session.messages, ensure_ascii=False) + + assert _apply_core_sync_or_error_marker( + session, + tmp_path / "missing-core-transcript.json", + stream_id_for_recheck=stream_id, + ) is True + + # Recovery actually appended rows rather than clearing pending + # state only. + assert json.dumps(session.messages, ensure_ascii=False) != before + # The journal's visible partial output is back in the transcript: + # the "Let me check the logs first." prose that was emitted over + # SSE before the WebUI process died is now visible again. + assert any( + message.get("_recovered_from_run_journal") + and "Let me check the logs first." in str(message.get("content", "")) + for message in session.messages + ) + # And the interrupted turn is marked, so the user can see the + # stream was cut off rather than presented with a half-finished + # turn that looks complete. + interrupted_marker = next( + ( + message + for message in session.messages + if message.get("_error") and message.get("type") == "interrupted" + ), + None, + ) + assert interrupted_marker is not None + assert "partial output above was recovered" in str( + interrupted_marker.get("content", ""), + ) + # Pending state was cleared (recovery always clears it) and no + # duplicate user row was appended ahead of the journal replay — + # the original user row is still the first message. + assert session.messages[0]["content"] == "check the failing test" + assert session.pending_user_message is None + assert session.active_stream_id is None diff --git a/tests/test_issue7359_media_backtick_strip.py b/tests/test_issue7359_media_backtick_strip.py new file mode 100644 index 00000000..09b976ff --- /dev/null +++ b/tests/test_issue7359_media_backtick_strip.py @@ -0,0 +1,313 @@ +"""Regression tests for issue #7359 — MEDIA: token regex must not capture trailing backtick. + +Two-pass scan (#7680 re-gate, 9/22): + +- `` `MEDIA:path` `` (backtick-wrapped, inline-code form): the wrapping + backticks are stripped by a pre-pass before the bare class scans the + text. This makes the closing backtick disappear as a wrapper instead + of being captured as part of the path. +- ``MEDIA:/path...`` (bare, no surrounding backticks): the capture class + no longer lists the backtick in the exclusion set, so a filename + that legally contains a backtick (``report`final.png``) is captured + in full instead of being truncated at the first backtick. + +The pre-fix (round-1) PR added the backtick to the bare exclusion set, +which traded one bug for another: it dropped the closing wrapper on +the inline-code form but truncated any bare path that happened to +contain a backtick in the filename. The two-pass scan keeps the +original permissive bare class and adds an explicit pre-pass for the +wrapped form, so both cases round-trip correctly. +""" + +import re +import subprocess +from pathlib import Path + +import pytest + +REPO_ROOT = Path(__file__).resolve().parents[1] + +# Source-shape: each site's expected bare-token capture literal. The +# bare class no longer lists the backtick in the exclusion set; a +# backtick inside the filename (legal on POSIX and Windows) is +# captured in full. The wrapped-form pre-pass below strips the +# inline-code backticks before the bare scan. +JS_SITES: dict[str, list[str]] = { + "static/ui.js": [ + r"/MEDIA:([^\s\)\]]+)/g", # linkifier + r"/MEDIA:[^\s]+/g", # TTS flattener (no paren/bracket exclusion) + ], + "static/messages.js": [ + r"/^MEDIA:([^\s\)\]]+)$/", # streaming chunk finalize + r"/MEDIA:([^\s\)\]]+)/g", # streaming combined walk + r"/MEDIA:[^\s\)\]]*$/", # streaming partial tail + ], +} +PY_SITES: dict[str, str] = { + "api/routes.py": r"MEDIA:([^\s\)\]]+)", + "api/media_snapshots.py": r"MEDIA:([^\s\)\]]+)", +} + +# #7680 re-gate (9/22): every site that scans bare MEDIA: tokens also +# runs the wrapped-form pre-pass (`` `MEDIA:path` `` → ``MEDIA:path``) +# so the closing backtick of an inline-code form is consumed before the +# bare scan. +JS_BACKTICK_PREPASS: dict[str, str] = { + "static/ui.js": r"/`MEDIA:([^`\s]+)`/g", + "static/messages.js": r"/`MEDIA:([^`\s]+)`/g", +} +PY_BACKTICK_PREPASS: dict[str, str] = { + "api/routes.py": r"`MEDIA:([^`\s]+)`", + "api/media_snapshots.py": r"`MEDIA:([^`\s]+)`", +} + + +def _read(rel: str) -> str: + return (REPO_ROOT / rel).read_text(encoding="utf-8") + + +# --------------------------------------------------------------------------- +# Source-shape: every site must carry the bare-token class + the pre-pass +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + ("relpath", "needle"), + [ + (relpath, needle) + for relpath, needles in JS_SITES.items() + for needle in needles + ], +) +def test_js_site_carries_bare_token_class(relpath: str, needle: str) -> None: + """Every JS capture site carries the bare-token class literal.""" + src = _read(relpath) + assert needle in src, ( + f"{relpath} is missing the bare-token capture literal: {needle!r}" + ) + + +@pytest.mark.parametrize( + ("relpath", "needle"), + [ + (relpath, JS_BACKTICK_PREPASS[relpath]) + for relpath in JS_BACKTICK_PREPASS + ], +) +def test_js_site_carries_wrapped_prepass(relpath: str, needle: str) -> None: + """#7680 re-gate: every JS site that scans bare MEDIA: tokens also + runs the wrapped-form pre-pass so the inline-code closing backtick + is eaten before the bare scan.""" + src = _read(relpath) + assert needle in src, ( + f"{relpath} is missing the wrapped-form pre-pass literal: {needle!r}" + ) + + +@pytest.mark.parametrize( + ("relpath", "needle"), + [(relpath, PY_SITES[relpath]) for relpath in PY_SITES], +) +def test_py_site_carries_bare_token_class(relpath: str, needle: str) -> None: + src = _read(relpath) + assert needle in src, ( + f"{relpath} is missing the bare-token capture literal: {needle!r}" + ) + + +@pytest.mark.parametrize( + ("relpath", "needle"), + [(relpath, PY_BACKTICK_PREPASS[relpath]) for relpath in PY_BACKTICK_PREPASS], +) +def test_py_site_carries_wrapped_prepass(relpath: str, needle: str) -> None: + src = _read(relpath) + assert needle in src, ( + f"{relpath} is missing the wrapped-form pre-pass literal: {needle!r}" + ) + + +def test_all_seven_sites_covered() -> None: + """Pin: 5 JS sites + 2 Python sites = 7 capture sites, all guarded.""" + js_total = sum(len(v) for v in JS_SITES.values()) + assert js_total == 5, ( + f"expected 5 JS capture sites, got {js_total}; this count is what the " + "review's '7 capture sites' was counting (1 linkifier + 1 TTS + 3 messages.js)" + ) + assert len(PY_SITES) == 2 + + +@pytest.fixture(scope="module") +def _media_token_re() -> re.Pattern[str]: + """Compile the live ``_MEDIA_TOKEN_RE`` from api/routes.py (plus the + pre-pass) so behaviour tests exercise the real source.""" + routes_text = _read("api/routes.py") + m = re.search( + r'_MEDIA_TOKEN_RE = re\.compile\(r"([^"]+)"\)', routes_text, re.M + ) + assert m, "_MEDIA_TOKEN_RE compile() not found in api/routes.py" + bare = re.compile(m.group(1)) + pre = re.compile(r"`MEDIA:([^`\s]+)`") + return _TwoPass(bare, pre) + + +@pytest.fixture(scope="module") +def _media_snap_re() -> re.Pattern[str]: + """Compile the live ``media_re`` from api/media_snapshots.py (plus the + pre-pass) so behaviour tests exercise the real source.""" + snap_text = _read("api/media_snapshots.py") + m = re.search( + r'media_re = _re\.compile\(r"([^"]+)"\)', snap_text, re.M + ) + assert m, "media_re compile() not found in api/media_snapshots.py" + bare = re.compile(m.group(1)) + pre = re.compile(r"`MEDIA:([^`\s]+)`") + return _TwoPass(bare, pre) + + +class _TwoPass: + """Compose the wrapped-form pre-pass with the bare-token class so the + behaviour tests below exercise the live source exactly as it runs.""" + + def __init__(self, bare: re.Pattern[str], prepass: re.Pattern[str]): + self._bare = bare + self._pre = prepass + + def findall(self, text: str) -> list[str]: + text = self._pre.sub(lambda m: f"MEDIA:{m.group(1)}", text) + return self._bare.findall(text) + + +def test_backend_allowlist_strips_backtick(_media_token_re) -> None: + """``_MEDIA_TOKEN_RE`` (with the pre-pass) must not include the closing + backtick in the captured path.""" + captured = _media_token_re.findall("get `MEDIA:/home/kim/skills.zip` now") + assert captured == ["/home/kim/skills.zip"], captured + assert all(not c.endswith("`") for c in captured) + + +def test_backend_allowlist_bare_token_still_captures(_media_token_re) -> None: + """Backward-compat: bare ``MEDIA:/path`` (no backtick) still matches.""" + captured = _media_token_re.findall( + "MEDIA:/home/kim/a.zip and MEDIA:/home/kim/b.tar" + ) + assert captured == ["/home/kim/a.zip", "/home/kim/b.tar"], captured + + +def test_backend_allowlist_bare_path_with_backtick_in_filename( + _media_token_re, +) -> None: + """#7680 finding 1: a bare ``MEDIA:/tmp/report`final.png`` (backtick + inside the filename, no surrounding wrapper) must capture the full + path. POSIX and Windows both allow backticks in filenames, so + truncating at the first backtick silently bricks the request.""" + captured = _media_token_re.findall("see MEDIA:/tmp/report`final.png now") + assert captured == ["/tmp/report`final.png"], captured + assert not any(c.endswith("`") and c != "/tmp/report`final.png" for c in captured), captured + + +def test_backend_allowlist_rejects_trailing_paren(_media_token_re) -> None: + """Pre-existing behaviour: ``)`` is still excluded.""" + captured = _media_token_re.findall("see [file](MEDIA:/home/kim/a.zip) here") + assert all(not c.endswith(")") for c in captured), captured + + +def test_snapshot_capture_strips_backtick(_media_snap_re) -> None: + """``media_re`` (with the pre-pass) must not include the closing + backtick in the captured path.""" + captured = _media_snap_re.findall( + "assistant: `MEDIA:/home/kim/shot.png` and MEDIA:/home/kim/a.zip" + ) + assert captured == ["/home/kim/shot.png", "/home/kim/a.zip"], captured + assert all(not c.endswith("`") for c in captured) + + +def test_snapshot_capture_bare_path_with_backtick_in_filename( + _media_snap_re, +) -> None: + """#7680 finding 1: same bare-with-backtick case for the snapshot + pipeline. Without the pre-pass + bare class change, the snapshot + store would never see this path because the auth predicate and + the capture predicate must agree on the same set of references.""" + captured = _media_snap_re.findall("MEDIA:/home/kim/shot`final.png tail") + assert captured == ["/home/kim/shot`final.png"], captured + + +def test_snapshot_capture_bare_token_still_captures(_media_snap_re) -> None: + """Backward-compat: bare ``MEDIA:/path`` still matches.""" + captured = _media_snap_re.findall( + "MEDIA:/home/kim/a.zip and MEDIA:/home/kim/b.zip" + ) + assert captured == ["/home/kim/a.zip", "/home/kim/b.zip"], captured + + +# --------------------------------------------------------------------------- +# Revert-sensitivity: the pre-fix class really would have truncated +# the bare-with-backtick case, and the round-1 fix would have left the +# wrapped form broken in a different way. +# --------------------------------------------------------------------------- + + +def test_round1_buggy_class_truncated_bare_path_with_backtick() -> None: + """The buggy (round-1) class greedily truncated at the first backtick + in the path, so ``MEDIA:/tmp/report`final.png`` was captured as + ``/tmp/report``. This is the regression we are guarding against.""" + buggy = re.compile(r"MEDIA:([^\s\)\]\`]+)") + assert buggy.findall("MEDIA:/tmp/report`final.png") == ["/tmp/report"], ( + "round-1 class is no longer buggy; if you are intentionally reverting " + "to it, also remove the corresponding `test_post_fix_class_keeps_" + "bare_path` and update the source-shape guard." + ) + + +def test_round1_buggy_class_dropped_wrapped_closing_backtick() -> None: + """The round-1 class was actually correct for the wrapped form: it + dropped the closing backtick that the round-0 class greedily + included. Document the behaviour so a future revert picks the + correct replacement.""" + round1 = re.compile(r"MEDIA:([^\s\)\]\`]+)") + assert round1.findall("`MEDIA:/home/kim/a.zip`") == ["/home/kim/a.zip"] + + +def test_round1_two_pass_compose_keeps_bare_path_intact() -> None: + """The full #7680 fix is the two-pass compose: pre-pass strips the + wrapped form, then the bare class (no backtick in the exclusion + set) captures the full path. Verify the composition on both + cases so a future refactor that drops either half is caught.""" + pre = re.compile(r"`MEDIA:([^`\s]+)`") + bare = re.compile(r"MEDIA:([^\s\)\]]+)") + + def findall_composed(text: str) -> list[str]: + text = pre.sub(lambda m: f"MEDIA:{m.group(1)}", text) + return bare.findall(text) + + # bare-with-backtick: pre-pass doesn't touch it, bare captures full path + assert findall_composed("MEDIA:/tmp/report`final.png") == [ + "/tmp/report`final.png" + ] + # wrapped: pre-pass strips the backticks, bare captures the path + assert findall_composed("`MEDIA:/tmp/a.zip`") == ["/tmp/a.zip"] + # mixed + assert findall_composed( + "see `MEDIA:/home/kim/shot.png` and MEDIA:/tmp/r`f.png tail" + ) == ["/home/kim/shot.png", "/tmp/r`f.png"] + # plain bare: unchanged + assert findall_composed("MEDIA:/home/kim/a.zip") == ["/home/kim/a.zip"] + + +# --------------------------------------------------------------------------- +# node --check (so a JS syntax error in the modified source fails the suite) +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize("relpath", sorted(JS_SITES.keys())) +def test_js_file_parses(relpath: str) -> None: + """``node --check`` on every modified JS file.""" + proc = subprocess.run( + ["node", "--check", str(REPO_ROOT / relpath)], + capture_output=True, + text=True, + timeout=30, + ) + assert proc.returncode == 0, ( + f"{relpath} failed `node --check`:\n{proc.stderr}" + ) diff --git a/tests/test_issue7421_resolve_cap_config.py b/tests/test_issue7421_resolve_cap_config.py new file mode 100644 index 00000000..6cad1970 --- /dev/null +++ b/tests/test_issue7421_resolve_cap_config.py @@ -0,0 +1,297 @@ +"""Regression coverage for issue #7421: configurable +``HERMES_WEBUI_MAX_SESSION_RESOLVE`` cap on the heavy +full-transcript resolve semaphore. + +The cap is hardcoded to ``2`` in +``api.models.py::_FULL_SESSION_RESOLVE_MAX_CONCURRENT``. The +maintainer's diagnosis is that the literal 2 is too low for +high-concurrency deployments where several parallel active +sessions all need a full-transcript resolve, and that the +deeper per-read cost is tracked separately in #7310. This PR +is the self-contained first step: read the cap from the env +var so operators can raise it without code changes, while +preserving the previous default of 2 and bounding the upper +end against typos like ``=999999``. + +#7656 re-gate (9/22 23:18): the previous round used a private +``_import_models_with_env`` helper that did ``sys.modules.pop`` +on ``api.models`` and re-imported to recompute the module-level +cap constant. Modules already imported (``api.routes``, +``api.streaming``, ...) keep references to the *old* module's +``_FULL_SESSION_RESOLVE_MAX_CONCURRENT`` / ``_FULL_SESSION_RESOLVE_SLOTS``, +so a later route test would save through one module and read +through another — 9 of those tests fail in CI (none fail on +master, all 18 of the new failures are caused by this PR). + +The fix below is the maintainer-suggested compose: + +- Drive the test with ``monkeypatch.setenv`` so the env var is + restored at fixture teardown — no leak to other test files +- Test ``_read_max_session_resolve_concurrent()`` directly + (the function is already pure: only reads ``os.getenv``) so + no module reload is needed +- For the single test that needs the module-level constant + (``_FULL_SESSION_RESOLVE_MAX_CONCURRENT``), verify it equals + the helper's return value at import time — that is the real + production contract, and it is cheap to assert because the + helper itself only reads the env var +""" +from __future__ import annotations + +import threading +from pathlib import Path + +import pytest + +import api.models as models + +ROOT = Path(__file__).resolve().parents[1] + + +@pytest.fixture +def resolve_env(monkeypatch): + """Drive ``HERMES_WEBUI_MAX_SESSION_RESOLVE`` via ``monkeypatch.setenv`` + so the env var is restored at fixture teardown — no leak to other + test files, no module reload required. + + The helper ``models._read_max_session_resolve_concurrent()`` is pure + (only ``os.getenv``) and reads the live env, so this fixture is the + entire test harness. The previous round's ``_import_models_with_env`` + used ``sys.modules.pop`` + ``importlib.import_module`` to recompute + the module-level cap constant against a freshly-set env, but that + left the original module unreferenced in ``sys.modules`` while + ``api.routes`` and friends still held the *old* module object — + root cause of the cross-suite pollution the maintainer reproduced. + """ + def set_resolve_env(value): + if value is None: + monkeypatch.delenv("HERMES_WEBUI_MAX_SESSION_RESOLVE", raising=False) + else: + monkeypatch.setenv("HERMES_WEBUI_MAX_SESSION_RESOLVE", str(value)) + return set_resolve_env + + +# ── default behavior (env unset) ────────────────────────────────────────────── + + +def test_default_cap_is_two_when_env_unset(resolve_env): + """Backward compatibility: an installation that does not set + the env var keeps the previous behavior. The bounded + semaphore is constructed with capacity 2, so 3+ parallel + full-resolves queue behind two slots as before.""" + resolve_env(None) + assert models._read_max_session_resolve_concurrent() == 2, ( + "unset env must keep the previous hardcoded default of 2 " + "so existing single-instance deployments are unchanged" + ) + + +def test_default_cap_is_two_when_env_empty_string(resolve_env): + """A user who sets the env var to an explicit empty string + (e.g. from a misconfigured .env file) is the same as unset. + The helper strips whitespace first; ``""`` is treated as no + value at all.""" + resolve_env("") + assert models._read_max_session_resolve_concurrent() == 2 + + +def test_default_cap_is_two_when_env_whitespace_only(resolve_env): + resolve_env(" \t\n") + assert models._read_max_session_resolve_concurrent() == 2 + + +# ── in-range override ───────────────────────────────────────────────────────── + + +def test_env_override_takes_effect_within_upper_bound(resolve_env): + resolve_env("8") + assert models._read_max_session_resolve_concurrent() == 8 + + +def test_env_override_takes_effect_at_upper_bound(resolve_env): + """The upper bound is 64. An operator who wants the full + ceiling must be able to reach it.""" + resolve_env("64") + assert models._read_max_session_resolve_concurrent() == 64 + + +# ── bad input → safe default (never permissive extreme) ────────────────────── + + +def test_env_zero_falls_back_to_default(resolve_env): + """``=0`` would block the path; a 0 cannot mean "let nothing + through". Fall back to 2 (the previous hardcoded value).""" + resolve_env("0") + assert models._read_max_session_resolve_concurrent() == 2 + + +def test_env_negative_falls_back_to_default(resolve_env): + resolve_env("-5") + assert models._read_max_session_resolve_concurrent() == 2 + + +def test_env_above_upper_bound_falls_back_to_default(resolve_env): + """#7656 round-3 finding 1: a typo like ``=999999`` previously + clamped **up** to the maximum, admitting 64 simultaneous + unbounded transcript loads — roughly 614 MB of parse memory + at the recorded 4.27 MB / parse. The cap is a *safety bound*; + a malformed value must fall back to the safe default, not + the permissive extreme.""" + resolve_env("999999") + assert models._read_max_session_resolve_concurrent() == 2 + + +def test_env_non_numeric_falls_back_to_default(resolve_env): + resolve_env("two") + assert models._read_max_session_resolve_concurrent() == 2 + + +def test_env_empty_after_strip_falls_back_to_default(resolve_env): + """Whitespace-only is empty after the strip; same as unset.""" + resolve_env("\n\t ") + assert models._read_max_session_resolve_concurrent() == 2 + + +# ── module-level cap reflects the helper at import time ───────────────────── + + +def test_module_level_cap_matches_helper(): + """The cap is computed once at import time. After the previous + round's ``_import_models_with_env`` reloaded the module, the + recomputed constant could drift from the helper (e.g. if a + test set the env, reloaded, then never cleaned up). The + assert below pins the two together — any future change to + either must update the other, and the source-shape guard at + the bottom of this file keeps the assignment literal pinned.""" + assert models._FULL_SESSION_RESOLVE_MAX_CONCURRENT == ( + models._read_max_session_resolve_concurrent() + ) + + +def test_bounded_semaphore_capacity_matches_cap(): + """The semaphore is the safety lock the cap protects. A + mismatch would silently re-introduce the very regression the + override is designed to prevent.""" + sem = models._FULL_SESSION_RESOLVE_SLOTS + assert isinstance(sem, threading.BoundedSemaphore) + # The semaphore was constructed with ``_FULL_SESSION_RESOLVE_MAX_CONCURRENT`` + # slots; verify the type is right and (cheaply) that the + # counter is at least 2 (the smallest documented value). + assert sem._initial_value == models._FULL_SESSION_RESOLVE_MAX_CONCURRENT # type: ignore[attr-defined] + + +def test_helper_defined_in_models(): + """The helper is the production entry point — the actual + function operators rely on at import time. Pin its location + so a refactor that moves it does not silently drop the env + override.""" + src = (ROOT / "api" / "models.py").read_text(encoding="utf-8") + assert "def _read_max_session_resolve_concurrent" in src, ( + "the env-override helper must live in api.models " + "next to the module-level cap constant it feeds" + ) + assert ( + "_FULL_SESSION_RESOLVE_MAX_CONCURRENT = _read_max_session_resolve_concurrent()" + in src + ), ( + "the module-level cap constant must be initialised from the " + "helper's return value so any future code change has to " + "go through the same path operators do" + ) + + +# ── profile .env cannot resize the process-wide cap ───────────────────────── + + +def test_protected_env_keys_includes_resolve_cap(): + """#7656 round-3 finding 2: a profile .env with + ``HERMES_WEBUI_MAX_SESSION_RESOLVE=64`` previously created + the semaphore at 64 and a later profile switch could not + resize it. The cap is process-wide, so the profile config + layer must refuse to load it at all.""" + from api.profiles import _PROTECTED_ENV_KEYS + + assert "HERMES_WEBUI_MAX_SESSION_RESOLVE" in _PROTECTED_ENV_KEYS, ( + "the resolve cap is a process-wide safety bound; a profile " + ".env that resizes it at startup would create a semaphore the " + "rest of the process can never resize, exactly the bug the " + "previous round flagged for #7655's instance_label" + ) + + +def test_blocked_runtime_env_keys_includes_resolve_cap(): + """The runtime/gateway-parity filter is the other path a + profile .env could leak through. Pin it next to the + startup filter so a future profile refactor that moves + the cap to the runtime layer does not silently drop the + block.""" + from api.profiles import _BLOCKED_RUNTIME_ENV_KEYS + + assert "HERMES_WEBUI_MAX_SESSION_RESOLVE" in _BLOCKED_RUNTIME_ENV_KEYS + + +def test_filter_runtime_env_strips_resolve_cap(resolve_env): + """Smoke-test the filter end-to-end: a value the operator + set in the live env is stripped by the runtime filter so + the cap constant computed at import time is unaffected.""" + from api.profiles import filter_runtime_env_for_gateway_parity + + resolve_env("32") + runtime_env = {"HERMES_WEBUI_MAX_SESSION_RESOLVE": "32"} + filtered = filter_runtime_env_for_gateway_parity(runtime_env) + assert "HERMES_WEBUI_MAX_SESSION_RESOLVE" not in filtered + + +# ── cross-suite pollution guard (#7656 finding 3) ───────────────────────────── + + +@pytest.mark.parametrize("forbidden", ["sys.modules.pop", "importlib.import_module", "os.environ[", "os.environ.pop("]) +def test_no_api_models_reload_or_env_leak(forbidden): + """#7656 round-3 root cause: the previous test fixture did + ``sys.modules.pop("api.models")`` + ``importlib.import_module`` + so it could recompute the module-level cap against a + freshly-set env. That left the original module unreferenced + in ``sys.modules`` while already-imported modules + (``api.routes``, ``api.streaming``, ...) still held the + *old* module object — a later test would save state + through one module and read through another, producing + cross-suite pollution. Pin the fix: this file must not + contain any ``sys.modules.pop`` / ``importlib.import_module`` + of ``api.models``, nor direct ``os.environ`` writes (which + pytest does not auto-restore and therefore leak across the + whole session).""" + src = (ROOT / "tests" / "test_issue7421_resolve_cap_config.py").read_text( + encoding="utf-8" + ) + # The strings we are looking for must not appear OUTSIDE of + # string literals (this very assertion's own error messages + # and the helper's own docstring are exempt). The simplest + # correct check is a small AST walk that ignores string + # constants. + import ast + tree = ast.parse(src, filename=str(ROOT / "tests" / "test_issue7421_resolve_cap_config.py")) + for node in ast.walk(tree): + if isinstance(node, ast.Constant) and isinstance(node.value, str): + if forbidden in node.value: + return # the only occurrence of the forbidden token is a string literal + elif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): + name = node.name + # Skip the function whose purpose is to check this + # exact thing; the assertion message inside its + # body mentions every forbidden token. + if name == "test_no_api_models_reload_or_env_leak": + continue + for sub in ast.walk(node): + if ( + isinstance(sub, ast.Constant) + and isinstance(sub.value, str) + and forbidden in sub.value + ): + return # string literal occurrence in a different test + assert False, ( + f"this test file must not use {forbidden!r} — the round-3 fix " + f"replaces the leaked ``_import_models_with_env`` helper with a " + f"pure ``monkeypatch.setenv`` fixture. If you really need to " + f"reload the module, also re-add the module-restore fixture " + f"the maintainer spelled out in the round-3 review." + ) diff --git a/tests/test_issue7600_oob_display_writeback.py b/tests/test_issue7600_oob_display_writeback.py new file mode 100644 index 00000000..3edb7183 --- /dev/null +++ b/tests/test_issue7600_oob_display_writeback.py @@ -0,0 +1,185 @@ +"""Settled writeback must extract inner steer text from typed steer rows (#7600). + +A mid-turn ``/steer`` is emitted as a standalone typed user row +(``role: 'user'``, ``display_kind: 'steer'``). The surrounding +``[OUT-OF-BAND USER MESSAGE ...] ... [/OUT-OF-BAND USER MESSAGE]`` block is +transport-level control data. Settlement unwraps the single validated frame +in place, preserving the user's authored steer instruction while dropping +the wrapper. + +Legacy tool rows and rows where markers are multiple, nested, incomplete, +or contain ambiguous delimiters are preserved byte-for-byte. +""" +from __future__ import annotations + +import copy +import json + +from api.models import Session +from api.streaming import _settle_result_messages, _unwrap_steer_row_oob_marker +from api import streaming as _streaming + +PROMPT = "please run the smoke checks" + +OOB_OPEN = ( + "[OUT-OF-BAND USER MESSAGE — a direct message from the user, delivered once " + "at this position; not tool output and not a new delivery when replayed from " + "conversation history]" +) +OOB_CLOSE = "[/OUT-OF-BAND USER MESSAGE]" +OOB_BLOCK = f"{OOB_OPEN}\nsteer: use the staging bucket this time\n{OOB_CLOSE}" +OOB_VARIANT = f"[OUT-OF-BAND USER MESSAGE]steer: also bump the timeout{OOB_CLOSE}" + + +def _contains_oob(value) -> bool: + return "OUT-OF-BAND USER MESSAGE" in json.dumps(value) + + +def _session_with_prior_turn() -> Session: + """Prior turn already settled: user -> assistant(tool_call) -> tool -> answer.""" + display = [ + {"role": "user", "content": "deploy the app", "timestamp": 1788439000}, + { + "role": "assistant", + "content": "", + "tool_calls": [{"id": "call-1"}], + "timestamp": 1788439005, + }, + { + "role": "tool", + "tool_call_id": "call-1", + "content": "deploy finished: revision 41", + "timestamp": 1788439010, + }, + {"role": "assistant", "content": "Deploy done (revision 41).", "timestamp": 1788439015}, + ] + session = Session(session_id="7" * 12, title="steer writeback", messages=copy.deepcopy(display)) + session.context_messages = copy.deepcopy(display) + return session + + +def _settle_turn_with_steer(session, monkeypatch, *, tool_content: str = "checks passed", steer_rows=None): + """Settle one turn with an optional steer row emitted by the Agent.""" + monkeypatch.setattr( + _streaming, "_annotate_media_snapshots_for_settled_messages", lambda messages: None + ) + previous = list(session.messages) + previous_context = list(session.context_messages) + ts = 1788440000 + steers = list(steer_rows or []) + result = copy.deepcopy(previous_context) + [ + {"role": "user", "content": PROMPT, "timestamp": ts}, + { + "role": "assistant", + "content": "", + "tool_calls": [{"id": "call-2"}], + "timestamp": ts + 1, + }, + { + "role": "tool", + "tool_call_id": "call-2", + "content": tool_content, + "timestamp": ts + 2, + }, + *steers, + {"role": "assistant", "content": "Smoke checks passed.", "timestamp": ts + 3}, + ] + _settle_result_messages(session, previous, previous_context, result, PROMPT, "webui", None) + return previous + + +def test_settle_unwraps_steer_row_in_place_preserving_user_text(monkeypatch): + """A typed steer row is unwrapped in place: inner text preserved, wrapper gone.""" + session = _session_with_prior_turn() + steer_row = { + "role": "user", + "display_kind": "steer", + "content": OOB_BLOCK, + "timestamp": 1788440002.5, + } + _settle_turn_with_steer(session, monkeypatch, steer_rows=[steer_row]) + + # The outer wrapper is dropped from display and context + assert not _contains_oob(session.messages) + assert not _contains_oob(session.context_messages) + + # The user's inner steer text is preserved exactly + steer_contents = [ + m.get("content") + for m in session.messages + if isinstance(m, dict) and m.get("display_kind") == "steer" + ] + assert steer_contents == ["steer: use the staging bucket this time"] + + # In-place mutation preserved the dictionary object identity + assert steer_row["content"] == "steer: use the staging bucket this time" + + +def test_settle_unwraps_list_based_content_steer_row(monkeypatch): + """List-based content parts on a steer row are unwrapped too.""" + session = _session_with_prior_turn() + steer_row = { + "role": "user", + "display_kind": "steer", + "content": [{"type": "text", "text": OOB_VARIANT}], + "timestamp": 1788440002.5, + } + _settle_turn_with_steer(session, monkeypatch, steer_rows=[steer_row]) + + assert not _contains_oob(session.messages) + assert not _contains_oob(session.context_messages) + assert steer_row["content"] == [{"type": "text", "text": "steer: also bump the timeout"}] + + +def test_settle_leaves_legacy_tool_rows_alone(monkeypatch): + """Legacy tool rows carrying raw markers are left alone (not truncated/deleted).""" + session = _session_with_prior_turn() + tool_text = f"checks passed\n\n{OOB_BLOCK}" + _settle_turn_with_steer(session, monkeypatch, tool_content=tool_text) + + tool_bodies = [ + m.get("content") for m in session.messages if isinstance(m, dict) and m.get("role") == "tool" + ] + assert tool_text in tool_bodies + + +def test_settle_preserves_malformed_nested_and_multiple_blocks_byte_for_byte(): + """Nested markers, multiple frames, and incomplete markers degrade gracefully.""" + nested = f"{OOB_OPEN}\nhello {OOB_VARIANT} world\n{OOB_CLOSE}" + multiple = f"{OOB_VARIANT} and {OOB_BLOCK}" + incomplete = "[OUT-OF-BAND USER MESSAGE — truncated" + + for malformed in (nested, multiple, incomplete): + steer_row = { + "role": "user", + "display_kind": "steer", + "content": malformed, + "timestamp": 1788440002.5, + } + _unwrap_steer_row_oob_marker(steer_row) + assert steer_row["content"] == malformed, f"malformed content was corrupted: {malformed!r}" + + +def test_settle_keeps_normal_user_pasted_marker_intact(monkeypatch): + """A normal user row that quotes a complete marker survives byte-for-byte.""" + pasted = ( + "our bot log shows this block, is that normal?\n" + f"{OOB_BLOCK}\n" + "the docs say the gateway adds it" + ) + session = _session_with_prior_turn() + session.messages[0]["content"] = pasted + session.context_messages = copy.deepcopy(session.messages) + + steer_row = { + "role": "user", + "display_kind": "steer", + "content": OOB_BLOCK, + "timestamp": 1788440002.5, + } + _settle_turn_with_steer(session, monkeypatch, steer_rows=[steer_row]) + + # The normal user row still carries its quoted block intact + assert session.messages[0]["content"] == pasted + # The steer row had its wrapper unwrapped + assert session.messages[-2]["content"] == "steer: use the staging bucket this time" diff --git a/tests/test_issue7622_initial_locale_from_browser.py b/tests/test_issue7622_initial_locale_from_browser.py new file mode 100644 index 00000000..e66b3bee --- /dev/null +++ b/tests/test_issue7622_initial_locale_from_browser.py @@ -0,0 +1,527 @@ +"""Regression coverage for #7622 — initial install locale from browser hint. + +The renderer's `loadLocale()` was strictly a localStorage-or-English +fallback — non-English speakers had to dig into Settings after every +fresh install. This fix consults `navigator.languages[0]` / +`navigator.language` when no stored preference exists, so the very +first visit lands on the browser's preferred language. + +The test drives the real `loadLocale()` in `static/i18n.js` via node +rather than relying on source-text assertions — the same forward-gate +principle the renderer-mirror test files spell out, because +"is `navigator.languages` consulted?" is a runtime fact that +source-text grepping can easily miss. +""" + +from __future__ import annotations + +import json +import re +import shutil +import subprocess +from pathlib import Path + +import pytest + +REPO_ROOT = Path(__file__).resolve().parent.parent +I18N_JS_PATH = REPO_ROOT / "static" / "i18n.js" + +NODE = shutil.which("node") + +pytestmark = pytest.mark.skipif(NODE is None, reason="node not on PATH") + + +def _extract_block(src: str, start_marker: str) -> str: + """Slice out a `const LOCALES = { ... };` (or any `… = { … };`) block + starting at the first occurrence of ``start_marker``.""" + idx = src.index(start_marker) + brace_idx = src.index("{", idx) + depth = 1 + i = brace_idx + 1 + while depth and i < len(src): + if src[i] == "{": + depth += 1 + elif src[i] == "}": + depth -= 1 + i += 1 + if depth: + raise ValueError(f"unterminated block starting at {start_marker!r}") + return src[idx:i] + + +def _build_driver(stored_value, navigator_obj): + """Build a node driver that loads i18n.js, mocks localStorage + + navigator, exposes ``_lastSetLang``, and writes the final + ``setLocale(lang)`` argument to stdout. + + The driver runs the full i18n.js (with the auto-``loadLocale()`` at + the bottom of the file removed) inside a function scope so that + each test gets a clean module instance. + """ + src = I18N_JS_PATH.read_text(encoding="utf-8") + # Strip the auto-loadLocale() call at the bottom so the test can + # drive the function explicitly. It's a single top-level call. + src = re.sub(r"\nloadLocale\(\);\s*$", "", src, count=1) + + # Build a string for the navigator object literal. + if navigator_obj is None: + nav_literal = "undefined" + elif isinstance(navigator_obj, str): + # Convenience: a bare string means "navigator.languages is missing, + # navigator.language is ". + nav_literal = f'{{ language: {navigator_obj!r}, languages: undefined }}' + elif isinstance(navigator_obj, list): + nav_literal = ( + f'{{ language: {navigator_obj[0]!r}, languages: {navigator_obj!r} }}' + ) + else: + raise TypeError(navigator_obj) + + # Build a string for the localStorage mock. + if stored_value is None: + stored_literal = 'undefined' + else: + stored_literal = f'{{ getItem: () => {stored_value!r} }}' + + return f""" +const fs = require('fs'); +const src = {src!r}; + +// Mock localStorage (must be set BEFORE eval, i18n.js may read on load) +global.localStorage = {stored_literal}; + +// Mock document (i18n.js calls setLocale → document.documentElement.lang = ...) +global.document = {{ + documentElement: {{ set lang(_) {{}} }}, + querySelectorAll: () => [], + querySelector: () => null, + addEventListener: () => {{}}, +}}; + +// Eval the i18n source — this installs `loadLocale`, `setLocale`, `t`, +// `resolveLocale`, `_locale`, and `LOCALES` in the current scope. Node +// 22+ has its own `navigator` global we cannot simply reassign with +// `global.navigator = ...`; we use `Object.defineProperty` so the +// shadow lands regardless of the read-only per-property defaults. +eval(src); + +// Build the navigator mock object AFTER eval so the test value wins +// over Node's built-in default. `Object.defineProperty` is needed +// because `navigator` in Node 22+ is defined on the global with a +// non-writable / non-configurable data descriptor on the property +// level — we replace the entire object instead. +const _navigatorMock = {nav_literal}; +Object.defineProperty(global, 'navigator', {{ + value: _navigatorMock, + writable: true, + configurable: true, + enumerable: true, +}}); + +// Wrap setLocale in-place by re-assigning the function name. Eval +// installs setLocale as a function declaration in the surrounding +// scope, so `setLocale = ...` rebinds it for any subsequent caller +// in the same scope. +const _origSetLocale = setLocale; +setLocale = function(lang) {{ + global._lastSetLang = lang; + _origSetLocale(lang); +}}; + +// Drive loadLocale() now (after our mocks and setLocale override are +// in place) +loadLocale(); + +// Emit the chosen lang on stdout, JSON-encoded, then exit. Wrap +// the navigator read in a try/catch — when the test deletes navigator +// (see test_missing_navigator_falls_back_to_en), `navigator.language` +// throws; we want the test to see the chosen lang without that +// secondary failure. +let _navInfo = {{}}; +try {{ _navInfo = {{ nav_language: navigator.language, nav_languages: navigator.languages }}; }} catch (_) {{}} +process.stdout.write(JSON.stringify({{ lang: global._lastSetLang, ..._navInfo }})); +""" + + +@pytest.fixture(scope="module") +def i18n_src() -> str: + return I18N_JS_PATH.read_text(encoding="utf-8") + + +def _run(driver_src: str) -> str: + """Run a driver in a fresh node subprocess and return stdout.""" + import tempfile + + with tempfile.NamedTemporaryFile("w", suffix=".js", delete=False) as f: + f.write(driver_src) + path = f.name + try: + result = subprocess.run( + [NODE, path], capture_output=True, text=True, timeout=30 + ) + finally: + Path(path).unlink(missing_ok=True) + if result.returncode != 0: + raise RuntimeError( + f"node driver failed: rc={result.returncode} stderr={result.stderr!r}" + ) + return result.stdout.strip() + + +# ── 1. test_first_visit_zh_cn_falls_back_to_zh ──────────────────────────── + + +class TestInitialLocaleFromBrowserHint: + + def test_first_visit_zh_cn_falls_back_to_zh(self, i18n_src): + """#7622 repro: a fresh install on a Chinese browser must default + to ``zh`` without the user having to open Settings. + + `navigator.language` reports the primary preference; for a + Chinese browser that's ``zh-CN``. `resolveLocale()` strips the + region subtag and lands on the available ``zh`` locale. + """ + driver = _build_driver(stored_value=None, navigator_obj="zh-CN") + out = _run(driver) + assert json.loads(out)["lang"] == "zh", ( + f"first-visit zh-CN browser must default to 'zh' (the " + f"available Chinese locale). Got: {out!r}" + ) + + def test_first_visit_en_us_falls_back_to_en(self, i18n_src): + """First-visit en-US must still default to en (regression guard + — the fix must not break the most common case).""" + driver = _build_driver(stored_value=None, navigator_obj="en-US") + out = _run(driver) + assert json.loads(out)["lang"] == "en", ( + f"first-visit en-US browser must default to 'en'. " + f"Got: {out!r}" + ) + + def test_first_visit_fr_fr_falls_back_to_fr(self, i18n_src): + """First-visit fr-FR must default to fr (one of the 14 + supported locales).""" + driver = _build_driver(stored_value=None, navigator_obj="fr-FR") + out = _run(driver) + assert json.loads(out)["lang"] == "fr", ( + f"first-visit fr-FR browser must default to 'fr'. " + f"Got: {out!r}" + ) + + def test_first_visit_unsupported_locale_falls_back_to_en(self, i18n_src): + """First-visit on a browser that reports a locale with no + translation available must still default to English (not crash + or hang).""" + driver = _build_driver(stored_value=None, navigator_obj="kl-GL") + out = _run(driver) + assert json.loads(out)["lang"] == "en", ( + f"unsupported browser locale must fall back to 'en'. " + f"Got: {out!r}" + ) + + def test_stored_preference_overrides_browser_hint(self, i18n_src): + """If the user has already chosen a language (or the server + has persisted one), the stored value MUST win — the browser + hint is only a first-visit fallback. A French browser + visiting again after having picked Japanese must still see + Japanese, not French. + """ + driver = _build_driver(stored_value="ja", navigator_obj="fr-FR") + out = _run(driver) + assert json.loads(out)["lang"] == "ja", ( + f"stored preference must override the browser hint. " + f"Got: {out!r}" + ) + + def test_navigator_languages_takes_precedence_over_navigator_language( + self, i18n_src + ): + """`navigator.languages[0]` (the ordered preference list) is + preferred over `navigator.language` (the single primary) when + the list is available. Chrome and Firefox both expose the + list. + """ + driver = _build_driver( + stored_value=None, + navigator_obj=["ja", "en-US", "en"], + ) + out = _run(driver) + assert json.loads(out)["lang"] == "ja", ( + f"navigator.languages[0] must drive the first-visit hint. " + f"Got: {out!r}" + ) + + def test_missing_navigator_falls_back_to_en(self, i18n_src): + """Some embedded webviews (Tauri desktop shell, certain kiosk + modes) report no `navigator` at all. loadLocale() must + not crash and must default to English. + """ + driver = _build_driver(stored_value=None, navigator_obj=None) + out = _run(driver) + assert json.loads(out)["lang"] == "en", ( + f"missing navigator must default to 'en'. Got: {out!r}" + ) + + def test_browser_hint_never_consulted_when_stored_exists( + self, i18n_src + ): + """Belt-and-braces: even with a stored English preference, a + browser hint for, say, Japanese must not sneak through on a + second visit. This pins the 'stored wins' contract.""" + driver = _build_driver( + stored_value="en", navigator_obj=["ja-JP", "en-US"] + ) + out = _run(driver) + assert json.loads(out)["lang"] == "en", ( + f"stored 'en' must win over a browser ja-JP hint. " + f"Got: {out!r}" + ) + + +# ── 2. Cross-file test for the composed resolver (#7622 round 3) ───────── +# +# The round-2 fix added a 3-arg resolver but kept a `primary === 'en'` +# special case to skip the server's schema default. That heuristic +# overrode users who had genuinely picked English on purpose: when the +# server-stored `language` was `"en"`, the resolver treated it as +# "schema default" and skipped it, falling through to localStorage / +# the browser hint. The round-3 fix drops the schema default at the +# source (`api/config.py:_SETTINGS_DEFAULTS`) so the server reports +# `None` for a fresh install and the resolver's 3-arg chain is +# sufficient: explicit server `primary` always wins, then +# `localStorage` `fallback`, then the browser hint `fallback2`, then +# `'en'`. +# +# A user who picked English on purpose has `language: "en"` in their +# stored settings file. The client receives that explicit value as +# `primary` and must preserve it, regardless of the browser hint. A +# user on a fresh install has no `language` key — `primary` is +# `None`/undefined — and the browser hint (or `'en'`) is the right +# outcome. These two cases must not be conflated. + + +class TestComposedResolverPrecedence: + """Pin the precedence chain in the 3-arg resolver used by boot.js + (settings hydration) and panels.js (settings modal). Round-3 + contract: explicit server `primary` always wins, then + `localStorage` `fallback`, then the browser hint `fallback2`, + then `'en'`. No `primary === 'en'` skip.""" + + def _build_resolver_driver(self, i18n_src, primary, fallback, fallback2): + """Build a node driver that extracts `resolvePreferredLocale` + from the eval'd i18n.js and calls it once with the given args. + Returns the resolved lang on stdout. + """ + # Python `None` must become JS `null`; otherwise the JS resolver + # sees the *string* `"None"` and resolveLocale("None") returns + # null, hiding the precedence chain. + def _js(v): + return "null" if v is None else repr(v) + return f""" +const fs = require('fs'); +const src = {i18n_src!r}; + +// strip the auto-loadLocale() so the test drives the resolver directly +eval(src.replace(/\\nloadLocale\\(\\);\\s*$/, '')); + +const _lang = resolvePreferredLocale({_js(primary)}, {_js(fallback)}, {_js(fallback2)}); +process.stdout.write(JSON.stringify({{ lang: _lang }})); +""" + + def test_fresh_install_no_primary_uses_browser_hint(self, i18n_src): + """#7622 round 3: a fresh install (server returns no `language` + key, so `primary` is `None`) on a Chinese browser must land on + `zh`. This is the first-visit case the round-2 schema-default + skip was originally written for — in round-3 the same outcome + falls out naturally because the server no longer reports a + schema default to skip. + """ + driver = self._build_resolver_driver( + i18n_src, primary=None, fallback=None, fallback2="zh-CN" + ) + out = _run(driver) + assert json.loads(out)["lang"] == "zh", ( + f"fresh install + zh-CN browser must default to 'zh' via " + f"the browser hint. Got: {out!r}" + ) + + def test_fresh_install_with_stored_ja_returns_ja(self, i18n_src): + """Stored preference must always win over the browser hint.""" + driver = self._build_resolver_driver( + i18n_src, primary=None, fallback="ja", fallback2="zh-CN" + ) + out = _run(driver) + assert json.loads(out)["lang"] == "ja", ( + f"a stored 'ja' must win over the browser hint 'zh-CN'. " + f"Got: {out!r}" + ) + + def test_explicit_server_zh_with_browser_ja_returns_zh(self, i18n_src): + """An explicit server language is treated as a real preference + and must win over the browser hint, even with no stored + choice.""" + driver = self._build_resolver_driver( + i18n_src, primary="zh", fallback=None, fallback2="ja-JP" + ) + out = _run(driver) + assert json.loads(out)["lang"] == "zh", ( + f"an explicit server 'zh' must win over the 'ja-JP' " + f"browser hint. Got: {out!r}" + ) + + def test_explicit_server_zh_with_stored_fr_returns_zh( + self, i18n_src + ): + """Server explicit > stored > browser > 'en'.""" + driver = self._build_resolver_driver( + i18n_src, primary="zh", fallback="fr", fallback2="ja-JP" + ) + out = _run(driver) + assert json.loads(out)["lang"] == "zh", ( + f"explicit server 'zh' must win over stored 'fr' and " + f"browser 'ja-JP'. Got: {out!r}" + ) + + def test_no_preference_anywhere_falls_back_to_en(self, i18n_src): + """No server, no stored, no browser hint -> 'en' (the safety + net). Replaces the round-2 `test_schema_default_en_with_no_browser_falls_back_to_en`: + the same outcome, but the `primary` is now genuinely `None` + instead of a round-2-pretending-to-be-skipped `'en'`.""" + driver = self._build_resolver_driver( + i18n_src, primary=None, fallback=None, fallback2=None + ) + out = _run(driver) + assert json.loads(out)["lang"] == "en", ( + f"no preference at all must default to 'en'. Got: {out!r}" + ) + + def test_explicit_saved_english_wins_over_browser_zh(self, i18n_src): + """#7622 round-3 BRICK regression: a user who explicitly saved + English (server has `language: "en"`, localStorage may or may + not have a value yet) on a non-English browser must still land + on English. This is the case the round-2 + `primary === 'en'` skip overrode. + + Tests two flavours: + - stored is empty, primary is the saved 'en', browser is zh + (boot.js call shape on a brand-new browser profile) + - stored is 'en' (the legacy migration shape), primary is + the saved 'en', browser is ja (panels.js re-hydrate + after a stale browser update) + """ + # Brand-new browser, server has the user's saved 'en'. + driver = self._build_resolver_driver( + i18n_src, primary="en", fallback=None, fallback2="zh-CN" + ) + out = _run(driver) + assert json.loads(out)["lang"] == "en", ( + f"server-saved 'en' must beat a zh-CN browser hint. " + f"Got: {out!r}" + ) + + # Stale browser, server still has the user's saved 'en'. + driver = self._build_resolver_driver( + i18n_src, primary="en", fallback="ja", fallback2="en-US" + ) + out = _run(driver) + assert json.loads(out)["lang"] == "en", ( + f"server-saved 'en' must beat both a stale 'ja' storage " + f"value and an 'en-US' browser hint. Got: {out!r}" + ) + + +# ── 3. Reviewer 4-row table regression (#7730 round 4) ────────────────── +# +# The maintainer reproduced the round-2 bug by loading the real +# `static/i18n.js` in a Node `vm` sandbox and calling +# `resolvePreferredLocale` EXACTLY as `static/boot.js` calls it: +# +# resolvePreferredLocale(s.language, +# localStorage.getItem('hermes-lang'), +# _detectBrowserLanguageHint()) +# +# Their table (server `language` / localStorage `hermes-lang` / browser) +# must all land on the server value once the server reports an explicit +# tri-state (`null` = unset, "en" = explicitly saved English, other): +# +# | en (chosen) | empty (new browser) | zh-CN | en | +# | en (chosen) | stale ja | en-US | en | +# | en | empty | en-US | en | +# | fr | empty | zh-CN | fr | +# +# Round-2's `primary === 'en'` skip produced zh / ja for the first two +# rows (the bug). Round-3 removed the server default so a fresh install +# reports `None`; round-4 makes that `None` explicit in `load_settings()` +# and trusts the server value verbatim — no guessing whether "en" was +# chosen. This test drives all four rows through the same vm sandbox +# call shape the maintainer used, against the real file. + + +class TestReviewerTableServerTriState: + """Drive `resolvePreferredLocale` exactly as boot.js calls it, in a + Node `vm` sandbox, for every row of the maintainer's reproduction + table. The server's explicit language value must win each row; the + browser hint must never override a saved choice.""" + + def _build_boot_shape_driver(self, i18n_src, server_lang, stored, browser_langs): + # Python None -> JS null (Trap 2 in the skill); strings -> JS strings. + js_server = "null" if server_lang is None else json.dumps(server_lang) + stored_seed = ( + "" + if stored is None + else f"storage['hermes-lang'] = {json.dumps(stored)};" + ) + nav_literal = json.dumps( + { + "languages": browser_langs, + "language": browser_langs[0] if browser_langs else "", + } + ) + src = re.sub(r"\nloadLocale\(\);\s*$", "", i18n_src, count=1) + call = ( + f"resolvePreferredLocale({js_server}, " + f"localStorage.getItem('hermes-lang'), _detectBrowserLanguageHint())" + ) + return f""" +const fs = require('fs'); +const vm = require('vm'); +const src = {src!r}; +const storage = {{}}; +{stored_seed} +const ctx = {{ + localStorage: {{ + getItem: (k) => Object.prototype.hasOwnProperty.call(storage, k) ? storage[k] : null, + setItem: (k, v) => {{ storage[k] = String(v); }}, + }}, + document: {{ + documentElement: {{ lang: '' }}, + querySelectorAll: () => [], + }}, + navigator: {nav_literal}, +}}; +vm.createContext(ctx); +vm.runInContext(src, ctx); +// Same call shape as static/boot.js:3443-3445. +const out = vm.runInContext({json.dumps(call)}, ctx); +process.stdout.write(JSON.stringify(out)); +""" + + def test_four_row_table_server_tri_state_boot_shape(self, i18n_src): + """#7730 reviewer table — all four rows, server value must win.""" + rows = [ + # (server language, localStorage, browser langs, expected) + ("en", None, ["zh-CN"], "en"), # explicit English, brand-new browser + ("en", "ja", ["en-US"], "en"), # explicit English, stale ja cookie + ("en", None, ["en-US"], "en"), # explicit English, en browser + ("fr", None, ["zh-CN"], "fr"), # explicit French, zh browser + ] + for server_lang, stored, browser_langs, expected in rows: + driver = self._build_boot_shape_driver( + i18n_src, server_lang, stored, browser_langs + ) + out = _run(driver) + assert json.loads(out) == expected, ( + f"server={server_lang!r} stored={stored!r} " + f"browser={browser_langs!r} -> expected {expected!r}, " + f"got {out!r}" + ) diff --git a/tests/test_language_precedence.py b/tests/test_language_precedence.py index 2d477c75..de8427f0 100644 --- a/tests/test_language_precedence.py +++ b/tests/test_language_precedence.py @@ -9,10 +9,22 @@ REPO_ROOT = pathlib.Path(__file__).parent.parent.resolve() I18N_JS = (REPO_ROOT / "static" / "i18n.js").read_text(encoding="utf-8") BOOT_JS = (REPO_ROOT / "static" / "boot.js").read_text(encoding="utf-8") PANELS_JS = (REPO_ROOT / "static" / "panels.js").read_text(encoding="utf-8") +CONFIG_PY = (REPO_ROOT / "api" / "config.py").read_text(encoding="utf-8") -def _run_i18n_case(script_expr: str) -> dict: +def _run_i18n_case(script_expr: str, *, navigator_obj: object | None = None) -> dict: wrapped_expr = f"(() => ({script_expr}))()" + if navigator_obj is None: + navigator_src = "undefined" + elif navigator_obj == "throwing": + # Mimic some embedded webviews: a `navigator` object whose + # `languages` / `language` accessors throw on read. + navigator_src = ( + "{ get languages(){ throw new Error('navigator access denied'); }," + " get language(){ throw new Error('navigator access denied'); } }" + ) + else: + navigator_src = json.dumps(navigator_obj) script = textwrap.dedent( f""" const fs = require('fs'); @@ -28,6 +40,7 @@ def _run_i18n_case(script_expr: str) -> dict: documentElement: {{ lang: '' }}, querySelectorAll: () => [], }}, + navigator: {navigator_src}, }}; vm.createContext(ctx); vm.runInContext(src, ctx); @@ -260,3 +273,156 @@ def test_set_locale_normalizes_alias_and_persists_canonical_key(): def test_boot_and_settings_panel_use_shared_locale_precedence(): assert _has_precedence_call(BOOT_JS, "s.language") assert _has_precedence_call(PANELS_JS, "settings.language") + + +# --- #7622 round-3 behavioural pins ----------------------------------------- +# +# These four tests pin the new contract end-to-end so a future +# maintainer cannot silently reintroduce the round-2 regressions: +# - the schema default was dropped (server) +# - the resolver no longer treats primary='en' as "no preference" +# - all three call sites use the shared guarded helper +# - the guarded helper survives a throwing `navigator` accessor +# +# Each test loads the real `static/i18n.js` in a Node `vm` sandbox, so +# the assertions are against the production code, not a copy. + + +def test_settings_defaults_drop_language_default(): + """`language` is intentionally absent from `_SETTINGS_DEFAULTS` so a + fresh install returns `None` (the key is missing) and the client + can distinguish "no preference" from a genuine saved choice.""" + # The dict literal still exists, but the line carrying the + # `"language": "en"` default must be gone. A grep for the + # default covers future renames / re-shuffles of the dict. + assert '"language": "en"' not in CONFIG_PY + # Defence in depth: even with the line removed, the comment that + # documents the round-3 contract must remain so the next maintainer + # doesn't quietly re-add it. + assert "language is intentionally absent" in CONFIG_PY + + +def test_i18n_exposes_guarded_browser_hint_helper(): + """`_detectBrowserLanguageHint` must exist and survive a `navigator` + accessor that throws — used by loadLocale, boot.js, and panels.js + to keep a single bad read from aborting boot / settings hydration.""" + assert "function _detectBrowserLanguageHint" in I18N_JS + + # Happy path: navigator.languages[0] wins. + out = _run_i18n_case( + "_detectBrowserLanguageHint()", + navigator_obj={"languages": ["pt-BR", "en-US"], "language": "en-US"}, + ) + assert out == "pt-BR" + + # Fallback: only navigator.language present. + out = _run_i18n_case( + "_detectBrowserLanguageHint()", + navigator_obj={"language": "fr-FR"}, + ) + assert out == "fr-FR" + + # Throwing accessor: helper must swallow and return null. + out = _run_i18n_case( + "_detectBrowserLanguageHint()", + navigator_obj="throwing", + ) + assert out is None + + # Empty / missing: null in, null out. + out = _run_i18n_case( + "_detectBrowserLanguageHint()", + navigator_obj={"languages": [], "language": ""}, + ) + assert out is None + + +def test_composed_resolver_preserves_explicit_english(): + """#7622 BRICK regression: with the round-2 `primary === 'en'` skip + removed, an explicit saved English must still beat a non-English + browser hint (the user picked English on purpose, do not override).""" + result = _run_i18n_case( + """ +{ + // server-stored 'en' (user picked English) + empty localStorage + zh-CN browser + explicitEnWins: resolvePreferredLocale('en', null, 'zh-CN'), + explicitEnBeatsStale: resolvePreferredLocale('en', 'ja', 'en-US'), + // explicit non-English server value still wins + explicitZh: resolvePreferredLocale('zh', null, 'en-US'), + explicitZhBeatsStored: resolvePreferredLocale('zh', 'ja', 'en-US'), + // non-English primary resolves through (no skip) + primaryResolves: resolvePreferredLocale('zh-CN', 'en', 'en-US'), + primaryNotEnResolves: resolvePreferredLocale('fr', 'en', 'en-US'), +} + """ + ) + assert result["explicitEnWins"] == "en" + assert result["explicitEnBeatsStale"] == "en" + assert result["explicitZh"] == "zh" + assert result["explicitZhBeatsStored"] == "zh" + assert result["primaryResolves"] == "zh" + assert result["primaryNotEnResolves"] == "fr" + + +def test_load_locale_first_visit_uses_browser_hint_when_no_preference(): + """End-to-end cross-file case: empty localStorage + zh-CN browser + + no stored server preference → loadLocale() must land on 'zh', not + the round-2 'en' default. The browser hint is also safe against a + throwing navigator accessor (must fall through to 'en').""" + # Fresh install: empty localStorage, browser is zh-CN. + out = _run_i18n_case( + """ +{ + ...(loadLocale(), {}), + saved: localStorage.getItem('hermes-lang'), + htmlLang: document.documentElement.lang, +} + """, + navigator_obj={"languages": ["zh-CN", "en"], "language": "zh-CN"}, + ) + assert out["saved"] == "zh" + assert out["htmlLang"] == "zh-CN" + + # Throwing navigator accessor: loadLocale() must NOT abort, the + # final locale must fall through to 'en' (the safety net). + out = _run_i18n_case( + """ +{ + ...(loadLocale(), {}), + saved: localStorage.getItem('hermes-lang'), +} + """, + navigator_obj="throwing", + ) + assert out["saved"] == "en" + + # Stored value already present: browser hint is ignored, even when + # the stored value is the previously-detected 'zh' and the browser + # now says 'en'. This is the "second visit" contract. + out = _run_i18n_case( + """ +{ + ...(loadLocale(), {}), + saved: localStorage.getItem('hermes-lang'), +} + """, + navigator_obj={"languages": ["en-US", "en"], "language": "en-US"}, + ) + # Default sandbox storage is empty, so the browser hint fires. + # When a prior setLocale('zh') already ran, localStorage wins. + assert out["saved"] in {"zh", "en"} # either is acceptable; round-3 + # specifically tests the *first* visit below. + + # Explicit "stored wins" contract: pre-seed localStorage, then + # loadLocale() must not consult the browser hint. + out = _run_i18n_case( + """ +{ + ...(setLocale('fr'), {}), // pre-seed localStorage + ...(loadLocale(), {}), + saved: localStorage.getItem('hermes-lang'), +} + """, + navigator_obj={"languages": ["zh-CN"], "language": "zh-CN"}, + ) + assert out["saved"] == "fr" diff --git a/tests/test_runtime_model_observation.py b/tests/test_runtime_model_observation.py new file mode 100644 index 00000000..7c1b87e3 --- /dev/null +++ b/tests/test_runtime_model_observation.py @@ -0,0 +1,206 @@ +"""Observed Agent runtime, durable replay and HTTP reconnect projection.""" +import json +import queue +import sys +import types +from unittest import mock + +import pytest + +from api.run_journal import RunJournalWriter, latest_run_summary, read_run_events + + +def observation(model="backup", *, session_id="session-a", stream_id="run-a"): + return {"session_id": session_id, "stream_id": stream_id, "model": model, + "provider": "custom", "fallback_active": True, "phase": "observed_output"} + + +def test_journal_runtime_is_owned_and_invalidated(tmp_path): + writer = RunJournalWriter("session-a", "run-a", session_dir=tmp_path) + writer.append_sse_event("token", {"text": "answer"}) + first = writer.append_sse_event("runtime_model", observation()) + assert read_run_events("session-a", "run-a", session_dir=tmp_path)["events"][-1] == first + summary = latest_run_summary("session-a", "run-a", session_dir=tmp_path) + assert summary["runtime_model"] == observation() + summary["runtime_model"]["model"] = "corrupted" + assert latest_run_summary("session-a", "run-a", session_dir=tmp_path)["runtime_model"] == observation() + writer.append_sse_event("warning", {"type": "fallback", "message": "Trying another route"}) + assert latest_run_summary("session-a", "run-a", session_dir=tmp_path)["runtime_model"] is None + writer.append_sse_event("runtime_model", {**observation(), "stream_id": "foreign"}) + assert latest_run_summary("session-a", "run-a", session_dir=tmp_path)["runtime_model"] is None + writer.append_sse_event("runtime_model", observation("recovered")) + assert latest_run_summary("session-a", "run-a", session_dir=tmp_path)["runtime_model"]["model"] == "recovered" + writer.append_sse_event("runtime_model", "malformed observation") + assert latest_run_summary("session-a", "run-a", session_dir=tmp_path)["runtime_model"] is None + writer.append_sse_event("runtime_model", observation("recovered-again")) + assert latest_run_summary("session-a", "run-a", session_dir=tmp_path)["runtime_model"]["model"] == "recovered-again" + assert latest_run_summary("session-a", "another-run", session_dir=tmp_path)["runtime_model"] is None + + +@pytest.mark.parametrize("malformed", ["bad observation", ["bad observation"], None]) +def test_non_object_observation_invalidates_summary_and_http_snapshot(tmp_path, monkeypatch, malformed): + from api import routes, run_journal + writer = RunJournalWriter("session-a", "run-a", session_dir=tmp_path) + writer.append_sse_event("runtime_model", observation("first")) + writer.append_sse_event("token", malformed) + assert latest_run_summary("session-a", "run-a", session_dir=tmp_path)["runtime_model"]["model"] == "first" + writer.append_sse_event("runtime_model", malformed if malformed is not None else {"placeholder": True}) + if malformed is None: + # The writer normalizes None to {}; exercise a legacy/raw null journal row. + path = run_journal._run_path("session-a", "run-a", session_dir=tmp_path) + rows = [json.loads(line) for line in path.read_text().splitlines()] + rows[-1]["payload"] = None + path.write_text("\n".join(json.dumps(row) for row in rows) + "\n") + assert latest_run_summary("session-a", "run-a", session_dir=tmp_path)["runtime_model"] is None + monkeypatch.setattr(routes, "find_run_summary", lambda rid: latest_run_summary("session-a", rid, session_dir=tmp_path)) + monkeypatch.setattr(routes, "read_run_events", lambda sid, rid: read_run_events(sid, rid, session_dir=tmp_path)) + snapshot = routes._run_journal_live_snapshot("run-a") + assert snapshot["runtime_model"] is None + assert routes._runtime_journal_snapshot_for_session_payload(snapshot)["runtime_model"] is None + + +@pytest.mark.parametrize("suffix,expected", [("", "backup"), ("warning", None), ("foreign", None)]) +def test_http_snapshot_projects_exact_journal_window(tmp_path, monkeypatch, suffix, expected): + import api.routes as routes + writer = RunJournalWriter("session-a", "run-a", session_dir=tmp_path) + writer.append_sse_event("runtime_model", observation()) + writer.append_sse_event("token", {"text": "answer"}) + if suffix == "warning": + writer.append_sse_event("warning", {"type": "fallback"}) + elif suffix == "foreign": + writer.append_sse_event("runtime_model", {**observation(), "session_id": "foreign"}) + monkeypatch.setattr(routes, "find_run_summary", lambda rid: latest_run_summary("session-a", rid, session_dir=tmp_path)) + monkeypatch.setattr(routes, "read_run_events", lambda sid, rid: read_run_events(sid, rid, session_dir=tmp_path)) + snapshot = routes._run_journal_live_snapshot("run-a") + assert snapshot["last_assistant_text"] == "answer" + value = snapshot["runtime_model"] + assert (value["model"] if value else None) == expected + assert routes._runtime_journal_snapshot_for_session_payload(snapshot)["runtime_model"] == value + + +def run_worker(steps, *, fail=False, ephemeral=False, tmp_path=None, monkeypatch=None): + """Drive the production local worker; fake only external Agent/provider calls.""" + from api import streaming, run_journal + from api.config import SESSION_AGENT_CACHE + SESSION_AGENT_CACHE.clear() + monkeypatch.setattr(run_journal, "_default_session_dir", lambda: tmp_path) + class Session: + session_id = "session-a" + title = "Session" + workspace = "/tmp" + model = "primary" + model_provider = "anthropic" + profile = None + personality = None + messages = [{"role": "user", "content": "old"}, {"role": "assistant", "content": "old answer"}] + context_messages = messages + input_tokens = output_tokens = cache_read_tokens = cache_write_tokens = 0 + estimated_cost = 0.0 + tool_calls = [] + gateway_routing = None + gateway_routing_history = [] + active_stream_id = "run-a" + pending_user_message = None + pending_attachments = [] + pending_started_at = None + context_length = threshold_tokens = last_prompt_tokens = 0 + llm_title_generated = True + def save(self, *args, **kwargs): pass + def compact(self): return {"session_id": self.session_id} + + class Agent: + def __init__(self, model=None, provider=None, session_id=None, + stream_delta_callback=None, reasoning_callback=None, + status_callback=None, **kwargs): + self.model, self.provider, self.session_id = model, provider, session_id + self.stream_delta_callback = stream_delta_callback + self.reasoning_callback = reasoning_callback + self.status_callback = status_callback + self._provider_fallback_active = False + self.context_compressor = None + self.session_prompt_tokens = self.session_completion_tokens = 0 + self.session_estimated_cost_usd = None + self.session_cache_read_tokens = self.session_cache_write_tokens = 0 + self.reasoning_config = self.ephemeral_system_prompt = self._last_error = None + def run_conversation(self, **kwargs): + for model, provider, fallback, channel, text in steps: + self.model, self.provider, self._provider_fallback_active = model, provider, fallback + if channel == "token": self.stream_delta_callback(text) + elif channel == "reasoning": self.reasoning_callback(text) + elif channel == "status": self.status_callback("lifecycle", text) + if fail: return {"error": "provider unavailable", "messages": []} + return {"messages": kwargs.get("conversation_history", []) + [ + {"role": "user", "content": kwargs["persist_user_message"]}, + {"role": "assistant", "content": "served answer"}]} + def interrupt(self, message): pass + + runtime_module = types.ModuleType("hermes_cli.runtime_provider") + runtime_module.resolve_runtime_provider = mock.Mock(return_value={ + "provider": "anthropic", "base_url": None, "api_key": "sk-test", + "api_mode": "chat_completions", "command": None, "args": [], "credential_pool": None}) + cli_module = types.ModuleType("hermes_cli") + cli_module.runtime_provider = runtime_module + state_module = types.ModuleType("hermes_state") + state_module.SessionDB = mock.Mock(return_value=None) + injected = {"hermes_cli": cli_module, "hermes_cli.runtime_provider": runtime_module, + "hermes_state": state_module} + sentinel = object() + saved = {name: sys.modules.get(name, sentinel) for name in injected} + sys.modules.update(injected) + q = queue.Queue() + s = Session() + try: + with mock.patch.object(streaming, "get_session", return_value=s), \ + mock.patch.object(streaming, "_get_ai_agent", return_value=Agent), \ + mock.patch.object(streaming, "resolve_model_provider", return_value=("primary", "anthropic", None)), \ + mock.patch("api.config.get_config", return_value={}), \ + mock.patch("api.config._resolve_cli_toolsets", return_value=[]): + streaming.STREAMS["run-a"] = q + streaming._run_agent_streaming("session-a", "new turn", "primary", "/tmp", "run-a", ephemeral=ephemeral) + finally: + for name, original in saved.items(): + if original is sentinel: sys.modules.pop(name, None) + else: sys.modules[name] = original + streaming.STREAMS.pop("run-a", None) + SESSION_AGENT_CACHE.clear() + return s, list(q.queue) + + +def test_worker_observes_serving_runtime_not_requested(tmp_path, monkeypatch): + s, events = run_worker([ + ("primary", "anthropic", False, "reasoning", "thinking"), + ("backup", "custom", True, "token", "answer"), + ("backup", "custom", True, "status", "Switched to fallback: backup (custom)"), + ], tmp_path=tmp_path, monkeypatch=monkeypatch) + names = [name for name, _ in events] + assert names.count("runtime_model") == 3 # primary, backup, post-warning correction + observed = [value for name, value in events if name == "runtime_model"] + assert [value["model"] for value in observed] == ["primary", "backup", "backup"] + assert observed[-1] == observation() + assert names.index("runtime_model") < names.index("reasoning") + assert names.index("warning") < len(names) - 1 - names[::-1].index("runtime_model") + assert latest_run_summary("session-a", "run-a", session_dir=tmp_path)["runtime_model"] == observation() + assert s.model == "primary" # selection is not overwritten + + +def test_worker_does_not_observe_attempt_or_missing_model(tmp_path, monkeypatch): + _, events = run_worker([ + ("backup", "custom", True, "status", "Trying fallback backup"), + (None, "custom", True, "token", "partial"), + ], fail=True, tmp_path=tmp_path, monkeypatch=monkeypatch) + assert not any(name == "runtime_model" for name, _ in events) + + +def test_missing_agent_model_does_not_stamp_requested_as_served(tmp_path, monkeypatch): + s, events = run_worker([(None, "custom", True, "token", "partial")], + tmp_path=tmp_path, monkeypatch=monkeypatch) + assert not any(name == "runtime_model" for name, _ in events) + assert "_usedModel" not in s.messages[-1] + usage = next(value["usage"] for name, value in events if name == "done") + assert not usage.get("used_model") + + +def test_worker_nonstreaming_success_observes_fallback(tmp_path, monkeypatch): + _, events = run_worker([("backup", "custom", True, "route", "")], + tmp_path=tmp_path, monkeypatch=monkeypatch, ephemeral=True) + assert [value["model"] for name, value in events if name == "runtime_model"] == ["backup"] diff --git a/tests/test_save_count_without_full_parse.py b/tests/test_save_count_without_full_parse.py new file mode 100644 index 00000000..543ace41 --- /dev/null +++ b/tests/test_save_count_without_full_parse.py @@ -0,0 +1,386 @@ +"""A save must not read and parse the whole sidecar just to count its messages. + +The #1558 backup safeguard in `Session.save()` compares the on-disk message +count with the incoming one, and writes `.json.bak` only when the save +would SHRINK the array. Sound. But it obtained the on-disk count by +`json.loads(self.path.read_text())` -- the entire file -- on EVERY save, +including the 99.9% of saves that grow the conversation and never back +anything up. `session_recovery._msg_count()` did the same on every boot, for +every live file that has a `.bak`. + +Measured in production on 2026-09-15, in the running container: + + sessions/fa3bca34a0c6.json 203,439,398 bytes / 266,940 messages + read_text 17,453 ms + json.loads 2,924 ms + TOTAL to obtain one integer: 20,377 ms <- paid on every save + +That was the 39,357 ms `/api/session` request stuck at stage +`t6_after_json_write`, after the read-side re-parse (#4633 recurrence) had +already been fixed. The cost scales linearly with the file, and the file grows +for as long as the conversation does. + +What the check needs is ONE integer -- how many messages the file on disk +holds -- and `save()` already writes that integer into the metadata prefix, +before `messages`, as `message_count`, exactly so readers can have it without +parsing the body (that is how `load_metadata_only()` and the sidebar freshness +check work). So `save()` reads that, through the same bounded 64 KiB prefix +reader, and falls back to the full read + parse whenever the prefix does not +carry a usable count. The `.bak` body is the one thing that still needs the +full text, and a shrink is the one time it is needed. + +The count is read from the FILE, not cached in the object, and that is the +point. An earlier revision of this PR trusted an in-memory (inode, size, +mtime_ns) signature: "this object wrote the file and stat says nothing changed, +so its length is already known". Review (nesquena-hermes, 2026-09-21) showed +that is not a content identity -- a same-length in-place rewrite inside one +mtime tick keeps all three fields, because ext4 stamps mtime from a coarse +clock, so a stale cached count reads a real shrink as a growth and skips the +#1558 backup. That is the test at the bottom of this file. A count read from +the bytes cannot be fooled that way: any rewrite of the bytes rewrites it. + +Fail-open is the contract these tests pin -- whenever the prefix has no usable +count (a legacy pre-#5854 sidecar, a file with no top-level `messages` key, a +corrupt or truncated prefix, metadata that overflows the budget), the behaviour +must be exactly today's full parse, never "assume no shrink". + +`session_recovery._msg_count()` pays the same cost at boot and is left alone +on purpose: its "torn file -> -1" contract is what makes recovery restore a +`.bak` over a truncated live file, and no bounded read can prove a file is +not truncated. +""" +import json +import os +import pathlib +from collections import OrderedDict + +import pytest + +import api.models as M + + +@pytest.fixture +def session_store(tmp_path, monkeypatch): + sdir = tmp_path / "sessions" + sdir.mkdir(parents=True, exist_ok=True) + monkeypatch.setattr(M, "SESSION_DIR", sdir) + monkeypatch.setattr(M, "SESSIONS", OrderedDict()) + return sdir + + +def _msgs(n): + return [{"role": "user" if i % 2 == 0 else "assistant", "content": f"m{i}"} for i in range(n)] + + +def _make(session_store, sid, n): + s = M.Session(session_id=sid, title="T", workspace=str(session_store.parent), + model="glm", messages=_msgs(n)) + s.save() + return s + + +def _spy_full_reads(monkeypatch, target): + """Count `Path.read_text()` calls on ONE path. + + `read_text` is the whole-file read; the metadata-prefix reader uses `open()` + with a byte budget and never goes through it. So this counter is exactly + "how many times was this file read in full", which is the cost being removed. + """ + calls = {"n": 0} + real = pathlib.Path.read_text + target = pathlib.Path(target) + + def counting(self, *a, **k): + if pathlib.Path(self) == target: + calls["n"] += 1 + return real(self, *a, **k) + + monkeypatch.setattr(type(pathlib.Path()), "read_text", counting) + return calls + + +# ── save(): the grow path must be free of full reads ──────────────────────── + +def test_grow_save_does_not_read_the_existing_file_in_full(session_store, monkeypatch): + """The common case. Appending messages must not read the previous file + in full -- this object wrote that file and it is untouched, so the on-disk + count is already known.""" + s = _make(session_store, "g1", 5) + calls = _spy_full_reads(monkeypatch, s.path) + + s.messages = _msgs(7) + s.save() + + assert calls["n"] == 0, f"grow-save read the existing sidecar in full {calls['n']}x" + assert not s.path.with_suffix(".json.bak").exists(), "a grow-save must not produce a backup" + assert len(M.Session.load("g1").messages) == 7 + + +def test_same_size_save_does_not_read_the_existing_file_in_full(session_store, monkeypatch): + """Metadata-only saves (title, flags, stream state) keep the array as-is. + They are the most frequent save of all and must be just as cheap.""" + s = _make(session_store, "g2", 5) + calls = _spy_full_reads(monkeypatch, s.path) + + s.title = "renamed" + s.save() + + assert calls["n"] == 0 + assert not s.path.with_suffix(".json.bak").exists() + + +# ── save(): the safeguards must be untouched ──────────────────────────────── + +def test_shrink_save_still_writes_bak_with_the_pre_shrink_content(session_store, monkeypatch): + """#1558 must keep working. A shrink still produces a `.bak` holding the + pre-shrink array -- and reads the file in full exactly once, for that body. + Not twice: knowing the count must not be followed by a redundant parse.""" + s = _make(session_store, "s1", 5) + calls = _spy_full_reads(monkeypatch, s.path) + + s.messages = _msgs(3) + s.save() + + reads_by_save = calls["n"] # taken before load() below, which reads the file too + bak = s.path.with_suffix(".json.bak") + assert bak.exists(), "a shrinking save must leave a recoverable backup" + assert len(json.loads(bak.read_text(encoding="utf-8"))["messages"]) == 5 + assert len(M.Session.load("s1").messages) == 3 + assert reads_by_save == 1, f"shrink-save read the sidecar in full {reads_by_save}x; the .bak body needs exactly one" + + +def test_empty_active_snapshot_is_still_refused(session_store, monkeypatch): + """The other guard in the same block: an empty array with a live stream or + pending prompt must NOT overwrite a populated file (the #1558 data-loss + shape). This object was neither loaded nor saved by this process, so it has + no signature to trust and refusing may cost today's single full read -- + never more, and it must still refuse.""" + _make(session_store, "r1", 5) + empty = M.Session(session_id="r1", title="T", workspace=str(session_store.parent), + model="glm", messages=[], active_stream_id="a" * 32, + pending_user_message="still typing") + calls = _spy_full_reads(monkeypatch, empty.path) + + empty.save() + + reads_by_save = calls["n"] # taken before load() below, which reads the file too + assert len(M.Session.load("r1").messages) == 5, "the populated file must survive" + assert reads_by_save <= 1, f"refusing read the file {reads_by_save}x; an unknown identity costs at most one" + + +# ── save(): fail-open whenever the prefix carries no usable count ─────────── + +def _write_legacy_without_count(session_store, sid, n): + """A pre-#5854 sidecar: no persisted `message_count` at all.""" + doc = {"session_id": sid, "title": "Legacy", "workspace": str(session_store.parent), + "model": "glm", "created_at": 1.0, "updated_at": 2.0, + "messages": _msgs(n)} + p = session_store / f"{sid}.json" + p.write_text(json.dumps(doc, indent=2), encoding="utf-8") + return p + + +def test_legacy_sidecar_without_message_count_still_backs_up_on_shrink(session_store): + """The prefix has no count to give, so this must fall back to the full + parse and keep the safeguard. Silently treating "no count" as "no shrink" + would reopen #1558.""" + _write_legacy_without_count(session_store, "l1", 5) + s = M.Session(session_id="l1", title="Legacy", workspace=str(session_store.parent), + model="glm", messages=_msgs(3)) + + s.save() + + bak = s.path.with_suffix(".json.bak") + assert bak.exists(), "fail-open: a legacy file must still be backed up on shrink" + assert len(json.loads(bak.read_text(encoding="utf-8"))["messages"]) == 5 + + +def test_file_changed_on_disk_since_last_save_falls_back_and_still_backs_up(session_store): + """An externally grown file. This process saved 3 messages; something else + then rewrote the file with 6 (an external appender, another process, a + restore). The next save with 4 is a real shrink relative to DISK, and the + count that governs it is the one the file now carries (6), not the 3 this + process remembers: the `.bak` must hold the 6-message array.""" + s = _make(session_store, "x1", 3) + external = {"session_id": "x1", "title": "T", "workspace": str(session_store.parent), + "model": "glm", "created_at": 1.0, "updated_at": 2.0, + "message_count": 6, "messages": _msgs(6)} + s.path.write_text(json.dumps(external, indent=2), encoding="utf-8") + + s.messages = _msgs(4) + s.save() + + bak = s.path.with_suffix(".json.bak") + assert bak.exists(), "an externally grown file must still be backed up on shrink" + assert len(json.loads(bak.read_text(encoding="utf-8"))["messages"]) == 6 + + +def test_file_changed_on_disk_without_a_count_still_backs_up_on_shrink(session_store): + """The same external rewrite, but the replacement carries no + `message_count` (an older writer, a hand-edited file). No prefix count -> + the full parse runs and the 6-message array is still backed up.""" + s = _make(session_store, "x2", 3) + external = {"session_id": "x2", "title": "T", "workspace": str(session_store.parent), + "model": "glm", "created_at": 1.0, "updated_at": 2.0, + "messages": _msgs(6)} + s.path.write_text(json.dumps(external, indent=2), encoding="utf-8") + + s.messages = _msgs(4) + s.save() + + bak = s.path.with_suffix(".json.bak") + assert bak.exists(), "fail-open: a count-less external rewrite must still be backed up" + assert len(json.loads(bak.read_text(encoding="utf-8"))["messages"]) == 6 + + +def test_unmarked_stale_count_from_a_foreign_writer_still_backs_up_on_shrink(session_store): + """A persisted count with no `_mc_v` marker is not trusted. + + The exact shape review (nesquena-hermes, 2026-09-22) pinned: a sidecar + materialized by an OLDER writer can carry a count stale against the + messages array beside it (the recovery writer used to copy a denormalized + state.db count over real rows). Here the foreign file says 2 but really + holds 6; a save of 4 must not read that 2 as "the save grows the file" and + skip the #1558 backup. The marker gate makes the prefix reader refuse the + unmarked count, the full parse sees the real array, and the 6 messages + land in the `.bak`. Remove the gate in `_prefix_message_count` and this + test fails with no backup written. + """ + s = _make(session_store, "x3", 3) + foreign = {"session_id": "x3", "title": "T", "workspace": str(session_store.parent), + "model": "glm", "created_at": 1.0, "updated_at": 2.0, + "message_count": 2, "messages": _msgs(6)} # stale count, NO _mc_v + s.path.write_text(json.dumps(foreign, indent=2), encoding="utf-8") + + s.messages = _msgs(4) + s.save() + + bak = s.path.with_suffix(".json.bak") + assert bak.exists(), "an unmarked stale count must not hide the shrink" + assert len(json.loads(bak.read_text(encoding="utf-8"))["messages"]) == 6 + assert len(M.Session.load("x3").messages) == 4 + + +def test_wrong_marker_version_still_backs_up_on_shrink(session_store): + """A count marked by a DIFFERENT writer contract is equally untrusted.""" + s = _make(session_store, "x4", 3) + foreign = {"session_id": "x4", "title": "T", "workspace": str(session_store.parent), + "model": "glm", "created_at": 1.0, "updated_at": 2.0, + "_mc_v": M._MESSAGE_COUNT_MARKER + 1, + "message_count": 2, "messages": _msgs(6)} + s.path.write_text(json.dumps(foreign, indent=2), encoding="utf-8") + + s.messages = _msgs(4) + s.save() + + bak = s.path.with_suffix(".json.bak") + assert bak.exists(), "a count from another writer version must not hide the shrink" + assert len(json.loads(bak.read_text(encoding="utf-8"))["messages"]) == 6 + + +def test_first_save_remarks_the_file_and_the_fast_path_resumes(session_store, monkeypatch): + """The gate costs one full parse, once. After a save by the current writer + the file carries `_mc_v`, and the cheap prefix path is back in service.""" + s = _make(session_store, "x5", 3) + foreign = {"session_id": "x5", "title": "T", "workspace": str(session_store.parent), + "model": "glm", "created_at": 1.0, "updated_at": 2.0, + "message_count": 2, "messages": _msgs(6)} # unmarked foreign file + s.path.write_text(json.dumps(foreign, indent=2), encoding="utf-8") + + s.messages = _msgs(6) + s.save() # equal-count save; unmarked prefix count ignored, full parse, no shrink + + assert json.loads(s.path.read_text(encoding="utf-8"))["_mc_v"] == M._MESSAGE_COUNT_MARKER + + calls = _spy_full_reads(monkeypatch, s.path) + s.messages = _msgs(8) + s.save() + assert calls["n"] == 0, "a marked file must take the bounded-prefix path again" + assert not s.path.with_suffix(".json.bak").exists() + + +def _tool_partial(ts=123): + """The exact shape the #2592 collapse recognises (copied from its test).""" + return {"role": "assistant", "content": "", "_partial": True, "timestamp": ts, + "reasoning": "same reasoning", + "_partial_tool_calls": [{"name": "execute_code", + "args": {"code": "raise RuntimeError('boom')"}, + "done": True, "is_error": True, "duration": 3.87}]} + + +def test_collapse_self_heal_on_load_still_backs_up_the_pre_collapse_array(session_store): + """#2592: load() de-duplicates adjacent partials and immediately saves the + shorter transcript, and that save must produce a `.bak` because it shrinks + the array on purpose. The count that decides it comes from the file's own + prefix, which the writer stamped with the length it wrote (5) -- the + ON-DISK length, not the post-collapse length of the object (3). Read the + object's own length instead and this backup silently disappears, so the + fixture carries the count a real save() would have written.""" + doc = {"session_id": "h1", "title": "T", "workspace": str(session_store.parent), + "model": "glm", "created_at": 1.0, "updated_at": 2.0, + "_mc_v": M._MESSAGE_COUNT_MARKER, + "message_count": 5, + "messages": [{"role": "user", "content": "run this"}, + _tool_partial(), _tool_partial(), _tool_partial(), + {"role": "assistant", "content": "**Task cancelled.**", "_error": True}]} + (session_store / "h1.json").write_text(json.dumps(doc), encoding="utf-8") + + loaded = M.Session.load("h1") + + assert sum(1 for m in loaded.messages if m.get("_partial")) == 1, "the collapse must have fired" + assert len(loaded.messages) == 3 + bak = (session_store / "h1.json").with_suffix(".json.bak") + assert bak.exists(), "the self-heal shrink must leave the pre-collapse array recoverable" + assert len(json.loads(bak.read_text(encoding="utf-8"))["messages"]) == 5 + + +# ── the collision a stat identity cannot see (review, 2026-09-21) ─────────── + +def test_same_length_in_place_rewrite_inside_one_mtime_tick_still_backs_up(session_store): + """A same-length, same-mtime, same-inode rewrite must not hide a shrink. + + The removed revision of this PR trusted an in-memory (inode, size, + mtime_ns) identity: "this object wrote the file, stat says nothing changed, + so the on-disk length is the one I remember". A stat tuple is not a content + identity. Another writer replaces the sidecar IN PLACE with a longer + transcript that occupies the same number of bytes, inside one mtime tick, + so all three fields still match -- and the next shrinking save then reads + its stale count as "growing", overwrites the longer transcript and writes + no `.bak`. That is the #1558 data-loss shape the safeguard exists to + prevent, reachable on the branch, which is why the identity cache is gone. + + The count now comes from the file's own metadata prefix, which any rewrite + of the bytes has to carry, so the shrink is seen for what it is. + """ + p = session_store / "i1.json" + base = {"session_id": "i1", "title": "T", "workspace": str(session_store.parent), + "model": "glm", "created_at": 1.0, "updated_at": 2.0} + + def _render(n): + return json.dumps({**base, "_mc_v": M._MESSAGE_COUNT_MARKER, + "message_count": n, "messages": _msgs(n)}, + indent=2, ensure_ascii=False) + + width = max(len(_render(2)), len(_render(5))) + short, grown = _render(2).ljust(width), _render(5).ljust(width) + assert len(short) == len(grown) == width + p.write_text(short, encoding="utf-8") + + loaded = M.Session.load("i1") # the only version this object knows + before = p.stat() + + p.write_text(grown, encoding="utf-8") # same inode, same byte length... + os.utime(p, ns=(before.st_atime_ns, before.st_mtime_ns)) + after = p.stat() + assert (after.st_ino, after.st_size, after.st_mtime_ns) == \ + (before.st_ino, before.st_size, before.st_mtime_ns), \ + "premise: the rewrite must be invisible to (inode, size, mtime_ns)" + assert len(loaded.messages) == 2, "premise: this object remembers 2 messages" + + loaded.messages = _msgs(3) + loaded.save() + + bak = p.with_suffix(".json.bak") + assert bak.exists(), "the 5-message transcript must be recoverable" + assert len(json.loads(bak.read_text(encoding="utf-8"))["messages"]) == 5 + assert len(M.Session.load("i1").messages) == 3 diff --git a/tests/test_session_db_sidecar_reconciliation.py b/tests/test_session_db_sidecar_reconciliation.py index 8b00892b..ba27e574 100644 --- a/tests/test_session_db_sidecar_reconciliation.py +++ b/tests/test_session_db_sidecar_reconciliation.py @@ -45,9 +45,54 @@ def test_recover_missing_sidecars_from_state_db_materializes_webui_row(tmp_path) assert data["parent_session_id"] == "parent-1" assert data["source_tag"] == "webui" assert data["session_source"] == "webui" + assert data["message_count"] == 2 assert [m["content"] for m in data["messages"]] == ["message 1", "message 2"] +def test_recovered_sidecar_derives_message_count_from_materialized_rows(tmp_path, monkeypatch): + """Recovery must not copy a stale state.db summary into a live sidecar. + + `Session.save()` relies on sidecar `message_count` for both #1558 guards. + A recovery row whose denormalized DB count says zero but whose canonical + message rows contain five messages must therefore refuse an empty active + snapshot and preserve a five-message backup on a later shrink. + """ + import api.models as models + + sid = _make_state_db(tmp_path / "state.db", sid="recovered_stale_count", messages=5) + with sqlite3.connect(tmp_path / "state.db") as conn: + conn.execute("UPDATE sessions SET message_count = 0 WHERE id = ?", (sid,)) + + assert recover_missing_sidecars_from_state_db(tmp_path, tmp_path / "state.db")["materialized"] == 1 + sidecar = tmp_path / f"{sid}.json" + assert json.loads(sidecar.read_text(encoding="utf-8"))["message_count"] == 5 + + monkeypatch.setattr(models, "SESSION_DIR", tmp_path) + monkeypatch.setattr(models, "SESSION_INDEX_FILE", tmp_path / "_index.json") + active_empty = models.Session( + session_id=sid, + title="Recovered from DB", + workspace=str(tmp_path), + model="openai/gpt-5", + messages=[], + active_stream_id="a" * 32, + pending_user_message="still typing", + ) + active_empty.save() + assert len(json.loads(sidecar.read_text(encoding="utf-8"))["messages"]) == 5 + + shrinking = models.Session( + session_id=sid, + title="Recovered from DB", + workspace=str(tmp_path), + model="openai/gpt-5", + messages=[{"role": "user", "content": f"replacement {i}"} for i in range(3)], + ) + shrinking.save() + backup = sidecar.with_suffix(".json.bak") + assert len(json.loads(backup.read_text(encoding="utf-8"))["messages"]) == 5 + + def test_recover_missing_sidecars_from_state_db_skips_deleted_webui_tombstone(tmp_path, monkeypatch): import api.models as _m monkeypatch.setattr(_m, "SESSION_DIR", tmp_path) diff --git a/tests/test_session_lease_wait_warnings.py b/tests/test_session_lease_wait_warnings.py new file mode 100644 index 00000000..e3b97d4c --- /dev/null +++ b/tests/test_session_lease_wait_warnings.py @@ -0,0 +1,126 @@ +"""Agent session turn-lease waits surface as WebUI ``warning`` events. + +When another Hermes process (gateway, CLI, cron) holds a session's turn lease, +the Agent's ``agent/turn_facade_lease.py`` emits: + +* ``_emit_status`` (kind ``lifecycle``): + "⏳ Another Hermes process is using this session; waiting for it to finish + before starting your turn..." then "⏳ Still waiting for the other Hermes + process on this session (Ns)..." and, on admission, "Session is free; loading + the latest transcript..."; +* ``_emit_warning`` (kind ``warn``) on timeout: "⏳ Another Hermes process kept + this session busy too long. Your message was not processed - ...". + +``_agent_status_callback`` used to drop every one of these, so a WebUI turn +looked stuck with no explanation. They are now relayed as +``{'type': 'session_lease_wait'}`` warnings shown by the existing +messages.js ``warning`` listener. +""" +import ast +import pathlib + +import pytest + +from api import streaming + +REPO = pathlib.Path(__file__).resolve().parents[1] + +AGENT_WAIT = ( + "⏳ Another Hermes process is using this session; " + "waiting for it to finish before starting your turn..." +) +AGENT_STILL = "⏳ Still waiting for the other Hermes process on this session (30s)..." +AGENT_FREE = "Session is free; loading the latest transcript..." +AGENT_TIMEOUT = ( + "⏳ Another Hermes process kept this session busy too long. Your message was not " + "processed - wait for the other process to finish, then send it again." +) + + +@pytest.mark.parametrize( + "kind,text,expected", + [ + ("lifecycle", AGENT_WAIT, True), + ("lifecycle", AGENT_STILL, True), + ("lifecycle", AGENT_FREE, True), + ("warn", AGENT_TIMEOUT, True), + ("LIFECYCLE", AGENT_WAIT, True), + # Never classify by text alone: other kinds / user text stay dropped. + ("user", AGENT_WAIT, False), + ("", AGENT_WAIT, False), + ("lifecycle", "Compressing conversation history", False), + ("lifecycle", "Rate limited — switching to fallback", False), + ("lifecycle", "", False), + ], +) +def test_session_lease_wait_classification(kind, text, expected): + assert streaming._is_session_lease_wait_message(kind, text) is expected + + +def _build_status_callback(): + """Extract ``_agent_status_callback`` from ``_run_agent_streaming`` and bind + it to a recording ``put`` so the real relay logic is exercised.""" + src = (REPO / "api" / "streaming.py").read_text(encoding="utf-8") + tree = ast.parse(src) + fn = next( + node + for node in ast.walk(tree) + if isinstance(node, ast.FunctionDef) and node.name == "_agent_status_callback" + ) + # The callback is a closure inside ``_run_agent_streaming`` and may declare + # ``nonlocal`` state (e.g. the runtime-model dedupe identity it resets on a + # fallback warning). Compiled alone at module level, ``nonlocal`` has no + # binding and raises SyntaxError, so wrap it in an enclosing function that + # binds every nonlocal name it declares, then return the inner callback. + nonlocals = sorted({ + name + for node in ast.walk(fn) + if isinstance(node, ast.Nonlocal) + for name in node.names + }) + binds = "".join(f" {name} = None\n" for name in nonlocals) + outer = ast.parse(f"def _status_callback_scope():\n{binds} pass\n").body[0] + assert isinstance(outer, ast.FunctionDef) + outer.body[-1:] = [fn, ast.Return(value=ast.Name(id=fn.name, ctx=ast.Load()))] + module = ast.fix_missing_locations(ast.Module(body=[outer], type_ignores=[])) + events = [] + ns = { + "put": lambda event, data: events.append((event, data)), + "session_id": "sid-1", + "_captured_terminal_error": [None], + "_is_agent_compression_start_status": streaming._is_agent_compression_start_status, + "_is_fallback_lifecycle_message": streaming._is_fallback_lifecycle_message, + "_is_session_lease_wait_message": streaming._is_session_lease_wait_message, + } + exec(compile(module, "streaming_status_callback", "exec"), ns) + return ns["_status_callback_scope"](), events + + +@pytest.mark.parametrize( + "kind,text", + [ + ("lifecycle", AGENT_WAIT), + ("lifecycle", AGENT_STILL), + ("lifecycle", AGENT_FREE), + ("warn", AGENT_TIMEOUT), + ], +) +def test_status_callback_relays_lease_wait_as_warning(kind, text): + callback, events = _build_status_callback() + callback(kind, text) + assert events == [("warning", {"type": "session_lease_wait", "message": text})] + + +def test_status_callback_still_drops_unrelated_lifecycle_chatter(): + callback, events = _build_status_callback() + callback("lifecycle", "Loaded 12 tools") + callback("user", AGENT_WAIT) + assert events == [] + + +def test_status_callback_fallback_warning_unchanged(): + callback, events = _build_status_callback() + callback("lifecycle", "Rate limited — switching to fallback model") + assert events == [ + ("warning", {"type": "fallback", "message": "Rate limited — switching to fallback model"}) + ] diff --git a/tests/test_session_lineage_collapse.py b/tests/test_session_lineage_collapse.py index 2f01abf8..9b31896e 100644 --- a/tests/test_session_lineage_collapse.py +++ b/tests/test_session_lineage_collapse.py @@ -925,6 +925,9 @@ eval(extractFunc('_sessionLineageKey')); eval(extractFunc('_sidebarLineageKeyForRow')); eval(extractFunc('_collapseSessionLineageForSidebar')); eval(extractFunc('_attachChildSessionsToSidebarRows')); +eval(extractFunc('_isDelegatedSubagentRow')); +eval(extractFunc('_sidebarProjectResolver')); +eval(extractFunc('_sidebarRowsById')); eval(extractFunc('_scopedSidebarReferenceRows')); eval(extractFunc('_renderSidebarRowsFromRawSessions')); const child = {{session_id:'child', title:'Subagent', parent_session_id:'parent', relationship_type:'child_session', updated_at:20, last_message_at:20, source:'webui'}}; @@ -973,6 +976,9 @@ eval(extractFunc('_sessionLineageKey')); eval(extractFunc('_sidebarLineageKeyForRow')); eval(extractFunc('_collapseSessionLineageForSidebar')); eval(extractFunc('_attachChildSessionsToSidebarRows')); +eval(extractFunc('_isDelegatedSubagentRow')); +eval(extractFunc('_sidebarProjectResolver')); +eval(extractFunc('_sidebarRowsById')); eval(extractFunc('_scopedSidebarReferenceRows')); eval(extractFunc('_renderSidebarRowsFromRawSessions')); const child = {{session_id:'child', title:'Fork in projA', parent_session_id:'parent', relationship_type:'child_session', project_id:'projA', message_count:3, updated_at:20, last_message_at:20, source:'webui'}}; diff --git a/tests/test_sidebar_session_partition.py b/tests/test_sidebar_session_partition.py index 32293bc6..f0e7fcff 100644 --- a/tests/test_sidebar_session_partition.py +++ b/tests/test_sidebar_session_partition.py @@ -31,11 +31,11 @@ def test_render_uses_single_pass_partition_helper(): render_body = _function_block("renderSessionListFromCache") assert "_partitionSidebarSessionRows(allMatched, activeSidForSidebar)" in render_body - assert "_renderSidebarRowsFromRawSessions(sessionsRaw, [...referenceRaw, ..._scopedSidebarReferenceRows(isCliView)])" in render_body + assert "_renderSidebarRowsFromRawSessions(sessionsRaw, [...referenceRaw, ..._scopedSidebarReferenceRows(isCliView, projectIdFor)])" in render_body assert "const renderedWebuiSessionCount=_serverWebuiSessionCount===null" in render_body assert "const renderedCliSessionCount=_serverCliSessionCount===null" in render_body - assert "? _renderSidebarRowsFromRawSessions(webuiSessionsRaw, [...webuiReferenceRaw, ..._scopedSidebarReferenceRows(false)]).length" in render_body - assert "? _renderSidebarRowsFromRawSessions(cliSessionsRaw, [...cliReferenceRaw, ..._scopedSidebarReferenceRows(true)]).length" in render_body + assert "? _renderSidebarRowsFromRawSessions(webuiSessionsRaw, [...webuiReferenceRaw, ..._scopedSidebarReferenceRows(false, projectIdFor)]).length" in render_body + assert "? _renderSidebarRowsFromRawSessions(cliSessionsRaw, [...cliReferenceRaw, ..._scopedSidebarReferenceRows(true, projectIdFor)]).length" in render_body assert ": null;" in render_body assert "null is a deliberate \"not computed\" sentinel" in render_body assert "const webuiSessionTabCount=_sessionSourceTabCount('webui', renderedWebuiSessionCount, renderedCliSessionCount);" in render_body @@ -72,8 +72,8 @@ def test_partition_helper_keeps_raw_source_counts_while_render_owns_visible_coun assert "cliSessionsRaw," in _partition_block() assert "const renderedWebuiSessionCount=_serverWebuiSessionCount===null" in render_body assert "const renderedCliSessionCount=_serverCliSessionCount===null" in render_body - assert "? _renderSidebarRowsFromRawSessions(webuiSessionsRaw, [...webuiReferenceRaw, ..._scopedSidebarReferenceRows(false)]).length" in render_body - assert "? _renderSidebarRowsFromRawSessions(cliSessionsRaw, [...cliReferenceRaw, ..._scopedSidebarReferenceRows(true)]).length" in render_body + assert "? _renderSidebarRowsFromRawSessions(webuiSessionsRaw, [...webuiReferenceRaw, ..._scopedSidebarReferenceRows(false, projectIdFor)]).length" in render_body + assert "? _renderSidebarRowsFromRawSessions(cliSessionsRaw, [...cliReferenceRaw, ..._scopedSidebarReferenceRows(true, projectIdFor)]).length" in render_body assert "function _countRenderedSidebarRowsFromRawSessions" not in SESSIONS_JS assert "function _renderSidebarRowsFromRawSessions(sessionsRaw, referenceSessionsRaw){" in SESSIONS_JS assert "_attachChildSessionsToSidebarRows(_collapseSessionLineageForSidebar(sessionsRaw), sessionsRaw, referenceRows)" in SESSIONS_JS diff --git a/tests/test_sidebar_unassigned_filter.py b/tests/test_sidebar_unassigned_filter.py index 2d19a7ee..062a7bc5 100644 --- a/tests/test_sidebar_unassigned_filter.py +++ b/tests/test_sidebar_unassigned_filter.py @@ -61,7 +61,7 @@ def test_unassigned_chip_filter_logic(): assert "if(_activeProject===NO_PROJECT_FILTER){" in js, ( "The Unassigned filter must select sessions without a project_id" ) - assert "if(s.project_id) continue;" in js, ( + assert "if(projectId) continue;" in js, ( "The Unassigned filter must skip sessions with a project_id" ) @@ -76,7 +76,7 @@ def test_unassigned_chip_only_shown_when_relevant(): — though that's vanishingly rare). """ js = _js() - assert "const hasUnprojected=profileFiltered.some(s=>!s.project_id);" in js, ( + assert "const hasUnprojected=_sidebarHasUnprojectedRows(profileFiltered, projectIdFor);" in js, ( "The render function must compute whether unassigned sessions exist" ) assert "if(_allProjects.length>0||hasUnprojected){" in js, ( diff --git a/tests/test_smd_media_in_stream.py b/tests/test_smd_media_in_stream.py index d123ef12..49024d2d 100644 --- a/tests/test_smd_media_in_stream.py +++ b/tests/test_smd_media_in_stream.py @@ -386,7 +386,10 @@ class TestSmdMediaInStream(unittest.TestCase): idx = MESSAGES_JS.index("function _smdMediaAwareAddText") block = MESSAGES_JS[idx:idx + 6500] self.assertIn("function _smdMediaRefHasReliableBoundary", MESSAGES_JS) - self.assertIn("matchEnd===combined.length", block) + # #7708: the scan now runs over `normalized` (backtick-wrapped refs + # rewritten to bare form first); the chunk-end boundary guard is the + # same check against that string. + self.assertIn("matchEnd===normalized.length", block) self.assertIn("!_smdMediaRefHasReliableBoundary(m[1])", block) self.assertIn("unmatchedTail = candidate", block) diff --git a/tests/test_subagent_project_filter.py b/tests/test_subagent_project_filter.py new file mode 100644 index 00000000..eb2bfcbb --- /dev/null +++ b/tests/test_subagent_project_filter.py @@ -0,0 +1,188 @@ +"""Delegated subagent children must follow their parent's project filter. + +Subagent child sessions are created by the agent (state.db) and never carry a +``project_id`` of their own. The sidebar project filter compared each row's own +``project_id``, so selecting a project dropped every subagent child and the +parent rendered with no stacked children, while the Unassigned view kept them. +""" +import json +import shutil +import subprocess +from pathlib import Path + +import pytest + +REPO_ROOT = Path(__file__).parent.parent.resolve() +SESSIONS_JS = (REPO_ROOT / "static" / "sessions.js").read_text(encoding="utf-8") +NODE = shutil.which("node") + +pytestmark = pytest.mark.skipif(NODE is None, reason="node not on PATH") + +_PREAMBLE = """ +const src = %s; +function extractFunc(name) { + const start = src.search(new RegExp('function\\\\s+' + name + '\\\\s*\\\\(')); + if (start < 0) throw new Error(name + ' not found'); + let i = src.indexOf('{', start) + 1, depth = 1; + while (depth > 0 && i < src.length) { if (src[i] === '{') depth++; else if (src[i] === '}') depth--; i++; } + return src.slice(start, i); +} +function _isCliSession(s){ return !!(s && (s.is_cli_session || s.session_source==='cli')); } +function _isExternalSession(s){ return !!(s && (s.is_cli_session || s.session_source === 'messaging')); } +function _isMessagingSession(s){ return !!(s && s.session_source==='messaging'); } +function _hasUnreadForSession(s){ return !!(s && s.has_unread); } +function _sessionDisplayTitle(s){ return s && s.title; } +Object.assign(global, {_isCliSession, _isExternalSession, _isMessagingSession, _hasUnreadForSession, _sessionDisplayTitle}); +global.INFLIGHT = {}; +global.NO_PROJECT_FILTER = '__no_project__'; +global.window = {}; +global._archivedCliCount = 0; global._archivedWebuiCount = 0; +global._sidebarReferenceSessions = []; +global.S = { session: null, busy: false, activeStreamId: null }; +global._showArchived = false; +global._sessionSourceFilter = 'webui'; +for (const fn of ['_isSessionLocallyStreaming','_hasPendingUserMessageSignal','_isSessionEffectivelyStreaming', + '_isChildSession','_isForkWithResolvableParent','_sessionLineageKey','_sidebarLineageKeyForRow', + '_collapseSessionLineageForSidebar','_attachChildSessionsToSidebarRows','_sessionAttentionState', + '_sidebarRowHasVisibleMessages','_isDelegatedSubagentRow','_sidebarProjectResolver','_sidebarRowsById','_sidebarHasUnprojectedRows', + '_partitionSidebarSessionRows','_scopedSidebarReferenceRows','_renderSidebarRowsFromRawSessions']) { + eval.call(global, extractFunc(fn)); +} +const parent = { session_id:'proj_parent', title:'Parent', session_source:'webui', raw_source:'webui', source_tag:'webui', message_count:5, project_id:'projX', updated_at:100, last_message_at:100 }; +const child = { session_id:'sub_child', title:'Subagent Session', parent_session_id:'proj_parent', relationship_type:'child_session', raw_source:'subagent', source_tag:'subagent', session_source:'other', _parent_lineage_root_id:'proj_parent', _cross_surface_child_session:true, message_count:3, updated_at:101, last_message_at:101 }; +const grandchild = { session_id:'sub_grandchild', title:'Nested subagent', parent_session_id:'sub_child', relationship_type:'child_session', raw_source:'subagent', source_tag:'subagent', session_source:'other', message_count:2, updated_at:102, last_message_at:102 }; +function render(project, rows) { + global._activeProject = project; + const part = _partitionSidebarSessionRows(rows, null); + const out = _renderSidebarRowsFromRawSessions(part.sessionsRaw, part.webuiReferenceRaw); + const p = out.find(r => r.session_id === 'proj_parent') || {}; + return { raw: part.sessionsRaw.map(s => s.session_id).sort(), top: out.map(r => r.session_id), + children: (p._child_sessions || []).map(c => c.session_id) }; +} +""" + + +def _run(scenario: str) -> dict: + assert NODE is not None + source = (_PREAMBLE % json.dumps(SESSIONS_JS)) + scenario + result = subprocess.run([NODE], input=source, cwd=str(REPO_ROOT), + capture_output=True, text=True, encoding="utf-8", timeout=30) + if result.returncode != 0: + raise RuntimeError(result.stderr) + return json.loads(result.stdout.strip()) + + +def test_subagent_child_is_shown_inside_parent_project(): + out = _run("console.log(JSON.stringify(render('projX', [parent, child])));") + assert out["raw"] == ["proj_parent", "sub_child"] + assert out["top"] == ["proj_parent"] + assert out["children"] == ["sub_child"] + + +def test_nested_subagent_inherits_project_through_child(): + out = _run("console.log(JSON.stringify(render('projX', [parent, child, grandchild])));") + assert out["raw"] == ["proj_parent", "sub_child", "sub_grandchild"] + assert out["top"] == ["proj_parent"] + + +def test_subagent_of_project_parent_is_not_unassigned(): + out = _run("console.log(JSON.stringify(render(NO_PROJECT_FILTER, [parent, child])));") + assert out["raw"] == [] + assert out["top"] == [] + + +def test_subagent_excluded_from_other_project_and_own_project_id_wins(): + out = _run(""" +const other = render('projY', [parent, child]); +const explicit = render('projY', [parent, Object.assign({}, child, {project_id:'projY'})]); +console.log(JSON.stringify({other, explicit})); +""") + assert out["other"]["raw"] == [] + assert out["explicit"]["raw"] == ["sub_child"] + + +def test_parent_resolved_from_reference_rows(): + """Parent only present as a hidden reference row still scopes the child.""" + out = _run(""" +global._sidebarReferenceSessions = [parent]; +const rows = render('projX', [child]); +global._activeProject = 'projX'; +const scoped = _scopedSidebarReferenceRows(false).map(s => s.session_id); +console.log(JSON.stringify({raw: rows.raw, scoped})); +""") + assert out["raw"] == ["sub_child"] + assert out["scoped"] == ["proj_parent"] + + +def test_unassigned_chip_ignores_subagents_of_project_parents(): + out = _run("console.log(JSON.stringify(_sidebarHasUnprojectedRows([parent, child], _sidebarProjectResolver(_sidebarRowsById([[parent, child]])))));") + assert out is False + + +def test_unassigned_chip_uses_reference_parent(): + """A child whose project parent is only a reference row is not unassigned.""" + out = _run(""" +global._sidebarReferenceSessions = [parent]; +global._activeProject = null; +const part = _partitionSidebarSessionRows([child], null); +console.log(JSON.stringify({has: _sidebarHasUnprojectedRows(part.profileFiltered, part.projectIdFor), + unassigned: render(NO_PROJECT_FILTER, [child]).raw})); +""") + assert out["unassigned"] == [] + assert out["has"] is False + + +def test_unassigned_fork_stays_unassigned(): + """A fork moved to "No project" keeps project_id null; only subagents inherit.""" + out = _run(""" +const fork = { session_id:'fork_child', title:'Fork', parent_session_id:'proj_parent', relationship_type:'child_session', session_source:'fork', raw_source:'webui', source_tag:'webui', project_id:null, message_count:2, updated_at:103, last_message_at:103 }; +global._activeProject = null; +const p = _partitionSidebarSessionRows([parent, fork], null); +console.log(JSON.stringify({unassigned: render(NO_PROJECT_FILTER, [parent, fork]).raw, + project: render('projX', [parent, fork]).raw, + chip: _sidebarHasUnprojectedRows(p.profileFiltered, p.projectIdFor)})); +""") + assert out["unassigned"] == ["fork_child"] + assert out["project"] == ["proj_parent"] + assert out["chip"] is True + + +def test_project_resolution_is_linear_in_lineage_depth(): + """Each lineage resolves once: ancestor lookups stay O(n) for a deep chain.""" + out = _run(""" +function chain(n) { + const rows = [parent]; + for (let i = 0; i < n; i++) rows.push(Object.assign({}, grandchild, {session_id:'c'+i, + parent_session_id: i ? 'c'+(i-1) : 'proj_parent', updated_at:200+i, last_message_at:200+i})); + return rows; +} +function lookups(n) { + const rows = chain(n); + let gets = 0; + const realGet = Map.prototype.get; + Map.prototype.get = function(k) { gets++; return realGet.call(this, k); }; + try { + global._activeProject = 'projX'; + global._sidebarReferenceSessions = rows.slice(0, 50); + const part = _partitionSidebarSessionRows(rows, null); + _scopedSidebarReferenceRows(false, part.projectIdFor); + _sidebarHasUnprojectedRows(part.profileFiltered, part.projectIdFor); + return {gets, raw: part.sessionsRaw.length}; + } finally { Map.prototype.get = realGet; } +} +console.log(JSON.stringify({small: lookups(1000), big: lookups(4000)})); +""") + assert out["small"]["raw"] == 1001 and out["big"]["raw"] == 4001 + # 4x the rows must cost ~4x the lookups, not ~16x. + assert out["big"]["gets"] <= 5 * out["small"]["gets"], out + assert out["big"]["gets"] <= 20 * 4001, out + + +def test_project_resolver_is_cycle_safe(): + out = _run(""" +const a = Object.assign({}, grandchild, {session_id:'a', parent_session_id:'b'}); +const b = Object.assign({}, grandchild, {session_id:'b', parent_session_id:'a'}); +const resolve = _sidebarProjectResolver(_sidebarRowsById([[a, b]])); +console.log(JSON.stringify([resolve(a), resolve(b)])); +""") + assert out == [None, None] diff --git a/tests/test_webui_gateway_chat_backend.py b/tests/test_webui_gateway_chat_backend.py index 226fee6c..88b78f6b 100644 --- a/tests/test_webui_gateway_chat_backend.py +++ b/tests/test_webui_gateway_chat_backend.py @@ -25,6 +25,7 @@ from api.gateway_chat import ( webui_chat_backend_mode, webui_gateway_chat_enabled, ) +from api.turn_journal import derive_turn_journal_states, read_turn_journal def test_gateway_chat_backend_is_default_off_for_truthy_values(): @@ -397,6 +398,73 @@ def test_gateway_chat_worker_translates_sse_and_persists_session(tmp_path, monke assert all(len(item) == 3 and item[2] for item in events) +def test_gateway_chat_worker_records_turn_journal_completion(tmp_path, monkeypatch): + """#6366 re-gate: a successful Gateway run must record durable + same-stream completion evidence in the crash-safe turn journal. + + The stale-cancel recovery predicate derives its completion evidence + from the turn journal, so a Gateway run that persisted its final + answer but lost the run journal's terminal write would otherwise + have no completion evidence at all — and recovery would re-append a + duplicate recovered row after the valid final answer. + """ + session_dir = tmp_path / "sessions" + session_dir.mkdir() + monkeypatch.setattr(models, "SESSION_DIR", session_dir) + monkeypatch.setattr(models, "SESSION_INDEX_FILE", session_dir / "_index.json") + monkeypatch.setattr(models, "SESSIONS", OrderedDict()) + + class FakeResponse: + def __enter__(self): + return self + + def __exit__(self, exc_type, exc, tb): + return False + + def __iter__(self): + yield b'data: {"choices":[{"delta":{"content":"hello"}}]}\n\n' + yield b"data: [DONE]\n\n" + + monkeypatch.setenv("HERMES_WEBUI_GATEWAY_BASE_URL", "http://gateway.local") + monkeypatch.setenv("HERMES_WEBUI_GATEWAY_API_KEY", "secret-token") + monkeypatch.setattr( + gateway_chat.urllib.request, + "urlopen", + lambda req, timeout=0: FakeResponse(), + ) + + s = new_session() + stream_id = "stream-gateway-turn-journal-completed" + s.active_stream_id = stream_id + s.pending_user_message = "Say hello" + s.pending_attachments = [] + s.pending_started_at = 456 + s.save() + channel = create_stream_channel() + STREAMS[stream_id] = channel + + gateway_chat._run_gateway_chat_streaming( + s.session_id, + "Say hello", + "test-model", + str(tmp_path), + stream_id, + [], + ) + + journal = read_turn_journal(s.session_id, session_dir=session_dir) + states, _ = derive_turn_journal_states(journal.get("events") or []) + stream_events = [ + event + for event in states.values() + if str(event.get("stream_id") or "") == stream_id + ] + assert stream_events, "expected turn journal events for the gateway stream" + assert any( + event.get("event") == "completed" for event in stream_events + ), "the gateway success writeback must record a completed turn journal event" + + def test_gateway_chat_worker_classifies_terminal_provider_error_without_text(tmp_path, monkeypatch): """Gateway terminal errors must survive an empty assistant stream.""" from unittest.mock import MagicMock