Release v0.51.336 — Release KZ (fix per-token inline-thinking perf regression, #3633 follow-up) (#3854)

* fix(streaming): make per-token inline-thinking extraction linear (#3633 follow-up)

Codex post-merge perf catch on #3633: _parseStreamState() and
syncInflightAssistantMessage() call _extractInlineThinkingFromContent on the
FULL accumulated assistantText on EVERY streamed token. The #3633 rewrite made
that a full char-by-char walk, so cost was O(n^2) over a stream — a Node harness
measured ~88s (no-tag) / ~103s (leading <think> block) for 2000x100-char tokens,
which would freeze the main thread on long reasoning-model responses.

Two fixes (Python api/streaming.py + JS static/messages.js twin, line-for-line parity):
1. Fast path: if the text contains no complete thinking opener AND (when
   streaming) its tail is not a prefix of an opener, return unchanged without the
   char walk — two cheap substring scans. Handles the common no-tag case.
2. Bulk-skip plain trailing content: track the next complete opener via
   str.find/indexOf (_next_inline_thinking_opener / _nextThinkingOpener); once no
   opener remains ahead, append the remainder and stop instead of walking it
   (streaming still suppresses a trailing partial-opener prefix). Handles the
   leading-block-then-long-answer case.

Result: ~88s/103s → ~0.5s/0.9s (Python), ~0.18s/0.21s (JS). All behavioral cases
(persist + streaming, code-awareness, position-aware unclosed, leading whitespace)
verified unchanged in both twins. Added a per-token streaming perf regression test
and wired _nextThinkingOpener into the node driver harness.

Co-authored-by: rodboev <rodboev@users.noreply.github.com>

* fix(streaming): perf bulk-skip must respect code context for partial-opener tails

Codex catch on the perf fix: the no-complete-opener bulk-skip suppressed a
trailing partial opener (e.g. '<thi') unconditionally during streaming, but a
partial opener INSIDE inline-backtick / fenced / indented code must stay visible
(master parity). Now, when streaming and the tail is a partial opener, fall
through to the code-aware char walk (bounded — a partial tail is a transient
single token) instead of bulk-skipping; only a PLAIN-text partial opener is
suppressed as a forming block. Added _text_tail_is_partial_opener /
_textTailIsPartialOpener (Python + JS parity) + regression tests for the
inside-code vs plain partial-tail cases.

Co-authored-by: rodboev <rodboev@users.noreply.github.com>

* docs(changelog): v0.51.336 KZ — inline-thinking streaming perf fix

---------

Co-authored-by: rodboev <rodboev@users.noreply.github.com>
Co-authored-by: Hermes Agent <hermes-agent@nesquena-hermes.local>

api/streaming.py

@@ -1517,6 +1517,30 @@ def _inline_thinking_fence_marker_at(text, index):
     return ''
 
 
+def _next_inline_thinking_opener(text, start):
+    """Index of the earliest complete thinking opener at/after `start`, or -1.
+    Cheap str.find per opener — lets the scanner bulk-skip plain trailing content
+    instead of walking it char-by-char (#3633 Codex per-token perf catch)."""
+    best = -1
+    for open_tag, _close in _INLINE_THINKING_TAG_PAIRS:
+        i = text.find(open_tag, start)
+        if i != -1 and (best == -1 or i < best):
+            best = i
+    return best
+
+
+def _text_tail_is_partial_opener(text):
+    """True when the END of `text` is a non-empty proper prefix of some thinking
+    opener (e.g. ``<thi`` for ``<think>``). Used to decide whether a streaming
+    tail might be a forming block worth code-aware handling."""
+    for open_tag, _close in _INLINE_THINKING_TAG_PAIRS:
+        m = min(len(open_tag) - 1, len(text))
+        for n in range(m, 0, -1):
+            if open_tag.startswith(text[-n:]):
+                return True
+    return False
+
+
 def _line_is_indented_code(text, line_start):
     """True when the line beginning at `line_start` is a markdown indented code
     block line (>=4 leading spaces or a leading tab, and not blank). `line_start`
@@ -1570,6 +1594,26 @@ def _extract_inline_thinking_from_content(raw_content, existing_reasoning='', *,
     text = '' if raw_content is None else str(raw_content)
     if not text:
         return text, str(existing_reasoning or '').strip()
+    # Fast path (#3633 Codex perf catch — _parseStreamState / syncInflight call
+    # this on the FULL accumulator on every streamed token, so the common no-tag
+    # case must not do the O(length) char walk per call). If the text contains no
+    # complete thinking opener AND — when streaming — its tail is not a prefix of
+    # any opener (a partial opener mid-stream), there is nothing to extract:
+    # return the text unchanged. Two cheap substring scans instead of a full walk.
+    if not any(open_tag in text for open_tag, _close in _INLINE_THINKING_TAG_PAIRS):
+        tail_is_partial_opener = False
+        if streaming:
+            for open_tag, _close in _INLINE_THINKING_TAG_PAIRS:
+                # Does the END of text look like the START of an opener?
+                max_prefix = min(len(open_tag) - 1, len(text))
+                for n in range(max_prefix, 0, -1):
+                    if open_tag.startswith(text[-n:]):
+                        tail_is_partial_opener = True
+                        break
+                if tail_is_partial_opener:
+                    break
+        if not tail_is_partial_opener:
+            return text, str(existing_reasoning or '').strip()
     visible = []
     extracted = []
     cursor = 0
@@ -1589,7 +1633,30 @@ def _extract_inline_thinking_from_content(raw_content, existing_reasoning='', *,
     # indented code / whitespace and has NO leading thinking wrapper keeps its
     # leading whitespace — #3633 Codex catch).
     leading_removed = False
+    # Index of the next opener at/after `index` (recomputed only when we pass it).
+    # When no opener remains ahead, the rest of the text is plain and can be
+    # appended in one slice — this keeps a stream that DID contain a leading
+    # thinking block from re-walking the whole growing answer tail every token
+    # (#3633 Codex perf catch: the per-token full walk was O(n^2) over a stream).
+    next_opener = _next_inline_thinking_opener(text, 0)
     while index < length:
+        if next_opener == -1 or index > next_opener:
+            next_opener = _next_inline_thinking_opener(text, index)
+        if next_opener == -1:
+            # No further COMPLETE opener ahead. The remaining tail is plain
+            # visible content and can be appended in one slice — EXCEPT during
+            # streaming when the tail is a prefix of an opener (e.g. "...<thi"):
+            # that may be a forming block and must be suppressed, but ONLY if it
+            # is outside code context (a partial opener inside inline-backtick /
+            # fenced / indented code stays visible — master parity). Determining
+            # code state needs the char walk, so in that case fall through to the
+            # normal loop (bounded — a partial tail is a transient single token)
+            # rather than bulk-skipping. Otherwise stop (avoids re-walking the
+            # growing answer tail every token — #3633 perf catch).
+            if streaming and _text_tail_is_partial_opener(text):
+                pass  # fall through to the code-aware char walk for the tail
+            else:
+                break
         ch = text[index]
         if index > 0 and text[index - 1] == '\n':
             line_is_indented_code = _line_is_indented_code(text, index)