Release v0.51.222 — Release GP (stage-p4 — backend bugfix batch: title language drift #3293 + orphaned CLI sidecar prune #3238 + pin-quota lineage #3288) (#3452)

* fix: reject cross-script drifted auto-generated session titles (#3293)

The title-language mismatch guard only knew two states: German (de) or empty,
and _title_language_mismatch early-returned False whenever the user start
wasn't German. So an English conversation whose LLM-generated title came back
in Chinese / Spanish / Russian sailed through and persisted with
llm_title_generated=true. The German case was the only one covered because
that's the one prior report it was built for.

Generalize from a German-specific binary to a language-agnostic cross-script
check. Add _script_counts() + _dominant_script() (cheap, dependency-free
Unicode-block classification: latin / cjk / cyrillic / arabic / hebrew / greek
/ devanagari). _title_language_mismatch now rejects a title that introduces a
substantial amount (>=35% of alphabetic chars, min 2) of a script different
from the conversation start's dominant script — so short titles that embed a
borrowed Latin technical term still trip, while an English title with a single
foreign place-name does not. The legacy German->English same-script heuristic
is preserved verbatim.

Kept api/streaming.py ASCII-only (the test_title_generation_source_has_no_cjk_
literals guard) — all CJK examples live in the test file, not the source.

Closes #3293

Co-authored-by: andrewkangkr <andrewkangkr@users.noreply.github.com>

* fix: prune orphaned imported-CLI sidecars from the WebUI sidebar (#3238)

When a CLI/agent session is opened in WebUI it gets a WebUI-owned sidecar
(webui/sessions/<id>.json + _index.json row) so it can render and reopen;
all_sessions() then returns it independently of the agent state.db. If the
user later deletes that session from the CLI / local Hermes storage, nothing
pruned the sidecar — the merge loop only overlays CLI metadata when a matching
state.db row exists and otherwise continues, so the stale row lingered in the
sidebar indefinitely (there is no WebUI delete affordance for CLI rows).

Add api.models.agent_session_row_exists(): an exact, uncapped existence probe
against the state.db sessions table. The sidebar merge loop now drops a row
that is_cli_session_row + not WebUI-native + absent from cli_by_id + whose
state.db row is genuinely gone, and calls prune_session_from_index() so
_index.json self-heals.

The state.db probe is deliberate: get_cli_sessions() caps at
CLI_VISIBLE_SESSION_LIMIT (20), so a still-existing session can fall out of
that window and look deleted — pruning on cli_by_id absence alone would delete
live sessions. WebUI-native rows with a CLI ancestor are never pruned, and any
probe error degrades to keep-the-row so a transient failure can't lose data.

Closes #3238

Co-authored-by: Luxciax <Luxciax@users.noreply.github.com>

* fix: count pin quota by visible session lineage

* docs(changelog): v0.51.222 — backend bugfix batch (#3293 title drift, #3238 sidecar prune, #3288 pin lineage)

* fix(pins): forks count as own pin lineage, not collapsed to parent (#3288 Codex follow-up)

Codex review of the batch found a pin-limit UNDERCOUNT: _session_row_lineage_root_id
followed any parent_session_id to the root, but /api/session/branch creates
independent visible fork sessions that also carry parent_session_id (session_source=
'fork'). Two pinned forks of the same parent collapsed to one quota lineage, letting
a user exceed pinned_sessions_limit with no 400. Fix: a fork returns its own id as
its lineage root (it's a separately-visible session); only compression/continuation
rows still collapse to a shared root. Adds a regression test with two pinned forks +
the parent counting as three distinct lineages, and confirms the existing
pre-compression-snapshot collapse case still passes.

* test(pins): update #2508/#2821 source-match tests for #3288 lineage rename

#3288 replaced the raw-session-id pin counter (pinned_ids set) with a
visible-lineage counter (pinned_lineage_ids via _visible_pinned_lineage_ids over
persisted_rows/candidate_rows). Two pre-existing source-string-matching tests
asserted the OLD implementation literals (pinned_ids = {, _session_field(existing,
session_id...), len(pinned_ids) >=). Updated both to assert the new mechanism while
preserving the invariants they actually guard: snapshot computed BEFORE LOCK (no
all_sessions()-inside-LOCK deadlock), quota filtering via the shared
_session_counts_toward_pin_quota helper, and the limit/400 guard. Behaviour
unchanged; these were implementation-detail assertions, not behaviour tests.

---------

Co-authored-by: nesquena-hermes <[email protected]>
Co-authored-by: andrewkangkr <andrewkangkr@users.noreply.github.com>
Co-authored-by: Luxciax <Luxciax@users.noreply.github.com>
Co-authored-by: Andy Kang <andrewkang.kr@gmail.com>

api/streaming.py

@@ -1576,24 +1576,109 @@ def _detect_title_language(text: str) -> str:
     return ''
 
 
+def _script_counts(text: str) -> dict:
+    """Return per-script alphabetic character counts for *text*.
+
+    Buckets: ``latin``, ``cjk`` (Han/Hiragana/Katakana/Hangul), ``cyrillic``,
+    ``arabic``, ``hebrew``, ``greek``, ``devanagari``. Non-alphabetic and
+    unclassified characters are ignored.
+    """
+    counts: dict[str, int] = {}
+    for ch in str(text or ''):
+        if not ch.isalpha():
+            continue
+        o = ord(ch)
+        if (0x0041 <= o <= 0x024F) or (0x1E00 <= o <= 0x1EFF):
+            bucket = 'latin'
+        elif (
+            (0x4E00 <= o <= 0x9FFF) or (0x3400 <= o <= 0x4DBF)   # Han
+            or (0x3040 <= o <= 0x30FF)                            # Hiragana/Katakana
+            or (0xAC00 <= o <= 0xD7A3) or (0x1100 <= o <= 0x11FF) # Hangul
+        ):
+            bucket = 'cjk'
+        elif 0x0400 <= o <= 0x04FF:
+            bucket = 'cyrillic'
+        elif (0x0600 <= o <= 0x06FF) or (0x0750 <= o <= 0x077F):
+            bucket = 'arabic'
+        elif 0x0590 <= o <= 0x05FF:
+            bucket = 'hebrew'
+        elif 0x0370 <= o <= 0x03FF:
+            bucket = 'greek'
+        elif 0x0900 <= o <= 0x097F:
+            bucket = 'devanagari'
+        else:
+            continue
+        counts[bucket] = counts.get(bucket, 0) + 1
+    return counts
+
+
+def _dominant_script(text: str) -> str:
+    """Return a coarse writing-script bucket for *text*, or '' when undecidable.
+
+    Script-level (not language-level) classification is cheap and dependency-free.
+    Returns the dominant script only when it holds a clear (≥60%) majority of the
+    alphabetic characters, so mixed/borrowed text doesn't flip the bucket. Used
+    to establish the conversation start's expected script for cross-script title
+    drift detection (#3293).
+    """
+    counts = _script_counts(text)
+    total = sum(counts.values())
+    if total < 2:
+        return ''
+    top, top_n = max(counts.items(), key=lambda kv: kv[1])
+    if top_n / total >= 0.6:
+        return top
+    return ''
+
+
 def _title_prompt_language_rule(user_text: str) -> str:
     return "Match the language of the user question.\n"
 
 
 def _title_language_mismatch(user_text: str, title: str) -> bool:
-    """Reject obvious English titles for German conversation starts."""
-    if _detect_title_language(user_text) != 'de':
-        return False
-    candidate = str(title or '').strip().lower()
+    """Reject titles whose language clearly diverges from the conversation start.
+
+    Two independent signals:
+    1. Cross-script drift (#3293): when the conversation start has a clear
+       dominant writing script (e.g. latin/English) and the generated title
+       introduces a *substantial* amount of a different script (e.g. CJK or
+       Cyrillic), reject. This is language-agnostic and catches the common
+       "English chat -> Chinese/Spanish/Russian title" drift. Because titles are
+       short and frequently embed a borrowed Latin technical term (e.g. a CJK
+       title containing the word "Python"), the title side uses a proportion
+       threshold (>=35% of the title's alphabetic characters in a non-start
+       script, min 2 chars) rather than a strict majority -- so a CJK title with
+       one English word still trips, while an English title with a single
+       foreign place-name does not.
+    2. The legacy German-start → English-title heuristic, preserved verbatim so
+       the original behavior keeps working for same-script (latin) drift that
+       the script check can't see.
+    """
+    candidate = str(title or '').strip()
     if not candidate:
         return False
-    if _detect_title_language(candidate) == 'de':
+
+    # (1) Cross-script mismatch — language-agnostic.
+    user_script = _dominant_script(user_text)
+    if user_script:
+        title_counts = _script_counts(candidate)
+        title_total = sum(title_counts.values())
+        if title_total >= 2:
+            for script, n in title_counts.items():
+                if script != user_script and n >= 2 and (n / title_total) >= 0.35:
+                    return True
+
+    # (2) Legacy same-script German→English heuristic.
+    if _detect_title_language(user_text) != 'de':
+        return False
+    candidate_lower = candidate.lower()
+    if _detect_title_language(candidate_lower) == 'de':
         return False
     english_markers = {
         'old', 'image', 'display', 'issue', 'problem', 'discussion', 'conversation',
         'session', 'title', 'fix', 'bug', 'attachment', 'attachments', 'context',
     }
-    tokens = re.findall(r'[a-z]+', candidate)
+    tokens = re.findall(r'[a-z]+', candidate_lower)
     english_hits = sum(1 for tok in tokens if tok in english_markers)
     return english_hits >= 2