release: multimodal title-generation fix (#7306) (#7484)

* fix(title): preserve multimodal title generation

Assisted-by: Hermes Agent:gpt-5.6-sol
Assisted-by: Codex:gpt-5.6-luna
Assisted-by: Claude Code:claude-opus-5

* docs(title): explain multimodal title generation

Assisted-by: Hermes Agent:gpt-6-astra

* docs(changelog): stamp #7306 multimodal title-generation fix

---------

Co-authored-by: starship-s <45587122+starship-s@users.noreply.github.com>
Co-authored-by: n <a@n>

CHANGELOG.md

@@ -5,6 +5,8 @@
 
 ### Fixed
 
+- **Automatic session titles no longer keep an internal placeholder for workspace-prefixed multimodal first turns, and internal scaffolding no longer leaks into generated titles.** Title eligibility compared persisted content that still carried WebUI workspace/attachment scaffolding against a sanitized visible title, so a workspace-prefixed structured multimodal first turn could never be recognized as provisional and stayed stuck on its placeholder title; the same internal scaffolding could also reach the title provider's input. Title generation now sanitizes title-facing structured text (stripping versioned workspace sentinels and generated attachment suffixes) across eligibility, first-exchange generation, manual prefer-latest generation, and adaptive refresh, while preserving genuine user content and native image parts. Persisted and model-facing message content is unchanged (titles only read it), and a real user-chosen title is never misclassified as provisional. Thanks @starship-s. (#7306)
+
 - **The composer footer no longer jitters the transcript while a reply streams.** During SSE streaming, the composer footer's fit-stage probe transiently applied and reverted layout classes, which nudged the transcript viewport a few pixels backward on each frame (a visible clamp/jump on the crown-jewel chat surface). The fit probe now runs synchronously with the transcript geometry frozen and the caller's exact styles restored in a `finally`, so footer sizing is measured without moving the reader's scroll position; adaptive stage selection is unchanged. Thanks @ruizanthony. (#7275)
 
 - **A Windows self-restart now relaunches the actual server instead of replaying whatever process launched it.** The Windows restart path replayed the launching process's `sys.argv`, so a server started indirectly (e.g. under pytest) would restart *that* process rather than the server. It now resolves the canonical replacement command per packaging mode — a source or pip install relaunches `server.py` directly (preferring windowless `pythonw.exe` when present), while frozen/packaged argv is left unchanged — and preserves host, port, state, workspace, and agent environment across the restart. The POSIX (exec) restart path is unchanged. Thanks @rodboev. (#7270, closes #7195)

api/streaming.py

@@ -3843,6 +3843,71 @@ def _strip_workspace_prefix(text: str, *, include_legacy: bool = False) -> str:
     return stripped.strip()
 
 
+_TITLE_ATTACHMENT_SUFFIX_RE = re.compile(
+    r'(?:\n\n|\r\n\r\n)\[Attached files(?: for this steer)?: [^\]]+\]\s*$'
+)
+
+
+def _strip_title_attachment_suffix(text) -> str:
+    """Remove one exact WebUI-generated terminal attachment suffix."""
+    return _TITLE_ATTACHMENT_SUFFIX_RE.sub('', str(text or ''))
+
+
+def _strip_title_input_metadata(text) -> str:
+    """Remove only internal title metadata from one selected text value."""
+    value = _strip_title_attachment_suffix(text)
+    return _strip_workspace_prefix(value).strip()
+
+
+def _title_structured_text_parts(
+    content, allowed_types, *, normalize_types: bool = False
+) -> list[str]:
+    """Sanitize accepted structured title parts without mutating the content."""
+    parts = []
+    workspace_prefix_stripped = False
+    for part in content:
+        if not isinstance(part, dict):
+            continue
+        part_type = str(part.get('type') or '').lower() if normalize_types else part.get('type')
+        if part_type not in allowed_types:
+            continue
+        text = (
+            str(part.get('text') or '')
+            if not normalize_types
+            else _message_content_part_text(part)
+        )
+        if text.strip() and not workspace_prefix_stripped:
+            text = _strip_title_input_metadata(text)
+            workspace_prefix_stripped = bool(text)
+        parts.append(text)
+    return parts
+
+
+def _title_input_text(content) -> str:
+    """Extract raw title text using the same content rules as title_from."""
+    if content is None:
+        return ''
+    if isinstance(content, list):
+        return ' '.join(
+            _title_structured_text_parts(content, ('text',), normalize_types=False)
+        ).strip()
+    return _strip_title_input_metadata(str(content))
+
+
+_TITLE_MIXED_PART_TYPES = ('', 'text', 'input_text', 'output_text')
+
+
+def _title_exchange_input_text(content) -> str:
+    """Extract sanitized first/latest-exchange title input text."""
+    if isinstance(content, list):
+        return '\n'.join(
+            _title_structured_text_parts(
+                content, _TITLE_MIXED_PART_TYPES, normalize_types=True
+            )
+        ).strip()
+    return _strip_title_input_metadata(str(content or '').strip())
+
+
 def _looks_like_current_user_turn(msg, msg_text) -> bool:
     """Match the current human turn even if an internal workspace tag leaked mid-text.
 
@@ -3877,7 +3942,7 @@ def _first_exchange_snippets(messages):
             continue
         role = m.get('role')
         if role == 'user':
-            candidate = _message_text(m.get('content'))
+            candidate = _strip_thinking_markup(_title_exchange_input_text(m.get('content')))
             if not user_text and candidate:
                 user_text = candidate
                 continue
@@ -3918,9 +3983,13 @@ def _latest_exchange_snippets(messages):
                 continue
             if candidate:
                 asst_text = candidate
-        elif role == 'user' and not user_text:
+        elif role == 'user':
-            candidate = _message_text(m.get('content'))
+            candidate = _strip_thinking_markup(_title_exchange_input_text(m.get('content')))
-            if candidate:
+            if not candidate:
+                user_text = ''
+                asst_text = ''
+                break
+            if not user_text:
                 user_text = candidate
         if user_text and asst_text:
             break
@@ -3953,14 +4022,51 @@ def _get_title_refresh_interval() -> int:
 
 def _is_provisional_title(current_title: str, messages) -> bool:
     """Heuristic: title equals first-message substring placeholder."""
-    derived = title_from(messages, '') or ''
+    first_user_text = ''
-    if not derived:
+    for message in messages or []:
+        if not isinstance(message, dict) or message.get('role') != 'user':
+            continue
+        first_user_text = _title_input_text(message.get('content'))
+        if first_user_text:
+            break
+    if not first_user_text:
         return False
+    sanitized_derived = title_from([{'role': 'user', 'content': first_user_text}], '') or ''
+    raw_derived = title_from(messages or [], '') or ''
+    if not sanitized_derived:
+        return False
+
+    def _normalize_candidate(value):
+        return re.sub(r'\s+', ' ', str(value or '')[:64]).strip()
+
     current = re.sub(r'\s+', ' ', str(current_title or '')).strip()
-    candidate = re.sub(r'\s+', ' ', str(derived[:64] or '')).strip()
+    candidates = (
-    if not current or not candidate:
+        _normalize_candidate(sanitized_derived),
+        _normalize_candidate(raw_derived),
+    )
+    if not current:
         return False
-    return current == candidate
+    return any(candidate and current == candidate for candidate in candidates)
+
+
+def _background_title_generation_inputs(session):
+    """Return sanitized first-exchange inputs when background title generation is eligible."""
+    messages = getattr(session, 'messages', None) or []
+    title = getattr(session, 'title', '')
+    invalid_existing_title = _looks_invalid_generated_title(title)
+    eligible_title = (
+        title == 'Untitled'
+        or title == 'New Chat'
+        or not title
+        or _is_provisional_title(title, messages)
+        or invalid_existing_title
+    )
+    if not eligible_title or (
+        getattr(session, 'llm_title_generated', False) and not invalid_existing_title
+    ):
+        return None
+    user_text, assistant_text = _first_exchange_snippets(messages)
+    return (user_text, assistant_text) if user_text and assistant_text else None
 
 
 def _detect_title_language(text: str) -> str:
@@ -11568,17 +11674,7 @@ def _run_agent_streaming(
                 # Only auto-generate title when still default; preserves user renames
                 if s.title == 'Untitled' or s.title == 'New Chat' or not s.title:
                     s.title = title_from(s.messages, s.title)
-                _looks_default = (s.title == 'Untitled' or s.title == 'New Chat' or not s.title)
+                _bg_title_inputs = _background_title_generation_inputs(s)
-                _looks_provisional = _is_provisional_title(s.title, s.messages)
-                _invalid_existing_title = _looks_invalid_generated_title(s.title)
-                _should_bg_title = (
-                    (_looks_default or _looks_provisional or _invalid_existing_title)
-                    and (not getattr(s, 'llm_title_generated', False) or _invalid_existing_title)
-                )
-                _u0 = ''
-                _a0 = ''
-                if _should_bg_title:
-                    _u0, _a0 = _first_exchange_snippets(s.messages)
                 # Read token/cost usage from the agent object (if available).
                 # Per-turn overwrite (#1857): replace cumulative session totals with the
                 # agent's most recent values, which already represent the current turn's
@@ -12340,10 +12436,10 @@ def _run_agent_streaming(
                 # misbehaving log handler here would otherwise skip the
                 # background-title thread spawn below. (#4923 gate hardening)
                 pass
-            if _should_bg_title and _u0 and _a0:
+            if _bg_title_inputs:
                 threading.Thread(
                     target=_run_background_title_update,
-                    args=(s.session_id, _u0, _a0, str(s.title or '').strip(), put, agent),
+                    args=(s.session_id, *_bg_title_inputs, str(s.title or '').strip(), put, agent),
                     daemon=True,
                 ).start()
             else:

docs/advanced-chat-setup.md

@@ -55,6 +55,23 @@ Hermes WebUI derives a provisional session title from the first user message
 and, after the first response, may call an LLM to generate a better title
 (and periodically refresh it for long sessions).
 
+For structured messages containing text and native images, title generation
+uses the user text without flattening or modifying the stored message. Title
+comparison and title-model inputs remove the internal `[Workspace::v1: ...]`
+prefix and one terminal `[Attached files: ...]` or
+`[Attached files for this steer: ...]` suffix separated by a blank line.
+For structured content, this cleanup applies to the first text part that
+provides title content. Literal legacy `[Workspace: ...]` text and later text
+parts remain unchanged. Initial generation, explicit regeneration, and adaptive
+refresh use this title-specific cleanup.
+
+Background generation requires user text and a substantive assistant response.
+It recognizes the sanitized provisional title as well as the existing raw
+placeholder, so internal metadata does not make an image-containing turn look
+manually titled. Image-only or metadata-only content does not provide title
+text. Existing manual-title protection and the title-generation setting still
+apply; this cleanup does not rewrite the transcript or native image parts.
+
 Automatic title-generation LLM calls honor the active Hermes profile's
 `auxiliary.title_generation.enabled` setting (default: `true`):
 

tests/test_sprint41.py

@@ -10,6 +10,7 @@ Covers:
 import pathlib
 import re
 import unittest
+from types import SimpleNamespace
 
 REPO_ROOT = pathlib.Path(__file__).parent.parent
 CSS = (REPO_ROOT / "static" / "style.css").read_text(encoding="utf-8")
@@ -223,6 +224,278 @@ class TestIssue495TitleStreaming(unittest.TestCase):
             "Whitespace-normalized provisional titles should still be recognized",
         )
 
+    def test_structured_workspace_prefixed_provisional_title_is_eligible(self):
+        """The persisted multimodal first turn remains eligible for auto-title."""
+        from api.streaming import (
+            _background_title_generation_inputs,
+            _is_provisional_title,
+            title_from,
+        )
+
+        visible_text = (
+            "[Workspace::v1: user-authored]\n"
+            "Describe uploads naturally.\n\n"
+            "Attached files are mentioned in ordinary prose."
+        )
+        messages = [
+            {
+                "role": "user",
+                "content": [
+                    {
+                        "type": "text",
+                        "text": (
+                            "[Workspace::v1: /internal]\n"
+                            f"{visible_text}\n\n"
+                            "[Attached files: /attachments/uploads.png]"
+                        ),
+                    },
+                    {
+                        "type": "image_url",
+                        "image_url": {"url": "data:image/png;base64,workflow"},
+                    },
+                ],
+            },
+            {"role": "assistant", "content": "Here is the completed workflow."},
+        ]
+        session = SimpleNamespace(
+            title=title_from([{"role": "user", "content": visible_text}], ""),
+            messages=messages,
+            llm_title_generated=False,
+        )
+
+        self.assertEqual(session.title, visible_text[:64])
+        self.assertTrue(_is_provisional_title(session.title, session.messages))
+        title_inputs = _background_title_generation_inputs(session)
+        expected_user_text = " ".join(visible_text.split())
+        self.assertEqual(title_inputs, (expected_user_text, "Here is the completed workflow."))
+        user_text, _ = title_inputs
+        self.assertIn("Attached files are mentioned in ordinary prose.", user_text)
+        self.assertNotIn("[Attached files: /attachments/uploads.png]", user_text)
+        for forbidden in ("/internal",):
+            self.assertNotIn(forbidden, user_text)
+
+    def test_structured_title_input_skips_empty_and_image_only_users(self):
+        """The first title-bearing user row wins over empty/image-only rows."""
+        from api.streaming import (
+            _background_title_generation_inputs,
+            _is_provisional_title,
+        )
+
+        visible_text = (
+            "Describe the three-step workflow with enough detail to exercise title truncation "
+            "after skipping empty image-only rows."
+        )
+        long_workspace = "/workspace/" + ("long-title-sentinel-" * 4)
+        messages = [
+            {"role": "user", "content": None},
+            {
+                "role": "user",
+                "content": [
+                    {"type": "image_url", "image_url": {"url": "data:image/png;base64,empty"}}
+                ],
+            },
+            {
+                "role": "user",
+                "content": (
+                    "[Workspace::v1: /internal-only-title-metadata]\n\n"
+                    "[Attached files: /attachments/internal-only.png]"
+                ),
+            },
+            {
+                "role": "user",
+                "content": [
+                    {
+                        "type": "text",
+                        "text": (
+                            "[Workspace::v1: /internal-only-title-part]\n\n"
+                            "[Attached files: /attachments/internal-only-part.png]"
+                        ),
+                    },
+                    {
+                        "type": "text",
+                        "text": (
+                            f"[Workspace::v1: {long_workspace}]\n{visible_text}\n\n"
+                            "[Attached files: /attachments/workflow.png]"
+                        ),
+                    },
+                    {"type": "image_url", "image_url": {"url": "data:image/png;base64,workflow"}},
+                ],
+            },
+            {"role": "assistant", "content": "The workflow is complete."},
+        ]
+        session = SimpleNamespace(
+            title=visible_text[:64], messages=messages, llm_title_generated=False
+        )
+
+        self.assertGreater(len(long_workspace), 64)
+        self.assertTrue(_is_provisional_title(session.title, session.messages))
+        self.assertEqual(
+            _background_title_generation_inputs(session),
+            (visible_text, "The workflow is complete."),
+        )
+
+    def test_input_text_title_snippet_remains_background_eligible(self):
+        """Mixed structured input_text content remains a title input."""
+        from api.streaming import (
+            _background_title_generation_inputs,
+            _first_exchange_snippets,
+            _is_provisional_title,
+        )
+
+        visible_text = "Describe the input-text workflow."
+        assistant_text = "The workflow is complete."
+        messages = [
+            {
+                "role": "user",
+                "content": [{"type": "input_text", "text": visible_text}],
+            },
+            {"role": "assistant", "content": assistant_text},
+        ]
+        session = SimpleNamespace(
+            title="Untitled", messages=messages, llm_title_generated=False
+        )
+
+        self.assertEqual(
+            _first_exchange_snippets(messages),
+            (visible_text, assistant_text),
+        )
+        self.assertEqual(
+            _background_title_generation_inputs(session),
+            (visible_text, assistant_text),
+        )
+        self.assertFalse(
+            _is_provisional_title(
+                "Alias text",
+                [{"role": "user", "content": [{"type": "text", "input_text": "Alias text"}]}],
+            )
+        )
+
+    def test_later_structured_text_part_remains_literal(self):
+        """Only the leading structured text part is metadata-normalized."""
+        from api.streaming import (
+            _background_title_generation_inputs,
+            _is_provisional_title,
+            title_from,
+        )
+
+        first_text = "Describe the literal multi-part request."
+        later_text = (
+            "[Workspace::v1: /literal-user-text] must remain literal user content.\n\n"
+            "[Attached files: literal.txt]"
+        )
+        messages = [
+            {
+                "role": "user",
+                "content": [
+                    {
+                        "type": "text",
+                        "text": (
+                            "[Workspace::v1: /workspace/example]\n"
+                            f"{first_text}\n\n"
+                            "[Attached files: /attachments/workflow.png]"
+                        ),
+                    },
+                    {"type": "text", "text": later_text},
+                    {"type": "image_url", "image_url": {"url": "data:image/png;base64,literal"}},
+                ],
+            },
+            {"role": "assistant", "content": "The literal content is preserved."},
+        ]
+        sanitized_title_text = f"{first_text} {later_text}"
+        expected_title_input = " ".join(sanitized_title_text.split())
+        session = SimpleNamespace(
+            title=title_from([{"role": "user", "content": sanitized_title_text}], ""),
+            messages=messages,
+            llm_title_generated=False,
+        )
+
+        self.assertTrue(_is_provisional_title(session.title, session.messages))
+        title_inputs = _background_title_generation_inputs(session)
+        self.assertEqual(
+            title_inputs,
+            (expected_title_input, "The literal content is preserved."),
+        )
+        user_text, _ = title_inputs
+        for forbidden in ("/workspace/example", "/attachments/workflow.png"):
+            self.assertNotIn(forbidden, user_text)
+        self.assertIn(" ".join(later_text.split()), user_text)
+        self.assertIn("[Attached files: literal.txt]", user_text)
+
+    def test_latest_structured_title_input_is_sanitized(self):
+        """Latest title-refresh input strips workspace and attachment metadata."""
+        from api.streaming import _latest_exchange_snippets
+
+        visible_text = "Summarize the latest workflow."
+        assistant_text = "Here is the refreshed summary."
+        messages = [
+            {"role": "user", "content": "Earlier request."},
+            {"role": "assistant", "content": "Earlier answer."},
+            {
+                "role": "user",
+                "content": [
+                    {
+                        "type": "text",
+                        "text": (
+                            "[Workspace::v1: /workspace/latest]\n"
+                            f"{visible_text}\n\n"
+                            "[Attached files: /attachments/latest.png]"
+                        ),
+                    },
+                    {"type": "image_url", "image_url": {"url": "data:image/png;base64:latest"}},
+                ],
+            },
+            {"role": "assistant", "content": assistant_text},
+        ]
+
+        user_text, returned_assistant_text = _latest_exchange_snippets(messages)
+
+        self.assertEqual(user_text, visible_text)
+        self.assertEqual(returned_assistant_text, assistant_text)
+        self.assertNotIn("/workspace/latest", user_text)
+        self.assertNotIn("/attachments/latest.png", user_text)
+
+        metadata_only_messages = [
+            {"role": "user", "content": "Earlier request."},
+            {"role": "assistant", "content": "Earlier answer."},
+            {
+                "role": "user",
+                "content": [
+                    {
+                        "type": "text",
+                        "text": (
+                            "[Workspace::v1: /workspace/latest]\n\n"
+                            "[Attached files: /attachments/latest.png]"
+                        ),
+                    },
+                    {"type": "image_url", "image_url": {"url": "data:image/png;base64:latest"}},
+                ],
+            },
+            {"role": "assistant", "content": "Latest answer."},
+        ]
+        self.assertEqual(_latest_exchange_snippets(metadata_only_messages), ("", ""))
+
+    def test_literal_legacy_workspace_text_is_preserved_in_title_inputs(self):
+        """Legacy workspace text remains user content rather than metadata."""
+        from api.streaming import (
+            _background_title_generation_inputs,
+            _is_provisional_title,
+            title_from,
+        )
+
+        messages = [
+            {"role": "user", "content": "[Workspace: /literal-user-text]\nExplain this literal prefix."},
+            {"role": "assistant", "content": "The legacy prefix is literal."},
+        ]
+        session = SimpleNamespace(
+            title=title_from(messages, ""),
+            messages=messages,
+            llm_title_generated=False,
+        )
+
+        self.assertTrue(_is_provisional_title(session.title, session.messages))
+        user_text, _ = _background_title_generation_inputs(session)
+        self.assertIn("[Workspace: /literal-user-text]", user_text)
+
     def test_title_snippet_keeps_tool_call_with_substantive_text(self):
         """An assistant row with tool_calls AND a substantive answer text
         must still be used as the first-exchange snippet — it's not a