* fix(dictation): preserve composer text when server STT commits
Dictation via the server STT path (MediaRecorder → POST /api/transcribe)
replaced any text already in the composer with the recognized transcript.
Browser SpeechRecognition worked correctly; only the async path was broken.
Root cause: the MediaRecorder.onstop handler calls _setRecording(false)
before awaiting _transcribeBlob's fetch. _setRecording(false) clears the
module-level _prefix. By the time the fetch resolves and _commitTranscript
runs, _prefix is '' — so the existing append logic collapses to
'just the transcript' and overwrites the textarea.
Browser SR dodged this because sr.onend builds the merged string into a
local variable before invoking _setRecording(false). Server STT is async
and could not use that pattern without help.
Fix:
- _commitTranscript accepts an optional prefixOverride parameter.
- _transcribeBlob snapshots _prefix into a local before the await.
- Pass the snapshot through to _commitTranscript.
Add a Settings → Voice → 'Append dictation to composer' toggle (default
ON) so users can opt into replace mode if they prefer the old buggy
behavior. The bug fix changes the default to append, matching what the
browser SR path always did.
i18n: new keys dictation_append_label / dictation_append_desc added to
all 14 locale blocks in static/i18n.js (English fallback where no native
translation exists).
Files: static/boot.js (+43/-7), static/index.html (+7), static/i18n.js (+30).
* fix(dictation): snapshot _prefix in onstop, not inside _transcribeBlob
First attempt at the append-mode fix took the snapshot inside
_transcribeBlob via 'const prefixSnapshot = _prefix'. By the time
_transcribeBlob runs (awaited from recorder.onstop), the sync block
above has already called _setRecording(false), which clears _prefix
inside _setRecording's !on branch. So the snapshot was always '' and
the append path fell through to the replace branch.
Move the snapshot into recorder.onstop, before _setRecording(false)
runs. Pass it as a parameter into _transcribeBlob so the call site
doesn't need to re-read the (now-cleared) module-level _prefix.
No behavior change for the browser-SR path — sr.onend still builds
the merged string into a local variable before invoking
_setRecording(false), and was already correct.
* fix(dictation): trim comments to fit test_590_recording_stops_before_transcribe
test_bugbatch_apr2026.py::test_590_recording_stops_before_transcribe slices
700 chars from 'recorder.onstop' and asserts that _setRecording(false)
comes before _sendRawAudio(/_transcribeBlob( inside that window. With
the verbose 6-line comment block from the previous fix, the actual
call sites landed at offsets 834/890 — past the 700-char window — so
the test's .find() returned -1 and the assertion failed mechanically.
Also: the verbose comment in _transcribeBlob contained the substring
'recorder.onstop', which made the test's first .find() land in the
comment instead of the actual handler, returning a wrong onstop_start.
Trim both comment blocks to single lines that fit within the test's
window and don't contain the literal 'recorder.onstop' substring.
Functional behavior unchanged from the previous fix; this is purely
a comment-shape adjustment to satisfy the brittle test.
* fix(tests): anchor test_590 on handler definition + widen window per review
The reviewer's fix-spec for test_590_recording_stops_before_transcribe
was more thorough than my single-shot comment trim:
(1) Reword the _transcribeBlob comment so it doesn't contain the
literal 'recorder.onstop' substring.
(2) Make the test robust to the handler growing: anchor on
'recorder.onstop=async' instead of 'recorder.onstop', and
widen the fixed 700-char window to 1200.
Applied both. With these changes the test passes regardless of how
much comment text gets added in or around the handler in the future.
Also restored the verbose explanatory comments in both locations
(the previous a7f84027 had trimmed them to fit the brittle 700-char
window — that workaround is no longer needed).
Backward-compat check: the new test logic also passes against
origin/master (verified locally), so this change doesn't regress
the existing green-master invariant.
* fix(dictation): preserve user's mid-transcription edits in async server STT
Greptile flagged this as a 4/5 concern on the append-mode branch:
the async server-STT path appended the dictated text to the stale
snapshot (`_prefix` captured at recording start), which clobbered
any keystrokes the user made during the ~1s transcription round-trip.
Before this fix, with append mode ON:
User typed before mic: 'Hello, '
User typed during STT: 'Hello, from K' (typed while waiting)
Dictated text: 'world'
Result: 'Hello, world' (lost 'from K')
After this fix:
User typed before mic: 'Hello, '
User typed during STT: 'Hello, from K'
Dictated text: 'world'
Result: 'Hello, from K world' (live state preserved)
The fix prefers live `ta.value` as the append base when the textarea
is non-empty, falling back to the `prefixOverride` snapshot only
when the textarea is genuinely empty (the original empty-composer path
where nothing to race with).
Adds a regression test in test_raw_audio_upload.py that asserts the
new `ta.value ||` fallback structure on `_commitTranscript`.
* fix(dictation): don't resurrect cleared text on async transcription resolve
Greptile P1 (re-reviewed at 20:07 UTC): the previous race fix
(`ta.value || (prefixOverride ?? _prefix)`) treats an empty
`ta.value` as falsy and falls back to the recording-start snapshot.
If the user explicitly clears the textarea while server transcription
is in flight, the cleared text reappears with the dictated text
appended — the user's intent is violated.
Fix: when prefixOverride IS passed (server-STT path), trust live
ta.value UNCONDITIONALLY — even when empty. The fallback to _prefix
only fires when no prefixOverride is given AND ta.value is empty,
which is purely defensive (no current caller hits this branch — the
browser-SR path commits inline in sr.onend without calling this
function).
Concrete repro:
User typed before mic: 'Hello, '
User clears textarea during STT
Dictated text: 'world'
OLD (0239bb4e) result: 'Hello, world' (cleared text resurrected)
NEW result: 'world' (user intent respected)
Greptile's suggested one-liner:
const base = prefixOverride !== undefined ? ta.value : (ta.value || _prefix);
Applied with expanded inline comment documenting the three concerns
this function balances (race, clear-during-STT, defensive fallback).
* Release exp-v0.52.49: preserve composer text on server STT commit (#5895)
---------
Co-authored-by: Konstantin M <kopamedve@gmail.com>
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
205 lines
8.4 KiB
Python
205 lines
8.4 KiB
Python
"""Tests for raw audio upload flow — sending audio as attachment vs transcribing."""
|
|
import io
|
|
import json
|
|
import sys
|
|
import types
|
|
|
|
from api.upload import handle_upload, handle_transcribe
|
|
|
|
|
|
def _multipart_body(fields=None, files=None, boundary=b"testboundary"):
|
|
fields = fields or {}
|
|
files = files or {}
|
|
body = b""
|
|
for name, value in fields.items():
|
|
body += b"--" + boundary + b"\r\n"
|
|
body += f'Content-Disposition: form-data; name="{name}"\r\n\r\n'.encode()
|
|
body += str(value).encode() + b"\r\n"
|
|
for name, (filename, data, content_type) in files.items():
|
|
body += b"--" + boundary + b"\r\n"
|
|
body += (
|
|
f'Content-Disposition: form-data; name="{name}"; filename="{filename}"\r\n'
|
|
f"Content-Type: {content_type}\r\n\r\n"
|
|
).encode()
|
|
body += data + b"\r\n"
|
|
body += b"--" + boundary + b"--\r\n"
|
|
return body, f"multipart/form-data; boundary={boundary.decode()}"
|
|
|
|
|
|
class _FakeHandler:
|
|
def __init__(self, body: bytes, content_type: str, session_id: str | None = None):
|
|
self.rfile = io.BytesIO(body)
|
|
self.wfile = io.BytesIO()
|
|
self.headers = {
|
|
"Content-Type": content_type,
|
|
"Content-Length": str(len(body)),
|
|
}
|
|
self._session_id = session_id
|
|
self.status = None
|
|
self.sent_headers = {}
|
|
|
|
def send_response(self, status):
|
|
self.status = status
|
|
|
|
def send_header(self, key, value):
|
|
self.sent_headers[key] = value
|
|
|
|
def end_headers(self):
|
|
pass
|
|
|
|
def payload(self):
|
|
return json.loads(self.wfile.getvalue().decode("utf-8"))
|
|
|
|
|
|
# ── Raw Audio Upload Tests ────────────────────────────────────────────────
|
|
|
|
|
|
def test_raw_audio_upload_accepts_audio_file():
|
|
"""Audio files uploaded via /api/upload should succeed and return audio mime."""
|
|
body, content_type = _multipart_body(
|
|
fields={"session_id": "test-session-raw-audio"},
|
|
files={"file": ("voice.webm", b"RIFF\x1a\x9f\x01fake_opus_data", "audio/webm")},
|
|
)
|
|
handler = _FakeHandler(body, content_type)
|
|
try:
|
|
handle_upload(handler)
|
|
except KeyError:
|
|
# Session not found is expected without a real session store
|
|
pass
|
|
# The handler should at least parse the request without crashing
|
|
# Real session check happens after parsing
|
|
assert handler.status is not None
|
|
|
|
|
|
def test_raw_audio_upload_rejects_missing_file():
|
|
"""Upload without file field should return 400."""
|
|
body, content_type = _multipart_body(fields={"session_id": "test-session"})
|
|
handler = _FakeHandler(body, content_type)
|
|
handle_upload(handler)
|
|
assert handler.status == 400
|
|
assert handler.payload()["error"] == "No file field in request"
|
|
|
|
|
|
def test_raw_audio_vs_transcribe_no_regression(monkeypatch):
|
|
"""The /api/transcribe endpoint continues working independently of raw audio mode."""
|
|
fake_mod = types.ModuleType("tools.transcription_tools")
|
|
fake_mod.transcribe_audio = lambda path: {"success": True, "transcript": "hello from audio"}
|
|
monkeypatch.setitem(sys.modules, "tools.transcription_tools", fake_mod)
|
|
|
|
body, content_type = _multipart_body(
|
|
files={"file": ("voice.webm", b"RIFFfakeaudio", "audio/webm")}
|
|
)
|
|
handler = _FakeHandler(body, content_type)
|
|
handle_transcribe(handler)
|
|
|
|
assert handler.status == 200
|
|
assert handler.payload() == {"ok": True, "transcript": "hello from audio"}
|
|
|
|
|
|
def test_raw_audio_upload_requires_session():
|
|
"""Upload should return 404 for missing session."""
|
|
body, content_type = _multipart_body(
|
|
fields={"session_id": "nonexistent-session-id"},
|
|
files={"file": ("voice.webm", b"fake", "audio/webm")},
|
|
)
|
|
handler = _FakeHandler(body, content_type)
|
|
try:
|
|
handle_upload(handler)
|
|
except KeyError:
|
|
# Expected — _FakeHandler doesn't have a real session store
|
|
pass
|
|
assert handler.status is None or handler.status == 404
|
|
|
|
|
|
def test_raw_audio_upload_mime_type_detection():
|
|
"""Uploaded audio file should return correct audio MIME type."""
|
|
body, content_type = _multipart_body(
|
|
fields={"session_id": "test-session-mime"},
|
|
files={"file": ("recording.webm", b"RIFF\x1a\x9f\x01fake", "audio/webm")},
|
|
)
|
|
handler = _FakeHandler(body, content_type)
|
|
try:
|
|
handle_upload(handler)
|
|
except KeyError:
|
|
pass
|
|
# The upload handler should at minimum parse the multipart and
|
|
# guess the mime type from filename extension
|
|
# (real session check happens after parsing)
|
|
assert handler.status is not None
|
|
|
|
|
|
def test_raw_audio_upload_different_formats():
|
|
"""Audio files in different formats (ogg, wav) should all be accepted."""
|
|
for filename, mime in [("voice.ogg", "audio/ogg"), ("voice.wav", "audio/wav"), ("voice.mp3", "audio/mpeg")]:
|
|
body, content_type = _multipart_body(
|
|
fields={"session_id": "test-session-fmt"},
|
|
files={"file": (filename, b"fake audio", mime)},
|
|
)
|
|
handler = _FakeHandler(body, content_type)
|
|
try:
|
|
handle_upload(handler)
|
|
except KeyError:
|
|
pass
|
|
assert handler.status is not None, f"Failed for {filename} ({mime})"
|
|
|
|
|
|
# ── Front-end: raw-audio mic backend pinning (boot.js source invariants) ──────
|
|
# Regression guard for the #3169 Codex-review fix: _stopMic / onstop must act on
|
|
# the capture backend that was ACTIVE WHEN RECORDING STARTED (pinned in
|
|
# _activeCaptureMode), not whatever _rawAudioMode says now — otherwise toggling
|
|
# Settings → Sound mid-recording orphans the wrong backend. Also: an explicit
|
|
# Send click (_micPendingSend) must send even with text in the composer.
|
|
import pathlib as _pathlib
|
|
|
|
_BOOT_JS = (_pathlib.Path(__file__).parent.parent / "static" / "boot.js").read_text(encoding="utf-8")
|
|
|
|
|
|
def test_stop_mic_uses_pinned_active_capture_mode_not_current_rawmode():
|
|
assert "let _activeCaptureMode" in _BOOT_JS, "_activeCaptureMode must be declared"
|
|
# _stopMic decides backend from the pinned mode, not _rawAudioMode.
|
|
assert "_activeCaptureMode==='speech'" in _BOOT_JS
|
|
# onstop dispatches raw-vs-transcribe from the pinned mode too.
|
|
assert "if(captureMode==='media-raw')" in _BOOT_JS
|
|
# the mode is pinned at both start branches.
|
|
assert "_activeCaptureMode='speech'" in _BOOT_JS
|
|
assert "const captureMode=_rawAudioMode?'media-raw':'media-transcribe';" in _BOOT_JS
|
|
assert "_activeCaptureMode=captureMode;" in _BOOT_JS
|
|
|
|
|
|
def test_send_raw_audio_honors_explicit_pending_send():
|
|
# An explicit Send-button click (sets _micPendingSend) must send the raw
|
|
# audio even when the composer already has text.
|
|
assert "if(window._micPendingSend){" in _BOOT_JS
|
|
# and it lives inside _sendRawAudio (before the empty-composer fallback).
|
|
idx = _BOOT_JS.index("async function _sendRawAudio")
|
|
end = _BOOT_JS.index("function _commitTranscript", idx)
|
|
body = _BOOT_JS[idx:end]
|
|
assert "window._micPendingSend" in body and "send()" in body
|
|
|
|
|
|
def test_commit_transcript_appends_to_live_text_not_stale_snapshot():
|
|
# Regression: server-STT transcription is async (~1s round-trip). During
|
|
# that window the user can keep typing OR clear the textarea. The original
|
|
# implementation appended to `_prefix` (the recording-start snapshot),
|
|
# which clobbered live edits AND resurrected text the user had cleared.
|
|
#
|
|
# Fix: when prefixOverride IS passed (server-STT path), trust live ta.value
|
|
# unconditionally — even when empty. The fallback to _prefix only applies
|
|
# when no prefixOverride is given AND ta.value is empty (defensive only —
|
|
# no current caller hits this branch; browser-SR commits inline in sr.onend).
|
|
idx = _BOOT_JS.index("function _commitTranscript")
|
|
end = _BOOT_JS.index("\n function ", idx + 1)
|
|
body = _BOOT_JS[idx:end]
|
|
# The append base must be `prefixOverride !== undefined ? ta.value : ...`
|
|
# so live ta.value (including empty) wins for the server-STT path.
|
|
assert "prefixOverride !== undefined ? ta.value" in body, (
|
|
"_commitTranscript must use live ta.value when prefixOverride is set, "
|
|
"even when empty — otherwise cleared textarea resurrects on resolve "
|
|
"(Greptile P1 finding on 0239bb4e)"
|
|
)
|
|
# Sanity: the fallback branch should still exist for the defensive case.
|
|
assert "_prefix" in body, (
|
|
"_commitTranscript must keep _prefix as a fallback for any future "
|
|
"caller that passes no prefixOverride"
|
|
)
|