`_terminals_to_reap()` snapshots victims under `_LOCK` and then releases it,
but `subscribe()` clears `unwatched_since` and appends to `_subscribers` under
the terminal's own `_sub_lock`, which the selection pass never held. A viewer
reconnecting in that gap left object identity — all `close_terminal(expected=…)`
ever checked — still true, so the reaper killed a terminal that now had a live
viewer.
- `_claim_reap_victim()` re-establishes the *whole* selection predicate while
holding `_LOCK` and the terminal's `_sub_lock`, and removes the registry
entry in that same critical section: same object, no subscribers, and
`unwatched_since` still set and still expired. A dead process is still
reaped unconditionally — it cannot come back, and a viewer watching a corpse
must not pin the entry forever.
- `attach_terminal()` does the registry lookup and the subscribe atomically
under `_LOCK`, replacing `get_terminal()` + `term.subscribe()` in the SSE
route. It takes the same two locks in the same order as the claim, so
exactly one side wins: the viewer attaches (and the reap is refused), or the
entry is already gone (and the route answers "terminal not running" before
committing a 200, instead of streaming from a corpse). Registration is the
authority — a registered-but-`closed` terminal still attaches so the viewer
receives `terminal_closed`.
- Teardown moved to `_teardown_terminal()` and runs after both locks are
released; `killpg`+`wait` can block ~2.5s and must not pin the registry.
- Dropped the duplicated `self.last_activity = time.time()` the rebase left in
`put_output()`.
Tests: a barrier regression that pauses the reaper after selection, attaches a
viewer through the production path, resumes, and asserts the terminal stays
registered/live and still receives output; the inverse where the reap claims
first and the attach reports not-running; refreshed-grace and replaced-entry
claims; dead-with-viewer; and a probe that `_LOCK` is free during teardown.
Three of them fail against an identity-only claim.
The fd-leak fix retires a terminal whose shell EXITS, and _MAX_TERMINALS bounds
the worst case, but an interactive shell abandoned by its client (tab closed
without POSTing /api/terminal/close, browser crash, network drop) keeps running
forever — there is deliberately no PDEATHSIG. A background reaper now closes any
terminal that has had zero attached output streams for longer than a 15-minute
idle grace, plus any dead-process terminal as a belt-and-suspenders for the
reader-loop retire.
Attachment is tracked on the subscriber model from the output-broadcast change:
subscribe() clears unwatched_since, the last unsubscribe() sets it. A terminal
is born unwatched, so a spawn nobody ever attaches to is reaped too; the grace
spans a tab refresh / brief drop so a real reconnect keeps the session. Eviction
uses close_terminal(sid, expected=term) so a restart replacement is never torn
down. The reaper thread is a lazily-started daemon singleton like the spawn
supervisor.
Stacked on the terminal fd-leak + output-broadcast changes (same file).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two embedded-terminal resource leaks (#4633):
1. fd + dict entry on shell exit. When a shell exited on its own (user typed
`exit`, the process died), _reader_loop broke out of its loop but only set
`closed` and emitted terminal_closed — it never closed the pty master_fd nor
removed the _TERMINALS entry. Only an explicit close_terminal / restart /
atexit did that, so every self-exited shell leaked its master fd and dict
entry for the rest of the WebUI's uptime. The reader loop now retires the
session (close_terminal with an `expected=` identity guard so a restart's
replacement is never torn down).
2. Unbounded accumulation of abandoned terminals. A client that drops its output
stream without POSTing /api/terminal/close (tab close, crash, network drop)
leaves the shell running (no PDEATHSIG, by design). _MAX_TERMINALS (32) now
caps the live population, evicting a dead-proc terminal first, else the
least-recently-active one, to make room.
Also hardens an fd-reuse TOCTOU that #1 would otherwise make reachable on the
common `exit` path: write_terminal / resize now re-check `closed` and touch
master_fd under a per-terminal io_lock that close_terminal also holds around
os.close, so input can never land on a closed-and-recycled fd number.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(terminal): broadcast output to every viewer instead of one shared queue
TerminalSession.output was a single queue.Queue read destructively by the SSE
handler. Two tabs/windows viewing the SAME session each open their own
EventSource, so two _handle_terminal_output handlers competed on that one queue:
every PTY chunk was delivered to exactly one of them. Each tab saw a disjoint
half of the byte stream, and only one ever received terminal_closed.
Output now fans out, mirroring StreamChannel/SessionChannel: each SSE consumer
subscribe()s its own queue (seeded with a bounded backlog so a first/late attach
still replays the recent scrollback, preserving the old buffer-until-first-
consumer behaviour), and put_output broadcasts to all subscribers. A slow
viewer's queue drops its own oldest chunk (drop-oldest, isolated per subscriber)
so one lagging tab can't starve another. The handler unsubscribes in a finally
so the subscriber list can't grow.
Stacked on the terminal fd-leak fix (same file).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: cover terminal broadcast lifecycle
* fix: serialize terminal subscriber fanout
* test: cover terminal unsubscribe publication race
* restore timing-flaky pytest.skip on test_terminal_survives_short_lived_request_thread (keep API-updated body); changelog #5836
---------
Co-authored-by: ai-ag2026 <261867348+ai-ag2026@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
* fix(terminal): reap reparented terminal descendants by process group (#3725, #2577)
Embedded-terminal descendants reparented to the WebUI process could linger as
zombies. The reaper now calls os.waitpid(-terminal_pgid, WNOHANG) scoped to the
terminal's own process group (terminals spawn with start_new_session=True, so
proc.pid == pgid) rather than process-wide waitpid(-1), which would otherwise
reap unrelated WebUI subprocess children and silently coerce their exit codes to
0. Bounded by a 64-iteration limit and lock-guarded. Runs on reader cleanup and
terminal close.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* docs(docker): add opt-in GPU runtime image path (#3721, #3243)
The default image stays CPU-only. A new INSTALL_GPU_LIBS=1 build arg installs
VA-API user-space libraries for users passing through host GPU devices, and
docker_init.bash preserves Docker --group-add supplemental groups (e.g. render/
video for /dev/dri) when dropping privileges to the runtime user. Default
(INSTALL_GPU_LIBS=0) is a no-op. Docs + regression test included.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* docs(changelog): stamp v0.51.304 — Release JT (stage-p2a #3725#3721)
---------
Co-authored-by: nesquena-hermes <[email protected]>
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
## Release v0.51.236 — Release HD (stage-q7)
First Phase 3 (deep-review) release — picked by the 3-factor framework (contributor × impact × mitigated-risk): high-impact (#1952 native Windows support), backend-only (no screenshots), well-mitigated risk (POSIX path provably unchanged), from a contributor active this session (@rodboev, #3446/#3486 shipped earlier today).
### Added
| PR | Author | Fix |
|----|--------|-----|
| #1952 | @rodboev | Native Windows support for `bootstrap.py` + the embedded terminal: POSIX-only `fcntl`/`termios`/`select` guarded behind `_TERMINAL_SUPPORTED`; terminal entry points raise `NotImplementedError`/no-op on Windows; bootstrap Windows block → warning; auto-install errors clearly on native Windows (WSL unaffected); foreground uses `Popen`+exit on Windows instead of `os.execv`. **POSIX behavior unchanged on every path.** |
### Absorbed on the way in (fix-it-ourselves, reviewed fresh)
- `subprocess.CREATE_NEW_PROCESS_GROUP` → `getattr(subprocess, ..., 0)` — the constant is Windows-only, so a win32-simulating test `AttributeError`'d on Linux. Mirrors the `SO_EXCLUSIVEADDRUSE` getattr guard.
- Fixed 2 over-reaching tests in `test_windows_native_support.py` — one was launching a **real installer subprocess** via an unstubbed `subprocess.run` (now stubbed; harness 2.8s vs 80s); removed unused imports.
- Updated `test_onboarding_static.py` — it asserted the OLD "Native Windows is not supported" hard-block string this PR intentionally replaces; now asserts the new experimental-warning + auto-install guard.
- Help-text accuracy: `--foreground` help now describes the Windows Popen path (Opus nit).
### Gate results
- **Full pytest suite**: 7478 passed, 9 skipped, 3 xpassed, **0 failed**
- **ruff forward gate**: CLEAN
- **browser-smoke gate**: CLEAN (gate hardened mid-release to auto-detect the cached chromium revision)
- **Codex (regression)**: SAFE TO SHIP (simulated `sys.platform=win32`, verified POSIX modules not imported + all terminal guards complete + POSIX foreground still uses execv)
- **Opus (correctness)**: SAFE TO SHIP (POSIX path provably unchanged, all fcntl/termios/select guarded, Popen+exit correct; noted inherent-Windows trade-offs that aren't PR bugs)
Note: the Windows *runtime* path can't be executed on the Linux CI box; it was reviewed statically by both reviewers + the contributor's 209-line test (win32 simulated via monkeypatch). Linux/POSIX no-regression is fully verified.
Closes#1952.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
Fixes#2853. The `_terminal_shell_preexec_fn` added in `71d8a8fb` called
`prctl(PR_SET_PDEATHSIG, SIGTERM)` so orphaned PTY shells would die when
the WebUI process crashed. But that signal is **per-thread**, not
per-process, and WebUI runs `ThreadingHTTPServer`: every HTTP request is
handled in its own short-lived worker thread.
Flow that broke every Linux user:
1. User clicks the terminal toggle → frontend hits `POST /api/terminal/start`.
2. ThreadingHTTPServer spins up a worker thread to handle that one request.
3. The worker thread calls `subprocess.Popen(..., preexec_fn=...)`.
4. The shell calls `prctl(PR_SET_PDEATHSIG, SIGTERM)` in its preexec_fn.
Its registered "parent" is now the WebUI worker thread that called Popen.
5. The handler returns its JSON response and the worker thread exits.
6. The kernel sees the pdeathsig-parent thread has died and sends SIGTERM
to the PTY shell. The shell dies within ~10 ms of being created.
7. The reader loop sees EIO on the master FD, emits `terminal_closed`, and
the frontend writes `[terminal closed]`.
macOS users were unaffected because `libc.prctl` doesn't exist there —
`ctypes.CDLL(None)` returns a libc handle, `libc.prctl` raises
`AttributeError`, the bare-`except` swallows it, and the shell starts
with no pdeathsig configured.
Empirical verification on this Linux host (real PTY + `subprocess.Popen`
inside a `threading.Thread` that joins immediately):
with preexec_fn → proc.poll() == -15 (SIGTERM), master FD returns EIO
without preexec_fn → proc.poll() == None (alive), master FD returns "HELLO\\r\\n"
Same shell, same PTY, same threading topology as WebUI.
Fix
---
Drop the `preexec_fn` entirely. The orphan-shell-on-crash case the original
PR was navigating is rare for self-hosted single-user installs, and the
existing `atexit.register(close_all_terminals)` + explicit `close_terminal`
paths cover graceful shutdown. A future fix (option B in the issue) can
re-introduce pdeathsig pinned to a long-lived supervisor thread, but that
is a follow-up — this PR is the smallest unbricks-Linux-today change.
Tests
-----
- Invert `test_terminal_shell_uses_parent_death_signal_preexec` →
`test_terminal_shell_does_not_use_pdeathsig_preexec`: asserts
`preexec_fn` is NOT in the Popen kwargs.
- Add `test_pty_shell_survives_when_spawning_thread_exits`: spawns a
real PTY shell via `start_terminal` from a worker thread, waits for
the worker to join, asserts the shell is still alive after a half-second
grace window. This is the contract the original tests never exercised.
- Update `test_terminal_module_registers_graceful_shutdown_reaper` to
refuse re-introduction of the preexec_fn or the `libc.prctl(1, SIGTERM)`
call (treats either as a regression).
All 27 terminal-related tests pass locally.
Refs #2853
B1: fix stored XSS in MCP delete button — replace inline onclick with
data-mcp-name attribute + event delegation (panels.js)
B2: fix zip/tar-slip via startswith prefix collision — use
is_relative_to(); track actual extracted bytes instead of trusting
member.file_size (upload.py)
B3: add NVIDIA NIM endpoint to _OPENAI_COMPAT_ENDPOINTS and
_SUPPORTED_PROVIDER_SETUPS so provider is reachable (routes.py,
onboarding.py)
H1: add terminalResizeHandle element to index.html and return it from
_terminalEls() so resize-by-drag works (index.html, terminal.js)
H2: fix dead get_terminal() branch — return None for dead terminals
instead of always returning term (terminal.py)
H3: replace os.environ.copy() with a safe allowlist in PTY shell env
so API keys are not exposed inside the terminal (terminal.py)
H5: make model dedup deterministic — sort groups by provider_id
alphabetically before first-occurrence assignment (config.py)
H7: add pid regex validation before OAuth probe; constrain key_source
to a closed set of safe values (providers.py)
M8: add double-run guard for cron run-now — reject if job is already
tracked as running (routes.py)
ai-ag2026