A threading.Timer per RequestDiagnostics spawns one OS thread per request, held
alive until finish() cancels it. Under sustained /api/sessions poll load this
exhausts the per-process thread cap (RuntimeError: can't start new thread) and
the server stops accepting connections. Replace with a single lazy-init
watchdog daemon thread that scans pending diags every ~1s; public API
unchanged. Caps timeout-tracking at 1 thread/process regardless of request rate.
On the default local in-process backend, an approval card whose stream
ended while still pending (cancel/fork/provider-error/completion) left the
agent queue entry dropped and the _pending mirror reconciled away. Clicking
Approve/Deny then sent an approval_id that matched nothing, so the handler
returned a bare {ok:false}; since #4771 the frontend surfaces that as
'Approval response not accepted.' with a stuck card (reported by
@santastabber on v0.51.666 and b3nw).
Distinguish a genuinely stale card (no pending approval for the session ->
benign {ok:true, stale_cleared:true} so the UI clears the orphan) from a
stale-id click made while a DIFFERENT approval is live (still ok:false so it
can never resolve the wrong command -- #527 guard preserved). Frontend
clears the orphan card on stale_cleared even if the displayed id drifted.
Co-authored-by: b3nw <b3nw@users.noreply.github.com>
#4771 added an explicit relay-failure 409 (code: gateway_run_unavailable)
to /api/approval/respond so a gateway approval whose run is gone keeps its
card actionable instead of silently failing. But the 409 was emitted
regardless of backend.
On the DEFAULT local in-process backend, every guarded command parks an
entry in tools.approval._gateway_queues (via _await_gateway_decision), and
the local streaming notify callback mirrors it into _pending tagged with
_GATEWAY_MIRROR_FLAG. There is no gateway run and no _STREAM_RUN_IDS entry
by design. _gateway_pending_approval_without_run_id() therefore returned
True for that purely local approval, so the handler 409'd
("active run unavailable") and refused to resolve an approval that resolves
perfectly well locally. Local-backend users hit this on every approval
click (reported by b3nw on Discord; independently filed as #4948 by
claw-io, which also notes the local-runtime trigger).
Fix: gate the 409 on the WebUI actually running the gateway chat backend
(webui_gateway_chat_enabled). Local approvals fall through to the local
resolution path; the legitimate #4771 gateway behaviour is unchanged.
Tests:
- New tests/test_issue4771_local_approval_regression.py drives the real
_handle_approval_respond handler and asserts: local backend -> 200 +
agent thread unblocked; gateway backend -> 409 preserved. These fail
before the fix (409) and pass after.
- The two existing #4771 tests that asserted the 409 are updated to pin
HERMES_WEBUI_CHAT_BACKEND=gateway at respond time, since the 409 is now
gateway-deployment behaviour.
Reported-by: b3nw
Reported-by: claw-io (#4948)
Opus SHIP-WITH-FIXES (no code change required): the comment/docstring overstated
that the session-list-change listener covers ALL structural mutations. In-app
mutations fire it, but externally-driven changes (scheduled cron completion,
external CLI writes) don't — for those the 30s streaming TTL is the load-bearing
backstop (≤30s bounded, self-healing lag). Comment-only accuracy fix; logic
unchanged (both gates: perf fix correct, freeze effective, staleness bounded).
jkobject