_merge_state_row overwrote an already-represented lineage unconditionally, and
recovery pass 2 (the unassigned refill) merges into the same lineage-keyed
table pass 1 fills. A row already carrying a RESOLVED project_id could
therefore be replaced by a bare unassigned projection — deleting the chip this
PR exists to restore and swapping the row's identity from the lineage tip back
to its root.
Reachable wherever the SQL project_assignment filter and the Python lineage
projection disagree about where a compression lineage starts. They differ today
on a blank `source`: _is_continuation_session treats an empty/whitespace source
as "unknown, same conversation", while the lineage CTE compares
LOWER(TRIM(parent.source)) = LOWER(TRIM(child.source)) and so will not walk
from an assigned tip up to a blank-source root. The mixed first pass collapses
the pair into one assigned row; pass 2 then hands the root back as its own
"unassigned" conversation and clobbers it.
The merge is now one-way: overwrite only when the incoming row ADDS a resolved
assignment, or when the row on file has none to lose. Fixing the merge rather
than each possible divergence — it is the one place where "already resolved" is
knowable locally, and pass 2 by construction has less assignment context than
pass 1.
tests/test_issue2841_show_cron_sessions_toggle.py reads this call site as
SOURCE TEXT (`'("cron", "webhook", "kanban") if source_filter is None'`), so
routing it through the new BACKGROUND_CLI_SOURCES constant turned a
pre-existing test in the maintainer's own shard gate red for a
behaviour-neutral refactor.
Put the literal back at that one call site. BACKGROUND_CLI_SOURCES stays where
the kanban-chip fix actually needs it (the background-source check in
_interactive_row_project_id, and interactive_excluded for both recovery
passes). A new test builds the expected literal FROM the constant and asserts
it appears verbatim in api/models.py, so the two spellings cannot drift
without failing here rather than in an unrelated file.
Recovery pass 2 was gated on `first_pass_count >= unassigned_target`, reading a
short first pass as "the database has nothing left". It can also mean the mixed
pass ran out of RAW candidate window: compression segments are spent from it
before the logical slice, so a lineage-heavy profile gets fewer conversations
than it asked for and the unassigned quota stays short. Also run the refill when
assigned conversations inside the window displaced unassigned candidates, which
is the case a narrower unassigned-only query can actually repair. A profile with
no assignments and a small state.db still issues no extra query.
While here, spend the per-project assigned budget through `_merge_state_row`'s
return value instead of pre-incrementing, so the budget counts logical
conversations the way PROJECT_ASSIGNED_CLI_LIMIT documents, and seed it from the
assigned rows already inside the window so the bound stays a total rather than
one bound stacked on another. That accounting is behaviour-neutral today (the
assigned query is newest-first, so window rows are always the first rows it
returns) — it makes the documented contract hold by construction rather than by
that ordering accident, and removes an unused return value.
`_interactive_row_project_id` resolved state.db assignments for anything that
was not cron or webhook, but the interactive projection loop is also what a
`source_filter='kanban'` scan walks. The same kanban row therefore reported two
different chips depending on the caller: None in the default sidebar (its own
bounded second pass, upstream behaviour) and a resolved project_id under
?source=kanban.
Route every source that owns a bounded project-chip pass back through
`_state_row_project_id`, and name that set once (BACKGROUND_CLI_SOURCES) so the
exclusion list and the chip decision cannot drift apart.
`?all_profiles=1` passed `project_assigned_limit=False`, on the premise that
its `visible_session_limit=None` first pass already returns every conversation.
It does not: `_load_cli_sessions_uncached` maps `visible_session_limit is None`
to CLI_VISIBLE_SESSION_LIMIT for the interactive pass, so that view truncates
by recency exactly like the single-profile one. Only cron/webhook/kanban limits
reach the reader as `limit=` directly, where None does mean unbounded.
Net effect of the old value: in the "All profiles" sidebar a project-assigned
CLI session older than the newest 20 interactive conversations was missing from
/api/sessions while its project chip still claimed it — the #6659 bug itself,
in the one view that shows every profile at once. Recovery pass 2 (the
unassigned refill) still ran there, so the two passes disagreed as well.
Pass PROJECT_ASSIGNED_CLI_LIMIT instead of None so the per-project bound and
the PROJECT_ASSIGNED_CLI_SCAN_CEILING guard apply per profile, rather than
issuing an unbounded assigned scan for every profile on the box.
Letting assigned sessions past the *recent* cap was right; letting them past
*all* bounds was not. Three linked defects from review of #6659:
1. The assigned list was effectively unbounded. models.py's 200 cap only applied
to state.db's own pass; imported CLI sidecars from all_sessions() never pass
through it and _cap_recent_cli_sessions() retained every assigned row (1,000
assigned rows -> 1,000 returned). The FINAL MERGED payload is the only place
that sees every source of assigned rows, so bound it there - per project, so
one busy project cannot evict another's history (the >200-assigned case
greptile flagged), and counting logical conversations, not raw rows.
2. Assigned rows spent the unassigned quota: 3 assigned + 20 unassigned yielded
17 unassigned, and the model fetched 20 rows total up front so bumping only
the route counter could not refill. Count the two separately in the route cap
and add a bounded unassigned refill pass, keyed on the lineage, so 20 unique
unassigned logical conversations survive sidecar/lineage dedup. The refill
only runs when the window was actually saturated AND came up short.
3. A deleted or cross-profile project_id hid the session outright: it was
surfaced unchecked, so the row left "Unassigned" and its overflow became
default_hidden with no chip left to reveal it. Resolve non-system ids against
profile-scoped load_projects() and coerce a miss to None BEFORE either cap -
the mirror of the bug this PR fixes.
read_importable_agent_session_rows() grows a `project_assignment` filter
('assigned'/'unassigned') replacing the require_project_id boolean. Both arms
key on the whole compression lineage and are exact complements, so neither
budget can double-count a lineage.
_state_row_project_id() keeps its (sid, source) signature: master's kanban pass
calls it that way, and giving it a row-dict signature turned that whole pass
into a swallowed TypeError that dropped every kanban row.
read_importable_agent_session_rows()'s `limit` is a bound on LOGICAL
conversations (the slice runs after lineage projection), but the bounded
candidate CTE spends its raw window on compression segments and rows the
post-projection visibility filters drop. A lineage-heavy profile therefore got
back far fewer conversations than it asked for, and silently: with twelve
ten-segment chains, limit=20 returned 8.
Retry once with a wider window when the first one was fully consumed AND still
under-delivered. The common case is unchanged - the first window is still 8x the
requested limit and the retry never runs when the db simply has nothing more to
give - so the #2628 capped-scan guarantee holds.
compression_tip() resolves a project assignment across the whole lineage, so it
can find one recorded on a newer EMPTY continuation. When the root is still the
freshest *importable* segment, the `tip is row` early return appended the root
without applying that resolved id and the assignment was silently dropped.
Copy the root and apply lineage_project_id before appending, matching what the
merged-tip path below already does.
Two independent 20-session caps hid older project-assigned CLI sessions:
models.py only queried the 20 most recent, and routes.py capped the API
payload again. Fixing the query alone was not enough.
- agent_sessions.py: optional project_id filter; propagate the freshest
project_id along a compression lineage (compression_tip now returns a
3-tuple) so a Project set only on the tip is not lost
- models.py: second bounded query for older project-assigned sessions;
skipped when the schema has no project_id; dedupe by id/lineage
- routes.py: keep the 20 recent, drop unassigned overflow, preserve
assigned overflow as default_hidden (revealed by the project chip)
Adds regression tests for the limit, the root/tip lineage cases and the
route cap.
Layer three additive pieces on top of the #7021 / #6931 continuation
predicate without changing its shape or CONTINUATION_STARTED_AT_TOLERANCE_SECONDS:
- _reset_from is a boundary marker alongside _branched_from/_delegate_from
(Hermes Agent stamps it in gateway/session_recovery.py and filters it in
_NON_CONTINUATION_CHILD_FILTER_SQL). The same predicate drives the sidebar
listing, lineage metadata/report and the transcript stitcher.
- _branch_markers() returns ('none' | 'markers' | 'unknown', markers);
unparsable, non-object, too-deep, or non-string marker payloads are
'unknown' and fail closed as a lineage boundary instead of degrading to
"no markers" (fail open).
- gateway_watcher._snapshot_hash() hashes the complete canonicalized published
payload, and the cheap fingerprint covers model_config, so a projection change
to title/created_at/metadata with fixed (id, updated_at, message_count) emits
sessions_changed.
The row-id fast path for a marked native-image mirror row continued
without advancing state_replay_idx, unlike the ordinary and multimodal-
mirror replay paths. The edited checkpoint then never read as consumed,
so a rerun reply that exists only in state.db was truncated. Master (no
marker branch) keeps that reply.
Advance the replay index when the resolved sidecar row is the next
expected visible row, the same guard the mirror branch uses. Adds a
regression that fails without the fix: sidecar [u1,a1,img#42,a_img,uE],
state [.., marked img#42, a_img, u_old, a_old, uE, aE], watermark 900,
boundary 861 -> aE survives, u_old/a_old stay out.
Found by the release-manager advisor gate on #7754.
Apply the existing truncation and checkpoint-order eligibility to marked
state.db image rows before insertion. Keep removed turns out of full,
limited, and model-context reads without changing no-watermark replay.
Preserve conflicting provider payloads for a uniquely proven retained
image row across Undo without restoring distinct removed rows.
Assisted-by: Hermes Agent:gpt-6-sol
Assisted-by: Codex:gpt-6-luna
Review gate on #7720 (two regressions reproduced against master with the
installed Agent):
1. Deleting a profile's last MCP server left ADOPTED tools callable after
/reload-mcp. The scoped teardown only stops connections the profile owns,
and the Agent's discover_mcp_tools() returns before its reconcile step
when the profile's config has no server, so an adopted shared connection
was never judged against the config. /reload-mcp now calls
register_mcp_servers({}) under the profile's scope before rediscovery:
the Agent's overlay heal detaches only this profile's overlay for servers
its config no longer has; the owner's connection keeps running (master's
wildcard shutdown stopped the owner too).
2. Portable plugin MCP servers are merged by the Agent at runtime and never
appear in the profile's raw ``mcp_servers``; the registry fallback
required a config entry and dropped every plugin tool from
/api/mcp/tools and the Notes inventory. The profile's own registry slot
(registry_tool_owned_by_view) is the isolation check; the config is not
an allowlist any more.
Tests (red before, green after): agent-free unit tests for both (the fake
agent now models the overlay heal and discovery's empty-config early
return), and a real-Agent integration test for the adopted-then-deleted
case (owner connection survives, adopter slot emptied, "Removed" reported).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review follow-up on #7720 (Greptile P2): the external Notes drawer reads the
same profile-bound MCP inventory as the MCP panel but dropped the
``runtime_scope`` label, so a profile whose runtime scope could not be
confirmed rendered its note sources as inactive/configured with no
explanation. ``/api/notes/sources`` now returns ``runtime_scope`` and the
drawer shows the existing ``mcp_runtime_scope_unavailable`` notice (same key,
no new locale strings) when it is ``unavailable``.
Tests: the skewed-routing unit test now also asserts the notes response
carries ``runtime_scope: "unavailable"`` with its sources inactive.
docs/troubleshooting.md mentions the drawer next to the panel.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two profiles configuring a same-named MCP server shared one connection and
the MCP settings panel showed another profile's runtime ("Active, N tools"
next to an empty inventory); /reload-mcp stopped every profile's servers and
rediscovered from the process-default config.
- api.profiles._set_hermes_home() is the single writer of the process-profile
home: it records get_process_profile_home() and hands it to Hermes Agent's
hermes_constants.pin_process_hermes_home when available, at startup and on
switch_profile(process_wide=True), so the per-turn HERMES_HOME mirror no
longer makes every turn look like the process profile (bare-name,
cross-profile MCP keys) and the anchor cannot drift from HERMES_HOME.
- Run get_mcp_status/registry reads and /reload-mcp inside
api.mcp_runtime.mcp_runtime_scope(): request profile's Hermes-home override
and secret scope (root included), no os.environ mutation, restored on exit.
- One ledger predicate, ledger_key_serves_view() (owned via
_server_scope_keys/key scope, or adopted via _server_tool_scopes), is shared
by status, inventory and the reload summary. The Agent's launch-profile view
is process-wide and name-indexed, so filter_runtime_status_to_view()
rebuilds connected/tools/status/error from the entries serving the profile:
the root profile never shows a routed profile's connection (nor lets its own
parked failed-spawn task borrow a routed profile's live same-named one).
- Inventory lists only tools in the profile's own registry slot for servers it
configures; responses carry runtime_scope.
- Fail closed when the agent's routing does not match the resolved profile:
hide runtime data (runtime_scope "unavailable", shown as a notice in the MCP
panel, 14 locales) and refuse /reload-mcp.
- /reload-mcp shuts down only the profile's own ledger names (parked
included), drops its own connect cooldown/error entries so a failed spawn is
retried (the wildcard cleared every cooldown; a scoped shutdown only clears
the live keys it tears down), rediscovers under the profile's home and
counts live connections only. The bare wildcard remains for agents without
profile-scoped MCP.
- Signature introspection is centralized (accepts_keywords) and falls back to
the bare call instead of blanking every status; ledger key helpers resolve
to the Agent's tools.mcp_tool_scope when present; tools.mcp_tool is
resolved by full name so stubbed agents still work.
- ARCHITECTURE.md §4.10 and docs/troubleshooting.md document the boundary.
- Tests: agent-free contract (models adoption, parked tasks, cooldowns and the
process-wide root view) and real-Agent integration (fake stdio MCP server,
skips without hermes-agent + mcp SDK).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Keep image-turn context tagging and upstream's typed steer-wrapper cleanup
in the same settlement path. Verify the combined recovery and streaming
behavior against current master.
Assisted-by: Hermes Agent:gpt-6-sol
Count a proved WebUI-owned pending row in the display baseline before
accepting a state.db self-heal. Without assistant output past that turn,
leave pending state for the run journal to restore partial text and the
interruption marker instead of clearing it prematurely.
Assisted-by: Hermes Agent:gpt-6-sol
Assisted-by: Codex:gpt-6-luna
Assisted-by: Claude Code:claude-opus-5-5
When a dead stream is reconciled from state.db, retain the submitted
prompt and attachment in the sidecar while keeping the Agent-enriched
row in model context. Use one parent snapshot for full-session conflict
projection so the sidecar owner is stable across the display merge.
Assisted-by: Hermes Agent:gpt-6-sol
Assisted-by: Codex:gpt-6-luna
Assisted-by: Claude Code:claude-opus-5-5
Project conflicting provider payloads after eligible parent-only display
rows are stitched, so a partial continuation keeps the parent bubble and
its attachments without duplicating the durable user row.
Assisted-by: Hermes Agent:gpt-6-sol
Assisted-by: Codex:gpt-6-luna
Apply the same display-only conflict projection to full and paginated
session reads before cache and window selection. Keep both provider payloads
for model replay while retaining a single visible sidecar bubble.
Assisted-by: Hermes Agent:gpt-6-sol
Assisted-by: Codex:gpt-6-luna
Record pending-turn proof under caller-owned session locks during normal
runs and credential recovery. Preserve both conflicting provider payloads
for replay, but display one sidecar-owned bubble when row identity and
visible content are unambiguous.
Assisted-by: Hermes Agent:gpt-6-sol
Assisted-by: Codex:gpt-6-luna
Persist trusted pending Agent timestamp evidence before the model call so
session reads, including metadata polls and cold reloads, keep only the
submitted prompt and attachment visible. Preserve conflicting provider
payloads without guessing which copy is newer.
Assisted-by: Hermes Agent:gpt-6-sol
Assisted-by: Codex:gpt-6-luna
The suppression predicate _transcript_already_advanced_past_pending()
required a run-journal terminal 'completed' state, but its caller in
_apply_core_sync_or_error_marker already returns early on that same
condition. The predicate branch was therefore unreachable dead code: it
could never suppress anything, and stale-cancel recovery could still
append a duplicate recovered user row, a journal clone and an
interruption marker for a turn that actually finished.
Derive the completion evidence from the crash-safe turn journal instead:
_turn_journal_records_completion() reports True when the exact-stream
'completed' event is the latest terminal state for the pending stream id.
The run journal's terminal write is not durable evidence for this path
(it is consumed by the caller's early return), so the turn journal is the
only still-observable source. A cancelled/crashed turn (interrupted)
keeps recovery active, and completion for another stream of the same
session never settles the pending one.
Also record the same completion event on the Gateway success writeback
path, so Gateway-backed runs have the equivalent completion persistence
instead of leaving the run journal's lost terminal write unrecoverable.
Regression coverage:
- test_turn_journal_helper_reports_completed_for_matching_stream
- test_turn_journal_helper_rejects_interrupted_turn
- test_advances_past_pending_from_turn_journal_without_run_journal_terminal
- test_full_recovery_suppresses_duplicates_on_turn_journal_completion
- test_full_recovery_appends_rows_on_turn_journal_completion_loss
- test_gateway_chat_worker_records_turn_journal_completion
Reverting the predicate gate turns the two new CORE regressions red;
reverting the Gateway write turns the Gateway regression red.
A persisted message_count is only authoritative when the write that
produced the messages array also vouched for it. Sidecars written by an
older writer (recovery materialization copied the denormalized state.db
count over real rows) could carry a count stale against the array beside
it, and the prefix shortcut then reads a real shrink as a growth and
skips the #1558 backup.
save() now stamps _mc_v alongside message_count, and _prefix_message_count
refuses any count without the current marker: those files get the full
parse, exactly as on the legacy path. One save re-marks the file, so the
fast path resumes for steady state.
Regression covers the unmarked stale-count shape and the re-marking
transition; mutation-checked by removing the gate (the unmarked fixture
then skips the backup, as the review's reproduction predicted).
The #7535 ownership guard skipped the global model.base_url backfill for
two shapes that master handled:
- A `model` section with `base_url` but no `provider` declares no owner,
so `_providers_match_for_context("", <session provider>)` was false and
the endpoint was dropped, silently falling back to 256,000 instead of
querying the configured /v1/models (the profile-setup path writes this
shape).
- `opencode_go` and `opencode-go` are two spellings of the same built-in
provider, but `_canonical_context_provider` does not fold underscores,
so the owner session lost its URL.
Allow the backfill when no owner is declared, and compare both sides
through api.config._canonicalise_provider_id, which folds `_` to `-` while
keeping distinct `custom:*` slugs distinct. The #7535 case (a custom:ark
owner vs an opencode-go session) still resolves to an empty base_url.
Rodrigo Gomes da Silva