fix(sessions): share the widening pool fairly across starved projects

The pass-level widening pool introduced by the previous commit was
billed per whole grant: the first starved project whose binding raw
window asked for a retry had its query widened to the full grant and
the pool was debited the entire width, even though the project only
read grant - share rows beyond its own share. Every later starved
project then found the pool empty, got no retry, and its conversations
behind its own raw window stayed absent from its sidebar on every
rebuild (greptile P1 on #6659, 2026-09-22 re-review).

Two changes, both inside _load_cli_sessions_uncached's recovery pass:

- the pool is debited only for what a retry READS beyond the project's
  own first-query width (widened - scoped_limit), not the grant; and
- each retry is funded by a fair share of what the pool still holds —
  one slice per starved project not yet tried — so a later project
  always keeps pool for its own first retry.

Granting min(query_limit, pool) whole would have re-created the same
starvation by reads instead of by billing: a project whose raw window
is consumed re-reads roughly its own width once before delivering, so
an honest retry must buy about one more width, and the first project
would read the pool down to zero. The share converges there in a few
bounded steps; anything still short after the pool is spent stays for
the next build. The pass's aggregate read bound moves from
2 * query_limit to 3 * query_limit — still constant in the number of
starved projects, which is what retires the original
len(starved) * query_limit blow-up.

test_compression_heavy_widening_is_one_pass_budget_not_per_project now
pins the new contract: every starved project gets a funded retry and
recovers the conversation behind its window; the pass stays inside the
aggregate bound.
This commit is contained in:
Rodrigo Gomes da Silva committed 2026-09-24 11:03:47 -03:00
1 parent 634f66774b
commit 0fba8b9654
2 files changed
+105 -31

No files matched your search

+66 -15
View File
@@ -8766,14 +8766,35 @@ def _load_cli_sessions_uncached(
# to every starved project made a build scan
# len(starved) * query_limit — an order of magnitude past
# the ceiling on a profile with ten compression-heavy
# projects. One aggregate pool, spent at the full width of
# each retry, caps the pass at 2 * query_limit of scoped
# reading regardless of how many projects are starved.
# projects. One aggregate pool caps the scoped reading of
# a pass at a constant multiple of `query_limit` however
# many projects are starved.
# Only the widening is rationed; the first query of every
# starved project is never skipped, which is what keeps the
# completeness guarantee above intact.
#
# (greptile P1, 2026-09-22 re-review) The pool is debited
# only for what a retry reads BEYOND the project's own
# first-query width — `widened - scoped_limit`, not the
# whole `widened` grant. Debiting the whole grant spent the
# entire pool on the first starved project whose window
# merely touched its own 8x oversample: it read
# `grant - share` extra rows but was billed the full grant,
# so every later starved project lost its retry and its
# older conversations stayed absent from its sidebar on
# every rebuild. With incremental debits a later project
# retries whenever real reads have left it pool; if an
# earlier project genuinely read the pool down to zero,
# that was real scan work — the aggregate bound, not a
# reservation, is what caps the pass. The bound is
# `3 * query_limit`: every starved project's first query
# sums to at most `query_limit`, a retrying project re-reads
# at most its own first width again, and the funded
# increments sum to at most the pool — still constant in
# the number of starved projects, which is what retires the
# original `len(starved) * query_limit` blow-up.
widening_budget = query_limit
for _kept, project_id in starved:
for starved_index, (_kept, project_id) in enumerate(starved):
remaining = effective_limit - kept_per_project.get(project_id, 0)
if remaining <= 0:
continue
@@ -8787,10 +8808,11 @@ def _load_cli_sessions_uncached(
# the loop would advance to the next project and leave
# this one under-delivered on every rebuild — the same
# failure the global query already guards against
# (greptile P1 on #6659). Re-query once, widened to
# whatever the pass's widening budget has left, which is
# what bounds the retry at the PASS level instead of per
# project (greptile P1 on #6659).
# (greptile P1 on #6659). Re-query once, widened to the
# project's own width plus whatever the pass's widening
# pool still has, which is what bounds the retry at the
# PASS level instead of per project (greptile P1 on
# #6659).
scoped_limit = remaining
while True:
scoped_rows, scoped_window_exhausted = cast(
@@ -8816,14 +8838,43 @@ def _load_cli_sessions_uncached(
or scoped_limit >= query_limit
):
break
widened = min(query_limit, widening_budget)
if widened <= scoped_limit:
# The pass has spent its widening budget. A
# retry now could only re-read a window this
# query already consumed, so stop instead of
# paying for a read that cannot deliver.
# (greptile P1, 2026-09-22 re-review) Each
# retry is funded by a FAIR SHARE of what the
# pool still holds — one slice per starved
# project not yet tried, so later projects
# always keep pool for their own first retry.
# The retry goes to `scoped_limit + share`:
# debiting the whole grant spent the entire
# pool on the first starved project whose
# window merely touched its own 8x oversample
# and left every later one unfunded, but
# granting `min(query_limit, pool)` whole also
# let the first project READ the pool down to
# zero — every project whose raw window is
# consumed re-reads its own width once before
# delivering, so an honest retry must buy
# roughly one more width. The share converges
# there in a few bounded steps; anything still
# short after the pool is spent stays for the
# next build — the alternative is the
# per-project grant that made the pass scan
# `len(starved) * query_limit` (the original
# greptile P1 on #6659).
pool_share = (
widening_budget
// max(len(starved) - starved_index, 1)
)
if pool_share <= 0:
# The pool is spent: no further starved
# project's retry can be funded. Whatever a
# project could not reach this pass stays
# for the next build — the alternative is
# buying reads nobody budgeted for.
break
widening_budget -= widened
widened = min(
query_limit, scoped_limit + pool_share
)
widening_budget -= widened - scoped_limit
scoped_limit = widened
except Exception:
logger.debug("Project-assigned CLI recovery pass failed", exc_info=True)
@@ -2398,11 +2398,15 @@ def test_compression_heavy_widening_is_one_pass_budget_not_per_project(
``query_limit`` — and it was granted to every starved project in turn. A
profile with ten compression-heavy projects therefore scanned ten times the
ceiling in a single build, growing with the number of starved projects
(greptile P1 on #6659). The widening now comes out of one pass-level pool,
so the scoped reading of a build is bounded by ``2 * query_limit`` however
many projects are starved — while every starved project still pays its own
first query, the completeness guarantee that a fixed per-build project cap
used to break (greptile P1 on #6659).
(greptile P1 on #6659). The widening now comes out of one pass-level pool
shared FAIRLY: each starved project's retry is funded by a slice of what
the pool still holds (one slice per project not yet tried), and the pool is
debited only for what a retry reads beyond the project's own width, so the
scoped reading of a build is bounded by ``3 * query_limit`` however many
projects are starved — while every starved project still pays its own first
query and still keeps pool for its own first retry, the completeness
guarantees neither a per-build project cap nor a first-come whole-grant
debit may break (both greptile P1 reviews on #6659).
Parametrized over the number of starved projects because that is exactly
what the budget must not scale with.
@@ -2465,16 +2469,35 @@ def test_compression_heavy_widening_is_one_pass_budget_not_per_project(
# A widened query is one that asks for more than the project's own share.
widened = [limit for _, limit in scoped_queries if limit > effective_limit]
# The pool is one query_limit wide and each retry is charged its full width,
# so exactly one project can take the widest retry — not one per project.
assert widened == [query_limit]
# And the whole pass stays inside the documented 2 * query_limit bound,
# The pool is shared fairly: EVERY starved project gets a widening retry —
# not just the first in line, and not zero of them. With one starved
# project the pool is entirely its own; with several, each retry is funded
# by a slice of what remains so later projects keep pool for theirs.
# Bounded by the number of retrying projects, not by the pool arithmetic
# (a single starved project legitimately spends the whole pool on one
# retry to `query_limit`).
assert len(widened) == min(quiet_count, len(quiet))
# Every retry covers the whole project: the reader's own 32x oversample
# turns a query at `limit` into a raw read of at most
# `limit * max(CANDIDATE_WINDOW_MULTIPLIERS)` rows, so the retry grants
# stay under `query_limit * 32` — but the retries must never buy the
# unbounded per-project read the original P1 described.
assert all(
limit <= query_limit * max(agent_sessions.CANDIDATE_WINDOW_MULTIPLIERS)
for limit in widened
)
# The retries are PAID FROM THE POOL, not granted per project: the sum of
# what they read beyond their projects' own widths stays within the pool
# plus each project's one honest re-read of its own width — the aggregate
# bound that keeps the pass constant in the number of starved projects.
assert sum(widened) <= len(widened) * query_limit
# And the whole pass stays inside the documented aggregate bound,
# whatever the number of starved projects.
assert sum(limit for _, limit in scoped_queries) <= 2 * query_limit
assert sum(limit for _, limit in scoped_queries) <= 3 * query_limit
# The widening the budget did fund still recovers the conversation behind
# the neediest project's window — the retry was rationed, not disabled.
widened_project = quiet[0]
assert f"{widened_project}-old" in {
session["session_id"] for session in sessions
}, "the one funded widening must still reach behind the binding window"
# The widening the budget funded still recovers the conversation behind
# EVERY starved project's window — the retry was rationed, not disabled.
for project_id in quiet:
assert f"{project_id}-old" in {
session["session_id"] for session in sessions
}, f"the funded widening must still reach behind {project_id}'s window"