fix(scheduler): re-queue undispatchable intake tasks + de-flake board tests #352

Merged

clawdie merged 2 commits from fix-scheduler-drop-and-flaky-tests into main 2026-07-01 07:25:17 +02:00

Conversation 0 Commits 2 Files changed 2 +111 -61

clawdie commented 2026-07-01 07:07:48 +02:00

Owner

Copy link

Does both #1 (deterministic tests) and #3 (fix the intake drop) from the flaky-test discussion — the deadline bump was abandoned because it didn't actually fix the flake.

#3 — production: stop silently dropping intake tasks

The scheduler's intake path drained its queue and dropped any request that found no capable/present agent that tick. The task stayed queued in the store but was never re-routed — a silent orphan. Now an undispatchable request is re-queued so a later tick routes it once a capable agent registers/returns. A claim that loses the race (task already handled) is still dropped, so there's no spin.

• New deterministic unit test intake_requeues_until_a_capable_agent_appears.

#1 — tests: drive the scheduler deterministically

Two multi_agent_board tests were flaky under cargo test parallel load. Diagnosis (the deadline was a red herring — 15s still failed 2/8):

• scheduler_routes_intake_tasks_by_capability: the 50ms background loop got CPU-starved past the poll deadline. It's contention, not slowness.
• one_agent_handles_two_tasks_isolated_sessions: run_loop's startup agent-prune (#348) deleted the manually-registered agent (it has no spawn handle) before the manual claim-task, so the claim came back a conflict → ok=false.

Both now run without a background loop: the routing test drives one explicit scheduler.tick() and checks once; the manual-claim test needs no loop at all. Neither races a timed loop or the startup prune.

Verification

• Full daemon suite: 0 failures across 12 stress runs (was ~2/3).
• fmt + clippy -D warnings clean; scheduler unit tests green.

🤖 Generated with Claude Code

Does both #1 (deterministic tests) and #3 (fix the intake drop) from the flaky-test discussion — the deadline bump was abandoned because it didn't actually fix the flake. ## #3 — production: stop silently dropping intake tasks The scheduler's intake path drained its queue and **dropped** any request that found no capable/present agent that tick. The task stayed `queued` in the store but was never re-routed — a silent orphan. Now an undispatchable request is **re-queued** so a later tick routes it once a capable agent registers/returns. A claim that loses the race (task already handled) is still dropped, so there's no spin. - New deterministic unit test `intake_requeues_until_a_capable_agent_appears`. ## #1 — tests: drive the scheduler deterministically Two `multi_agent_board` tests were flaky under `cargo test` parallel load. Diagnosis (the deadline was a red herring — 15s still failed 2/8): - **`scheduler_routes_intake_tasks_by_capability`**: the 50ms background loop got CPU-starved past the poll deadline. It's contention, not slowness. - **`one_agent_handles_two_tasks_isolated_sessions`**: `run_loop`'s startup agent-prune (#348) deleted the **manually-registered** agent (it has no spawn handle) before the manual `claim-task`, so the claim came back a conflict → `ok=false`. Both now run **without a background loop**: the routing test drives one explicit `scheduler.tick()` and checks once; the manual-claim test needs no loop at all. Neither races a timed loop or the startup prune. ## Verification - **Full daemon suite: 0 failures across 12 stress runs** (was ~2/3). - fmt + clippy `-D warnings` clean; scheduler unit tests green. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

clawdie added 1 commit 2026-07-01 07:07:54 +02:00

fix(scheduler): re-queue undispatchable intake tasks; de-flake board tests ... 66aaf48e10

#3 (production): the intake path drained the queue and *dropped* any request
with no capable/present agent that tick — silently orphaning the task (it
stayed `queued` in the store but was never re-routed). Now such a request is
re-queued and a later tick routes it once a capable agent appears. Claim races
(task already handled) are still dropped, so there's no spin.

#1 (tests): two multi_agent_board tests were flaky under parallel load because
they spawned the timed background loop and raced it. Root causes found:
- routing test: the 50ms scheduler loop could be CPU-starved past the poll
  deadline (a bigger deadline did NOT fix it — it was contention, not slowness);
- single-agent test: run_loop's startup agent-prune (#348) deleted the
  manually-registered agent (no spawn handle) before the manual claim, so the
  claim returned a conflict.
Both now run deterministically without a background loop — the routing test
drives one explicit `scheduler.tick()`; the manual-claim test needs no loop at
all. Full daemon suite: 0 failures across 12 stress runs (was ~2/3).

- scheduler: re-queue-on-no-agent + deterministic unit test
- tests: drop bg loop; explicit tick / manual claims

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

claude-domedog added 1 commit 2026-07-01 07:19:39 +02:00

test: de-flake two_agents_claim_distinct_tasks_on_the_board too ... 22a2167549

Same latent bug as the other manual-claim board test: it spawned a timed
scheduler loop whose startup agent-prune (#348) can delete the just-registered
agents before the manual claim, and only passed because the prune usually wins
the race by microseconds. It does manual claims only, so it needs no loop —
dropped it (and the now-unused run_loop/DaemonLoopConfig imports).

All three board tests are now deterministic; full daemon suite 0/12 stress.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

clawdie merged commit d3887bd899 into main 2026-07-01 07:25:17 +02:00

clawdie referenced this pull request from a commit 2026-07-01 07:25:20 +02:00

Merge pull request 'fix(scheduler): re-queue undispatchable intake tasks + de-flake board tests' (#352) from fix-scheduler-drop-and-flaky-tests into main

clawdie deleted branch fix-scheduler-drop-and-flaky-tests 2026-07-01 07:25:23 +02:00

clawdie referenced this pull request from a commit 2026-07-01 08:09:44 +02:00

docs(wiki): refresh dispatch, harness, task-board, and event pages for 0.12.0 (Sam & Hermes)

clawdie referenced this pull request 2026-07-01 08:09:55 +02:00

docs(wiki): refresh dispatch, harness, task-board, and event pages for 0.12.0 (Sam & Hermes) #357

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

doctor-fix

colibri doctor --fix dispatch targets and upgrade self-heal

  

first-proof blocker

gates the first proven end-to-end

  

hardening

security/robustness follow-up, not a first-proof gate

No labels

doctor-fix

first-proof blocker

hardening

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

clawdie/colibri!352

No description provided.