From 978bb180114845f74223aa7062e8c374e4304737 Mon Sep 17 00:00:00 2001 From: Sam & Claude Date: Fri, 19 Jun 2026 17:29:30 +0200 Subject: [PATCH] docs: multi-agent multi-host gap analysis + implementation plan (Sam & Hermes) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New doc: docs/MULTI-AGENT-HOST-PLAN.md — gap analysis of the multi-host coordination stack after PR #83 (socat bridge + Python pollers). Defines 5 phases: Phase 1 (multi-agent task-board tests), Phase 2 (merge CLI surface gaps), Phase 3 (agent presence schema, deferred), Phase 4 (polling workflow test, deferred), Phase 5 (TCP bridge validation, FreeBSD-only). Immediate scope: Phases 1-2, all Linux-testable. Updated PRIORITY-HANDOFF-ISO-SPAWN-COST.md to mark Priorities 2 (poll_tasks) and 3 (cost mode enforcement) as done, fix stale test count (164 → 211), and cross-reference the new plan as the next sprint. 211 tests pass, clippy clean, fmt clean. --- docs/MULTI-AGENT-HOST-PLAN.md | 225 ++++++++++++++++++++++++ docs/PRIORITY-HANDOFF-ISO-SPAWN-COST.md | 25 ++- docs/README.md | 3 +- 3 files changed, 245 insertions(+), 8 deletions(-) create mode 100644 docs/MULTI-AGENT-HOST-PLAN.md diff --git a/docs/MULTI-AGENT-HOST-PLAN.md b/docs/MULTI-AGENT-HOST-PLAN.md new file mode 100644 index 0000000..969f9e6 --- /dev/null +++ b/docs/MULTI-AGENT-HOST-PLAN.md @@ -0,0 +1,225 @@ +# Multi-Agent Multi-Host — Gap Analysis & Implementation Plan + +**Created:** 2026-06-19 (Sam & Hermes) +**Status:** Phase 1-2 ready for implementation + +## Context + +The 0.10.0 milestone (ISO build, rc.d lifecycle, SIGTERM/socket fixes, release +gate) is staged for the FreeBSD build host. The next milestone is proving the +multi-agent, multi-host coordination model: multiple agents on different hosts +reading from the same Colibri task board, each picking up work by capability, +and reporting results back. + +PR #83 landed the first cross-host plumbing — a socat TCP bridge, Python polling +scripts, and a Hermes cronjob configuration. But the gap analysis below shows +that **the multi-host plane is packaged and documented but almost entirely +untested**. This document defines what needs to happen to close that gap. + +--- + +## Current architecture (as of PR #83) + +The multi-host stack lives **outside the Rust daemon**: + +``` + FreeBSD host (colibri-daemon) + └── Unix socket: /var/run/colibri/colibri.sock + └── socat bridge (colibri_bridge rc.d, port 9190, Tailscale) + └── TCP reachable from: debby, domedog + └── colibri_poll.py (Python, raw JSON-over-socket) + └── Hermes cronjob (2min poll / 5min work) +``` + +- **Transport:** `tokio::net::UnixListener` only — zero TCP in Rust. The socat + bridge is a shell-level relay. +- **Agent model:** `register-agent` stores name + capabilities + status + (`active`/`idle`/`offline`). No `host` field, no `last_seen`, no heartbeat, + no lease/TTL. +- **Task assignment:** `pick_agent()` matches by capability score (partial + match counts, highest score wins, tie → later-in-slice). `claim_task()` is a + blind UPDATE with no concurrency guard. +- **Polling:** `colibri_poll.py` queries `list-tasks status=started` filtered + by `agent_id`. `colibri_task_done.py` calls `transition-task`. +- **Spawning:** `poll_tasks()` in daemon.rs spawns agents for `Claimed` tasks, + skipping those with an existing session (idempotency guard). + +### Socket command inventory (17 commands, all Unix-socket) + +| Category | Commands | +|----------|----------| +| Daemon | `status`, `glasspane-snapshot`, `set-cost-mode` | +| Session | `list-sessions`, `get-session`, `compact-session` | +| Agent process | `spawn-agent`, `kill-agent` | +| Board | `list-tasks`, `create-task`, `transition-task`, `claim-task`, `intake-task` | +| Agent registry | `register-agent`, `list-agents` | +| Skills | `list-skills`, `register-skill` | + +### CLI surface (10 of 17 commands exposed) + +Missing from CLI: `claim-task`, `transition-task`, `register-agent`, +`list-agents`, `set-cost-mode`, `register-skill` (register-skill IS in CLI; +the others are socket-only). Remote agents currently must use raw Python +socket calls. + +--- + +## Gap analysis + +### What IS tested (single-host, single-agent) + +- Agent spawn → JSONL → glasspane → Done lifecycle +- Task create/intake/claim/start/done over socket +- SIGTERM cleanup + stale socket safety +- Session isolation with 2 agents (bypasses task board) +- Cost mode derivation in background rotation +- `pick_agent` unit tests: best match, offline exclusion, no-match, empty-required +- Scheduler tick drains intake queue +- `poll_tasks` spawns agent for a claimed task + +### What is NOT tested + +| # | Gap | Severity | Linux-doable? | +|---|-----|----------|---------------| +| 1 | **Multi-agent task-board contention** — `pick_agent` only tested with 0-1 agents; no capability-based multi-agent assignment test; no same-agent-multiple-tasks test | High | Yes | +| 2 | **CLI surface gaps** — `claim-task`, `transition-task`, `register-agent`, `list-agents` have no CLI; remote agents forced to use raw Python | Medium | Yes | +| 3 | **Agent presence model** — no `host` column, no `last_seen`, no heartbeat/lease; cannot detect stale remote agents | High | Yes (schema change) | +| 4 | **Remote-safe task claim** — `claim_task` is a blind UPDATE, no concurrency safety, no lease/TTL | Medium | Yes | +| 5 | **Python polling scripts** — `colibri_poll.py` and `colibri_task_done.py` have zero test coverage | Medium | Yes | +| 6 | **TCP bridge round-trip** — socat bridge untested end-to-end | Medium | Partial (needs socat or FreeBSD) | +| 7 | **Cross-host coordination** — no test simulates a remote agent claiming/transitioning a task over the bridge | High | FreeBSD only | + +--- + +## Implementation phases + +### Phase 1: Multi-agent task board tests (Linux, highest impact) + +#### 1a. Pure `pick_agent` unit tests — extend `scheduler.rs` test module + +Existing tests cover: best match (2 agents, different caps), offline exclusion, +no-match, empty-required. Add: + +| Test | What it proves | +|------|---------------| +| `test_pick_agent_partial_match_wins_over_no_match` | Agent with `["rust","freebsd"]` beats agent with `["python"]` for required `["freebsd"]` | +| `test_pick_agent_tie_breaking` | Two agents with same score — verify deterministic tie-break (later name wins) | +| `test_pick_agent_multiple_required_capabilities` | Required `["rust","freebsd"]` — agent with both beats agent with one | +| `test_pick_agent_active_status_eligible` | `status: "active"` is treated same as `"idle"` (both eligible) | + +#### 1b. Multi-agent board integration test — new file `crates/colibri-daemon/tests/multi_agent_board.rs` + +Full lifecycle: register 2 agents with different capabilities, submit 2 intake +tasks with matching capabilities, run scheduler tick, verify correct assignment, +run `poll_tasks`, verify both agents spawn and reach Done. + +``` +Register agent "freebsd-agent" with ["freebsd"] +Register agent "rust-agent" with ["rust"] +Submit intake "build on freebsd" required ["freebsd"] +Submit intake "write rust code" required ["rust"] +Run scheduler.tick(&state) + → verify task A agent_id == freebsd-agent.id + → verify task B agent_id == rust-agent.id +Run poll_tasks(&state) + → verify 2 agent handles in state.agents + → verify both tasks transitioned Claimed → Started + → wait for glasspane Done on both panes +``` + +This proves the core multi-agent coordination loop: **different agents get +different tasks by capability**. + +#### 1c. Same-capability multi-task test + +``` +Register agent "worker" with ["freebsd"] +Submit 2 intake tasks both requiring ["freebsd"] +Run tick + poll_tasks + → verify both tasks assigned to same agent (documents current behavior) + → verify both agents spawn independently (session isolation) + → verify both reach Done +``` + +Documents the current contention behavior (no guard against same agent getting +multiple tasks) and proves session isolation when one agent handles multiple +tasks. + +### Phase 2: Merge `feat/cli-register-agent` + add claim/transition CLI + +#### 2a. Merge `feat/cli-register-agent` (existing branch, 64 lines, client-only) + +The branch is clean and ready: +- `Command::RegisterAgent { name, capabilities }` + `Command::ListAgents` +- `parse_capabilities()` helper (reuses `--capability`/`--capabilities` pattern) +- `DaemonClient::register_agent()` + `DaemonClient::list_agents()` +- Usage text + +Enables: `colibri register-agent osa-agent --capability freebsd` and +`colibri list-agents`. + +#### 2b. Add `claim-task` and `transition-task` to CLI + +The two commands `colibri_task_done.py` currently does via raw socket. Adding +them to the CLI means remote agents can work entirely through the `colibri` +binary: + +``` +colibri claim-task --task-id --agent-id +colibri transition-task --task-id --status done|failed +``` + +Implementation: +- Add `Command::ClaimTask { task_id, agent_id }` and + `Command::TransitionTask { task_id, status }` variants +- Add `DaemonClient::claim_task()` and `DaemonClient::transition_task()` +- Add CLI parsing (follow existing `--flag value` pattern) + +#### 2c. Add CLI unit tests for new commands + +Parse tests matching existing `parses_task_commands` style. + +### Phase 3: Agent presence schema (deferred) + +Add `host` and `last_seen` columns to the agents table. Update `register-agent` +to accept an optional `host` parameter and update `last_seen` on each call. Add +a `heartbeat` socket command for liveness. Enables detecting stale remote agents. + +**Deferred** — requires schema migration and broader design discussion about +lease semantics. Not blocking the multi-agent test coverage goal. + +### Phase 4: Polling workflow integration test (deferred) + +Test the full poll → claim → work → done cycle from the agent's perspective, +simulating what `colibri_poll.py` does. Register two agents, create tasks with +different capabilities, verify each agent sees only its tasks via the poll +path, transition tasks to done. + +**Deferred** — depends on Phase 2 CLI additions (so the test can use CLI +commands instead of raw socket replication of the Python scripts). + +### Phase 5: Bridge validation (FreeBSD-only) + +Start `colibri_bridge` with socat on the FreeBSD host. Connect from a second +host via Tailscale TCP. Verify round-trip: status, list-tasks, claim-task all +work over the bridge. **Can only be done on FreeBSD 15 with the Tailscale +mesh.** + +--- + +## Summary + +| Phase | What | Files | Linux? | Status | +|-------|------|-------|--------|--------| +| 1a | `pick_agent` unit tests | `scheduler.rs` tests | Yes | Ready | +| 1b | Multi-agent board integration test | `tests/multi_agent_board.rs` (new) | Yes | Ready | +| 1c | Same-capability multi-task test | Same file | Yes | Ready | +| 2a | Merge `feat/cli-register-agent` | `colibri.rs` + `lib.rs` | Yes | Branch exists | +| 2b | Add `claim-task` + `transition-task` CLI | `colibri.rs` + `lib.rs` | Yes | Ready | +| 2c | CLI parse tests | `colibri.rs` tests | Yes | Ready | +| 3 | Agent presence schema | `schema.rs` + `lib.rs` + `socket.rs` | Yes | Deferred | +| 4 | Polling workflow test | `tests/` | Yes | Deferred (needs Phase 2) | +| 5 | TCP bridge validation | FreeBSD host | No | FreeBSD lane | + +**Immediate scope:** Phases 1-2. All testable on Linux with `cargo test` + +`cargo clippy` gate. No FreeBSD dependency for implementation. diff --git a/docs/PRIORITY-HANDOFF-ISO-SPAWN-COST.md b/docs/PRIORITY-HANDOFF-ISO-SPAWN-COST.md index 4481cda..e92ecab 100644 --- a/docs/PRIORITY-HANDOFF-ISO-SPAWN-COST.md +++ b/docs/PRIORITY-HANDOFF-ISO-SPAWN-COST.md @@ -1,16 +1,27 @@ # Priority Handoff — Three Focus Items Toward ISO Gate 1 -**Created:** 2026-06-14 (Sam & Hermes) -**Status:** open for any agent to pick up -**Replaces:** ad-hoc ISO work-lane priorities +**Created:** 2026-06-14 (Sam & Hermes) · **Updated:** 2026-06-19 +**Status:** Priorities 2 & 3 **done** · Priority 1 **staged for FreeBSD build** +**Superseded by:** `MULTI-AGENT-HOST-PLAN.md` for the next sprint -Round 2 audit is fully closed. All repos are green (164 tests, clippy clean, -fmt clean). The three items below are the highest-leverage work toward getting +Round 2 audit is fully closed. All repos are green (211 tests, clippy clean, +fmt clean). The three items below were the highest-leverage work toward getting a Colibri-backed ISO candidate and delivering on the core cost-discipline promise. -Each item is independently implementable on Linux with FreeBSD validation as -the final step. Items can be worked in parallel by different agents. +**Current status of each item:** + +- **Priority 1 (ISO boot validation):** Build wiring done, release runbook + landed (`clawdie-iso/docs/RELEASE-BUILD-RUNBOOK.md`), artifacts built on + FreeBSD host. Awaiting the 0.10.0 release build execution. +- **Priority 2 (Pi spawn end-to-end):** **Done** — `poll_tasks()` wired in + `9d443a4`, integration test `poll_tasks_spawns_agent_for_claimed_task` passes. +- **Priority 3 (Cost mode enforcement):** **Done** — cost mode is single source + of truth; `session_max_bytes`/`max_uncompacted_turns` removed from + `DaemonConfig`; per-append compaction derives from `CostMode::parse()`. + +The next sprint is multi-agent multi-host coordination — see +[`MULTI-AGENT-HOST-PLAN.md`](MULTI-AGENT-HOST-PLAN.md). --- diff --git a/docs/README.md b/docs/README.md index eb7599e..f662b94 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,4 +14,5 @@ A quick-reference guide to every document in this folder. | [`INTEGRATION-LAYERED-SOUL.md`](INTEGRATION-LAYERED-SOUL.md) | How Colibri consumes `layered-soul` reviewed context today vs planned | Agents | | [`ISO-ACCEPTANCE-RUNBOOK.md`](ISO-ACCEPTANCE-RUNBOOK.md) | Post-boot acceptance commands after staging Colibri into an ISO | Codex (FreeBSD) | | [`ISO-SERVICE-LAYOUT.md`](ISO-SERVICE-LAYOUT.md) | `rc.conf` service layout for the ISO image | All | -| [`PRIORITY-HANDOFF-ISO-SPAWN-COST.md`](PRIORITY-HANDOFF-ISO-SPAWN-COST.md) | **Current sprint**: ISO staging wiring, Pi spawn path, cost mode enforcement | All agents | +| [`MULTI-AGENT-HOST-PLAN.md`](MULTI-AGENT-HOST-PLAN.md) | **Current sprint**: multi-agent task-board tests + CLI surface gaps | All agents | +| [`PRIORITY-HANDOFF-ISO-SPAWN-COST.md`](PRIORITY-HANDOFF-ISO-SPAWN-COST.md) | ISO boot validation, Pi spawn path, cost mode enforcement (P2/P3 done) | All agents | -- 2.45.3