docs: add ISO integration plan

2026-05-27 22:43:51 +02:00 · 2026-05-27 22:43:51 +02:00 · 2b01d8dad8
commit 2b01d8dad8
parent 2883151b5f
2 changed files with 305 additions and 0 deletions
--- a/README.md
+++ b/README.md
@ -7,6 +7,7 @@ with cache-first cost discipline (byte-stable prompt prefixes, cache-hit meterin
 **Status:** 8 crates; workspace gates are expected to be fmt/clippy/test/release green. Avoid fixed test-count status here — run the gate commands below for the current count. Phase 3 (coordination core) is in progress.

 Design doc + cutover plan: `docs/COLIBRI-CUTOVER-PLAN.md`.
+Next ISO integration plan: `docs/ISO-INTEGRATION-PLAN.md`.

 ## Workspace — 8 crates

--- a/docs/ISO-INTEGRATION-PLAN.md
+++ b/docs/ISO-INTEGRATION-PLAN.md
@ -0,0 +1,304 @@
+# Colibri ISO Integration Plan
+
+**Goal:** land a Rust-first Colibri control plane in the next Clawdie ISO build, replacing the TypeScript Clawdie control/supervision path with a FreeBSD-native `rc.conf` service stack.
+
+This plan intentionally shifts from prototype smoke testing to wiring the missing production pieces together. The live USB/Zed/GPU check remains a final acceptance gate, not a blocker while the test machine is useful for active development.
+
+## Direction
+
+Colibri becomes the source-of-truth control service on FreeBSD:
+
+```text
+rc.conf
+  -> colibri_daemon rc.d service
+    -> colibri-daemon
+      -> SQLite coordination store
+      -> scheduler/intake task board
+      -> session/prompt/cache discipline
+      -> glasspane supervision state
+      -> Pi agent/process spawning
+      -> future jail-backed Pi workers
+
+operator/display
+  -> colibri CLI
+  -> colibri-glasspane-tui / future native dashboard
+  -> optional Herdr remote/display plane from Linux/macOS only
+```
+
+Herdr is not being ported to FreeBSD. On the ISO, the native answer is Colibri daemon + glasspane/TUI/harness.
+
+## Current baseline
+
+Already landed and validated:
+
+- `colibri-daemon` starts socket server and background daemon loop together.
+- SQLite coordination store is embedded and works with WAL.
+- Scheduler/intake drains `intake-task` into SQLite on tick.
+- `colibri` CLI exists after `colibri-ctl` rename.
+- `colibri` task commands exist:
+  - `colibri create-task`
+  - `colibri list-tasks`
+  - `colibri intake-task`
+- FreeBSD `/tmp` daemon/scheduler/task CLI smokes passed on OSA.
+- Prototype FreeBSD rc.d service exists at `packaging/freebsd/colibri_daemon.in`.
+- T1.4 cache discipline exists:
+  - `CostMode::{Fast,Smart,Max}`
+  - `COLIBRI_COST_MODE` startup env
+  - escalation helpers
+  - tool-result compaction helpers
+  - `set-cost-mode` socket acknowledgement is runtime-only/status-intent for now.
+
+## Target ISO service layout
+
+### Installed files
+
+```text
+/usr/local/bin/colibri-daemon
+/usr/local/bin/colibri
+/usr/local/bin/colibri-smoke-agent          # optional diagnostic helper
+/usr/local/bin/colibri-tui                  # if TUI is included in image
+/usr/local/etc/rc.d/colibri_daemon
+/var/db/colibri/                            # persistent data + SQLite
+/var/run/colibri/                           # runtime socket/pid/log if used
+```
+
+### User/group
+
+Prefer a dedicated service account:
+
+```text
+user:  colibri
+group: colibri
+home:  /var/db/colibri
+shell: /usr/sbin/nologin
+```
+
+### Runtime paths
+
+```text
+COLIBRI_DAEMON_DATA_DIR=/var/db/colibri
+COLIBRI_DB_PATH=/var/db/colibri/colibri.sqlite
+COLIBRI_DAEMON_SOCKET=/var/run/colibri/colibri.sock
+COLIBRI_HOST=<hostname or image role>
+COLIBRI_COST_MODE=smart
+```
+
+### rc.conf contract
+
+Target knobs:
+
+```sh
+colibri_daemon_enable="YES"
+colibri_daemon_user="colibri"
+colibri_daemon_group="colibri"
+colibri_daemon_data_dir="/var/db/colibri"
+colibri_daemon_socket="/var/run/colibri/colibri.sock"
+colibri_daemon_db_path="/var/db/colibri/colibri.sqlite"
+colibri_daemon_host="$(hostname)"
+colibri_cost_mode="smart"
+```
+
+Optional later knobs:
+
+```sh
+colibri_daemon_log="/var/log/colibri-daemon.log"
+colibri_agent_binary="/usr/local/bin/pi"
+colibri_provider="deepseek"
+colibri_jail_enable="NO"
+colibri_jail_pool="colibri-pi"
+```
+
+## TypeScript-to-Rust migration inventory
+
+| Existing/legacy Clawdie role | Rust replacement | ISO readiness | Notes |
+|---|---|---:|---|
+| process supervisor / runner | `colibri-daemon` + `spawner` | partial | Local provider works; Pi binary/jail provider still needs hardening. |
+| session JSONL persistence | `colibri-daemon::session` | partial | Existing 3-region assembler is present; needs production fit with Pi flows. |
+| prompt/cache discipline | `colibri-daemon::cost` + session config | partial | Startup `COLIBRI_COST_MODE` works; live config mutation is future work. |
+| task board / coordination | `colibri-store` SQLite | yes for prototype | Needs ISO backup/inspection story but core is present. |
+| scheduler/intake | `colibri-daemon::scheduler` | yes for prototype | Socket intake + daemon loop validated on FreeBSD. |
+| Herdr display/supervision | `colibri-glasspane`, `colibri-glasspane-tui`, `colibri` | partial | Native FreeBSD path exists; responsive dashboard polish remains. |
+| operator CLI | `colibri` | partial | Core status/snapshot/spawn/task commands exist; status should expose more runtime details. |
+| remote display plane | Herdr on Linux/macOS | separate | Optional; not an ISO blocker for FreeBSD-native supervision. |
+| Pi worker isolation | future jail provider | missing | Important for ISO worker architecture; start with design + local Pi process first. |
+
+## Work plan
+
+### Lane A — ISO service hardening
+
+1. **Finalize rc.d service behavior**
+   - Keep review-only until package/image install step.
+   - Ensure `start_precmd` creates `/var/run/colibri` and `/var/db/colibri` with correct owner.
+   - Ensure `daemon(8)` wrapper uses pidfile/log/restart/user correctly.
+   - Verify stop removes socket and leaves DB intact.
+
+2. **Package/install manifest**
+   - Define exact files copied into ISO.
+   - Define ownership/mode for binaries, rc.d script, DB dir, runtime dir.
+   - Add a script or Make target for staging into an image root.
+
+3. **First-boot behavior**
+   - Empty DB is created automatically.
+   - WAL mode is enabled.
+   - Socket appears under `/var/run/colibri`.
+   - `colibri status` works as the unprivileged operator path.
+
+### Lane B — Operator visibility
+
+1. **Improve `colibri status`**
+   Include enough runtime state for ISO debugging:
+
+   ```text
+   daemon/version/host
+   data_dir
+   db_path
+   socket_path
+   cost_mode
+   scheduler_interval/status
+   sessions count
+   agents count/list
+   glasspane pane count
+   task counts by status
+   ```
+
+2. **Add `colibri service-check` or equivalent**
+   Optional but useful for ISO acceptance:
+
+   ```text
+   socket reachable
+   DB reachable
+   WAL active
+   daemon loop active
+   scheduler drains intake
+   glasspane snapshot available
+   ```
+
+3. **Keep task CLI tight**
+   Current commands are sufficient for now:
+
+   ```sh
+   colibri create-task --title "..."
+   colibri list-tasks --status queued
+   colibri intake-task --title "..." --capability freebsd
+   ```
+
+   Do not add full scheduler admin until after ISO service basics are stable.
+
+### Lane C — Pi spawning path
+
+1. **Local Pi process path**
+   - Confirm daemon can spawn the actual Pi binary in JSONL mode.
+   - Ensure stdout JSONL updates glasspane state.
+   - Ensure process kill/cleanup is reliable.
+   - Ensure session IDs are stable and visible.
+
+2. **DeepSeek optimized harness**
+   - Wire cost mode thresholds into actual session rotation/compaction decisions.
+   - Confirm immutable prefix remains byte-stable.
+   - Confirm appendable log and volatile scratch are not mixed.
+   - Add visible escalation logging.
+
+3. **Jail provider design**
+   Define but do not overbuild initially:
+
+   ```text
+   Provider::Jail
+     -> creates/starts named jail or reuses pool member
+     -> runs Pi inside jail
+     -> streams JSONL stdout to daemon
+     -> maps workdir/session volume safely
+     -> reports lifecycle to glasspane/store
+   ```
+
+4. **Jail provider implementation phases**
+   - Phase 1: static pre-created jail, daemon executes command inside it.
+   - Phase 2: pool of named Pi jails.
+   - Phase 3: task capability matching assigns work to jail workers.
+   - Phase 4: per-task ephemeral jail option if needed.
+
+### Lane D — Native dashboard / Herdr replacement
+
+1. **FreeBSD-native dashboard**
+   - Continue with `colibri-glasspane-tui` / harness rather than porting Herdr.
+   - Optimize for responsiveness on the ISO console/XFCE environment.
+   - Show panes, task queue, stalled agents, cost mode, host info.
+
+2. **Herdr remote/display plane**
+   - Keep Herdr Linux/macOS path as optional external observer.
+   - Do not block FreeBSD ISO on Herdr.
+
+3. **Zed/GPU acceptance**
+   - Previous image run is good enough to proceed with wiring.
+   - Final live USB acceptance should verify:
+     - XFCE starts.
+     - Zed launches with expected GPU behavior.
+     - Colibri service starts at boot.
+     - Native dashboard/CLI works locally.
+
+## Hermes task queue
+
+Implementation-heavy tasks suitable for Hermes:
+
+1. Add richer fields to daemon status response.
+2. Add tests for status exposing `cost_mode`, paths, task counts, pane counts.
+3. Wire cost mode thresholds into actual session rotation/compaction decisions if not already fully connected.
+4. Harden rc.d script against reboot/tmpfs `/var/run` behavior.
+5. Add image-root staging script or packaging notes for binaries + rc.d + directories.
+6. Start `Provider::Jail` design stub and type-level wiring without committing to full jail lifecycle yet.
+
+## Planning/documentation task queue
+
+Coordination/planning tasks:
+
+1. Keep this ISO plan current as implementation lands.
+2. Maintain Clawdie TS -> Colibri Rust migration matrix.
+3. Define final live USB acceptance checklist.
+4. Keep Herdr positioning clear:
+   - Linux/macOS remote/display plane.
+   - FreeBSD native path is Colibri/glasspane.
+5. Avoid stale fixed test counts in docs; cite commands and commits instead.
+
+## Final ISO acceptance checklist
+
+Before cutting the next ISO candidate:
+
+```sh
+service colibri_daemon start
+service colibri_daemon status
+colibri status
+colibri snapshot
+colibri create-task --title "iso smoke"
+colibri list-tasks --status queued
+colibri intake-task --title "iso intake smoke" --capability freebsd
+# wait one scheduler tick
+colibri list-tasks --status queued
+service colibri_daemon stop
+```
+
+Expected:
+
+- service starts as `colibri` user
+- socket exists under `/var/run/colibri`
+- DB exists under `/var/db/colibri`
+- WAL is active
+- `colibri status` reports correct paths and cost mode
+- direct task creation persists
+- intake drains through scheduler into SQLite
+- glasspane snapshot is available
+- service stop removes socket and leaves DB intact
+
+Live desktop acceptance:
+
+- boot ISO from USB
+- XFCE session starts
+- Zed launches and GPU behavior is acceptable for our requirements
+- Colibri native dashboard or CLI is responsive
+- no TypeScript Clawdie control service is required for core supervision
+
+## Non-goals for the next ISO
+
+- No Herdr-on-FreeBSD port.
+- No full remote scheduler administration unless it falls out naturally.
+- No mandatory Postgres path.
+- No root-required smoke outside service installation/startup.
+- No full dynamic config mutation; `set-cost-mode` remains runtime-only acknowledgement until live config update is designed.