From 123f86196d25d13d9537e05331171af09381767c Mon Sep 17 00:00:00 2001 From: Sam & Claude Date: Fri, 26 Jun 2026 10:50:08 +0200 Subject: [PATCH] docs(guide): correct install docs to the clawdie binary (not just/clawdie-ai) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The install/ section was clawdie-ai docs ported into colibri: it documented a `just install` Node/tsx 20-step orchestrator + Ansible, none of which exists here (colibri is a Cargo workspace). colibri's real installer is the `clawdie` binary (crates/clawdie): `discover` / `plan` / `apply --yes` → ZFS datasets + clawdie service user + rc.d/systemd unit running colibri-daemon. - install.md: rewritten to the clawdie flow (commands, storage strategy, what apply provisions, --create-pool safety) from crates/clawdie/README.md. Adds a Scope section marking PF/jails/PostgreSQL/CMS provisioning as legacy clawdie-ai (not yet in the colibri installer) — no invented commands. - controlplane-install.md: rewritten to `cargo build -p clawdie` + apply. - index.md: "Install orchestrator / just install" → clawdie discover/plan/apply. - requirements.md: toolchain → Cargo/clawdie (drop Node/tsx), install command fixed, and a note that the runtime-philosophy prose is still clawdie-ai pending reconciliation. Out of scope (flagged for follow-up): first-boot.md (ISO QML rewrite pending), fresh-install-checklist.md, and the `just install/setup/doctor` mentions in the architecture pages — all broader clawdie-ai legacy. Co-Authored-By: Claude Opus 4.8 --- docs/guide/install/controlplane-install.md | 29 +- docs/guide/install/index.md | 2 +- docs/guide/install/install.md | 326 ++++----------------- docs/guide/install/requirements.md | 14 +- 4 files changed, 81 insertions(+), 290 deletions(-) diff --git a/docs/guide/install/controlplane-install.md b/docs/guide/install/controlplane-install.md index a07ee2f..141e529 100644 --- a/docs/guide/install/controlplane-install.md +++ b/docs/guide/install/controlplane-install.md @@ -1,24 +1,23 @@ --- title: 'Control Plane Install' -description: Install the Clawdie control plane with the standard orchestrator. +description: Install the Clawdie control-plane host service with the clawdie binary. --- -The control plane is installed by the standard orchestrator: +The control plane runs as the host-managed `clawdie` service (`colibri-daemon`), +provisioned by the `clawdie` binary: ```bash -just install +cargo build -p clawdie --release +clawdie apply --yes ``` -The root install runs the control plane as the host-managed Clawdie service. -The orchestrator configures PF, worker jails, PostgreSQL, hostd, service -installation, checkpoints, and resume state together. +This installs the storage datasets, the `clawdie` service user, and the rc.d +(FreeBSD) / systemd (Linux) unit that runs `colibri-daemon`. Preview first with +`clawdie plan`; `apply` is a dry-run until you pass `--yes`. See +[Install](./install/) for the full command set, storage strategy, and safety +notes. -If an install fails, resume from the failed step: - -```bash -just install-from-db -just install-from hosts -just install -- --dry-run -``` - -The canonical Bastille/Warden bridge name is `warden0`. +> **Scope.** This provisions the control-plane host service only. PF egress, +> worker jails, PostgreSQL, and the CMS/web service were handled by the legacy +> clawdie-ai `just` + Ansible orchestrator and are not yet part of the colibri +> installer. The canonical Bastille/Warden bridge name remains `warden0`. diff --git a/docs/guide/install/index.md b/docs/guide/install/index.md index 5dbbf1a..eeda9b8 100644 --- a/docs/guide/install/index.md +++ b/docs/guide/install/index.md @@ -21,5 +21,5 @@ Use this path for a new machine or a reinstall from boot media. Use this path when FreeBSD 15.x is already installed and you manage the host. - [Requirements](./requirements/) — host, runtime, and toolchain requirements. -- [Install orchestrator](./install/) — run `just install` and resume failed steps. +- [Install](./install/) — provision the host service with the `clawdie` binary (`clawdie discover` / `plan` / `apply --yes`). - [Fresh install checklist](./fresh-install-checklist/) — verify the completed install. diff --git a/docs/guide/install/install.md b/docs/guide/install/install.md index f018c51..2bb2a8c 100644 --- a/docs/guide/install/install.md +++ b/docs/guide/install/install.md @@ -1,301 +1,91 @@ --- -title: Install Orchestrator -description: Single-command install flow for Clawdie. +title: Install +description: Provision the Clawdie host service with the clawdie binary. --- -**Command:** `just install` +**Command:** `clawdie apply --yes` -## Quick start +Colibri-based Clawdie is installed by the **`clawdie`** binary (crate +`crates/clawdie`). It discovers the host's ZFS layout and provisions the +`clawdie` service: storage datasets, a service user, and the rc.d (FreeBSD) or +systemd (Linux) unit that runs `colibri-daemon`. -Clawdie tracks the FreeBSD 15.x line. The installer rejects FreeBSD 14.x and any unvalidated future major version. +This repo is a **Cargo workspace** — there is no `just`/npm install flow here. +Build the installer from the workspace: ```bash -just install +cargo build -p clawdie --release ``` -## ISO path - -Current ISO validation is converging on: live QML installer → first installed -boot → loopback-bound controlplane `/setup` with a one-time bootstrap token. -Provider keys and Telegram are configured there after install. The older -[`setup.txt` first-boot contract](./first-boot/) remains documented for legacy -non-interactive work and will be rewritten after live ISO validation. - -After the initial boot, the same `install` flow described here runs in the -background. - -Resume from a specific step after a failure: +## Commands ```bash -just install-from-db +clawdie discover # read-only: OS, ZFS pools, datasets, spare disks +clawdie plan [--pool NAME] # show the deploy plan (dry-run, no writes) +clawdie apply [--pool NAME] # dry-run unless --yes is given +clawdie apply --yes # provision: storage layout + install the service ``` -Skip service jails (db, git, cms) when running `DB_RUNTIME=host` or installing a second agent on an existing host: +`apply` is a **dry-run by default** and prints the full step plan; it writes to +disk only with `--yes`. When the host has exactly one ZFS pool it is selected +automatically; with several, pass `--pool NAME`. -```bash -just install-from hosts -``` +## Storage strategy -Print the step plan without running anything: +| Host | Behavior | +| ------------------ | ---------------------------------------------------------------------------------------------------------- | +| FreeBSD | ZFS is **required**; datasets are created under the chosen pool. | +| Linux + ZFS + pool | Same — datasets under the pool. | +| Linux, no ZFS/pool | Falls back to plain directories, and reports the ZFS benefits plus any spare disks that could host a pool. | -```bash -just install -- --dry-run -``` - -Skip ZFS checkpoints (e.g. no ZFS pool): - -```bash -just install -- --no-snapshots -``` - ---- - -## Step flow - -``` -just install - │ - ▼ - ┌─────────────────────────────────────────────────────────┐ - │ Detect ZFS dataset (zroot/bastille) │ - │ Parse --from / --no-snapshots / --dry-run │ - └─────────────────────────────────────────────────────────┘ - │ - ▼ -[ 1] onboarding first-boot setup seed or TUI fallback REQUIRED - │ -[ 2] environment host pkg baseline, bridge, locale REQUIRED - │ -[ 3] agent-config validate/write agent provider optional ─── warn on missing provider auth - │ └── pi missing → warn, continue -[ 4] pf write PF include (NAT egress) REQUIRED - │ └── 📸 snapshot: post-pf -[ 5] jails create worker jail (--create) REQUIRED - │ └── 📸 snapshot: post-jails -[ 6] git Git Service (git jail) DEFAULT ─── DB_RUNTIME=host → skip or use install-from hosts - │ └── 📸 snapshot: post-git -[ 7] forgejo Forgejo for Git Service DEFAULT ─── FEATURE_GITEA=NO → skip - │ -[ 8] db Data Service (PostgreSQL) DEFAULT ─── DB_RUNTIME=host → skip; DB on host instead - │ └── 📸 snapshot: post-db -[ 9] skills-memory built-in knowledge import DEFAULT ─── artifact.sql ships in tarball - │ -[10] skills-init Skills engine init (.nanoclaw) DEFAULT - │ -[11] cms Web Service (cms jail: Astro+nginx) DEFAULT ─── use install-from hosts to skip - │ └── 📸 snapshot: post-cms - │ -[12] ollama Local AI Models (Ollama jail .5) optional ─── FEATURE_OLLAMA≠YES → skip - │ -[13] llama-cpp Local AI Models (llama-cpp jail .5) optional ─── FEATURE_LLAMA_CPP≠YES → skip - │ -[14] hosts /etc/hosts + jail hostnames REQUIRED - │ -[15] mounts validate jail mounts REQUIRED - │ -[16] telegram-auth verify bot token optional ─── TELEGRAM_BOT_TOKEN unset → skip - │ -[17] service build + install rc.d service REQUIRED - │ -[18] hostd privileged host daemon (hostd) REQUIRED - │ -[19] identity-restore Supabase restore optional ─── SUPABASE_URL unset → skip - │ -[20] verify integrity check optional - │ └── 📸 snapshot: install-complete - │ - ▼ - ┌─────────────────────────────────────────────────────────┐ - │ Summary: N ok N warnings N skipped N failed │ - │ Snapshots taken: zroot/bastille@post-pf-… … │ - │ LLM providers: anthropic ✓ openai ✗ ollama ✗ … │ - └─────────────────────────────────────────────────────────┘ -``` - -The onboarding step prefers the first-boot setup file (`setup.txt`; see -[First boot](./first-boot/)). The interactive TUI wizard is the fallback when -the first-boot setup is absent or invalid; it sources locales from FreeBSD -itself, so any installed locale can be selected (`en_US.UTF-8`, -`zh_CN.UTF-8`, etc.) and is applied consistently. -`setup.txt` is the operator-intent contract, and `system.env` is the matching -hardware-intent contract. Inspect can populate both before the installer runs. -Set `DB_RUNTIME=host` in `.env` to provision PostgreSQL directly on the host instead of a db jail; `DB_HOST` defaults to `${AGENT_SUBNET_BASE}.1` so jails can reach it. Use `DB_COMPRESSION=lz4` (default) or `DB_COMPRESSION=zstd` for ZFS compression on host datasets. - -The root install owns shared platform services. It is not modeled as tenant zero. -`ASSISTANT_NAME` is display-only. Later additive tenants consume shared services such as: - -- Git Service -- Web Service -- Local AI Models - ---- - -## ZFS snapshots - -Snapshots are taken after each milestone step if a Bastille ZFS dataset is -detected. Two paths: - -| Context | Method | -| ------------------------------ | ------------------------------------------------------------- | -| Running as root | `zfs snapshot zroot/bastille@` directly | -| Non-root, `sudo` available | `sudo zfs snapshot zroot/bastille@` | -| Non-root, hostd socket present | `nc -U /var/run/-hostd.sock` (hostd `zfs-snapshot` op) | -| Neither | skip silently | - -Snapshot tags are suffixed with a Unix timestamp to prevent collisions on re-runs. - ---- - -## LLM providers - -The orchestrator never exits on missing LLM provider auth. OpenAI/OpenRouter/Anthropic -subscription auth is the recommended path for performance/price, but it is not a -hard installer requirement and all supported providers are peer options. At the -end of the run Clawdie prints a table of known providers: - -``` -△ LLM providers - anthropic ANTHROPIC_API_KEY ✓ configured - openai OPENAI_API_KEY ✗ not set - openrouter OPENROUTER_API_KEY ✗ not set - deepseek DEEPSEEK_API_KEY ✗ not set - zai ZAI_API_KEY ✗ not set - groq GROQ_API_KEY ✗ not set - azure AZURE_OPENAI_API_KEY ✗ not set - ollama OLLAMA_HOST ✗ not set -``` - -If no provider auth is found, it prints instructions: - -``` - No LLM provider auth found. Configure one after install and restart: - Set DEEPSEEK_API_KEY=... in provider.env for the default agent, - or add provider keys such as ANTHROPIC_API_KEY=sk-ant-... - service clawdie restart -``` - -The entire infrastructure (PF, jails, PostgreSQL, nginx, ZFS) has zero -LLM dependency. Provider auth is only consumed when the jail-runner spawns a -live response. Install and service start succeed without it. - -After install, use the controlplane setup page to finish provider and optional -Telegram configuration: +ZFS layout under the pool: ```text -https:///setup +/clawdie (container, canmount=off) +/clawdie/db -> /var/db/clawdie +/clawdie/log -> /var/log/clawdie ``` -Before setup is complete, do not expose this URL directly to the public internet. -Use local console/loopback access, or a tailnet/LAN path protected by TLS and -network allowlisting. The bootstrap token is a first-setup key, not a substitute -for bind/PF/TLS policy. +## What `apply --yes` provisions -If the printed bootstrap token is lost before setup completes, rotate a fresh -one on the host: +1. **Storage** — the datasets above (or plain `/var/db/clawdie` + + `/var/log/clawdie` on the plain-dirs fallback). +2. **Service user** — `clawdie` (nologin), owning the state directories + (`clawdie:clawdie`). +3. **Service** — the rc.d script (FreeBSD, via `daemon -u clawdie`) or systemd + unit (Linux, `User=clawdie`), installed and enabled to run + `/usr/local/bin/colibri-daemon`. -```sh -npm run setup-token -- rotate -``` +## Creating a pool (destructive) -### Agent runtime setup - -The agent runtime uses `DEEPSEEK_API_KEY` from `provider.env`. -Colibri autospawns zot in RPC mode — no separate login or auth store -needed. zot reads its identity from `$ZOT_HOME/AGENTS.md` (installed -automatically from the seed partition). - -Quick verification: +On a host with a spare disk and no usable pool: ```bash -zot --no-session --print "Reply with exactly: zot-ok" +clawdie apply --pool NAME --create-pool /dev/DEV --yes ``` -Expected output: +`--create-pool` runs `zpool create` on `DEV` and **destroys all data on it**, so +it is refused unless the disk is detected as empty (no partitions/filesystem/ +mount, not the root disk). Override that guard with `--force` only if you are +certain. -```text -zot-ok -``` +## Safety -### Control plane API auth +Disk-touching steps (`zfs`/`zpool create`, service install) run as root on the +target host. `discover`, `plan`, and a bare `apply` never write — preview with +`plan` first. -Agent subprocesses (pi, aider) authenticate back to the control plane -API using a shared secret. Generate one after install: +## Scope: what the installer does _not_ do (yet) -```bash -echo "CONTROLPLANE_SHARED_SECRET=$(openssl rand -base64 32)" >> .env -sudo service ${AGENT_NAME} restart -``` +`clawdie` provisions the **host service** only. The wider service mesh — PF +egress, worker jails, PostgreSQL, Forgejo/Git, and the CMS/web service — was +provided by the legacy clawdie-ai `just` + Ansible orchestrator and is **not yet +part of the colibri installer**. Older pages that reference `just install`, +`just setup-db`, `just doctor`, and similar describe that legacy flow, not this +repo; they are being reconciled to the `clawdie` model. -Without it, the startup log warns and agent-to-API calls are rejected. +## FreeBSD validation -### Service wrapper scripts - -The rc.d service (`/usr/local/etc/rc.d/${AGENT_NAME}`) runs -`run-${AGENT_NAME}.sh` which is **generated** by `just setup-service` at -install time. It is not tracked in git (`run-*.sh` is in `.gitignore`). -If missing, re-run `just setup-service` to regenerate it. - -Local LLM runtime is optional and configured via: - -``` -LOCAL_LLM_PROVIDER=none|ollama|llama_cpp -FEATURE_OLLAMA=YES|NO -FEATURE_LLAMA_CPP=YES|NO -FEATURE_OLLAMA_HPP=YES|NO # optional C++ bindings for Ollama clients -``` - ---- - -## Required vs optional steps - -Steps are **required** (failure stops install), **default** (runs out of the box -but can be disabled), or **optional** (skipped unless explicitly enabled). - -| Step | Status | Skip condition | -| ---------------- | -------- | ------------------------------------------------------------------ | -| onboarding | required | — | -| environment | required | — | -| agent-config | optional | warn on missing provider auth or missing agent binary | -| pf | required | — | -| jails | required | — | -| git | default | `DB_RUNTIME=host` or `install-from hosts` | -| forgejo | default | `FEATURE_GITEA` = NO | -| db | default | `DB_RUNTIME=host` skips jail, provisions on host instead | -| skills-memory | default | `bootstrap/skills-memory/artifact.sql` not present | -| skills-init | default | — | -| cms | default | `install-from hosts` skips; cms is a shared service, not per-agent | -| ollama | optional | `FEATURE_OLLAMA` ≠ YES | -| llama-cpp | optional | `FEATURE_LLAMA_CPP` ≠ YES | -| hosts | required | — | -| mounts | required | — | -| telegram-auth | optional | `TELEGRAM_BOT_TOKEN` not set | -| service | required | — | -| hostd | required | — | -| identity-restore | optional | `SUPABASE_URL` not set | -| verify | optional | warn on most check failures; fail on broken runtime integrity | - -A required step failure stops the install immediately and prints the resume -command. Default steps ship enabled (`FEATURE_GITEA=YES`, artifact.sql bundled) -— they run unless the operator explicitly opts out. - ---- - -## Individual steps - -The orchestrator calls each step through `setup/index.ts` as a subprocess -(`spawnSync` with `tsx`). This isolates `process.exit(1)` calls in individual -step modules from killing the orchestrator. - -Individual steps can still be run directly: - -```bash -just setup-db -just setup -- --step verify -``` - -`verify` checks more than the shared docs build. If tenant sites are -declared, it also checks their served output and publish-manifest consistency in -the CMS webroot. A site that is merely declared but not yet manually published -does not fail the step by itself; an inconsistent live publish state does. - -The step registry is implemented in `setup/index.ts`. +The disk-touching and service-install paths are validated on real FreeBSD/Linux +hosts; discovery, plan, and disk-candidacy logic are unit-tested. See +`docs/CLAWDIE-INSTALLER-HANDOFF.md` for the FreeBSD validation runbook. diff --git a/docs/guide/install/requirements.md b/docs/guide/install/requirements.md index 45b1d8a..916853b 100644 --- a/docs/guide/install/requirements.md +++ b/docs/guide/install/requirements.md @@ -5,7 +5,7 @@ description: Host, runtime, and toolchain requirements for Clawdie. What you need on the host before installing Clawdie. The ISO installer provisions most of this automatically; existing-host installs should verify -these requirements before running `just install`. +these requirements before running `clawdie apply`. ## Host @@ -18,13 +18,15 @@ these requirements before running `just install`. ## Toolchain -- **Rust** via rustup, installed under `/opt/clawdie/rustup`. - Required for native dependencies (SWC, tree-sitter). The ISO - installs this; existing-host installs run it from - [`Install orchestrator`](./install/). -- **Node.js + tsx** for the install scripts and runtime. +- **Rust** (Cargo) — the colibri control plane and the `clawdie` + installer are a Cargo workspace. Build the installer with + `cargo build -p clawdie --release`. See [Install](./install/). - **Bastille** for jail management. +> The legacy clawdie-ai `just` + Node/`tsx` orchestrator is not used in this +> repo; the sections below still describe the clawdie-ai product runtime and are +> being reconciled to the colibri model. + ## Runtime philosophy Clawdie is a single Node.js process that orchestrates jailed