clawdie/colibri
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 48

docs(guide): correct install docs to the clawdie binary (drop clawdie-ai just/Ansible) #213

Merged
clawdie merged 1 commit from docs/install-correct-to-clawdie-binary into main 2026-06-26 15:05:25 +02:00
Conversation 0 Commits 1 Files changed 4 +81 -290
4 changed files with 81 additions and 290 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
docs/guide/install/controlplane-install.md
View file

@ -1,24 +1,23 @@
---
title: 'Control Plane Install'
description: Install the Clawdie control plane with the standard orchestrator.
description: Install the Clawdie control-plane host service with the clawdie binary.
---
The control plane is installed by the standard orchestrator:
The control plane runs as the host-managed `clawdie` service (`colibri-daemon`),
provisioned by the `clawdie` binary:
```bash
just install
cargo build -p clawdie --release
clawdie apply --yes
```
The root install runs the control plane as the host-managed Clawdie service.
The orchestrator configures PF, worker jails, PostgreSQL, hostd, service
installation, checkpoints, and resume state together.
This installs the storage datasets, the `clawdie` service user, and the rc.d
(FreeBSD) / systemd (Linux) unit that runs `colibri-daemon`. Preview first with
`clawdie plan`; `apply` is a dry-run until you pass `--yes`. See
[Install](./install/) for the full command set, storage strategy, and safety
notes.
If an install fails, resume from the failed step:
```bash
just install-from-db
just install-from hosts
just install -- --dry-run
```
The canonical Bastille/Warden bridge name is `warden0`.
> **Scope.** This provisions the control-plane host service only. PF egress,
> worker jails, PostgreSQL, and the CMS/web service were handled by the legacy
> clawdie-ai `just` + Ansible orchestrator and are not yet part of the colibri
> installer. The canonical Bastille/Warden bridge name remains `warden0`.

2
docs/guide/install/index.md
View file

@ -21,5 +21,5 @@ Use this path for a new machine or a reinstall from boot media.
Use this path when FreeBSD 15.x is already installed and you manage the host.
- [Requirements](./requirements/) — host, runtime, and toolchain requirements.
- [Install orchestrator](./install/) — run `just install` and resume failed steps.
- [Install](./install/) — provision the host service with the `clawdie` binary (`clawdie discover` / `plan` / `apply --yes`).
- [Fresh install checklist](./fresh-install-checklist/) — verify the completed install.

326
docs/guide/install/install.md
View file

@ -1,301 +1,91 @@
---
title: Install Orchestrator
description: Single-command install flow for Clawdie.
title: Install
description: Provision the Clawdie host service with the clawdie binary.
---
**Command:** `just install`
**Command:** `clawdie apply --yes`
## Quick start
Colibri-based Clawdie is installed by the **`clawdie`** binary (crate
`crates/clawdie`). It discovers the host's ZFS layout and provisions the
`clawdie` service: storage datasets, a service user, and the rc.d (FreeBSD) or
systemd (Linux) unit that runs `colibri-daemon`.
Clawdie tracks the FreeBSD 15.x line. The installer rejects FreeBSD 14.x and any unvalidated future major version.
This repo is a **Cargo workspace** — there is no `just`/npm install flow here.
Build the installer from the workspace:
```bash
just install
cargo build -p clawdie --release
```
## ISO path
Current ISO validation is converging on: live QML installer → first installed
boot → loopback-bound controlplane `/setup` with a one-time bootstrap token.
Provider keys and Telegram are configured there after install. The older
[`setup.txt` first-boot contract](./first-boot/) remains documented for legacy
non-interactive work and will be rewritten after live ISO validation.
After the initial boot, the same `install` flow described here runs in the
background.
Resume from a specific step after a failure:
## Commands
```bash
just install-from-db
clawdie discover # read-only: OS, ZFS pools, datasets, spare disks
clawdie plan [--pool NAME] # show the deploy plan (dry-run, no writes)
clawdie apply [--pool NAME] # dry-run unless --yes is given
clawdie apply --yes # provision: storage layout + install the service
```
Skip service jails (db, git, cms) when running `DB_RUNTIME=host` or installing a second agent on an existing host:
`apply` is a **dry-run by default** and prints the full step plan; it writes to
disk only with `--yes`. When the host has exactly one ZFS pool it is selected
automatically; with several, pass `--pool NAME`.
```bash
just install-from hosts
```
## Storage strategy
Print the step plan without running anything:
| Host | Behavior |
| ------------------ | ---------------------------------------------------------------------------------------------------------- |
| FreeBSD | ZFS is **required**; datasets are created under the chosen pool. |
| Linux + ZFS + pool | Same — datasets under the pool. |
| Linux, no ZFS/pool | Falls back to plain directories, and reports the ZFS benefits plus any spare disks that could host a pool. |
```bash
just install -- --dry-run
```
Skip ZFS checkpoints (e.g. no ZFS pool):
```bash
just install -- --no-snapshots
```
---
## Step flow
```
just install
│
▼
┌─────────────────────────────────────────────────────────┐
│ Detect ZFS dataset (zroot/bastille) │
│ Parse --from / --no-snapshots / --dry-run │
└─────────────────────────────────────────────────────────┘
│
▼
[ 1] onboarding first-boot setup seed or TUI fallback REQUIRED
│
[ 2] environment host pkg baseline, bridge, locale REQUIRED
│
[ 3] agent-config validate/write agent provider optional ─── warn on missing provider auth
│ └── pi missing → warn, continue
[ 4] pf write PF include (NAT egress) REQUIRED
│ └── 📸 snapshot: post-pf
[ 5] jails create worker jail (--create) REQUIRED
│ └── 📸 snapshot: post-jails
[ 6] git Git Service (git jail) DEFAULT ─── DB_RUNTIME=host → skip or use install-from hosts
│ └── 📸 snapshot: post-git
[ 7] forgejo Forgejo for Git Service DEFAULT ─── FEATURE_GITEA=NO → skip
│
[ 8] db Data Service (PostgreSQL) DEFAULT ─── DB_RUNTIME=host → skip; DB on host instead
│ └── 📸 snapshot: post-db
[ 9] skills-memory built-in knowledge import DEFAULT ─── artifact.sql ships in tarball
│
[10] skills-init Skills engine init (.nanoclaw) DEFAULT
│
[11] cms Web Service (cms jail: Astro+nginx) DEFAULT ─── use install-from hosts to skip
│ └── 📸 snapshot: post-cms
│
[12] ollama Local AI Models (Ollama jail .5) optional ─── FEATURE_OLLAMA≠YES → skip
│
[13] llama-cpp Local AI Models (llama-cpp jail .5) optional ─── FEATURE_LLAMA_CPP≠YES → skip
│
[14] hosts /etc/hosts + jail hostnames REQUIRED
│
[15] mounts validate jail mounts REQUIRED
│
[16] telegram-auth verify bot token optional ─── TELEGRAM_BOT_TOKEN unset → skip
│
[17] service build + install rc.d service REQUIRED
│
[18] hostd privileged host daemon (hostd) REQUIRED
│
[19] identity-restore Supabase restore optional ─── SUPABASE_URL unset → skip
│
[20] verify integrity check optional
│ └── 📸 snapshot: install-complete
│
▼
┌─────────────────────────────────────────────────────────┐
│ Summary: N ok N warnings N skipped N failed │
│ Snapshots taken: zroot/bastille@post-pf-… … │
│ LLM providers: anthropic ✓ openai ✗ ollama ✗ … │
└─────────────────────────────────────────────────────────┘
```
The onboarding step prefers the first-boot setup file (`setup.txt`; see
[First boot](./first-boot/)). The interactive TUI wizard is the fallback when
the first-boot setup is absent or invalid; it sources locales from FreeBSD
itself, so any installed locale can be selected (`en_US.UTF-8`,
`zh_CN.UTF-8`, etc.) and is applied consistently.
`setup.txt` is the operator-intent contract, and `system.env` is the matching
hardware-intent contract. Inspect can populate both before the installer runs.
Set `DB_RUNTIME=host` in `.env` to provision PostgreSQL directly on the host instead of a db jail; `DB_HOST` defaults to `${AGENT_SUBNET_BASE}.1` so jails can reach it. Use `DB_COMPRESSION=lz4` (default) or `DB_COMPRESSION=zstd` for ZFS compression on host datasets.
The root install owns shared platform services. It is not modeled as tenant zero.
`ASSISTANT_NAME` is display-only. Later additive tenants consume shared services such as:
- Git Service
- Web Service
- Local AI Models
---
## ZFS snapshots
Snapshots are taken after each milestone step if a Bastille ZFS dataset is
detected. Two paths:
| Context | Method |
| ------------------------------ | ------------------------------------------------------------- |
| Running as root | `zfs snapshot zroot/bastille@<tag>` directly |
| Non-root, `sudo` available | `sudo zfs snapshot zroot/bastille@<tag>` |
| Non-root, hostd socket present | `nc -U /var/run/<agent>-hostd.sock` (hostd `zfs-snapshot` op) |
| Neither | skip silently |
Snapshot tags are suffixed with a Unix timestamp to prevent collisions on re-runs.
---
## LLM providers
The orchestrator never exits on missing LLM provider auth. OpenAI/OpenRouter/Anthropic
subscription auth is the recommended path for performance/price, but it is not a
hard installer requirement and all supported providers are peer options. At the
end of the run Clawdie prints a table of known providers:
```
△ LLM providers
anthropic ANTHROPIC_API_KEY ✓ configured
openai OPENAI_API_KEY ✗ not set
openrouter OPENROUTER_API_KEY ✗ not set
deepseek DEEPSEEK_API_KEY ✗ not set
zai ZAI_API_KEY ✗ not set
groq GROQ_API_KEY ✗ not set
azure AZURE_OPENAI_API_KEY ✗ not set
ollama OLLAMA_HOST ✗ not set
```
If no provider auth is found, it prints instructions:
```
No LLM provider auth found. Configure one after install and restart:
Set DEEPSEEK_API_KEY=... in provider.env for the default agent,
or add provider keys such as ANTHROPIC_API_KEY=sk-ant-...
service clawdie restart
```
The entire infrastructure (PF, jails, PostgreSQL, nginx, ZFS) has zero
LLM dependency. Provider auth is only consumed when the jail-runner spawns a
live response. Install and service start succeed without it.
After install, use the controlplane setup page to finish provider and optional
Telegram configuration:
ZFS layout under the pool:
```text
https://<agent-domain>/setup
<pool>/clawdie (container, canmount=off)
<pool>/clawdie/db -> /var/db/clawdie
<pool>/clawdie/log -> /var/log/clawdie
```
Before setup is complete, do not expose this URL directly to the public internet.
Use local console/loopback access, or a tailnet/LAN path protected by TLS and
network allowlisting. The bootstrap token is a first-setup key, not a substitute
for bind/PF/TLS policy.
## What `apply --yes` provisions
If the printed bootstrap token is lost before setup completes, rotate a fresh
one on the host:
1. **Storage** — the datasets above (or plain `/var/db/clawdie` +
`/var/log/clawdie` on the plain-dirs fallback).
2. **Service user** — `clawdie` (nologin), owning the state directories
(`clawdie:clawdie`).
3. **Service** — the rc.d script (FreeBSD, via `daemon -u clawdie`) or systemd
unit (Linux, `User=clawdie`), installed and enabled to run
`/usr/local/bin/colibri-daemon`.
```sh
npm run setup-token -- rotate
```
## Creating a pool (destructive)
### Agent runtime setup
The agent runtime uses `DEEPSEEK_API_KEY` from `provider.env`.
Colibri autospawns zot in RPC mode — no separate login or auth store
needed. zot reads its identity from `$ZOT_HOME/AGENTS.md` (installed
automatically from the seed partition).
Quick verification:
On a host with a spare disk and no usable pool:
```bash
zot --no-session --print "Reply with exactly: zot-ok"
clawdie apply --pool NAME --create-pool /dev/DEV --yes
```
Expected output:
`--create-pool` runs `zpool create` on `DEV` and **destroys all data on it**, so
it is refused unless the disk is detected as empty (no partitions/filesystem/
mount, not the root disk). Override that guard with `--force` only if you are
certain.
```text
zot-ok
```
## Safety
### Control plane API auth
Disk-touching steps (`zfs`/`zpool create`, service install) run as root on the
target host. `discover`, `plan`, and a bare `apply` never write — preview with
`plan` first.
Agent subprocesses (pi, aider) authenticate back to the control plane
API using a shared secret. Generate one after install:
## Scope: what the installer does _not_ do (yet)
```bash
echo "CONTROLPLANE_SHARED_SECRET=$(openssl rand -base64 32)" >> .env
sudo service ${AGENT_NAME} restart
```
`clawdie` provisions the **host service** only. The wider service mesh — PF
egress, worker jails, PostgreSQL, Forgejo/Git, and the CMS/web service — was
provided by the legacy clawdie-ai `just` + Ansible orchestrator and is **not yet
part of the colibri installer**. Older pages that reference `just install`,
`just setup-db`, `just doctor`, and similar describe that legacy flow, not this
repo; they are being reconciled to the `clawdie` model.
Without it, the startup log warns and agent-to-API calls are rejected.
## FreeBSD validation
### Service wrapper scripts
The rc.d service (`/usr/local/etc/rc.d/${AGENT_NAME}`) runs
`run-${AGENT_NAME}.sh` which is **generated** by `just setup-service` at
install time. It is not tracked in git (`run-*.sh` is in `.gitignore`).
If missing, re-run `just setup-service` to regenerate it.
Local LLM runtime is optional and configured via:
```
LOCAL_LLM_PROVIDER=none|ollama|llama_cpp
FEATURE_OLLAMA=YES|NO
FEATURE_LLAMA_CPP=YES|NO
FEATURE_OLLAMA_HPP=YES|NO # optional C++ bindings for Ollama clients
```
---
## Required vs optional steps
Steps are **required** (failure stops install), **default** (runs out of the box
but can be disabled), or **optional** (skipped unless explicitly enabled).
| Step | Status | Skip condition |
| ---------------- | -------- | ------------------------------------------------------------------ |
| onboarding | required | — |
| environment | required | — |
| agent-config | optional | warn on missing provider auth or missing agent binary |
| pf | required | — |
| jails | required | — |
| git | default | `DB_RUNTIME=host` or `install-from hosts` |
| forgejo | default | `FEATURE_GITEA` = NO |
| db | default | `DB_RUNTIME=host` skips jail, provisions on host instead |
| skills-memory | default | `bootstrap/skills-memory/artifact.sql` not present |
| skills-init | default | — |
| cms | default | `install-from hosts` skips; cms is a shared service, not per-agent |
| ollama | optional | `FEATURE_OLLAMA` ≠ YES |
| llama-cpp | optional | `FEATURE_LLAMA_CPP` ≠ YES |
| hosts | required | — |
| mounts | required | — |
| telegram-auth | optional | `TELEGRAM_BOT_TOKEN` not set |
| service | required | — |
| hostd | required | — |
| identity-restore | optional | `SUPABASE_URL` not set |
| verify | optional | warn on most check failures; fail on broken runtime integrity |
A required step failure stops the install immediately and prints the resume
command. Default steps ship enabled (`FEATURE_GITEA=YES`, artifact.sql bundled)
— they run unless the operator explicitly opts out.
---
## Individual steps
The orchestrator calls each step through `setup/index.ts` as a subprocess
(`spawnSync` with `tsx`). This isolates `process.exit(1)` calls in individual
step modules from killing the orchestrator.
Individual steps can still be run directly:
```bash
just setup-db
just setup -- --step verify
```
`verify` checks more than the shared docs build. If tenant sites are
declared, it also checks their served output and publish-manifest consistency in
the CMS webroot. A site that is merely declared but not yet manually published
does not fail the step by itself; an inconsistent live publish state does.
The step registry is implemented in `setup/index.ts`.
The disk-touching and service-install paths are validated on real FreeBSD/Linux
hosts; discovery, plan, and disk-candidacy logic are unit-tested. See
`docs/CLAWDIE-INSTALLER-HANDOFF.md` for the FreeBSD validation runbook.

14
docs/guide/install/requirements.md
View file

@ -5,7 +5,7 @@ description: Host, runtime, and toolchain requirements for Clawdie.
What you need on the host before installing Clawdie. The ISO installer
provisions most of this automatically; existing-host installs should verify
these requirements before running `just install`.
these requirements before running `clawdie apply`.
## Host
@ -18,13 +18,15 @@ these requirements before running `just install`.
## Toolchain
- **Rust** via rustup, installed under `/opt/clawdie/rustup`.
Required for native dependencies (SWC, tree-sitter). The ISO
installs this; existing-host installs run it from
[`Install orchestrator`](./install/).
- **Node.js + tsx** for the install scripts and runtime.
- **Rust** (Cargo) — the colibri control plane and the `clawdie`
installer are a Cargo workspace. Build the installer with
`cargo build -p clawdie --release`. See [Install](./install/).
- **Bastille** for jail management.
> The legacy clawdie-ai `just` + Node/`tsx` orchestrator is not used in this
> repo; the sections below still describe the clawdie-ai product runtime and are
> being reconciled to the colibri model.
## Runtime philosophy
Clawdie is a single Node.js process that orchestrates jailed