clawdie-iso/docs/LIVE-COLIBRI-REBUILD.md

# Live USB Colibri rebuild lane

This runbook documents how a running Clawdie live USB can rebuild and redeploy
Colibri without waiting for a full ISO rebuild.

Use this for **field repair and validation** when the USB boots but a Colibri
binary, rc.d script, or small runtime behavior needs testing. A successful live
rebuild does **not** replace the release ISO process; it proves the fix before
the next image is rebuilt.

## Why this exists

New live USB builds stage `/home/clawdie/ai/colibri` as a shallow git checkout
with a `.git` directory and a `.clawdie-source.json` provenance file. You can
rebuild directly from that checkout, or clone a separate working checkout such
as `/home/clawdie/ai/colibri-build` if you want to keep the shipped source
untouched while testing a branch.

## Required live packages

Live operator ISOs include these packages to make this path available out of the
box:

```text
rust
pkgconf
git
go
```

`git` is already in the live operator package list. `rust` provides `cargo` on
FreeBSD. `pkgconf` is needed by common Rust native dependency build scripts.
`go` builds the zot agent (see "Rebuild zot" below).

Optional if future dependencies need them:

```text
gmake
cmake
```

## One-time readiness check

Run on the live USB:

```sh
rustc --version
cargo --version
git --version
pkgconf --version
```

If `rustc` or `cargo` is missing, the current USB cannot rebuild Colibri from
source without installing packages first or using a newer ISO. Likewise check
`go version` before attempting a zot rebuild.

## Clone or update Colibri

Use HTTPS unless the live USB has an SSH key configured for Forgejo.

```sh
mkdir -p /home/clawdie/ai
cd /home/clawdie/ai

if [ ! -d colibri-build/.git ]; then
  git clone https://code.smilepowered.org/clawdie/colibri.git colibri-build
fi

cd /home/clawdie/ai/colibri-build
git fetch --prune origin
git switch main
git pull --ff-only
```

To test a specific commit or branch:

```sh
git checkout c967698
# or:
# git switch fix/some-branch
```

## Build Colibri release binaries

Minimal live USB build set:

```sh
cd /home/clawdie/ai/colibri-build
cargo build --release \
  -p colibri-daemon \
  -p colibri-client \
  -p colibri-mcp \
  -p colibri-glasspane-tui
```

Expected outputs:

```text
target/release/colibri-daemon
target/release/colibri
target/release/colibri-mcp
target/release/colibri-tui
# target/release/colibri-test-agent is installed only for explicit
# COLIBRI_STAGE_TEST_AGENT=YES validation builds.
```

## Install rebuilt Colibri into the running USB

The live USB does not use `sudo`. Privileged actions use `mdo`.

```sh
cd /home/clawdie/ai/colibri-build

mdo -u root service colibri_daemon stop || true
mdo -u root pkill -f colibri-daemon || true
mdo -u root rm -f /var/run/colibri/colibri.sock
mdo -u root rm -f /var/run/colibri/colibri-daemon.pid
mdo -u root rm -f /var/run/colibri/colibri-daemon-supervisor.pid

mdo -u root install -m 0555 target/release/colibri-daemon /usr/local/bin/colibri-daemon
mdo -u root install -m 0555 target/release/colibri /usr/local/bin/colibri
mdo -u root install -m 0555 target/release/colibri-mcp /usr/local/bin/colibri-mcp
if [ "${COLIBRI_STAGE_TEST_AGENT:-NO}" = "YES" ]; then
  mdo -u root install -m 0555 target/release/colibri-test-agent /usr/local/bin/colibri-test-agent
fi
mdo -u root install -m 0555 target/release/colibri-tui /usr/local/bin/colibri-tui
mdo -u root install -m 0555 packaging/freebsd/colibri_daemon.in /usr/local/etc/rc.d/colibri_daemon

mdo -u root chown -R colibri:colibri /var/run/colibri /var/db/colibri /var/log/colibri
mdo -u root service colibri_daemon start
```

`service colibri_daemon start` should return to the shell after a few seconds. If
it does not, stop the command, collect the service file and daemon log, and hand
those back through git before continuing.

## Validate runtime

```sh
service colibri_daemon status
colibri status
colibri list-skills | head
colibri-mcp tools
```

Useful deeper checks:

```sh
service colibri_daemon health
tail -80 /var/log/colibri/daemon.log
ls -l /var/run/colibri
sockstat -u | grep colibri || true
```

## Record what was installed

After a successful live rebuild, write a small local manifest:

```sh
cd /home/clawdie/ai/colibri-build
commit=$(git rev-parse HEAD)
branch=$(git symbolic-ref --short -q HEAD || echo detached)
now=$(date -u +%Y-%m-%dT%H:%M:%SZ)

cat > /tmp/colibri-live-rebuild.json <<EOF
{
  "repo": "https://code.smilepowered.org/clawdie/colibri.git",
  "branch": "${branch}",
  "commit": "${commit}",
  "installed_at": "${now}",
  "note": "Built and installed on running live USB"
}
EOF

mdo -u root install -m 0644 /tmp/colibri-live-rebuild.json /var/db/colibri/live-rebuild.json
mdo -u root sha256 /usr/local/bin/colibri-daemon /usr/local/bin/colibri /usr/local/bin/colibri-mcp /usr/local/bin/colibri-tui \
  > /tmp/colibri-live-rebuild.sha256
mdo -u root install -m 0644 /tmp/colibri-live-rebuild.sha256 /var/db/colibri/live-rebuild.sha256
```

Report both files with validation notes when handing findings back through git.

## Rebuild zot (agent)

zot is the Go agent staged as a prebuilt binary at `/usr/local/bin/zot`. New
live USB builds seed `/home/clawdie/ai/zot` as a shallow git checkout with
`.clawdie-source.json` provenance, so you can rebuild there or clone a separate
`zot-build` checkout for branch testing.

```sh
mkdir -p /home/clawdie/ai
cd /home/clawdie/ai
if [ ! -d zot-build/.git ]; then
  git clone https://code.smilepowered.org/clawdie/zot.git zot-build
fi
cd zot-build
git fetch --prune origin
git checkout v0.2.42          # the zot tag this image ships (see build-manifest.json)
ZOT_BUILD_VERSION="${ZOT_VERSION:-v0.2.42}"
VERSION="${ZOT_BUILD_VERSION#v}" make build
./bin/zot --version
```

Install into the running USB (privileged steps via `mdo`):

```sh
mdo -u root install -m 0555 bin/zot /usr/local/bin/zot
zot --version
```

zot has no rc.d service of its own — Colibri spawns it — so no service restart
is needed. Validate with `zot --version` and a Colibri spawn check.

## Fast rc.d-only repair

If only the service script is broken and no binary rebuild is needed, fetch the
current Colibri rc.d source directly:

```sh
mdo -u root fetch -o /usr/local/etc/rc.d/colibri_daemon \
  https://code.smilepowered.org/clawdie/colibri/raw/branch/main/packaging/freebsd/colibri_daemon.in
mdo -u root chmod 555 /usr/local/etc/rc.d/colibri_daemon
mdo -u root service colibri_daemon restart
```

Sanity checks:

```sh
sh -n /usr/local/etc/rc.d/colibri_daemon
service colibri_daemon status
colibri status
```

Expected:

- the rc.d script has valid shell syntax.
- `service colibri_daemon start` returns to the shell.
- `colibri status` can connect to `/var/run/colibri/colibri.sock`.

## One-command helper

The whole Colibri runbook above is wrapped in `colibri-live-rebuild`, staged at
`/usr/local/bin/colibri-live-rebuild` whenever the image carries the Colibri
service. It clones/updates a build checkout, optionally checks out a ref, builds
the release binaries, stops the daemon, installs binaries + rc.d, clears stale
socket/pid files, restarts, validates `colibri status`, and writes
`/var/db/colibri/live-rebuild.json`.

```sh
colibri-live-rebuild                 # rebuild from the build checkout's HEAD
colibri-live-rebuild --ref main      # check out a branch/tag/commit first
colibri-live-rebuild --build-dir DIR # use an alternate build checkout
```

The manual steps above remain the reference for partial repairs (e.g. the zot
rebuild or the rc.d-only fix). This keeps the live USB usable as a self-hosted
Colibri validation target while preserving ISO rebuilds for release artifacts.