clawdie-iso/docs/POUDRIERE-BUILD-SERVER.md

Poudriere + bhyve Build Server Plan — v2.0.0

Build colibri/clawdie as proper FreeBSD packages instead of raw cargo build binaries copied into the ISO. Plus bhyve virtualization for build test VMs. Target: the mother-build host (FreeBSD VPS) serving pkg.clawdie.si — the package half of the trusted supply chain (layered-soul HIVE-ONBOARDING.md §10).

Date: 4 Jun 2026 (v1.0.0) · retargeted 20 Jun 2026 (v2.0.0) Target server: mother-build — a FreeBSD VPS with a ZFS pool (OVHcloud candidate; see layered-soul HOST-MATRIX.md). The original ML350p Gen8 target is retired (boot-looping); host provisioning is now provider-specific, not iLO/IPMI. Hardware- specific sizing below (32 GB split, Xeon VT-x) is kept only as reference numbers from the ML350p — scale them to the chosen mother-build plan.

Scripts: the reproducible Phase 2–3 steps are implemented in ../scripts/poudriere/ — poudriere-setup.sh, poudriere-build.sh, clawdie-repo.conf.in. This doc is the rationale, host- provisioning context, and the manual reference those scripts encode.

---

Why

Current build flow:

colibri host (dev machine)
  cargo build --release
  → stage-colibri-iso.sh copies binaries to ISO
  → no version tracking, no pkg metadata, manual

Proposed flow:

mother-build server (Poudriere)
  → FreeBSD port: sysutils/colibri
  → poudriere builds .pkg files in clean jail
  → signed pkg repo hosted at pkg.clawdie.si
  → ISO build does: pkg install colibri

Benefits:

• Versioned packages with dependencies
• Clean-room builds (no host contamination)
• pkg upgrade colibri on deployed machines
• No Rust toolchain needed on the ISO build host
• Proper pkg info colibri metadata
• Less pressure to treat the OSA ISO builder as disposable package scratch space. OSA still needs a meaningful local operator/runtime toolkit (Codex, Hermes/image helpers, media/rendering deps, VNC/screenshot helpers). In practice those pull a shared GTK/X11/Wayland/media dependency closure. When disk gets tight, deleting repo-local build caches is safer than trying to strip that host closure. A real mother-build shifts bulky package-build churn off OSA instead of forcing the builder host to choose between free space and capability.

Memory split (reference sizing — ML350p 32 GB)

These numbers are the original ML350p reference; on a smaller cloud mother-build plan, poudriere alone needs little (drop USE_TMPFS if RAM is tight) and the bhyve guests are optional. 32GB total — 16GB host, 16GB bhyve:

Host (16GB):
  6GB  — ZFS ARC (primary cache)
  4GB  — Poudriere tmpfs (builds in RAM)
  6GB  — headroom (ARC bursts, nginx, clawdie, SSH)

bhyve (16GB):
  4GB  — FreeBSD build jail (Poudriere test guest)
  4GB  — Linux test guest (cross-compile validation)
  4GB  — FreeBSD ISO test VM (boot clawdie-iso after build)
  4GB  — spare / future CI runner

CPU: Xeon E5-2400 v1/v2 — all SKUs have VT-x + EPT + VT-d. bhyve uses hardware-accelerated virt (no emulation penalty). With 8-10 cores, 2 vCPUs per guest is comfortable.

Architecture

┌──────────────────────────────────────────────────┐
│  mother-build host (FreeBSD 15)                  │
│                                                   │
│  ZFS pool: zroot                                 │
│    zroot/ROOT/default     ← base system           │
│    zroot/poudriere        ← build jails           │
│    zroot/poudriere/data   ← packages              │
│    zroot/poudriere/ports  ← ports tree            │
│    zroot/bhyve            ← VM disk images        │
│    zroot/bhyve/iso-test   ← ISO boot test VM      │
│                                                   │
│  Services:                                        │
│    poudriere (bulk builder)                       │
│    bhyve (VM host: FreeBSD + Linux + ISO test)    │
│    nginx (pkg repo)                               │
│    clawdie agent (self-hosting)                   │
│    tailscale (mesh)                               │
└──────────────────────────────────────────────────┘

Phase 1 - server provision (provider-specific)

1.1 Get a booted FreeBSD host

Host provisioning is provider-specific and out of scope for the scripts. On OVHcloud, install FreeBSD via the rescue system or a FreeBSD template/image; the old ML350p iLO/IPMI virtual-media path is retired. The poudriere scripts assume you already have a booted FreeBSD host with a ZFS pool (default zroot).

1.2 Disk survey + ZFS

camcontrol devlist
geom disk list

# Single disk or mirror
zpool create -o ashift=12 zroot /dev/ada0
zfs create -o mountpoint=/ zroot/ROOT/default
zfs create zroot/poudriere
zfs create -o mountpoint=/usr/local/poudriere zroot/poudriere/root

1.3 Install base system

# Install FreeBSD 15 base + kernel to zroot
# Set up /boot, /etc, and the future deployed-system service hook
# Configure network (DHCP, tailscale)

Phase 2 - Poudriere setup

Scripted (DONE). poudriere-setup.sh performs 2.1–2.3 idempotently (install, poudriere.conf incl. PKG_REPO_SIGNING_KEY, signing-key generation, jail, ports tree) after validating root / version / pool. poudriere-build.sh performs 2.5 (bulk + automatic repo signing). The steps below are the reference they encode; only 2.4 (creating the sysutils/colibri port) is still manual.

2.1 Install Poudriere

pkg install poudriere

2.2 Configure

# /usr/local/etc/poudriere.conf
ZPOOL=zroot
ZROOTFS=/poudriere
FREEBSD_HOST=download.freebsd.org
RESOLV_CONF=/etc/resolv.conf
BASEFS=/usr/local/poudriere
USE_TMPFS=yes         # build in RAM (we have 32GB)
MAX_MEMORY=4          # limit per-jail RAM
MAX_FILES=2048
DISTFILES_CACHE=/usr/local/poudriere/distfiles
PARALLEL_JOBS=8

2.3 Create jail + ports tree

# poudriere-setup.sh does this with these names by default:
poudriere jail -c -j clawdie-amd64 -v 15.0-RELEASE -a amd64
poudriere ports -c -p clawdie -m git -B main

2.4 Colibri port — canonical in the colibri repo

The sysutils/colibri port is owned by the colibri repo, kept with the code it builds at packaging/freebsd/port/sysutils/colibri/:

sysutils/colibri/
├── Makefile      USES=cargo; ships 6 binaries + rc.d services
├── pkg-descr
├── pkg-plist
└── files/        rc.d templates (colibri_daemon.in, colibri_bridge.in)

Copy that directory into the poudriere ports tree before building. This repo keeps no duplicate — build.sh's release gate fails if ports/sysutils/colibri/ reappears here. Key facts:

• LICENSE= MIT (per colibri/Cargo.toml; same as layered-soul).
• USES= cargo; source from the Forgejo archive (tagged v${DISTVERSION}).
• Ships runtime binaries: clawdie, colibri, colibri-daemon, colibri-mcp, colibri-tui — plus the colibri_daemon / colibri_bridge rc.d services. Keep colibri-test-agent available for poudriere validation builds, but treat it as a development/test helper rather than a default production ISO payload.
• CARGO_CRATES is committed and kept in sync with Cargo.lock by check-cargo-crates.sh (colibri CI). distinfo is generated on the build host with make makesum.

2.5 Build

cd /usr/local/poudriere/ports/clawdie/sysutils/colibri
make makesum        # generate distinfo
make cargo-crates > Makefile.crates  # generate crate list

# then build via the wrapper (validates + signs):
poudriere-build.sh --jail clawdie-amd64 --ports clawdie sysutils/colibri

Phase 3 - pkg repository

3.1 Nginx pkg repo

pkg install nginx
# /usr/local/etc/nginx/nginx.conf:
#   server { listen 443 ssl; server_name pkg.clawdie.si;
#            root /usr/local/poudriere/data/packages/clawdie-amd64-clawdie; autoindex on; }
# TLS via acme.sh (osa/mother-build already carry nginx + acme).

service nginx enable
service nginx start

3.2 Client config

On ISO builds and deployed jails, generate from ../scripts/poudriere/clawdie-repo.conf.in and ship the public key from Phase 2. The repo is signed, so clients verify:

sed "s#__PKG_URL__#https://pkg.clawdie.si/#; s#__PUBKEY_PATH__#/usr/share/keys/pkg/clawdie.pub#" \
    clawdie-repo.conf.in > /usr/local/etc/pkg/repos/clawdie.conf
install -m 0444 /usr/local/etc/ssl/clawdie-pkg.pub /usr/share/keys/pkg/clawdie.pub

For first-party-only (paid) tenants, lower the stock FreeBSD repo priority so resolution prefers signed Clawdie packages.

Phase 4 - ISO integration

Instead of stage-colibri-iso.sh copying raw binaries:

# build.sh - install colibri packages during ISO build
pkg -r ${MOUNT_POINT} install colibri

This gives us:

• colibri
• colibri-daemon
• colibri-mcp
• colibri-tui (optional in current raw-binary staging, desired for operator USB)
• colibri-test-agent only in validation/dev package builds, not default production images

All with proper pkg metadata, upgradeable, with dependencies tracked.

Phase 5 — self-hosted clawdie

Once the deployed-system service implementation lands and the server builds itself:

# The mother-build host runs its own deployed-system clawdie service
service clawdie enable
service clawdie start

# clawdie monitors build jails, runs periodic rebuilds
# clawdie skills include "rebuild-colibri" and "poudriere-status"

The current live USB does not stage service clawdie; it runs colibri_daemon directly.

Phase 6 — bhyve test VMs

Largely DONE (on the ISO-build host, not yet on mother-build). The ISO-boot test VM exists and is wired as a build gate: ../scripts/bhyve-test.sh boots the built image in bhyve with com1,stdio, and ../scripts/run-bhyve-test.sh scans the serial console for required boot markers / forbidden failures (panics, restart loops, the colibri permission-denied regression) and exits non-zero on a critical miss. ../scripts/bhyve-pf-allow.sh opens the guest network. What remains: replicate this on mother-build and add the Linux cross-compile / poudriere-validation guests (6.4) if wanted.

6.1 Enable bhyve kernel module

kldload vmm
sysrc kld_list+="vmm"

6.2 Install bhyve packages

pkg install bhyve-firmware edk2-bhyve vm-bhyve

6.3 Create ZFS datasets for VMs

zfs create zroot/bhyve
zfs create zroot/bhyve/iso-test
zfs create zroot/bhyve/linux-test
zfs create zroot/bhyve/freebsd-test

6.4 Test VMs

FreeBSD ISO test VM (boots clawdie-iso after each build):

vm create -t freebsd iso-test
vm install iso-test clawdie-iso.iso
vm start iso-test
# → verifies ISO boots, colibri starts, clawdie health passes

Linux cross-compile test VM (validates non-FreeBSD targets):

vm create -t linux linux-test
# → test colibri builds on Linux target

FreeBSD Poudriere test jail VM (full pkg build validation):

vm create -t freebsd freebsd-test
# → clone poudriere setup, run bulk build as validation

6.5 Packages for ISO

Add to pkg-list-disk-install-extras.txt (installed on deployed server, not live USB):

bhyve-firmware
edk2-bhyve
vm-bhyve

Timeline

Step	Effort	Status / depends on
1. Provision mother-build host	~1h	PENDING — OVH FreeBSD VPS + ZFS pool (cost-gated)
2. Poudriere setup	~30m	SCRIPTED — poudriere-setup.sh; needs the host
3. colibri port creation	~1h	PENDING — write sysutils/colibri (manual, §2.4)
4. First pkg build	~30m (compile)	SCRIPTED — poudriere-build.sh; needs the port
5. pkg repo + nginx (pkg.clawdie.si)	~15m	PENDING — nginx + acme + DNS
6. ISO integration	~15m	PENDING — pkg install in build.sh (Phase 4)
7. bhyve ISO-boot test gate	done	DONE — bhyve-test.sh + run-bhyve-test.sh

Critical path: provision mother-build → write the colibri port → run the two scripts → stand up nginx/DNS. The host purchase is gated on the HOST-MATRIX cost rows.

Why this matters operationally on OSA:

• repo-local ISO artifacts (tmp/packages, sparse work.img, cached memsticks) are the right things to delete under pressure
• host GUI/media/runtime packages are not just "old desktop leftovers" anymore; they are shared dependencies for Codex-adjacent tooling, Hermes/image helpers, and remote-display/debug paths
• pushing package-build churn to mother-build is the clean fix when OSA starts trading free space against operator capability

Notes

• Rust target x86_64-unknown-freebsd stays — that's the compiler triple. The package name is colibri (or clawdie-colibri), no "unknown" in package branding.
• Poudriere builds each package in a clean jail — no host Rust toolchain pollution. The jail installs lang/rust from ports automatically.
• USE_TMPFS=yes keeps builds in RAM (fast, no SSD wear) when the host has the memory; drop it on a small cloud plan.
• Tailscale mesh means the operator USB can install packages from this server even after deploying to a different machine.
• bhyve needs a host with VT-x + EPT (the ML350p's Xeon E5-2400 had it; confirm the chosen OVH plan exposes nested virt before relying on Phase 6 there).