diff --git a/docs/POUDRIERE-BUILD-SERVER.md b/docs/POUDRIERE-BUILD-SERVER.md index 093146d9..7897c0dd 100644 --- a/docs/POUDRIERE-BUILD-SERVER.md +++ b/docs/POUDRIERE-BUILD-SERVER.md @@ -1,9 +1,10 @@ -# Poudriere Build Server Plan — v1.0.0 +# Poudriere + bhyve Build Server Plan — v1.0.0 > Build colibri/clawdie as proper FreeBSD packages instead of raw `cargo build` -> binaries copied into the ISO. Target: HPE ML350p Gen8, 32GB RAM. +> binaries copied into the ISO. Plus bhyve virtualization for build test VMs. +> Target: HPE ML350p Gen8, 32GB RAM, Xeon E5-2400. -**Date:** 4 Jun 2026 +**Date:** 4 Jun 2026 **Target server:** HPE ProLiant ML350p Gen8 (iLO at 10.0.0.2) --- @@ -36,27 +37,51 @@ Benefits: - No Rust toolchain needed on the ISO build host - Proper `pkg info colibri` metadata +## Memory split + +32GB total — 16GB host, 16GB bhyve: + +``` +Host (16GB): + 6GB — ZFS ARC (primary cache) + 4GB — Poudriere tmpfs (builds in RAM) + 6GB — headroom (ARC bursts, nginx, clawdie, SSH) + +bhyve (16GB): + 4GB — FreeBSD build jail (Poudriere test guest) + 4GB — Linux test guest (cross-compile validation) + 4GB — FreeBSD ISO test VM (boot clawdie-iso after build) + 4GB — spare / future CI runner +``` + +CPU: Xeon E5-2400 v1/v2 — all SKUs have VT-x + EPT + VT-d. +bhyve uses hardware-accelerated virt (no emulation penalty). +With 8-10 cores, 2 vCPUs per guest is comfortable. + ## Architecture ``` -┌─────────────────────────────────────────┐ -│ HPE ML350p Gen8 (FreeBSD 15) │ -│ │ -│ ZFS pool: zroot │ -│ zroot/ROOT/default ← base system │ -│ zroot/poudriere ← build jails │ -│ zroot/poudriere/data ← packages │ -│ zroot/poudriere/ports ← ports tree │ -│ │ -│ Services: │ -│ poudriere (bulk builder) │ -│ nginx (pkg repo) │ -│ clawdie agent (self-hosting) │ -│ tailscale (mesh) │ -└─────────────────────────────────────────┘ +┌──────────────────────────────────────────────────┐ +│ HPE ML350p Gen8 (FreeBSD 15) │ +│ │ +│ ZFS pool: zroot │ +│ zroot/ROOT/default ← base system │ +│ zroot/poudriere ← build jails │ +│ zroot/poudriere/data ← packages │ +│ zroot/poudriere/ports ← ports tree │ +│ zroot/bhyve ← VM disk images │ +│ zroot/bhyve/iso-test ← ISO boot test VM │ +│ │ +│ Services: │ +│ poudriere (bulk builder) │ +│ bhyve (VM host: FreeBSD + Linux + ISO test) │ +│ nginx (pkg repo) │ +│ clawdie agent (self-hosting) │ +│ tailscale (mesh) │ +└──────────────────────────────────────────────────┘ ``` -## Phase 1 — server provision +## Phase 1 - server provision ### 1.1 Boot FreeBSD from ISO @@ -89,7 +114,7 @@ zfs create -o mountpoint=/usr/local/poudriere zroot/poudriere/root # Configure network (DHCP, tailscale) ``` -## Phase 2 — Poudriere setup +## Phase 2 - Poudriere setup ### 2.1 Install Poudriere @@ -141,7 +166,7 @@ MASTER_SITES= https://code.smilepowered.org/clawdie/colibri/archive/ DISTNAME= ${PORTNAME}-${PORTVERSION} MAINTAINER= hello@clawdie.si -COMMENT= Colibri control plane — agent supervision, skills, tasks +COMMENT= Colibri control plane - agent supervision, skills, tasks WWW= https://code.smilepowered.org/clawdie/colibri LICENSE= MIT @@ -164,7 +189,7 @@ make cargo-crates > Makefile.crates # generate crate list poudriere bulk -j 150-amd64 -p default sysutils/colibri ``` -## Phase 3 — pkg repository +## Phase 3 - pkg repository ### 3.1 Nginx pkg repo @@ -188,12 +213,12 @@ clawdie: { } ``` -## Phase 4 — ISO integration +## Phase 4 - ISO integration Instead of `stage-colibri-iso.sh` copying raw binaries: ```sh -# build.sh — install colibri packages during ISO build +# build.sh - install colibri packages during ISO build pkg -r ${MOUNT_POINT} install colibri ``` @@ -218,6 +243,62 @@ service clawdie start # clawdie skills include "rebuild-colibri" and "poudriere-status" ``` +## Phase 6 — bhyve test VMs + +### 6.1 Enable bhyve kernel module + +```sh +kldload vmm +sysrc kld_list+="vmm" +``` + +### 6.2 Install bhyve packages + +```sh +pkg install bhyve-firmware edk2-bhyve vm-bhyve +``` + +### 6.3 Create ZFS datasets for VMs + +```sh +zfs create zroot/bhyve +zfs create zroot/bhyve/iso-test +zfs create zroot/bhyve/linux-test +zfs create zroot/bhyve/freebsd-test +``` + +### 6.4 Test VMs + +**FreeBSD ISO test VM** (boots clawdie-iso after each build): +```sh +vm create -t freebsd iso-test +vm install iso-test clawdie-iso.iso +vm start iso-test +# → verifies ISO boots, colibri starts, clawdie health passes +``` + +**Linux cross-compile test VM** (validates non-FreeBSD targets): +```sh +vm create -t linux linux-test +# → test colibri builds on Linux target +``` + +**FreeBSD Poudriere test jail VM** (full pkg build validation): +```sh +vm create -t freebsd freebsd-test +# → clone poudriere setup, run bulk build as validation +``` + +### 6.5 Packages for ISO + +Add to `pkg-list-disk-install-extras.txt` (installed on deployed server, not live USB): + +``` +bhyve-firmware +edk2-bhyve +vm-bhyve +``` + ## Timeline | Step | Effort | Depends on | @@ -228,8 +309,9 @@ service clawdie start | 4. First pkg build | ~30m (compile) | port ready | | 5. pkg repo + nginx | ~15m | packages built | | 6. ISO integration | ~15m | repo hosted | +| 7. bhyve + test VMs | ~30m | base system + ZFS | -**Total: ~3.5h** once iLO password is available. +**Total: ~4h** once iLO password is available. ## Notes @@ -241,3 +323,6 @@ service clawdie start - 32GB RAM + tmpfs means builds stay in memory (fast, no SSD wear). - Tailscale mesh means the operator USB can install packages from this server even after deploying to a different machine. +- Xeon E5-2400 has VT-x + EPT — full hardware virt acceleration. +- bhyve guests can run concurrently with Poudriere builds (16GB reserved + for VMs, 16GB for host). diff --git a/packages/pkg-list-disk-install-extras.txt b/packages/pkg-list-disk-install-extras.txt index 33ffc794..e5ed5acf 100644 --- a/packages/pkg-list-disk-install-extras.txt +++ b/packages/pkg-list-disk-install-extras.txt @@ -38,4 +38,10 @@ xls2txt p5-docx2txt odt2txt mpv +# bhyve virtualization — FreeBSD-native hypervisor for test VMs +# (ISO boot verification, cross-compile test guests, CI runners) +bhyve-firmware +edk2-bhyve +vm-bhyve + simplescreenrecorder