From 4235d4e1070d5caefea2b73e1a420fe1a2b6260e Mon Sep 17 00:00:00 2001 From: Sam & Claude Date: Tue, 17 Mar 2026 12:02:02 +0100 Subject: [PATCH] docs: add Poudriere hybrid package system implementation plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Option 3 from brainstorming session — two-tier package system: - Base layer: stock FreeBSD packages from official latest repo - Clawdie layer: custom-built packages from Poudriere (priority 100) Covers: - Phase 1-7 implementation steps - Jail setup and configuration - Package list and make.conf - ISO integration and automation - Resource requirements and update flow Co-Authored-By: Claude Sonnet 4.6 --- POUDRIERE-IMPLEMENTATION.md | 414 ++++++++++++++++++++++++++++++++++++ 1 file changed, 414 insertions(+) create mode 100644 POUDRIERE-IMPLEMENTATION.md diff --git a/POUDRIERE-IMPLEMENTATION.md b/POUDRIERE-IMPLEMENTATION.md new file mode 100644 index 00000000..c1d71af7 --- /dev/null +++ b/POUDRIERE-IMPLEMENTATION.md @@ -0,0 +1,414 @@ +# Poudriere Implementation Plan — Hybrid Package System + +**Status:** Planning — not yet implemented +**Last updated:** 17.mar.2026 +**Target:** Clawdie-ISO v1.1+ + +--- + +## Overview + +**Goal:** Two-tier package system for Clawdie-ISO +- **Base layer:** Stock FreeBSD packages from official `latest` repo +- **Clawdie layer:** Custom-built packages from Poudriere (priority: 100) + +**Result:** USB image with offline packages that match Clawdie requirements, while still benefiting from FreeBSD security updates for base system. + +--- + +## Why Hybrid? + +| Approach | Pros | Cons | +|----------|------|------| +| Stock only | Simple, security updates from FreeBSD | Can't customize compile options, may lag on Node/Bastille versions | +| Full custom | Complete control | Long builds, must track all security updates manually | +| **Hybrid** | Best of both — security from base, control over Clawdie layer | Two repos to manage | + +--- + +## Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ controlplane host │ +│ │ +│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ +│ │ git jail │ │ db jail │ │ cms jail │ │ +│ │ (.1) │ │ (.3) │ │ (.4) │ │ +│ │ clawdie-iso │ │ postgres │ │ nginx │ │ +│ └─────────────┘ └─────────────┘ └─────────────┘ │ +│ │ +│ ┌─────────────────────────────────────────────────────┐ │ +│ │ poudriere jail (new) │ │ +│ │ (.5) │ │ +│ │ ┌─────────────────────────────────────────────┐ │ │ +│ │ │ ZFS: zroot/jails/poudriere/data │ │ │ +│ │ │ - /usr/local/poudriere/data/packages │ │ │ +│ │ │ - /usr/local/poudriere/data/logs │ │ │ +│ │ │ - /usr/local/poudriere/jails/15amd64 │ │ │ +│ │ └─────────────────────────────────────────────┘ │ │ +│ │ │ │ +│ │ Services: │ │ +│ │ - poudriere (build tool) │ │ +│ │ - nginx (pkg repo server on :8080) │ │ +│ └─────────────────────────────────────────────────────┘ │ +│ │ +│ Build output → rsync → git jail → injected into ISO │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## Phase 1: Poudriere Jail Setup + +### 1.1 Create the jail + +```bash +# On controlplane host +bastille create poudriere 15.0-RELEASE 10.0.0.5 lagg0 +bastille config poudriere set host.hostname poudriere.local +bastille start poudriere +``` + +### 1.2 Install Poudriere + +```bash +bastille pkg poudriere install poudriere dialog4ports +``` + +### 1.3 Configure ZFS dataset + +```bash +# Create dedicated ZFS dataset for build data +zfs create -o mountpoint=/usr/local/poudriere/data zroot/poudriere +bastille config poudriere set mount.fstab += "zroot/poudriere /usr/local/poudriere/data nullfs rw 0 0" +``` + +### 1.4 Poudriere configuration + +```bash +# /usr/local/etc/poudriere.conf (inside jail) +ZROOTFS=zroot/poudriere +FREEBSD_HOST=https://download.freebsd.org +RESOLV_CONF=/etc/resolv.conf +BASEFS=/usr/local/poudriere +USE_PORTLINT=no +USE_TMPFS=yes +NOLINUX=yes +``` + +--- + +## Phase 2: Build Environment + +### 2.1 Create build jail + +```bash +# Inside poudriere jail +poudriere jail -c -j 15amd64 -v 15.0-RELEASE -a amd64 +``` + +### 2.2 Create ports tree + +```bash +poudriere ports -c -p clawdie -m git+https +``` + +### 2.3 Define package list + +File: `/usr/local/etc/poudriere.d/clawdie-pkglist` + +``` +# Clawdie custom packages (built with specific options) +www/node24 +sysutils/bastille +x11/desktop-installer +databases/postgresql17-client +net/rsync +devel/git +sysutils/tmux +editors/vim +shells/bash +security/sudo + +# Custom Clawdie ports (to be created) +sysutils/clawdie-setup +``` + +### 2.4 Custom make.conf + +File: `/usr/local/etc/poudriere.d/15amd64-clawdie-make.conf` + +```makefile +# Node 24 with specific options +www_node24_SET=NPM OPTIMIZED_CFLAGS + +# Bastille with ZFS +sysutils_bastille_SET=ZFS + +# Minimal Xorg (no VNC, no VBox) +x11_servers_xorg_server_UNSET=VNC +x11_servers_xorg_server_SET=UDEV KMS + +# Desktop-installer without VirtualBox +x11_desktop-installer_UNSET=VIRTUALBOX +``` + +--- + +## Phase 3: Custom Ports (Optional) + +### 3.1 Create local ports overlay + +``` +/usr/local/poudriere/ports/clawdie/ +├── sysutils/ +│ └── clawdie-setup/ +│ ├── Makefile +│ ├── pkg-descr +│ ├── pkg-plist +│ └── files/ +│ └── setup.sh +``` + +### 3.2 Example Makefile for clawdie-setup + +```makefile +PORTNAME= clawdie-setup +PORTVERSION= 0.1.0 +CATEGORIES= sysutils + +MAINTAINER= hello@clawdie.si +COMMENT= Clawdie AI post-install setup wizard + +RUN_DEPENDS= bash:shells/bash \ + tmux:sysutils/tmux \ + node:www/node24 + +USE_GITHUB= yes +GH_ACCOUNT= Clawdie +GH_PROJECT= Clawdie-AI + +NO_BUILD= yes + +do-install: + ${MKDIR} ${STAGEDIR}${PREFIX}/share/clawdie + ${INSTALL_SCRIPT} ${WRKSRC}/setup.sh ${STAGEDIR}${PREFIX}/share/clawdie/ + +.include +``` + +--- + +## Phase 4: Build Process + +### 4.1 Initial build + +```bash +# Inside poudriere jail +poudriere bulk -j 15amd64 -p clawdie -f /usr/local/etc/poudriere.d/clawdie-pkglist +``` + +### 4.2 Output location + +``` +/usr/local/poudriere/data/packages/15amd64-clawdie/.latest/ +├── All/ +│ ├── node24-24.x.x.pkg +│ ├── bastille-0.x.x.pkg +│ └── ... +├── Latest/ +│ └── (symlinks) +├── meta.txz +├── digests.txz +└── packagesite.txz +``` + +### 4.3 Serve via nginx + +```nginx +# /usr/local/etc/nginx/nginx.conf (in poudriere jail) +server { + listen 8080; + server_name localhost; + + location /packages/ { + alias /usr/local/poudriere/data/packages/15amd64-clawdie/.latest/; + autoindex on; + } +} +``` + +--- + +## Phase 5: ISO Integration + +### 5.1 Sync packages to git jail + +```bash +# From controlplane host +rsync -avz poudriere.local:/usr/local/poudriere/data/packages/15amd64-clawdie/.latest/ \ + /usr/local/bastille/jails/git/root/srv/git/clawdie-iso/packages/ +``` + +### 5.2 Update build.sh in clawdie-iso + +```bash +# In clawdie-iso/build.sh + +# Step 1: Fetch FreeBSD base packages (stock) +pkg fetch -d -o ${ISO_ROOT}/packages/base -r latest $(cat pkg-list-base.txt) + +# Step 2: Copy Clawdie custom packages (priority) +cp -r /srv/git/clawdie-iso/packages/* ${ISO_ROOT}/packages/clawdie/ + +# Step 3: Generate combined repo +pkg repo ${ISO_ROOT}/packages/base +pkg repo ${ISO_ROOT}/packages/clawdie +``` + +### 5.3 Configure pkg priority on installed system + +```bash +# In firstboot.sh, write /usr/local/etc/pkg/repos/clawdie.conf + +cat > /usr/local/etc/pkg/repos/clawdie.conf << 'EOF' +Clawdie: { + url: "file:///usr/local/share/clawdie-iso/packages/clawdie", + mirror_type: "none", + enabled: yes, + priority: 100 +} + +FreeBSD: { + url: "pkg+https://pkg.FreeBSD.org/${ABI}/latest", + mirror_type: "srv", + enabled: yes, + priority: 50 +} +EOF +``` + +--- + +## Phase 6: Automation + +### 6.1 Build script + +File: `scripts/build-packages.sh` (in clawdie-iso repo) + +```bash +#!/bin/sh +set -e + +echo "=== Building Clawdie packages ===" + +# Trigger build in poudriere jail +jexec poudriere poudriere bulk -j 15amd64 -p clawdie -f /usr/local/etc/poudriere.d/clawdie-pkglist + +# Sync to git jail +rsync -avz --delete \ + poudriere.local:/usr/local/poudriere/data/packages/15amd64-clawdie/.latest/ \ + /srv/git/clawdie-iso/packages/ + +echo "=== Package build complete ===" +echo "Packages available at: /srv/git/clawdie-iso/packages/" +``` + +### 6.2 Cron job (optional) + +```bash +# /etc/cron.d/clawdie-build +# Rebuild packages weekly on Sunday at 03:00 +0 3 * * 0 root /home/clawdie/clawdie-iso/scripts/build-packages.sh >> /var/log/clawdie-build.log 2>&1 +``` + +--- + +## Phase 7: Update Flow + +### 7.1 When Clawdie-AI changes + +```bash +# 1. Update ports tree +jexec poudriere poudriere ports -u -p clawdie + +# 2. Rebuild affected packages +jexec poudriere poudriere bulk -j 15amd64 -p clawdie -f /usr/local/etc/poudriere.d/clawdie-pkglist + +# 3. Sync to git jail +rsync -avz poudriere.local:/usr/local/poudriere/data/packages/15amd64-clawdie/.latest/ \ + /srv/git/clawdie-iso/packages/ + +# 4. Rebuild ISO +cd /srv/git/clawdie-iso && ./build.sh +``` + +### 7.2 When FreeBSD releases security update + +```bash +# Stock packages are fetched fresh on each ISO build +# Custom packages only need rebuild if they depend on updated base +``` + +--- + +## Resource Requirements + +| Resource | Value | +|----------|-------| +| **Jail disk** | 10 GB ZFS dataset | +| **Build jail** | 5 GB (15amd64 build environment) | +| **Ports tree** | 1 GB | +| **Output packages** | 500 MB - 1 GB | +| **Build time (first)** | 30-60 minutes | +| **Build time (incremental)** | 5-15 minutes | +| **RAM** | 2 GB recommended for build jail | + +--- + +## File Summary + +| File | Location | Purpose | +|------|----------|---------| +| `poudriere.conf` | poudriere jail:/usr/local/etc/ | Main config | +| `clawdie-pkglist` | poudriere jail:/usr/local/etc/poudriere.d/ | Package list | +| `15amd64-clawdie-make.conf` | poudriere jail:/usr/local/etc/poudriere.d/ | Build options | +| `build-packages.sh` | clawdie-iso repo | Build automation | +| `packages/` | git jail:/srv/git/clawdie-iso/ | Built packages | + +--- + +## Implementation Checklist + +- [ ] Phase 1: Create poudriere jail on controlplane +- [ ] Phase 1: Install and configure Poudriere +- [ ] Phase 2: Define package list (start with ~10 packages) +- [ ] Phase 2: Create custom make.conf +- [ ] Phase 3: Create local ports overlay (optional) +- [ ] Phase 4: Run first build and verify output +- [ ] Phase 5: Integrate with build.sh in clawdie-iso +- [ ] Phase 5: Configure pkg priority on target system +- [ ] Phase 6: Create build-packages.sh automation +- [ ] Phase 7: Test ISO build with custom packages +- [ ] Phase 7: Document update procedure + +--- + +## Alternative Options Considered + +| Option | Description | Why not chosen | +|--------|-------------|----------------| +| Minimal (pre-build only) | Build ~20 custom packages, rest stock | Less control, potential ABI conflicts | +| Full custom repo | Build all ~500 packages | Long builds, must track all security updates | +| CI/CD pipeline | Automated builds on commit | Complex setup, requires self-hosted runner | +| Poudriere-as-a-Service | Dedicated build jail serving live systems | Good for later, overkill for v1 | +| Remote build service | Outsource to cloud VM | External dependency, credential management | + +--- + +## References + +- [Poudriere Wiki](https://github.com/freebsd/poudriere/wiki) +- [FreeBSD Handbook: Building Packages with poudriere](https://docs.freebsd.org/en/books/handbook/ports/#ports-poudriere) +- [CLAWDIE-ISO.md](CLAWDIE-ISO.md) — USB installer master plan