clawdie/clawdie-iso
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: add Poudriere hybrid package system implementation plan

Browse source 
Option 3 from brainstorming session — two-tier package system:
- Base layer: stock FreeBSD packages from official latest repo
- Clawdie layer: custom-built packages from Poudriere (priority 100)

Covers:
- Phase 1-7 implementation steps
- Jail setup and configuration
- Package list and make.conf
- ISO integration and automation
- Resource requirements and update flow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Sam & Claude 2026-03-17 12:02:02 +01:00
parent 798e149305
commit 4235d4e107
1 changed files with 414 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

414
POUDRIERE-IMPLEMENTATION.md Normal file
View file

@ -0,0 +1,414 @@
# Poudriere Implementation Plan — Hybrid Package System
**Status:** Planning — not yet implemented
**Last updated:** 17.mar.2026
**Target:** Clawdie-ISO v1.1+
---
## Overview
**Goal:** Two-tier package system for Clawdie-ISO
- **Base layer:** Stock FreeBSD packages from official `latest` repo
- **Clawdie layer:** Custom-built packages from Poudriere (priority: 100)
**Result:** USB image with offline packages that match Clawdie requirements, while still benefiting from FreeBSD security updates for base system.
---
## Why Hybrid?
| Approach | Pros | Cons |
|----------|------|------|
| Stock only | Simple, security updates from FreeBSD | Can't customize compile options, may lag on Node/Bastille versions |
| Full custom | Complete control | Long builds, must track all security updates manually |
| **Hybrid** | Best of both — security from base, control over Clawdie layer | Two repos to manage |
---
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ controlplane host │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ git jail │ │ db jail │ │ cms jail │ │
│ │ (.1) │ │ (.3) │ │ (.4) │ │
│ │ clawdie-iso │ │ postgres │ │ nginx │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ poudriere jail (new) │ │
│ │ (.5) │ │
│ │ ┌─────────────────────────────────────────────┐ │ │
│ │ │ ZFS: zroot/jails/poudriere/data │ │ │
│ │ │ - /usr/local/poudriere/data/packages │ │ │
│ │ │ - /usr/local/poudriere/data/logs │ │ │
│ │ │ - /usr/local/poudriere/jails/15amd64 │ │ │
│ │ └─────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ Services: │ │
│ │ - poudriere (build tool) │ │
│ │ - nginx (pkg repo server on :8080) │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ Build output → rsync → git jail → injected into ISO │
└─────────────────────────────────────────────────────────────┘
```
---
## Phase 1: Poudriere Jail Setup
### 1.1 Create the jail
```bash
# On controlplane host
bastille create poudriere 15.0-RELEASE 10.0.0.5 lagg0
bastille config poudriere set host.hostname poudriere.local
bastille start poudriere
```
### 1.2 Install Poudriere
```bash
bastille pkg poudriere install poudriere dialog4ports
```
### 1.3 Configure ZFS dataset
```bash
# Create dedicated ZFS dataset for build data
zfs create -o mountpoint=/usr/local/poudriere/data zroot/poudriere
bastille config poudriere set mount.fstab += "zroot/poudriere /usr/local/poudriere/data nullfs rw 0 0"
```
### 1.4 Poudriere configuration
```bash
# /usr/local/etc/poudriere.conf (inside jail)
ZROOTFS=zroot/poudriere
FREEBSD_HOST=https://download.freebsd.org
RESOLV_CONF=/etc/resolv.conf
BASEFS=/usr/local/poudriere
USE_PORTLINT=no
USE_TMPFS=yes
NOLINUX=yes
```
---
## Phase 2: Build Environment
### 2.1 Create build jail
```bash
# Inside poudriere jail
poudriere jail -c -j 15amd64 -v 15.0-RELEASE -a amd64
```
### 2.2 Create ports tree
```bash
poudriere ports -c -p clawdie -m git+https
```
### 2.3 Define package list
File: `/usr/local/etc/poudriere.d/clawdie-pkglist`
```
# Clawdie custom packages (built with specific options)
www/node24
sysutils/bastille
x11/desktop-installer
databases/postgresql17-client
net/rsync
devel/git
sysutils/tmux
editors/vim
shells/bash
security/sudo
# Custom Clawdie ports (to be created)
sysutils/clawdie-setup
```
### 2.4 Custom make.conf
File: `/usr/local/etc/poudriere.d/15amd64-clawdie-make.conf`
```makefile
# Node 24 with specific options
www_node24_SET=NPM OPTIMIZED_CFLAGS
# Bastille with ZFS
sysutils_bastille_SET=ZFS
# Minimal Xorg (no VNC, no VBox)
x11_servers_xorg_server_UNSET=VNC
x11_servers_xorg_server_SET=UDEV KMS
# Desktop-installer without VirtualBox
x11_desktop-installer_UNSET=VIRTUALBOX
```
---
## Phase 3: Custom Ports (Optional)
### 3.1 Create local ports overlay
```
/usr/local/poudriere/ports/clawdie/
├── sysutils/
│ └── clawdie-setup/
│ ├── Makefile
│ ├── pkg-descr
│ ├── pkg-plist
│ └── files/
│ └── setup.sh
```
### 3.2 Example Makefile for clawdie-setup
```makefile
PORTNAME= clawdie-setup
PORTVERSION= 0.1.0
CATEGORIES= sysutils
MAINTAINER= hello@clawdie.si
COMMENT= Clawdie AI post-install setup wizard
RUN_DEPENDS= bash:shells/bash \
tmux:sysutils/tmux \
node:www/node24
USE_GITHUB= yes
GH_ACCOUNT= Clawdie
GH_PROJECT= Clawdie-AI
NO_BUILD= yes
do-install:
${MKDIR} ${STAGEDIR}${PREFIX}/share/clawdie
${INSTALL_SCRIPT} ${WRKSRC}/setup.sh ${STAGEDIR}${PREFIX}/share/clawdie/
.include <bsd.port.mk>
```
---
## Phase 4: Build Process
### 4.1 Initial build
```bash
# Inside poudriere jail
poudriere bulk -j 15amd64 -p clawdie -f /usr/local/etc/poudriere.d/clawdie-pkglist
```
### 4.2 Output location
```
/usr/local/poudriere/data/packages/15amd64-clawdie/.latest/
├── All/
│ ├── node24-24.x.x.pkg
│ ├── bastille-0.x.x.pkg
│ └── ...
├── Latest/
│ └── (symlinks)
├── meta.txz
├── digests.txz
└── packagesite.txz
```
### 4.3 Serve via nginx
```nginx
# /usr/local/etc/nginx/nginx.conf (in poudriere jail)
server {
listen 8080;
server_name localhost;
location /packages/ {
alias /usr/local/poudriere/data/packages/15amd64-clawdie/.latest/;
autoindex on;
}
}
```
---
## Phase 5: ISO Integration
### 5.1 Sync packages to git jail
```bash
# From controlplane host
rsync -avz poudriere.local:/usr/local/poudriere/data/packages/15amd64-clawdie/.latest/ \
/usr/local/bastille/jails/git/root/srv/git/clawdie-iso/packages/
```
### 5.2 Update build.sh in clawdie-iso
```bash
# In clawdie-iso/build.sh
# Step 1: Fetch FreeBSD base packages (stock)
pkg fetch -d -o ${ISO_ROOT}/packages/base -r latest $(cat pkg-list-base.txt)
# Step 2: Copy Clawdie custom packages (priority)
cp -r /srv/git/clawdie-iso/packages/* ${ISO_ROOT}/packages/clawdie/
# Step 3: Generate combined repo
pkg repo ${ISO_ROOT}/packages/base
pkg repo ${ISO_ROOT}/packages/clawdie
```
### 5.3 Configure pkg priority on installed system
```bash
# In firstboot.sh, write /usr/local/etc/pkg/repos/clawdie.conf
cat > /usr/local/etc/pkg/repos/clawdie.conf << 'EOF'
Clawdie: {
url: "file:///usr/local/share/clawdie-iso/packages/clawdie",
mirror_type: "none",
enabled: yes,
priority: 100
}
FreeBSD: {
url: "pkg+https://pkg.FreeBSD.org/${ABI}/latest",
mirror_type: "srv",
enabled: yes,
priority: 50
}
EOF
```
---
## Phase 6: Automation
### 6.1 Build script
File: `scripts/build-packages.sh` (in clawdie-iso repo)
```bash
#!/bin/sh
set -e
echo "=== Building Clawdie packages ==="
# Trigger build in poudriere jail
jexec poudriere poudriere bulk -j 15amd64 -p clawdie -f /usr/local/etc/poudriere.d/clawdie-pkglist
# Sync to git jail
rsync -avz --delete \
poudriere.local:/usr/local/poudriere/data/packages/15amd64-clawdie/.latest/ \
/srv/git/clawdie-iso/packages/
echo "=== Package build complete ==="
echo "Packages available at: /srv/git/clawdie-iso/packages/"
```
### 6.2 Cron job (optional)
```bash
# /etc/cron.d/clawdie-build
# Rebuild packages weekly on Sunday at 03:00
0 3 * * 0 root /home/clawdie/clawdie-iso/scripts/build-packages.sh >> /var/log/clawdie-build.log 2>&1
```
---
## Phase 7: Update Flow
### 7.1 When Clawdie-AI changes
```bash
# 1. Update ports tree
jexec poudriere poudriere ports -u -p clawdie
# 2. Rebuild affected packages
jexec poudriere poudriere bulk -j 15amd64 -p clawdie -f /usr/local/etc/poudriere.d/clawdie-pkglist
# 3. Sync to git jail
rsync -avz poudriere.local:/usr/local/poudriere/data/packages/15amd64-clawdie/.latest/ \
/srv/git/clawdie-iso/packages/
# 4. Rebuild ISO
cd /srv/git/clawdie-iso && ./build.sh
```
### 7.2 When FreeBSD releases security update
```bash
# Stock packages are fetched fresh on each ISO build
# Custom packages only need rebuild if they depend on updated base
```
---
## Resource Requirements
| Resource | Value |
|----------|-------|
| **Jail disk** | 10 GB ZFS dataset |
| **Build jail** | 5 GB (15amd64 build environment) |
| **Ports tree** | 1 GB |
| **Output packages** | 500 MB - 1 GB |
| **Build time (first)** | 30-60 minutes |
| **Build time (incremental)** | 5-15 minutes |
| **RAM** | 2 GB recommended for build jail |
---
## File Summary
| File | Location | Purpose |
|------|----------|---------|
| `poudriere.conf` | poudriere jail:/usr/local/etc/ | Main config |
| `clawdie-pkglist` | poudriere jail:/usr/local/etc/poudriere.d/ | Package list |
| `15amd64-clawdie-make.conf` | poudriere jail:/usr/local/etc/poudriere.d/ | Build options |
| `build-packages.sh` | clawdie-iso repo | Build automation |
| `packages/` | git jail:/srv/git/clawdie-iso/ | Built packages |
---
## Implementation Checklist
- [ ] Phase 1: Create poudriere jail on controlplane
- [ ] Phase 1: Install and configure Poudriere
- [ ] Phase 2: Define package list (start with ~10 packages)
- [ ] Phase 2: Create custom make.conf
- [ ] Phase 3: Create local ports overlay (optional)
- [ ] Phase 4: Run first build and verify output
- [ ] Phase 5: Integrate with build.sh in clawdie-iso
- [ ] Phase 5: Configure pkg priority on target system
- [ ] Phase 6: Create build-packages.sh automation
- [ ] Phase 7: Test ISO build with custom packages
- [ ] Phase 7: Document update procedure
---
## Alternative Options Considered
| Option | Description | Why not chosen |
|--------|-------------|----------------|
| Minimal (pre-build only) | Build ~20 custom packages, rest stock | Less control, potential ABI conflicts |
| Full custom repo | Build all ~500 packages | Long builds, must track all security updates |
| CI/CD pipeline | Automated builds on commit | Complex setup, requires self-hosted runner |
| Poudriere-as-a-Service | Dedicated build jail serving live systems | Good for later, overkill for v1 |
| Remote build service | Outsource to cloud VM | External dependency, credential management |
---
## References
- [Poudriere Wiki](https://github.com/freebsd/poudriere/wiki)
- [FreeBSD Handbook: Building Packages with poudriere](https://docs.freebsd.org/en/books/handbook/ports/#ports-poudriere)
- [CLAWDIE-ISO.md](CLAWDIE-ISO.md) — USB installer master plan