clawdie-iso/TESTING.md

9.6 KiB
Raw Blame History

Clawdie-ISO Testing Guide

Version: v0.9.0 Date: 26.mar.2026 Tested On: FreeBSD 15.0-RELEASE amd64 Test Platform: bhyve (KVM)


Overview

This guide covers testing the Clawdie-ISO installer end-to-end. Testing happens in two levels:

  1. Module Integration Test — Verify all 5 shell modules work together (5 min, offline)
  2. Full Bhyve Boot Test — Boot ISO in VM, run bsdinstall + firstboot (2025 min, interactive)

Before running bhyve tests, run the read-only host preflight:

./scripts/preflight-host.sh

If the host uses an isolated bhyve bridge, confirm the test VM runs on bhyve0 (10.99.0.0/24) rather than warden0 to avoid cross-traffic with Clawdie jails.

After the bhyve boot test completes, validate the installed system with the fresh-install checklist in the Clawdie-AI repo:

/home/clawdie/clawdie-ai/docs/FRESH-INSTALL-CHECKLIST.md


Level 1: Module Integration Test

Quick validation that all shell modules execute in sequence with proper state handoff.

Run the test

cd /home/clawdie/clawdie-iso
sh firstboot/integration-test.sh

Expected output

╔════════════════════════════════════════════════════════════════╗
║ Clawdie Shell v0.9.0 Integration Test                          ║
║ 6-module sequential execution (cloud/VM scenario)              ║
╚════════════════════════════════════════════════════════════════╝

[1/6] clawdie-shell-env.sh → Identity + .env
  ✓ .env created (65 variables)

[2/6] clawdie-shell-pkg.sh → Repos (online + offline USB)
  ✓ FreeBSD repo configured
  ✓ USB repo configured

[3/6] clawdie-shell-gpu.sh → GPU detection (Intel)
  ✓ GPU detection: intel
  ✓ rc.conf updated

[4/6] clawdie-shell-nvidia.sh → NVIDIA driver (skipped for Intel)
  ✓ NVIDIA module: skipped (correct for Intel GPU)

[5/6] clawdie-shell-system.sh → Hostname, timezone, services
  ✓ Hostname configured
  ✓ Profile environment setup

[6/6] clawdie-shell-deploy.sh → Clawdie-AI deployment
  ✓ Clawdie-AI deployed
  ✓ node_modules installed

╔════════════════════════════════════════════════════════════════╗
║ Integration Test Complete                                      ║
║ ✓ All 6 modules executed with state handoff
╚════════════════════════════════════════════════════════════════╝

What this tests

  • Environment variable export between modules
  • File creation (rc.conf, .env, repos)
  • Error handling (functions return non-zero on failure)
  • Logging to progress file
  • No undefined function calls

If test passes: Ready for bhyve testing. If test fails: Check /tmp/clawdie-int-*/var/log/integration.log for errors.


Level 2: Full Bhyve Boot Test

Complete end-to-end installation and configuration in a virtual machine.

Prerequisites

# Check bhyve is available
which bhyve
kldstat | grep vmm

# Check disk space
df -h /

# Create tap0 if not already present
ifconfig tap0 create 2>/dev/null || echo "tap0 already exists"

All should succeed. You need:

  • bhyve binary
  • vmm kernel module loaded
  • 30GB free disk (25GB ISO + 50GB VM disk)
  • Network interface

Run the test

# This will create a bhyve VM and boot the ISO
cd /home/clawdie/clawdie-iso
sudo ./scripts/bhyve-test.sh

Boot sequence

The script will:

  1. Create VM — bhyve VM named clawdie-test with 2 CPUs, 2GB RAM
  2. Boot ISO — Load clawdie-iso-baremetal-26.mar.2026.img as CDROM
  3. FreeBSD Installer — bsdinstall drops to interactive prompt
  4. Disk partitioning — Default options (ZFS root)
  5. Reboot — After install, reboots into HDD
  6. First boot — rc.d/clawdie-firstboot triggers firstboot.sh
  7. Firstboot wizard — Prompts for assistant name, domain, timezone, and Forgejo web UI toggle (non-interactive in this test, uses defaults)
  8. Module execution — GPU detection → deployment → service startup

Timeline

Phase Duration What happens
bsdinstall 510 min Disk layout, base system, reboot
First boot 35 min rc.d startup sequence
Firstboot wizard 23 min bsddialog prompts (or auto answers)
GPU detection <1 min pciconf → rc.conf update
Package setup 23 min Repo config + cache seeding
Clawdie deploy 510 min Extract tarball + npm install-all
Total ~2025 min System ready for login

What to verify

After boot completes, check the VM console for:

GPU detection

[gpu] Detected GPU vendor: intel
[gpu] Matched driver: i915kms
[gpu] Wrote kld_list=i915kms to /etc/rc.conf

Package configuration

[pkg] System ABI: FreeBSD:15:amd64
[pkg] Wrote /etc/pkg/repos/FreeBSD.conf with branch=latest
[pkg] Wrote /etc/pkg/repos/Clawdie-USB.conf pointing to /usr/local/share/clawdie-iso/packages

Environment generation

[env] Wrote .env with 45 configuration lines
[env] .env validation passed (45 lines)

System configuration

[system] Updated rc.conf
[system] Set hostname to clawdie.local (or provided domain)
[system] Enabled services: dbus, hald, seatd, lightdm

Clawdie deployment

[deploy] Extracting /usr/local/share/clawdie-iso/clawdie-ai.tar.gz
[deploy] Tarball extracted to /home/clawdie/clawdie-ai
[deploy] Running npm run install-all...
[deploy] npm install-all completed successfully
[deploy] ✓ Jails active: 4 jails
[deploy] ✓ clawdie service configured in rc.conf
[deploy] Clawdie-AI deployment complete

Forgejo (optional)

When Forgejo is enabled, verify the service and port:

sudo bastille cmd ${AGENT_NAME}-git service forgejo onestatus
sockstat -4l | grep ':3000'

Git jail storage (always)

sudo bastille cmd ${AGENT_NAME}-git ls -la /srv/git

Forgejo UI reachability (optional)

fetch -o - http://git.${AGENT_NAME}.home.arpa:3000 | head -n 5

Desktop ready

Login prompt appears (Lumina greeter via lightdm)
User: clawdie (password auto-generated, shown in .env)

Stopping the VM

Once testing is complete:

# From another terminal, destroy the VM
bhyvectl --vm=clawdie-test --destroy

Or press Ctrl-C in the bhyve console (may not work cleanly; destroy is cleaner).


Troubleshooting

VM won't boot

Symptom: Freezes at bhyve boot prompt

Cause: Usually missing or corrupted ISO

Fix:

# Verify ISO
file /home/clawdie/clawdie-iso/clawdie-iso-baremetal-*.img
# Should show: DOS/MBR boot sector

# Check size
ls -lh /home/clawdie/clawdie-iso/clawdie-iso-baremetal-*.img
# Should be ~25G

# Rebuild if needed
cd /home/clawdie/clawdie-iso
sudo ./build.sh --skip-fetch

bsdinstall hangs

Symptom: Installer stops responding after disk selection

Cause: bhyve disk I/O issue (rare)

Fix:

# Destroy and retry
bhyvectl --vm=clawdie-test --destroy
cd /home/clawdie/clawdie-iso
sudo ./scripts/bhyve-test.sh

Firstboot doesn't trigger

Symptom: System boots to login prompt, no installation started

Cause: rc.d service not enabled or firstboot payload missing

Verify:

# In VM console (login as root if possible)
grep clawdie_firstboot /etc/rc.conf
ls -la /usr/local/share/clawdie-iso/firstboot/
service clawdie-firstboot status

Module errors in logs

Symptom: One of the [gpu], [pkg], [env], [deploy] stages fails

Debug:

# SSH into VM (if network working)
ssh root@<vm-ip>

# Check logs
tail -f /var/log/clawdie-firstboot.log

# Check progress
cat /var/log/clawdie-firstboot.progress

Test Results Checklist

Before marking test as passed, verify:

  • bsdinstall completed without errors
  • System rebooted from HDD (not USB)
  • rc.d/clawdie-firstboot triggered on first boot
  • GPU detection ran and set correct kld_list
  • Package repos configured
  • .env file created with secrets
  • Hostname and timezone set
  • Clawdie-AI extracted and npm install-all ran
  • At least one jail is running
  • Lumina desktop login appears
  • No critical errors in /var/log/clawdie-firstboot.log

Next Steps

If all tests pass:

  • Commit ISO to git: git add clawdie-iso-baremetal-*.img && git commit -m "build: Final ISO for v0.9.0"
  • Test on real hardware (Intel, AMD, NVIDIA systems)
  • Begin Phase 3 (clawdie-admin.sh UI)

If tests fail:

  • Check logs in VM: /var/log/clawdie-firstboot.log
  • Review SHELL-MODULES.md for module details
  • Fix issue in shell module
  • Rebuild ISO: sudo ./build.sh --skip-fetch
  • Retry test

CI/CD Integration

For automated testing in CI pipeline:

#!/bin/sh
# ci-test-installer.sh

# 1. Module integration test (fast, no root)
sh firstboot/integration-test.sh || exit 1

# 2. Build ISO
sudo ./build.sh --skip-fetch || exit 1

# 3. Boot in bhyve (requires interactive console capture)
# [Not yet automated — manual testing or expect script needed]

echo "✓ Installer tests passed"

See Also