clawdie-iso/TESTING.md

# Clawdie-ISO Testing Guide

**Version:** v0.9.0
**Date:** 26.mar.2026
**Tested On:** FreeBSD 15.0-RELEASE amd64
**Test Platform:** bhyve (KVM)

---

## Overview

This guide covers testing the Clawdie-ISO installer end-to-end. Testing happens in two levels:

1. **Module Integration Test** — Verify all 5 shell modules work together (5 min, offline)
2. **Full Bhyve Boot Test** — Boot ISO in VM, run bsdinstall + firstboot (20–25 min, interactive)

---

## Level 1: Module Integration Test

Quick validation that all shell modules execute in sequence with proper state handoff.

### Run the test

```bash
cd /home/clawdie/clawdie-iso
sh firstboot/integration-test.sh
```

### Expected output

```
╔════════════════════════════════════════════════════════════════╗
║ Clawdie Shell v0.9.0 Integration Test                          ║
║ 6-module sequential execution (cloud/VM scenario)              ║
╚════════════════════════════════════════════════════════════════╝

[1/6] clawdie-shell-env.sh → Identity + .env
  ✓ .env created (65 variables)

[2/6] clawdie-shell-pkg.sh → Repos (online + offline USB)
  ✓ FreeBSD repo configured
  ✓ USB repo configured

[3/6] clawdie-shell-gpu.sh → GPU detection (Intel)
  ✓ GPU detection: intel
  ✓ rc.conf updated

[4/6] clawdie-shell-nvidia.sh → NVIDIA driver (skipped for Intel)
  ✓ NVIDIA module: skipped (correct for Intel GPU)

[5/6] clawdie-shell-system.sh → Hostname, timezone, services
  ✓ Hostname configured
  ✓ Profile environment setup

[6/6] clawdie-shell-deploy.sh → Clawdie-AI deployment
  ✓ Clawdie-AI deployed
  ✓ node_modules installed

╔════════════════════════════════════════════════════════════════╗
║ Integration Test Complete                                      ║
║ ✓ All 6 modules executed with state handoff
╚════════════════════════════════════════════════════════════════╝
```

### What this tests

- ✅ Environment variable export between modules
- ✅ File creation (rc.conf, .env, repos)
- ✅ Error handling (functions return non-zero on failure)
- ✅ Logging to progress file
- ✅ No undefined function calls

**If test passes:** Ready for bhyve testing.
**If test fails:** Check `/tmp/clawdie-int-*/var/log/integration.log` for errors.

---

## Level 2: Full Bhyve Boot Test

Complete end-to-end installation and configuration in a virtual machine.

### Prerequisites

```bash
# Check bhyve is available
which bhyve
kldstat | grep vmm

# Check disk space
df -h /

# Create tap0 if not already present
ifconfig tap0 create 2>/dev/null || echo "tap0 already exists"
```

All should succeed. You need:
- bhyve binary
- vmm kernel module loaded
- 30GB free disk (25GB ISO + 50GB VM disk)
- Network interface

### Run the test

```bash
# This will create a bhyve VM and boot the ISO
cd /home/clawdie/clawdie-iso
sudo ./tmp/bhyve-test-setup.sh
```

### Boot sequence

The script will:

1. **Create VM** — bhyve VM named `clawdie-test` with 2 CPUs, 2GB RAM
2. **Boot ISO** — Load clawdie-iso-baremetal-26.mar.2026.img as CDROM
3. **FreeBSD Installer** — bsdinstall drops to interactive prompt
4. **Disk partitioning** — Default options (ZFS root)
5. **Reboot** — After install, reboots into HDD
6. **First boot** — rc.d/clawdie-firstboot triggers firstboot.sh
7. **Firstboot wizard** — Prompts for assistant name, domain, timezone (non-interactive in this test, uses defaults)
8. **Module execution** — GPU detection → deployment → service startup

### Timeline

| Phase | Duration | What happens |
|-------|----------|--------------|
| bsdinstall | 5–10 min | Disk layout, base system, reboot |
| First boot | 3–5 min | rc.d startup sequence |
| Firstboot wizard | 2–3 min | bsddialog prompts (or auto answers) |
| GPU detection | <1 min | pciconf → rc.conf update |
| Package setup | 2–3 min | Repo config + cache seeding |
| Clawdie deploy | 5–10 min | Extract tarball + npm install-all |
| **Total** | **~20–25 min** | System ready for login |

### What to verify

After boot completes, check the VM console for:

#### ✅ GPU detection
```
[gpu] Detected GPU vendor: intel
[gpu] Matched driver: i915kms
[gpu] Wrote kld_list=i915kms to /etc/rc.conf
```

#### ✅ Package configuration
```
[pkg] System ABI: FreeBSD:15:amd64
[pkg] Wrote /etc/pkg/repos/FreeBSD.conf with branch=latest
[pkg] Wrote /etc/pkg/repos/Clawdie-USB.conf pointing to /usr/local/share/clawdie-iso/packages
```

#### ✅ Environment generation
```
[env] Wrote .env with 45 configuration lines
[env] .env validation passed (45 lines)
```

#### ✅ System configuration
```
[system] Updated rc.conf
[system] Set hostname to clawdie.local (or provided domain)
[system] Enabled services: dbus, hald, seatd, lightdm
```

#### ✅ Clawdie deployment
```
[deploy] Extracting /usr/local/share/clawdie-iso/clawdie-ai.tar.gz
[deploy] Tarball extracted to /home/clawdie/clawdie-ai
[deploy] Running npm run install-all...
[deploy] npm install-all completed successfully
[deploy] ✓ Jails active: 4 jails
[deploy] ✓ clawdie service configured in rc.conf
[deploy] Clawdie-AI deployment complete
```

#### ✅ Desktop ready
```
Login prompt appears (Lumina greeter via lightdm)
User: clawdie (password auto-generated, shown in .env)
```

### Stopping the VM

Once testing is complete:

```bash
# From another terminal, destroy the VM
bhyvectl --vm=clawdie-test --destroy
```

Or press `Ctrl-C` in the bhyve console (may not work cleanly; destroy is cleaner).

---

## Troubleshooting

### VM won't boot

**Symptom:** Freezes at bhyve boot prompt

**Cause:** Usually missing or corrupted ISO

**Fix:**
```bash
# Verify ISO
file /home/clawdie/clawdie-iso/clawdie-iso-baremetal-*.img
# Should show: DOS/MBR boot sector

# Check size
ls -lh /home/clawdie/clawdie-iso/clawdie-iso-baremetal-*.img
# Should be ~25G

# Rebuild if needed
cd /home/clawdie/clawdie-iso
sudo ./build.sh --skip-fetch
```

### bsdinstall hangs

**Symptom:** Installer stops responding after disk selection

**Cause:** bhyve disk I/O issue (rare)

**Fix:**
```bash
# Destroy and retry
bhyvectl --vm=clawdie-test --destroy
cd /home/clawdie/clawdie-iso
sudo ./tmp/bhyve-test-setup.sh
```

### Firstboot doesn't trigger

**Symptom:** System boots to login prompt, no installation started

**Cause:** rc.d service not enabled or firstboot payload missing

**Verify:**
```bash
# In VM console (login as root if possible)
grep clawdie_firstboot /etc/rc.conf
ls -la /usr/local/share/clawdie-iso/firstboot/
service clawdie-firstboot status
```

### Module errors in logs

**Symptom:** One of the [gpu], [pkg], [env], [deploy] stages fails

**Debug:**
```bash
# SSH into VM (if network working)
ssh root@<vm-ip>

# Check logs
tail -f /var/log/clawdie-firstboot.log

# Check progress
cat /var/log/clawdie-firstboot.progress
```

---

## Test Results Checklist

Before marking test as passed, verify:

- [ ] bsdinstall completed without errors
- [ ] System rebooted from HDD (not USB)
- [ ] rc.d/clawdie-firstboot triggered on first boot
- [ ] GPU detection ran and set correct kld_list
- [ ] Package repos configured
- [ ] .env file created with secrets
- [ ] Hostname and timezone set
- [ ] Clawdie-AI extracted and npm install-all ran
- [ ] At least one jail is running
- [ ] Lumina desktop login appears
- [ ] No critical errors in /var/log/clawdie-firstboot.log

---

## Next Steps

**If all tests pass:**
- Commit ISO to git: `git add clawdie-iso-baremetal-*.img && git commit -m "build: Final ISO for v0.9.0"`
- Test on real hardware (Intel, AMD, NVIDIA systems)
- Begin Phase 3 (clawdie-admin.sh UI)

**If tests fail:**
- Check logs in VM: `/var/log/clawdie-firstboot.log`
- Review SHELL-MODULES.md for module details
- Fix issue in shell module
- Rebuild ISO: `sudo ./build.sh --skip-fetch`
- Retry test

---

## CI/CD Integration

For automated testing in CI pipeline:

```bash
#!/bin/sh
# ci-test-installer.sh

# 1. Module integration test (fast, no root)
sh firstboot/integration-test.sh || exit 1

# 2. Build ISO
sudo ./build.sh --skip-fetch || exit 1

# 3. Boot in bhyve (requires interactive console capture)
# [Not yet automated — manual testing or expect script needed]

echo "✓ Installer tests passed"
```

---

## See Also

- [IMPLEMENTATION-PLAN.md](IMPLEMENTATION-PLAN.md) — Task status
- [SHELL-MODULES.md](SHELL-MODULES.md) — Module documentation
- [BUILD.md](BUILD.md) — Building the ISO