clawdie-iso/SHELL-ARCHITECTURE.md

Clawdie Shell Architecture — Modular Shell Design

Purpose: Explain the shell-based deployment architecture and why this approach

Audience: Developers, operators, contributors

---

Design Philosophy

Traditional installers often do one of two things:

1. Monolithic: Single script that does everything (hard to test, debug, reuse)
2. Heavy framework: External tool (Python, Perl, Ansible) with VM overhead and dependencies

Clawdie Shell chooses a third path: Modular shell functions organized by concern, sourceable independently, POSIX-compliant.

---

Architecture Overview

firstboot.sh (52 lines)
  ├─ Parse bsddialog wizard input
  ├─ Source: clawdie-shell-env.sh
  ├─ Source: clawdie-shell-pkg.sh
  ├─ Source: clawdie-shell-gpu.sh
  ├─ Source: clawdie-shell-system.sh
  ├─ Source: clawdie-shell-clawdie.sh
  └─ Call functions in sequence:
      ├─ clawdie_shell_pkg_setup()
      ├─ clawdie_shell_gpu_detect()
      ├─ clawdie_shell_system_config()
      ├─ clawdie_shell_env_generate()
      └─ clawdie_shell_clawdie_setup()

Each module is:

• Standalone: Can be sourced without others
• Idempotent: Safe to run twice (checks state first)
• Testable: Functions have clear inputs/outputs
• Debuggable: Can run clawdie_shell_gpu_detect() in isolation

---

Module 1: clawdie-shell-env.sh

Purpose: Generate .env file with all 65 environment variables

Inputs:

• $ASSISTANT_NAME — from wizard (e.g., "Clawdie")
• $AGENT_DOMAIN — from wizard (e.g., "clawdie.local")
• $TZ — from wizard (e.g., "Europe/Ljubljana")
• $LLM_PROVIDER — from wizard (e.g., "anthropic")
• $TELEGRAM_TOKEN — from wizard (optional)

Outputs:

• /home/clawdie/clawdie-ai/.env — complete env file
• Logs to /var/log/clawdie-firstboot.log

Functions:

clawdie_shell_env_generate() {
  # Main entry point
  # Generates 9 random secrets (32-char base64)
  # Calculates 34 structural vars (jail IPs, names, etc.)
  # Writes .env with all 65 vars

  # Calls internal helpers:
  clawdie_shell_env_gen_secrets     # openssl rand
  clawdie_shell_env_derive_names    # AGENT_NAME from ASSISTANT_NAME
  clawdie_shell_env_derive_ips      # Jail subnet allocation
  clawdie_shell_env_derive_providers # LLM provider vars
}

clawdie_shell_env_gen_secrets() {
  # Output 9 secrets (DB pwd, API keys, JWT secrets, etc.)
}

clawdie_shell_env_derive_names() {
  # Lowercase, strip spaces: "Clawdie Smith" → "clawdie-smith"
  # Generate machine-safe names
}

clawdie_shell_env_derive_ips() {
  # Allocate jail IPs from base subnet (10.0.0.0/24)
  # worker: 10.0.0.101, db: 10.0.0.102, cms: 10.0.0.103, etc.
}

clawdie_shell_env_validate() {
  # Check .env is readable, all vars present
  # Run before npm install
}

---

Module 2: clawdie-shell-pkg.sh

Purpose: Configure package repositories and prep for offline installation

Inputs:

• USB mounted at /mnt/media (package cache)
• System ABI detected from pkg config

Outputs:

• /etc/pkg/repos/FreeBSD.conf — online repo (priority 0, fallback)
• /etc/pkg/repos/Clawdie-USB.conf — offline repo (priority 100, used first)
• Package cache seeded to /var/cache/pkg/bastille

Functions:

clawdie_shell_pkg_setup() {
  # Main entry point
  # 1. Detect system ABI
  # 2. Write FreeBSD.conf (online, fallback)
  # 3. Write Clawdie-USB.conf (offline, preferred)
  # 4. Update pkg metadata
  # 5. Seed bastille pkg cache from USB

  clawdie_shell_pkg_detect_abi
  clawdie_shell_pkg_write_config
  clawdie_shell_pkg_update_metadata
  clawdie_shell_pkg_seed_cache
}

clawdie_shell_pkg_detect_abi() {
  # Query: pkg config abi
  # Output: FreeBSD:15:amd64 (or FreeBSD:15:arm64, etc.)
}

clawdie_shell_pkg_write_config() {
  # Write /etc/pkg/repos/FreeBSD.conf
  # URL: pkg+https://pkg.FreeBSD.org/${ABI}/latest

  # Write /etc/pkg/repos/Clawdie-USB.conf
  # URL: file:///mnt/media/packages/
  # Priority: 100 (higher = preferred)
}

clawdie_shell_pkg_seed_cache() {
  # Copy USB packages to /var/cache/pkg/bastille
  # Jails will use local cache for offline provisioning
}

---

Module 3: clawdie-shell-gpu.sh

Purpose: Detect GPU hardware and load appropriate kernel modules

Inputs:

• System PCI bus (via pciconf -lv)
• No interactive input (detection automatic)

Outputs:

• kld_list written to /etc/rc.conf
• /etc/X11/xorg.conf.d/ updated if needed

Functions:

clawdie_shell_gpu_detect() {
  # Main entry point
  # 1. Query PCI devices
  # 2. Match against known IDs
  # 3. Select appropriate kld(s)
  # 4. Write rc.conf
  # 5. Load modules live (if possible)

  clawdie_shell_gpu_detect_pci
  clawdie_shell_gpu_match_driver
  clawdie_shell_gpu_write_rcconf
  clawdie_shell_gpu_load_live
}

clawdie_shell_gpu_detect_pci() {
  # pciconf -lv | grep -i vga
  # Extract vendor:device IDs
  # Returns: intel | amd | nvidia | vmware | vesa (fallback)
}

clawdie_shell_gpu_match_driver() {
  # Lookup table:
  # Intel → i915kms
  # AMD → amdgpu
  # NVIDIA → nvidia-modeset + nvidia
  # VMware → vmwgfx
  # Unknown → vesa (software fallback)
}

clawdie_shell_gpu_write_rcconf() {
  # Example:
  # kld_list="i915kms"
}

clawdie_shell_gpu_load_live() {
  # Optional: kldload i915kms
  # (May fail in chroot; safe to skip)
}

---

Module 4: clawdie-shell-system.sh

Purpose: System-level configuration (hostname, timezone, services)

Inputs:

• $TZ — from wizard
• GPU kld selection (from gpu module)
• Desktop choice (Lumina, fixed)

Outputs:

• /etc/rc.conf — updated with timezone, kld_list, services
• /etc/hostname — set to agent domain
• /etc/profile.d/clawdie.sh — environment setup
• dbus, hald services enabled

Functions:

clawdie_shell_system_config() {
  # Main entry point
  clawdie_shell_system_write_rcconf
  clawdie_shell_system_set_hostname
  clawdie_shell_system_setup_env
  clawdie_shell_system_enable_services
}

clawdie_shell_system_write_rcconf() {
  # Write to /etc/rc.conf:
  # timezone="Europe/Ljubljana"
  # kld_list="i915kms"
  # dbus_enable="YES"
  # hald_enable="YES"
  # seatd_enable="YES"
  # display_manager="lightdm"
  # lightdm_enable="YES"
}

clawdie_shell_system_set_hostname() {
  # /etc/hostname = $AGENT_DOMAIN
  # hostname $AGENT_DOMAIN (set live)
}

clawdie_shell_system_setup_env() {
  # /etc/profile.d/clawdie.sh:
  # export PATH="$HOME/.npm-global/bin:$PATH"
  # export npm_config_prefix="$HOME/.npm-global"
}

clawdie_shell_system_enable_services() {
  # service dbus onestart
  # service hald onestart
  # sysrc dbus_enable=YES
}

---

Module 5: clawdie-shell-clawdie.sh

Purpose: Extract Clawdie-AI, npm install, provision jails, start service

Inputs:

• /usr/local/share/clawdie-iso/clawdie-ai.tar.gz on USB
• .env from env module
• Package repos configured by pkg module

Outputs:

• /home/clawdie/clawdie-ai/ extracted and ready
• Bastille jails: worker, db, cms provisioned
• /etc/rc.d/clawdie service enabled
• PostgreSQL seeded, nginx configured

Functions:

clawdie_shell_clawdie_setup() {
  # Main entry point
  clawdie_shell_clawdie_extract
  clawdie_shell_clawdie_npm_install
  clawdie_shell_clawdie_install_all
  clawdie_shell_clawdie_verify
}

clawdie_shell_clawdie_extract() {
  # cd /home/clawdie
  # tar xzf /usr/local/share/clawdie-iso/clawdie-ai.tar.gz
  # chown -R clawdie:clawdie clawdie-ai/
}

clawdie_shell_clawdie_npm_install() {
  # cd /home/clawdie/clawdie-ai
  # npm install --offline (from USB cache)
  # npm ci (clean install, respects package-lock.json)
}

clawdie_shell_clawdie_install_all() {
  # cd /home/clawdie/clawdie-ai
  # npm run install-all
  # (Calls setup/index.ts → jails, db, cms, service)
}

clawdie_shell_clawdie_verify() {
  # Check:
  # - Jails running
  # - PostgreSQL responding
  # - nginx config valid
  # - rc.d clawdie service present
}

---

Error Handling Strategy

All modules use POSIX-compliant error handling (FreeBSD-first, no bash-isms):

#!/bin/sh
set -eu  # Exit on error AND undefined variables (POSIX safe)
trap 'echo "ERROR in clawdie_shell_FUNCTION at line $LINENO" >&2; exit 1' ERR

clawdie_shell_FUNCTION() {
  local status_file="/var/log/clawdie-firstboot.progress"

  # Log progress checkpoint
  echo "[MODULE] FUNCTION_START" >> "$status_file"

  # Do work
  # ... actual setup code ...

  # Log completion
  echo "[MODULE] FUNCTION_COMPLETE" >> "$status_file"
}

Key principles:

1. set -eu — Exit on error, fail on undefined variables (POSIX, no bash-isms)
2. trap ERR — Catch unhandled errors with line numbers
3. Progress checkpoints — Each tier/module logs completion
4. No silent failures — Every error appears in logs with context

Recovery on Failure

If a module fails:

# Check what completed
cat /var/log/clawdie-firstboot.progress

# Resume from last checkpoint
clawdie-firstboot --resume

# Or start over
clawdie-firstboot --reset

The --resume flag parses progress file and skips completed tiers.

Logging

All errors logged to /var/log/clawdie-firstboot.log:

[2026-03-23 15:34:12] [pkg] Starting package setup
[2026-03-23 15:34:15] [pkg] Detected ABI: FreeBSD:15:amd64
[2026-03-23 15:34:18] [pkg] ERROR at line 42: repo config failed
[2026-03-23 15:34:18] ERROR in clawdie_shell_pkg_setup at line 42

User can:

1. Read logs: tail -50 /var/log/clawdie-firstboot.log
2. SSH in and debug
3. Resume: clawdie-firstboot --resume

---

Testing Individual Modules

After install, operator can test/rerun modules:

# Source the module library
. /usr/local/libexec/clawdie-shell-env.sh

# Test a function
clawdie_shell_env_validate

# Or rerun a setup phase
clawdie_shell_pkg_setup
clawdie_shell_gpu_detect

Use case: Troubleshooting without full reinstall.

---

Design Patterns

1. Idempotence

Functions check state before modifying:

clawdie_shell_system_write_rcconf() {
  # Check if already done
  if grep -q "^timezone=" /etc/rc.conf; then
    return 0  # Already set, skip
  fi

  # Write new config
  sysrc timezone="$TZ"
}

2. Logging

Every function logs its progress:

clawdie_shell_pkg_setup() {
  LOG="/var/log/clawdie-firstboot.log"
  echo "[pkg] Starting package setup" >> $LOG
  echo "[pkg] Detected ABI: $ABI" >> $LOG
  echo "[pkg] Setup complete" >> $LOG
}

3. Configuration Over Code

Variables at top of each module:

# clawdie-shell-pkg.sh
REPO_PRIORITY_ONLINE=0
REPO_PRIORITY_USB=100
REPO_PATH_USB="/mnt/media/packages"

Easier to customize than editing function logic.

4. Exit Codes

Consistent semantics:

clawdie_shell_validate() {
  [[ -f /home/clawdie/.env ]] && return 0  # Success
  return 1  # Failure
}

if clawdie_shell_validate; then
  echo "Environment ready"
else
  echo "Error: .env not found"
  exit 1
fi

---

Comparison: Shell vs. Python vs. Node

Aspect	Shell	Python	Node.js
Startup time	5 ms	100 ms	300 ms
Footprint	1 file	~50 MB	~100 MB
External deps	None	venv, pip	npm modules
Portability	POSIX (40+ years)	Python 3.8+	Node 18+
Error handling	set -e, trap	try/except	try/catch
Testing	bats, bash_unit	pytest, unittest	jest, mocha
First-boot overhead	Negligible	100+ ms	300+ ms

For a one-time installer, shell wins on speed and simplicity.

---

Future Extensibility

To add a new setup phase (e.g., clawdie-shell-ollama.sh for local AI):

1. Create module: /usr/local/libexec/clawdie-shell-ollama.sh
2. Define function: clawdie_shell_ollama_setup()
3. Update firstboot.sh:

   . /usr/local/libexec/clawdie-shell-ollama.sh
   clawdie_shell_ollama_setup  # Call after clawdie setup
   

4. Document in this file

No framework changes needed. Just add a new module.

---

Debugging Guide

Check logs

tail -100 /var/log/clawdie-firstboot.log

Test a module manually

. /usr/local/libexec/clawdie-shell-env.sh
export ASSISTANT_NAME="Clawdie"
export AGENT_DOMAIN="clawdie.local"
clawdie_shell_env_generate  # Run one function

Verify module output

# After env module:
grep ASSISTANT_NAME /home/clawdie/clawdie-ai/.env

# After pkg module:
cat /etc/pkg/repos/Clawdie-USB.conf

# After gpu module:
sysctl hw.pci.dump | grep -i vga
grep kld_list /etc/rc.conf

# After clawdie module:
jls -N  # List jails
service clawdie status

---

POSIX Compliance: FreeBSD-First

Mandate: No bash-isms, no Linux-isms. Pure POSIX shell for maximum portability.

Why POSIX Over Bash

Aspect	POSIX sh	Bash
Default on FreeBSD	✅ /bin/sh	❌ Requires pkg install bash
First-boot availability	✅ Always present	❌ Needs to be installed first
Dependencies	None	libreadline, ncurses, etc.
Startup overhead	~5 ms	~50 ms
Portability	40+ years, all Unix systems	Linux-centric, optional on BSD

Decision: Use POSIX sh exclusively. Our scripts run on stock FreeBSD without extra packages.

POSIX-Safe Error Handling

❌ Bash-only (not allowed):

set -eo pipefail  # pipefail is bash-only
BASH_COMMAND      # bash-only variable

✅ POSIX-safe (required):

set -eu           # POSIX: exit on error, undefined vars
trap '...' ERR    # POSIX: error trap

# For debugging:
PS4='+ ${0##*/}: line $LINENO: '  # POSIX way to show line numbers

Avoiding Linux-Specific Patterns

❌ systemd (not on FreeBSD):

systemctl start service      # ❌ No systemd

✅ FreeBSD rc.d:

service clawdie start        # ✅ FreeBSD standard

❌ Linux package names:

apt-get install package      # ❌ Debian-only
pkg install package-devel    # ✅ FreeBSD pkg

✅ FreeBSD packages:

pkg install libsndfile       # ✅ FreeBSD ports

❌ Linux device paths:

/dev/sda1                     # ❌ Linux naming
/dev/da0s1                    # ✅ FreeBSD naming

---

Philosophy: "Shells Done Right"

Shell scripting has a bad reputation because it's often abused:

• Unquoted variables → word splitting bugs
• No error checking → silent failures
• Global state → hard to test

Clawdie Shell enforces best practices:

#!/bin/sh
set -e  # Exit on error
set -u  # Exit on undefined vars

# Use functions, not global scope
clawdie_shell_main() {
  local assistant_name="$1"  # Local variables
  local agent_domain="$2"

  # Quote variables, use explicit paths
  /bin/echo "Assistant: $assistant_name"
}

clawdie_shell_main "Clawdie" "clawdie.local"

Result: Shell that's as reliable and testable as Python/Node, but 50× faster.

---

References

• POSIX Shell Reference
• FreeBSD rc.d Services
• bsddialog Manual
• Bastille Jails Documentation

---

Last updated: 23.mar.2026 Maintained by: Clawdie Project