clawdie-ai/.agent/skills/docs-deployment/INTEGRATION.md
Clawdie AI 84cc452497 docs: complete Codex's public/internal restructure alignment
Codex restructured docs on 5 Apr (cc37d2c) into public/ and internal/
subdirectories. This commit completes the alignment work by updating all
references across skills, agents, and generated files.

Changes:

Skills Documentation:
- docs-deployment/SKILL.md: Update doc paths (docs/INSTALL.md → docs/public/install/install.md)
- docs-deployment/INTEGRATION.md: Update example paths for new structure
- agent-setup/SKILL.md: Update references
- ansible-freebsd/SKILL.md: Update references

Agents & Conventions:
- AGENTS.md: Add tmux "testing" window guideline for long-running commands
- AGENTS.md: Update doc path references (INSTALL.md → install.md lowercase)
- README-CLAWDIE.md: Update doc paths
- CROWDIN.md: Update structure references

Build System:
- setup/cms.ts: Improve frontmatter generation (skip files with existing h1)
- scripts/memory/embed-docs.py: Adjust for new structure
- docs-deployment/DOCUMENTATION-POLICY.md: Update guidelines

HTML/Web Output:
- html/: Version bumps, link fixes, manifest updated
- Reflect new public/ structure in rendered docs

Validation:
- All skills reference docs correctly
- Install/setup docs now at docs/public/install/
- Docs build and serve correctly (verified 6 Apr)
- 603 tests passing (from Codex's build)

Co-Authored-By: Codex (Agent) <codex@clawdie.si>
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

---
Build: pass | Tests: pass — Tests  603 passed (603)
2026-04-06 13:22:24 +00:00

467 lines
13 KiB
Markdown

# Clawdie-AI: Build to Deployment Integration Guide
**Phase:** 3.0+
**Status:** Cross-repo reference (clawdie-ai + clawdie-shell)
**Created:** 24.mar.2026
---
## Overview
This guide shows how **clawdie-shell** (ISO building) and **clawdie-ai** (deployment & operations) work together to deliver Clawdie-AI as a complete stack.
```
clawdie-shell (Build) clawdie-ai (Deploy)
─────────────────────────────────────────────────────────
build-iso skill → docs-deployment skill
build.sh (orchestrate) → docs-sync.cron.sh
firstboot.sh (provision) → nginx (serve)
shell-*.sh modules → Live system
```
---
## The Journey: Build → Deploy → Serve
### Phase 1: ISO Creation (clawdie-shell repo)
**Where:** `clawdie-shell/`
**Skill:** `build-iso`
**What happens:**
```bash
cd /home/clawdie/clawdie-shell
# Trigger build
npm run build-iso
# What build-iso does:
# 1. Clones clawdie-iso from git jail
# 2. Runs build.sh with cloud/baremetal variants
# 3. Injects config (--target cloud, --domain, --tz, etc.)
# 4. Creates versioned ISO: clawdie-iso-cloud-24.mar.2026.img
# 5. Publishes to https://clawdie.si/downloads/
```
**Output:**
```
clawdie-iso-cloud-24.mar.2026.img (cloud VPS variant)
clawdie-iso-amd-24.mar.2026.img (baremetal AMD GPU)
clawdie-iso-intel-24.mar.2026.img (baremetal Intel GPU)
clawdie-iso-baremetal-24.mar.2026.img (auto-detect)
```
**Baked into ISO:**
- `build.cfg` — TARGET, ASSISTANT_NAME, AGENT_DOMAIN, TZ, SSH_PUBLIC_KEY, etc.
- `scripts/docs-compile.sh` — markdown → HTML compiler
- `scripts/docs-sync.cron.sh` — daily sync orchestrator
- Shell modules — gpu.sh, pkg.sh, env.sh, deploy.sh, etc.
---
### Phase 2: ISO Boot & Firstboot (FreeBSD host)
**Where:** User downloads ISO and boots on target hardware
**Scripts:** `firstboot.sh` + shell modules (in ISO payload)
**What happens:**
```
User boots ISO
bsddialog wizard (baremetal only)
├─ Basic config (name, domain, timezone)
├─ ZFS RAID selection
├─ Backup destination (optional)
└─ Review + confirm
firstboot.sh orchestrator runs:
├─ clawdie-shell-gpu.sh (GPU detection, cloud: skips)
├─ clawdie-shell-nvidia.sh (NVIDIA setup if needed)
├─ clawdie-shell-pkg.sh (Package repo config, offline cache)
├─ clawdie-shell-env.sh (Generate .env with secrets)
├─ clawdie-shell-system.sh (hostname, rc.conf, services)
└─ clawdie-shell-deploy.sh (Extract tarball, npm install, jails)
System boots into FreeBSD with:
✓ Jails configured
✓ Services running
✓ Documentation source in docs/
✓ Deployment scripts ready
✓ API keys in .env (blank, ready for web UI)
```
---
### Phase 3: Host-Level Operations (clawdie-ai repo)
**Where:** `/home/clawdie/clawdie-ai/` (on deployed host)
**Skill:** `docs-deployment`
**What happens:**
#### 3a. Author writes documentation
```bash
cd /home/clawdie/clawdie-ai
# Write English docs
vim docs/public/install/install.md
git add docs/public/install/install.md
git commit -m "docs: Add installation guide"
git push origin main
# Crowdin webhook detects change
# Translators start working
```
#### 3b. Crowdin syncs translations
```
Crowdin auto-fetches (webhook)
Translators work (crowdin.com/project/clawdie-ai)
Translations complete (24-48 hours)
Crowdin auto-commits:
docs/de/INSTALL.md (German)
docs/fr/INSTALL.md (French)
docs/es/INSTALL.md (Spanish)
Git pulls on host
```
#### 3c. Daily sync @ 05:00 UTC (cron)
```bash
# docs-sync.cron.sh runs automatically
1. git pull
✓ Gets latest markdown + translations
2. docs-compile.sh --languages en,de,fr,es
✓ Compiles all languages to versioned dirs:
docs-v0.9.0_24.mar.2026/
├─ en/ (29 HTML files)
├─ de/ (29 HTML files)
├─ fr/ (29 HTML files)
└─ es/ (29 HTML files)
3. Validate output
✓ Check all language indexes exist
✓ Verify HTML structure
4. Atomic symlink swap (zero-downtime)
en-current → docs-v0.9.0_24.mar.2026/en/
de-current → docs-v0.9.0_24.mar.2026/de/
fr-current → docs-v0.9.0_24.mar.2026/fr/
es-current → docs-v0.9.0_24.mar.2026/es/
5. nginx reload
✓ Serves new docs (no downtime)
6. Cleanup
✓ Delete versions older than 30 days
✓ Update metadata
```
#### 3d. Users access documentation
```
Browser: https://docs.clawdie.si/
Language selector (root page)
User picks language: /en/, /de/, /fr/, /es/
Nginx resolves symlink:
en-current → docs-v0.9.0_24.mar.2026/en/
User sees English docs (compiled from markdown)
Zero-downtime updates:
- Author commits docs in clawdie-ai
- Cron syncs automatically @ 05:00 UTC
- Symlink swaps (no 404s, no downtime)
```
---
## Key Integration Points
### 1. ISO Payload Contains Deployment Scripts
**In clawdie-shell build.sh:**
```bash
# Inject deployment scripts into ISO
cp /home/clawdie/clawdie-ai/scripts/docs-compile.sh \
/mnt/media/scripts/
cp /home/clawdie/clawdie-ai/scripts/docs-sync.cron.sh \
/mnt/media/scripts/
```
**In firstboot.sh:**
```bash
# Scripts are available on first boot
/usr/local/share/clawdie-iso/scripts/docs-compile.sh --help
/usr/local/share/clawdie-iso/scripts/docs-sync.cron.sh
```
### 2. build.cfg Bridges Both Repos
**In clawdie-shell (build-time):**
```bash
# build.cfg is baked into ISO
TARGET="cloud" # cloud | baremetal
ASSISTANT_NAME="natasha" # pre-configured
AGENT_DOMAIN="natasha.internal"
TZ="Europe/Ljubljana"
```
**In clawdie-ai (runtime):**
```bash
# scripts/docs-compile.sh and docs-sync.cron.sh read LANGUAGES
# Slovenian is primary (prototype default), others follow
LANGUAGES="sl,en,de,fr,es" # Slovenian primary, then English, German, French, Spanish
# Can be overridden by build.cfg or environment for cloud builds:
# LANGUAGES="sl,en" # Cloud: Slovenian primary + English only
```
### 3. Crowdin ↔ Documentation Sync
**Source of truth:** `clawdie-ai` repo (markdown)
```
docs/public/install/install.md (English) ──→ Crowdin ──→ docs/sl/INSTALL.md (Slovenian primary)
├→ docs/de/INSTALL.md (German)
├→ docs/fr/INSTALL.md (French)
└→ docs/es/INSTALL.md (Spanish)
```
**Deployment:** Both repos pull from clawdie-ai
```
clawdie-shell (build-iso)
├─ References clawdie-ai/docs/ for docs in ISO?
└─ Or: Operators maintain separate docs for ISO releases
clawdie-ai (docs-deployment)
├─ Compiles docs/ to HTML
├─ Deploys daily @ 05:00 UTC
└─ Crowdin keeps languages in sync
```
---
## Workflow Scenarios
### Scenario 1: Release a New ISO (clawdie-shell)
```
Operator in clawdie-shell repo:
1. Update build.cfg with new version
2. Run: npm run build-iso
3. build-iso skill:
├─ Clones clawdie-iso
├─ Runs build.sh
├─ Injects docs scripts (from clawdie-ai)
├─ Creates ISO: clawdie-iso-cloud-24.mar.2026.img
└─ Publishes to https://clawdie.si/downloads/
4. Users download and boot ISO
```
### Scenario 2: Update Documentation (clawdie-ai)
```
Author in clawdie-ai repo:
1. Write English docs: docs/public/install/install.md
2. Commit & push
3. Crowdin detects (webhook)
4. Translators work (24-48 hours)
5. Crowdin commits: docs/de/, docs/fr/, docs/es/
6. Cron @ 05:00 UTC:
├─ git pull (gets translations)
├─ Compile all languages
├─ Symlink swap (zero-downtime)
└─ nginx reload
7. Users see updated docs in all languages
```
### Scenario 3: Add New Language to Docs
```
Process:
1. Admin adds language to Crowdin project
crowdin.com/project/clawdie-ai → Add: Italian (it)
2. Author adds to docs-sync.cron.sh:
LANGUAGES="en,de,fr,es,it"
3. Crowdin auto-syncs translations to docs/it/
4. Next cron run (05:00 UTC):
├─ Compile includes Italian
├─ Create: docs-v0.9.0_24.mar.2026/it/
└─ Symlink: it-current → docs-v0.9.0_24.mar.2026/it/
5. Nginx location block added (manually or via template):
location /it/ { ... }
6. Users access: https://docs.clawdie.si/it/
```
---
## Dependency Chain
```
clawdie-shell (Build)
├─ build-iso skill
├─ build.sh
│ └─ Fetches from clawdie-iso repo
│ └─ Contains: firstboot.sh + shell-*.sh modules
│ └─ Injects into ISO:
│ ├─ build.cfg (TARGET, ASSISTANT_NAME, etc.)
│ ├─ docs-compile.sh (from clawdie-ai)
│ ├─ docs-sync.cron.sh (from clawdie-ai)
│ └─ shell-*.sh modules
└─ Output: clawdie-iso-{cloud,amd,baremetal}-24.mar.2026.img
└─ Users download & boot
└─ firstboot.sh runs shell modules
└─ System boots with deployment scripts ready
clawdie-ai (Deploy)
├─ docs-deployment skill
├─ docs-compile.sh (used in ISO and on host)
├─ docs-sync.cron.sh (runs daily @ 05:00 UTC on host)
├─ Crowdin integration (markdown → translations)
│ └─ docs/ + docs/de/, docs/fr/, docs/es/
└─ Output: Zero-downtime documented system
└─ Users access: https://docs.clawdie.si/{en,de,fr,es}/
```
---
## File Cross-References
### clawdie-shell must know about:
- `clawdie-ai/scripts/docs-compile.sh` — Injected into ISO
- `clawdie-ai/scripts/docs-sync.cron.sh` — Injected into ISO
- `clawdie-ai/docs/` — Source of truth for documentation
- `clawdie-ai/.agent/skills/docs-deployment/` — Integration guide (this file)
### clawdie-ai must know about:
- `clawdie-shell/skills/build-iso/` — How ISOs are created
- `clawdie-shell/build.sh` — What gets baked into ISO
- `clawdie-shell/build.cfg` — Template for cloud/baremetal variants
### Both repos reference:
- `clawdie-iso/` — FreeBSD ISO build system (not in this guide, external)
- `crowdin.com/project/clawdie-ai` — Translation management
---
## Deployment Architectures
### Cloud Deployment (VPS)
```
1. User downloads: clawdie-iso-cloud-24.mar.2026.img
2. Boots in VPS (no GUI, headless)
3. Firstboot runs:
├─ Cloud preset (no GPU detection, no bsddialog)
├─ All values pre-baked (ASSISTANT_NAME, DOMAIN, TZ)
├─ System configures fully automated
└─ Ready for API key injection via web UI
4. Documentation:
└─ docs-sync.cron.sh @ 05:00 UTC
├─ English only (cloud: reduced languages)
└─ Zero-downtime updates
Docs serve: https://docs.natasha.internal/en/
```
### Baremetal Deployment (Physical Hardware)
```
1. User downloads: clawdie-iso-amd-24.mar.2026.img (GPU-specific)
2. Boots on physical hardware (with GUI desktop)
3. Firstboot runs:
├─ Interactive wizard (bsddialog)
├─ User selects: name, domain, timezone, ZFS RAID config
├─ GPU auto-detected (AMD/Intel/NVIDIA)
├─ System configures interactively
└─ Desktop available for API key entry (browser)
4. Documentation:
└─ docs-sync.cron.sh @ 05:00 UTC
├─ All languages (en, de, fr, es)
└─ Zero-downtime updates
Docs serve: https://docs.natasha.internal/en/ (default)
or /de/, /fr/, /es/ (via language selector)
```
---
## Version Timeline
### v0.9.0 Release (Target)
**clawdie-shell:**
- ✅ build-iso skill (publish ISO)
- ✅ Cloud vs baremetal variants
- ✅ SSH key + password configuration
- ✅ ZFS RAID selection
**clawdie-ai:**
- ✅ docs-deployment skill
- ✅ Multi-language (en, de, fr, es)
- ✅ Crowdin integration ready
- ✅ Zero-downtime deployment
- ✅ 30-day version retention
**Together:**
- ✅ Complete ISO → deployment pipeline
- ✅ Build (clawdie-shell) → Deploy (clawdie-ai)
- ✅ Documentation in 4 languages
- ✅ Automated daily sync @ 05:00 UTC
---
## Quick Links
**clawdie-shell:**
- `.agent/skills/build-iso/SKILL.md` — ISO building
- `build.sh` — Build orchestrator
- `build.cfg` — Build configuration
**clawdie-ai:**
- `.agent/skills/docs-deployment/SKILL.md` — This skill ⬅️
- `scripts/docs-compile.sh` — Markdown → HTML
- `scripts/docs-sync.cron.sh` — Daily sync orchestrator
- `CROWDIN.md` — Translation workflow
- `docs/DOCUMENTATION-SYNC-RUNBOOK.md` — Operator manual
**External:**
- Crowdin: https://crowdin.com/project/clawdie-ai
- ISO Downloads: https://clawdie.si/downloads/
- Docs: https://docs.clawdie.si/
---
## Questions & Support
**For ISO building issues:** See `clawdie-shell/.agent/skills/build-iso/SKILL.md`
**For documentation deployment:** See `clawdie-ai/.agent/skills/docs-deployment/SKILL.md`
**For troubleshooting:** See `clawdie-ai/docs/DOCUMENTATION-SYNC-RUNBOOK.md`
**For translation workflow:** See `clawdie-ai/CROWDIN.md`
**For Crowdin setup (admin):** See `clawdie-ai/.agent/skills/docs-deployment/CROWDIN-SETUP.md`
---
**Last Updated:** 24.mar.2026
**Phase:** 3.5 (Crowdin integration — Phase 3.0 foundation complete, ready for v0.9.0 release)