clawdie-ai/.agent/skills/docs-deployment/SKILL.md

name	description
docs-deployment	Manage multi-language documentation deployment on FreeBSD host. Handles Crowdin integration, markdown compilation, symlink-based zero-downtime deployment, and nginx language routing. Use when: setting up docs.clawdie.si, multi-language support, Crowdin integration, documentation deployment, language routing, translation workflow, documentation monitoring.

Documentation Deployment System

Phase: 3.0+ Status: Production-ready (v0.9.0+) Created: 24.mar.2026

---

Overview

This skill covers the host-level documentation deployment system for Clawdie-AI.

Handles multi-language markdown→HTML compilation, Crowdin translation integration, zero-downtime symlink-based deployments, and automated daily syncing @ 05:00 UTC.

Not in scope: CMS jail nginx (see nginx skill for cms jail vhost management)

---

Architecture at a Glance

Source (git)              Compilation           Deployment              Serving
─────────────────────────────────────────────────────────────────────────────────
docs/public/install/install.md ──→ docs-compile.sh ──→ docs-v0.9.0_24.mar.2026/ ──→ nginx
docs/de/INSTALL.md        (28 files)        ├─ en/ (symlink)         /en/
docs/fr/INSTALL.md        (4 langs)         ├─ de/ (symlink)         /de/
docs/es/INSTALL.md                         ├─ fr/ (symlink)         /fr/
                                            └─ es/ (symlink)         /es/
     ↓
Crowdin auto-syncs translations
(crowdin.com/project/clawdie-ai)
     ↓
docs-sync.cron.sh @ 05:00 UTC
├─ git pull
├─ compile (all 4 languages)
├─ atomic symlink swap (zero-downtime)
├─ nginx reload
└─ cleanup (30-day retention)

---

Key Components

1. Markdown Source (Git)

• English source: docs/*.md (you maintain)
• Translations: docs/de/*.md, docs/fr/*.md, docs/es/*.md (Crowdin maintains)
• Single source of truth: English markdown is authoritative
• Git strategy: Markdown committed, HTML never committed

Locations:

/home/clawdie/clawdie-ai/docs/
├─ INSTALL.md (English)
├─ REQUIREMENTS.md (English)
├─ de/
│  ├─ INSTALL.md (German — auto-synced by Crowdin)
│  └─ REQUIREMENTS.md
├─ fr/
│  ├─ INSTALL.md (French)
│  └─ REQUIREMENTS.md
└─ es/
   ├─ INSTALL.md (Spanish)
   └─ REQUIREMENTS.md

2. Compilation Pipeline (docs-compile.sh)

• Input: Markdown files from docs/ (filtered by .docignore)
• Output: Versioned HTML directories docs-v0.9.0_24.mar.2026/{en,de,fr,es}/
• Features:
  • No external dependencies (pure shell, no pandoc/markdown packages needed)
  • Filters internal/sensitive docs via .docignore
  • Auto-generates language-specific indexes
  • Version naming: docs-v{SEMVER}_{DD.mon.YYYY} (user-friendly Slovenian date format)
  • Supports any number of languages

Usage:

# Single language (backward compatible)
scripts/docs-compile.sh --semver 0.9.0 /usr/local/www/docs.clawdie.si/

# Multi-language
scripts/docs-compile.sh --semver 0.9.0 --languages en,de,fr,es \
    /usr/local/www/docs.clawdie.si/

Performance: ~10s for 4 languages, 28 markdown files

3. Sync Orchestrator (docs-sync.cron.sh)

• Runs: Daily @ 05:00 UTC via cron
• Steps:
  1. Git pull (fetch latest markdown + translations from Crowdin)
  2. Compile markdown → HTML (all languages)
  3. Validate output (check all language indexes exist)
  4. Atomic symlink swap (zero-downtime deployment)
  5. Nginx reload (serve new content)
  6. Cleanup old versions (30-day retention)

Execution time: ~15-20s total

4. Symlink-Based Deployment

• Strategy: Atomic swaps for zero-downtime updates

• How it works:

  # Before swap
  en-current → docs-v0.9.0_20260317/en/  (old version)
  
  # Atomic swap (ln -sfn is atomic on modern filesystems)
  ln -sfn docs-v0.9.0_24.mar.2026/en en-current
  
  # After swap
  en-current → docs-v0.9.0_24.mar.2026/en/  (new version)
  
  # Total downtime: < 1 millisecond
  

• Rollback: Also < 1 second (just repoint symlink to previous version)

• Retention: Keep 30 days of versions (~60MB for 3 versions)

5. Crowdin Integration

• Project: https://crowdin.com/project/clawdie-ai

• Workflow:

  1. You write English docs (docs/*.md)
  2. Crowdin auto-fetches via GitHub webhook
  3. Translators work in Crowdin UI (no git knowledge needed)
  4. Crowdin auto-commits translations (docs/de/, docs/fr/, etc.)
  5. Cron pulls translations automatically

• No manual file editing: Translations are managed by Crowdin, not manually

• Translation memory: Crowdin reuses past translations for consistency

• Community: Open to community translators

6. Nginx Configuration

• Vhost: /usr/local/etc/nginx/vhosts/docs.clawdie.si.conf

• Language routing:

  location /en/ { try_files $uri /en/index.html; }  # English
  location /de/ { try_files $uri /de/index.html; }  # German
  location /fr/ { try_files $uri /fr/index.html; }  # French
  location /es/ { try_files $uri /es/index.html; }  # Spanish
  location = / { return 301 /en/; }                 # Default to English
  

• Symlink following: Nginx follows symlinks (en-current, de-current, etc.)

• Language selector: Root URL (docs.clawdie.si/) shows language picker

---

Deployment Checklist

# 1. Crowdin Setup (manual, 15 min)
[ ] Create account: https://crowdin.com
[ ] Create project: "Clawdie-AI"
[ ] Connect GitHub (OAuth)
[ ] Add languages: de, fr, es
[ ] Configure file mappings:
    Source: /docs/*.md
    German: /docs/de/%filename%
    French: /docs/fr/%filename%
    Spanish: /docs/es/%filename%

# 2. Repository Structure (10 min)
[ ] mkdir -p docs/{de,fr,es}
[ ] Add to .gitignore (language dirs auto-synced by Crowdin)
[ ] Create CROWDIN.md (workflow guide)
[ ] Update .docignore (filter language dirs from English build)

# 3. Code Modifications (3 hours with testing)
[ ] Modify docs-compile.sh (add --languages flag)
[ ] Modify docs-sync.cron.sh (language-aware sync)
[ ] Test: sh docs-compile.sh --languages en,de,fr tmp/test
[ ] Verify: ls -la tmp/test/docs-v*/en/ /de/ /fr/

# 4. Nginx Configuration (20 min)
[ ] Update docs.clawdie.si.conf (language locations)
[ ] Create language selector HTML (root page)
[ ] Test: sudo nginx -t && sudo service nginx reload
[ ] Verify: curl https://docs.clawdie.si/{en,de,fr}

# 5. Testing (1 hour)
[ ] Manual compile test
[ ] Nginx routing test
[ ] Crowdin integration test (commit test file, wait for sync)
[ ] Symlink swap test
[ ] Rollback test

# 6. Git Commits (10 min)
[ ] 6 commits (structure, compile.sh, sync.sh, nginx, docs, metadata)
[ ] Push to origin

---

Quick Reference

Task	Command	Time
Check sync status	tail -f /var/log/clawdie-docs-sync.log	Real-time
View current version	ls -l /usr/local/www/docs.clawdie.si/en-current	Instant
Manual compile	scripts/docs-compile.sh --languages en,de,fr /usr/local/www/docs.clawdie.si/	~10s
Trigger immediate sync	sudo scripts/docs-sync.cron.sh	~15-20s
Instant rollback	ln -sfn docs-v0.9.0_20260317 /usr/local/www/docs.clawdie.si/en-current && sudo service nginx reload	<1s
List versions	ls -lt /usr/local/www/docs.clawdie.si/docs-v* | head -3	Instant

---

Common Tasks

Publishing New Documentation

cd /home/clawdie/clawdie-ai

# Write English doc
vim docs/MY-FEATURE.md
git add docs/MY-FEATURE.md
git commit -m "docs: Add MY-FEATURE guide"
git push origin main

# Crowdin detects change (webhook)
# Translators translate (within 24 hours)
# Cron syncs automatically (05:00 UTC next day)
# Users see docs in all languages (zero-downtime)

Excluding Docs from Public Sites

Files matching .docignore patterns are NOT deployed:

# Current filters (docs/.docignore)
cat /home/clawdie/clawdie-ai/docs/.docignore

# Examples:
#   POSTGRES-MEMORY.md (internal notes)
#   *-INTERNAL.md (internal docs)
#   */private/* (private directories)

To exclude new content:

echo "*-DRAFT.md" >> /home/clawdie/clawdie-ai/docs/.docignore

# File will be in git but NOT synced to public sites

Instant Rollback

If new version is broken:

# Find previous working version
ls -lt /usr/local/www/docs.clawdie.si/docs-v* | head -3

# Revert to previous
sudo ln -sfn docs-v0.9.0_20260317 /usr/local/www/docs.clawdie.si/en-current
sudo ln -sfn docs-v0.9.0_20260317 /usr/local/www/docs.clawdie.si/de-current
# ... (for all languages)

# Reload nginx
sudo service nginx reload

# Total time: < 1 second

Monitor Sync Health

# Last sync timestamp
jq '.last_sync' /home/clawdie/clawdie-ai/docs/.sync-metadata.json

# Current versions deployed
ls -l /usr/local/www/docs.clawdie.si/{en,de,fr,es}-current

# Sync logs (watch real-time)
tail -f /var/log/clawdie-docs-sync.log

# Check for broken symlinks
find /usr/local/www/docs.clawdie.si -maxdepth 1 -type l ! -exec test -d {} \; -print
# Should be empty

---

Troubleshooting

Symlink Broken (Points to Missing Directory)

# Check
ls -l /usr/local/www/docs.clawdie.si/en-current
# en-current -> docs-v0.9.0_24.mar.2026 (BROKEN LINK)

# Fix: Find working version
ls -d /usr/local/www/docs.clawdie.si/docs-v*

# Repoint
sudo ln -sfn docs-v0.9.0_20260317 /usr/local/www/docs.clawdie.si/en-current
sudo service nginx reload

Nginx 404 After Symlink Swap

# Check nginx config
sudo nginx -t

# Check permissions (nginx needs read+execute)
stat /usr/local/www/docs.clawdie.si/docs-v0.9.0_24.mar.2026
# Should show: (0755) drwxr-xr-x

# Fix if needed
sudo chmod -R a+rx /usr/local/www/docs.clawdie.si/docs-v*/

# Reload
sudo service nginx reload

Cron Sync Not Running

# Check cron job
sudo crontab -l | grep docs-sync
# Should show: 0 5 * * * root /home/clawdie/clawdie-ai/scripts/docs-sync.cron.sh

# Check system logs
sudo grep docs-sync /var/log/cron | tail -10

# If missing, reinstall:
sudo crontab -e
# Add: 0 5 * * * root /home/clawdie/clawdie-ai/scripts/docs-sync.cron.sh >> /var/log/clawdie-docs-sync.log 2>&1

Git Pull Failing in Sync

# Check git status
cd /home/clawdie/clawdie-ai
git status

# Check for merge conflicts
git diff HEAD origin/implementation -- docs/

# Resolve manually
git fetch origin
git merge origin/implementation
# Fix conflicts, commit, push

# Retry sync
sudo /home/clawdie/clawdie-ai/scripts/docs-sync.cron.sh

---

Performance & Capacity

Metric	Value	Notes
Compile time	~10s	4 languages, 28 markdown files
Sync time	~15-20s	git pull + compile + deploy + cleanup
Symlink swap	<1ms	Atomic operation
Rollback time	<1s	Repoint symlink + reload nginx
Version size	~20MB each	Per language version
3-version storage	~60MB	30-day retention
Daily bandwidth	~100-200MB	git pull + sync logs

---

Integration Points

Related files in repo:

• scripts/docs-compile.sh — Compilation implementation
• scripts/docs-sync.cron.sh — Sync orchestrator
• docs/DOCUMENTATION-POLICY.md — Policy and governance
• docs/DOCUMENTATION-SYNC-RUNBOOK.md — Operator manual (detailed)
• docs/.docignore — Filter rules
• docs/.sync-metadata.json — Deployment metadata
• CROWDIN.md — Translation workflow

External services:

• Crowdin (https://crowdin.com/project/clawdie-ai) — Translation management
• GitHub (oauth for Crowdin) — Source control

Host files:

• /usr/local/etc/nginx/vhosts/docs.clawdie.si.conf — Nginx config
• /usr/local/www/docs.clawdie.si/ — Webroot
• /var/log/clawdie-docs-sync.log — Sync logs
• Cron job (root crontab) — Daily trigger @ 05:00 UTC

---

See Also

• DOCUMENTATION-SYNC-RUNBOOK.md — Detailed operator manual with all troubleshooting steps
• CROWDIN.md — Translation workflow guide for authors and translators
• html/docs-clawdie-si/VERSIONING.md — Deep dive into symlink-based versioning architecture
• nginx skill — Handles cms jail nginx (not host-level docs)

---

Last Updated: 24.mar.2026 Skill Version: 1.0 Phase: 3.0+ (Production-ready)