clawdie-ai/docs/internal/DOCUMENTATION.md

Documentation System

Last Updated: 16.apr.2026

---

Part 1: Policy & Sources

Canonical Sources

Markdown is the single source of truth. All documentation starts as .md files in docs/public/ (public) or docs/internal/ (internal).

• Never edit HTML directly — it is generated, not authored
• All docs must be markdown — easier to version, review, and sync
• HTML is a build artifact — generated via Pandoc, then deployed to webroots

Deployment Targets

Target	URL	Source	Schedule
docs.clawdie.si	https://docs.clawdie.si	docs/public/ (subset)	Daily @ 05:00 UTC
osa.smilepowered.org	https://osa.smilepowered.org	docs/public/OSA*.md	Daily @ 05:00 UTC
clawdie.si	https://clawdie.si	Manual HTML	On release

Content Classification

Public (auto-synced to webroots):

• docs/public/ — Operator guides (install, architecture, operate, reference, roadmap, localization)

Internal (excluded by .docignore):

• docs/internal/ — Technical internals, session logs, plans (not published)
• .env.example — Secrets template (not published)

Editing Workflow

Adding a new public document:

1. Create docs/public/topic.md with front-matter
2. Next cron run (05:00 UTC) syncs automatically
3. Or run just docs-sync immediately

Adding a new internal document:

1. Create docs/internal/TOPIC.md
2. Stays in git, NOT synced to public webroots

Excluding from sync: Add pattern to docs/public/.docignore

Version Control

Commit: .md files, .docignore, .sync-metadata.json, sync scripts, nginx configs Don't commit: Generated HTML, symlinks, log files, lock files

---

Part 2: Sync Operations

Architecture

docs/public/*.md → docs-compile.sh → versioned HTML dir → symlink swap → nginx

Components:

1. Markdown Source (docs/public/) — single source of truth, version controlled
2. Compilation Pipeline (scripts/docs-compile.sh) — markdown → versioned HTML
3. Sync Orchestrator (scripts/docs-sync.cron.sh) — pull, compile, validate, deploy, cleanup
4. Symlink Deployment — nginx points to docs-current, atomic swap on update
5. Metadata (.sync-metadata.json) — tracks last sync, commit, targets

Quick Reference

Task	Command
Check sync status	tail -f /var/log/clawdie-docs-sync.log
View current version	ls -l /usr/local/www/docs.clawdie.si/docs-current
Manual compile	scripts/docs-compile.sh --semver 0.10.0 /usr/local/www/docs.clawdie.si/
Trigger immediate sync	sudo scripts/docs-sync.cron.sh
Rollback to previous	sudo ln -sfn docs-v0.9.0_YYYYMMDD /usr/local/www/docs.clawdie.si/docs-current && sudo service nginx reload

Daily Cron

Runs automatically at 05:00 UTC:

05:00 UTC → git pull → compile markdown → validate → symlink swap → nginx reload → cleanup old versions

Log: /var/log/clawdie-docs-sync.log

Cron entry:

0 5 * * * root ${PROJECT_ROOT}/scripts/docs-sync.cron.sh >> /var/log/${AGENT_NAME}-docs-sync.log 2>&1

Monitoring

# Watch sync in progress
tail -f /var/log/clawdie-docs-sync.log

# Disk usage (3 versions ~60MB)
du -sh /usr/local/www/docs.clawdie.si/

# Check metadata staleness
jq '.last_sync' docs/public/.sync-metadata.json

Troubleshooting

Sync failed: Check log with grep -i "error\|failed" /var/log/clawdie-docs-sync.log. Test compile manually with scripts/docs-compile.sh.

Symlink broken: Find working version with ls -d /usr/local/www/docs.clawdie.si/docs-v*, repoint symlink, reload nginx.

Nginx 404 after swap: Check sudo nginx -t, verify permissions (chmod -R a+rx), reload nginx.

Cron not running: Verify with sudo crontab -l | grep docs-sync. Check system cron logs.

Git pull failing: Check for merge conflicts in docs/public/. Resolve and retry.

Instant Rollback

ls -lt /usr/local/www/docs.clawdie.si/docs-v* | head -5   # find previous version
sudo ln -sfn docs-v0.9.0_YYYYMMDD /usr/local/www/docs.clawdie.si/docs-current
sudo service nginx reload

Disk Management

• Retention: Last 30 days
• Size: ~20MB per version (~60MB for 3 versions)
• Cleanup: Automatic at end of daily cron

Manual cleanup:

find /usr/local/www/docs.clawdie.si -maxdepth 1 -name "docs-v*" -type d -mtime +30 -exec rm -rf {} \;

Disaster Recovery

Recompile from git:

cd /home/clawdie/clawdie-ai && git pull origin main
scripts/docs-compile.sh --semver 0.10.0 /usr/local/www/docs.clawdie.si/
sudo ln -sfn docs-v0.10.0_YYYYMMDD /usr/local/www/docs.clawdie.si/docs-current
sudo service nginx reload

Pre-Commit Hook

hooks/pre-commit (tracked, activated via just install-hooks) prevents committing HTML to docs/public/:

if git diff --cached --name-only | grep '^docs/public/.*\.html$'; then
  echo "ERROR: HTML files must not be committed to docs/public/"
  exit 1
fi