clawdie-ai/docs/internal/DOCUMENTATION-POLICY.md

Documentation Policy & Sync Strategy

Version: Phase 3.0 Effective: 24.mar.2026 Last Updated: 24.mar.2026

---

Canonical Sources

Markdown is the single source of truth. All documentation starts as .md files in /home/clawdie/clawdie-ai/docs/public/ (public) or /home/clawdie/clawdie-ai/docs/internal/ (internal).

• Never edit HTML directly — it is generated, not authored
• All docs must be markdown — easier to version, review, and sync
• HTML is a build artifact — generated via Pandoc, then deployed to webroots

---

Deployment Targets

Target	URL	Source	Schedule
docs.clawdie.si	https://docs.clawdie.si	docs/public/ (subset)	Daily @ 05:00 UTC
osa.smilepowered.org	https://osa.smilepowered.org	docs/public/OSA*.md	Daily @ 05:00 UTC
clawdie.si	https://clawdie.si	Manual HTML	On release

---

Content Classification

Public (Auto-synced to webroots)

Documents intended for external users, contributors, operators:

• install.md — Installation guide
• spec.md — System specification
• sdk-deep-dive.md — SDK documentation
• OSA-PLATFORM.md — OSA project overview
• OSA-GOVERNANCE.md — OSA governance model
• OSA-ROADMAP.md — OSA roadmap
• CONTRIBUTING.md — Contribution guidelines
• LICENSE.md — License information
• DEPLOYMENT.md — Deployment procedures
• Anything in docs/public/ — Operator guides

Internal Only (Excluded by .docignore)

Documents for maintainers, team, not for public consumption:

• *-INTERNAL.md — Any file with INTERNAL in name
• DEBUG_CHECKLIST.md — Debugging procedures (internal)
• POSTGRES-MEMORY.md — PostgreSQL internals (technical depth)
• SECURITY.md — Security notes (sensitive)
• Anything in docs/internal/ — Internal docs (not published)
• .env.example — Secrets template (not published)

---

Sync Process

Automated Daily (Cron @ 05:00 UTC)

1. Pull latest from origin/main
2. Compile markdown → HTML via Pandoc
3. Validate HTML output, nginx config
4. Deploy via rsync to webroot targets
5. Reload nginx
6. Cleanup versions older than 30 days
7. Log all actions to /var/log/clawdie-docs-sync.log

Manual Sync (On-Demand)

npm run docs:sync      # Full sync (fetch, compile, deploy)
npm run docs:compile  # Compile only (no deployment)
npm run docs:validate # Validate HTML

Metadata Tracking

File: .sync-metadata.json

{
  "canonical_path": "/home/clawdie/clawdie-ai/docs/public",
  "last_sync": "2026-03-24T05:00:00Z",
  "last_commit": "abc1234",
  "targets": [
    {
      "name": "docs-clawdie-si",
      "path": "/usr/local/www/docs.clawdie.si",
      "type": "static-html"
    },
    {
      "name": "osa-smilepowered.org",
      "path": "/usr/local/www/osa.smilepowered.org",
      "type": "static-html"
    }
  ]
}

Updated after every sync (automated).

---

Editing Workflow

Adding a New Public Document

1. Create docs/public/topic.md
2. Add front-matter:

   ---
   title: My Topic
   visibility: public
   last_updated: 2026-03-24
   ---
   
   # My Topic
   ...
   

3. Next cron run (05:00 UTC) syncs automatically
4. Or run npm run docs:sync immediately

Adding a New Internal Document

1. Create docs/internal/TOPIC.md
2. Add front-matter:

   ---
   title: My Internal Topic
   visibility: internal
   last_updated: 2026-03-24
   ---
   

3. Stays in git, NOT synced to public webroots
4. .docignore filters extra exclusions automatically

Editing Existing Documents

• Edit the .md file in docs/public/ or docs/internal/
• Commit to git
• Next cron run syncs changes
• Or run npm run docs:sync for immediate deployment

Excluding a Document from Sync

Add pattern to docs/public/.docignore:

TEMP-DRAFT.md
*/private/
*-SENSITIVE.md

Next sync removes it from deployed sites if it was previously synced.

---

Version Control Rules

DO Commit

• ✓ .md files in docs/public/ and docs/internal/
• ✓ docs/public/.docignore (filtering rules)
• ✓ docs/public/.sync-metadata.json (timestamps, safe)
• ✓ scripts/docs-*.sh (sync scripts)
• ✓ DOCUMENTATION-POLICY.md (this file)
• ✓ nginx vhost configs (html/*/nginx/)
• ✓ .git/hooks/pre-commit (git hooks)

DO NOT Commit

• ✗ Generated HTML (html/docs-clawdie-si/docs-v*/)
• ✗ Generated HTML (html/osa/en-v*/, de-v*/, etc.)
• ✗ Symlinks (docs-current, en-current)
• ✗ Log files (/var/log/clawdie-docs-sync.log)
• ✗ Lock files (tmp/clawdie-docs-sync.lock)

Why? HTML is generated from markdown (deterministic). Committing HTML:

• Inflates repo size
• Creates merge conflicts
• Defeats version control benefits

---

Git Hooks & Safety

Pre-Commit Hook

File: .git/hooks/pre-commit

Prevents:

1. Committing HTML directly to docs/public/
2. Committing symlinks to versioned directories
3. Accidentally committing generated output

Triggers: git commit (automatic)

What it does:

if git diff --cached --name-only | grep '^docs/public/.*\.html$'; then
  echo "ERROR: HTML files must not be committed to docs/public/"
  echo "Run: npm run docs:sync instead"
  exit 1
fi

---

Sync Failure & Rollback

If Sync Fails

1. Check log:

   tail -100 /var/log/clawdie-docs-sync.log
   

2. Common issues:

   • Git fetch failed (network issue)
   • Pandoc error (markdown syntax)
   • nginx config invalid
   • rsync permission denied

3. Symlink remains unchanged (old version still serving)

4. Users see NO errors (old content continues)

5. Admin can investigate at leisure

Instant Rollback (If New Version Broken)

# Find previous working version
ls -lt /home/clawdie/clawdie-ai/html/docs-clawdie-si/docs-v* | head -3

# Repoint symlink
ln -sfn docs-v0.8.1_20260316 /home/clawdie/clawdie-ai/html/docs-clawdie-si/docs-current

# Reload nginx
sudo service nginx reload

# Site now serving OLD version (< 1 second)

---

Monitoring & Status

Check Sync Status

# Last sync timestamp
jq .last_sync /home/clawdie/clawdie-ai/docs/public/.sync-metadata.json

# View recent log
tail -50 /var/log/clawdie-docs-sync.log

# List all versions (with dates)
ls -lt /home/clawdie/clawdie-ai/html/docs-clawdie-si/docs-v* | head -10

# Check current symlink
ls -l /home/clawdie/clawdie-ai/html/docs-clawdie-si/docs-current

Monitor Cron Execution

# See cron job runs
grep docs-sync /var/log/cron

# Check for errors
grep ERROR /var/log/clawdie-docs-sync.log

---

Future Enhancements

Phase 3.5: Crowdin Integration

• Multi-language translations via Crowdin API
• Automated daily pull of translated markdown
• Per-language version directories

Phase 4: Astro/CMS Migration

• Move docs/public/ to CMS jail (Astro project)
• Deployment from inside jail
• Markdown becomes collaborative CMS content

---

Questions & Support

• How do I add a doc? → See "Editing Workflow" section
• Why is my doc not syncing? → Check .docignore patterns
• How do I rollback? → See "Sync Failure & Rollback" section
• When does sync run? → Daily @ 05:00 UTC (check cron log)
• Can I sync manually? → Yes: npm run docs:sync

---

This policy is the source of truth for documentation sync. Questions? Check the runbook: docs/internal/DOCUMENTATION-SYNC-RUNBOOK.md (Phase 3.4)