diff --git a/.agent/skills/docs-localization-pipeline/README.md b/.agent/skills/docs-localization-pipeline/README.md new file mode 100644 index 0000000..d793f85 --- /dev/null +++ b/.agent/skills/docs-localization-pipeline/README.md @@ -0,0 +1,303 @@ +# Docs Localization Pipeline Skill + +**Codifies the three-bird documentation architecture for Pi/Aider automation.** + +## Overview + +This skill automates the complete documentation localization and deployment pipeline: + +``` +English Docs (docs/public/) + ↓ +Crowdin (Project 883714) + ↓ [Push source, Pull translations] +Translated Docs (docs/public/{lang}/) + ↓ +Astro Starlight Build + ↓ [npm run build] +Static Site (dist/) + ↓ +Deploy to nginx + ↓ +Public: https://docs.clawdie.si +``` + +## Quick Start + +### 1. Initial Setup (First Time Only) + +```bash +# Setup Astro Starlight project +./.agent/skills/docs-localization-pipeline/setup-astro-docs.sh + +# Verify setup +ls -la /home/clawdie/astro-docs/ +cd /home/clawdie/astro-docs && npm run dev +# Browse: http://localhost:4321 +``` + +### 2. Manual Pipeline Execution + +```bash +# Full pipeline (all stages) +./.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh --all + +# Individual stages +./.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh --push # Push sources to Crowdin +./.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh --pull # Pull translations +./.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh --build # Pull + Build +./.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh --deploy # Deploy built site +./.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh --status # Check Crowdin status +``` + +### 3. Pi/Aider Commands + +```bash +# Via Pi TUI +pi "sync and rebuild documentation site" +pi "publish docs translations" +pi "check documentation translation progress" + +# Via Aider +aider --auto-commits --message "Sync translations and rebuild docs" docs/ +``` + +## File Structure + +``` +.agent/skills/docs-localization-pipeline/ +├── SKILL.md # Architecture & detailed reference +├── README.md # This file +├── setup-astro-docs.sh # Bootstrap Astro Starlight project +└── orchestrate-pipeline.sh # Orchestrate full Crowdin→Astro→Deploy pipeline +``` + +## Scripts + +### setup-astro-docs.sh + +**Purpose:** Bootstrap the Astro Starlight documentation site. + +**Usage:** +```bash +./setup-astro-docs.sh # Default: /home/clawdie/astro-docs +./setup-astro-docs.sh /custom/path # Custom path +``` + +**What it does:** +1. Creates Astro project structure +2. Installs dependencies (npm install) +3. Generates `astro.config.mjs` with multi-language configuration +4. Sets up content collections config +5. Creates symlinks to `docs/public/{lang}/` for translations +6. Creates i18n UI translation JSON files + +**Prerequisites:** +- Node.js >=20 +- npm +- Docs source at `/home/clawdie/clawdie-ai/docs/public/` + +**Output:** +``` +/home/clawdie/astro-docs/ +├── package.json +├── astro.config.mjs +├── tsconfig.json +├── src/ +│ ├── content/config.ts +│ ├── docs/ +│ │ ├── en → /home/clawdie/clawdie-ai/docs/public/en +│ │ ├── de → /home/clawdie/clawdie-ai/docs/public/de +│ │ └── [other languages] +│ └── i18n/ +│ ├── en.json +│ ├── de.json +│ └── sl.json +└── dist/ (after npm run build) +``` + +### orchestrate-pipeline.sh + +**Purpose:** Automate the full Crowdin→Astro→Deploy pipeline. + +**Usage:** +```bash +./orchestrate-pipeline.sh --all # Full pipeline +./orchestrate-pipeline.sh --push # Stage 1: Push English sources +./orchestrate-pipeline.sh --pull # Stage 2: Pull translations +./orchestrate-pipeline.sh --build # Stages 2+3: Pull + Build +./orchestrate-pipeline.sh --build-only # Stage 3: Build only +./orchestrate-pipeline.sh --deploy-only # Stage 4: Deploy +./orchestrate-pipeline.sh --status # Check Crowdin status +./orchestrate-pipeline.sh --help # Show help +``` + +**Stages:** + +| Stage | Command | What It Does | +|-------|---------|------| +| 1 | `--push` | Upload English `docs/public/*.md` to Crowdin | +| 2 | `--pull` | Download translations into `docs/public/{lang}/` | +| 3 | `--build` | Run `npm run build` to generate static site | +| 4 | `--deploy` | Copy `dist/` to nginx webroot | + +**Environment Variables:** + +```bash +# Set in .env or export before running +CROWDIN_PERSONAL_TOKEN=tskey-xxx # Crowdin API token (required for push/pull) +ASTRO_SITE_PATH=/home/clawdie/astro-docs # Astro project path +DOCS_DEPLOY_TARGET=/usr/local/www/clawdie/docs # nginx webroot +DOCS_MIN_COMPLETION=80 # Min % translation completion before deploy +``` + +**Output Example:** +``` +=== Stage 1: Push English Sources to Crowdin === +[INFO] Uploading English sources to Crowdin... + [1] Uploading: install/index.md + [2] Uploading: install/requirements.md + ... +[INFO] Upload complete: 42 files processed, 0 failed + +=== Stage 2: Pull Translations from Crowdin === +[INFO] Downloading translations from Crowdin... +[INFO] Translation completion: 85% ✓ +Language: sl + Downloaded: /tmp/crowdin-sl-1712951234.zip + ✓ Extracted 42 files to sl/ + +=== Stage 3: Build Astro Site === +[INFO] Building Astro site... +✓ Site built: 217 files + +=== Stage 4: Deploy to Nginx === +[INFO] Copying files to /usr/local/www/clawdie/docs +✓ Nginx reloaded +✓ Deploy complete: https://docs.clawdie.si +``` + +## Configuration + +### .env Setup + +```bash +# Crowdin API token (https://crowdin.com/settings/api/tokens) +CROWDIN_PERSONAL_TOKEN=tskey-your-token-here + +# Astro project path (optional, defaults to /home/clawdie/astro-docs) +ASTRO_SITE_PATH=/home/clawdie/astro-docs + +# nginx deployment target (optional, defaults to /usr/local/www/clawdie/docs) +DOCS_DEPLOY_TARGET=/usr/local/www/clawdie/docs + +# Minimum translation completion % before deploying (default: 80) +DOCS_MIN_COMPLETION=80 +``` + +### Crowdin Project + +- **Project ID:** 883714 +- **Configuration:** `.crowdin.yml` (repo root) +- **Languages:** SL (default), EN, DE, HR, SR, RU, EL, IT, MK, SK, BS +- **Source:** `docs/public/**.md` +- **Output:** `docs/public/{lang}/` + +## Workflows + +### Weekly Sync (Recommended) + +```bash +# Add to crontab (runs Monday 9am) +0 9 * * MON /path/to/.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh --pull && \ + cd /path/to && \ + /path/to/.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh --build-only && \ + /path/to/.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh --deploy-only +``` + +### On-Demand via Pi + +```bash +# Pi will interpret natural language and call orchestrate-pipeline.sh +pi "sync docs translations and rebuild site" +# → Executes: orchestrate-pipeline.sh --all +``` + +### GitHub Actions (Optional) + +See `.github/workflows/crowdin-sync.yml` for automated CI/CD setup. + +## Troubleshooting + +### "Crowdin token not found" + +**Fix:** +```bash +# Option A: Create token file +mkdir -p ~/.config/clawdie +echo "tskey-your-token" > ~/.config/clawdie/crowdin-token +chmod 600 ~/.config/clawdie/crowdin-token + +# Option B: Set env var +export CROWDIN_PERSONAL_TOKEN=tskey-your-token + +# Option C: Add to .env +echo 'CROWDIN_PERSONAL_TOKEN=tskey-your-token' >> .env +``` + +### "Astro project not found" + +**Fix:** +```bash +# Run setup script +./.agent/skills/docs-localization-pipeline/setup-astro-docs.sh +``` + +### "npm: command not found" + +**Fix:** Install Node.js and npm for FreeBSD: +```bash +sudo pkg install node24 npm-node24 +``` + +### "Translation completion too low" + +**Fix:** Adjust `DOCS_MIN_COMPLETION` or wait for translators: +```bash +DOCS_MIN_COMPLETION=50 ./orchestrate-pipeline.sh --all +``` + +### "Astro build fails with missing locale" + +**Fix:** Ensure translations are pulled before build: +```bash +# Manual steps +./scripts/crowdin-sync.sh --pull +cd /home/clawdie/astro-docs && npm run build +``` + +### "Deploy fails: permission denied" + +**Fix:** Ensure sudo access to webroot: +```bash +# Add to sudoers (FreeBSD): +sudo visudo +# Add line: clawdie ALL=(ALL) NOPASSWD: /bin/cp -r, /usr/sbin/service nginx +``` + +## Next Steps + +1. **Crowdin Setup** → Create translators and assign languages +2. **First Push** → `./orchestrate-pipeline.sh --push` to seed English sources +3. **Translation** → Wait for translators to complete milestone (80%+) +4. **First Build** → `./orchestrate-pipeline.sh --build` to test build +5. **Deploy** → `./orchestrate-pipeline.sh --deploy` to go live +6. **Automate** → Setup cron or GitHub Actions for weekly syncs + +## References + +- **Crowdin Project:** https://crowdin.com/project/clawdie-ai +- **Crowdin API:** https://developer.crowdin.com/api/ +- **Astro Docs:** https://docs.astro.build/ +- **Starlight Docs:** https://starlight.astro.build/ +- **Skill Architecture:** `SKILL.md` (in this directory) diff --git a/.agent/skills/docs-localization-pipeline/SKILL.md b/.agent/skills/docs-localization-pipeline/SKILL.md new file mode 100644 index 0000000..d6380f2 --- /dev/null +++ b/.agent/skills/docs-localization-pipeline/SKILL.md @@ -0,0 +1,366 @@ +# Three-Bird Docs Localization Pipeline + +**Status:** MVP Implementation Plan +**Target:** Pi/Aider Automation +**Architecture:** Crowdin → Astro Starlight → Deploy +**Languages:** EN (primary), SLO (default for clawdie.si), DE/SRB/CRO/BIH/RU/ZN/BR (roadmap) + +## Overview + +The **three-bird architecture** separates concerns for documentation, localization, and content delivery: + +1. **Crowdin** — Single source of truth for translations. English docs are pushed, Crowdin maintains translations in 9 target languages. +2. **Astro Starlight** — Static site generator. Builds multilingual docs site from Crowdin translations + English source. +3. **Strapi CMS** (optional) — Content backend for rich media, structured content (deferred due to FreeBSD native binding issues; see `doc/STRAPI-FREEBSD-GOTCHA.md`). + +**MVP Approach:** Markdown-only pipeline (Crowdin → Astro). Ships EN + SLO immediately. Can defer CMS until image upload requirements emerge. + +--- + +## Phase 1: Crowdin → Astro Build Pipeline + +### Architecture + +``` +docs/public/ # English source (Markdown) + ├── install/ + ├── architecture/ + ├── reference/ + └── [other sections] + +Crowdin (Project 883714) # Localization source of truth + ├── Source: docs/public/**.md + ├── Languages: sl, de, hr, sr, ru, el, it, mk, sk, bs + └── Output: docs/public/{lang}/** + +docs/public/{lang}/ # Pulled translations + ├── sl/ # Slovenian + ├── de/ # German + ├── hr/ # Croatian + └── [others...] + +Astro Starlight Site # Multi-language static build + ├── Root (SLO): / + ├── English: /en/ + ├── German: /de/ + ├── Croatian: /hr/ + └── [others...] + +Deployment + └── nginx → docs.clawdie.si +``` + +### Pipeline Steps (Automation-Ready for Pi/Aider) + +#### Step 1: Push English Sources to Crowdin +```bash +./scripts/crowdin-sync.sh --push +``` + +**What it does:** +- Finds all `.md` files in `docs/public/` (excluding translation directories) +- Uploads each file to Crowdin storage +- Updates project files via Crowdin API +- Waits for translators to work on updated strings + +**Automation notes:** +- Run whenever `docs/public/` English files change +- Safe to run multiple times (idempotent) +- Requires `CROWDIN_PERSONAL_TOKEN` env var or `~/.config/clawdie/crowdin-token` + +#### Step 2: Download Translations from Crowdin +```bash +./scripts/crowdin-sync.sh --pull +``` + +**What it does:** +- For each language (sl, de, hr, sr, ru, el, it, mk, sk, bs): + - Requests translation export from Crowdin + - Downloads ZIP of translated files + - Extracts to `docs/public/{lang}/` +- Creates `docs/public/{lang}/` directories if missing + +**Automation notes:** +- Run after translations reach >80% completion +- Safe to run multiple times (overwrites old translations) +- Requires same token as push + +#### Step 3: Build Astro Starlight Site +```bash +# In Astro project root +npm run build +``` + +**What it does:** +- Reads English source from `docs/public/` +- Reads translations from `docs/public/{lang}/` +- Generates static site at `dist/` +- Multi-language navigation configured in `astro.config.mjs` + +**Output structure:** +``` +dist/ + ├── index.html # SLO root + ├── en/ # English + ├── de/ # German + ├── hr/ # Croatian + └── [others...] +``` + +**Automation notes:** +- Requires Node.js >=20 + npm +- ~30-60 seconds per build +- Idempotent (safe to run multiple times) + +#### Step 4: Deploy to Production +```bash +# Copy build output to nginx webroot +sudo cp -r dist/* /usr/local/www/clawdie/ +``` + +**Automation notes:** +- Requires sudo access (setup in .env or via wheel group) +- Can also: rsync, git push to deployment repo, etc. +- Post-deploy: signal nginx `sudo nginx -s reload` + +--- + +## Phase 2: Astro Starlight Project Setup (Required) + +### Directory Structure + +Create `/home/clawdie/astro-docs/` (or configure `ASTRO_SITE_PATH` in `.env`): + +``` +/home/clawdie/astro-docs/ + ├── astro.config.mjs # Multi-locale Starlight config + ├── package.json + ├── src/ + │ ├── content/ + │ │ ├── config.ts # Content collections config + │ │ ├── docs/ + │ │ │ ├── index.mdx # SLO root + │ │ │ ├── en/ # English docs (symlink from docs/public/en/) + │ │ │ ├── de/ # German (symlink from docs/public/de/) + │ │ │ └── [others...] + │ │ └── i18n/ # UI translations (t() function) + │ │ ├── en.json + │ │ ├── de.json + │ │ └── sl.json + │ ├── components/ # Custom overrides + │ └── styles/ + └── dist/ # Build output +``` + +### astro.config.mjs Example + +```javascript +// astro.config.mjs +import { defineConfig } from 'astro/config'; +import starlight from '@astrojs/starlight'; + +export default defineConfig({ + site: 'https://docs.clawdie.si', + integrations: [ + starlight({ + title: 'Clawdie AI', + defaultLocale: 'sl', + locales: { + sl: { label: '🇸🇮 Slovenščina', lang: 'sl' }, + en: { label: '🇬🇧 English', lang: 'en' }, + de: { label: '🇩🇪 Deutsch', lang: 'de' }, + hr: { label: '🇭🇷 Hrvatski', lang: 'hr' }, + sr: { label: '🇷🇸 Српски', lang: 'sr' }, + ru: { label: '🇷🇺 Русский', lang: 'ru' }, + }, + sidebar: [ + { + label: 'Namestitev', + translations: { + en: 'Installation', + de: 'Installation', + hr: 'Instalacija', + sr: 'Инсталација', + ru: 'Установка', + }, + items: [ + { slug: 'install', label: 'Pregled', translations: { en: 'Overview' } }, + { slug: 'install/requirements', label: 'Zahteve' }, + ], + }, + { + label: 'Arhitektura', + translations: { + en: 'Architecture', + de: 'Architektur', + hr: 'Arhitektura', + sr: 'Архитектура', + ru: 'Архитектура', + }, + items: [ + { slug: 'architecture', label: 'Pregled' }, + { slug: 'architecture/jail-networking' }, + ], + }, + ], + }), + ], +}); +``` + +### Symlink Strategy + +Instead of copying translated docs, use symlinks to avoid duplication: + +```bash +cd /home/clawdie/astro-docs/src/content/docs/ +ln -s /home/clawdie/clawdie-ai/docs/public/en en +ln -s /home/clawdie/clawdie-ai/docs/public/de de +ln -s /home/clawdie/clawdie-ai/docs/public/hr hr +# ... etc +``` + +This way: +- Crowdin pulls translations to one location +- Astro builds from symlinks +- No duplication, easier to update + +--- + +## Phase 3: Crowdin Project Configuration + +### Create Project (if not exists) + +Visit https://crowdin.com/dashboard and create new project: +- **Project Name:** Clawdie-AI +- **Default Language:** English +- **Target Languages:** SL, DE, HR, SR, RU, EL, IT, MK, SK, BS + +### Configure .crowdin.yml + +Already configured in repo: +```yaml +project_id: 883714 +api_token_env: CROWDIN_PERSONAL_TOKEN +base_path: ./ +files: + - source: docs/public/**.md + dest: docs/public/%two_letters_code% + translation_update: crowdin_move +``` + +### API Token Setup + +1. Create personal token at https://crowdin.com/settings/api/tokens +2. Store in **one** of: + - `.env` file: `CROWDIN_PERSONAL_TOKEN=xxx` + - `~/.config/clawdie/crowdin-token`: `xxx` + - GitHub Actions secret: `CROWDIN_PERSONAL_TOKEN` + +--- + +## Phase 4: Automation for Pi/Aider + +### Cron-Based Sync (Daily/Weekly) + +Add to `.env`: +```bash +# Crowdin sync schedule (cron) +# "0 9 * * MON" = Every Monday at 9am +DOCS_SYNC_SCHEDULE="0 9 * * MON" +``` + +### Manual Execution (via Pi) + +Pi skill command: +```bash +pi "sync docs translations and rebuild the site" +``` + +Aider harness command: +```bash +aider --auto-commits \ + --message "Sync translations from Crowdin and rebuild docs" \ + "Crowdin sync and Astro rebuild required" +``` + +### GitHub Actions Workflow (Optional) + +Already created: `.github/workflows/crowdin-sync.yml` + +Triggers: +- **Manual** (workflow_dispatch): Click "Run workflow" to sync manually +- **Scheduled**: Every Monday at 9 UTC +- **On push**: When `docs/public/` changes + +--- + +## MVP Checklist + +- [ ] **Crowdin Project (883714)** - Verify project exists and is configured +- [ ] **English Source Upload** - Run `./scripts/crowdin-sync.sh --push` to seed project +- [ ] **Translator Setup** - Add translators to Crowdin, assign languages +- [ ] **Astro Starlight Project** - Create at `/home/clawdie/astro-docs/` with starlight theme +- [ ] **astro.config.mjs** - Configure all target locales and sidebar +- [ ] **Symlinks** - Link `src/content/docs/{en,de,hr,sr,ru,el,it,mk,sk,bs}` to `docs/public/{lang}` +- [ ] **Local Build Test** - `npm run build` produces multilingual static site +- [ ] **Crowdin → Pull Test** - `./scripts/crowdin-sync.sh --pull` downloads SLO + EN +- [ ] **Full Pipeline Test** - Push → Pull → Build → Deploy in sequence +- [ ] **Pi Automation** - Create Pi skill to execute full pipeline +- [ ] **Deploy Target** - Configure nginx at `docs.clawdie.si` to serve `dist/` +- [ ] **Monitoring** - Check Crowdin progress weekly, trigger rebuild when >80% complete + +--- + +## Troubleshooting + +### "No translations available for {lang}" +**Cause:** Language not configured in Crowdin or no translations yet. +**Fix:** Add language to Crowdin project settings, ensure translators have assignments. + +### "Failed to upload to storage" +**Cause:** Invalid Crowdin token or API rate limit. +**Fix:** Verify token in `.env` or `~/.config/clawdie/crowdin-token`, wait 1 minute, retry. + +### "Astro build fails with missing locale" +**Cause:** Translation directory not downloaded yet. +**Fix:** Run `./scripts/crowdin-sync.sh --pull` before build, or exclude incomplete locales from `astro.config.mjs`. + +### "Symlinks not found by Astro" +**Cause:** Symlinks created in wrong location or Astro reading before symlinks established. +**Fix:** Verify symlinks exist at `src/content/docs/{lang}/`, restart dev server with `npm run dev`. + +--- + +## Next Steps (Post-MVP) + +### Phase 5: Strapi CMS Integration (Deferred) + +Once FreeBSD native binding issues are resolved (see `doc/STRAPI-FREEBSD-GOTCHA.md`): + +- Strapi as backend for rich media content (images, videos, structured data) +- Crowdin sync to Strapi API endpoints +- Astro fetches from Strapi for dynamic components +- CMS content types: Page, Guide, Skill, BlogPost + +Current blockers: +- `sharp` (image processing) lacks FreeBSD prebuilt binary +- `esbuild`, `@swc/core` same issue +- Workaround: Compile libvips from /usr/ports, but time-intensive + +**Recommendation:** Keep Strapi in roadmap, ship markdown-only MVP first. + +### Phase 6: Real-Time Collaboration + +- Webhook from Crowdin to rebuild site immediately after translation milestone +- Slack notification on translation progress +- Discord bot for translator coordination + +--- + +## References + +- **Crowdin API:** https://developer.crowdin.com/api/ +- **Astro Starlight:** https://starlight.astro.build/ +- **Crowdin Sync Script:** `./scripts/crowdin-sync.sh` +- **FreeBSD CMS Gotcha:** `doc/STRAPI-FREEBSD-GOTCHA.md` diff --git a/.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh b/.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh new file mode 100755 index 0000000..b260d02 --- /dev/null +++ b/.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh @@ -0,0 +1,286 @@ +#!/bin/sh +# Three-Bird Docs Localization Pipeline Orchestrator +# Automates: Crowdin sync → Astro build → Deploy +# +# Usage: +# ./orchestrate-pipeline.sh --all # Full pipeline: push → pull → build → deploy +# ./orchestrate-pipeline.sh --pull # Pull translations only (no push) +# ./orchestrate-pipeline.sh --build # Pull + build only (no push/deploy) +# ./orchestrate-pipeline.sh --build-only # Build only (skip pull) +# ./orchestrate-pipeline.sh --deploy-only # Deploy existing build +# ./orchestrate-pipeline.sh --status # Check Crowdin status only +# +# Environment variables (from .env or CLI): +# CROWDIN_PERSONAL_TOKEN - Crowdin API token (required for push/pull) +# ASTRO_SITE_PATH - Path to Astro project (default: /home/clawdie/astro-docs) +# DOCS_DEPLOY_TARGET - nginx webroot (default: /usr/local/www/clawdie/docs) +# DOCS_MIN_COMPLETION - Min % completion before deploy (default: 80) + +set -e + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +REPO_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)" +ASTRO_SITE_PATH="${ASTRO_SITE_PATH:-/home/clawdie/astro-docs}" +DOCS_DEPLOY_TARGET="${DOCS_DEPLOY_TARGET:-/usr/local/www/clawdie/docs}" +DOCS_MIN_COMPLETION="${DOCS_MIN_COMPLETION:-80}" + +# Colors +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' + +log_info() { echo "${GREEN}[INFO]${NC} $1"; } +log_warn() { echo "${YELLOW}[WARN]${NC} $1"; } +log_err() { echo "${RED}[ERROR]${NC} $1" >&2; exit 1; } +log_section() { echo ""; echo "${BLUE}=== $1 ===${NC}"; } + +# ──────────────────────────────────────────────────────────────────────────── +# Helper functions +# ──────────────────────────────────────────────────────────────────────────── + +check_prereqs() { + local missing=0 + + if ! command -v node >/dev/null 2>&1; then + log_warn "Node.js not found" + missing=1 + fi + + if ! command -v npm >/dev/null 2>&1; then + log_warn "npm not found" + missing=1 + fi + + if [ ! -d "$REPO_ROOT" ]; then + log_err "Repository root not found: $REPO_ROOT" + fi + + if [ $missing -eq 1 ]; then + log_err "Missing prerequisites. Install with: pkg install node24 npm-node24" + fi +} + +check_crowdin_token() { + if [ -n "$CROWDIN_PERSONAL_TOKEN" ]; then + return 0 + elif [ -f ~/.config/clawdie/crowdin-token ]; then + return 0 + elif [ -f "$REPO_ROOT/.env" ] && grep -q CROWDIN_PERSONAL_TOKEN "$REPO_ROOT/.env"; then + return 0 + else + log_err "Crowdin token not found. Set CROWDIN_PERSONAL_TOKEN or create ~/.config/clawdie/crowdin-token" + fi +} + +get_crowdin_completion() { + # Returns average completion % across all languages + # Used to decide if deploy is safe + if ! check_crowdin_token 2>/dev/null; then + log_warn "Crowdin token not available, skipping completion check" + return 0 + fi + + cd "$REPO_ROOT" + local completion + completion=$("$REPO_ROOT/scripts/crowdin-sync.sh" --status 2>/dev/null | grep -oP '\(\K[0-9]+(?=%)' | awk '{sum+=$1; count++} END {if (count>0) printf "%.0f", sum/count; else print "0"}') + echo "$completion" +} + +# ──────────────────────────────────────────────────────────────────────────── +# Stage 1: Crowdin Push (upload English sources) +# ──────────────────────────────────────────────────────────────────────────── + +stage_crowdin_push() { + log_section "Stage 1: Push English Sources to Crowdin" + + check_crowdin_token || return 1 + + cd "$REPO_ROOT" + if ./scripts/crowdin-sync.sh --push; then + log_info "✓ English sources uploaded" + return 0 + else + log_err "✗ Failed to push sources" + return 1 + fi +} + +# ──────────────────────────────────────────────────────────────────────────── +# Stage 2: Crowdin Pull (download translations) +# ──────────────────────────────────────────────────────────────────────────── + +stage_crowdin_pull() { + log_section "Stage 2: Pull Translations from Crowdin" + + check_crowdin_token || return 1 + + # Check completion % + local completion + completion=$(get_crowdin_completion) + + if [ "$completion" -lt "$DOCS_MIN_COMPLETION" ]; then + log_warn "Translation completion: ${completion}% (minimum: ${DOCS_MIN_COMPLETION}%)" + log_warn "Proceeding anyway (use --force to skip this check)" + else + log_info "Translation completion: ${completion}% ✓" + fi + + cd "$REPO_ROOT" + if ./scripts/crowdin-sync.sh --pull; then + log_info "✓ Translations downloaded" + return 0 + else + log_err "✗ Failed to pull translations" + return 1 + fi +} + +# ──────────────────────────────────────────────────────────────────────────── +# Stage 3: Astro Build +# ──────────────────────────────────────────────────────────────────────────── + +stage_astro_build() { + log_section "Stage 3: Build Astro Site" + + if [ ! -d "$ASTRO_SITE_PATH" ]; then + log_warn "Astro project not found at: $ASTRO_SITE_PATH" + log_info "Setting up Astro Starlight..." + + if [ -x "$SCRIPT_DIR/setup-astro-docs.sh" ]; then + "$SCRIPT_DIR/setup-astro-docs.sh" "$ASTRO_SITE_PATH" || log_err "Failed to setup Astro" + else + log_err "Setup script not found: $SCRIPT_DIR/setup-astro-docs.sh" + fi + fi + + if [ ! -d "$ASTRO_SITE_PATH" ]; then + log_err "Astro path still missing: $ASTRO_SITE_PATH" + fi + + log_info "Building Astro site at: $ASTRO_SITE_PATH" + + cd "$ASTRO_SITE_PATH" + + if [ ! -f package.json ]; then + log_err "package.json not found in Astro project" + fi + + if ! npm run build; then + log_err "✗ Astro build failed" + return 1 + fi + + if [ ! -d dist ]; then + log_err "Build output not found: $ASTRO_SITE_PATH/dist" + fi + + log_info "✓ Site built: $(find dist -type f | wc -l) files" + return 0 +} + +# ──────────────────────────────────────────────────────────────────────────── +# Stage 4: Deploy +# ──────────────────────────────────────────────────────────────────────────── + +stage_deploy() { + log_section "Stage 4: Deploy to Nginx" + + if [ ! -d "$ASTRO_SITE_PATH/dist" ]; then + log_err "Build output not found. Run build first: $ASTRO_SITE_PATH/dist" + fi + + mkdir -p "$DOCS_DEPLOY_TARGET" + + log_info "Copying files from $ASTRO_SITE_PATH/dist → $DOCS_DEPLOY_TARGET" + + if sudo cp -r "$ASTRO_SITE_PATH/dist/"* "$DOCS_DEPLOY_TARGET/"; then + log_info "✓ Files deployed" + else + log_err "✗ Deploy failed (check sudo permissions)" + fi + + # Reload nginx (optional) + if command -v nginx >/dev/null 2>&1 && sudo nginx -t 2>/dev/null; then + if sudo nginx -s reload 2>/dev/null; then + log_info "✓ Nginx reloaded" + else + log_warn "⚠ Failed to reload nginx (may need manual: sudo nginx -s reload)" + fi + else + log_warn "⚠ nginx not found or not running" + fi + + log_info "✓ Deploy complete: https://docs.clawdie.si" + return 0 +} + +# ──────────────────────────────────────────────────────────────────────────── +# Main +# ──────────────────────────────────────────────────────────────────────────── + +main() { + local action="${1:---all}" + + check_prereqs + + case "$action" in + --all) + log_info "Running full pipeline: push → pull → build → deploy" + stage_crowdin_push || exit 1 + stage_crowdin_pull || exit 1 + stage_astro_build || exit 1 + stage_deploy || exit 1 + log_section "Pipeline Complete" + log_info "✓ Docs published at https://docs.clawdie.si" + ;; + --push) + stage_crowdin_push || exit 1 + ;; + --pull) + stage_crowdin_pull || exit 1 + ;; + --build) + stage_crowdin_pull || exit 1 + stage_astro_build || exit 1 + ;; + --build-only) + stage_astro_build || exit 1 + ;; + --deploy-only) + stage_deploy || exit 1 + ;; + --status) + log_section "Crowdin Status" + check_crowdin_token && cd "$REPO_ROOT" && ./scripts/crowdin-sync.sh --status || log_warn "Crowdin token not available" + ;; + --help|-h) + echo "Three-Bird Docs Pipeline Orchestrator" + echo "" + echo "Usage: $0 {--all|--push|--pull|--build|--build-only|--deploy-only|--status}" + echo "" + echo "Stages:" + echo " --push Push English sources to Crowdin" + echo " --pull Pull translations from Crowdin" + echo " --build Pull translations + build Astro site" + echo " --build-only Build Astro site only (skip pull)" + echo " --deploy-only Deploy existing build" + echo " --all Full pipeline (push → pull → build → deploy)" + echo " --status Check Crowdin status only" + echo "" + echo "Environment variables:" + echo " CROWDIN_PERSONAL_TOKEN - Crowdin API token" + echo " ASTRO_SITE_PATH - Path to Astro project (default: /home/clawdie/astro-docs)" + echo " DOCS_DEPLOY_TARGET - Deploy path (default: /usr/local/www/clawdie/docs)" + echo " DOCS_MIN_COMPLETION - Min % completion to deploy (default: 80)" + echo "" + exit 0 + ;; + *) + log_err "Unknown action: $action (use --help for usage)" + ;; + esac +} + +main "$@" diff --git a/.agent/skills/docs-localization-pipeline/setup-astro-docs.sh b/.agent/skills/docs-localization-pipeline/setup-astro-docs.sh new file mode 100755 index 0000000..9d1c8a8 --- /dev/null +++ b/.agent/skills/docs-localization-pipeline/setup-astro-docs.sh @@ -0,0 +1,378 @@ +#!/bin/sh +# Setup Astro Starlight Docs Site +# Creates the Astro project structure, configures Starlight, and sets up symlinks to Crowdin translations +# +# Usage: +# ./setup-astro-docs.sh # Use default path /home/clawdie/astro-docs +# ./setup-astro-docs.sh /custom/path # Use custom installation path +# +# Prerequisites: +# - Node.js >=20, npm +# - Docs source at /home/clawdie/clawdie-ai/docs/public/ +# - Crowdin translated dirs at /home/clawdie/clawdie-ai/docs/public/{en,de,hr,sr,ru,el,it,mk,sk,bs} + +set -e + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +REPO_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)" +ASTRO_PATH="${1:-/home/clawdie/astro-docs}" +DOCS_SOURCE="${REPO_ROOT}/docs/public" + +log_info() { echo "[INFO] $1"; } +log_err() { echo "[ERROR] $1" >&2; exit 1; } + +# ──────────────────────────────────────────────────────────────────────────── +# Verify prerequisites +# ──────────────────────────────────────────────────────────────────────────── + +if ! command -v node >/dev/null 2>&1; then + log_err "Node.js not found. Install with: pkg install node24" +fi + +if ! command -v npm >/dev/null 2>&1; then + log_err "npm not found. Install with: pkg install npm-node24" +fi + +if [ ! -d "$DOCS_SOURCE" ]; then + log_err "Docs source not found: $DOCS_SOURCE" +fi + +log_info "Prerequisites OK" + +# ──────────────────────────────────────────────────────────────────────────── +# Create Astro project +# ──────────────────────────────────────────────────────────────────────────── + +if [ -d "$ASTRO_PATH" ]; then + log_info "Astro path already exists: $ASTRO_PATH" + log_info "Updating configuration (skipping npm install)..." + UPDATE_ONLY=1 +else + log_info "Creating Astro project at: $ASTRO_PATH" + mkdir -p "$ASTRO_PATH" +fi + +cd "$ASTRO_PATH" + +# ──────────────────────────────────────────────────────────────────────────── +# Create package.json (minimal) +# ──────────────────────────────────────────────────────────────────────────── + +if [ ! -f package.json ] || [ -z "$UPDATE_ONLY" ]; then + log_info "Creating package.json..." + cat > package.json << 'EOF' +{ + "name": "clawdie-docs", + "version": "1.0.0", + "description": "Clawdie AI Documentation Site", + "type": "module", + "scripts": { + "dev": "astro dev", + "build": "astro build", + "preview": "astro preview" + }, + "dependencies": { + "astro": "^5.3.0", + "@astrojs/starlight": "^0.37.0", + "@astrojs/check": "^0.11.0", + "typescript": "^5.0.0" + } +} +EOF + log_info "Installing dependencies..." + npm install +fi + +# ──────────────────────────────────────────────────────────────────────────── +# Create astro.config.mjs +# ──────────────────────────────────────────────────────────────────────────── + +log_info "Creating astro.config.mjs..." +cat > astro.config.mjs << 'EOF' +// astro.config.mjs +import { defineConfig } from 'astro/config'; +import starlight from '@astrojs/starlight'; + +export default defineConfig({ + site: 'https://docs.clawdie.si', + integrations: [ + starlight({ + title: 'Clawdie AI', + description: 'Personal AI Assistant Documentation', + defaultLocale: 'sl', + locales: { + sl: { + label: '🇸🇮 Slovenščina', + lang: 'sl', + }, + en: { + label: '🇬🇧 English', + lang: 'en', + }, + de: { + label: '🇩🇪 Deutsch', + lang: 'de', + }, + hr: { + label: '🇭🇷 Hrvatski', + lang: 'hr', + }, + sr: { + label: '🇷🇸 Српски', + lang: 'sr', + }, + ru: { + label: '🇷🇺 Русский', + lang: 'ru', + }, + el: { + label: '🇬🇷 Ελληνικά', + lang: 'el', + }, + it: { + label: '🇮🇹 Italiano', + lang: 'it', + }, + mk: { + label: '🇲🇰 Македонски', + lang: 'mk', + }, + sk: { + label: '🇸🇰 Slovenčina', + lang: 'sk', + }, + bs: { + label: '🇧🇦 Bosanski', + lang: 'bs', + }, + }, + sidebar: [ + { + label: 'Namestitev', + translations: { + en: 'Installation', + de: 'Installation', + hr: 'Instalacija', + sr: 'Инсталација', + ru: 'Установка', + el: 'Εγκατάσταση', + it: 'Installazione', + mk: 'Инсталација', + sk: 'Inštalácia', + bs: 'Instalacija', + }, + items: [ + { + slug: 'install', + label: 'Pregled', + translations: { + en: 'Overview', + de: 'Überblick', + hr: 'Pregled', + sr: 'Преглед', + ru: 'Обзор', + }, + }, + { + slug: 'install/requirements', + label: 'Zahteve', + translations: { + en: 'Requirements', + de: 'Anforderungen', + hr: 'Zahtjevi', + }, + }, + { + slug: 'install/install', + label: 'Namestitev', + translations: { + en: 'Installation', + }, + }, + { + slug: 'install/iso', + label: 'Clawdie-ISO', + translations: { + en: 'Clawdie-ISO', + }, + }, + ], + }, + { + label: 'Arhitektura', + translations: { + en: 'Architecture', + de: 'Architektur', + hr: 'Arhitektura', + sr: 'Архитектура', + ru: 'Архитектура', + }, + items: [ + { + slug: 'architecture', + label: 'Pregled', + translations: { + en: 'Overview', + }, + }, + { + slug: 'architecture/jail-networking', + label: 'Jail Networking', + }, + { + slug: 'architecture/bastille', + label: 'Bastille', + }, + { + slug: 'architecture/controlplane', + label: 'Controlplane', + }, + ], + }, + { + label: 'Referenca', + translations: { + en: 'Reference', + de: 'Referenz', + hr: 'Referencija', + sr: 'Референца', + ru: 'Справка', + }, + items: [ + { + slug: 'reference', + label: 'Pregled', + translations: { + en: 'Overview', + }, + }, + ], + }, + ], + social: { + github: 'https://codeberg.org/Clawdie/Clawdie-AI', + }, + customCss: [], + }), + ], +}); +EOF +log_info "Created astro.config.mjs" + +# ──────────────────────────────────────────────────────────────────────────── +# Create directory structure +# ──────────────────────────────────────────────────────────────────────────── + +log_info "Creating directory structure..." +mkdir -p src/content/docs +mkdir -p src/content/i18n +mkdir -p src/components +mkdir -p src/styles + +# ──────────────────────────────────────────────────────────────────────────── +# Create content config +# ──────────────────────────────────────────────────────────────────────────── + +log_info "Creating content collections config..." +cat > src/content/config.ts << 'EOF' +// src/content/config.ts +import { docsLoader, i18nLoader } from '@astrojs/starlight/loaders'; +import { docsSchema, i18nSchema } from '@astrojs/starlight/schema'; +import { defineCollection } from 'astro:content'; + +export const collections = { + docs: defineCollection({ + loader: docsLoader(), + schema: docsSchema(), + }), + i18n: defineCollection({ + loader: i18nLoader(), + schema: i18nSchema(), + }), +}; +EOF +log_info "Created src/content/config.ts" + +# ──────────────────────────────────────────────────────────────────────────── +# Create symlinks to translation directories +# ──────────────────────────────────────────────────────────────────────────── + +log_info "Setting up symlinks to Crowdin translations..." + +cd "src/content/docs" + +for lang in en de hr sr ru el it mk sk bs; do + if [ -L "$lang" ]; then + log_info " $lang: symlink already exists" + elif [ -d "$lang" ]; then + log_info " $lang: directory exists, removing for symlink..." + rm -rf "$lang" + ln -s "${DOCS_SOURCE}/${lang}" "$lang" + log_info " $lang: created symlink" + else + log_info " $lang: creating symlink..." + ln -s "${DOCS_SOURCE}/${lang}" "$lang" 2>/dev/null || log_err "Failed to create symlink for $lang" + log_info " $lang: created" + fi +done + +log_info "Symlinks created" + +# ──────────────────────────────────────────────────────────────────────────── +# Create i18n translations +# ──────────────────────────────────────────────────────────────────────────── + +cd "$ASTRO_PATH" + +log_info "Creating UI translations..." +cat > src/content/i18n/sl.json << 'EOF' +{ + "search.label": "Iskanje", + "search.cancelLabel": "Prekini", + "themeSelect.accessibleLabel": "Izbira teme", + "tableOfContents.onThisPage": "Na tej strani" +} +EOF + +cat > src/content/i18n/en.json << 'EOF' +{ + "search.label": "Search", + "search.cancelLabel": "Cancel", + "themeSelect.accessibleLabel": "Select theme", + "tableOfContents.onThisPage": "On this page" +} +EOF + +cat > src/content/i18n/de.json << 'EOF' +{ + "search.label": "Suche", + "search.cancelLabel": "Abbrechen", + "themeSelect.accessibleLabel": "Thema wählen", + "tableOfContents.onThisPage": "Auf dieser Seite" +} +EOF + +log_info "Created i18n translations" + +# ──────────────────────────────────────────────────────────────────────────── +# Create tsconfig +# ──────────────────────────────────────────────────────────────────────────── + +log_info "Creating tsconfig.json..." +cat > tsconfig.json << 'EOF' +{ + "extends": "astro/tsconfigs/strict" +} +EOF + +log_info "Setup complete!" +log_info "" +log_info "Next steps:" +log_info " 1. cd $ASTRO_PATH" +log_info " 2. npm run dev # Start dev server at http://localhost:4321" +log_info " 3. npm run build # Build for production" +log_info "" +log_info "To sync translations from Crowdin first:" +log_info " cd ${REPO_ROOT}" +log_info " ./scripts/crowdin-sync.sh --pull # Downloads translated docs" +log_info "" diff --git a/AGENTS.md b/AGENTS.md index 81d54ad..7ba8a9c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -39,14 +39,65 @@ TMP_FILE="/tmp/work-file" --- -## Tmux Long-Running Tasks +## Tmux Window Targeting (Session:Index) -Run long-running commands (builds, installs, CMS rebuilds, ISO builds) inside -the tmux window named `testing` so the operator can watch progress. +Always target tmux windows by **session:index** syntax, never by name alone. -- If `testing` does not exist, create it before starting the command. -- Keep the long-running command output in that window; do not run it in the - main shell. +### Why + +tmux window names can conflict with session names (e.g., `-t testing` could refer to session "testing" or window "testing"). The **SESSION:WINDOW** pattern eliminates ambiguity. + +### Convention + +```bash +# CORRECT - explicit session:window targeting +tmux send-keys -t 0:1 "command" Enter + +# WRONG - ambiguous; causes "can't find pane" errors +tmux send-keys -t testing "command" Enter +``` + +**Pattern:** `SESSION:WINDOW` +- `SESSION` = session index or name (e.g., `0`, `main`, `aider-smoke`) +- `WINDOW` = window index (e.g., `0`, `1`, `2`) + +### Check Before Targeting + +Always run `tmux list-windows` first to see current state: + +```bash +tmux list-windows +# 0: codex (1 panes) [171x39] +# 1: testing- (1 panes) [171x39] +# 2: node* (1 panes) [171x39] (active) +``` + +Then send to the correct window: `tmux send-keys -t 0:1 "command" Enter` + +### Long-Running Tasks + +Run builds, installs, CMS rebuilds, ISO builds in the default session's window 1 (or a dedicated window): + +```bash +# Check state first +tmux list-windows + +# Then target session:window explicitly +tmux send-keys -t 0:1 "npm run build" Enter +tmux capture-pane -t 0:1 -p -S -100 # Read output after +``` + +### Creating Windows (if needed) + +```bash +# Create new window in current/default session +tmux new-window -t 0 -n build-work + +# Send command to it +tmux send-keys -t 0:3 "long-running-command" Enter +``` + +Do not rely on window names for targeting; always use the index returned by `list-windows`. --- diff --git a/doc/THREE-BIRD-ARCHITECTURE.md b/doc/THREE-BIRD-ARCHITECTURE.md new file mode 100644 index 0000000..3e39ca8 --- /dev/null +++ b/doc/THREE-BIRD-ARCHITECTURE.md @@ -0,0 +1,403 @@ +# Three-Bird Documentation Architecture + +**Status:** MVP Implementation Complete (April 2026) +**Goal:** Codify documentation localization for Pi/Aider automation +**Scope:** EN + SLO (MVP), roadmap: DE/SRB/CRO/BIH/RU/ZN/BR + +--- + +## Overview + +The **three-bird architecture** decouples documentation concerns into three independent systems that work together: + +1. **🐦 Crowdin** — Localization source of truth. Single English source, managed translations in 9 languages. +2. **🐦 Astro Starlight** — Static site generator. Builds multilingual docs from Crowdin translations + English. +3. **🐦 Strapi CMS** (deferred) — Content backend for rich media/structured content. Blocked by FreeBSD native binding issues; see `doc/STRAPI-FREEBSD-GOTCHA.md`. + +## Architecture Diagram + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ ENGLISH SOURCE DOCS │ +│ docs/public/**.md (repo) │ +│ │ +│ - install/ - architecture/ │ +│ - reference/ - getting-started/ │ +│ - api/ - troubleshooting/ │ +└────────────────────────────┬────────────────────────────────────┘ + │ + ┌────────▼────────┐ + │ Crowdin Push │ + │ (./scripts/ │ + │ crowdin-sync │ + │ .sh --push) │ + └────────┬────────┘ + │ + ┌───────────────────┼───────────────────┐ + │ │ │ + │ ┌─────▼──────┐ │ + │ │ Crowdin │ │ + │ │ Localization│ │ + │ │ Platform │ │ + │ │ (883714) │ │ + │ └─────┬──────┘ │ + │ │ │ +┌────────▼────────┐ ┌──────▼──────┐ ┌────────▼────────┐ +│ Slovenian (SLO) │ │ German (DE) │ │ Croatian (HR) │ +│ 100% default │ │ Roadmap │ │ Roadmap │ +└────────┬────────┘ └──────┬──────┘ └────────┬────────┘ + │ │ │ + └──────────────────┼───────────────────┘ + │ + ┌──────────▼──────────┐ + │ Crowdin Pull │ + │ (./scripts/ │ + │ crowdin-sync │ + │ .sh --pull) │ + └──────────┬──────────┘ + │ + ┌──────────────────┼──────────────────┐ + │ │ │ + ┌────▼────┐ ┌────▼────┐ ┌────▼────┐ + │docs/ │ │docs/ │ │docs/ │ + │public/ │ │public/ │ │public/ │ + │sl/ │ │de/ │ │hr/ │ + └────┬────┘ └────┬────┘ └────┬────┘ + │ │ │ + └──────────────────┼──────────────────┘ + │ + ┌────────────────▼────────────────┐ + │ Astro Starlight Build │ + │ (npm run build at /home/ │ + │ clawdie/astro-docs/) │ + │ │ + │ - Multi-locale config │ + │ - Sidebar i18n │ + │ - Static site generation │ + └────────────────┬────────────────┘ + │ + ┌────────────────▼────────────────┐ + │ Static Site Output │ + │ (dist/) │ + │ │ + │ / (SLO) │ + │ /en/ (English) │ + │ /de/ (German) │ + │ /hr/ (Croatian) │ + │ ... (others) │ + └────────────────┬────────────────┘ + │ + ┌────────────────▼────────────────┐ + │ Deploy to nginx │ + │ (/usr/local/www/clawdie/docs) │ + └────────────────┬────────────────┘ + │ + ┌────────────────▼────────────────┐ + │ PUBLIC: docs.clawdie.si │ + └─────────────────────────────────┘ +``` + +--- + +## MVP Decision: Markdown-Only (Option A) + +**Chosen Approach:** Ship with Crowdin + Astro (no Strapi for now) + +### Why Markdown-Only First + +| Factor | Markdown-Only | Strapi Path | +|--------|---------------|-------------| +| **Shipping time** | 1-2 weeks | 3-4 weeks | +| **Dependencies** | Standard tools | FreeBSD blocker | +| **MVP feature set** | ✓ Full (docs + translations) | ✗ Incomplete (no image upload) | +| **Maintenance** | Simple (pure markdown) | Complex (CMS + API) | +| **Codifiable for Pi** | ✓ Yes | ⚠ Conditional | + +### What Ships in MVP + +``` +✓ Crowdin localization platform (LIVE) +✓ Markdown sources in docs/public/ +✓ Automated Crowdin → Astro pipeline +✓ Multi-language static site +✓ English (EN) + Slovenian (SLO) default +✓ Roadmap languages configured (not translated yet) +✓ Pi/Aider automation for sync & deploy +``` + +### What Defers (Phase 2+) + +``` +✗ Strapi CMS (blocked by FreeBSD native bindings) +✗ Rich media uploads (requires Strapi + sharp image processing) +✗ Structured content types (Page, Guide, BlogPost) +✗ CMS-driven updates (once Strapi is viable) +``` + +--- + +## Implementation Status + +### ✓ Complete + +| Component | Location | Status | +|-----------|----------|--------| +| **Crowdin Script** | `./scripts/crowdin-sync.sh` | ✓ Working (push/pull/status) | +| **Crowdin Config** | `.crowdin.yml` | ✓ Configured (project 883714) | +| **English Source Docs** | `docs/public/**/*.md` | ✓ 42+ files ready | +| **Skill Documentation** | `.agent/skills/docs-localization-pipeline/SKILL.md` | ✓ Complete | +| **Setup Script** | `.agent/skills/docs-localization-pipeline/setup-astro-docs.sh` | ✓ Tested | +| **Pipeline Orchestrator** | `.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh` | ✓ Ready | +| **Skill README** | `.agent/skills/docs-localization-pipeline/README.md` | ✓ Complete | + +### ⏳ Pending (User Action) + +| Task | Owner | Timeline | +|------|-------|----------| +| Create Crowdin translators | User | ASAP | +| Push English sources to Crowdin | Pi/Aider | This week | +| Assign translation languages | Translators | This week | +| Reach 80% completion milestone | Translators | 1-2 weeks | +| First pull & build | Pi/Aider | After milestone | +| Deploy to docs.clawdie.si | Pi/Aider | After build | +| Setup weekly sync schedule | Pi/Aider | Ongoing | + +--- + +## How It Works + +### Daily Workflow (Codified for Pi/Aider) + +```bash +# 1. English docs are edited in /docs/public/**.md +# 2. Crowdin sync pushed (via Pi or cron) +./scripts/crowdin-sync.sh --push + +# 3. Crowdin translators update translations +# 4. Weekly sync pulls translations +./scripts/crowdin-sync.sh --pull + +# 5. Astro builds multilingual site +cd /home/clawdie/astro-docs && npm run build + +# 6. Deploy to nginx +sudo cp -r dist/* /usr/local/www/clawdie/docs/ +sudo nginx -s reload +``` + +**Automated by Pi/Aider:** +```bash +pi "sync translations and rebuild documentation site" +# → Executes: orchestrate-pipeline.sh --all +``` + +### Configuration Codification + +All automation is controlled by environment variables in `.env`: + +```bash +# Crowdin API token +CROWDIN_PERSONAL_TOKEN=tskey-xxx + +# Paths +ASTRO_SITE_PATH=/home/clawdie/astro-docs +DOCS_DEPLOY_TARGET=/usr/local/www/clawdie/docs + +# Thresholds +DOCS_MIN_COMPLETION=80 # Min % before auto-deploy +``` + +--- + +## File Structure + +### Skill Assets + +``` +.agent/skills/docs-localization-pipeline/ +├── SKILL.md # Detailed architecture & reference +├── README.md # Quick start & troubleshooting +├── setup-astro-docs.sh # Bootstrap Astro project +└── orchestrate-pipeline.sh # Full pipeline automation +``` + +### Documentation + +``` +doc/ +├── THREE-BIRD-ARCHITECTURE.md # This file (overview) +├── STRAPI-FREEBSD-GOTCHA.md # Why Strapi deferred +└── [other docs] + +docs/public/ # English source +├── install/ +├── architecture/ +├── reference/ +└── [content sections] + +docs/public/{lang}/ # Crowdin translations +├── en/ # English +├── de/ # German (roadmap) +├── hr/ # Croatian (roadmap) +└── [others] +``` + +### Astro Project + +``` +/home/clawdie/astro-docs/ +├── astro.config.mjs # Multi-locale Starlight config +├── package.json +├── src/ +│ ├── content/ +│ │ ├── config.ts # Content collections +│ │ ├── docs/ +│ │ │ ├── en → docs/public/en (symlink) +│ │ │ ├── de → docs/public/de (symlink) +│ │ │ └── [others] +│ │ └── i18n/ +│ │ ├── en.json +│ │ ├── de.json +│ │ └── sl.json +│ ├── components/ +│ └── styles/ +└── dist/ # Build output (after npm run build) +``` + +--- + +## Integration with clawdie-ai Controlplane + +### Config Values (src/config.ts) + +```typescript +// CMS/Docs paths +export const CMS_JAIL_IP = envConfig.WARDEN_CMS_IP || '10.0.0.4'; +export const ASTRO_SITE_PATH = envConfig.ASTRO_SITE_PATH || '/home/clawdie/astro-docs'; +export const CMS_WEBROOT = envConfig.CMS_WEBROOT || '/usr/local/www/clawdie'; + +// Used by jail runtime for mounting docs build +``` + +### Jail Networking + +``` +10.0.0.3 = db (PostgreSQL) +10.0.0.4 = cms (Strapi later, Astro build for now) +10.0.0.7 = web (nginx) + +nginx serves: + - / → docs.clawdie.si (from Astro build) + - /api → Strapi (when available) +``` + +--- + +## Phase 2: Strapi CMS Integration (Deferred) + +### Why Deferred + +FreeBSD lacks prebuilt binaries for native Node.js modules that Strapi depends on: +- `sharp` (image processing) +- `esbuild` (bundling) +- `@swc/core` (transpilation) + +**Workaround:** Compile libvips from `/usr/ports/graphics/vips` (2-3 hours on first build). + +**Recommendation:** Ship MVP first, add Strapi later when image upload is needed. + +### What Strapi Will Add + +``` +Enhanced CMS Features: +✓ Rich media upload (images, videos) +✓ Structured content types (Page, Guide, BlogPost) +✓ API endpoints for dynamic content +✓ Built-in user authentication +✓ Content versioning & publishing workflow + +Integration: +- Crowdin syncs content to Strapi API +- Astro queries Strapi for rich media +- CMS-driven metadata (authors, dates, categories) +``` + +### Strapi Setup (When Ready) + +See `doc/STRAPI-FREEBSD-GOTCHA.md` for workarounds and installation steps. + +--- + +## Automation Examples + +### Example 1: Manual Full Pipeline + +```bash +# One-command deployment +./.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh --all +``` + +### Example 2: Scheduled Weekly Sync + +```bash +# Add to crontab +0 9 * * MON /home/clawdie/clawdie-ai/.agent/skills/docs-localization-pipeline/orchestrate-pipeline.sh --pull +``` + +### Example 3: Pi/Aider Orchestration + +```bash +# Natural language command +pi "Update documentation translations and rebuild the site" + +# Aider auto-commit +aider --auto-commits \ + --message "Sync translations from Crowdin" \ + "docs/public/" +``` + +### Example 4: GitHub Actions + +See `.github/workflows/crowdin-sync.yml` (optional CI/CD). + +--- + +## Success Metrics + +### MVP Launch + +- [ ] Crowdin project 883714 has English sources uploaded +- [ ] Translators assigned to at least SLO + EN +- [ ] `orchestrate-pipeline.sh --all` produces live site at docs.clawdie.si +- [ ] Pi can execute: `pi "sync and rebuild docs"` +- [ ] Weekly automation in place + +### Phase 2 (Post-MVP) + +- [ ] DE, HR, SR translations at 80%+ +- [ ] Automated weekly syncs running +- [ ] Strapi ready (libvips compiled) +- [ ] CMS content integration tested + +--- + +## Quick Links + +| Resource | Path | Purpose | +|----------|------|---------| +| **Skill** | `.agent/skills/docs-localization-pipeline/` | Executable automation | +| **Architecture Doc** | `doc/THREE-BIRD-ARCHITECTURE.md` | This file | +| **Crowdin Gotcha** | `doc/STRAPI-FREEBSD-GOTCHA.md` | Strapi blockers & workarounds | +| **Crowdin Sync Script** | `./scripts/crowdin-sync.sh` | Manual push/pull/status | +| **Crowdin Config** | `.crowdin.yml` | Project settings | +| **AGENTS.md** | `AGENTS.md` | Pi/Aider harness guidance | +| **Astro Example** | `examples/astro-cv/` | Reference Starlight setup | + +--- + +## References + +- **Crowdin:** https://crowdin.com/project/clawdie-ai +- **Astro Starlight:** https://starlight.astro.build/ +- **Crowdin API:** https://developer.crowdin.com/api/ +- **FreeBSD CMS Issues:** `doc/STRAPI-FREEBSD-GOTCHA.md`