diff --git a/.agent/skills/postgres-memory/references/layout.md b/.agent/skills/postgres-memory/references/layout.md index 78d604a..33a6627 100644 --- a/.agent/skills/postgres-memory/references/layout.md +++ b/.agent/skills/postgres-memory/references/layout.md @@ -59,6 +59,7 @@ The jail dataset should be snapshotted at these points: ## Design note -SQLite remains the current operational state store. +All operational data lives in PostgreSQL (ops database). The memory database +holds user/agent memories and the skills database holds preloaded knowledge. -This PostgreSQL jail is the live split-brain backend for Agent System Skills and User/Agent Memory. +This PostgreSQL jail is the live split-brain backend for Agent System Skills, User/Agent Memory, and Operational State across three databases. diff --git a/AGENTS.md b/AGENTS.md index 65723a9..b1ba82b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -160,12 +160,12 @@ new Date().toISOString(); // "2024-03-06T14:30:00.000Z" ### Files Affected -| File | Change | -| ----------------------------------------- | --------------------------------------------------------------- | -| `setup/profile.ts` | detects FreeBSD locale/timezone and writes onboarding profile | -| `setup.sh:13` | bootstrap logs use `format_display_timestamp_now` | -| `src/display-date.ts` | user-facing display uses configured locale/timezone helpers | -| `scripts/memory/*.sh` | shell-side display uses shared locale/timezone helper | +| File | Change | +| --------------------- | ------------------------------------------------------------- | +| `setup/profile.ts` | detects FreeBSD locale/timezone and writes onboarding profile | +| `setup.sh:13` | bootstrap logs use `format_display_timestamp_now` | +| `src/display-date.ts` | user-facing display uses configured locale/timezone helpers | +| `scripts/memory/*.sh` | shell-side display uses shared locale/timezone helper | ### Files NOT Affected @@ -257,6 +257,7 @@ date '+%d.%m.%Y-%H%M' ``` Examples: + - `zroot/clawdie-runtime/jails/clawdie-worker@setup-complete-10.mar.2026` - `zroot/clawdie-runtime/jails/clawdie-worker@pre-update-10.mar.2026-1430` - `zroot/clawdie-runtime/jails/clawdie-browser-vm@pre-upgrade-08.mar.2026` @@ -264,6 +265,7 @@ Examples: ### Exceptions **Automated snapshots (Sanoid)** use ISO format and should NOT be changed: + - `autosnap_2026-03-10_04:00:00_hourly` - machine-generated, machine-consumed ### Rationale @@ -299,10 +301,10 @@ Inside the file, use **European format** for user-facing dates: ### Rationale -| Aspect | Format | Reason | -|--------|--------|--------| -| Filename | ISO (`YYYY-MM-DD`) | Sortable in directory listings | -| Content | `DD.mmm.YYYY` | User-facing, matches AGENTS.md convention | +| Aspect | Format | Reason | +| -------- | ------------------ | ----------------------------------------- | +| Filename | ISO (`YYYY-MM-DD`) | Sortable in directory listings | +| Content | `DD.mmm.YYYY` | User-facing, matches AGENTS.md convention | ### Do Not Mix @@ -313,19 +315,19 @@ Never use ISO dates in user-facing content, and never use European dates in file Use different language depending on audience. The rule: **operator-facing content** uses humanized service names; **code, config, and developer docs** use exact technical terms. -| Context | Use | -|---|---| +| Context | Use | +| --------------------------------------------------- | ---------------------------------------------------------------------- | | HTML docs, README, setup messages, user-facing logs | data service, web publishing service, code service, worker environment | -| TypeScript, setup scripts, config, developer docs | jail, db jail, cms jail, git jail, pf, zfs, bastille, hostd | +| TypeScript, setup scripts, config, developer docs | jail, db jail, cms jail, git jail, pf, zfs, bastille, hostd | Mapping: -| Operator name | Technical term | -|---|---| -| **Data Service** | `db jail` | -| **Web Service** | `cms jail` | -| **Code Service** | `git jail` | -| **Worker** | worker jail | +| Operator name | Technical term | +| ---------------- | -------------- | +| **Data Service** | `db jail` | +| **Web Service** | `cms jail` | +| **Code Service** | `git jail` | +| **Worker** | worker jail | In prose, lead with the capitalized name and follow with the technical term in parentheses on first use: @@ -335,7 +337,7 @@ The Data Service (db jail) runs PostgreSQL at 10.0.0.3. In subsequent references within the same page or section, either form is fine. -Rationale: `jail`, `pf`, `zfs`, `bastille` are what the runtime actually uses in code and logs. Changing them in code creates confusion. But operator-facing docs should describe what a service *does*, not its implementation container type. +Rationale: `jail`, `pf`, `zfs`, `bastille` are what the runtime actually uses in code and logs. Changing them in code creates confusion. But operator-facing docs should describe what a service _does_, not its implementation container type. When in doubt: if the word appears in a TypeScript import, a Bastille command, or a config key, keep it technical. If it appears in a sentence a human reads before they know what Bastille is, use the humanized form. @@ -461,11 +463,11 @@ of what was passing at commit time. Run `npm run release` only when a **minor or major** version bump has been committed and pushed — never on patch bumps or regular commits. -| Version change | Tag? | -|---|---| +| Version change | Tag? | +| ------------------------- | ----------------------- | | `0.8.0` → `0.9.0` (minor) | ✓ run `npm run release` | | `0.8.0` → `1.0.0` (major) | ✓ run `npm run release` | -| `0.8.0` → `0.8.1` (patch) | ✗ skip | +| `0.8.0` → `0.8.1` (patch) | ✗ skip | The script reads the version from `package.json`, creates an annotated git tag, and pushes it to Codeberg. This is what makes the version appear in the @@ -528,12 +530,12 @@ appear as `_`) in every terminal session until manually corrected. Strip any existing charset suffix from the detected locale tag, then append `.UTF-8`. For example: -| Detected | Written | -|----------|---------| -| `sl_SI` | `sl_SI.UTF-8` | +| Detected | Written | +| ----------------- | ------------- | +| `sl_SI` | `sl_SI.UTF-8` | | `sl_SI.ISO8859-2` | `sl_SI.UTF-8` | -| `sl_SI.UTF-8` | `sl_SI.UTF-8` | -| `en_US.UTF-8` | `en_US.UTF-8` | +| `sl_SI.UTF-8` | `sl_SI.UTF-8` | +| `en_US.UTF-8` | `en_US.UTF-8` | ### After writing `~/.login_conf` @@ -550,21 +552,23 @@ environment. Agents working on this repo must identify themselves for attribution and handoff tracking. -| Identity | Platform | Capabilities | Restrictions | -|----------|----------|-------------|--------------| -| Claude (Linux) | Linux dev machine | Code search, file editing, git push | Cannot run tests (Linux), cannot build ISO, cannot run FreeBSD commands | -| Codex / FreeBSD agent | FreeBSD host | Runs tests, builds ISO, validates on real system | Receives handoff docs, cannot push without operator review | -| Operator (Sam) | FreeBSD host + Linux | Final review, merge decisions, ISO builds, deploys | Human — all commits require approval | +| Identity | Platform | Capabilities | Restrictions | +| --------------------- | -------------------- | -------------------------------------------------- | ----------------------------------------------------------------------- | +| Claude (Linux) | Linux dev machine | Code search, file editing, git push | Cannot run tests (Linux), cannot build ISO, cannot run FreeBSD commands | +| Codex / FreeBSD agent | FreeBSD host | Runs tests, builds ISO, validates on real system | Receives handoff docs, cannot push without operator review | +| Operator (Sam) | FreeBSD host + Linux | Final review, merge decisions, ISO builds, deploys | Human — all commits require approval | ### Linux Agent Constraints The Linux agent (Claude/Codex on Linux) MUST NOT: + - Attempt to build the ISO (`./build.sh`) - Run `npm test` or `npx vitest` (unreliable on Linux) - Run any FreeBSD-specific commands (`bastille`, `service`, `pkg`, `jls`) - Assume packages or runtime behavior matches FreeBSD The Linux agent SHOULD: + - Write code, tests, and documentation - Push to Codeberg for the FreeBSD agent to validate - Create handoff docs for FreeBSD-specific tasks @@ -573,6 +577,7 @@ The Linux agent SHOULD: ### FreeBSD Agent Constraints The FreeBSD agent MUST: + - Read and follow any active `doc/*-HANDOFF.md` files on startup - Check off tasks as they complete them - Report test results in the handoff doc or commit message @@ -584,7 +589,7 @@ The FreeBSD agent MUST: The control plane now pivots to a terminal-first harness (extensions + safety). -- Do not build or deploy Paperclip UI. +- The controlplane UI is agent-generated dashboards, not a standalone UI app. - Operator access stays via the control plane API at `http://127.0.0.1:3100` (or nginx TLS for remote access). - PF should allow `tailscale0` ingress to `3100` (direct API) and `443` (nginx proxy) when testing from the tailnet. @@ -594,9 +599,9 @@ The control plane now pivots to a terminal-first harness (extensions + safety). Clawdie spans two repos. Changes often require coordinated updates. -| Repo | Purpose | Remote | -|------|---------|--------| -| `Clawdie-AI` | Agent runtime, control plane, channels | `git@codeberg.org:Clawdie/Clawdie-AI.git` | +| Repo | Purpose | Remote | +| ------------- | ---------------------------------------- | ------------------------------------------ | +| `Clawdie-AI` | Agent runtime, control plane, channels | `git@codeberg.org:Clawdie/Clawdie-AI.git` | | `clawdie-iso` | ISO builder, firstboot wizard, installer | `git@codeberg.org:Clawdie/Clawdie-ISO.git` | ### When Changes Span Both Repos @@ -705,6 +710,7 @@ The receiving agent: Use attribution in commit messages, not in code comments. Labels: + - `Sam & Codex` — changes made by Sam and Codex - `Sam & Claude` — changes made by Sam and Claude - `C&C` — joint change with equal credit for Claude and Codex diff --git a/README_zh.md b/README_zh.md index 000a9fe..b46acb5 100644 --- a/README_zh.md +++ b/README_zh.md @@ -125,60 +125,52 @@ claude ## 架构 ``` -WhatsApp (baileys) --> SQLite --> 轮询循环 --> 容器 (Claude Agent SDK) --> 响应 +Telegram + 计划任务 + IPC + ↓ +PostgreSQL ops 数据库(消息、群组、任务、会话) + ↓ +主机编排器 (Node.js) + ↓ +FreeBSD worker jail(s) + ↓ +智能体响应 ``` -单一 Node.js 进程。智能体在具有挂载目录的隔离 Linux 容器中执行。每个群组的消息队列都带有全局并发控制。通过文件系统进行进程间通信(IPC)。 +单一 Node.js 进程。智能体在通过 nullfs 挂载实现文件系统隔离的 FreeBSD jail 中执行。每个群组拥有独立的消息队列和并发控制。PostgreSQL 部署在专用的 `{agent}-db` jail 中,以三库分离模式运行:系统技能库(预加载,只读)、用户/智能体记忆库(动态增长)、操作状态库(消息、任务、会话、路由)。 关键文件: - `src/index.ts` - 编排器:状态管理、消息循环、智能体调用 -- `src/channels/whatsapp.ts` - WhatsApp 连接、认证、收发消息 +- `src/channels/telegram.ts` - 内置 Telegram 通道 - `src/ipc.ts` - IPC 监听与任务处理 -- `src/router.ts` - 消息格式化与出站路由 - `src/group-queue.ts` - 各带全局并发限制的群组队列 -- `src/container-runner.ts` - 生成流式智能体容器 +- `src/agent-runner.ts` - 在 jail 中运行智能体进程 - `src/task-scheduler.ts` - 运行计划任务 - `src/db.ts` - PostgreSQL ops 操作(消息、群组、会话、状态、路由) -- `groups/*/AGENT.md` - 各群组的记忆 +- `setup/*.ts` - 模块化安装入口 +- `groups/*/AGENTS.md` - 各群组的记忆 ## FAQ -**为什么是 WhatsApp 而不是 Telegram/Signal 等?** +**为什么使用 Telegram 而不是 WhatsApp?** -因为我用 WhatsApp。Fork 这个项目然后运行一个技能来改变它。正是这个项目的核心理念。 +Telegram 的 Bot API 原生支持、文档完善、无需手机配对。Fork 此项目并运行技能即可添加其他通道。 -**为什么是 Docker?** +**为什么使用 FreeBSD Jails 而不是 Docker?** -Docker 提供跨平台支持(macOS 和 Linux)和成熟的生态系统。在 macOS 上,您可以选择通过运行 `/convert-to-apple-container` 切换到 Apple Container,以获得更轻量级的原生运行时体验。 - -**我可以在 Linux 上运行吗?** - -可以。Docker 是默认的容器运行时,在 macOS 和 Linux 上都可以使用。只需运行 `/setup`。 +FreeBSD Jails 提供原生性能、更简单的架构(更少层次)、与 ZFS 的原生集成、以及出色的安全性。 **这个项目安全吗?** -智能体在容器中运行,而不是在应用级别的权限检查之后。它们只能访问被明确挂载的目录。您仍然应该审查您运行的代码,但这个代码库小到您真的可以做到。完整的安全模型请见 [docs/public/operate/security.md](docs/public/operate/security.md)。 +智能体在 FreeBSD jail 中运行,通过 nullfs 挂载实现文件系统隔离,而非依赖应用级权限检查。智能体只能访问明确挂载的目录。完整的安全模型请见 [docs/public/operate/security.md](docs/public/operate/security.md)。 **为什么没有配置文件?** -我们不希望配置泛滥。每个用户都应该定制它,让代码完全符合他们的需求,而不是去配置一个通用的系统。如果您喜欢用配置文件,告诉 Claude 让它加上。 +我们不希望配置泛滥。每个用户都应该定制它,让代码完全符合他们的需求,而不是去配置一个通用的系统。 **我该如何调试问题?** -问 Claude Code。"为什么计划任务没有运行?" "最近的日志里有什么?" "为什么这条消息没有得到回应?" 这就是 AI 原生的方法。 - -**为什么我的安装不成功?** - -我不知道。运行 `claude`,然后运行 `/debug`。如果 Claude 发现一个可能影响其他用户的问题,请开一个 PR 来修改 `SKILL.md` 安装文件。 - -**什么样的代码更改会被接受?** - -安全修复、bug 修复,以及对基础配置的明确改进。仅此而已。 - -其他一切(新功能、操作系统兼容性、硬件支持、增强功能)都应该作为技能来贡献。 - -这使得基础系统保持最小化,并让每个用户可以定制他们的安装,而无需继承他们不想要的功能。 +问智能体。"为什么计划任务没有运行?" "最近的日志里有什么?" 这就是 AI 原生的方法。也可以运行 `npm run doctor` 检查系统健康状态。 ## 社区 diff --git a/doc/CONTROLPLANE-AIDER-PI.md b/doc/CONTROLPLANE-AIDER-PI.md index 895507a..7818b1a 100644 --- a/doc/CONTROLPLANE-AIDER-PI.md +++ b/doc/CONTROLPLANE-AIDER-PI.md @@ -48,12 +48,12 @@ nginx serves as a static site. This replaces Paperclip as the controlplane UI. ## What Changes (Summary) -- Paperclip UI build/deploy paths become obsolete. +- Agent-generated dashboards replace the standalone UI model. - Controlplane UI is generated by agents, not a standalone app. - Controlplane API stays intact and is not replaced. ## Next Steps 1. Update install and README docs to state Aider+Pi as the primary harness. -2. Remove Paperclip-specific build steps from setup/verify. -3. Add a controlplane doc page describing the dashboard output contract. +2. Write a controlplane doc page describing the dashboard output contract. +3. Implement agent dashboard generation in the controlplane runner. diff --git a/docs/internal/POSTGRES-PERMISSIONS.md b/docs/internal/POSTGRES-PERMISSIONS.md index 105f167..e2d76d9 100644 --- a/docs/internal/POSTGRES-PERMISSIONS.md +++ b/docs/internal/POSTGRES-PERMISSIONS.md @@ -12,6 +12,7 @@ The application connects as a separate user (e.g., `clawdie_brain`). Without explicit GRANT statements, the app user cannot INSERT, SELECT, or UPDATE. Error example: + ``` DatabaseError: permission denied for table memories at storeMemory (dist/memory-pg.js:93:22) @@ -22,12 +23,14 @@ DatabaseError: permission denied for table memories ## Solution: Two-Layer Permission Model ### 1. Existing Tables (one-time fix) + ```sql GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO :db_user; GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO :db_user; ``` ### 2. Future Tables (permanent fix) + ```sql ALTER DEFAULT PRIVILEGES FOR ROLE postgres IN SCHEMA public @@ -40,12 +43,12 @@ This ensures new tables created by migrations automatically inherit permissions. ## Files Changed -| File | Purpose | -|------|---------| -| `docs/internal/sql/permissions.sql` | Core permission logic (GRANT + DEFAULT PRIVILEGES) | -| `docs/internal/scripts/fix-permissions.sh` | Quick fix script for broken databases | -| `docs/internal/scripts/setup-db-jail.sh` | Runs permissions.sql after migrations | -| `setup/skills-memory.ts` | Applies permissions after artifact import | +| File | Purpose | +| ------------------------------------------ | -------------------------------------------------- | +| `docs/internal/sql/permissions.sql` | Core permission logic (GRANT + DEFAULT PRIVILEGES) | +| `docs/internal/scripts/fix-permissions.sh` | Quick fix script for broken databases | +| `docs/internal/scripts/setup-db-jail.sh` | Runs permissions.sql after migrations | +| `setup/skills-memory.ts` | Applies permissions after artifact import | --- @@ -76,16 +79,17 @@ The setup process now includes permissions: --- -## Two Databases, One Pattern +## Three Databases, One Pattern -Clawdie uses two databases in the db jail: +Clawdie uses three databases in the db jail: -| Database | Purpose | User | Permissions | -|----------|---------|------|-------------| -| `clawdie_brain` | User memories, agent state | `clawdie_brain` | `permissions.sql` | -| `clawdie_skills` | Built-in knowledge | `clawdie_skills` | `permissions.sql` | +| Database | Purpose | User | Permissions | +| ---------------- | ----------------------------------------------------- | ---------------- | ----------------- | +| `clawdie_brain` | User memories, agent state | `clawdie_brain` | `permissions.sql` | +| `clawdie_skills` | Built-in knowledge | `clawdie_skills` | `permissions.sql` | +| `clawdie_ops` | Operational data (messages, tasks, sessions, routing) | `clawdie_ops` | `permissions.sql` | -Both databases run through the same permission flow. +All three databases run through the same permission flow. --- @@ -108,6 +112,7 @@ DELETE FROM memories WHERE summary = 'test permission'; ``` Expected output: + ``` grantee | table_name | privilege_type ----------------+-----------------------+---------------- @@ -123,13 +128,13 @@ Expected output: ## Why This Approach? -| Aspect | Why It Matters | -|--------|----------------| -| **DEFAULT PRIVILEGES** | New migrations don't require permission updates | -| **Idempotent** | Running permissions.sql twice is safe | -| **Standard pattern** | Same approach as Rails, Django, TypeORM | -| **Single source of truth** | Permissions defined in one file | -| **Easy debugging** | `psql -c '\ddp'` shows all grants | +| Aspect | Why It Matters | +| -------------------------- | ----------------------------------------------- | +| **DEFAULT PRIVILEGES** | New migrations don't require permission updates | +| **Idempotent** | Running permissions.sql twice is safe | +| **Standard pattern** | Same approach as Rails, Django, TypeORM | +| **Single source of truth** | Permissions defined in one file | +| **Easy debugging** | `psql -c '\ddp'` shows all grants | --- @@ -138,6 +143,7 @@ Expected output: ### "permission denied for table X" Run fix script: + ```bash ./docs/internal/scripts/fix-permissions.sh ``` @@ -145,6 +151,7 @@ Run fix script: ### "role X does not exist" Create the user first: + ```sql CREATE USER clawdie_brain WITH PASSWORD '...' LOGIN; ``` @@ -152,6 +159,7 @@ CREATE USER clawdie_brain WITH PASSWORD '...' LOGIN; ### "database X does not exist" Create the database first: + ```sql CREATE DATABASE clawdie_brain OWNER clawdie_brain; ``` @@ -159,6 +167,7 @@ CREATE DATABASE clawdie_brain OWNER clawdie_brain; ### "must be owner of table X" You're connected as the wrong user. Connect as `postgres` superuser: + ```bash sudo -u postgres psql -d clawdie_brain ``` diff --git a/docs/internal/REFACTOR-PLAN.md b/docs/internal/REFACTOR-PLAN.md index e81b642..9316c29 100644 --- a/docs/internal/REFACTOR-PLAN.md +++ b/docs/internal/REFACTOR-PLAN.md @@ -38,21 +38,21 @@ firstboot wizard already does that job better, in real shell. ### What this eliminates -| Component | Lines | Why it's dead | -|-----------|-------|---------------| -| TUI setup wizard (`setup-wizard.ts`) | 1,698 | clawdie-iso firstboot does this | -| Screenshot wizard (`screenshot-wizard.ts`) | 666 | Lumina has native screenshot | -| ANSI-to-HTML converter (`ansi-to-html.ts`) | 355 | Only existed for TUI screenshots | -| Browser VM jail profile + skill | ~200 | Lumina has native Chromium/Firefox | -| bhyve VMM detection in wizard | ~80 | No Linux VM needed | -| Warden-specific skills (5 skills) | ~1,500 MD | Jails no longer primary runtime | +| Component | Lines | Why it's dead | +| ------------------------------------------ | --------- | ---------------------------------- | +| TUI setup wizard (`setup-wizard.ts`) | 1,698 | clawdie-iso firstboot does this | +| Screenshot wizard (`screenshot-wizard.ts`) | 666 | Lumina has native screenshot | +| ANSI-to-HTML converter (`ansi-to-html.ts`) | 355 | Only existed for TUI screenshots | +| Browser VM jail profile + skill | ~200 | Lumina has native Chromium/Firefox | +| bhyve VMM detection in wizard | ~80 | No Linux VM needed | +| Warden-specific skills (5 skills) | ~1,500 MD | Jails no longer primary runtime | **Total removable: ~3,000 lines TS + ~1,500 lines skill docs** ### What stays - Core orchestrator (`index.ts`, `group-queue.ts`, `task-scheduler.ts`) -- SQLite + PostgreSQL memory +- PostgreSQL memory (skills, memory, ops) - Telegram channel (grammy) - IPC system - Health monitoring (simplified) @@ -132,6 +132,7 @@ src/jail-config.ts 358 lines — jail profiles Replace with a single `src/agent-runner.ts` modeled on NanoClaw's `container-runner.ts` (707 lines → our simpler version ~200): + - Spawns `pi` as a subprocess (no tmux pane needed for daemon mode) - Passes secrets via env vars directly (no jail stdin trick) - Same IPC file protocol (file-based, no changes to IPC layer) @@ -176,52 +177,52 @@ export function cleanupOrphans() { ... } ### Skills to DROP (jail/infra-specific, replaced by clawdie-iso) -| Skill | Reason | -|-------|--------| -| `warden-bootstrap` | clawdie-iso firstboot does this | -| `warden-health` | Jails gone | -| `warden-pf` | PF rules for jail networking | -| `warden-zfs` | ZFS for jail snapshots (see Appendix A) | -| `bastille-network` | Bastille jail manager | -| `browser-vm` | Lumina has native browser | -| `nginx-glasspane` | Rethink for Lumina | -| `tmux-screenshot` | Lumina has native screenshot | +| Skill | Reason | +| ------------------ | --------------------------------------- | +| `warden-bootstrap` | clawdie-iso firstboot does this | +| `warden-health` | Jails gone | +| `warden-pf` | PF rules for jail networking | +| `warden-zfs` | ZFS for jail snapshots (see Appendix A) | +| `bastille-network` | Bastille jail manager | +| `browser-vm` | Lumina has native browser | +| `nginx-glasspane` | Rethink for Lumina | +| `tmux-screenshot` | Lumina has native screenshot | ### Skills to KEEP (still relevant) -| Skill | Notes | -|-------|-------| -| `ansible-freebsd` | Host provisioning on top of clawdie-iso | -| `coding-agent` | Core: configures pi agent engine and API keys | -| `nginx` | Still need reverse proxy | -| `ollama` | Local LLM, runs on host (great upstream candidate) | +| Skill | Notes | +| ----------------- | -------------------------------------------------- | +| `ansible-freebsd` | Host provisioning on top of clawdie-iso | +| `coding-agent` | Core: configures pi agent engine and API keys | +| `nginx` | Still need reverse proxy | +| `ollama` | Local LLM, runs on host (great upstream candidate) | | `postgres-memory` | Supabase memory backend (great upstream candidate) | -| `telegram-admin` | Telegram bot management | -| `freebsd-admin` | Host sysadmin_agent tasks | -| `sanoid` | ZFS snapshots — now must cover `/home/clawdie` too | +| `telegram-admin` | Telegram bot management | +| `freebsd-admin` | Host sysadmin_agent tasks | +| `sanoid` | ZFS snapshots — now must cover `/home/clawdie` too | ### Skills to PORT FROM NanoClaw (we're missing) -| Skill | What it does | -|-------|-------------| -| `add-compact` | Compact mode for cheaper API usage | -| `add-image-vision` | Image analysis via Claude vision | -| `add-parallel` | Parallel agent execution | -| `add-pdf-reader` | PDF ingestion | -| `add-reactions` | Emoji reactions to messages | -| `add-voice-transcription` | Voice message → text | -| `use-local-whisper` | Local Whisper for voice (aligns with our ollama skill) | -| `update-skills` | Pull latest skills from upstream | +| Skill | What it does | +| ------------------------- | ------------------------------------------------------ | +| `add-compact` | Compact mode for cheaper API usage | +| `add-image-vision` | Image analysis via Claude vision | +| `add-parallel` | Parallel agent execution | +| `add-pdf-reader` | PDF ingestion | +| `add-reactions` | Emoji reactions to messages | +| `add-voice-transcription` | Voice message → text | +| `use-local-whisper` | Local Whisper for voice (aligns with our ollama skill) | +| `update-skills` | Pull latest skills from upstream | ### Skills we could UPSTREAM to NanoClaw -| Skill | Value to upstream | -|-------|------------------| -| `postgres-memory` | Persistent memory beyond SQLite — high demand | -| `ollama` | Local LLM — popular request | -| `freebsd-admin` | Expands platform support | -| `ansible-freebsd` | Infrastructure-as-code pattern | -| `coding-agent` | Pi agent engine configuration | +| Skill | Value to upstream | +| ----------------- | ------------------------------- | +| `postgres-memory` | Persistent memory — high demand | +| `ollama` | Local LLM — popular request | +| `freebsd-admin` | Expands platform support | +| `ansible-freebsd` | Infrastructure-as-code pattern | +| `coding-agent` | Pi agent engine configuration | **Best upstream candidates:** `postgres-memory` and `ollama` — platform-agnostic, add real value, no FreeBSD-specific code. @@ -238,6 +239,7 @@ multi-channel registry (`src/channels/registry.ts`). ### NanoClaw's approach Channels are **skill branches** merged via git. Each channel: + 1. Adds `src/channels/{name}.ts` implementing `Channel` interface 2. Self-registers via class-based pattern in `src/channels/registry.ts` 3. Auto-enables when its env var is present (e.g., `TELEGRAM_BOT_TOKEN`) @@ -249,6 +251,7 @@ registry costs ~100 lines and keeps us upstream-compatible. Discord, Slack, WhatsApp become `/add-discord` away. Action items: + 1. Port `src/channels/registry.ts` from NanoClaw 2. Refactor `src/channels/telegram.ts` to implement `Channel` interface 3. Remove hardcoded Telegram init from `src/index.ts` @@ -264,6 +267,7 @@ Note: clawdie-iso's `shell-env.sh` already writes `TELEGRAM_BOT_TOKEN` to ### Phase 0: Snapshot before anything (NEW — see Appendix A) Add `zroot/home/clawdie` to sanoid config before touching anything: + ```ini [zroot/home/clawdie] use_template = production @@ -279,7 +283,7 @@ Add `zroot/home/clawdie` to sanoid config before touching anything: 1. Delete TUI wizard + screenshot wizard + ansi-to-html 2. Delete jail-specific setup modules (network.ts, jail.ts) 3. Delete jail runtime files (jail-runner, jail-runtime, jail-ops, jail-config) -4. Delete dead skills (warden-*, bastille-*, browser-vm, tmux-screenshot) +4. Delete dead skills (warden-_, bastille-_, browser-vm, tmux-screenshot) 5. Delete `container/` directory (legacy Docker) **Result:** ~3,500 lines removed, project compiles but can't run agents @@ -323,17 +327,17 @@ Add `zroot/home/clawdie` to sanoid config before touching anything: ## Summary -| Metric | Before | After | Change | -|--------|--------|-------|--------| -| TS source lines | ~10,500 | ~6,500 | -38% | -| Setup lines | ~4,300 | ~800 | -81% | -| Runtime files | 4 (jail) | 2 (process) | -50% | -| Skills | 34 | ~22 | -35% | -| Dependencies | 8 | 7 | -12% | -| Install method | TUI wizard | `/setup` skill | Aligned with upstream | -| Machine setup | DIY jails | clawdie-iso firstboot | Done before agent starts | -| Agent isolation | FreeBSD jails | Process + permissions | Simpler | -| Browser | Future bhyve VM | Native Lumina | Done | +| Metric | Before | After | Change | +| --------------- | --------------- | --------------------- | ------------------------ | +| TS source lines | ~10,500 | ~6,500 | -38% | +| Setup lines | ~4,300 | ~800 | -81% | +| Runtime files | 4 (jail) | 2 (process) | -50% | +| Skills | 34 | ~22 | -35% | +| Dependencies | 8 | 7 | -12% | +| Install method | TUI wizard | `/setup` skill | Aligned with upstream | +| Machine setup | DIY jails | clawdie-iso firstboot | Done before agent starts | +| Agent isolation | FreeBSD jails | Process + permissions | Simpler | +| Browser | Future bhyve VM | Native Lumina | Done | --- @@ -361,6 +365,7 @@ The `ZAI_API_KEY` was still in `.env` but pi reads from `auth.json` first. ### Why ZFS couldn't save us Sanoid was snapshotting **jail datasets only**: + ``` zroot/clawdie-runtime/jails/controlplane — hourly ✓ zroot/clawdie-runtime/jails/db — hourly ✓ @@ -372,10 +377,11 @@ No rollback possible for `~/.pi/agent/`. Diagnosis took ~20 min. ### Fix applied (27.Mar.2026) Restored `auth.json` with both keys: + ```json { "ollama": { "type": "api_key", "key": "c1**ab**...sU*Z**V*..." }, - "zai": { "type": "api_key", "key": "9d**d8**c7*b4**...Jt*Q9*..." } + "zai": { "type": "api_key", "key": "9d**d8**c7*b4**...Jt*Q9*..." } } ``` @@ -387,12 +393,14 @@ AI. Local llama-cpp replaces it — see [`LOCAL-LLM.md`](./LOCAL-LLM.md). Current direction: pi points at `http://10.0.0.5:8081/v1` (local llama-server chat instance). Model choice scales with available RAM: + - 12 GB: `dolphin3.0-phi4-mini` Q4_K_M (~2.4 GB) — stepping stone - 32 GB: `Qwen3-14B` Q4_K_M (~9 GB) — target for reliable tool use + HEARTBEAT ### Action items baked into Phase 0 Before any refactor work starts: + 1. Add `zroot/home/clawdie` to sanoid (daily + hourly snapshots) 2. `/setup` skill must **append** to `auth.json`, never overwrite it 3. `/setup` skill must snapshot `zroot/home/clawdie` before touching pi config diff --git a/docs/internal/SKILLS-ARTIFACT-V1-PLAN.md b/docs/internal/SKILLS-ARTIFACT-V1-PLAN.md index d542b83..7fbf916 100644 --- a/docs/internal/SKILLS-ARTIFACT-V1-PLAN.md +++ b/docs/internal/SKILLS-ARTIFACT-V1-PLAN.md @@ -13,6 +13,7 @@ whenever a source file changes. Embeddings are expensive to generate, take space in the repo, and create a false sense of stability. **Trigger for v1 artifact generation:** + - Install flow is stable (install orchestrator written and tested) - Source allowlist is confirmed and frozen for v1 - Docs for FreeBSD setup, install, split-brain, and FreeBSD guides @@ -27,8 +28,8 @@ development. ## Architecture decision — PostgreSQL path for skills -The runtime already relies on PostgreSQL + pgvector. We keep skills and memory -in the same db jail and remove the SQLite/sqlite-vec path entirely. +The runtime already relies on PostgreSQL + pgvector. We keep skills, memory, +and ops in the same db jail. ``` Target: @@ -40,7 +41,7 @@ Target: **Rationale:** -- Avoids SQLite native bindings and sqlite-vec dependencies. +- Single database engine (PostgreSQL + pgvector), no SQLite dependencies. - Keeps a single vector query path (pgvector) for skills + memory. - Uses the existing db jail, so no new service footprint is introduced. - Aligns with the long-term architecture and backup strategy. @@ -63,18 +64,19 @@ with any metadata tables defined by `docs/internal/BUILTIN-KNOWLEDGE-SPEC.md`. Clawdie v1 knowledge base scope: -| Source | Est. chunks | -|---|---| -| FreeBSD setup guide | ~80 | -| Install guide | ~60 | -| Split-brain docs | ~40 | -| Nginx + SSL guide | ~50 | -| Tailscale guide | ~40 | -| .agent/skills/ | ~200 | -| docs/public/ + docs/internal/ (relevant files) | ~200 | -| **Total** | **~670 chunks** | +| Source | Est. chunks | +| ---------------------------------------------- | --------------- | +| FreeBSD setup guide | ~80 | +| Install guide | ~60 | +| Split-brain docs | ~40 | +| Nginx + SSL guide | ~50 | +| Tailscale guide | ~40 | +| .agent/skills/ | ~200 | +| docs/public/ + docs/internal/ (relevant files) | ~200 | +| **Total** | **~670 chunks** | Storage (pg_dump, plain SQL): + ``` 670 chunks × 1536 dims × 4 bytes = ~4 MB raw embeddings + text content = ~1 MB @@ -96,6 +98,7 @@ The operator installing Clawdie NEVER runs this. This is a maintainer step. 4. Commit the SQL dump. The build script (to be written at v1) needs: + - an embedding API key on the maintainer machine - `python3.11` or `node` tooling (implementation choice) - PostgreSQL + pgvector locally available @@ -107,18 +110,22 @@ No key is needed on the operator's machine at install time. ## What changes in the codebase (deferred to v1) ### 1. `setup/skills-memory.ts` + Current: imports SQL artifact into PostgreSQL (skills DB). -Target: keep this path — no SQLite import path. +Target: keep this path — PostgreSQL only. ### 2. `src/builtin-knowledge-artifact.ts` + Current: PostgreSQL lookup for skills retrieval. -Target: keep this path — no SQLite query path. +Target: keep this path — PostgreSQL only. ### 3. `setup/db.ts` + Current: creates `{agent}_skills`, `{agent}_memory`, `{agent}_ops` databases. Target: keep this path and ensure the skills DB is available before import. ### 4. `scripts/build-skills-artifact.*` (new, to be written at v1) + Chunker + embedder + `pg_dump`. One-time maintainer script, not part of the operator install path. diff --git a/html/docs-clawdie-si/docs/install.html b/html/docs-clawdie-si/docs/install.html index 2d14f32..1524a26 100644 --- a/html/docs-clawdie-si/docs/install.html +++ b/html/docs-clawdie-si/docs/install.html @@ -386,8 +386,8 @@ npm run setup -- --step skills-memory --import .3 Data Service (db jail) — split-brain - PostgreSQL for built-in knowledge, memory, and the internal - Strapi database