diff --git a/README.md b/README.md index 4d7f8b3..10b9954 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,11 @@ Colibri, Codex, Claude Code, Zot, or future harnesses. The repository carries durable identity, reviewed user context, curated memories, and approved skills without copying a full runtime backup. +It is both the **template** and a **working reference**: the identity files, +curated memories, and skills here are real, usable content — not placeholders. +Spin off `-layered-soul` instances from it when you need a separate +identity bundle (see "Template usage"). + ## Why this exists Agent runtimes already keep useful state, but their native backups are not safe diff --git a/docs/CONNECT-HERMES-SOUL.md b/docs/CONNECT-HERMES-SOUL.md index 713f163..2c1085d 100644 --- a/docs/CONNECT-HERMES-SOUL.md +++ b/docs/CONNECT-HERMES-SOUL.md @@ -39,7 +39,7 @@ Do not mirror `hermes-soul` into `layered-soul`. Use a review/export workflow. - Reviewed user context from `memories/USER.md`. - Reviewed memory summaries from `memories/MEMORY.md`. -- Selected `skills/**/*.md` that are safe and useful across harnesses. +- Selected `skills/**/SKILL.md` (with their `references/`) that are safe and useful across harnesses. - Converted cron/task ideas after they are rewritten as explicit Ops manifests. ## What must stay private diff --git a/docs/PLAN-CONFIGURE-PRIVATE-REPO.md b/docs/PLAN-CONFIGURE-PRIVATE-REPO.md deleted file mode 100644 index 03f60f6..0000000 --- a/docs/PLAN-CONFIGURE-PRIVATE-REPO.md +++ /dev/null @@ -1,207 +0,0 @@ -# Plan: Configure layered-soul for Private Repo - -## State Analysis (2026-06-11) - -### What we have - -| Component | Status | Details | -| ---------------------------- | ---------------- | ----------------------------------------------------------------------- | -| `hermes-soul` | ✅ Working | Private backup — 539 skills, 136 sessions, 2 memories, config sanitized | -| `layered-soul` scaffolding | ✅ Complete | Schema, validation, 6 adapters, private-source planner | -| `layered-soul` content | ✅ Populated | SOUL/IDENTITY/USER/AGENTS.md filled, 5 curated memories, 9 skills | -| Colibri skills import | ✅ Working | `import-layered-soul.sh` imports skills into flat `skills` table | -| Colibri system_brain | ❌ Not built | `memories/curated/` detected but no DB table | -| Colibri system_ops | ❌ Not built | No schema, no manifest format | -| Colibri system_skills (full) | 📋 Planned | Schema designed, Phase 2-7 not implemented | -| Headroom-ai | ✅ Installed | 0.25.0 on debby, 40%+ savings, integration planned in 3.4 | -| Clawdie installer hardening | ✅ Merged PR #53 | Non-root daemon, ownership enforcement, pool validation | - -### Architecture - -```text - ┌──────────────────────────────────┐ - │ hermes-soul (PRIVATE) │ - │ sessions/ config/ cron/ │ - │ memories/ skills/ scripts/ │ - └──────────┬───────────────────────┘ - │ plan-private-source - │ (review, never copies secrets) - ▼ - ┌──────────────────────────────────┐ - │ layered-soul (PORTABLE) │ - │ SOUL.md USER.md IDENTITY.md │ - │ memories/curated/ skills/ │ - │ adapters/ manifest.json │ - └────┬──────────┬──────────┬───────┘ - │ │ │ - Hermes Colibri Pi/Codex/Claude/Zot - (context) (import) (AGENTS.md) -``` - -## Phase 1: Populate the Soul (operator + Hermes together) - -### 1.1 Fill SOUL.md - -Replace the empty SOUL.md with actual agent identity — what Layered Soul IS, its values, how it operates. Hermes can draft; operator reviews. - -### 1.2 Fill IDENTITY.md - -Short runtime identity — agent name, home harness, boundaries, contact. 3-4 lines. - -### 1.3 Import USER.md context - -Run the private-source planner, review hermes-soul's USER.md, write sanitized summaries into layered-soul's USER.md. Keep Hermes-native detail in hermes-soul; portable context in layered-soul. - -```sh -python3 scripts/layered_soul.py plan-private-source \ - examples/private-sources/hermes-soul.example.json \ - --source-root ~/hermes-soul -``` - -### 1.4 Curate memories - -Select 3-5 durable memory entries from Hermes that matter across harnesses (Tailscale layout, Forgejo setup, agent identities, project structure). Write each as a `memories/curated/.md`. - -```sh -# Create curated memories -# memories/curated/tailscale-network.md — Tailscale IPs and host roles -# memories/curated/forgejo-setup.md — Forgejo repo layout, SSH keys -# memories/curated/agent-roster.md — Which agents run where -# memories/curated/project-structure.md — Clawdie repos and their purposes -``` - -### 1.5 Select cross-harness skills - -From 539 Hermes skills, pick the ones useful outside Hermes: - -- `devops/forgejo-operations` — already cross-harness -- `devops/vaultwarden-secrets` — cross-harness secret management -- `mlops/vision-model-setup` — useful for any harness with vision -- `software-development/colibri-development` — obviously cross-harness -- `devops/herdr-deployment` — cross-machine deployment -- `devops/bootable-usb-images` — FreeBSD/ISO operations - -Do NOT import Hermes-specific skills (cron management, Telegram gateway, voice setup, etc.). - -### 1.6 Validate - -```sh -python3 scripts/layered_soul.py validate . -``` - -Must pass clean before committing. - -## Phase 2: Wire hermes-soul → layered-soul (automation) - -### 2.1 Create private-source config - -Replace `examples/private-sources/hermes-soul.example.json` with a real config pointing to the actual hermes-soul repo. - -### 2.2 Add export step to hermes-soul-backup.sh - -After backing up to hermes-soul, optionally run the planner to report what's new since last review: - -```sh -python3 ~/ai/layered-soul/scripts/layered_soul.py plan-private-source \ - ~/ai/layered-soul/examples/private-sources/hermes-soul.json \ - --source-root ~/hermes-soul -``` - -Operator reviews the diff and manually copies curated content. - -## Phase 3: Colibri Integration (code + test) - -### 3.1 Import skills into Colibri - -```sh -cd ~/ai/colibri -./scripts/import-layered-soul.sh ~/ai/layered-soul colibri.db -``` - -Verify: `sqlite3 colibri.db "SELECT COUNT(*) FROM skills;"` - -### 3.2 Plan system_brain schema - -Design the `system_brain` SQLite table matching the pattern from `COLIBRI-SKILLS-PLAN.md`: - -- `brain_id`, `source_path`, `content_hash`, `memory_type`, `heading`, `content`, `created_at` -- FTS5 virtual table for search -- Import script: `memories/curated/**/*.md` → `system_brain` - -### 3.3 Plan system_ops schema - -Design `system_ops` for task/job manifests: - -- `op_id`, `manifest_path`, `manifest_hash`, `op_type`, `title`, `schedule`, `status`, `created_at` -- Import script: converted cron/task manifests - -### 3.4 Headroom compression middleware (NEW — 2026-06-11) - -Headroom-ai (0.25.0) is installed on debby — 40%+ token savings on tool outputs. -Integrate as a Colibri Unix socket sidecar that compresses tool outputs before -they enter agent context. Works identically on Linux and FreeBSD. - -**Architecture:** - -``` -tool output → session.rs:build_prompt_messages() - → headroom_compress(raw) ← NEW - → compressed output → LLM prompt -``` - -**Files to modify (colibri-daemon):** - -| File | Change | -| ---------------- | ----------------------------------------------------------------- | -| `cost.rs` | Add `HeadroomSidecar` struct + `headroom_compress()` fn | -| `daemon.rs` | Spawn sidecar on startup (`DaemonState`) | -| `session.rs:557` | Call `headroom_compress()` on tool results before prompt assembly | -| `config.rs` | `COLIBRI_HEADROOM_ENABLED` env toggle (disabled by default) | - -**Sidecar**: Python script at `/usr/local/bin/headroom-sidecar.py` — simple loop: -read JSON line from Unix socket → `compress()` → write compressed JSON back. - -**Installer impact**: `ServiceSpec` in crates/clawdie extended with optional -`sidecar: Option` — same pattern already used for colibri-daemon. -The installer creates the sidecar user/unit, but the daemon manages lifecycle. - -**FreeBSD**: zero blockers. Python 3 in base, headroom pure Python, Unix sockets -identical semantics. Daemon runs as unprivileged `clawdie` (hardened in PR #53) -— sidecar inherits same security posture. - -**Clawdie installer hardening (PR #53) — confirmed orthogonal**: the installer -creates the service user, ZFS datasets, and rc.d script. It never touches prompts, -tool outputs, or compression. No adjustment needed to headroom plan. - -### 3.5 Test end-to-end - -1. Run hermes-soul-backup.sh → hermes-soul updated -2. Run private-source planner → see what's new -3. Import skills into Colibri → verify in DB -4. Run layered-soul validate → clean - -## Phase 4: Adapter Testing - -| Harness | Test | -| ----------- | -------------------------------------------------------------------------- | -| Hermes | Load SOUL.md + USER.md as profile context, verify Hermes sees the identity | -| Colibri | Run import-layered-soul.sh, verify skills table populated | -| Pi | Render prompt bundle, confirm Pi can consume it | -| Codex | Seed a task with identity context, verify no session leak | -| Claude Code | Same as Codex — task-scoped context only | -| Zot | Same pattern — task-scoped | - -## Immediate Next Steps (operator actions) - -1. **Review this plan** — approve, modify, or prioritize phases -2. **Fill SOUL.md** — Hermes drafts, Sam reviews -3. **Run private-source planner** — see what hermes-soul has to offer -4. **Curate 3-5 memories** — start small -5. **Select 5-10 skills** — the most cross-harness-useful ones - -## Notes - -- The `.gitignore` already covers secrets, sessions, config, locks, caches -- `plan-private-source` NEVER copies raw sessions or secrets — it only reports counts -- The import direction is strictly one-way: layered-soul (git) → Colibri (DB). No write-back. -- Colibri now has 12 crates (up from 9). `colibri-store`, `colibri-skills`, `colibri-mcp`, and `clawdie` are new since memory was last updated.