clawdie/layered-soul
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: match layered-soul docs to populated state #4

Merged
clawdie merged 1 commit from docs/soul-decisions-match-code into main 2026-06-14 02:13:14 +02:00
Conversation 0 Commits 1 Files changed 3 +6 -208
3 changed files with 6 additions and 208 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
README.md
View file

@ -7,6 +7,11 @@ Colibri, Codex, Claude Code, Zot, or future harnesses. The repository carries
durable identity, reviewed user context, curated memories, and approved skills
without copying a full runtime backup.
It is both the **template** and a **working reference**: the identity files,
curated memories, and skills here are real, usable content — not placeholders.
Spin off `<agent>-layered-soul` instances from it when you need a separate
identity bundle (see "Template usage").
## Why this exists
Agent runtimes already keep useful state, but their native backups are not safe

2
docs/CONNECT-HERMES-SOUL.md
View file

@ -39,7 +39,7 @@ Do not mirror `hermes-soul` into `layered-soul`. Use a review/export workflow.
- Reviewed user context from `memories/USER.md`.
- Reviewed memory summaries from `memories/MEMORY.md`.
- Selected `skills/**/*.md` that are safe and useful across harnesses.
- Selected `skills/**/SKILL.md` (with their `references/`) that are safe and useful across harnesses.
- Converted cron/task ideas after they are rewritten as explicit Ops manifests.
## What must stay private

207
docs/PLAN-CONFIGURE-PRIVATE-REPO.md
View file

@ -1,207 +0,0 @@
# Plan: Configure layered-soul for Private Repo
## State Analysis (2026-06-11)
### What we have
| Component | Status | Details |
| ---------------------------- | ---------------- | ----------------------------------------------------------------------- |
| `hermes-soul` | ✅ Working | Private backup — 539 skills, 136 sessions, 2 memories, config sanitized |
| `layered-soul` scaffolding | ✅ Complete | Schema, validation, 6 adapters, private-source planner |
| `layered-soul` content | ✅ Populated | SOUL/IDENTITY/USER/AGENTS.md filled, 5 curated memories, 9 skills |
| Colibri skills import | ✅ Working | `import-layered-soul.sh` imports skills into flat `skills` table |
| Colibri system_brain | ❌ Not built | `memories/curated/` detected but no DB table |
| Colibri system_ops | ❌ Not built | No schema, no manifest format |
| Colibri system_skills (full) | 📋 Planned | Schema designed, Phase 2-7 not implemented |
| Headroom-ai | ✅ Installed | 0.25.0 on debby, 40%+ savings, integration planned in 3.4 |
| Clawdie installer hardening | ✅ Merged PR #53 | Non-root daemon, ownership enforcement, pool validation |
### Architecture
```text
┌──────────────────────────────────┐
│ hermes-soul (PRIVATE) │
│ sessions/ config/ cron/ │
│ memories/ skills/ scripts/ │
└──────────┬───────────────────────┘
│ plan-private-source
│ (review, never copies secrets)
▼
┌──────────────────────────────────┐
│ layered-soul (PORTABLE) │
│ SOUL.md USER.md IDENTITY.md │
│ memories/curated/ skills/ │
│ adapters/ manifest.json │
└────┬──────────┬──────────┬───────┘
│ │ │
Hermes Colibri Pi/Codex/Claude/Zot
(context) (import) (AGENTS.md)
```
## Phase 1: Populate the Soul (operator + Hermes together)
### 1.1 Fill SOUL.md
Replace the empty SOUL.md with actual agent identity — what Layered Soul IS, its values, how it operates. Hermes can draft; operator reviews.
### 1.2 Fill IDENTITY.md
Short runtime identity — agent name, home harness, boundaries, contact. 3-4 lines.
### 1.3 Import USER.md context
Run the private-source planner, review hermes-soul's USER.md, write sanitized summaries into layered-soul's USER.md. Keep Hermes-native detail in hermes-soul; portable context in layered-soul.
```sh
python3 scripts/layered_soul.py plan-private-source \
examples/private-sources/hermes-soul.example.json \
--source-root ~/hermes-soul
```
### 1.4 Curate memories
Select 3-5 durable memory entries from Hermes that matter across harnesses (Tailscale layout, Forgejo setup, agent identities, project structure). Write each as a `memories/curated/<topic>.md`.
```sh
# Create curated memories
# memories/curated/tailscale-network.md — Tailscale IPs and host roles
# memories/curated/forgejo-setup.md — Forgejo repo layout, SSH keys
# memories/curated/agent-roster.md — Which agents run where
# memories/curated/project-structure.md — Clawdie repos and their purposes
```
### 1.5 Select cross-harness skills
From 539 Hermes skills, pick the ones useful outside Hermes:
- `devops/forgejo-operations` — already cross-harness
- `devops/vaultwarden-secrets` — cross-harness secret management
- `mlops/vision-model-setup` — useful for any harness with vision
- `software-development/colibri-development` — obviously cross-harness
- `devops/herdr-deployment` — cross-machine deployment
- `devops/bootable-usb-images` — FreeBSD/ISO operations
Do NOT import Hermes-specific skills (cron management, Telegram gateway, voice setup, etc.).
### 1.6 Validate
```sh
python3 scripts/layered_soul.py validate .
```
Must pass clean before committing.
## Phase 2: Wire hermes-soul → layered-soul (automation)
### 2.1 Create private-source config
Replace `examples/private-sources/hermes-soul.example.json` with a real config pointing to the actual hermes-soul repo.
### 2.2 Add export step to hermes-soul-backup.sh
After backing up to hermes-soul, optionally run the planner to report what's new since last review:
```sh
python3 ~/ai/layered-soul/scripts/layered_soul.py plan-private-source \
~/ai/layered-soul/examples/private-sources/hermes-soul.json \
--source-root ~/hermes-soul
```
Operator reviews the diff and manually copies curated content.
## Phase 3: Colibri Integration (code + test)
### 3.1 Import skills into Colibri
```sh
cd ~/ai/colibri
./scripts/import-layered-soul.sh ~/ai/layered-soul colibri.db
```
Verify: `sqlite3 colibri.db "SELECT COUNT(*) FROM skills;"`
### 3.2 Plan system_brain schema
Design the `system_brain` SQLite table matching the pattern from `COLIBRI-SKILLS-PLAN.md`:
- `brain_id`, `source_path`, `content_hash`, `memory_type`, `heading`, `content`, `created_at`
- FTS5 virtual table for search
- Import script: `memories/curated/**/*.md` → `system_brain`
### 3.3 Plan system_ops schema
Design `system_ops` for task/job manifests:
- `op_id`, `manifest_path`, `manifest_hash`, `op_type`, `title`, `schedule`, `status`, `created_at`
- Import script: converted cron/task manifests
### 3.4 Headroom compression middleware (NEW — 2026-06-11)
Headroom-ai (0.25.0) is installed on debby — 40%+ token savings on tool outputs.
Integrate as a Colibri Unix socket sidecar that compresses tool outputs before
they enter agent context. Works identically on Linux and FreeBSD.
**Architecture:**
```
tool output → session.rs:build_prompt_messages()
→ headroom_compress(raw) ← NEW
→ compressed output → LLM prompt
```
**Files to modify (colibri-daemon):**
| File | Change |
| ---------------- | ----------------------------------------------------------------- |
| `cost.rs` | Add `HeadroomSidecar` struct + `headroom_compress()` fn |
| `daemon.rs` | Spawn sidecar on startup (`DaemonState`) |
| `session.rs:557` | Call `headroom_compress()` on tool results before prompt assembly |
| `config.rs` | `COLIBRI_HEADROOM_ENABLED` env toggle (disabled by default) |
**Sidecar**: Python script at `/usr/local/bin/headroom-sidecar.py` — simple loop:
read JSON line from Unix socket → `compress()` → write compressed JSON back.
**Installer impact**: `ServiceSpec` in crates/clawdie extended with optional
`sidecar: Option<SidecarSpec>` — same pattern already used for colibri-daemon.
The installer creates the sidecar user/unit, but the daemon manages lifecycle.
**FreeBSD**: zero blockers. Python 3 in base, headroom pure Python, Unix sockets
identical semantics. Daemon runs as unprivileged `clawdie` (hardened in PR #53)
— sidecar inherits same security posture.
**Clawdie installer hardening (PR #53) — confirmed orthogonal**: the installer
creates the service user, ZFS datasets, and rc.d script. It never touches prompts,
tool outputs, or compression. No adjustment needed to headroom plan.
### 3.5 Test end-to-end
1. Run hermes-soul-backup.sh → hermes-soul updated
2. Run private-source planner → see what's new
3. Import skills into Colibri → verify in DB
4. Run layered-soul validate → clean
## Phase 4: Adapter Testing
| Harness | Test |
| ----------- | -------------------------------------------------------------------------- |
| Hermes | Load SOUL.md + USER.md as profile context, verify Hermes sees the identity |
| Colibri | Run import-layered-soul.sh, verify skills table populated |
| Pi | Render prompt bundle, confirm Pi can consume it |
| Codex | Seed a task with identity context, verify no session leak |
| Claude Code | Same as Codex — task-scoped context only |
| Zot | Same pattern — task-scoped |
## Immediate Next Steps (operator actions)
1. **Review this plan** — approve, modify, or prioritize phases
2. **Fill SOUL.md** — Hermes drafts, Sam reviews
3. **Run private-source planner** — see what hermes-soul has to offer
4. **Curate 3-5 memories** — start small
5. **Select 5-10 skills** — the most cross-harness-useful ones
## Notes
- The `.gitignore` already covers secrets, sessions, config, locks, caches
- `plan-private-source` NEVER copies raw sessions or secrets — it only reports counts
- The import direction is strictly one-way: layered-soul (git) → Colibri (DB). No write-back.
- Colibri now has 12 crates (up from 9). `colibri-store`, `colibri-skills`, `colibri-mcp`, and `clawdie` are new since memory was last updated.