layered-soul/skills/herdr-deployment/SKILL.md

name	description	triggers
herdr-deployment	Build, configure, and deploy Herdr (terminal workspace manager) across Tailscale-connected hosts. Covers Zig dependency pinning, --remote SSH bridge mode, interactive install requirement, and Tailscale-isolated SSH config.	deploy herdr herdr remote herdr --remote build herdr set up herdr over Tailscale herdr server on herdr tailscale

Herdr Deployment

Herdr is a terminal workspace manager for AI coding agents. It uses Unix sockets only (no TCP), with cross-machine communication via SSH stdio bridge.

Prerequisites

Herdr's vendored libghostty-vt requires Zig 0.15.2 exactly. Newer Zig (0.17+) breaks with @cImport removal and readFileAlloc signature changes.

# Correct Zig version
zig version  # must be 0.15.2

Manual install from ziglang.org:

https://ziglang.org/download/0.15.2/zig-linux-x86_64-0.15.2.tar.xz

Extract to ~/.local/, symlink zig into ~/.local/bin/.

Build

cd /path/to/herdr
cargo build --release
# binary at target/release/herdr

Architecture & Key Files

Herdr is a ratatui-based terminal multiplexer (alternative to tmux/zellij) with workspace/tab/pane BSP layout, PTY supervision, session persistence, detach/reattach, and an SSH-based --remote mode.

Key source files in the repo:

File	Role
build.rs	Zig target mapping, libghostty-vt build orchestrator
src/remote.rs	SSH remote mode, platform detection, binary install
src/platform/freebsd.rs	FreeBSD process detection via sysctl
src/server/headless.rs	Headless server, frame rendering, client socket protocol
src/persist/	Session save/restore (session.json, session-history.json)

FreeBSD 15 Porting Status

Local clone patches (commit ede2059, cannot push to upstream ogulcancelik/herdr):

• build.rs: Zig target mapping x86_64-unknown-freebsd → x86_64-freebsd (not -gnu)
• src/platform/freebsd.rs: process_cwd() uses sysctl kern.proc.cwd (FreeBSD native), with linprocfs fallback at /compat/linux/proc/<pid>/cwd

Validation (Codex on osa, FreeBSD 15.0-RELEASE-p8):

• Builds after Zig target fix
• 1450 passed, 3 failed (failures not FreeBSD-specific)
• sysctl(KERN_PROC) enumeration works; KERN_PROC_ARGS argv retrieval works
• /proc/ is NOT available on FreeBSD (only /compat/linux/proc/)
• sizeof(kinfo_proc) = 1088 verified
• Verdict: "needs more porting"

RemotePlatform::from_uname() in remote.rs only matches "Linux" and "Darwin". FreeBSD returns "FreeBSD" which causes the unsupported-platform error. Do not use herdr --remote against osa or any FreeBSD host.

Tailscale-only SSH Config

Herdr's --remote mode uses ssh(1) — if the target is a Tailscale IP, traffic stays inside the WireGuard tunnel. Add SSH config aliases:

Host <host>-ts-herdr
      HostName <tailscale-ip>
      User <user>
      IdentityFile ~/.ssh/<key>
      IdentitiesOnly yes
      PreferredAuthentications publickey
      StrictHostKeyChecking accept-new
      ForwardAgent no

See references/tailscale-ssh-config.md for the full template.

Remote Deployment (herdr --remote)

herdr --remote <host> does three things:

1. SSHes to the remote, detects platform via uname
2. Downloads and installs upstream herdr binary to ~/.local/bin/herdr
3. Starts headless server + bridges client socket over SSH stdin/stdout

Critical pitfall: interactive terminal required for first install

herdr --remote prompts for install confirmation on first run. Running it non-interactively fails with:

Error: "matching remote herdr 0.6.2 is not installed at $HOME/.local/bin/herdr;
run from an interactive terminal to approve installation"

Workarounds:

• Run herdr --remote <host> from the user's own terminal (approve the prompt)
• Pre-install herdr binary on the remote manually, then --remote detects it and skips the prompt
• Set HERDR_REMOTE_BINARY=<path> env var to use a local build instead of downloading

Limitations

• remote.rs:RemotePlatform::from_uname only accepts "Linux" and "Darwin" — FreeBSD is rejected. Do not use --remote against osa/FreeBSD hosts.
• The remote binary is upstream herdr 0.6.2, not the locally-patched build.

Reverse-direction deployment

When deploying herdr between two hosts (A ↔ B), both directions need:

1. SSH config on each host pointing to the other's Tailscale IP
2. Key coordination: each host's private key must have its public counterpart in the other host's authorized_keys. The keys do NOT need to be the same — domedog uses id_infra, debby uses id_123kupola. Add each public key to the destination's authorized_keys.

# On host A: add host B's public key
ssh host-b 'cat ~/.ssh/<key>.pub' >> ~/.ssh/authorized_keys

3. SSH daemon must be running on both hosts. See pitfall below.

Pitfall: sshd ListenAddress on Tailscale IP fails at boot

If sshd is configured with ListenAddress <tailscale-ip>, it will fail at boot because Tailscale hasn't connected yet (the IP doesn't exist). Symptom:

sshd[1203]: error: Bind to port 22 on 100.66.193.10 failed: Cannot assign requested address.
sshd[1203]: fatal: Cannot bind any address.

Fix: use ListenAddress 0.0.0.0 instead. The machine is behind NAT with no public IP on any interface — Tailscale is the only remote path.

sudo sed -i 's/^ListenAddress .*/ListenAddress 0.0.0.0/' /etc/ssh/sshd_config
sudo systemctl restart ssh

Server persistence

The headless server persists after client detach. herdr --remote <host> again to reattach to the same session. Server log and config live at ~/.config/herdr/.

Post-install verification

ssh <host>-ts-herdr 'ps aux | grep "[h]erdr server"'
ssh <host>-ts-herdr 'ls -la ~/.config/herdr/*.sock'
ssh <host>-ts-herdr '~/.local/bin/herdr --version'

PATH warning on install

The installer may warn: ~/.local/bin is not in the remote PATH. This is cosmetic — the server binary runs fine from its full path. The PATH may be present for interactive shells but missing in the installer's non-interactive SSH session.

Validated Smoke Test (2026-05-27)

Bidirectional validation on Tailscale WireGuard:

From	To	User	Key	Result
debby (100.66.193.10)	domedog (100.103.255.41)	clawdija	id_123kupola	Install → server start → attach → detach → persist
domedog (100.103.255.41)	debby (100.66.193.10)	samob	id_infra	Install → server start → attach → detach → persist

Report: docs/internal/sessions/2026-05-27-herdr-tailscale-remote-smoke.md

Both servers persist after client detach. Reattach with same herdr --remote <host>. Integrations (Claude, OpenCode) auto-installed on first server start.

Herdr as a Colibri Display Client

Herdr's --remote is a herdr→herdr protocol (PTY frame forwarding), not a generic GlasspaneSnapshot consumer. It cannot directly connect to a colibri-daemon socket.

Options for display:

1. Native herdr on Linux — build with Zig 0.15.2, use locally for Linux PTY supervision
2. colibri-glasspane-tui — render GlasspaneSnapshot with ratatui (Colibri-native, FreeBSD-compatible, no herdr dependency)
3. HTTP snapshot endpoint — daemon already has axum scaffold, quick browser display

Related

• Herdr repo: ogulcancelik/herdr (our clone at /home/samob/ai/herdr)
• Colibri control plane: Clawdie/Colibri at /home/samob/ai/colibri
  • 8 crates as of 2026-05-27 (added colibri-store for coordination)
  • Colibri daemon exposes its own Unix socket API (distinct from Herdr's)
  • Herdr consumes Colibri's glasspane-snapshot over socket; the two protocols are separate and independently versioned
  • Colibri glasspane TUI: alternative lightweight display for Colibri daemon (no herdr dependency, FreeBSD-native)
• references/tailscale-ssh-config.md — annotated SSH config template
• references/build-pitfalls.md — Zig version mismatches and other build issues
• references/zig-017-build-failure.md — full error transcript from Zig 0.17.0 build attempt
• references/colibri-harness.md — colibri-harness TUI (herdr-like dashboard on Colibri primitives)