# Agent Harness and LLM Provider Matrix

**Owner:** Opencode / Z.ai Integrator, Pi ISO Developer, Codex ISO Builder

This document separates three things that were previously mixed together:

1. **Agent harness** — the CLI/runtime that executes an agent (`pi`, `codex`,
   `claude`, `aider`, Opencode, etc.).
2. **LLM provider lane** — the provider/model endpoint used by a harness
   (`openai-codex/gpt-5.5`, `deepseek/deepseek-v4-flash`, `zai/glm-4.7`, etc.).
3. **Operator display/control fabric** — Colibri dashboards, tmux, or other
   panes/views. These are not LLM providers.

A provider working in one harness does **not** prove it works in another. For
Colibri, the preferred proof is a `pi --mode json` JSONL capture because that is
what the current parser and runtime seam consume.

## Platform proof boundaries

- Linux-side smokes prove only provider transport, JSONL shape, and integration
  direction on that Linux host.
- FreeBSD runtime claims require a FreeBSD host or mounted/booted live image.
- Live-USB claims require Codex ISO Builder evidence from the FreeBSD build host
  and, for GUI/input/audio, real hardware evidence.
- A dashboard/control-fabric result is never provider proof by itself; capture
  the harness command, provider/model, platform, and output shape.

## Agent Harness Matrix

| Harness             | Where observed / target                | Provider/model lanes known here                                                              | Status / caveat                                                                                                                 |
| ------------------- | -------------------------------------- | -------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| **Pi**              | FreeBSD host; live ISO pinned globally | `openai-codex/gpt-5.5`, `deepseek/deepseek-v4-flash`, `zai/*`, OpenRouter, Anthropic, others | Core Clawdie runtime target. JSONL mode is the Colibri contract. Check runtime inventory before claiming host/ISO version sync. |
| **Codex CLI**       | FreeBSD host and live ISO pkg          | OpenAI/Codex subscription models; local OSS mode                                             | Separate harness from Pi. Good for build/review work, not the Colibri JSONL provider contract.                                  |
| **Claude Code CLI** | FreeBSD host / Linux dev, not live ISO | Claude subscription/API models                                                               | Separate harness. Useful for review/design; excluded from current live USB policy.                                              |
| **Aider**           | FreeBSD host                           | LiteLLM-backed providers; `zai/glm-5-turbo` documented with corrected `ZAI_API_BASE`         | Separate harness. ZAI success here does not prove Pi ZAI JSONL.                                                                 |
| **Opencode**        | Linux lane, not FreeBSD/live ISO       | GLM-4.7 via z.ai                                                                             | External Linux proof only. Its config/key store is not Pi's auth store.                                                         |
| **Gemini CLI**      | Not present on this FreeBSD host       | Google Gemini                                                                                | Excluded from current live ISO policy unless intentionally restored later.                                                      |

## Pi Provider Lane Matrix

| Provider lane        | Pi selector / model                                  | Credential source                                                | Smoke status                                             | Notes                                                                               |
| -------------------- | ---------------------------------------------------- | ---------------------------------------------------------------- | -------------------------------------------------------- | ----------------------------------------------------------------------------------- |
| **OpenAI Codex**     | `--provider openai-codex --model gpt-5.5`            | `~/.pi/agent/auth.json` OAuth via Pi `/login`                    | Active FreeBSD config                                    | Current preferred Clawdie runtime lane when subscription auth is configured.        |
| **DeepSeek v4**      | `--provider deepseek --model deepseek-v4-flash`      | `DEEPSEEK_API_KEY` or `~/.pi/agent/auth.json` key `deepseek`     | PASS in Linux smoke; FreeBSD local runtime smoke pending | Direct DeepSeek API lane. `deepseek-v4-flash` is the default/heartbeat-safe choice. |
| **Z.ai / GLM**       | `--provider zai --model <exact-id-from-list-models>` | `ZAI_API_KEY` or `~/.pi/agent/auth.json` key `zai`               | Pi JSONL smoke pending                                   | Opencode GLM-4.7 transport is not the same as Pi ZAI JSONL proof.                   |
| **OpenRouter**       | `--provider openrouter --model <provider/model>`     | `OPENROUTER_API_KEY` or `~/.pi/agent/auth.json` key `openrouter` | Available when key exists                                | Broad fallback/catalog lane. Model IDs are OpenRouter-specific.                     |
| **Anthropic Claude** | `--provider anthropic --model <claude-model>`        | `ANTHROPIC_API_KEY` or Pi Claude Pro/Max OAuth                   | Not the current ISO lane                                 | Pi supports Anthropic, but Claude Code CLI is a separate harness.                   |
| **Google Gemini**    | `--provider google --model <gemini-model>`           | `GEMINI_API_KEY` or `~/.pi/agent/auth.json` key `google`         | Not current live ISO policy                              | Do not infer Gemini CLI presence from Pi Google-provider support.                   |

## Current Proofs and Non-Proofs

Confirmed:

- Pi can run `openai-codex/gpt-5.5` when Codex subscription auth is present
  (see the headless Codex login check in `../docs/public/install/install.md`).
- DeepSeek v4 works through Pi in JSONL mode on the Linux lane:
  `deepseek/deepseek-v4-flash` (see `DEEPSEEK-V4-SMOKE.md`).
- Opencode can run GLM-4.7 through its own z.ai setup on Linux (see
  `GLM-4.7-SMOKE.md`; this is Opencode transport proof, not Pi proof).
- Aider can use ZAI when LiteLLM is pointed at the corrected base URL:
  `https://api.z.ai/api/coding/paas/v4` (see Clawdie-AI:
  `doc/AIDER-FREEBSD-INSTALL.md`).

Not confirmed yet:

- Formal `pi --provider zai --mode json` GLM capture.
- FreeBSD-host DeepSeek runtime smoke after Pi is synchronized to the pinned
  version used by the ISO (`0.75.5`).
- Any claim that a dashboard/control fabric itself provides LLM/provider
  functionality.

## Quick Pi Smoke Pattern

Use repo-local `tmp/` for captures.

```sh
mkdir -p tmp
```

List models before choosing an ID:

```sh
pi --provider deepseek --list-models
```

Run JSONL mode for Colibri ingestion:

```sh
pi --provider deepseek --model deepseek-v4-flash -p --mode json "reply with: ok" 2>&1 | tee tmp/deepseek-smoke.jsonl
```

Output is newline-delimited JSON. Expect a session header event plus turn and
message lifecycle events. Exact event names can vary across Pi versions, so do
not overfit validation to one captured stream until the current runtime version
has produced a fresh sample.

## Provider Deep Links

| Lane        | Smoke doc                                      | API keys                                                                 | Pricing                                                           |
| ----------- | ---------------------------------------------- | ------------------------------------------------------------------------ | ----------------------------------------------------------------- |
| DeepSeek v4 | [DEEPSEEK-V4-SMOKE.md](./DEEPSEEK-V4-SMOKE.md) | [platform.deepseek.com/api_keys](https://platform.deepseek.com/api_keys) | [Pricing page](https://api-docs.deepseek.com/quick_start/pricing) |
| GLM / Z.ai  | [GLM-4.7-SMOKE.md](./GLM-4.7-SMOKE.md)         | Sam/operator private key path                                            | TBD                                                               |

## Adding a New Provider Lane

1. Identify the harness first (`pi`, `aider`, `codex`, etc.).
2. Get or configure the provider credential locally. Never commit keys.
3. Run the harness-specific model listing command.
4. Run a minimal smoke test.
5. For Pi lanes, capture JSONL with `--mode json` under repo-local `tmp/`.
6. Record the exact provider, model ID, harness, command shape, event types, and
   pass/fail result.
7. Add or update the relevant smoke doc.

## Key Rotation

When a key needs rotation:

1. Generate a new key at the provider dashboard.
2. Update the local harness credential store or shell profile.
3. Re-run the smoke to confirm.
4. Delete the old key at the provider dashboard.

Never paste provider keys into chat, commit messages, logs, or public docs.

## Verifying Key Cleanup

After revoking a key, confirm it's fully removed:

```sh
# 1. No stored key
cat ~/.pi/agent/auth.json   # should be {}

# 2. No env var
echo "Key set: ${DEEPSEEK_API_KEY:+yes}"   # should be blank

# 3. Revoked key is rejected
DEEPSEEK_API_KEY="<old-key>" pi --provider deepseek --model deepseek-v4-flash -p "test" 2>&1
# Expected: "401 Authentication Fails, Your api key: ****XXXX is invalid"

# 4. Clean "no key" state confirmed
pi --provider deepseek --model deepseek-v4-flash -p "test" 2>&1
# Expected: "No API key found for deepseek."
```

---

## Platform × Harness Matrix

Level 1 is the **platform** (FreeBSD main, Linux auxiliary).
Level 2 is the **harness** — which agent runtime lives where and which
LLM providers each can reach.

### Level 1: Platform

| Platform              | Role                          | Primary harness | Control fabric         | Notes                      |
| --------------------- | ----------------------------- | --------------- | ---------------------- | -------------------------- |
| **FreeBSD (main)**    | ISO build, live USB, validate | pi + codex      | Colibri CLI/TUI        | Authoritative build host   |
| **Linux (auxiliary)** | Review, deploy, Colibri, UX   | pi + opencode   | Colibri/terminal panes | Opencode + Claude sessions |

### Level 2: Harness × LLM Provider Combos

Each cell shows: provider/model — smoke status.

| Harness / Agent | DeepSeek v4                     | GLM-4.7 (z.ai)             | GPT-5.5 (OpenAI/Codex)        | Claude (Anthropic)        | Gemini (Google)        |
| --------------- | ------------------------------- | -------------------------- | ----------------------------- | ------------------------- | ---------------------- |
| **pi**          | `deepseek/v4-flash` PASS        | `zai/<tbd>` TRANSPORT OK   | `openai-codex/gpt-5.5` ACTIVE | `anthropic/claude-*` TODO | `google/gemini-*` TODO |
| **codex**       | n/a                             | n/a                        | built-in PASS                 | n/a                       | n/a                    |
| **claude-code** | DeepSeek via OpenAI compat TODO | n/a                        | n/a                           | native PASS               | n/a                    |
| **opencode**    | n/a (no DeepSeek config yet)    | native PASS (this session) | n/a                           | n/a                       | n/a                    |
| **gemini-cli**  | n/a                             | n/a                        | n/a                           | n/a                       | native TODO            |

### What populates the matrix

- **FreeBSD live USB**: ships `pi` + `codex` only (per live CLI policy)
- **FreeBSD build host**: `pi` with `openai-codex/gpt-5.5` for Colibri review/fixes
- **Linux auxiliary**: runs `pi`, `claude-code`, `codex`, `gemini-cli`,
  and `opencode` — visible through Colibri or terminal panes where wired
- **Opencode** (this session): currently GLM-4.7 via z.ai; other
  providers can be wired via opencode's config
- **pi**: supports all providers via `--provider` flag; key in env or
  `auth.json`

### How to add a combo

1. Pick an empty cell above.
2. Configure the agent for that provider (key, model).
3. Run the smoke (see Quick Start).
4. Update the cell with the exact `provider/model` string and PASS/FAIL.
5. Create a smoke doc if one doesn't exist.
6. Commit and push.
