Single Go module, four top-level packages under packages/. Import
paths become github.com/patriceckhart/zot/packages/<name>; downstream
consumers can depend on individual packages without pulling the rest.
Layout:
packages/provider/ LLM clients + catalog
packages/provider/auth/ credential store + OAuth + login server
packages/core/ agent loop, sessions, cost
packages/tui/ terminal toolkit + chat view
packages/agent/ CLI wiring, system prompt
extensions/ extproto/ modes/ tools/ skills/ swarm/
sdk/ (was pkg/zotcore, package renamed zotcore -> sdk)
ext/ (was pkg/zotext, package renamed zotext -> ext)
internal/ and pkg/ removed. The internal/assets logo moved into
packages/provider/auth/assets.
Public Go SDK identifiers renamed:
pkg/zotcore (package zotcore) -> packages/agent/sdk (package sdk)
pkg/zotext (package zotext) -> packages/agent/ext (package ext)
This breaks Go-based extensions and embedders; the JSON wire protocol
for extensions and RPC is unchanged, so non-Go extensions, already-
built extension binaries, and zot rpc consumers are unaffected.
Docs, examples, and the built-in write-zot-extension skill updated
for the new paths and identifiers. Shadow-bug fixes in code samples
(ext := ext.New -> e := ext.New).
Adds the todo panel example under examples/extensions, updates example manifests and READMEs to match the current extension API, and surfaces extension install/load commands in zot --help.
Notes are now project-local and survive zot restarts. The
extension reads its cwd from the hello_ack handshake, then:
- on /note appends one line of {"at":..,"text":..} JSONL
under <cwd>/.zot/scratchpad-notes.jsonl
- on /notes reads the in-memory cache (loaded on hello_ack)
- on /clear-notes truncates the file and clears the cache
- on read_notes (tool) returns the cached set
Single-writer assumption (one zot session per cwd at a time);
two concurrent zot processes writing to the same file would
interleave but JSONL line boundaries stay intact under POSIX
PIPE_BUF semantics. Good enough for an example.
Verified end-to-end: round 1 writes two notes via the slash
command, round 2 (fresh extension process, same cwd) loads
them and surfaces them via both /notes and the read_notes
tool.
Three independent fixes to startup latency:
1) Discover spawns extensions in parallel.
Before: each spawn synchronously waited on its child's hello
frame; multiple slow runtimes (e.g. tsx) added linearly.
After: every loadOne runs on its own goroutine; total time
collapses to max(spawn_time) instead of sum.
2) WaitForReady waits in parallel.
Before: one extension at a time, so a slow ready (or no ready
at all from a legacy SDK) blocked every other extension's wait
too.
After: one goroutine per extension, all sharing a single
deadline; total = max(per-ext wait), not sum.
3) Auto-ready idle watchdog for legacy extensions.
Phase-1 SDK builds didn't send the ready sentinel introduced
in phase 2. Without it, WaitForReady burned the full 3s grace
on every startup for every legacy extension. Fix: read loop
stamps lastFrameTime on each frame; a per-extension watchdog
closes readyCh as soon as no new frame has arrived for 250ms.
Native binaries register + go quiet within microseconds, so
this fires almost immediately. Newer extensions still trip
the explicit ready path before the watchdog matters.
Also updates the scratchpad example to invoke `tsx` directly
instead of `npx --yes tsx`, with the README explaining how to
install tsx globally and how to fall back to npx (and what it
costs in startup time).
Measured impact on a machine with 4 extensions installed
(guard / hello / weather / scratchpad):
before: 4.2-4.9s per zot launch
after: ~200ms per zot launch (cold-cache first run ~780ms)
The dominant remaining cost in the 200ms is normal node + tsx
boot for scratchpad, which only matters because it's still in
the spawn fan-out — Go extensions add nothing measurable.
examples/extensions/scratchpad: real .ts (not .js) extension, no
build step, no SDK. Runs via `npx --yes tsx index.ts` so authors
can use TypeScript without forcing a global install. Demonstrates:
/note <text> slash command (typed CommandResponse)
/notes slash command (display action)
/clear-notes slash command
read_notes LLM-callable tool (typed ToolResult)
Plus a typed wire-format subset inline so the file shows what the
protocol actually looks like from the consumer side. Pure node +
tsx, zero npm deps beyond tsx itself (~5 MB cached on first call).
Manager fix: extension exec paths are now resolved by shape:
absolute used as-is
starts with ./ or ../ joined to ext.Dir
contains a separator joined to ext.Dir (other relative form)
bare name (no sep) left as-is so $PATH lookup works
Before this, "exec": "npx" was being looked up at
extensions/scratchpad/npx and failing with a "no such file or
directory" error. With the fix, "node", "npx", "python3", "tsx",
etc. resolve via $PATH like users intuitively expect.
Bumped WaitForReady grace from 500ms to 3s so slow runtimes
(npx tsx cold-start ≈ 1.4s) get their register_tool frames
in before the agent's tool registry is built. Extensions that
send ready quickly still release the wait immediately; the
extra grace only applies to laggards.
Verified end-to-end live against anthropic:
prompt: "Use the read_notes tool now and tell me what's in the
scratchpad"
-> [tool_call] read_notes({})
-> [tool_result] (scratchpad is empty)
-> "The scratchpad is empty."
Two new capabilities, both ride on the existing subprocess
protocol with a couple of new frame types.
Event subscriptions (one-way notifications):
ext -> host: subscribe {events: [...], intercept: [...]}
host -> ext: event {event, ...payload}
Recognised events: session_start, turn_start, turn_end,
tool_call, assistant_message. Subscribers get fire-and-forget
notifications on each. Useful for telemetry, audit logs, custom
state widgets that follow live agent activity.
Tool-call interception (round-trip, can refuse):
host -> ext: event_intercept {id, event:"tool_call", tool_name, tool_args}
ext -> host: event_intercept_response {id, block?, reason?}
When at least one extension subscribed to "tool_call" intercept,
zot asks each one in turn before running every tool call. First
blocker wins; reason becomes the tool-result error text the model
sees. Per-extension 5s timeout treats unresponsive interceptors
as "allow" so a wedged extension never stalls the agent.
Wire format additions (internal/extproto):
ext -> host: SubscribeFromExt, EventInterceptResponseFromExt
host -> ext: EventFromHost, EventInterceptFromHost
Manager (internal/agent/extensions):
- per-extension eventSubs / interceptSubs sets, populated by the
subscribe frame
- EmitEvent fans out to every subscribed extension on its own
goroutine (won't block the agent on slow stdin writes)
- InterceptToolCall walks subscribers serially, returning the
first refusal; 5s timeout per subscriber (allow on timeout)
- readLoop handles event_intercept_response correlations the
same way it handles command/tool responses
Core (internal/core/agent.go):
- Agent.BeforeToolExecute hook called from runOneTool right
before tool.Execute. Returning (allowed=false, reason)
short-circuits with an IsError tool result containing reason.
- Agent.OnEvent observer fires for every emitted AgentEvent;
composed transparently with the per-Prompt sink via wrapSink
so neither the existing TUI nor the rpc loop need changes.
Wiring (internal/agent/cli.go, rpc.go):
- wireAgentExt sets BeforeToolExecute -> InterceptToolCall and
OnEvent -> fanoutAgentEvent for every freshly-built agent
(initial, login rebuild, model swap)
- fanoutAgentEvent translates core AgentEvent kinds into
extproto.EventFromHost. Internal-only events (text_delta,
tool_progress) are dropped to keep the per-extension stream
sane.
- session_start emitted once after extensions come up
SDK (pkg/zotext):
- On(name, EventHandler) registers per-event observers
- InterceptToolCall(InterceptHandler) registers a single
intercept callback
- Run() now also sends a subscribe frame before the ready
sentinel, with the union of subscribed events + intercept
- Frame loop handles "event" and "event_intercept" frames,
runs the handlers (intercepts on a goroutine to avoid
head-of-line blocking)
- Capabilities advertised: commands + tools + events
Example (examples/extensions/guard):
- subscribes to session_start / turn_start / tool_call / turn_end
and writes one-line audit entries
- intercepts every bash call; refuses commands matching
rm -rf, sudo, dd of=/, mkfs, the fork bomb, chmod -R 777
- end-to-end verified live: agent -> bash("rm -rf /tmp/foo")
-> guard refuses -> model sees the refusal text and surfaces
it in its reply ("the guard blocked it, as expected — the
pattern \brm\s+-rf\b matched")
Docs/extensions.md updated with all five new frame types and the
guard example.
Extensions can now register tools the LLM calls directly. The model
sees them in its tool list alongside the built-ins (read, write,
edit, bash, skill); when it invokes one, zot routes the tool_call
to the owning extension subprocess and feeds the tool_result back.
Wire format additions (internal/extproto):
ext -> host:
register_tool {name, description, schema}
ready # all initial regs flushed
tool_result {id, content[], is_error} # reply to a tool_call
host -> ext:
tool_call {id, name, args} # raw json args from the model
Manager (internal/agent/extensions):
- tracks per-extension RegisterToolFromExt frames
- validates schemas parse as JSON before registering (bad schema
skipped + logged, doesn't crash zot)
- toolIndex map for O(1) lookup
- WaitForReady(grace): blocks per extension on its readyCh until
a ready frame arrives or the grace expires; called once after
Discover so the agent's tool registry is built against the
final set
- Tools() / HasTool() / InvokeTool() public surface
- readLoop closes readyCh on stdout EOF so a wedged extension
doesn't permanently block WaitForReady
extensionTool (internal/agent/extensions/tool.go):
implements core.Tool. Execute() round-trips through
Manager.InvokeTool with a 60s default timeout, decodes
base64 image blocks, surfaces extension+tool name in
ToolResult.Details for the renderer.
internal/agent/build.go:
- new ExtensionToolSource interface (declared here to avoid the
build->extensions->core import cycle) + ExtensionToolInfo
mirror of extensions.ToolInfo
- Resolved.MergeExtensionTools(): folds extension tools into
ToolRegistry, re-renders the system prompt's tool summary
with both built-in and extension tools listed
- Resolved gains private bookkeeping fields so the rebuild
works without re-running Resolve
internal/agent/cli.go:
extension manager built BEFORE the agent in interactive mode
so MergeExtensionTools can fire before NewAgent. Same in
buildAgent + buildAgentFor closures so login / model-switch
rebuilds also include extension tools. extToolAdapter bridges
*extensions.Manager to ExtensionToolSource.
internal/agent/rpc.go:
extension lifecycle now also runs in `zot rpc` mode. Notify and
Display from extensions surface as `ext_notify` / `ext_display`
events on the rpc stream so any consumer can react.
pkg/zotext (Go SDK):
- ToolHandler, ToolResult, ToolContent types
- Tool(name, desc, schema, fn) registration method
- TextResult / TextErrorResult / Image / ImageBytes constructors
- Run() now also flushes register_tool frames + a final ready
sentinel after the last registration
examples/extensions/weather: working Go example registering one
tool. Deterministic fake weather (sha1 of city -> temp + cond) so
the demo is repeatable. Plus README explaining how to install.
Tests:
internal/agent/extensions/tool_test.go: spawns a mock /bin/sh
extension that registers a tool, sends ready, and echoes tool
calls. Verifies registration timing, lookup via HasTool/Tools,
invoke roundtrip via InvokeTool.
End-to-end verified against live anthropic backend:
prompt: "What is the weather in Berlin?"
-> [tool_call] weather({"city":"Berlin"})
-> [tool_result] Berlin: 16°C, fog (deterministic fake)
-> reply: "Berlin is 16°C."
Docs/extensions.md updated with phase 2 wire format, the new SDK
tool API, and the weather example.
extension commands appeared in the autocomplete popup but invoking
them produced "unknown command: /summon". The submit-handler path
already tried the extension manager before erroring, but the popup-
enter path (suggest.Selection -> runSlash) bypassed that check and
fell straight into runSlash's switch, where the default case bailed
with the generic error.
Fix: runSlash's default branch now also consults
cfg.Extensions.HasCommand and dispatches via invokeExtensionCommand
when matched. Both UI paths (typed-and-enter, popup-enter) now route
identically. Built-in cases above default still always win on
conflict.
Also adds examples/extensions/clock — a node extension demonstrating
the wire protocol from a non-Go runtime. Pure stdlib (readline +
process), no npm install. Registers /now (display) and /uptime
(prompt). Documented in its README; the protocol works the same
from any language.
Phase 1: extensions can register slash commands and push chat
notifications. Tools and event subscriptions land in later phases.
Architecture: each extension is its own subprocess. Zot launches
it on startup, completes a hello/hello_ack handshake over its
stdin/stdout, then routes slash commands the extension registered.
Crash isolation, language agnostic, works with any executable
that can read/write json lines.
What lands here:
- internal/extproto: shared wire-format types (Frame, HelloFromExt,
RegisterCommandFromExt, CommandResponseFromExt, NotifyFromExt,
HelloAckFromHost, CommandInvokedFromHost, ShutdownFromHost...).
Both the host and the SDK marshal/unmarshal the same types.
- internal/agent/extensions: discovery + lifecycle manager.
- Discover() walks $ZOT_HOME/extensions and ./.zot/extensions
(project-local first, global second; first wins for duplicates)
- Spawns each enabled extension, captures stderr to
$ZOT_HOME/logs/ext-<name>.log
- Reads frames in a goroutine, dispatches register_command and
notify, correlates command_response by id
- Stop() sends shutdown, waits 2s, then SIGTERM/SIGKILL
- HostHooks abstracts the tui callbacks (Notify/Submit/Insert/Display)
- Interactive bridge: extensions slot into the slash dispatcher
*after* the built-in catalog, so built-ins always win on conflict.
Extension-registered commands also flow into the autocomplete
popup and /help via slashSuggester.SetExtra. NotifyFromExt frames
render as muted [ext-name] notes above the editor.
- internal/agent/extcmd: `zot ext` CLI.
list / install <path|git-url> / remove / enable / disable / logs
- pkg/zotext: public Go SDK. Construct an Extension, register
Command(name, desc, fn), call Run(). Fn returns a Response built
with Prompt(), Insert(), Display(), Noop(), or Errorf(). Stderr
via Logf() so stdout stays clean for the protocol.
- examples/extensions/hello: working Go example registering /hello
and /summon, plus README + extension.json.
- docs/extensions.md: full protocol reference, including a
~30-line raw-Python example for users who don't want the SDK.
Tests: internal/agent/extensions/manager_test.go spawns a mock
extension via /bin/sh and exercises the full handshake -> register
-> invoke -> response cycle. Verifies the hello frame ordering,
correlation-by-id, and graceful shutdown.
Verified manually: built and installed the example, drove it via
stdin pipes, confirmed clean handshake + correct frame ordering
and shutdown_ack. Builds vet-clean on darwin / linux / windows.
Editor.Insert exported (was Editor.insert) so the extension hooks
can drop text into the input.
