perf(prompt): cut system prompt to the bone (410 -> 54 tokens)

Three levers, all dead-simple, compounded savings.

1) System prompt rewritten to a one-line identity.
   Was 410 tokens (identity + tool listing duplicating the tool
   schemas + operating guidelines that frontier models already
   internalise). Now 54 tokens:

       You are zot, a lightweight terminal coding agent. Be
       concise, act on the user's request directly, and reply
       with a short summary when done.

   The old 'You have the following tools available:' block listed
   every tool by name and description, which the provider sends
   alongside the actual tool schemas. Pure duplication. Dropped.
   Operating guidelines (prefer edit over write, read before
   editing, don't apologize, etc.) are ~150 tokens of advice the
   model already follows by default. Dropped.

2) Tool descriptions trimmed.
   read:  long paragraph   -> 'Read a file. Images (png/jpg/gif/webp) return inline.'
   write: long paragraph   -> 'Write a file. Creates parent dirs. Overwrites.'
   edit:  long paragraph   -> 'Edit a file via exact-match replacements. Each oldText must be unique in the file.'
   bash:  long paragraph   -> 'Run a shell command. stdout+stderr merged.'
   skill: 2-sentence para  -> 'Load a named skill's instructions. Use when the user's request matches a skill listed above.'

3) Tool schemas minified.
   Every schema was pretty-printed JSON with per-field descriptions
   that reiterated the tool's own description ('Path to the file
   to read (relative or absolute)'). The model infers the obvious
   from property names. Schemas now single-line, type+required
   only. Saves ~20-40 bytes per schema, 5 tools = ~150 bytes per
   request.

Net effect on a fresh OAuth turn, measured end-to-end:

   request body:   3205 bytes -> ~1600 bytes
   system prompt:  410 tokens -> 54 tokens
   tools payload:  ~400 tokens -> ~100 tokens

Escape hatches preserved: --system-prompt (per-run), --append-
system-prompt (per-run, repeatable), and $ZOT_HOME/SYSTEM.md
(persistent) all still work and take precedence over the built-
in identity when set.
This commit is contained in:
patriceckhart 2026-04-19 17:39:38 +02:00
parent f5719c6be1
commit 05d0df91b8
6 changed files with 31 additions and 93 deletions

View file

@ -6,8 +6,10 @@ import (
"time"
)
// ToolSummary is a name+one-line description, used when rendering the
// "available tools" section of the system prompt.
// ToolSummary is a name+one-line description. Kept for backwards
// compatibility with callers that still pass tool summaries in; the
// built-in system prompt no longer lists tools by name, since the
// tool schemas themselves already reach the model.
type ToolSummary struct {
Name string
Description string
@ -23,6 +25,22 @@ type SystemPromptOpts struct {
}
// BuildSystemPrompt constructs the system prompt.
//
// Design note: the prompt is intentionally tiny. Every byte here
// is re-sent on every request (cached after the first, but still
// counts toward cache-write on turn 1 and live context throughout).
// We avoid:
//
// - Listing the tool names and descriptions (the provider sends
// the tool schemas separately; duplicating them costs tokens
// for zero benefit, the model already sees the tools).
// - Repeating generic coding-assistant advice the frontier models
// already internalise ("always read before editing", "prefer
// minimal diffs", "don't apologize"). These were free tokens
// on older models; they are pure overhead now.
//
// Anything the user explicitly needs can still be added via
// --system-prompt, --append-system-prompt, or $ZOT_HOME/SYSTEM.md.
func BuildSystemPrompt(o SystemPromptOpts) string {
if o.Now.IsZero() {
o.Now = time.Now()
@ -39,10 +57,6 @@ func BuildSystemPrompt(o SystemPromptOpts) string {
sb.WriteString(o.Custom)
} else {
sb.WriteString(defaultIdentity)
sb.WriteString("\n\n")
sb.WriteString(renderToolsSection(o.Tools))
sb.WriteString("\n")
sb.WriteString(defaultGuidelines)
}
for _, a := range o.Append {
@ -57,26 +71,4 @@ func BuildSystemPrompt(o SystemPromptOpts) string {
return sb.String()
}
func renderToolsSection(tools []ToolSummary) string {
if len(tools) == 0 {
return "No tools are available in this session."
}
var sb strings.Builder
sb.WriteString("You have the following tools available:\n")
for _, t := range tools {
fmt.Fprintf(&sb, "- %s: %s\n", t.Name, t.Description)
}
return sb.String()
}
const defaultIdentity = `You are zot, a lightweight terminal coding assistant.
You help a developer by reading files, writing files, editing files, and running shell commands.
You are concise. You explain your plan briefly, then act. You do not apologize or hedge.`
const defaultGuidelines = `Operating guidelines:
- Prefer "edit" over "write" for existing files. Always read a file before editing it.
- Before running "bash", explain what the command will do in one short sentence.
- Avoid destructive commands (rm -rf, dropping tables, force-pushing, etc.) unless the user explicitly asks.
- Keep shell commands non-interactive (pass -y / --yes where needed; pipe "yes" if required).
- When unsure about a file's contents or structure, read it first rather than guess.
- When you are done, reply with a short summary of what you changed and any commands the user should run.`
const defaultIdentity = `You are zot, a lightweight terminal coding agent. Be concise, act on the user's request directly, and reply with a short summary when done.`

View file

@ -36,18 +36,10 @@ type bashArgs struct {
Timeout int `json:"timeout,omitempty"`
}
const bashSchema = `{
"type":"object",
"properties":{
"command":{"type":"string","description":"Shell command to execute."},
"timeout":{"type":"integer","description":"Timeout in seconds. No default timeout."}
},
"required":["command"],
"additionalProperties":false
}`
const bashSchema = `{"type":"object","properties":{"command":{"type":"string"},"timeout":{"type":"integer"}},"required":["command"]}`
func (t *BashTool) Name() string { return "bash" }
func (t *BashTool) Description() string { return "Run a shell command. stdout and stderr are merged." }
func (t *BashTool) Description() string { return "Run a shell command. stdout+stderr merged." }
func (t *BashTool) Schema() json.RawMessage { return json.RawMessage(bashSchema) }
func (t *BashTool) Execute(ctx context.Context, raw json.RawMessage, progress func(string)) (core.ToolResult, error) {

View file

@ -28,31 +28,11 @@ type editArgs struct {
Edits []editOp `json:"edits"`
}
const editSchema = `{
"type":"object",
"properties":{
"path":{"type":"string","description":"Path to the file to edit (relative or absolute)"},
"edits":{
"type":"array",
"description":"One or more targeted replacements. Each oldText must match exactly once in the original file.",
"items":{
"type":"object",
"properties":{
"oldText":{"type":"string","description":"Exact text to replace. Must be unique in the file."},
"newText":{"type":"string","description":"Replacement text."}
},
"required":["oldText","newText"],
"additionalProperties":false
}
}
},
"required":["path","edits"],
"additionalProperties":false
}`
const editSchema = `{"type":"object","properties":{"path":{"type":"string"},"edits":{"type":"array","items":{"type":"object","properties":{"oldText":{"type":"string"},"newText":{"type":"string"}},"required":["oldText","newText"]}}},"required":["path","edits"]}`
func (t *EditTool) Name() string { return "edit" }
func (t *EditTool) Description() string {
return "Edit an existing file via exact-match replacements. Preserves line endings and BOM."
return "Edit a file via exact-match replacements. Each oldText must be unique in the file."
}
func (t *EditTool) Schema() json.RawMessage { return json.RawMessage(editSchema) }

View file

@ -31,20 +31,11 @@ type readArgs struct {
Limit int `json:"limit,omitempty"`
}
const readSchema = `{
"type":"object",
"properties":{
"path":{"type":"string","description":"Path to the file to read (relative or absolute)"},
"offset":{"type":"integer","description":"Line number to start reading from (1-indexed)"},
"limit":{"type":"integer","description":"Maximum number of lines to read"}
},
"required":["path"],
"additionalProperties":false
}`
const readSchema = `{"type":"object","properties":{"path":{"type":"string"},"offset":{"type":"integer"},"limit":{"type":"integer"}},"required":["path"]}`
func (t *ReadTool) Name() string { return "read" }
func (t *ReadTool) Description() string {
return "Read the contents of a text file or an image (png/jpg/jpeg/gif/webp). Large files are truncated."
return "Read a file. Images (png/jpg/gif/webp) return inline."
}
func (t *ReadTool) Schema() json.RawMessage { return json.RawMessage(readSchema) }

View file

@ -22,19 +22,11 @@ type writeArgs struct {
Content string `json:"content"`
}
const writeSchema = `{
"type":"object",
"properties":{
"path":{"type":"string","description":"Path to the file to write (relative or absolute)"},
"content":{"type":"string","description":"File contents. Overwrites any existing file."}
},
"required":["path","content"],
"additionalProperties":false
}`
const writeSchema = `{"type":"object","properties":{"path":{"type":"string"},"content":{"type":"string"}},"required":["path","content"]}`
func (t *WriteTool) Name() string { return "write" }
func (t *WriteTool) Description() string {
return "Write content to a file. Creates parent directories. Overwrites existing files."
return "Write a file. Creates parent dirs. Overwrites."
}
func (t *WriteTool) Schema() json.RawMessage { return json.RawMessage(writeSchema) }

View file

@ -52,21 +52,12 @@ func (*Tool) Name() string { return "skill" }
// Description tells the model what this tool does. Kept blunt so the
// model reliably uses it instead of guessing what a "skill" is.
func (*Tool) Description() string {
return "Load the full body of a named skill. Use this when the user's request matches one of the skills listed in the system prompt; the tool returns the skill's instructions, which you should then follow."
return "Load a named skill's instructions. Use when the user's request matches a skill listed above."
}
// Schema is one required string parameter: the skill name.
func (*Tool) Schema() json.RawMessage {
return json.RawMessage(`{
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "The skill name (must match one listed in the system prompt)."
}
},
"required": ["name"]
}`)
return json.RawMessage(`{"type":"object","properties":{"name":{"type":"string"}},"required":["name"]}`)
}
// Execute returns the markdown body of the requested skill.