Fix Bedrock streaming and provider setup docs

2026-06-26 21:36:31 +02:00 · 2026-06-05 08:31:54 +02:00 · 2026-06-05 08:31:54 +02:00 · 7a7bf0b52c
commit 7a7bf0b52c
parent 498b769c07
5 changed files with 393 additions and 16 deletions
--- a/docs/providers.md
+++ b/docs/providers.md
@ -0,0 +1,210 @@
+# zot providers
+
+zot ships with built-in providers and a model catalog. You can select models
+with `/model`, list them with `zot --list-models`, and add private models in
+`$ZOT_HOME/models.json`.
+
+## Login methods
+
+Use `/login` in interactive mode.
+
+- `api key`: stores an API key in `$ZOT_HOME/auth.json` when the provider uses a normal key.
+- `subscription`: stores OAuth credentials for subscription-backed providers.
+
+Use `/logout` to remove stored credentials.
+
+Some providers need more than a single pasted key. For those providers,
+`/login` shows setup instructions instead of opening a localhost browser form.
+This avoids broken browser flows in SSH, containers, and `kubectl exec`
+sessions.
+
+Setup-instruction providers:
+
+- Amazon Bedrock
+- Google Vertex AI
+- Cloudflare Workers AI
+- Cloudflare AI Gateway
+- Azure OpenAI Responses
+
+## Subscription providers
+
+These providers support subscription login:
+
+| Provider | Notes |
+| --- | --- |
+| Anthropic | Claude Pro/Max OAuth credentials. |
+| OpenAI Codex | ChatGPT Plus/Pro Codex subscription route. Separate from the OpenAI API-key provider. |
+| Kimi | Kimi subscription login. |
+| GitHub Copilot | GitHub Copilot token flow. |
+
+OAuth tokens are stored in `$ZOT_HOME/auth.json` and refreshed when refresh is
+available.
+
+## API-key providers
+
+These providers can use environment variables. Simple API-key providers can
+also be configured through `/login`. Providers that require extra cloud setup
+show instructions and should be configured with environment variables.
+
+| Provider | Environment variable | Stored key |
+| --- | --- | --- |
+| Anthropic | `ANTHROPIC_API_KEY` | `anthropic` |
+| OpenAI | `OPENAI_API_KEY` | `openai` |
+| OpenAI Responses | `OPENAI_API_KEY` | `openai-responses` |
+| Kimi | `KIMI_API_KEY` or `MOONSHOT_API_KEY` | `kimi` |
+| Google Gemini | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | `google` |
+| DeepSeek | `DEEPSEEK_API_KEY` | `deepseek` |
+| Moonshot AI | `MOONSHOT_API_KEY` | `moonshotai` |
+| Moonshot AI China | `MOONSHOT_API_KEY` | `moonshotai-cn` |
+| Groq | `GROQ_API_KEY` | `groq` |
+| xAI | `XAI_API_KEY` | `xai` |
+| Cerebras | `CEREBRAS_API_KEY` | `cerebras` |
+| Together AI | `TOGETHER_API_KEY` | `together` |
+| Hugging Face | `HF_TOKEN` | `huggingface` |
+| OpenRouter | `OPENROUTER_API_KEY` | `openrouter` |
+| Mistral | `MISTRAL_API_KEY` | `mistral` |
+| ZAI | `ZAI_API_KEY` | `zai` |
+| Xiaomi MiMo | `XIAOMI_API_KEY` | `xiaomi` |
+| Xiaomi Token Plan Amsterdam | `XIAOMI_TOKEN_PLAN_AMS_API_KEY` | `xiaomi-token-plan-ams` |
+| Xiaomi Token Plan China | `XIAOMI_TOKEN_PLAN_CN_API_KEY` | `xiaomi-token-plan-cn` |
+| Xiaomi Token Plan Singapore | `XIAOMI_TOKEN_PLAN_SGP_API_KEY` | `xiaomi-token-plan-sgp` |
+| MiniMax | `MINIMAX_API_KEY` | `minimax` |
+| MiniMax China | `MINIMAX_CN_API_KEY` or `MINIMAX_API_KEY` | `minimax-cn` |
+| Fireworks | `FIREWORKS_API_KEY` | `fireworks` |
+| Vercel AI Gateway | `AI_GATEWAY_API_KEY` | `vercel-ai-gateway` |
+| OpenCode Zen | `OPENCODE_API_KEY` | `opencode` |
+| OpenCode Go | `OPENCODE_API_KEY` | `opencode-go` |
+| GitHub Copilot token | `COPILOT_GITHUB_TOKEN` or `GITHUB_COPILOT_TOKEN` | `github-copilot` |
+| Cloudflare Workers AI | `CLOUDFLARE_API_KEY` | `cloudflare-workers-ai` |
+| Cloudflare AI Gateway | `CLOUDFLARE_API_KEY` | `cloudflare-ai-gateway` |
+| Azure OpenAI Responses | `AZURE_OPENAI_API_KEY` | `azure-openai-responses` |
+
+Example:
+
+```bash
+export OPENROUTER_API_KEY=...
+zot --provider openrouter
+```
+
+## Cloud providers
+
+### Amazon Bedrock
+
+Bedrock is configured with AWS credentials, not a generic zot API-key entry.
+Use one of these credential sources:
+
+```bash
+# AWS profile
+export AWS_PROFILE=your-profile
+
+# IAM access keys
+export AWS_ACCESS_KEY_ID=AKIA...
+export AWS_SECRET_ACCESS_KEY=...
+export AWS_SESSION_TOKEN=... # only for temporary credentials
+
+# Bedrock API key bearer token
+export AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-...
+
+# Region
+export AWS_REGION=us-east-1
+```
+
+ECS task roles, IRSA, and other AWS SDK credential-chain sources are also
+supported.
+
+Example:
+
+```bash
+AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-... AWS_REGION=us-east-1 \
+  zot --provider amazon-bedrock --model anthropic.claude-sonnet-4-5-20250929-v1:0
+```
+
+Some Bedrock models require regional inference-profile IDs for on-demand
+throughput, such as `us.` or `eu.` prefixed model IDs. zot rewrites known
+families automatically where possible. Explicit profile IDs and ARNs are left
+unchanged.
+
+### Google Vertex AI
+
+Vertex can use a Google API key when available:
+
+```bash
+export GOOGLE_CLOUD_API_KEY=...
+zot --provider google-vertex
+```
+
+For service-account or application-default credentials, set the standard
+Google environment variables used by your deployment.
+
+### Cloudflare AI Gateway
+
+Cloudflare AI Gateway needs a Cloudflare token plus account and gateway IDs:
+
+```bash
+export CLOUDFLARE_API_KEY=...
+export CLOUDFLARE_ACCOUNT_ID=...
+export CLOUDFLARE_GATEWAY_ID=...
+zot --provider cloudflare-ai-gateway
+```
+
+### Cloudflare Workers AI
+
+Workers AI needs a Cloudflare token and account ID:
+
+```bash
+export CLOUDFLARE_API_KEY=...
+export CLOUDFLARE_ACCOUNT_ID=...
+zot --provider cloudflare-workers-ai
+```
+
+### Azure OpenAI Responses
+
+```bash
+export AZURE_OPENAI_API_KEY=...
+export AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
+export AZURE_OPENAI_API_VERSION=2024-02-01 # optional
+zot --provider azure-openai-responses
+```
+
+If your Azure deployment names differ from zot model IDs, add model overrides
+in `$ZOT_HOME/models.json`.
+
+## Auth file
+
+Credentials are stored in `$ZOT_HOME/auth.json` with user-only permissions
+when zot creates the file.
+
+Example:
+
+```json
+{
+  "anthropic": { "api_key": "sk-ant-..." },
+  "openai": { "api_key": "sk-..." },
+  "google": { "api_key": "..." },
+  "additional_api_key_creds": {
+    "openrouter": { "api_key": "..." },
+    "mistral": { "api_key": "..." }
+  }
+}
+```
+
+The top-level keys are used for providers with dedicated credential fields.
+Other API-key providers are stored under `additional_api_key_creds`. Prefer
+`/login` so zot writes the correct schema.
+
+## Custom providers and models
+
+Use `$ZOT_HOME/models.json` for private models, deployment aliases, local
+servers, or OpenAI-compatible gateways that are not in the built-in catalog.
+User entries override built-in entries with the same provider and model ID.
+
+## Credential resolution
+
+For each request, zot checks credentials in this order:
+
+1. Explicit CLI key, such as `--api-key`.
+2. Provider-specific environment variables.
+3. `$ZOT_HOME/auth.json`.
+4. Custom provider credentials from `$ZOT_HOME/models.json`, when configured.
+
+Bedrock then uses the AWS SDK credential chain for the actual request.
--- a/packages/agent/modes/interactive.go
+++ b/packages/agent/modes/interactive.go
@ -3405,7 +3405,79 @@ func (i *Interactive) doLogout(target string) {
 	}
 }

+func providerSetupInfo(provider string) (string, []string, bool) {
+	const docsURL = "https://raw.githubusercontent.com/patriceckhart/zot/main/docs/providers.md"
+	switch provider {
+	case "amazon-bedrock":
+		return "Amazon Bedrock setup", []string{
+			"Amazon Bedrock uses AWS credentials instead of a generic zot API-key entry.",
+			"Configure an AWS profile, IAM keys, bearer token, or role-based credentials.",
+			"",
+			"For Bedrock API keys, set:",
+			"  AWS_BEARER_TOKEN_BEDROCK=...",
+			"  AWS_REGION=us-east-1",
+			"",
+			"Docs:",
+			"  " + docsURL,
+		}, true
+	case "google-vertex":
+		return "Google Vertex AI setup", []string{
+			"Google Vertex AI usually uses Google Cloud credentials and project settings.",
+			"Set a Google API key, application-default credentials, or a service account.",
+			"",
+			"Common environment:",
+			"  GOOGLE_CLOUD_API_KEY=...",
+			"  GOOGLE_CLOUD_PROJECT=...",
+			"  GOOGLE_CLOUD_LOCATION=us-central1",
+			"",
+			"Docs:",
+			"  " + docsURL,
+		}, true
+	case "cloudflare-workers-ai":
+		return "Cloudflare Workers AI setup", []string{
+			"Cloudflare Workers AI needs both an API token and an account ID.",
+			"",
+			"Set:",
+			"  CLOUDFLARE_API_KEY=...",
+			"  CLOUDFLARE_ACCOUNT_ID=...",
+			"",
+			"Docs:",
+			"  " + docsURL,
+		}, true
+	case "cloudflare-ai-gateway":
+		return "Cloudflare AI Gateway setup", []string{
+			"Cloudflare AI Gateway needs an API token, account ID, and gateway ID.",
+			"",
+			"Set:",
+			"  CLOUDFLARE_API_KEY=...",
+			"  CLOUDFLARE_ACCOUNT_ID=...",
+			"  CLOUDFLARE_GATEWAY_ID=...",
+			"",
+			"Docs:",
+			"  " + docsURL,
+		}, true
+	case "azure-openai-responses":
+		return "Azure OpenAI Responses setup", []string{
+			"Azure OpenAI needs an API key plus your Azure endpoint or deployment setup.",
+			"",
+			"Set:",
+			"  AZURE_OPENAI_API_KEY=...",
+			"  AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com",
+			"  AZURE_OPENAI_API_VERSION=2024-02-01",
+			"",
+			"Docs:",
+			"  " + docsURL,
+		}, true
+	default:
+		return "", nil, false
+	}
+}
+
 func (i *Interactive) startAPIKeyFlow(provider string) {
+	if title, lines, ok := providerSetupInfo(provider); ok {
+		i.dialog.ShowInfo(title, lines)
+		return
+	}
 	if provider == "kimi" && i.cfg.SetKimiCLIFallbackDisabled != nil {
 		_ = i.cfg.SetKimiCLIFallbackDisabled(false)
 	}
--- a/packages/agent/modes/login_dialog.go
+++ b/packages/agent/modes/login_dialog.go
@ -3,6 +3,7 @@ package modes
 import (
 	"fmt"
 	"path/filepath"
+	"sort"

 	"github.com/patriceckhart/zot/packages/provider"
 	"github.com/patriceckhart/zot/packages/provider/auth"
@ -21,6 +22,7 @@ const (
 	loginStepProvider            // pick anthropic vs openai vs kimi
 	loginStepWaiting             // browser open, waiting for callback
 	loginStepPasteCode           // user pastes the auth code here
+	loginStepInfo                // informational setup guidance
 	loginStepDone                // success or error, waiting for key to dismiss
 )

@ -29,14 +31,16 @@ const loginProviderPageSize = 8
 // loginDialog is a tiny inline dialog rendered above the editor while
 // the user picks their login method and provider.
 type loginDialog struct {
-	step     loginStep
-	method   string // "apikey" | "oauth"
-	provider string // "anthropic" | "openai" | "openai-codex" | "kimi" | "google"
-	message  string
-	success  bool
-	url      string
-	cursor   int
-	codeEd   *tui.Editor
+	step      loginStep
+	method    string // "apikey" | "oauth"
+	provider  string // "anthropic" | "openai" | "openai-codex" | "kimi" | "google"
+	message   string
+	success   bool
+	url       string
+	cursor    int
+	codeEd    *tui.Editor
+	infoTitle string
+	infoLines []string

 	// status is a snapshot of the current login state for each
 	// provider, captured when Open() runs. Rendered above the
@ -68,6 +72,8 @@ func (d *loginDialog) Open(zotHome string) {
 	d.success = false
 	d.url = ""
 	d.cursor = 0
+	d.infoTitle = ""
+	d.infoLines = nil
 	d.status = map[string]string{}
 	for _, p := range providersForMethod("apikey") {
 		d.status[p] = ""
@ -199,6 +205,18 @@ func (d *loginDialog) Render(th tui.Theme, width int) []string {
 		lines = append(lines, "")
 		lines = append(lines, th.FG256(th.Muted, "enter submits - esc cancels"))
 		lines = append(lines, frameRule(th, width))
+	case loginStepInfo:
+		title := d.infoTitle
+		if title == "" {
+			title = "login - setup"
+		}
+		lines = append(lines, frameHeader(th, title, width))
+		for _, l := range d.infoLines {
+			lines = append(lines, l)
+		}
+		lines = append(lines, "")
+		lines = append(lines, th.FG256(th.Muted, "press any key to close"))
+		lines = append(lines, frameRule(th, width))
 	case loginStepDone:
 		title := "login - failed"
 		body := th.FG256(th.Error, d.message)
@ -221,10 +239,16 @@ func (d *loginDialog) Render(th tui.Theme, width int) []string {
 // consumer Gemini Advanced login does not, and DeepSeek has no
 // subscription product at all).
 func providersForMethod(method string) []string {
+	var providers []string
 	if method == "oauth" {
-		return []string{"anthropic", "openai-codex", "kimi", "github-copilot"}
+		providers = []string{"anthropic", "openai-codex", "kimi", "github-copilot"}
+	} else {
+		providers = auth.APIKeyProviders()
 	}
-	return auth.APIKeyProviders()
+	sort.Slice(providers, func(a, b int) bool {
+		return providerLabel(providers[a]) < providerLabel(providers[b])
+	})
+	return providers
 }

 // providerLabel returns the user-facing label for a provider id.
@ -347,6 +371,9 @@ func (d *loginDialog) HandleKey(k tui.Key) loginDialogAction {
 		return d.handleWaitingKey(k)
 	case loginStepPasteCode:
 		return d.handlePasteCodeKey(k)
+	case loginStepInfo:
+		d.Close()
+		return loginDialogAction{Close: true}
 	case loginStepDone:
 		d.Close()
 		return loginDialogAction{Close: true}
@ -430,6 +457,17 @@ func (d *loginDialog) ShowWaiting(url string) {
 	d.url = url
 }

+// ShowInfo transitions to an informational setup dialog.
+// No-op if the user has already dismissed the dialog.
+func (d *loginDialog) ShowInfo(title string, lines []string) {
+	if d.step == loginStepClosed {
+		return
+	}
+	d.step = loginStepInfo
+	d.infoTitle = title
+	d.infoLines = lines
+}
+
 // ShowResult transitions to the done state with the given outcome.
 // No-op if the user has already dismissed the dialog.
 func (d *loginDialog) ShowResult(success bool, message string) {
--- a/packages/provider/amazon_bedrock.go
+++ b/packages/provider/amazon_bedrock.go
@ -450,6 +450,9 @@ func (c *bedrockClient) runStream(ctx context.Context, resp *http.Response, req
 			return
 		}
 		eventType := evt.headerString(":event-type")
+		if eventType == "" {
+			eventType = bedrockEventTypeFromPayload(evt.payload)
+		}
 		messageType := evt.headerString(":message-type")
 		if messageType == "exception" {
 			out <- EventDone{Stop: StopError, Err: fmt.Errorf("bedrock exception (%s): %s", evt.headerString(":exception-type"), string(evt.payload)), Message: finalMsg}
@ -468,7 +471,7 @@ func (c *bedrockClient) runStream(ctx context.Context, resp *http.Response, req
 					} `json:"toolUse"`
 				} `json:"start"`
 			}
-			if err := json.Unmarshal(evt.payload, &d); err != nil {
+			if err := unmarshalBedrockEventPayload(evt.payload, "contentBlockStart", &d); err != nil {
 				continue
 			}
 			st := &bedrockBlockState{}
@ -489,12 +492,13 @@ func (c *bedrockClient) runStream(ctx context.Context, resp *http.Response, req
 					} `json:"toolUse"`
 				} `json:"delta"`
 			}
-			if err := json.Unmarshal(evt.payload, &d); err != nil {
+			if err := unmarshalBedrockEventPayload(evt.payload, "contentBlockDelta", &d); err != nil {
 				continue
 			}
 			st := contentBlocks[d.ContentBlockIndex]
 			if st == nil {
-				continue
+				st = &bedrockBlockState{}
+				contentBlocks[d.ContentBlockIndex] = st
 			}
 			if d.Delta.Text != "" {
 				st.text.WriteString(d.Delta.Text)
@ -508,7 +512,7 @@ func (c *bedrockClient) runStream(ctx context.Context, resp *http.Response, req
 			var d struct {
 				ContentBlockIndex int `json:"contentBlockIndex"`
 			}
-			if err := json.Unmarshal(evt.payload, &d); err != nil {
+			if err := unmarshalBedrockEventPayload(evt.payload, "contentBlockStop", &d); err != nil {
 				continue
 			}
 			st := contentBlocks[d.ContentBlockIndex]
@ -531,7 +535,7 @@ func (c *bedrockClient) runStream(ctx context.Context, resp *http.Response, req
 			var d struct {
 				StopReason string `json:"stopReason"`
 			}
-			_ = json.Unmarshal(evt.payload, &d)
+			_ = unmarshalBedrockEventPayload(evt.payload, "messageStop", &d)
 			switch d.StopReason {
 			case "tool_use":
 				stop = StopToolUse
@ -551,7 +555,7 @@ func (c *bedrockClient) runStream(ctx context.Context, resp *http.Response, req
 					CacheWriteInputTokens int `json:"cacheWriteInputTokens"`
 				} `json:"usage"`
 			}
-			if err := json.Unmarshal(evt.payload, &d); err == nil {
+			if err := unmarshalBedrockEventPayload(evt.payload, "metadata", &d); err == nil {
 				usage.InputTokens = d.Usage.InputTokens
 				usage.OutputTokens = d.Usage.OutputTokens
 				usage.CacheReadTokens = d.Usage.CacheReadInputTokens
@ -574,6 +578,32 @@ type bedrockBlockState struct {
 	text      strings.Builder
 }

+func bedrockEventTypeFromPayload(payload []byte) string {
+	var outer map[string]json.RawMessage
+	if err := json.Unmarshal(payload, &outer); err != nil {
+		return ""
+	}
+	for _, name := range []string{"messageStart", "contentBlockStart", "contentBlockDelta", "contentBlockStop", "messageStop", "metadata"} {
+		if _, ok := outer[name]; ok {
+			return name
+		}
+	}
+	return ""
+}
+
+func unmarshalBedrockEventPayload(payload []byte, eventType string, dst any) error {
+	var outer map[string]json.RawMessage
+	if err := json.Unmarshal(payload, &outer); err == nil {
+		if wrapped, ok := outer[eventType]; ok {
+			return json.Unmarshal(wrapped, dst)
+		}
+	}
+	if err := json.Unmarshal(payload, dst); err != nil {
+		return fmt.Errorf("bedrock: parse %s payload: %w", eventType, err)
+	}
+	return nil
+}
+
 // ---- event-stream binary framing parser ----

 type eventStreamMessage struct {
--- a/packages/provider/amazon_bedrock_test.go
+++ b/packages/provider/amazon_bedrock_test.go
@ -82,6 +82,33 @@ func TestReadAWSCredentialsFile(t *testing.T) {
 	}
 }

+func TestBedrockEventPayloadHelpers(t *testing.T) {
+	wrapped := []byte(`{"contentBlockDelta":{"contentBlockIndex":0,"delta":{"text":"Hello"}}}`)
+	if got := bedrockEventTypeFromPayload(wrapped); got != "contentBlockDelta" {
+		t.Fatalf("event type = %q, want contentBlockDelta", got)
+	}
+	var delta struct {
+		ContentBlockIndex int `json:"contentBlockIndex"`
+		Delta             struct {
+			Text string `json:"text"`
+		} `json:"delta"`
+	}
+	if err := unmarshalBedrockEventPayload(wrapped, "contentBlockDelta", &delta); err != nil {
+		t.Fatal(err)
+	}
+	if delta.ContentBlockIndex != 0 || delta.Delta.Text != "Hello" {
+		t.Fatalf("unexpected wrapped delta: %+v", delta)
+	}
+
+	direct := []byte(`{"contentBlockIndex":1,"delta":{"text":"world"}}`)
+	if err := unmarshalBedrockEventPayload(direct, "contentBlockDelta", &delta); err != nil {
+		t.Fatal(err)
+	}
+	if delta.ContentBlockIndex != 1 || delta.Delta.Text != "world" {
+		t.Fatalf("unexpected direct delta: %+v", delta)
+	}
+}
+
 func TestResolveBedrockInferenceProfileID(t *testing.T) {
 	cases := []struct {
 		model  string