hermes-bsd/agent
Teknium c769be344a
fix(agent): recover from providers rejecting list-type tool content (#27344) (#30259)
Some providers (Xiaomi MiMo, some Alibaba endpoints, a long tail of
OpenAI-compatible servers) follow the OpenAI spec strictly and require
tool message `content` to be a string — they reject our list-type
content (text + image_url parts) with HTTP 400 'text is not set' /
'tool message content must be a string'.

Instead of an allowlist of known-good providers (maintenance burden,
guaranteed to miss aggregators like OpenRouter where the underlying
model determines support, not the aggregator name), this lands a
reactive recovery:

1. New `FailoverReason.multimodal_tool_content_unsupported` with a
   small pattern list covering the common 400 wordings.
2. `AIAgent._try_strip_image_parts_from_tool_messages` walks the API
   message list, downgrades any `role:tool` message whose content is
   list-with-image to a plain text summary (preserves text parts) in
   place, AND records the active (provider, model) in a session-scoped
   `_no_list_tool_content_models` set.
3. `_tool_result_content_for_active_model` short-circuits to a text
   summary when (provider, model) is in the cache — so after the first
   400 + retry, subsequent screenshots in the same session skip the
   round trip entirely.
4. Retry hook in `agent.conversation_loop` mirrors the existing
   `image_too_large` recovery: detect the reason, run the helper,
   retry once, fall through to the normal error path if no list-type
   tool content was actually present.

Cache is transient (per-session) by design — next session retries in
case the provider added support, no persistent state to maintain.

Fixes #27344. Closes #27351 (allowlist approach superseded by reactive
recovery).
2026-05-21 23:40:16 -07:00
..
lsp chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) 2026-05-17 02:29:41 -07:00
secret_sources feat(secrets): Bitwarden Secrets Manager integration with lazy bws install (#30035) 2026-05-21 14:10:34 -07:00
transports fix(xai): restore encrypted reasoning replay across turns 2026-05-20 23:12:45 -07:00
__init__.py
account_usage.py
agent_init.py fix(agent): widen toolset gate to context engine tools (#5544 sibling) 2026-05-21 23:18:37 -07:00
agent_runtime_helpers.py fix(gateway): harden kanban and provider cleanup races 2026-05-20 14:31:22 -07:00
anthropic_adapter.py feat(azure-foundry): add Microsoft Entra ID auth 2026-05-18 10:14:38 -07:00
async_utils.py fix(async): close unscheduled coroutines in all threadsafe bridges (#26584) 2026-05-15 14:00:01 -07:00
auxiliary_client.py fix(xai-oauth): pin inference base_url to x.ai origin (#28952) 2026-05-19 14:51:21 -07:00
azure_identity_adapter.py feat(azure-foundry): add Microsoft Entra ID auth 2026-05-18 10:14:38 -07:00
background_review.py chore: trim verbose comments/docstrings, add AUTHOR_MAP entry 2026-05-21 12:49:21 +05:30
bedrock_adapter.py chore(deps): lazy-install boto3/botocore for bedrock adapter 2026-05-17 02:31:18 -07:00
browser_provider.py fix(browser): self-review pass — dead-import, log levels, future-proofing 2026-05-17 04:04:15 -07:00
browser_registry.py fix(browser): self-review pass — dead-import, log levels, future-proofing 2026-05-17 04:04:15 -07:00
chat_completion_helpers.py fix(gateway): harden kanban and provider cleanup races 2026-05-20 14:31:22 -07:00
codex_responses_adapter.py fix(xai): restore encrypted reasoning replay across turns 2026-05-20 23:12:45 -07:00
codex_runtime.py fix(xai): surface provider 'error' SSE frame in Codex fallback stream (#27184) 2026-05-16 23:41:09 -07:00
context_compressor.py fix(compress): make abort-on-summary-failure opt-in via config flag (#28117) 2026-05-18 10:28:20 -07:00
context_engine.py
context_references.py
conversation_compression.py refactor(session-log): drop branch/compress re-point of session_log_file 2026-05-20 11:44:10 -07:00
conversation_loop.py fix(agent): recover from providers rejecting list-type tool content (#27344) (#30259) 2026-05-21 23:40:16 -07:00
copilot_acp_client.py fix: guard yaml.safe_load, flock unlock, TOCTOU races, and atomic writes 2026-05-19 00:12:41 -07:00
credential_pool.py fix(codex-oauth): quarantine terminal refresh errors so dead tokens are not replayed across sessions 2026-05-18 10:31:40 -07:00
credential_sources.py feat(xai-oauth): add xAI Grok OAuth (SuperGrok Subscription) provider 2026-05-15 12:11:32 -07:00
curator.py
curator_backup.py fix(skills): prune dependency/venv dirs from all skill scanners (#30042) 2026-05-21 14:18:02 -07:00
display.py chore: remove Atropos RL environments and tinker-atropos integration (#26106) 2026-05-15 10:36:38 +05:30
error_classifier.py fix(agent): recover from providers rejecting list-type tool content (#27344) (#30259) 2026-05-21 23:40:16 -07:00
file_safety.py security(file-safety): also write-deny <root>/.env when running under a profile (#15981) 2026-05-20 23:37:37 -07:00
gemini_cloudcode_adapter.py
gemini_native_adapter.py
gemini_schema.py
google_code_assist.py
google_oauth.py fix(security): guard os.chmod(parent) against / and top-level dirs 2026-05-20 22:56:55 -07:00
i18n.py
image_gen_provider.py
image_gen_registry.py
image_routing.py fix(agent): consult supports_vision override in auto-mode routing 2026-05-20 23:27:10 -07:00
insights.py
iteration_budget.py refactor(run_agent): extract OpenAI proxy, safe stdio, IterationBudget 2026-05-16 17:59:32 -07:00
lmstudio_reasoning.py
manual_compression_feedback.py
markdown_tables.py
memory_manager.py 🐛 fix(memory): require newline after context tag 2026-05-18 10:53:08 -07:00
memory_provider.py
message_sanitization.py refactor(run_agent): extract message sanitization to agent/message_sanitization.py 2026-05-16 17:41:09 -07:00
model_metadata.py fix(metadata): qwen3.6-plus has a 1M context window (#27008) 2026-05-17 02:31:18 -07:00
models_dev.py
moonshot_schema.py fix(moonshot): strip $ref siblings and collapse tuple items in tool schemas (#27104) 2026-05-16 13:02:19 -07:00
nous_rate_guard.py
onboarding.py
plugin_llm.py
portal_tags.py
process_bootstrap.py refactor(run_agent): extract OpenAI proxy, safe stdio, IterationBudget 2026-05-16 17:59:32 -07:00
prompt_builder.py fix(kanban): stale reclaim must not tick failure counter (#28680) 2026-05-19 03:15:18 -07:00
prompt_caching.py
rate_limit_tracker.py
redact.py perf(agent-loop): cut 47% of per-conversation function calls via 3 targeted hot-path optimizations (#28866) 2026-05-19 14:25:10 -07:00
retry_utils.py
shell_hooks.py fix: guard yaml.safe_load, flock unlock, TOCTOU races, and atomic writes 2026-05-19 00:12:41 -07:00
skill_bundles.py feat(skills): add skill bundles — alias /<name> loads multiple skills (#28373) 2026-05-18 21:38:05 -07:00
skill_commands.py fix(skills): load symlinked skill slash commands 2026-05-18 00:34:29 -07:00
skill_preprocessing.py fix: treat inline-shell timeout guard as timeout 2026-05-18 19:36:04 -07:00
skill_utils.py fix(skills): load Linux-tagged skills on Termux (android sys.platform) 2026-05-21 19:08:38 -07:00
stream_diag.py refactor(run_agent): extract stream diagnostics to agent/stream_diag.py 2026-05-16 18:28:17 -07:00
subdirectory_hints.py
system_prompt.py perf(prompt): cache kanban worker guidance at session init 2026-05-18 20:56:44 -07:00
think_scrubber.py
title_generator.py
tool_dispatch_helpers.py fix(agent): set tool_name on tool-result messages at construction time 2026-05-19 20:49:11 +01:00
tool_executor.py fix(agent): set tool_name on tool-result messages at construction time 2026-05-19 20:49:11 +01:00
tool_guardrails.py fix: add recovery hints to loop guard warnings 2026-05-19 00:12:12 -07:00
tool_result_classification.py
trajectory.py
usage_pricing.py
video_gen_provider.py
video_gen_registry.py
web_search_provider.py
web_search_registry.py