hermes-bsd/agent
Teknium 769ee86cd2
feat(kanban): attach images referenced in task bodies to worker vision (#34210)
Kanban workers now scan the task body for local image paths and
http(s) image URLs and attach them to the worker's first user turn —
matching the CLI/gateway behaviour for inbound images. Before, a
user pasting `/home/me/screenshot.png` or `https://example.com/img.png`
into a kanban task description had it sent to the model as plain
text and the pixels were never seen.

How it works:
* agent/image_routing.py gains extract_image_refs(text) → (paths, urls)
  that mirrors gateway/platforms/base.py:extract_local_files (absolute /
  ~-relative paths, image extensions only, ignores fenced/inline code).
* build_native_content_parts() accepts an optional image_urls= kwarg
  and emits passthrough image_url parts for remote URLs alongside the
  base64 data: URLs used for local paths.
* cli.py (single-query/quiet branch — the path every dispatcher-spawned
  worker takes) detects HERMES_KANBAN_TASK, reads the task body via
  kanban_db.get_task, runs extract_image_refs, and threads the results
  into the existing image-routing decision (native vs text). Best-effort:
  enrichment failures never block worker startup.

Tested:
* tests/agent/test_image_routing.py — 22 new tests for extract_image_refs
  and URL pass-through in build_native_content_parts.
* tests/hermes_cli/test_kanban_worker_image_extraction.py — 10 new tests
  driving real kanban_db round-trip (create task → read body → extract
  refs → build parts).
* E2E: created a fake kanban task with a body referencing both a local
  PNG and an https URL; verified the worker pipeline produces a
  multimodal user turn with 1 text part + 2 image_url parts (data URL
  for the local file, passthrough URL for the remote).
2026-05-28 17:50:42 -07:00
..
lsp
secret_sources perf(cli): cut hermes startup 63% — flip head-to-head vs codex (#31968) 2026-05-25 03:06:39 -07:00
transports fix(codex): omit tools key from Codex Responses kwargs when no tools registered 2026-05-27 11:46:17 -07:00
__init__.py fix(agent): preload jiter native parser 2026-05-28 00:20:11 -07:00
account_usage.py
agent_init.py feat(context-engine): host contract for external context engines 2026-05-28 01:45:30 -07:00
agent_runtime_helpers.py fix(agent): re-pad reasoning_content on cross-provider fallback to require-side providers 2026-05-28 03:21:00 -07:00
anthropic_adapter.py feat: add claude-opus-4.8 and claude-opus-4.8-fast (#34003) 2026-05-28 10:31:59 -07:00
async_utils.py
auxiliary_client.py feat(auth) normalise the way in which we check whether a user has free/paid access to nous portal so we can expose behaviour and error messages accordingly. 2026-05-28 00:19:31 -07:00
azure_identity_adapter.py
background_review.py test: cover ci-unblocker production regressions 2026-05-27 22:14:53 -07:00
bedrock_adapter.py
browser_provider.py
browser_registry.py
chat_completion_helpers.py feat(agent): buffer retry/fallback status, surface only on terminal failure (#33816) 2026-05-28 04:53:27 -07:00
codex_responses_adapter.py fix(xai-oauth): strip service_tier and add safety-net sanitization for slash enums 2026-05-27 05:25:38 -07:00
codex_runtime.py refactor(codex): drop SDK responses.stream() helper; consume events directly (#33042) 2026-05-27 00:30:06 -07:00
context_compressor.py fix(compressor): ABC compliance — total_tokens, api_mode, logger consistency 2026-05-23 17:38:19 -07:00
context_engine.py feat(context-engine): host contract for external context engines 2026-05-28 01:45:30 -07:00
context_references.py
conversation_compression.py feat(context-engine): host contract for external context engines 2026-05-28 01:45:30 -07:00
conversation_loop.py feat: expose completed-turn message context to memory providers 2026-05-29 02:16:43 +05:30
copilot_acp_client.py fix: guard yaml.safe_load, flock unlock, TOCTOU races, and atomic writes 2026-05-19 00:12:41 -07:00
credential_persistence.py fix: avoid persisting borrowed credential secrets (#31416) 2026-05-25 00:32:08 -07:00
credential_pool.py fix(anthropic): API-key path skips OAuth autodiscovery + prunes stale entries 2026-05-25 17:41:40 -07:00
credential_sources.py docs(auth): replace stale 'hermes login' references with 'hermes auth add' 2026-05-26 15:41:11 -07:00
curator.py fix: preserve skill packages during curator consolidation 2026-05-27 13:39:58 -07:00
curator_backup.py fix(skills): prune dependency/venv dirs from all skill scanners (#30042) 2026-05-21 14:18:02 -07:00
display.py chore(web): remove web_crawl tool + provider crawl plumbing (#33824) 2026-05-28 04:52:42 -07:00
error_classifier.py fix(agent): fallback immediately on provider content-policy blocks (#33883) 2026-05-28 07:28:24 -07:00
file_safety.py fix(security): block read_file on project-local .env files 2026-05-25 03:40:47 -07:00
gemini_cloudcode_adapter.py
gemini_native_adapter.py
gemini_schema.py
google_code_assist.py
google_oauth.py docs(auth): replace stale 'hermes login' references with 'hermes auth add' 2026-05-26 15:41:11 -07:00
i18n.py
image_gen_provider.py fix(image_gen): cache xAI ephemeral URL responses to disk (#26942) (#31759) 2026-05-24 18:10:47 -07:00
image_gen_registry.py
image_routing.py feat(kanban): attach images referenced in task bodies to worker vision (#34210) 2026-05-28 17:50:42 -07:00
insights.py
iteration_budget.py
jiter_preload.py fix(agent): preload jiter native parser 2026-05-28 00:20:11 -07:00
lmstudio_reasoning.py
manual_compression_feedback.py
markdown_tables.py
memory_manager.py feat: expose completed-turn message context to memory providers 2026-05-29 02:16:43 +05:30
memory_provider.py feat: expose completed-turn message context to memory providers 2026-05-29 02:16:43 +05:30
message_sanitization.py
model_metadata.py fix: stop probe stepdown without provider context limit 2026-05-28 12:26:53 -07:00
models_dev.py remove Vercel AI Gateway and Vercel Sandbox (#33067) 2026-05-27 00:43:32 -07:00
moonshot_schema.py
nous_rate_guard.py
onboarding.py
plugin_llm.py
portal_tags.py
process_bootstrap.py
prompt_builder.py remove Vercel AI Gateway and Vercel Sandbox (#33067) 2026-05-27 00:43:32 -07:00
prompt_caching.py
rate_limit_tracker.py
redact.py fix(redact): pass web URLs through unchanged (#34029) 2026-05-28 11:32:39 -07:00
retry_utils.py
shell_hooks.py fix: guard yaml.safe_load, flock unlock, TOCTOU races, and atomic writes 2026-05-19 00:12:41 -07:00
skill_bundles.py feat(skills): add skill bundles — alias /<name> loads multiple skills (#28373) 2026-05-18 21:38:05 -07:00
skill_commands.py
skill_preprocessing.py fix: treat inline-shell timeout guard as timeout 2026-05-18 19:36:04 -07:00
skill_utils.py fix(skills): load Linux-tagged skills on Termux (android sys.platform) 2026-05-21 19:08:38 -07:00
stream_diag.py feat(agent): buffer retry/fallback status, surface only on terminal failure (#33816) 2026-05-28 04:53:27 -07:00
subdirectory_hints.py fix(subdirectory_hints): prevent loading AGENTS.md outside workspace 2026-05-25 23:17:33 -07:00
system_prompt.py fix(profiles): cross-profile soft guard on file-write tools + system-prompt hint (#31290) 2026-05-24 00:38:17 -07:00
think_scrubber.py
title_generator.py
tool_dispatch_helpers.py feat(security): promptware defense — shared threat patterns + memory load-time scan + tool-result delimiters (#32269) 2026-05-25 14:52:24 -07:00
tool_executor.py fix(cli): surface tool failures with specific error messages 2026-05-23 21:03:51 -07:00
tool_guardrails.py fix: add recovery hints to loop guard warnings 2026-05-19 00:12:12 -07:00
tool_result_classification.py
trajectory.py
transcription_provider.py feat(stt): add register_transcription_provider() plugin hook 2026-05-25 01:41:19 -07:00
transcription_registry.py feat(stt): add register_transcription_provider() plugin hook 2026-05-25 01:41:19 -07:00
tts_provider.py feat(tts): add register_tts_provider() plugin hook (closes #30398) 2026-05-24 18:04:54 -07:00
tts_registry.py feat(tts): add register_tts_provider() plugin hook (closes #30398) 2026-05-24 18:04:54 -07:00
usage_pricing.py feat: add claude-opus-4.8 and claude-opus-4.8-fast (#34003) 2026-05-28 10:31:59 -07:00
video_gen_provider.py
video_gen_registry.py
web_search_provider.py chore(web): remove web_crawl tool + provider crawl plumbing (#33824) 2026-05-28 04:52:42 -07:00
web_search_registry.py chore(web): remove web_crawl tool + provider crawl plumbing (#33824) 2026-05-28 04:52:42 -07:00