hermes-bsd/hermes_cli
Peetwan ebb81f10cb fix(tui_gateway): prevent WS disconnect under GIL pressure
Three targeted fixes for Desktop GUI WebSocket stability when agent
turns starve the uvicorn event loop of CPU (GIL contention):

1. Loosen ws_ping_timeout for loopback binds (QW-1)
   - Loopback (Desktop): ping 30s interval / 60s timeout
   - Non-loopback (Cloudflare Tunnel): unchanged 20/20
   - A GIL-heavy agent turn can stall the event loop past 20s;
     uvicorn's keepalive ping runs on that same starved loop, so a
     20s timeout kills an otherwise-healthy local connection over a
     recoverable stall. 60s rides out the stall without affecting
     half-open detection on public binds.

2. Coalesce streaming token frames in WSTransport (CF-2)
   - Buffer high-frequency delta frames (message.delta, reasoning.delta,
     thinking.delta) and flush as a batch every ~33ms (~30fps)
   - Non-streaming frames (RPC responses, control/tool/completion events)
     flush pending tokens first — wire ordering preserved
   - Thread-safe via threading.Lock; worker threads return immediately
     instead of blocking on per-token loop wakeups
   - Reduces event-loop wakeup churn by orders of magnitude during model
     streaming, directly cutting GIL pressure

3. Loop heartbeat watchdog (CF-1)
   - Self-rearming call_later tick (2s) measures drift between expected
     and actual fire time using loop.time() (monotonic)
   - Logs 'event loop stalled Ns (GIL pressure suspected)' when drift >5s
   - Turns mysterious WS drops into diagnosable log entries
   - Uses call_later chain (not a task) — dies with the loop, nothing
     to cancel on shutdown

Root cause: uvicorn's ws keepalive ping (20/20s) runs on the same
starved event loop as agent turns. Under GIL pressure from heavy agent
turns or delegation, the loop can't service the ping within 20s, so
the websockets protocol declares the connection dead. Reconnects fail
with ready_send_failed because the old process's loop is still wedged.

None of these fixes touch the model-facing message array, prompt
caching, message role alternation, or the wire protocol — they are
strictly display-transport improvements plus a config tweak and a
diagnostic log.

Tests: 762 passed, 17 skipped (0 failures) across test_tui_gateway_ws,
test_tui_gateway_server, test_web_server, and tui_gateway/ suites.
2026-06-30 03:11:13 -07:00
..
dashboard_auth refactor(dashboard-auth): drop redundant _interactive_providers helper 2026-06-29 04:25:18 -07:00
proxy fix(auth): honor NOUS_INFERENCE_BASE_URL env override for Nous OAuth sessions (#52270) 2026-06-25 00:11:15 -07:00
subcommands test(cli): pin the hermes serve decoupling contract 2026-06-28 22:11:48 -05:00
__init__.py chore: release v0.17.0 (2026.6.19) 2026-06-19 12:38:31 -07:00
_parser.py
_subprocess_compat.py revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853) 2026-06-27 15:59:00 -07:00
active_sessions.py fix(tui): preserve live session identity across compression (#49041) 2026-06-24 00:54:18 +05:30
auth.py fix(credentials): prefer ~/.hermes/.env over stale os.environ on key rotation (#55528) 2026-06-30 09:49:52 +00:00
auth_commands.py feat(providers): remove google-gemini-cli + google-antigravity OAuth providers (#50492) 2026-06-21 19:53:27 -07:00
azure_detect.py
backup.py fix(backup): include projects.db, kanban boards, and sibling stores in pre-update snapshot (#52889) 2026-06-26 19:23:33 +05:30
banner.py revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853) 2026-06-27 15:59:00 -07:00
blueprint_cmd.py
browser_connect.py
build_info.py
bundles.py
callbacks.py
checkpoints.py
claw.py revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853) 2026-06-27 15:59:00 -07:00
cli_agent_setup_mixin.py fix(cli): publish agent ref to cli module so memory on_session_end fires on exit 2026-06-19 16:59:43 -07:00
cli_commands_mixin.py revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853) 2026-06-27 15:59:00 -07:00
cli_output.py
clipboard.py revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853) 2026-06-27 15:59:00 -07:00
codex_models.py
codex_runtime_plugin_migration.py
codex_runtime_switch.py
colors.py
commands.py feat(moa): make /moa one-shot only; route preset switching through the model picker 2026-06-27 03:09:09 -07:00
completion.py
config.py fix(delegation): budget subagent summaries against parent context headroom 2026-06-30 03:07:40 -07:00
container_boot.py fix(container-boot): also autostart a gateway stranded in 'degraded' 2026-06-29 21:12:36 -07:00
context_switch_guard.py fix(cli): log instead of swallow preflight-warning errors; consistent TUI warning field 2026-06-21 16:31:56 +05:30
copilot_auth.py fix(windows): hide console-window flash on backend git/gh/wmic/bash subprocess spawns 2026-06-28 05:28:45 -07:00
cron.py fix(cron): don't report a false 'gateway not running' on external-provider instances (#54600) 2026-06-29 14:03:02 +10:00
curator.py
curses_ui.py
dashboard_register.py fix(cli): correct stale hermes auth login nous hints to hermes auth add nous (#53929) 2026-06-27 21:30:37 -07:00
debug.py
default_soul.py fix(soul): installers seed the real default persona, upgrade legacy empty templates (#52246) 2026-06-24 18:56:26 -07:00
dep_ensure.py fix(security): centralize credential-safe subprocess env (#29157) 2026-06-27 20:45:31 -07:00
dingtalk_auth.py
doctor.py revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853) 2026-06-27 15:59:00 -07:00
dump.py revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853) 2026-06-27 15:59:00 -07:00
env_loader.py perf(startup): parse config + plugin manifests with libyaml CSafeLoader (#54486) 2026-06-28 15:38:39 -07:00
fallback_cmd.py
fallback_config.py
gateway.py fix(windows): cover remaining console-flash spawn legs (#54417) 2026-06-28 13:49:08 -07:00
gateway_enroll.py fix(cli): correct stale hermes auth login nous hints to hermes auth add nous (#53929) 2026-06-27 21:30:37 -07:00
gateway_windows.py revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853) 2026-06-27 15:59:00 -07:00
goals.py feat(goals): completion contracts for /goal — evidence-based judging (#50501) 2026-06-22 12:20:09 -07:00
gui_uninstall.py
hooks.py feat(agent): add pre_verify hook and verify-on-stop coding guidance 2026-06-30 00:59:29 -05:00
inventory.py refactor(moa): share one virtual-provider row builder across pickers 2026-06-27 03:43:38 -07:00
kanban.py feat(kanban): typed block reasons + unblock-loop breaker (#52848) 2026-06-25 21:46:58 -07:00
kanban_db.py fix(kanban): retry write_txn on transient SQLITE_BUSY 2026-06-28 02:44:04 -07:00
kanban_decompose.py
kanban_diagnostics.py
kanban_specify.py
kanban_swarm.py refactor(kanban): fold worker/orchestrator skills into injected guidance (#50473) 2026-06-21 17:06:48 -07:00
logs.py
main.py feat(cli): add headless hermes serve backend; desktop no longer launches dashboard 2026-06-28 22:04:22 -05:00
managed_scope.py fix(managed-scope): honor managed scope in all standalone config loaders 2026-06-19 07:46:33 -07:00
managed_uv.py revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853) 2026-06-27 15:59:00 -07:00
mcp_catalog.py revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853) 2026-06-27 15:59:00 -07:00
mcp_config.py fix(mcp): auto-recover from invalid_client on stale OAuth client registration 2026-06-26 00:35:27 -07:00
mcp_picker.py
mcp_security.py fix(security): close hermes-0day MCP-persistence attack surface 2026-06-21 19:05:27 -07:00
mcp_startup.py fix(mcp): late-refresh must see desktop/dashboard discovery thread owner (#55514) 2026-06-30 02:08:37 -07:00
memory_oauth.py feat(memory): Honcho OAuth connect — desktop and CLI flows + token refresh (#44335) 2026-06-22 19:16:47 -05:00
memory_providers.py
memory_setup.py
middleware.py
migrate.py
moa_cmd.py feat(moa): expose MoA presets as selectable virtual models (#46081) 2026-06-25 13:52:06 -07:00
moa_config.py fix(moa): tolerate non-list reference_models in hand-edited MoA preset config 2026-06-27 03:43:16 -07:00
model_catalog.py
model_cost_guard.py
model_normalize.py
model_setup_flows.py fix(model): show MoA preset picker on selection and label MoA in the banner 2026-06-27 11:45:07 -07:00
model_switch.py fix(models): scope live-first picker merge to opencode aggregators only 2026-06-27 21:23:25 -07:00
models.py fix: normalize lmstudio base urls 2026-06-28 20:46:44 -07:00
nous_account.py
nous_auth_keepalive.py fix Nous auth refresh for idle agents 2026-06-21 22:43:48 -07:00
nous_billing.py
nous_subscription.py fix(browser): validate agent-browser is runnable, not just present (#51740) 2026-06-24 00:14:49 -07:00
oneshot.py fix(agent,cli): surface empty-body API errors and fail oneshot exit code 2026-06-28 02:05:20 -07:00
pairing.py
partial_compress.py
pets.py feat(pets): generation RPCs, non-blocking gallery + gateway plumbing 2026-06-24 13:48:38 -05:00
platforms.py
plugins.py feat(agent): add pre_verify hook and verify-on-stop coding guidance 2026-06-30 00:59:29 -05:00
plugins_cmd.py
portal_cli.py
profile_describer.py
profile_distribution.py revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853) 2026-06-27 15:59:00 -07:00
profiles.py perf(profiles): fix list_profiles O(N*M) wrapper rescan (6.4s -> 0.4s) 2026-06-29 02:35:57 -07:00
projects_cmd.py feat(projects): add per-profile project store 2026-06-25 16:40:26 -05:00
projects_db.py feat(projects): add per-profile project store 2026-06-25 16:40:26 -05:00
prompt_size.py
provider_catalog.py feat(moa): expose MoA presets as selectable virtual models (#46081) 2026-06-25 13:52:06 -07:00
providers.py fix: propagate key_env from custom_providers into ProviderDef 2026-06-29 02:25:48 -07:00
psutil_android.py
pt_input_extras.py
pty_bridge.py
relaunch.py revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853) 2026-06-27 15:59:00 -07:00
runtime_provider.py fix: normalize lmstudio base urls 2026-06-28 20:46:44 -07:00
secret_prompt.py
secrets_cli.py
security_advisories.py
security_audit.py
security_audit_startup.py style(security-audit): add explicit encoding to read_text calls (ruff PLW1514) 2026-06-21 19:05:27 -07:00
send_cmd.py fix(managed-scope): honor managed scope in config→env bridges too 2026-06-19 07:46:33 -07:00
service_manager.py fix(s6): dot-prefix gateway staging dir so svscan ignores it mid-build (#54834) 2026-06-29 21:33:00 +10:00
session_listing.py
session_recap.py
setup.py fix(gateway): only offer system-scope gateway install to root sessions (#53975) 2026-06-27 21:24:08 -07:00
setup_whatsapp_cloud.py
skills_config.py
skills_hub.py fix(skills-hub): surface per-tap providers (NVIDIA/OpenAI/...) in runtime search (#53191) 2026-06-26 11:04:41 -07:00
skin_engine.py
slack_cli.py fix(slack): subscribe to message.mpim + mpim scopes so group DMs work 2026-06-29 01:02:53 -07:00
sqlite_util.py feat(projects): add per-profile project store 2026-06-25 16:40:26 -05:00
status.py
stdio.py
suggestions_cmd.py
telegram_managed_bot.py
timeouts.py
tips.py fix: use os.pathsep, add tests, update tips for multi-root support 2026-06-27 04:01:12 +05:30
tools_config.py feat(xai): Imagine public-URL storage, chaining & video edit/extend 2026-06-29 21:11:58 -07:00
toolset_validation.py fix(config): surface invalid platform_toolsets instead of silently dropping tools (#38798) 2026-06-26 14:07:43 +05:30
uninstall.py
voice.py
web_git.py refactor(web_git): unify porcelain-v2 parsing into one walker 2026-06-28 14:29:59 -05:00
web_server.py fix(tui_gateway): prevent WS disconnect under GIL pressure 2026-06-30 03:11:13 -07:00
webhook.py
win_pty_bridge.py
write_approval_commands.py
xai_retirement.py