2026-02-19 00:57:31 -08:00
|
|
|
#!/usr/bin/env python3
|
|
|
|
|
"""
|
|
|
|
|
Memory Tool Module - Persistent Curated Memory
|
|
|
|
|
|
|
|
|
|
Provides bounded, file-backed memory that persists across sessions. Two stores:
|
|
|
|
|
- MEMORY.md: agent's personal notes and observations (environment facts, project
|
|
|
|
|
conventions, tool quirks, things learned)
|
|
|
|
|
- USER.md: what the agent knows about the user (preferences, communication style,
|
|
|
|
|
expectations, workflow habits)
|
|
|
|
|
|
|
|
|
|
Both are injected into the system prompt as a frozen snapshot at session start.
|
|
|
|
|
Mid-session writes update files on disk immediately (durable) but do NOT change
|
|
|
|
|
the system prompt -- this preserves the prefix cache for the entire session.
|
|
|
|
|
The snapshot refreshes on the next session start.
|
|
|
|
|
|
|
|
|
|
Entry delimiter: § (section sign). Entries can be multiline.
|
|
|
|
|
Character limits (not tokens) because char counts are model-independent.
|
|
|
|
|
|
|
|
|
|
Design:
|
|
|
|
|
- Single `memory` tool with action parameter: add, replace, remove, read
|
|
|
|
|
- replace/remove use short unique substring matching (not full text or IDs)
|
|
|
|
|
- Behavioral guidance lives in the tool schema description
|
|
|
|
|
- Frozen snapshot pattern: system prompt is stable, tool responses show live state
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
import json
|
Harden agent attack surface: scan writes to memory, skills, cron, and context files
The security scanner (skills_guard.py) was only wired into the hub install path.
All other write paths to persistent state — skills created by the agent, memory
entries, cron prompts, and context files — bypassed it entirely. This closes
those gaps:
- file_operations: deny-list blocks writes to ~/.ssh, ~/.aws, ~/.hermes/.env, etc.
- code_execution_tool: filter secret env vars from sandbox child process
- skill_manager_tool: wire scan_skill() into create/edit/patch/write_file with rollback
- skills_guard: add "agent-created" trust level (same policy as community)
- memory_tool: scan content for injection/exfil before system prompt injection
- prompt_builder: scan AGENTS.md, .cursorrules, SOUL.md for prompt injection
- cronjob_tools: scan cron prompts for critical threats before scheduling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 23:43:15 -05:00
|
|
|
import logging
|
2026-02-19 00:57:31 -08:00
|
|
|
import os
|
2026-02-20 02:32:15 -08:00
|
|
|
import tempfile
|
fix(memory): guard against external drift in MEMORY.md/USER.md (#26045) (#30877)
Reproduction (production, 2026-05-14): two concurrent sessions on the
same agent. Session A patches MEMORY.md directly via the patch tool,
appending ~8KB of structured content (Vendor Master, Standing Orders,
Pin Board) — none of it through the memory tool, so no § delimiters.
Session B starts later with stale in-memory state (1 entry, ~331
chars). Session B calls memory(action=replace) on its one known
entry. The tool's _read_file parses A's content as a single 8KB
'entry' (no § splits), then replace truncates that entry to B's new
333-byte content. ~8KB of structured content silently destroyed.
The atomic-rename write path is fine in isolation. The bug is the
implicit contract: the tool assumes MEMORY.md is exclusively a
§-delimited list of small entries it wrote, but the v0.13 install
runbook itself uses 'cat >> MEMORY.md' for onboarding, the patch tool
edits the file directly, and operators do too.
Fix: a drift guard in MemoryStore._detect_external_drift that fires
on either signal:
1. Re-parse + re-serialize doesn't produce identical bytes
(catches oddly-encoded delimiters / partial writes).
2. Any single parsed entry exceeds the store's whole-file char
limit. The tool budgets the ENTIRE store against that limit
(2200 chars for memory, 1375 for user), so no tool-written
entry can legitimately be larger. An entry bigger than the
store limit means an external writer dropped free-form content
into what the tool will treat as one entry.
When drift fires, _reload_target writes a .bak.<ts> snapshot of the
on-disk file, then add/replace/remove refuse to flush. The original
file stays untouched. The error dict surfaces the .bak path AND a
remediation string ('integrate missing entries via memory(add=...)
one at a time, then rewrite the file clean') so the model can act on
it without escalating to the operator.
Tests:
- test_replace_refuses_on_drift, test_add_refuses_on_drift,
test_remove_refuses_on_drift — all three mutators refuse
- test_clean_file_does_not_trigger_drift — false-positive check
- test_error_message_points_at_remediation — error string shape
- test_drift_guard_also_protects_user_target — USER.md too
- test_drift_backup_filename_is_unique_per_invocation — bak.<ts>
naming pin
144 memory tests passing (was 137; +7).
Fixes #26045
2026-05-23 02:51:29 -07:00
|
|
|
import time
|
2026-03-17 04:19:11 -07:00
|
|
|
from contextlib import contextmanager
|
2026-02-19 00:57:31 -08:00
|
|
|
from pathlib import Path
|
refactor: consolidate get_hermes_home() and parse_reasoning_effort() (#3062)
Centralizes two widely-duplicated patterns into hermes_constants.py:
1. get_hermes_home() — Path resolution for ~/.hermes (HERMES_HOME env var)
- Was copy-pasted inline across 30+ files as:
Path(os.getenv("HERMES_HOME", Path.home() / ".hermes"))
- Now defined once in hermes_constants.py (zero-dependency module)
- hermes_cli/config.py re-exports it for backward compatibility
- Removed local wrapper functions in honcho_integration/client.py,
tools/website_policy.py, tools/tirith_security.py, hermes_cli/uninstall.py
2. parse_reasoning_effort() — Reasoning effort string validation
- Was copy-pasted in cli.py, gateway/run.py, cron/scheduler.py
- Same validation logic: check against (xhigh, high, medium, low, minimal, none)
- Now defined once in hermes_constants.py, called from all 3 locations
- Warning log for unknown values kept at call sites (context-specific)
31 files changed, net +31 lines (125 insertions, 94 deletions)
Full test suite: 6179 passed, 0 failed
2026-03-25 15:54:28 -07:00
|
|
|
from hermes_constants import get_hermes_home
|
2026-02-19 00:57:31 -08:00
|
|
|
from typing import Dict, Any, List, Optional
|
|
|
|
|
|
refactor: consolidate symlink-safe atomic replace into shared helper
Extract the islink/realpath guard from the 16743 fix into a single
atomic_replace() helper in utils.py, then migrate every os.replace()
call site in the codebase to use it.
The original PR #16777 correctly identified and fixed the bug, but
only patched 9 of ~24 call sites. The same bug class (managed
deployments that symlink state files silently losing the link on
every write) still existed at auth.json, sessions file, gateway
config, env_loader, webhook subscriptions, debug store, model
catalog, pairing, google OAuth, nous rate guard, and more.
Rather than add another 10+ copies of the same three-line guard,
consolidate into atomic_replace(tmp, target) which:
- resolves symlinks via os.path.realpath before os.replace
- returns the resolved real path so callers can re-apply permissions
- is a drop-in replacement for os.replace at the use sites
Changes:
- utils.py: new atomic_replace() helper + atomic_json_write /
atomic_yaml_write now call it instead of inlining the guard
- 16 files: all os.replace() call sites migrated to atomic_replace()
- agent/{google_oauth, nous_rate_guard, shell_hooks}.py
- cron/jobs.py
- gateway/{pairing, session, platforms/telegram}.py
- hermes_cli/{auth, config, debug, env_loader, model_catalog, webhook}.py
- tools/{memory_tool, skill_manager_tool, skills_sync}.py
Tests: tests/test_atomic_replace_symlinks.py pins the invariant for
atomic_replace + atomic_json_write + atomic_yaml_write, covers plain
files, first-time creates, broken symlinks, and permission preservation.
Refs #16743
Builds on #16777 by @vominh1919.
2026-04-28 04:51:38 -07:00
|
|
|
from utils import atomic_replace
|
|
|
|
|
|
2026-04-14 10:17:37 -07:00
|
|
|
# fcntl is Unix-only; on Windows use msvcrt for file locking
|
|
|
|
|
msvcrt = None
|
2026-04-14 14:35:06 +03:00
|
|
|
try:
|
|
|
|
|
import fcntl
|
|
|
|
|
except ImportError:
|
|
|
|
|
fcntl = None
|
2026-04-14 10:17:37 -07:00
|
|
|
try:
|
|
|
|
|
import msvcrt
|
|
|
|
|
except ImportError:
|
|
|
|
|
pass
|
2026-04-14 14:35:06 +03:00
|
|
|
|
Harden agent attack surface: scan writes to memory, skills, cron, and context files
The security scanner (skills_guard.py) was only wired into the hub install path.
All other write paths to persistent state — skills created by the agent, memory
entries, cron prompts, and context files — bypassed it entirely. This closes
those gaps:
- file_operations: deny-list blocks writes to ~/.ssh, ~/.aws, ~/.hermes/.env, etc.
- code_execution_tool: filter secret env vars from sandbox child process
- skill_manager_tool: wire scan_skill() into create/edit/patch/write_file with rollback
- skills_guard: add "agent-created" trust level (same policy as community)
- memory_tool: scan content for injection/exfil before system prompt injection
- prompt_builder: scan AGENTS.md, .cursorrules, SOUL.md for prompt injection
- cronjob_tools: scan cron prompts for critical threats before scheduling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 23:43:15 -05:00
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
2026-04-03 13:10:11 -07:00
|
|
|
# Where memory files live — resolved dynamically so profile overrides
|
|
|
|
|
# (HERMES_HOME env var changes) are always respected. The old module-level
|
|
|
|
|
# constant was cached at import time and could go stale if a profile switch
|
|
|
|
|
# happened after the first import.
|
|
|
|
|
def get_memory_dir() -> Path:
|
|
|
|
|
"""Return the profile-scoped memories directory."""
|
|
|
|
|
return get_hermes_home() / "memories"
|
|
|
|
|
|
2026-02-19 00:57:31 -08:00
|
|
|
ENTRY_DELIMITER = "\n§\n"
|
|
|
|
|
|
|
|
|
|
|
Harden agent attack surface: scan writes to memory, skills, cron, and context files
The security scanner (skills_guard.py) was only wired into the hub install path.
All other write paths to persistent state — skills created by the agent, memory
entries, cron prompts, and context files — bypassed it entirely. This closes
those gaps:
- file_operations: deny-list blocks writes to ~/.ssh, ~/.aws, ~/.hermes/.env, etc.
- code_execution_tool: filter secret env vars from sandbox child process
- skill_manager_tool: wire scan_skill() into create/edit/patch/write_file with rollback
- skills_guard: add "agent-created" trust level (same policy as community)
- memory_tool: scan content for injection/exfil before system prompt injection
- prompt_builder: scan AGENTS.md, .cursorrules, SOUL.md for prompt injection
- cronjob_tools: scan cron prompts for critical threats before scheduling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 23:43:15 -05:00
|
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
# Memory content scanning — lightweight check for injection/exfiltration
|
|
|
|
|
# in content that gets injected into the system prompt.
|
feat(security): promptware defense — shared threat patterns + memory load-time scan + tool-result delimiters (#32269)
Hardens the context window against Brainworm-class promptware attacks
(see #496). Three changes:
1. tools/threat_patterns.py — single source of truth for injection/promptware
patterns. Replaces the duplicated pattern lists in prompt_builder.py and
memory_tool.py. Adds ~15 new Brainworm/C2 patterns (node registration,
heartbeat/beacon, pull tasking, anti-forensic disk avoidance, identity
override, known framework names). Three scopes — 'all' (narrow, classic
injection), 'context' (adds promptware/role-play, broader detection),
'strict' (adds persistence/SSH-backdoor patterns for user-mediated writes).
2. MemoryStore.load_from_disk() now scans entries at snapshot-build time.
Poisoned entries are replaced with [BLOCKED: ...] placeholders in the
frozen system-prompt snapshot. Live state keeps the original so the
user can still inspect + remove via memory(action=read/remove). Scan is
deterministic from disk bytes — prefix-cache invariant holds.
3. make_tool_result_message() wraps results from high-risk tools
(web_extract, web_search, browser_*, mcp_*) in
<untrusted_tool_result source="...">...</untrusted_tool_result>
delimiters with framing prose telling the model the content is data,
not instructions. Architectural defense against indirect injection
from poisoned web pages, GitHub issues, MCP responses — does NOT
regex-scan tool results (pattern arms race + per-iteration latency).
Multimodal content lists pass through unwrapped to preserve adapter
compatibility.
Pattern philosophy: anchor on C2-specific vocabulary or unambiguous attack
behavior, NOT on bossy English. Dropped patterns suggested in #496 that
would have tripped legitimate content: standalone 'you are obligated to',
'do not respond immediately', 'you must X' without a C2-verb anchor.
Validation:
- 257/257 targeted tests pass (test_threat_patterns + test_memory_tool +
test_tool_dispatch_helpers + test_prompt_builder)
- E2E run with real Brainworm payload: blocked from AGENTS.md context-file
path, blocked from MEMORY.md snapshot, wrapped in delimiters when
arriving via web_extract. Legitimate 'you must follow conventions'
phrasing not flagged.
Explicitly NOT in this PR (per #496 discussion):
- Per-tool-result regex scanning (pattern arms race)
- SessionBehaviorMonitor / polling-loop detection (wrong layer)
- Outbound network gating (Docker backend already covers this)
- security.context_scanning warn|block knob (current behavior is always
block-with-placeholder — there's no warn mode that makes sense)
Closes #496 for Phase 1 + the architectural delimiter piece of Phase 2.
Phase 3 stays in tracking issue territory.
2026-05-25 14:52:24 -07:00
|
|
|
#
|
|
|
|
|
# Patterns live in ``tools/threat_patterns.py`` — the single source of truth
|
|
|
|
|
# shared with the context-file scanner and the tool-result delimiter system.
|
|
|
|
|
# Memory uses the "strict" scope (broadest pattern set) because:
|
|
|
|
|
# - memory entries are user-curated; the user can rewrite a flagged entry
|
|
|
|
|
# - memory enters the system prompt as a FROZEN snapshot, so a poisoned
|
|
|
|
|
# entry persists for the entire session and across sessions until
|
|
|
|
|
# explicitly removed.
|
Harden agent attack surface: scan writes to memory, skills, cron, and context files
The security scanner (skills_guard.py) was only wired into the hub install path.
All other write paths to persistent state — skills created by the agent, memory
entries, cron prompts, and context files — bypassed it entirely. This closes
those gaps:
- file_operations: deny-list blocks writes to ~/.ssh, ~/.aws, ~/.hermes/.env, etc.
- code_execution_tool: filter secret env vars from sandbox child process
- skill_manager_tool: wire scan_skill() into create/edit/patch/write_file with rollback
- skills_guard: add "agent-created" trust level (same policy as community)
- memory_tool: scan content for injection/exfil before system prompt injection
- prompt_builder: scan AGENTS.md, .cursorrules, SOUL.md for prompt injection
- cronjob_tools: scan cron prompts for critical threats before scheduling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 23:43:15 -05:00
|
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
feat(security): promptware defense — shared threat patterns + memory load-time scan + tool-result delimiters (#32269)
Hardens the context window against Brainworm-class promptware attacks
(see #496). Three changes:
1. tools/threat_patterns.py — single source of truth for injection/promptware
patterns. Replaces the duplicated pattern lists in prompt_builder.py and
memory_tool.py. Adds ~15 new Brainworm/C2 patterns (node registration,
heartbeat/beacon, pull tasking, anti-forensic disk avoidance, identity
override, known framework names). Three scopes — 'all' (narrow, classic
injection), 'context' (adds promptware/role-play, broader detection),
'strict' (adds persistence/SSH-backdoor patterns for user-mediated writes).
2. MemoryStore.load_from_disk() now scans entries at snapshot-build time.
Poisoned entries are replaced with [BLOCKED: ...] placeholders in the
frozen system-prompt snapshot. Live state keeps the original so the
user can still inspect + remove via memory(action=read/remove). Scan is
deterministic from disk bytes — prefix-cache invariant holds.
3. make_tool_result_message() wraps results from high-risk tools
(web_extract, web_search, browser_*, mcp_*) in
<untrusted_tool_result source="...">...</untrusted_tool_result>
delimiters with framing prose telling the model the content is data,
not instructions. Architectural defense against indirect injection
from poisoned web pages, GitHub issues, MCP responses — does NOT
regex-scan tool results (pattern arms race + per-iteration latency).
Multimodal content lists pass through unwrapped to preserve adapter
compatibility.
Pattern philosophy: anchor on C2-specific vocabulary or unambiguous attack
behavior, NOT on bossy English. Dropped patterns suggested in #496 that
would have tripped legitimate content: standalone 'you are obligated to',
'do not respond immediately', 'you must X' without a C2-verb anchor.
Validation:
- 257/257 targeted tests pass (test_threat_patterns + test_memory_tool +
test_tool_dispatch_helpers + test_prompt_builder)
- E2E run with real Brainworm payload: blocked from AGENTS.md context-file
path, blocked from MEMORY.md snapshot, wrapped in delimiters when
arriving via web_extract. Legitimate 'you must follow conventions'
phrasing not flagged.
Explicitly NOT in this PR (per #496 discussion):
- Per-tool-result regex scanning (pattern arms race)
- SessionBehaviorMonitor / polling-loop detection (wrong layer)
- Outbound network gating (Docker backend already covers this)
- security.context_scanning warn|block knob (current behavior is always
block-with-placeholder — there's no warn mode that makes sense)
Closes #496 for Phase 1 + the architectural delimiter piece of Phase 2.
Phase 3 stays in tracking issue territory.
2026-05-25 14:52:24 -07:00
|
|
|
from tools.threat_patterns import first_threat_message as _first_threat_message
|
Harden agent attack surface: scan writes to memory, skills, cron, and context files
The security scanner (skills_guard.py) was only wired into the hub install path.
All other write paths to persistent state — skills created by the agent, memory
entries, cron prompts, and context files — bypassed it entirely. This closes
those gaps:
- file_operations: deny-list blocks writes to ~/.ssh, ~/.aws, ~/.hermes/.env, etc.
- code_execution_tool: filter secret env vars from sandbox child process
- skill_manager_tool: wire scan_skill() into create/edit/patch/write_file with rollback
- skills_guard: add "agent-created" trust level (same policy as community)
- memory_tool: scan content for injection/exfil before system prompt injection
- prompt_builder: scan AGENTS.md, .cursorrules, SOUL.md for prompt injection
- cronjob_tools: scan cron prompts for critical threats before scheduling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 23:43:15 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
def _scan_memory_content(content: str) -> Optional[str]:
|
|
|
|
|
"""Scan memory content for injection/exfil patterns. Returns error string if blocked."""
|
feat(security): promptware defense — shared threat patterns + memory load-time scan + tool-result delimiters (#32269)
Hardens the context window against Brainworm-class promptware attacks
(see #496). Three changes:
1. tools/threat_patterns.py — single source of truth for injection/promptware
patterns. Replaces the duplicated pattern lists in prompt_builder.py and
memory_tool.py. Adds ~15 new Brainworm/C2 patterns (node registration,
heartbeat/beacon, pull tasking, anti-forensic disk avoidance, identity
override, known framework names). Three scopes — 'all' (narrow, classic
injection), 'context' (adds promptware/role-play, broader detection),
'strict' (adds persistence/SSH-backdoor patterns for user-mediated writes).
2. MemoryStore.load_from_disk() now scans entries at snapshot-build time.
Poisoned entries are replaced with [BLOCKED: ...] placeholders in the
frozen system-prompt snapshot. Live state keeps the original so the
user can still inspect + remove via memory(action=read/remove). Scan is
deterministic from disk bytes — prefix-cache invariant holds.
3. make_tool_result_message() wraps results from high-risk tools
(web_extract, web_search, browser_*, mcp_*) in
<untrusted_tool_result source="...">...</untrusted_tool_result>
delimiters with framing prose telling the model the content is data,
not instructions. Architectural defense against indirect injection
from poisoned web pages, GitHub issues, MCP responses — does NOT
regex-scan tool results (pattern arms race + per-iteration latency).
Multimodal content lists pass through unwrapped to preserve adapter
compatibility.
Pattern philosophy: anchor on C2-specific vocabulary or unambiguous attack
behavior, NOT on bossy English. Dropped patterns suggested in #496 that
would have tripped legitimate content: standalone 'you are obligated to',
'do not respond immediately', 'you must X' without a C2-verb anchor.
Validation:
- 257/257 targeted tests pass (test_threat_patterns + test_memory_tool +
test_tool_dispatch_helpers + test_prompt_builder)
- E2E run with real Brainworm payload: blocked from AGENTS.md context-file
path, blocked from MEMORY.md snapshot, wrapped in delimiters when
arriving via web_extract. Legitimate 'you must follow conventions'
phrasing not flagged.
Explicitly NOT in this PR (per #496 discussion):
- Per-tool-result regex scanning (pattern arms race)
- SessionBehaviorMonitor / polling-loop detection (wrong layer)
- Outbound network gating (Docker backend already covers this)
- security.context_scanning warn|block knob (current behavior is always
block-with-placeholder — there's no warn mode that makes sense)
Closes #496 for Phase 1 + the architectural delimiter piece of Phase 2.
Phase 3 stays in tracking issue territory.
2026-05-25 14:52:24 -07:00
|
|
|
return _first_threat_message(content, scope="strict")
|
Harden agent attack surface: scan writes to memory, skills, cron, and context files
The security scanner (skills_guard.py) was only wired into the hub install path.
All other write paths to persistent state — skills created by the agent, memory
entries, cron prompts, and context files — bypassed it entirely. This closes
those gaps:
- file_operations: deny-list blocks writes to ~/.ssh, ~/.aws, ~/.hermes/.env, etc.
- code_execution_tool: filter secret env vars from sandbox child process
- skill_manager_tool: wire scan_skill() into create/edit/patch/write_file with rollback
- skills_guard: add "agent-created" trust level (same policy as community)
- memory_tool: scan content for injection/exfil before system prompt injection
- prompt_builder: scan AGENTS.md, .cursorrules, SOUL.md for prompt injection
- cronjob_tools: scan cron prompts for critical threats before scheduling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 23:43:15 -05:00
|
|
|
|
|
|
|
|
|
fix(memory): guard against external drift in MEMORY.md/USER.md (#26045) (#30877)
Reproduction (production, 2026-05-14): two concurrent sessions on the
same agent. Session A patches MEMORY.md directly via the patch tool,
appending ~8KB of structured content (Vendor Master, Standing Orders,
Pin Board) — none of it through the memory tool, so no § delimiters.
Session B starts later with stale in-memory state (1 entry, ~331
chars). Session B calls memory(action=replace) on its one known
entry. The tool's _read_file parses A's content as a single 8KB
'entry' (no § splits), then replace truncates that entry to B's new
333-byte content. ~8KB of structured content silently destroyed.
The atomic-rename write path is fine in isolation. The bug is the
implicit contract: the tool assumes MEMORY.md is exclusively a
§-delimited list of small entries it wrote, but the v0.13 install
runbook itself uses 'cat >> MEMORY.md' for onboarding, the patch tool
edits the file directly, and operators do too.
Fix: a drift guard in MemoryStore._detect_external_drift that fires
on either signal:
1. Re-parse + re-serialize doesn't produce identical bytes
(catches oddly-encoded delimiters / partial writes).
2. Any single parsed entry exceeds the store's whole-file char
limit. The tool budgets the ENTIRE store against that limit
(2200 chars for memory, 1375 for user), so no tool-written
entry can legitimately be larger. An entry bigger than the
store limit means an external writer dropped free-form content
into what the tool will treat as one entry.
When drift fires, _reload_target writes a .bak.<ts> snapshot of the
on-disk file, then add/replace/remove refuse to flush. The original
file stays untouched. The error dict surfaces the .bak path AND a
remediation string ('integrate missing entries via memory(add=...)
one at a time, then rewrite the file clean') so the model can act on
it without escalating to the operator.
Tests:
- test_replace_refuses_on_drift, test_add_refuses_on_drift,
test_remove_refuses_on_drift — all three mutators refuse
- test_clean_file_does_not_trigger_drift — false-positive check
- test_error_message_points_at_remediation — error string shape
- test_drift_guard_also_protects_user_target — USER.md too
- test_drift_backup_filename_is_unique_per_invocation — bak.<ts>
naming pin
144 memory tests passing (was 137; +7).
Fixes #26045
2026-05-23 02:51:29 -07:00
|
|
|
def _drift_error(path: "Path", bak_path: str) -> Dict[str, Any]:
|
|
|
|
|
"""Build the error dict returned when external drift is detected.
|
|
|
|
|
|
|
|
|
|
The on-disk memory file contains content that wouldn't round-trip
|
|
|
|
|
through the tool's parser/serializer — flushing would discard the
|
|
|
|
|
appended/edited content from a patch tool, shell append, manual edit,
|
|
|
|
|
or sister-session write. We refuse the mutation, point the operator at
|
|
|
|
|
the .bak.<ts> snapshot we took, and tell them what to do next.
|
|
|
|
|
"""
|
|
|
|
|
return {
|
|
|
|
|
"success": False,
|
|
|
|
|
"error": (
|
|
|
|
|
f"Refusing to write {path.name}: file on disk has content that "
|
|
|
|
|
f"wouldn't round-trip through the memory tool (likely added by "
|
|
|
|
|
f"the patch tool, a shell append, a manual edit, or a "
|
|
|
|
|
f"concurrent session). A snapshot was saved to {bak_path}. "
|
|
|
|
|
f"Resolve the drift first — either rewrite the file as a clean "
|
|
|
|
|
f"§-delimited list of entries, or move the extra content out — "
|
|
|
|
|
f"then retry. This guard exists to prevent silent data loss "
|
|
|
|
|
f"(issue #26045)."
|
|
|
|
|
),
|
|
|
|
|
"drift_backup": bak_path,
|
|
|
|
|
"remediation": (
|
|
|
|
|
"Open the .bak file, integrate the missing entries into the "
|
|
|
|
|
"memory tool one at a time via memory(action=add, content=...), "
|
|
|
|
|
"then remove or rewrite the original file to a clean state."
|
|
|
|
|
),
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
2026-02-19 00:57:31 -08:00
|
|
|
class MemoryStore:
|
|
|
|
|
"""
|
|
|
|
|
Bounded curated memory with file persistence. One instance per AIAgent.
|
|
|
|
|
|
|
|
|
|
Maintains two parallel states:
|
|
|
|
|
- _system_prompt_snapshot: frozen at load time, used for system prompt injection.
|
|
|
|
|
Never mutated mid-session. Keeps prefix cache stable.
|
|
|
|
|
- memory_entries / user_entries: live state, mutated by tool calls, persisted to disk.
|
|
|
|
|
Tool responses always reflect this live state.
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
def __init__(self, memory_char_limit: int = 2200, user_char_limit: int = 1375):
|
|
|
|
|
self.memory_entries: List[str] = []
|
|
|
|
|
self.user_entries: List[str] = []
|
|
|
|
|
self.memory_char_limit = memory_char_limit
|
|
|
|
|
self.user_char_limit = user_char_limit
|
|
|
|
|
# Frozen snapshot for system prompt -- set once at load_from_disk()
|
|
|
|
|
self._system_prompt_snapshot: Dict[str, str] = {"memory": "", "user": ""}
|
|
|
|
|
|
|
|
|
|
def load_from_disk(self):
|
feat(security): promptware defense — shared threat patterns + memory load-time scan + tool-result delimiters (#32269)
Hardens the context window against Brainworm-class promptware attacks
(see #496). Three changes:
1. tools/threat_patterns.py — single source of truth for injection/promptware
patterns. Replaces the duplicated pattern lists in prompt_builder.py and
memory_tool.py. Adds ~15 new Brainworm/C2 patterns (node registration,
heartbeat/beacon, pull tasking, anti-forensic disk avoidance, identity
override, known framework names). Three scopes — 'all' (narrow, classic
injection), 'context' (adds promptware/role-play, broader detection),
'strict' (adds persistence/SSH-backdoor patterns for user-mediated writes).
2. MemoryStore.load_from_disk() now scans entries at snapshot-build time.
Poisoned entries are replaced with [BLOCKED: ...] placeholders in the
frozen system-prompt snapshot. Live state keeps the original so the
user can still inspect + remove via memory(action=read/remove). Scan is
deterministic from disk bytes — prefix-cache invariant holds.
3. make_tool_result_message() wraps results from high-risk tools
(web_extract, web_search, browser_*, mcp_*) in
<untrusted_tool_result source="...">...</untrusted_tool_result>
delimiters with framing prose telling the model the content is data,
not instructions. Architectural defense against indirect injection
from poisoned web pages, GitHub issues, MCP responses — does NOT
regex-scan tool results (pattern arms race + per-iteration latency).
Multimodal content lists pass through unwrapped to preserve adapter
compatibility.
Pattern philosophy: anchor on C2-specific vocabulary or unambiguous attack
behavior, NOT on bossy English. Dropped patterns suggested in #496 that
would have tripped legitimate content: standalone 'you are obligated to',
'do not respond immediately', 'you must X' without a C2-verb anchor.
Validation:
- 257/257 targeted tests pass (test_threat_patterns + test_memory_tool +
test_tool_dispatch_helpers + test_prompt_builder)
- E2E run with real Brainworm payload: blocked from AGENTS.md context-file
path, blocked from MEMORY.md snapshot, wrapped in delimiters when
arriving via web_extract. Legitimate 'you must follow conventions'
phrasing not flagged.
Explicitly NOT in this PR (per #496 discussion):
- Per-tool-result regex scanning (pattern arms race)
- SessionBehaviorMonitor / polling-loop detection (wrong layer)
- Outbound network gating (Docker backend already covers this)
- security.context_scanning warn|block knob (current behavior is always
block-with-placeholder — there's no warn mode that makes sense)
Closes #496 for Phase 1 + the architectural delimiter piece of Phase 2.
Phase 3 stays in tracking issue territory.
2026-05-25 14:52:24 -07:00
|
|
|
"""Load entries from MEMORY.md and USER.md, capture system prompt snapshot.
|
|
|
|
|
|
|
|
|
|
The frozen snapshot is what enters the system prompt. We scan each
|
|
|
|
|
entry for injection/promptware patterns at snapshot-build time —
|
|
|
|
|
ANY hit replaces the entry text in the snapshot with a placeholder
|
|
|
|
|
like ``[BLOCKED: …]``, so a poisoned-on-disk memory file (supply
|
|
|
|
|
chain, compromised tool, sister-session write) cannot inject into
|
|
|
|
|
the system prompt.
|
|
|
|
|
|
|
|
|
|
The live ``memory_entries`` / ``user_entries`` lists keep the
|
|
|
|
|
original text so the user can still SEE poisoned entries via
|
|
|
|
|
``memory(action=read)`` and remove them — silently dropping them
|
|
|
|
|
would hide the attack from the user.
|
|
|
|
|
|
|
|
|
|
Scanning is deterministic from disk bytes, so the snapshot remains
|
|
|
|
|
stable for the entire session (prefix-cache invariant holds).
|
|
|
|
|
"""
|
2026-04-03 13:10:11 -07:00
|
|
|
mem_dir = get_memory_dir()
|
|
|
|
|
mem_dir.mkdir(parents=True, exist_ok=True)
|
2026-02-19 00:57:31 -08:00
|
|
|
|
2026-04-03 13:10:11 -07:00
|
|
|
self.memory_entries = self._read_file(mem_dir / "MEMORY.md")
|
|
|
|
|
self.user_entries = self._read_file(mem_dir / "USER.md")
|
2026-02-19 00:57:31 -08:00
|
|
|
|
2026-02-20 02:32:15 -08:00
|
|
|
# Deduplicate entries (preserves order, keeps first occurrence)
|
|
|
|
|
self.memory_entries = list(dict.fromkeys(self.memory_entries))
|
|
|
|
|
self.user_entries = list(dict.fromkeys(self.user_entries))
|
|
|
|
|
|
feat(security): promptware defense — shared threat patterns + memory load-time scan + tool-result delimiters (#32269)
Hardens the context window against Brainworm-class promptware attacks
(see #496). Three changes:
1. tools/threat_patterns.py — single source of truth for injection/promptware
patterns. Replaces the duplicated pattern lists in prompt_builder.py and
memory_tool.py. Adds ~15 new Brainworm/C2 patterns (node registration,
heartbeat/beacon, pull tasking, anti-forensic disk avoidance, identity
override, known framework names). Three scopes — 'all' (narrow, classic
injection), 'context' (adds promptware/role-play, broader detection),
'strict' (adds persistence/SSH-backdoor patterns for user-mediated writes).
2. MemoryStore.load_from_disk() now scans entries at snapshot-build time.
Poisoned entries are replaced with [BLOCKED: ...] placeholders in the
frozen system-prompt snapshot. Live state keeps the original so the
user can still inspect + remove via memory(action=read/remove). Scan is
deterministic from disk bytes — prefix-cache invariant holds.
3. make_tool_result_message() wraps results from high-risk tools
(web_extract, web_search, browser_*, mcp_*) in
<untrusted_tool_result source="...">...</untrusted_tool_result>
delimiters with framing prose telling the model the content is data,
not instructions. Architectural defense against indirect injection
from poisoned web pages, GitHub issues, MCP responses — does NOT
regex-scan tool results (pattern arms race + per-iteration latency).
Multimodal content lists pass through unwrapped to preserve adapter
compatibility.
Pattern philosophy: anchor on C2-specific vocabulary or unambiguous attack
behavior, NOT on bossy English. Dropped patterns suggested in #496 that
would have tripped legitimate content: standalone 'you are obligated to',
'do not respond immediately', 'you must X' without a C2-verb anchor.
Validation:
- 257/257 targeted tests pass (test_threat_patterns + test_memory_tool +
test_tool_dispatch_helpers + test_prompt_builder)
- E2E run with real Brainworm payload: blocked from AGENTS.md context-file
path, blocked from MEMORY.md snapshot, wrapped in delimiters when
arriving via web_extract. Legitimate 'you must follow conventions'
phrasing not flagged.
Explicitly NOT in this PR (per #496 discussion):
- Per-tool-result regex scanning (pattern arms race)
- SessionBehaviorMonitor / polling-loop detection (wrong layer)
- Outbound network gating (Docker backend already covers this)
- security.context_scanning warn|block knob (current behavior is always
block-with-placeholder — there's no warn mode that makes sense)
Closes #496 for Phase 1 + the architectural delimiter piece of Phase 2.
Phase 3 stays in tracking issue territory.
2026-05-25 14:52:24 -07:00
|
|
|
# Sanitize entries for the system-prompt snapshot only. Live state
|
|
|
|
|
# (memory_entries / user_entries) keeps the raw text so the user
|
|
|
|
|
# can see + remove poisoned entries via the memory tool.
|
|
|
|
|
sanitized_memory = self._sanitize_entries_for_snapshot(self.memory_entries, "MEMORY.md")
|
|
|
|
|
sanitized_user = self._sanitize_entries_for_snapshot(self.user_entries, "USER.md")
|
|
|
|
|
|
2026-02-19 00:57:31 -08:00
|
|
|
# Capture frozen snapshot for system prompt injection
|
|
|
|
|
self._system_prompt_snapshot = {
|
feat(security): promptware defense — shared threat patterns + memory load-time scan + tool-result delimiters (#32269)
Hardens the context window against Brainworm-class promptware attacks
(see #496). Three changes:
1. tools/threat_patterns.py — single source of truth for injection/promptware
patterns. Replaces the duplicated pattern lists in prompt_builder.py and
memory_tool.py. Adds ~15 new Brainworm/C2 patterns (node registration,
heartbeat/beacon, pull tasking, anti-forensic disk avoidance, identity
override, known framework names). Three scopes — 'all' (narrow, classic
injection), 'context' (adds promptware/role-play, broader detection),
'strict' (adds persistence/SSH-backdoor patterns for user-mediated writes).
2. MemoryStore.load_from_disk() now scans entries at snapshot-build time.
Poisoned entries are replaced with [BLOCKED: ...] placeholders in the
frozen system-prompt snapshot. Live state keeps the original so the
user can still inspect + remove via memory(action=read/remove). Scan is
deterministic from disk bytes — prefix-cache invariant holds.
3. make_tool_result_message() wraps results from high-risk tools
(web_extract, web_search, browser_*, mcp_*) in
<untrusted_tool_result source="...">...</untrusted_tool_result>
delimiters with framing prose telling the model the content is data,
not instructions. Architectural defense against indirect injection
from poisoned web pages, GitHub issues, MCP responses — does NOT
regex-scan tool results (pattern arms race + per-iteration latency).
Multimodal content lists pass through unwrapped to preserve adapter
compatibility.
Pattern philosophy: anchor on C2-specific vocabulary or unambiguous attack
behavior, NOT on bossy English. Dropped patterns suggested in #496 that
would have tripped legitimate content: standalone 'you are obligated to',
'do not respond immediately', 'you must X' without a C2-verb anchor.
Validation:
- 257/257 targeted tests pass (test_threat_patterns + test_memory_tool +
test_tool_dispatch_helpers + test_prompt_builder)
- E2E run with real Brainworm payload: blocked from AGENTS.md context-file
path, blocked from MEMORY.md snapshot, wrapped in delimiters when
arriving via web_extract. Legitimate 'you must follow conventions'
phrasing not flagged.
Explicitly NOT in this PR (per #496 discussion):
- Per-tool-result regex scanning (pattern arms race)
- SessionBehaviorMonitor / polling-loop detection (wrong layer)
- Outbound network gating (Docker backend already covers this)
- security.context_scanning warn|block knob (current behavior is always
block-with-placeholder — there's no warn mode that makes sense)
Closes #496 for Phase 1 + the architectural delimiter piece of Phase 2.
Phase 3 stays in tracking issue territory.
2026-05-25 14:52:24 -07:00
|
|
|
"memory": self._render_block("memory", sanitized_memory),
|
|
|
|
|
"user": self._render_block("user", sanitized_user),
|
2026-02-19 00:57:31 -08:00
|
|
|
}
|
|
|
|
|
|
feat(security): promptware defense — shared threat patterns + memory load-time scan + tool-result delimiters (#32269)
Hardens the context window against Brainworm-class promptware attacks
(see #496). Three changes:
1. tools/threat_patterns.py — single source of truth for injection/promptware
patterns. Replaces the duplicated pattern lists in prompt_builder.py and
memory_tool.py. Adds ~15 new Brainworm/C2 patterns (node registration,
heartbeat/beacon, pull tasking, anti-forensic disk avoidance, identity
override, known framework names). Three scopes — 'all' (narrow, classic
injection), 'context' (adds promptware/role-play, broader detection),
'strict' (adds persistence/SSH-backdoor patterns for user-mediated writes).
2. MemoryStore.load_from_disk() now scans entries at snapshot-build time.
Poisoned entries are replaced with [BLOCKED: ...] placeholders in the
frozen system-prompt snapshot. Live state keeps the original so the
user can still inspect + remove via memory(action=read/remove). Scan is
deterministic from disk bytes — prefix-cache invariant holds.
3. make_tool_result_message() wraps results from high-risk tools
(web_extract, web_search, browser_*, mcp_*) in
<untrusted_tool_result source="...">...</untrusted_tool_result>
delimiters with framing prose telling the model the content is data,
not instructions. Architectural defense against indirect injection
from poisoned web pages, GitHub issues, MCP responses — does NOT
regex-scan tool results (pattern arms race + per-iteration latency).
Multimodal content lists pass through unwrapped to preserve adapter
compatibility.
Pattern philosophy: anchor on C2-specific vocabulary or unambiguous attack
behavior, NOT on bossy English. Dropped patterns suggested in #496 that
would have tripped legitimate content: standalone 'you are obligated to',
'do not respond immediately', 'you must X' without a C2-verb anchor.
Validation:
- 257/257 targeted tests pass (test_threat_patterns + test_memory_tool +
test_tool_dispatch_helpers + test_prompt_builder)
- E2E run with real Brainworm payload: blocked from AGENTS.md context-file
path, blocked from MEMORY.md snapshot, wrapped in delimiters when
arriving via web_extract. Legitimate 'you must follow conventions'
phrasing not flagged.
Explicitly NOT in this PR (per #496 discussion):
- Per-tool-result regex scanning (pattern arms race)
- SessionBehaviorMonitor / polling-loop detection (wrong layer)
- Outbound network gating (Docker backend already covers this)
- security.context_scanning warn|block knob (current behavior is always
block-with-placeholder — there's no warn mode that makes sense)
Closes #496 for Phase 1 + the architectural delimiter piece of Phase 2.
Phase 3 stays in tracking issue territory.
2026-05-25 14:52:24 -07:00
|
|
|
@staticmethod
|
|
|
|
|
def _sanitize_entries_for_snapshot(entries: List[str], filename: str) -> List[str]:
|
|
|
|
|
"""Return ``entries`` with any threat-matching entry replaced by a placeholder.
|
|
|
|
|
|
|
|
|
|
Each entry is scanned with the shared threat-pattern library at the
|
|
|
|
|
``"strict"`` scope (same as memory writes). On match, the entry is
|
|
|
|
|
replaced in the returned list with ``"[BLOCKED: <filename> entry
|
|
|
|
|
contained threat pattern: <ids>. Removed from system prompt.]"`` —
|
|
|
|
|
the placeholder enters the snapshot, the original entry stays in
|
|
|
|
|
live state for the user to inspect and delete.
|
|
|
|
|
|
|
|
|
|
Empty or already-block-marker entries pass through unchanged.
|
|
|
|
|
"""
|
|
|
|
|
from tools.threat_patterns import scan_for_threats
|
|
|
|
|
|
|
|
|
|
sanitized: List[str] = []
|
|
|
|
|
for entry in entries:
|
|
|
|
|
if not entry or entry.startswith("[BLOCKED:"):
|
|
|
|
|
sanitized.append(entry)
|
|
|
|
|
continue
|
|
|
|
|
findings = scan_for_threats(entry, scope="strict")
|
|
|
|
|
if findings:
|
|
|
|
|
logger.warning(
|
|
|
|
|
"Memory entry from %s blocked at load time: %s",
|
|
|
|
|
filename, ", ".join(findings),
|
|
|
|
|
)
|
|
|
|
|
sanitized.append(
|
|
|
|
|
f"[BLOCKED: {filename} entry contained threat pattern(s): "
|
|
|
|
|
f"{', '.join(findings)}. Removed from system prompt; "
|
|
|
|
|
f"use memory(action=read) to inspect and memory(action=remove) "
|
|
|
|
|
f"to delete the original.]"
|
|
|
|
|
)
|
|
|
|
|
else:
|
|
|
|
|
sanitized.append(entry)
|
|
|
|
|
return sanitized
|
|
|
|
|
|
2026-03-17 04:19:11 -07:00
|
|
|
@staticmethod
|
|
|
|
|
@contextmanager
|
|
|
|
|
def _file_lock(path: Path):
|
|
|
|
|
"""Acquire an exclusive file lock for read-modify-write safety.
|
|
|
|
|
|
|
|
|
|
Uses a separate .lock file so the memory file itself can still be
|
|
|
|
|
atomically replaced via os.replace().
|
|
|
|
|
"""
|
|
|
|
|
lock_path = path.with_suffix(path.suffix + ".lock")
|
|
|
|
|
lock_path.parent.mkdir(parents=True, exist_ok=True)
|
2026-04-14 14:35:06 +03:00
|
|
|
|
|
|
|
|
if fcntl is None and msvcrt is None:
|
|
|
|
|
yield
|
|
|
|
|
return
|
|
|
|
|
|
2026-05-15 18:28:45 +03:00
|
|
|
fd = open(lock_path, "a+", encoding="utf-8")
|
2026-03-17 04:19:11 -07:00
|
|
|
try:
|
2026-04-14 14:35:06 +03:00
|
|
|
if fcntl:
|
|
|
|
|
fcntl.flock(fd, fcntl.LOCK_EX)
|
|
|
|
|
else:
|
|
|
|
|
fd.seek(0)
|
|
|
|
|
msvcrt.locking(fd.fileno(), msvcrt.LK_LOCK, 1)
|
2026-03-17 04:19:11 -07:00
|
|
|
yield
|
|
|
|
|
finally:
|
2026-04-14 14:35:06 +03:00
|
|
|
if fcntl:
|
fix: guard yaml.safe_load, flock unlock, TOCTOU races, and atomic writes
1. trajectory_compressor.py: yaml.safe_load() returns None on empty
files, crashing with TypeError on `if 'tokenizer' in data`. Fix by
adding `or {}` fallback. (HIGH — blocks startup with empty config)
2. 6 files with fcntl.flock(LOCK_UN) in finally blocks without
try/except: cron/scheduler.py, hermes_cli/auth.py,
agent/shell_hooks.py, tools/skill_usage.py,
tools/environments/file_sync.py, tools/memory_tool.py. If unlock
raises OSError, fd.close() is skipped and the lock is held forever.
The msvcrt branches already had try/except; the fcntl branches did
not. Fix by wrapping in try/except (OSError, IOError): pass.
3. agent/copilot_acp_client.py line 639: TOCTOU race — path.exists()
followed by path.read_text() with no try/except. If file is deleted
between the check and the read, FileNotFoundError propagates. Fix
by using try/except FileNotFoundError.
4. gateway/sticker_cache.py: non-atomic write via Path.write_text()
can leave truncated JSON on crash, causing JSONDecodeError on next
load. Fix by writing to tempfile + fsync + os.replace (atomic).
2026-05-19 00:12:34 -07:00
|
|
|
try:
|
|
|
|
|
fcntl.flock(fd, fcntl.LOCK_UN)
|
|
|
|
|
except (OSError, IOError):
|
|
|
|
|
pass
|
2026-04-14 14:35:06 +03:00
|
|
|
elif msvcrt:
|
|
|
|
|
try:
|
|
|
|
|
fd.seek(0)
|
|
|
|
|
msvcrt.locking(fd.fileno(), msvcrt.LK_UNLCK, 1)
|
|
|
|
|
except (OSError, IOError):
|
|
|
|
|
pass
|
2026-03-17 04:19:11 -07:00
|
|
|
fd.close()
|
|
|
|
|
|
|
|
|
|
@staticmethod
|
|
|
|
|
def _path_for(target: str) -> Path:
|
2026-04-03 13:10:11 -07:00
|
|
|
mem_dir = get_memory_dir()
|
2026-03-17 04:19:11 -07:00
|
|
|
if target == "user":
|
2026-04-03 13:10:11 -07:00
|
|
|
return mem_dir / "USER.md"
|
|
|
|
|
return mem_dir / "MEMORY.md"
|
2026-03-17 04:19:11 -07:00
|
|
|
|
fix(memory): guard against external drift in MEMORY.md/USER.md (#26045) (#30877)
Reproduction (production, 2026-05-14): two concurrent sessions on the
same agent. Session A patches MEMORY.md directly via the patch tool,
appending ~8KB of structured content (Vendor Master, Standing Orders,
Pin Board) — none of it through the memory tool, so no § delimiters.
Session B starts later with stale in-memory state (1 entry, ~331
chars). Session B calls memory(action=replace) on its one known
entry. The tool's _read_file parses A's content as a single 8KB
'entry' (no § splits), then replace truncates that entry to B's new
333-byte content. ~8KB of structured content silently destroyed.
The atomic-rename write path is fine in isolation. The bug is the
implicit contract: the tool assumes MEMORY.md is exclusively a
§-delimited list of small entries it wrote, but the v0.13 install
runbook itself uses 'cat >> MEMORY.md' for onboarding, the patch tool
edits the file directly, and operators do too.
Fix: a drift guard in MemoryStore._detect_external_drift that fires
on either signal:
1. Re-parse + re-serialize doesn't produce identical bytes
(catches oddly-encoded delimiters / partial writes).
2. Any single parsed entry exceeds the store's whole-file char
limit. The tool budgets the ENTIRE store against that limit
(2200 chars for memory, 1375 for user), so no tool-written
entry can legitimately be larger. An entry bigger than the
store limit means an external writer dropped free-form content
into what the tool will treat as one entry.
When drift fires, _reload_target writes a .bak.<ts> snapshot of the
on-disk file, then add/replace/remove refuse to flush. The original
file stays untouched. The error dict surfaces the .bak path AND a
remediation string ('integrate missing entries via memory(add=...)
one at a time, then rewrite the file clean') so the model can act on
it without escalating to the operator.
Tests:
- test_replace_refuses_on_drift, test_add_refuses_on_drift,
test_remove_refuses_on_drift — all three mutators refuse
- test_clean_file_does_not_trigger_drift — false-positive check
- test_error_message_points_at_remediation — error string shape
- test_drift_guard_also_protects_user_target — USER.md too
- test_drift_backup_filename_is_unique_per_invocation — bak.<ts>
naming pin
144 memory tests passing (was 137; +7).
Fixes #26045
2026-05-23 02:51:29 -07:00
|
|
|
def _reload_target(self, target: str) -> Optional[str]:
|
2026-03-17 04:19:11 -07:00
|
|
|
"""Re-read entries from disk into in-memory state.
|
|
|
|
|
|
|
|
|
|
Called under file lock to get the latest state before mutating.
|
fix(memory): guard against external drift in MEMORY.md/USER.md (#26045) (#30877)
Reproduction (production, 2026-05-14): two concurrent sessions on the
same agent. Session A patches MEMORY.md directly via the patch tool,
appending ~8KB of structured content (Vendor Master, Standing Orders,
Pin Board) — none of it through the memory tool, so no § delimiters.
Session B starts later with stale in-memory state (1 entry, ~331
chars). Session B calls memory(action=replace) on its one known
entry. The tool's _read_file parses A's content as a single 8KB
'entry' (no § splits), then replace truncates that entry to B's new
333-byte content. ~8KB of structured content silently destroyed.
The atomic-rename write path is fine in isolation. The bug is the
implicit contract: the tool assumes MEMORY.md is exclusively a
§-delimited list of small entries it wrote, but the v0.13 install
runbook itself uses 'cat >> MEMORY.md' for onboarding, the patch tool
edits the file directly, and operators do too.
Fix: a drift guard in MemoryStore._detect_external_drift that fires
on either signal:
1. Re-parse + re-serialize doesn't produce identical bytes
(catches oddly-encoded delimiters / partial writes).
2. Any single parsed entry exceeds the store's whole-file char
limit. The tool budgets the ENTIRE store against that limit
(2200 chars for memory, 1375 for user), so no tool-written
entry can legitimately be larger. An entry bigger than the
store limit means an external writer dropped free-form content
into what the tool will treat as one entry.
When drift fires, _reload_target writes a .bak.<ts> snapshot of the
on-disk file, then add/replace/remove refuse to flush. The original
file stays untouched. The error dict surfaces the .bak path AND a
remediation string ('integrate missing entries via memory(add=...)
one at a time, then rewrite the file clean') so the model can act on
it without escalating to the operator.
Tests:
- test_replace_refuses_on_drift, test_add_refuses_on_drift,
test_remove_refuses_on_drift — all three mutators refuse
- test_clean_file_does_not_trigger_drift — false-positive check
- test_error_message_points_at_remediation — error string shape
- test_drift_guard_also_protects_user_target — USER.md too
- test_drift_backup_filename_is_unique_per_invocation — bak.<ts>
naming pin
144 memory tests passing (was 137; +7).
Fixes #26045
2026-05-23 02:51:29 -07:00
|
|
|
Returns the backup path if external drift was detected (the on-disk
|
|
|
|
|
file contains content that wouldn't round-trip through our
|
|
|
|
|
parser/serializer, OR an entry larger than the store's char limit).
|
|
|
|
|
When drift is detected the caller must abort the mutation —
|
|
|
|
|
flushing would discard the un-roundtrippable content.
|
|
|
|
|
Returns None on clean reload.
|
2026-03-17 04:19:11 -07:00
|
|
|
"""
|
fix(memory): guard against external drift in MEMORY.md/USER.md (#26045) (#30877)
Reproduction (production, 2026-05-14): two concurrent sessions on the
same agent. Session A patches MEMORY.md directly via the patch tool,
appending ~8KB of structured content (Vendor Master, Standing Orders,
Pin Board) — none of it through the memory tool, so no § delimiters.
Session B starts later with stale in-memory state (1 entry, ~331
chars). Session B calls memory(action=replace) on its one known
entry. The tool's _read_file parses A's content as a single 8KB
'entry' (no § splits), then replace truncates that entry to B's new
333-byte content. ~8KB of structured content silently destroyed.
The atomic-rename write path is fine in isolation. The bug is the
implicit contract: the tool assumes MEMORY.md is exclusively a
§-delimited list of small entries it wrote, but the v0.13 install
runbook itself uses 'cat >> MEMORY.md' for onboarding, the patch tool
edits the file directly, and operators do too.
Fix: a drift guard in MemoryStore._detect_external_drift that fires
on either signal:
1. Re-parse + re-serialize doesn't produce identical bytes
(catches oddly-encoded delimiters / partial writes).
2. Any single parsed entry exceeds the store's whole-file char
limit. The tool budgets the ENTIRE store against that limit
(2200 chars for memory, 1375 for user), so no tool-written
entry can legitimately be larger. An entry bigger than the
store limit means an external writer dropped free-form content
into what the tool will treat as one entry.
When drift fires, _reload_target writes a .bak.<ts> snapshot of the
on-disk file, then add/replace/remove refuse to flush. The original
file stays untouched. The error dict surfaces the .bak path AND a
remediation string ('integrate missing entries via memory(add=...)
one at a time, then rewrite the file clean') so the model can act on
it without escalating to the operator.
Tests:
- test_replace_refuses_on_drift, test_add_refuses_on_drift,
test_remove_refuses_on_drift — all three mutators refuse
- test_clean_file_does_not_trigger_drift — false-positive check
- test_error_message_points_at_remediation — error string shape
- test_drift_guard_also_protects_user_target — USER.md too
- test_drift_backup_filename_is_unique_per_invocation — bak.<ts>
naming pin
144 memory tests passing (was 137; +7).
Fixes #26045
2026-05-23 02:51:29 -07:00
|
|
|
path = self._path_for(target)
|
|
|
|
|
bak = self._detect_external_drift(target)
|
|
|
|
|
fresh = self._read_file(path)
|
2026-03-17 04:19:11 -07:00
|
|
|
fresh = list(dict.fromkeys(fresh)) # deduplicate
|
|
|
|
|
self._set_entries(target, fresh)
|
fix(memory): guard against external drift in MEMORY.md/USER.md (#26045) (#30877)
Reproduction (production, 2026-05-14): two concurrent sessions on the
same agent. Session A patches MEMORY.md directly via the patch tool,
appending ~8KB of structured content (Vendor Master, Standing Orders,
Pin Board) — none of it through the memory tool, so no § delimiters.
Session B starts later with stale in-memory state (1 entry, ~331
chars). Session B calls memory(action=replace) on its one known
entry. The tool's _read_file parses A's content as a single 8KB
'entry' (no § splits), then replace truncates that entry to B's new
333-byte content. ~8KB of structured content silently destroyed.
The atomic-rename write path is fine in isolation. The bug is the
implicit contract: the tool assumes MEMORY.md is exclusively a
§-delimited list of small entries it wrote, but the v0.13 install
runbook itself uses 'cat >> MEMORY.md' for onboarding, the patch tool
edits the file directly, and operators do too.
Fix: a drift guard in MemoryStore._detect_external_drift that fires
on either signal:
1. Re-parse + re-serialize doesn't produce identical bytes
(catches oddly-encoded delimiters / partial writes).
2. Any single parsed entry exceeds the store's whole-file char
limit. The tool budgets the ENTIRE store against that limit
(2200 chars for memory, 1375 for user), so no tool-written
entry can legitimately be larger. An entry bigger than the
store limit means an external writer dropped free-form content
into what the tool will treat as one entry.
When drift fires, _reload_target writes a .bak.<ts> snapshot of the
on-disk file, then add/replace/remove refuse to flush. The original
file stays untouched. The error dict surfaces the .bak path AND a
remediation string ('integrate missing entries via memory(add=...)
one at a time, then rewrite the file clean') so the model can act on
it without escalating to the operator.
Tests:
- test_replace_refuses_on_drift, test_add_refuses_on_drift,
test_remove_refuses_on_drift — all three mutators refuse
- test_clean_file_does_not_trigger_drift — false-positive check
- test_error_message_points_at_remediation — error string shape
- test_drift_guard_also_protects_user_target — USER.md too
- test_drift_backup_filename_is_unique_per_invocation — bak.<ts>
naming pin
144 memory tests passing (was 137; +7).
Fixes #26045
2026-05-23 02:51:29 -07:00
|
|
|
return bak
|
2026-03-17 04:19:11 -07:00
|
|
|
|
2026-02-19 00:57:31 -08:00
|
|
|
def save_to_disk(self, target: str):
|
|
|
|
|
"""Persist entries to the appropriate file. Called after every mutation."""
|
2026-04-03 13:10:11 -07:00
|
|
|
get_memory_dir().mkdir(parents=True, exist_ok=True)
|
2026-03-17 04:19:11 -07:00
|
|
|
self._write_file(self._path_for(target), self._entries_for(target))
|
2026-02-19 00:57:31 -08:00
|
|
|
|
|
|
|
|
def _entries_for(self, target: str) -> List[str]:
|
|
|
|
|
if target == "user":
|
|
|
|
|
return self.user_entries
|
|
|
|
|
return self.memory_entries
|
|
|
|
|
|
|
|
|
|
def _set_entries(self, target: str, entries: List[str]):
|
|
|
|
|
if target == "user":
|
|
|
|
|
self.user_entries = entries
|
|
|
|
|
else:
|
|
|
|
|
self.memory_entries = entries
|
|
|
|
|
|
|
|
|
|
def _char_count(self, target: str) -> int:
|
|
|
|
|
entries = self._entries_for(target)
|
|
|
|
|
if not entries:
|
|
|
|
|
return 0
|
|
|
|
|
return len(ENTRY_DELIMITER.join(entries))
|
|
|
|
|
|
|
|
|
|
def _char_limit(self, target: str) -> int:
|
|
|
|
|
if target == "user":
|
|
|
|
|
return self.user_char_limit
|
|
|
|
|
return self.memory_char_limit
|
|
|
|
|
|
|
|
|
|
def add(self, target: str, content: str) -> Dict[str, Any]:
|
|
|
|
|
"""Append a new entry. Returns error if it would exceed the char limit."""
|
|
|
|
|
content = content.strip()
|
|
|
|
|
if not content:
|
|
|
|
|
return {"success": False, "error": "Content cannot be empty."}
|
|
|
|
|
|
Harden agent attack surface: scan writes to memory, skills, cron, and context files
The security scanner (skills_guard.py) was only wired into the hub install path.
All other write paths to persistent state — skills created by the agent, memory
entries, cron prompts, and context files — bypassed it entirely. This closes
those gaps:
- file_operations: deny-list blocks writes to ~/.ssh, ~/.aws, ~/.hermes/.env, etc.
- code_execution_tool: filter secret env vars from sandbox child process
- skill_manager_tool: wire scan_skill() into create/edit/patch/write_file with rollback
- skills_guard: add "agent-created" trust level (same policy as community)
- memory_tool: scan content for injection/exfil before system prompt injection
- prompt_builder: scan AGENTS.md, .cursorrules, SOUL.md for prompt injection
- cronjob_tools: scan cron prompts for critical threats before scheduling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 23:43:15 -05:00
|
|
|
# Scan for injection/exfiltration before accepting
|
|
|
|
|
scan_error = _scan_memory_content(content)
|
|
|
|
|
if scan_error:
|
|
|
|
|
return {"success": False, "error": scan_error}
|
|
|
|
|
|
2026-03-17 04:19:11 -07:00
|
|
|
with self._file_lock(self._path_for(target)):
|
fix(memory): guard against external drift in MEMORY.md/USER.md (#26045) (#30877)
Reproduction (production, 2026-05-14): two concurrent sessions on the
same agent. Session A patches MEMORY.md directly via the patch tool,
appending ~8KB of structured content (Vendor Master, Standing Orders,
Pin Board) — none of it through the memory tool, so no § delimiters.
Session B starts later with stale in-memory state (1 entry, ~331
chars). Session B calls memory(action=replace) on its one known
entry. The tool's _read_file parses A's content as a single 8KB
'entry' (no § splits), then replace truncates that entry to B's new
333-byte content. ~8KB of structured content silently destroyed.
The atomic-rename write path is fine in isolation. The bug is the
implicit contract: the tool assumes MEMORY.md is exclusively a
§-delimited list of small entries it wrote, but the v0.13 install
runbook itself uses 'cat >> MEMORY.md' for onboarding, the patch tool
edits the file directly, and operators do too.
Fix: a drift guard in MemoryStore._detect_external_drift that fires
on either signal:
1. Re-parse + re-serialize doesn't produce identical bytes
(catches oddly-encoded delimiters / partial writes).
2. Any single parsed entry exceeds the store's whole-file char
limit. The tool budgets the ENTIRE store against that limit
(2200 chars for memory, 1375 for user), so no tool-written
entry can legitimately be larger. An entry bigger than the
store limit means an external writer dropped free-form content
into what the tool will treat as one entry.
When drift fires, _reload_target writes a .bak.<ts> snapshot of the
on-disk file, then add/replace/remove refuse to flush. The original
file stays untouched. The error dict surfaces the .bak path AND a
remediation string ('integrate missing entries via memory(add=...)
one at a time, then rewrite the file clean') so the model can act on
it without escalating to the operator.
Tests:
- test_replace_refuses_on_drift, test_add_refuses_on_drift,
test_remove_refuses_on_drift — all three mutators refuse
- test_clean_file_does_not_trigger_drift — false-positive check
- test_error_message_points_at_remediation — error string shape
- test_drift_guard_also_protects_user_target — USER.md too
- test_drift_backup_filename_is_unique_per_invocation — bak.<ts>
naming pin
144 memory tests passing (was 137; +7).
Fixes #26045
2026-05-23 02:51:29 -07:00
|
|
|
# Re-read from disk under lock to pick up writes from other sessions.
|
|
|
|
|
# If external drift was detected, the file was backed up to .bak.<ts>
|
|
|
|
|
# — refuse the mutation so we don't clobber the un-roundtrippable
|
|
|
|
|
# content the patch tool / shell append / sister session wrote.
|
|
|
|
|
bak = self._reload_target(target)
|
|
|
|
|
if bak:
|
|
|
|
|
return _drift_error(self._path_for(target), bak)
|
2026-03-17 04:19:11 -07:00
|
|
|
|
|
|
|
|
entries = self._entries_for(target)
|
|
|
|
|
limit = self._char_limit(target)
|
|
|
|
|
|
|
|
|
|
# Reject exact duplicates
|
|
|
|
|
if content in entries:
|
|
|
|
|
return self._success_response(target, "Entry already exists (no duplicate added).")
|
|
|
|
|
|
|
|
|
|
# Calculate what the new total would be
|
|
|
|
|
new_entries = entries + [content]
|
|
|
|
|
new_total = len(ENTRY_DELIMITER.join(new_entries))
|
|
|
|
|
|
|
|
|
|
if new_total > limit:
|
|
|
|
|
current = self._char_count(target)
|
|
|
|
|
return {
|
|
|
|
|
"success": False,
|
|
|
|
|
"error": (
|
|
|
|
|
f"Memory at {current:,}/{limit:,} chars. "
|
|
|
|
|
f"Adding this entry ({len(content)} chars) would exceed the limit. "
|
2026-06-07 22:16:28 -07:00
|
|
|
f"Consolidate now: use 'replace' to merge overlapping entries into "
|
|
|
|
|
f"shorter ones or 'remove' stale or less important entries (see "
|
|
|
|
|
f"current_entries below), then retry this add — all in this turn."
|
2026-03-17 04:19:11 -07:00
|
|
|
),
|
|
|
|
|
"current_entries": entries,
|
|
|
|
|
"usage": f"{current:,}/{limit:,}",
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
entries.append(content)
|
|
|
|
|
self._set_entries(target, entries)
|
|
|
|
|
self.save_to_disk(target)
|
2026-02-19 00:57:31 -08:00
|
|
|
|
|
|
|
|
return self._success_response(target, "Entry added.")
|
|
|
|
|
|
|
|
|
|
def replace(self, target: str, old_text: str, new_content: str) -> Dict[str, Any]:
|
|
|
|
|
"""Find entry containing old_text substring, replace it with new_content."""
|
|
|
|
|
old_text = old_text.strip()
|
|
|
|
|
new_content = new_content.strip()
|
|
|
|
|
if not old_text:
|
|
|
|
|
return {"success": False, "error": "old_text cannot be empty."}
|
|
|
|
|
if not new_content:
|
|
|
|
|
return {"success": False, "error": "new_content cannot be empty. Use 'remove' to delete entries."}
|
|
|
|
|
|
Harden agent attack surface: scan writes to memory, skills, cron, and context files
The security scanner (skills_guard.py) was only wired into the hub install path.
All other write paths to persistent state — skills created by the agent, memory
entries, cron prompts, and context files — bypassed it entirely. This closes
those gaps:
- file_operations: deny-list blocks writes to ~/.ssh, ~/.aws, ~/.hermes/.env, etc.
- code_execution_tool: filter secret env vars from sandbox child process
- skill_manager_tool: wire scan_skill() into create/edit/patch/write_file with rollback
- skills_guard: add "agent-created" trust level (same policy as community)
- memory_tool: scan content for injection/exfil before system prompt injection
- prompt_builder: scan AGENTS.md, .cursorrules, SOUL.md for prompt injection
- cronjob_tools: scan cron prompts for critical threats before scheduling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 23:43:15 -05:00
|
|
|
# Scan replacement content for injection/exfiltration
|
|
|
|
|
scan_error = _scan_memory_content(new_content)
|
|
|
|
|
if scan_error:
|
|
|
|
|
return {"success": False, "error": scan_error}
|
|
|
|
|
|
2026-03-17 04:19:11 -07:00
|
|
|
with self._file_lock(self._path_for(target)):
|
fix(memory): guard against external drift in MEMORY.md/USER.md (#26045) (#30877)
Reproduction (production, 2026-05-14): two concurrent sessions on the
same agent. Session A patches MEMORY.md directly via the patch tool,
appending ~8KB of structured content (Vendor Master, Standing Orders,
Pin Board) — none of it through the memory tool, so no § delimiters.
Session B starts later with stale in-memory state (1 entry, ~331
chars). Session B calls memory(action=replace) on its one known
entry. The tool's _read_file parses A's content as a single 8KB
'entry' (no § splits), then replace truncates that entry to B's new
333-byte content. ~8KB of structured content silently destroyed.
The atomic-rename write path is fine in isolation. The bug is the
implicit contract: the tool assumes MEMORY.md is exclusively a
§-delimited list of small entries it wrote, but the v0.13 install
runbook itself uses 'cat >> MEMORY.md' for onboarding, the patch tool
edits the file directly, and operators do too.
Fix: a drift guard in MemoryStore._detect_external_drift that fires
on either signal:
1. Re-parse + re-serialize doesn't produce identical bytes
(catches oddly-encoded delimiters / partial writes).
2. Any single parsed entry exceeds the store's whole-file char
limit. The tool budgets the ENTIRE store against that limit
(2200 chars for memory, 1375 for user), so no tool-written
entry can legitimately be larger. An entry bigger than the
store limit means an external writer dropped free-form content
into what the tool will treat as one entry.
When drift fires, _reload_target writes a .bak.<ts> snapshot of the
on-disk file, then add/replace/remove refuse to flush. The original
file stays untouched. The error dict surfaces the .bak path AND a
remediation string ('integrate missing entries via memory(add=...)
one at a time, then rewrite the file clean') so the model can act on
it without escalating to the operator.
Tests:
- test_replace_refuses_on_drift, test_add_refuses_on_drift,
test_remove_refuses_on_drift — all three mutators refuse
- test_clean_file_does_not_trigger_drift — false-positive check
- test_error_message_points_at_remediation — error string shape
- test_drift_guard_also_protects_user_target — USER.md too
- test_drift_backup_filename_is_unique_per_invocation — bak.<ts>
naming pin
144 memory tests passing (was 137; +7).
Fixes #26045
2026-05-23 02:51:29 -07:00
|
|
|
bak = self._reload_target(target)
|
|
|
|
|
if bak:
|
|
|
|
|
return _drift_error(self._path_for(target), bak)
|
2026-02-19 00:57:31 -08:00
|
|
|
|
2026-03-17 04:19:11 -07:00
|
|
|
entries = self._entries_for(target)
|
|
|
|
|
matches = [(i, e) for i, e in enumerate(entries) if old_text in e]
|
2026-02-19 00:57:31 -08:00
|
|
|
|
refactor: codebase-wide lint cleanup — unused imports, dead code, and inefficient patterns (#5821)
Comprehensive cleanup across 80 files based on automated (ruff, pyflakes, vulture)
and manual analysis of the entire codebase.
Changes by category:
Unused imports removed (~95 across 55 files):
- Removed genuinely unused imports from all major subsystems
- agent/, hermes_cli/, tools/, gateway/, plugins/, cron/
- Includes imports in try/except blocks that were truly unused
(vs availability checks which were left alone)
Unused variables removed (~25):
- Removed dead variables: connected, inner, channels, last_exc,
source, new_server_names, verify, pconfig, default_terminal,
result, pending_handled, temperature, loop
- Dropped unused argparse subparser assignments in hermes_cli/main.py
(12 instances of add_parser() where result was never used)
Dead code removed:
- run_agent.py: Removed dead ternary (None if False else None) and
surrounding unreachable branch in identity fallback
- run_agent.py: Removed write-only attribute _last_reported_tool
- hermes_cli/providers.py: Removed dead @property decorator on
module-level function (decorator has no effect outside a class)
- gateway/run.py: Removed unused MCP config load before reconnect
- gateway/platforms/slack.py: Removed dead SessionSource construction
Undefined name bugs fixed (would cause NameError at runtime):
- batch_runner.py: Added missing logger = logging.getLogger(__name__)
- tools/environments/daytona.py: Added missing Dict and Path imports
Unnecessary global statements removed (14):
- tools/terminal_tool.py: 5 functions declared global for dicts
they only mutated via .pop()/[key]=value (no rebinding)
- tools/browser_tool.py: cleanup thread loop only reads flag
- tools/rl_training_tool.py: 4 functions only do dict mutations
- tools/mcp_oauth.py: only reads the global
- hermes_time.py: only reads cached values
Inefficient patterns fixed:
- startswith/endswith tuple form: 15 instances of
x.startswith('a') or x.startswith('b') consolidated to
x.startswith(('a', 'b'))
- len(x)==0 / len(x)>0: 13 instances replaced with pythonic
truthiness checks (not x / bool(x))
- in dict.keys(): 5 instances simplified to in dict
- Redefined unused name: removed duplicate _strip_mdv2 import in
send_message_tool.py
Other fixes:
- hermes_cli/doctor.py: Replaced undefined logger.debug() with pass
- hermes_cli/config.py: Consolidated chained .endswith() calls
Test results: 3934 passed, 17 failed (all pre-existing on main),
19 skipped. Zero regressions.
2026-04-07 10:25:31 -07:00
|
|
|
if not matches:
|
2026-03-17 04:19:11 -07:00
|
|
|
return {"success": False, "error": f"No entry matched '{old_text}'."}
|
|
|
|
|
|
|
|
|
|
if len(matches) > 1:
|
|
|
|
|
# If all matches are identical (exact duplicates), operate on the first one
|
2026-05-11 11:20:58 -07:00
|
|
|
unique_texts = {e for _, e in matches}
|
2026-03-17 04:19:11 -07:00
|
|
|
if len(unique_texts) > 1:
|
|
|
|
|
previews = [e[:80] + ("..." if len(e) > 80 else "") for _, e in matches]
|
|
|
|
|
return {
|
|
|
|
|
"success": False,
|
|
|
|
|
"error": f"Multiple entries matched '{old_text}'. Be more specific.",
|
|
|
|
|
"matches": previews,
|
|
|
|
|
}
|
|
|
|
|
# All identical -- safe to replace just the first
|
2026-02-19 00:57:31 -08:00
|
|
|
|
2026-03-17 04:19:11 -07:00
|
|
|
idx = matches[0][0]
|
|
|
|
|
limit = self._char_limit(target)
|
2026-02-19 00:57:31 -08:00
|
|
|
|
2026-03-17 04:19:11 -07:00
|
|
|
# Check that replacement doesn't blow the budget
|
|
|
|
|
test_entries = entries.copy()
|
|
|
|
|
test_entries[idx] = new_content
|
|
|
|
|
new_total = len(ENTRY_DELIMITER.join(test_entries))
|
2026-02-19 00:57:31 -08:00
|
|
|
|
2026-03-17 04:19:11 -07:00
|
|
|
if new_total > limit:
|
2026-06-07 22:16:28 -07:00
|
|
|
current = self._char_count(target)
|
2026-03-17 04:19:11 -07:00
|
|
|
return {
|
|
|
|
|
"success": False,
|
|
|
|
|
"error": (
|
|
|
|
|
f"Replacement would put memory at {new_total:,}/{limit:,} chars. "
|
2026-06-07 22:16:28 -07:00
|
|
|
f"Shorten the new content, or 'remove' other stale or less important "
|
|
|
|
|
f"entries to make room (see current_entries below), then retry — all "
|
|
|
|
|
f"in this turn."
|
2026-03-17 04:19:11 -07:00
|
|
|
),
|
2026-06-07 22:16:28 -07:00
|
|
|
"current_entries": entries,
|
|
|
|
|
"usage": f"{current:,}/{limit:,}",
|
2026-03-17 04:19:11 -07:00
|
|
|
}
|
2026-02-19 00:57:31 -08:00
|
|
|
|
2026-03-17 04:19:11 -07:00
|
|
|
entries[idx] = new_content
|
|
|
|
|
self._set_entries(target, entries)
|
|
|
|
|
self.save_to_disk(target)
|
2026-02-19 00:57:31 -08:00
|
|
|
|
|
|
|
|
return self._success_response(target, "Entry replaced.")
|
|
|
|
|
|
|
|
|
|
def remove(self, target: str, old_text: str) -> Dict[str, Any]:
|
|
|
|
|
"""Remove the entry containing old_text substring."""
|
|
|
|
|
old_text = old_text.strip()
|
|
|
|
|
if not old_text:
|
|
|
|
|
return {"success": False, "error": "old_text cannot be empty."}
|
|
|
|
|
|
2026-03-17 04:19:11 -07:00
|
|
|
with self._file_lock(self._path_for(target)):
|
fix(memory): guard against external drift in MEMORY.md/USER.md (#26045) (#30877)
Reproduction (production, 2026-05-14): two concurrent sessions on the
same agent. Session A patches MEMORY.md directly via the patch tool,
appending ~8KB of structured content (Vendor Master, Standing Orders,
Pin Board) — none of it through the memory tool, so no § delimiters.
Session B starts later with stale in-memory state (1 entry, ~331
chars). Session B calls memory(action=replace) on its one known
entry. The tool's _read_file parses A's content as a single 8KB
'entry' (no § splits), then replace truncates that entry to B's new
333-byte content. ~8KB of structured content silently destroyed.
The atomic-rename write path is fine in isolation. The bug is the
implicit contract: the tool assumes MEMORY.md is exclusively a
§-delimited list of small entries it wrote, but the v0.13 install
runbook itself uses 'cat >> MEMORY.md' for onboarding, the patch tool
edits the file directly, and operators do too.
Fix: a drift guard in MemoryStore._detect_external_drift that fires
on either signal:
1. Re-parse + re-serialize doesn't produce identical bytes
(catches oddly-encoded delimiters / partial writes).
2. Any single parsed entry exceeds the store's whole-file char
limit. The tool budgets the ENTIRE store against that limit
(2200 chars for memory, 1375 for user), so no tool-written
entry can legitimately be larger. An entry bigger than the
store limit means an external writer dropped free-form content
into what the tool will treat as one entry.
When drift fires, _reload_target writes a .bak.<ts> snapshot of the
on-disk file, then add/replace/remove refuse to flush. The original
file stays untouched. The error dict surfaces the .bak path AND a
remediation string ('integrate missing entries via memory(add=...)
one at a time, then rewrite the file clean') so the model can act on
it without escalating to the operator.
Tests:
- test_replace_refuses_on_drift, test_add_refuses_on_drift,
test_remove_refuses_on_drift — all three mutators refuse
- test_clean_file_does_not_trigger_drift — false-positive check
- test_error_message_points_at_remediation — error string shape
- test_drift_guard_also_protects_user_target — USER.md too
- test_drift_backup_filename_is_unique_per_invocation — bak.<ts>
naming pin
144 memory tests passing (was 137; +7).
Fixes #26045
2026-05-23 02:51:29 -07:00
|
|
|
bak = self._reload_target(target)
|
|
|
|
|
if bak:
|
|
|
|
|
return _drift_error(self._path_for(target), bak)
|
2026-03-17 04:19:11 -07:00
|
|
|
|
|
|
|
|
entries = self._entries_for(target)
|
|
|
|
|
matches = [(i, e) for i, e in enumerate(entries) if old_text in e]
|
|
|
|
|
|
refactor: codebase-wide lint cleanup — unused imports, dead code, and inefficient patterns (#5821)
Comprehensive cleanup across 80 files based on automated (ruff, pyflakes, vulture)
and manual analysis of the entire codebase.
Changes by category:
Unused imports removed (~95 across 55 files):
- Removed genuinely unused imports from all major subsystems
- agent/, hermes_cli/, tools/, gateway/, plugins/, cron/
- Includes imports in try/except blocks that were truly unused
(vs availability checks which were left alone)
Unused variables removed (~25):
- Removed dead variables: connected, inner, channels, last_exc,
source, new_server_names, verify, pconfig, default_terminal,
result, pending_handled, temperature, loop
- Dropped unused argparse subparser assignments in hermes_cli/main.py
(12 instances of add_parser() where result was never used)
Dead code removed:
- run_agent.py: Removed dead ternary (None if False else None) and
surrounding unreachable branch in identity fallback
- run_agent.py: Removed write-only attribute _last_reported_tool
- hermes_cli/providers.py: Removed dead @property decorator on
module-level function (decorator has no effect outside a class)
- gateway/run.py: Removed unused MCP config load before reconnect
- gateway/platforms/slack.py: Removed dead SessionSource construction
Undefined name bugs fixed (would cause NameError at runtime):
- batch_runner.py: Added missing logger = logging.getLogger(__name__)
- tools/environments/daytona.py: Added missing Dict and Path imports
Unnecessary global statements removed (14):
- tools/terminal_tool.py: 5 functions declared global for dicts
they only mutated via .pop()/[key]=value (no rebinding)
- tools/browser_tool.py: cleanup thread loop only reads flag
- tools/rl_training_tool.py: 4 functions only do dict mutations
- tools/mcp_oauth.py: only reads the global
- hermes_time.py: only reads cached values
Inefficient patterns fixed:
- startswith/endswith tuple form: 15 instances of
x.startswith('a') or x.startswith('b') consolidated to
x.startswith(('a', 'b'))
- len(x)==0 / len(x)>0: 13 instances replaced with pythonic
truthiness checks (not x / bool(x))
- in dict.keys(): 5 instances simplified to in dict
- Redefined unused name: removed duplicate _strip_mdv2 import in
send_message_tool.py
Other fixes:
- hermes_cli/doctor.py: Replaced undefined logger.debug() with pass
- hermes_cli/config.py: Consolidated chained .endswith() calls
Test results: 3934 passed, 17 failed (all pre-existing on main),
19 skipped. Zero regressions.
2026-04-07 10:25:31 -07:00
|
|
|
if not matches:
|
2026-03-17 04:19:11 -07:00
|
|
|
return {"success": False, "error": f"No entry matched '{old_text}'."}
|
|
|
|
|
|
|
|
|
|
if len(matches) > 1:
|
|
|
|
|
# If all matches are identical (exact duplicates), remove the first one
|
2026-05-11 11:20:58 -07:00
|
|
|
unique_texts = {e for _, e in matches}
|
2026-03-17 04:19:11 -07:00
|
|
|
if len(unique_texts) > 1:
|
|
|
|
|
previews = [e[:80] + ("..." if len(e) > 80 else "") for _, e in matches]
|
|
|
|
|
return {
|
|
|
|
|
"success": False,
|
|
|
|
|
"error": f"Multiple entries matched '{old_text}'. Be more specific.",
|
|
|
|
|
"matches": previews,
|
|
|
|
|
}
|
|
|
|
|
# All identical -- safe to remove just the first
|
|
|
|
|
|
|
|
|
|
idx = matches[0][0]
|
|
|
|
|
entries.pop(idx)
|
|
|
|
|
self._set_entries(target, entries)
|
|
|
|
|
self.save_to_disk(target)
|
2026-02-19 00:57:31 -08:00
|
|
|
|
|
|
|
|
return self._success_response(target, "Entry removed.")
|
|
|
|
|
|
|
|
|
|
def format_for_system_prompt(self, target: str) -> Optional[str]:
|
|
|
|
|
"""
|
|
|
|
|
Return the frozen snapshot for system prompt injection.
|
|
|
|
|
|
|
|
|
|
This returns the state captured at load_from_disk() time, NOT the live
|
|
|
|
|
state. Mid-session writes do not affect this. This keeps the system
|
|
|
|
|
prompt stable across all turns, preserving the prefix cache.
|
|
|
|
|
|
|
|
|
|
Returns None if the snapshot is empty (no entries at load time).
|
|
|
|
|
"""
|
|
|
|
|
block = self._system_prompt_snapshot.get(target, "")
|
|
|
|
|
return block if block else None
|
|
|
|
|
|
|
|
|
|
# -- Internal helpers --
|
|
|
|
|
|
|
|
|
|
def _success_response(self, target: str, message: str = None) -> Dict[str, Any]:
|
|
|
|
|
entries = self._entries_for(target)
|
|
|
|
|
current = self._char_count(target)
|
|
|
|
|
limit = self._char_limit(target)
|
2026-03-28 14:55:18 -07:00
|
|
|
pct = min(100, int((current / limit) * 100)) if limit > 0 else 0
|
2026-02-19 00:57:31 -08:00
|
|
|
|
|
|
|
|
resp = {
|
|
|
|
|
"success": True,
|
|
|
|
|
"target": target,
|
|
|
|
|
"entries": entries,
|
|
|
|
|
"usage": f"{pct}% — {current:,}/{limit:,} chars",
|
|
|
|
|
"entry_count": len(entries),
|
|
|
|
|
}
|
|
|
|
|
if message:
|
|
|
|
|
resp["message"] = message
|
|
|
|
|
return resp
|
|
|
|
|
|
|
|
|
|
def _render_block(self, target: str, entries: List[str]) -> str:
|
|
|
|
|
"""Render a system prompt block with header and usage indicator."""
|
|
|
|
|
if not entries:
|
|
|
|
|
return ""
|
|
|
|
|
|
|
|
|
|
limit = self._char_limit(target)
|
|
|
|
|
content = ENTRY_DELIMITER.join(entries)
|
|
|
|
|
current = len(content)
|
2026-03-28 14:55:18 -07:00
|
|
|
pct = min(100, int((current / limit) * 100)) if limit > 0 else 0
|
2026-02-19 00:57:31 -08:00
|
|
|
|
|
|
|
|
if target == "user":
|
|
|
|
|
header = f"USER PROFILE (who the user is) [{pct}% — {current:,}/{limit:,} chars]"
|
|
|
|
|
else:
|
|
|
|
|
header = f"MEMORY (your personal notes) [{pct}% — {current:,}/{limit:,} chars]"
|
|
|
|
|
|
|
|
|
|
separator = "═" * 46
|
|
|
|
|
return f"{separator}\n{header}\n{separator}\n{content}"
|
|
|
|
|
|
|
|
|
|
@staticmethod
|
|
|
|
|
def _read_file(path: Path) -> List[str]:
|
2026-02-20 02:32:15 -08:00
|
|
|
"""Read a memory file and split into entries.
|
|
|
|
|
|
|
|
|
|
No file locking needed: _write_file uses atomic rename, so readers
|
|
|
|
|
always see either the previous complete file or the new complete file.
|
|
|
|
|
"""
|
2026-02-19 00:57:31 -08:00
|
|
|
if not path.exists():
|
|
|
|
|
return []
|
|
|
|
|
try:
|
2026-02-20 02:32:15 -08:00
|
|
|
raw = path.read_text(encoding="utf-8")
|
2026-02-19 00:57:31 -08:00
|
|
|
except (OSError, IOError):
|
|
|
|
|
return []
|
|
|
|
|
|
|
|
|
|
if not raw.strip():
|
|
|
|
|
return []
|
|
|
|
|
|
2026-02-28 01:33:41 +03:00
|
|
|
# Use ENTRY_DELIMITER for consistency with _write_file. Splitting by "§"
|
|
|
|
|
# alone would incorrectly split entries that contain "§" in their content.
|
|
|
|
|
entries = [e.strip() for e in raw.split(ENTRY_DELIMITER)]
|
2026-02-19 00:57:31 -08:00
|
|
|
return [e for e in entries if e]
|
|
|
|
|
|
fix(memory): guard against external drift in MEMORY.md/USER.md (#26045) (#30877)
Reproduction (production, 2026-05-14): two concurrent sessions on the
same agent. Session A patches MEMORY.md directly via the patch tool,
appending ~8KB of structured content (Vendor Master, Standing Orders,
Pin Board) — none of it through the memory tool, so no § delimiters.
Session B starts later with stale in-memory state (1 entry, ~331
chars). Session B calls memory(action=replace) on its one known
entry. The tool's _read_file parses A's content as a single 8KB
'entry' (no § splits), then replace truncates that entry to B's new
333-byte content. ~8KB of structured content silently destroyed.
The atomic-rename write path is fine in isolation. The bug is the
implicit contract: the tool assumes MEMORY.md is exclusively a
§-delimited list of small entries it wrote, but the v0.13 install
runbook itself uses 'cat >> MEMORY.md' for onboarding, the patch tool
edits the file directly, and operators do too.
Fix: a drift guard in MemoryStore._detect_external_drift that fires
on either signal:
1. Re-parse + re-serialize doesn't produce identical bytes
(catches oddly-encoded delimiters / partial writes).
2. Any single parsed entry exceeds the store's whole-file char
limit. The tool budgets the ENTIRE store against that limit
(2200 chars for memory, 1375 for user), so no tool-written
entry can legitimately be larger. An entry bigger than the
store limit means an external writer dropped free-form content
into what the tool will treat as one entry.
When drift fires, _reload_target writes a .bak.<ts> snapshot of the
on-disk file, then add/replace/remove refuse to flush. The original
file stays untouched. The error dict surfaces the .bak path AND a
remediation string ('integrate missing entries via memory(add=...)
one at a time, then rewrite the file clean') so the model can act on
it without escalating to the operator.
Tests:
- test_replace_refuses_on_drift, test_add_refuses_on_drift,
test_remove_refuses_on_drift — all three mutators refuse
- test_clean_file_does_not_trigger_drift — false-positive check
- test_error_message_points_at_remediation — error string shape
- test_drift_guard_also_protects_user_target — USER.md too
- test_drift_backup_filename_is_unique_per_invocation — bak.<ts>
naming pin
144 memory tests passing (was 137; +7).
Fixes #26045
2026-05-23 02:51:29 -07:00
|
|
|
def _detect_external_drift(self, target: str) -> Optional[str]:
|
|
|
|
|
"""Return a backup-path string if on-disk content shows external drift.
|
|
|
|
|
|
|
|
|
|
The memory file is supposed to be a list of small entries the tool
|
|
|
|
|
wrote, joined by §. Detect drift via two signals:
|
|
|
|
|
|
|
|
|
|
1. Round-trip mismatch — re-parsing and re-serializing the file
|
|
|
|
|
doesn't produce identical bytes (rare; would catch oddly-encoded
|
|
|
|
|
delimiters).
|
|
|
|
|
2. Entry-size overflow — any single parsed entry exceeds the
|
|
|
|
|
store's whole-file char limit. The tool budgets the ENTIRE store
|
|
|
|
|
against that limit; no single tool-written entry can exceed it.
|
|
|
|
|
When we see one entry larger than the limit, an external writer
|
|
|
|
|
(patch tool, shell append, manual edit, sister session) appended
|
|
|
|
|
free-form content into what the tool will treat as one entry.
|
|
|
|
|
Flushing would then truncate that entry to the model's new
|
|
|
|
|
content, discarding the appended bytes — issue #26045.
|
|
|
|
|
|
|
|
|
|
Returns the absolute path of the .bak file when drift was found and
|
|
|
|
|
backed up; returns None when the file looks tool-shaped.
|
|
|
|
|
|
|
|
|
|
Note: this is an INSTANCE method (not static) because we need the
|
|
|
|
|
per-target char_limit for signal #2.
|
|
|
|
|
"""
|
|
|
|
|
path = self._path_for(target)
|
|
|
|
|
if not path.exists():
|
|
|
|
|
return None
|
|
|
|
|
try:
|
|
|
|
|
raw = path.read_text(encoding="utf-8")
|
|
|
|
|
except (OSError, IOError):
|
|
|
|
|
return None
|
|
|
|
|
if not raw.strip():
|
|
|
|
|
return None
|
|
|
|
|
|
|
|
|
|
parsed = [e.strip() for e in raw.split(ENTRY_DELIMITER) if e.strip()]
|
|
|
|
|
roundtrip = ENTRY_DELIMITER.join(parsed)
|
|
|
|
|
|
|
|
|
|
char_limit = self._char_limit(target)
|
|
|
|
|
max_entry_len = max((len(e) for e in parsed), default=0)
|
|
|
|
|
|
|
|
|
|
drift_detected = (raw.strip() != roundtrip) or (max_entry_len > char_limit)
|
|
|
|
|
if not drift_detected:
|
|
|
|
|
return None
|
|
|
|
|
|
|
|
|
|
# Drift confirmed — snapshot the file so the operator can recover
|
|
|
|
|
# whatever the external writer added, then return the .bak path so
|
|
|
|
|
# the caller can refuse the mutation.
|
|
|
|
|
ts = int(time.time())
|
|
|
|
|
bak_path = path.with_suffix(path.suffix + f".bak.{ts}")
|
|
|
|
|
try:
|
|
|
|
|
bak_path.write_text(raw, encoding="utf-8")
|
|
|
|
|
except (OSError, IOError):
|
|
|
|
|
return str(bak_path) + " (BACKUP FAILED — file unchanged on disk)"
|
|
|
|
|
return str(bak_path)
|
|
|
|
|
|
2026-02-19 00:57:31 -08:00
|
|
|
@staticmethod
|
|
|
|
|
def _write_file(path: Path, entries: List[str]):
|
2026-02-20 02:32:15 -08:00
|
|
|
"""Write entries to a memory file using atomic temp-file + rename.
|
|
|
|
|
|
|
|
|
|
Previous implementation used open("w") + flock, but "w" truncates the
|
|
|
|
|
file *before* the lock is acquired, creating a race window where
|
|
|
|
|
concurrent readers see an empty file. Atomic rename avoids this:
|
|
|
|
|
readers always see either the old complete file or the new one.
|
|
|
|
|
"""
|
2026-02-19 00:57:31 -08:00
|
|
|
content = ENTRY_DELIMITER.join(entries) if entries else ""
|
|
|
|
|
try:
|
2026-02-20 02:32:15 -08:00
|
|
|
# Write to temp file in same directory (same filesystem for atomic rename)
|
|
|
|
|
fd, tmp_path = tempfile.mkstemp(
|
|
|
|
|
dir=str(path.parent), suffix=".tmp", prefix=".mem_"
|
|
|
|
|
)
|
|
|
|
|
try:
|
|
|
|
|
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
2026-02-19 00:57:31 -08:00
|
|
|
f.write(content)
|
2026-02-20 02:32:15 -08:00
|
|
|
f.flush()
|
|
|
|
|
os.fsync(f.fileno())
|
refactor: consolidate symlink-safe atomic replace into shared helper
Extract the islink/realpath guard from the 16743 fix into a single
atomic_replace() helper in utils.py, then migrate every os.replace()
call site in the codebase to use it.
The original PR #16777 correctly identified and fixed the bug, but
only patched 9 of ~24 call sites. The same bug class (managed
deployments that symlink state files silently losing the link on
every write) still existed at auth.json, sessions file, gateway
config, env_loader, webhook subscriptions, debug store, model
catalog, pairing, google OAuth, nous rate guard, and more.
Rather than add another 10+ copies of the same three-line guard,
consolidate into atomic_replace(tmp, target) which:
- resolves symlinks via os.path.realpath before os.replace
- returns the resolved real path so callers can re-apply permissions
- is a drop-in replacement for os.replace at the use sites
Changes:
- utils.py: new atomic_replace() helper + atomic_json_write /
atomic_yaml_write now call it instead of inlining the guard
- 16 files: all os.replace() call sites migrated to atomic_replace()
- agent/{google_oauth, nous_rate_guard, shell_hooks}.py
- cron/jobs.py
- gateway/{pairing, session, platforms/telegram}.py
- hermes_cli/{auth, config, debug, env_loader, model_catalog, webhook}.py
- tools/{memory_tool, skill_manager_tool, skills_sync}.py
Tests: tests/test_atomic_replace_symlinks.py pins the invariant for
atomic_replace + atomic_json_write + atomic_yaml_write, covers plain
files, first-time creates, broken symlinks, and permission preservation.
Refs #16743
Builds on #16777 by @vominh1919.
2026-04-28 04:51:38 -07:00
|
|
|
atomic_replace(tmp_path, path)
|
2026-02-20 02:32:15 -08:00
|
|
|
except BaseException:
|
|
|
|
|
# Clean up temp file on any failure
|
|
|
|
|
try:
|
|
|
|
|
os.unlink(tmp_path)
|
|
|
|
|
except OSError:
|
|
|
|
|
pass
|
|
|
|
|
raise
|
2026-02-19 00:57:31 -08:00
|
|
|
except (OSError, IOError) as e:
|
|
|
|
|
raise RuntimeError(f"Failed to write memory file {path}: {e}")
|
|
|
|
|
|
|
|
|
|
|
2026-06-09 21:51:43 -07:00
|
|
|
def _apply_write_gate(action: str, target: str, content: Optional[str],
|
|
|
|
|
old_text: Optional[str]) -> Optional[str]:
|
|
|
|
|
"""Evaluate the memory write gate. Returns a JSON tool-result string when
|
|
|
|
|
the write should NOT proceed normally (blocked or staged), or None when the
|
|
|
|
|
caller should perform the real write.
|
|
|
|
|
|
|
|
|
|
Only the mutating actions (add/replace/remove) are gated.
|
|
|
|
|
"""
|
|
|
|
|
if action not in {"add", "replace", "remove"}:
|
|
|
|
|
return None
|
|
|
|
|
|
|
|
|
|
try:
|
|
|
|
|
from tools import write_approval as wa
|
|
|
|
|
except Exception:
|
|
|
|
|
# If the gate module can't load, fail open (current behaviour) rather
|
|
|
|
|
# than blocking all memory writes.
|
|
|
|
|
return None
|
|
|
|
|
|
|
|
|
|
# Build a small inline summary/detail for the foreground approval prompt.
|
|
|
|
|
label = "user profile" if target == "user" else "memory"
|
|
|
|
|
if action == "add":
|
|
|
|
|
summary = f"add to {label}"
|
|
|
|
|
detail = content or ""
|
|
|
|
|
elif action == "replace":
|
|
|
|
|
summary = f"replace in {label}"
|
|
|
|
|
detail = f"old: {old_text}\nnew: {content}"
|
|
|
|
|
else: # remove
|
|
|
|
|
summary = f"remove from {label}"
|
|
|
|
|
detail = old_text or ""
|
|
|
|
|
|
|
|
|
|
decision = wa.evaluate_gate(wa.MEMORY, inline_summary=summary, inline_detail=detail)
|
|
|
|
|
|
|
|
|
|
if decision.allow:
|
|
|
|
|
return None
|
|
|
|
|
|
|
|
|
|
if decision.blocked:
|
|
|
|
|
return tool_error(decision.message, success=False)
|
|
|
|
|
|
|
|
|
|
# stage
|
|
|
|
|
payload = {
|
|
|
|
|
"action": action,
|
|
|
|
|
"target": target,
|
|
|
|
|
"content": content,
|
|
|
|
|
"old_text": old_text,
|
|
|
|
|
}
|
|
|
|
|
record = wa.stage_write(
|
|
|
|
|
wa.MEMORY, payload,
|
|
|
|
|
summary=f"{summary}: {detail[:120]}",
|
|
|
|
|
origin=wa.current_origin(),
|
|
|
|
|
)
|
|
|
|
|
return json.dumps(
|
|
|
|
|
{"success": True, "staged": True, "pending_id": record["id"],
|
|
|
|
|
"message": decision.message},
|
|
|
|
|
ensure_ascii=False,
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
2026-02-19 00:57:31 -08:00
|
|
|
def memory_tool(
|
|
|
|
|
action: str,
|
|
|
|
|
target: str = "memory",
|
|
|
|
|
content: str = None,
|
|
|
|
|
old_text: str = None,
|
|
|
|
|
store: Optional[MemoryStore] = None,
|
|
|
|
|
) -> str:
|
|
|
|
|
"""
|
|
|
|
|
Single entry point for the memory tool. Dispatches to MemoryStore methods.
|
|
|
|
|
|
|
|
|
|
Returns JSON string with results.
|
|
|
|
|
"""
|
|
|
|
|
if store is None:
|
refactor: add tool_error/tool_result helpers + read_raw_config, migrate 129 callsites
Add three reusable helpers to eliminate pervasive boilerplate:
tools/registry.py — tool_error() and tool_result():
Every tool handler returns JSON strings. The pattern
json.dumps({"error": msg}, ensure_ascii=False) appeared 106 times,
and json.dumps({"success": False, "error": msg}, ...) another 23.
Now: tool_error(msg) or tool_error(msg, success=False).
tool_result() handles arbitrary result dicts:
tool_result(success=True, data=payload) or tool_result(some_dict).
hermes_cli/config.py — read_raw_config():
Lightweight YAML reader that returns the raw config dict without
load_config()'s deep-merge + migration overhead. Available for
callsites that just need a single config value.
Migration (129 callsites across 32 files):
- tools/: browser_camofox (18), file_tools (10), homeassistant (8),
web_tools (7), skill_manager (7), cronjob (11), code_execution (4),
delegate (5), send_message (4), tts (4), memory (7), session_search (3),
mcp (2), clarify (2), skills_tool (3), todo (1), vision (1),
browser (1), process_registry (2), image_gen (1)
- plugins/memory/: honcho (9), supermemory (9), hindsight (8),
holographic (7), openviking (7), mem0 (7), byterover (6), retaindb (2)
- agent/: memory_manager (2), builtin_memory_provider (1)
2026-04-07 13:36:20 -07:00
|
|
|
return tool_error("Memory is not available. It may be disabled in config or this environment.", success=False)
|
2026-02-19 00:57:31 -08:00
|
|
|
|
2026-05-11 11:13:25 -07:00
|
|
|
if target not in {"memory", "user"}:
|
refactor: add tool_error/tool_result helpers + read_raw_config, migrate 129 callsites
Add three reusable helpers to eliminate pervasive boilerplate:
tools/registry.py — tool_error() and tool_result():
Every tool handler returns JSON strings. The pattern
json.dumps({"error": msg}, ensure_ascii=False) appeared 106 times,
and json.dumps({"success": False, "error": msg}, ...) another 23.
Now: tool_error(msg) or tool_error(msg, success=False).
tool_result() handles arbitrary result dicts:
tool_result(success=True, data=payload) or tool_result(some_dict).
hermes_cli/config.py — read_raw_config():
Lightweight YAML reader that returns the raw config dict without
load_config()'s deep-merge + migration overhead. Available for
callsites that just need a single config value.
Migration (129 callsites across 32 files):
- tools/: browser_camofox (18), file_tools (10), homeassistant (8),
web_tools (7), skill_manager (7), cronjob (11), code_execution (4),
delegate (5), send_message (4), tts (4), memory (7), session_search (3),
mcp (2), clarify (2), skills_tool (3), todo (1), vision (1),
browser (1), process_registry (2), image_gen (1)
- plugins/memory/: honcho (9), supermemory (9), hindsight (8),
holographic (7), openviking (7), mem0 (7), byterover (6), retaindb (2)
- agent/: memory_manager (2), builtin_memory_provider (1)
2026-04-07 13:36:20 -07:00
|
|
|
return tool_error(f"Invalid target '{target}'. Use 'memory' or 'user'.", success=False)
|
2026-02-19 00:57:31 -08:00
|
|
|
|
2026-06-10 02:57:15 -07:00
|
|
|
# Validate required params BEFORE the gate so an invalid write is rejected
|
|
|
|
|
# immediately instead of being staged and only failing at approve time.
|
|
|
|
|
if action == "add" and not content:
|
|
|
|
|
return tool_error("Content is required for 'add' action.", success=False)
|
|
|
|
|
if action == "replace" and (not old_text or not content):
|
|
|
|
|
missing = "old_text" if not old_text else "content"
|
|
|
|
|
return tool_error(f"{missing} is required for 'replace' action.", success=False)
|
|
|
|
|
if action == "remove" and not old_text:
|
|
|
|
|
return tool_error("old_text is required for 'remove' action.", success=False)
|
|
|
|
|
|
|
|
|
|
# Approval gate: when on, stages the write (background/gateway) or prompts
|
|
|
|
|
# inline (interactive CLI); when off (default) passes straight through.
|
2026-06-09 21:51:43 -07:00
|
|
|
gate_result = _apply_write_gate(action, target, content, old_text)
|
|
|
|
|
if gate_result is not None:
|
|
|
|
|
return gate_result
|
|
|
|
|
|
2026-02-19 00:57:31 -08:00
|
|
|
if action == "add":
|
|
|
|
|
result = store.add(target, content)
|
|
|
|
|
|
|
|
|
|
elif action == "replace":
|
|
|
|
|
result = store.replace(target, old_text, content)
|
|
|
|
|
|
|
|
|
|
elif action == "remove":
|
|
|
|
|
result = store.remove(target, old_text)
|
|
|
|
|
|
|
|
|
|
else:
|
refactor: add tool_error/tool_result helpers + read_raw_config, migrate 129 callsites
Add three reusable helpers to eliminate pervasive boilerplate:
tools/registry.py — tool_error() and tool_result():
Every tool handler returns JSON strings. The pattern
json.dumps({"error": msg}, ensure_ascii=False) appeared 106 times,
and json.dumps({"success": False, "error": msg}, ...) another 23.
Now: tool_error(msg) or tool_error(msg, success=False).
tool_result() handles arbitrary result dicts:
tool_result(success=True, data=payload) or tool_result(some_dict).
hermes_cli/config.py — read_raw_config():
Lightweight YAML reader that returns the raw config dict without
load_config()'s deep-merge + migration overhead. Available for
callsites that just need a single config value.
Migration (129 callsites across 32 files):
- tools/: browser_camofox (18), file_tools (10), homeassistant (8),
web_tools (7), skill_manager (7), cronjob (11), code_execution (4),
delegate (5), send_message (4), tts (4), memory (7), session_search (3),
mcp (2), clarify (2), skills_tool (3), todo (1), vision (1),
browser (1), process_registry (2), image_gen (1)
- plugins/memory/: honcho (9), supermemory (9), hindsight (8),
holographic (7), openviking (7), mem0 (7), byterover (6), retaindb (2)
- agent/: memory_manager (2), builtin_memory_provider (1)
2026-04-07 13:36:20 -07:00
|
|
|
return tool_error(f"Unknown action '{action}'. Use: add, replace, remove", success=False)
|
2026-02-19 00:57:31 -08:00
|
|
|
|
|
|
|
|
return json.dumps(result, ensure_ascii=False)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def check_memory_requirements() -> bool:
|
|
|
|
|
"""Memory tool has no external requirements -- always available."""
|
|
|
|
|
return True
|
|
|
|
|
|
|
|
|
|
|
2026-06-09 21:51:43 -07:00
|
|
|
def apply_memory_pending(payload: Dict[str, Any], store: "MemoryStore") -> Dict[str, Any]:
|
|
|
|
|
"""Replay a staged memory write directly against the store, bypassing the
|
|
|
|
|
write gate. Called by the /memory approve handler.
|
|
|
|
|
|
|
|
|
|
Returns the store's result dict.
|
|
|
|
|
"""
|
|
|
|
|
action = payload.get("action")
|
|
|
|
|
target = payload.get("target", "memory")
|
|
|
|
|
content = payload.get("content") or ""
|
|
|
|
|
old_text = payload.get("old_text") or ""
|
|
|
|
|
if action == "add":
|
|
|
|
|
return store.add(target, content)
|
|
|
|
|
if action == "replace":
|
|
|
|
|
return store.replace(target, old_text, content)
|
|
|
|
|
if action == "remove":
|
|
|
|
|
return store.remove(target, old_text)
|
|
|
|
|
return {"success": False, "error": f"Unknown staged action '{action}'."}
|
2026-02-19 00:57:31 -08:00
|
|
|
# OpenAI Function-Calling Schema
|
|
|
|
|
# =============================================================================
|
|
|
|
|
|
|
|
|
|
MEMORY_SCHEMA = {
|
|
|
|
|
"name": "memory",
|
|
|
|
|
"description": (
|
2026-03-14 11:26:18 -07:00
|
|
|
"Save durable information to persistent memory that survives across sessions. "
|
|
|
|
|
"Memory is injected into future turns, so keep it compact and focused on facts "
|
|
|
|
|
"that will still matter later.\n\n"
|
2026-02-22 02:31:52 -08:00
|
|
|
"WHEN TO SAVE (do this proactively, don't wait to be asked):\n"
|
2026-03-16 06:52:32 -07:00
|
|
|
"- User corrects you or says 'remember this' / 'don't do that again'\n"
|
2026-02-22 02:31:52 -08:00
|
|
|
"- User shares a preference, habit, or personal detail (name, role, timezone, coding style)\n"
|
|
|
|
|
"- You discover something about the environment (OS, installed tools, project structure)\n"
|
|
|
|
|
"- You learn a convention, API quirk, or workflow specific to this user's setup\n"
|
2026-03-14 11:26:18 -07:00
|
|
|
"- You identify a stable fact that will be useful again in future sessions\n\n"
|
2026-03-16 06:52:32 -07:00
|
|
|
"PRIORITY: User preferences and corrections > environment facts > procedural knowledge. "
|
|
|
|
|
"The most valuable memory prevents the user from having to repeat themselves.\n\n"
|
2026-03-14 11:26:18 -07:00
|
|
|
"Do NOT save task progress, session outcomes, completed-work logs, or temporary TODO "
|
|
|
|
|
"state to memory; use session_search to recall those from past transcripts.\n"
|
|
|
|
|
"If you've discovered a new way to do something, solved a problem that could be "
|
|
|
|
|
"necessary later, save it as a skill with the skill tool.\n\n"
|
2026-02-22 02:31:52 -08:00
|
|
|
"TWO TARGETS:\n"
|
|
|
|
|
"- 'user': who the user is -- name, role, preferences, communication style, pet peeves\n"
|
|
|
|
|
"- 'memory': your notes -- environment facts, project conventions, tool quirks, lessons learned\n\n"
|
|
|
|
|
"ACTIONS: add (new entry), replace (update existing -- old_text identifies it), "
|
2026-03-14 11:26:18 -07:00
|
|
|
"remove (delete -- old_text identifies it).\n\n"
|
|
|
|
|
"SKIP: trivial/obvious info, things easily re-discovered, raw data dumps, and temporary task state."
|
2026-02-19 00:57:31 -08:00
|
|
|
),
|
|
|
|
|
"parameters": {
|
|
|
|
|
"type": "object",
|
|
|
|
|
"properties": {
|
|
|
|
|
"action": {
|
|
|
|
|
"type": "string",
|
2026-02-19 01:03:08 -08:00
|
|
|
"enum": ["add", "replace", "remove"],
|
2026-02-19 00:57:31 -08:00
|
|
|
"description": "The action to perform."
|
|
|
|
|
},
|
|
|
|
|
"target": {
|
|
|
|
|
"type": "string",
|
|
|
|
|
"enum": ["memory", "user"],
|
|
|
|
|
"description": "Which memory store: 'memory' for personal notes, 'user' for user profile."
|
|
|
|
|
},
|
|
|
|
|
"content": {
|
|
|
|
|
"type": "string",
|
|
|
|
|
"description": "The entry content. Required for 'add' and 'replace'."
|
|
|
|
|
},
|
|
|
|
|
"old_text": {
|
|
|
|
|
"type": "string",
|
|
|
|
|
"description": "Short unique substring identifying the entry to replace or remove."
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
"required": ["action", "target"],
|
|
|
|
|
},
|
|
|
|
|
}
|
2026-02-20 02:32:15 -08:00
|
|
|
|
|
|
|
|
|
2026-02-21 20:22:33 -08:00
|
|
|
# --- Registry ---
|
refactor: add tool_error/tool_result helpers + read_raw_config, migrate 129 callsites
Add three reusable helpers to eliminate pervasive boilerplate:
tools/registry.py — tool_error() and tool_result():
Every tool handler returns JSON strings. The pattern
json.dumps({"error": msg}, ensure_ascii=False) appeared 106 times,
and json.dumps({"success": False, "error": msg}, ...) another 23.
Now: tool_error(msg) or tool_error(msg, success=False).
tool_result() handles arbitrary result dicts:
tool_result(success=True, data=payload) or tool_result(some_dict).
hermes_cli/config.py — read_raw_config():
Lightweight YAML reader that returns the raw config dict without
load_config()'s deep-merge + migration overhead. Available for
callsites that just need a single config value.
Migration (129 callsites across 32 files):
- tools/: browser_camofox (18), file_tools (10), homeassistant (8),
web_tools (7), skill_manager (7), cronjob (11), code_execution (4),
delegate (5), send_message (4), tts (4), memory (7), session_search (3),
mcp (2), clarify (2), skills_tool (3), todo (1), vision (1),
browser (1), process_registry (2), image_gen (1)
- plugins/memory/: honcho (9), supermemory (9), hindsight (8),
holographic (7), openviking (7), mem0 (7), byterover (6), retaindb (2)
- agent/: memory_manager (2), builtin_memory_provider (1)
2026-04-07 13:36:20 -07:00
|
|
|
from tools.registry import registry, tool_error
|
2026-02-21 20:22:33 -08:00
|
|
|
|
|
|
|
|
registry.register(
|
|
|
|
|
name="memory",
|
|
|
|
|
toolset="memory",
|
|
|
|
|
schema=MEMORY_SCHEMA,
|
|
|
|
|
handler=lambda args, **kw: memory_tool(
|
|
|
|
|
action=args.get("action", ""),
|
|
|
|
|
target=args.get("target", "memory"),
|
|
|
|
|
content=args.get("content"),
|
|
|
|
|
old_text=args.get("old_text"),
|
|
|
|
|
store=kw.get("store")),
|
|
|
|
|
check_fn=check_memory_requirements,
|
2026-03-15 20:21:21 -07:00
|
|
|
emoji="🧠",
|
2026-02-21 20:22:33 -08:00
|
|
|
)
|
2026-02-20 02:32:15 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|