fix(agent): bound remaining memory/history pollution paths from #3412
#3412 stopped the headline raw_archive bloat but left four adjacent leaks on the same pollution chain: - archive() success path appended uncapped LLM summaries to history.jsonl, so a misbehaving LLM could re-open the #3412 bug from the happy path. - maybe_consolidate_by_tokens did not advance last_consolidated when archive() fell back to raw_archive, causing duplicate [RAW] dumps of the same chunk on every subsequent call. - Dream's Phase 1/2 prompt injected MEMORY.md / SOUL.md / USER.md and each history entry without caps, so any legacy oversized record (or an unbounded user edit) would blow past the context window every dream. - append_history itself had no default cap, leaving future new callers one forgotten-cap-away from the same vector. Changes: - Cap LLM-produced summaries at 8K chars (_ARCHIVE_SUMMARY_MAX_CHARS) before writing to history.jsonl. - Advance session.last_consolidated after archive() regardless of whether it summarized or raw-archived — both outcomes materialize the chunk; still break the round loop on fallback so a degraded LLM isn't hammered. - Truncate MEMORY.md / SOUL.md / USER.md and each history entry in Dream's Phase 1 prompt preview (Phase 2 still reaches full files via read_file). - Add _HISTORY_ENTRY_HARD_CAP (64K) as belt-and-suspenders default in append_history with a once-per-store warning, so any new caller that forgets its own tighter cap gets caught and observable. Layer the caps by scope: raw_archive=16K, archive summary=8K, append_history default=64K. Tight per-caller values cover expected payloads; the wide default only catches regressions. Tests: +9 regression tests covering each fix. Full suite: 2372 passed. Made-with: Cursor
This commit is contained in:
@@ -5,7 +5,7 @@ from datetime import datetime
|
||||
|
||||
import pytest
|
||||
|
||||
from nanobot.agent.memory import MemoryStore
|
||||
from nanobot.agent.memory import MemoryStore, _HISTORY_ENTRY_HARD_CAP
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@@ -142,6 +142,49 @@ class TestHistoryWithCursor:
|
||||
assert entries[0]["cursor"] in {4, 5}
|
||||
|
||||
|
||||
class TestAppendHistoryHardCap:
|
||||
"""append_history has a defensive cap that catches new callers who forgot
|
||||
to set their own tighter cap. The default is intentionally larger than
|
||||
any current caller's per-call cap, so normal operation never trips it."""
|
||||
|
||||
def test_oversized_entry_is_truncated(self, store):
|
||||
"""An entry above _HISTORY_ENTRY_HARD_CAP is truncated before being persisted."""
|
||||
huge = "x" * (_HISTORY_ENTRY_HARD_CAP + 10_000)
|
||||
store.append_history(huge)
|
||||
entry = store.read_unprocessed_history(since_cursor=0)[0]
|
||||
assert len(entry["content"]) <= _HISTORY_ENTRY_HARD_CAP + 50
|
||||
|
||||
def test_oversize_warning_is_emitted_once(self, store, caplog):
|
||||
"""Repeated oversized writes should warn only on the first occurrence."""
|
||||
from loguru import logger as loguru_logger
|
||||
|
||||
records: list[str] = []
|
||||
handler_id = loguru_logger.add(lambda m: records.append(m), level="WARNING")
|
||||
try:
|
||||
huge = "x" * (_HISTORY_ENTRY_HARD_CAP + 1)
|
||||
store.append_history(huge)
|
||||
store.append_history(huge)
|
||||
store.append_history(huge)
|
||||
finally:
|
||||
loguru_logger.remove(handler_id)
|
||||
|
||||
oversize_warnings = [r for r in records if "exceeds" in r and "chars" in r]
|
||||
assert len(oversize_warnings) == 1
|
||||
|
||||
def test_custom_max_chars_overrides_default(self, store):
|
||||
"""Callers that pass max_chars should get their tighter cap applied."""
|
||||
store.append_history("a" * 500, max_chars=100)
|
||||
entry = store.read_unprocessed_history(since_cursor=0)[0]
|
||||
assert len(entry["content"]) <= 150 # 100 + "\n... (truncated)"
|
||||
|
||||
def test_normal_sized_entries_unaffected(self, store):
|
||||
"""The hard cap must not alter entries that fit within it."""
|
||||
msg = "normal short entry"
|
||||
store.append_history(msg)
|
||||
entry = store.read_unprocessed_history(since_cursor=0)[0]
|
||||
assert entry["content"] == msg
|
||||
|
||||
|
||||
class TestDreamCursor:
|
||||
def test_initial_cursor_is_zero(self, store):
|
||||
assert store.get_last_dream_cursor() == 0
|
||||
|
||||
Reference in New Issue
Block a user