Grayscale/nanobot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(agent): retain recent context during auto compact

Browse Source 
Keep a legal recent suffix in idle auto-compacted sessions so resumed chats preserve their freshest live context while older messages are summarized. Recover persisted summaries even when retained messages remain, and document the new behavior.
This commit is contained in:
Xubin Ren
2026-04-11 15:56:41 +08:00
committed by Xubin Ren
parent d03458f034
commit 1cb28b39a3
3 changed files with 104 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+3 -3
View File
@@ -1505,7 +1505,7 @@ MCP tools are automatically discovered and registered on startup. The LLM can us
### Auto Compact
When a user is idle for longer than a configured TTL, nanobot **proactively** compresses the session context into a summary. This reduces token cost and first-token latency when the user returns — instead of re-processing a long stale context with an expired KV cache, the model receives a compact summary and fresh input.
When a user is idle for longer than a configured TTL, nanobot **proactively** compresses the older part of the session context into a summary while keeping a recent legal suffix of live messages. This reduces token cost and first-token latency when the user returns — instead of re-processing a long stale context with an expired KV cache, the model receives a compact summary, the most recent live context, and fresh input.
```json
{
@@ -1523,8 +1523,8 @@ When a user is idle for longer than a configured TTL, nanobot **proactively** co
How it works:
1. **Idle detection**: On each idle tick (~1 s), checks all sessions for expiration.
2. **Background compaction**: Expired sessions are summarized via LLM, then cleared.
3. **Summary injection**: When the user returns, the summary is injected as runtime context (one-shot, not persisted).
2. **Background compaction**: Expired sessions summarize the older live prefix via LLM and keep the most recent legal suffix (currently 8 messages).
3. **Summary injection**: When the user returns, the summary is injected as runtime context (one-shot, not persisted) alongside the retained recent suffix.
> [!TIP]
> The summary survives bot restarts — it's stored in session metadata and recovered on the next message.