* fix(dream): ground commit messages and cursor advance in the real git diff
Dream consolidation could emit a /dream-log audit record that did not match
the actual file changes: build_dream_commit_message appended the LLM's
unverified resp.content, dream_run_completed only checked the stop reason, and
file contents were deliberately omitted from the prompt. The combination let a
single-turn self-report become the durable audit record.
- gitstore: add summarize_working_tree() — a structured, machine-derived
summary (per-file +N/-M, totals, capped unified diff) of working-tree
changes vs HEAD. Pure filesystem/git ground truth, never LLM narrative.
- memory: build_dream_commit_message now takes the diff body instead of resp;
dream_content_diff() exposes the real delta over SOUL/USER/MEMORY.md only
(excludes .dream_cursor so cursor writes aren't mistaken for edits);
build_dream_prompt embeds current file contents so the model edits reality,
not a stale mental model.
- builtin/cli: both Dream paths now compute the diff, gate cursor advance on
a non-empty delta (no-op runs no longer swallow history), and commit with
the diff-grounded message. Non-git workspaces fall back to the completion
check.
- dream.md: document that contents are embedded, and add a chain-of-
verification guardrail so the model's summary cannot claim unmade edits.
A regression test proves a lying resp.content never reaches the audit log
while the real diff does.
* fix(dream): mark non-UTF-8 memory files as binary in diff summary
Address review feedback (Q1 on PR #4673): summarize_working_tree read
working-tree files with errors="replace", which would emit U+FFFD
replacement chars into the audit record if a memory file ever held
invalid UTF-8 — misrepresenting the diff it is meant to make truthful.
Switch to errors="strict" and catch UnicodeDecodeError: a non-UTF-8
(or binary/corrupt) file is now recorded as "{path}: binary or
non-UTF-8 file changed" and omitted from the unified diff, so the
audit record stays honest. An empty diff block is also suppressed when
all changes are binary.
Adds a defensive regression test asserting no replacement char leaks.
maintainer edit: cron is also a trigger source, so keep the new CLI-delivered source explicitly named as local trigger across backend, WebUI, docs, and tests.
apply_final_call_ids iterated over all final tool calls, including
non-file-edit tools like read_file. The greedy path-match in
matches_final_tool_call could overwrite a correct unique id with a
stale one from a different streaming state, producing duplicate
tool_use ids that poison the persisted session.
Guard the loop with is_file_edit_tool() so only tracked file-edit
tools (write_file, edit_file, apply_patch) are subject to canonical
id remapping. Non-file-edit tools keep their authoritative id from
get_final_message().
Fixes#4595
Maintainer edit: make token truncation include the suffix within the budget and route the consolidator through the shared helper so recent-history and archive truncation keep the same semantics.
The recent-history section injected into the system prompt was capped by character count (_MAX_HISTORY_CHARS = 32_000). Characters are a poor proxy for tokens: ~32k chars of English is ~8k tokens, but the same char count of CJK text or code can be far more, so the cap could let the section blow well past its intended size on non English/code-heavy histories.
Add a reusable truncate_text_to_tokens() helper (reusing the tiktoken
cl100k_base encoder already used elsewhere, with a char-based fallback) and
switch the digest cap to a token budget (_MAX_HISTORY_TOKENS = 8_000),
matching the previous English-text size while holding regardless of
content.
Move the fenced-code-block-aware splitting logic out of the shared
split_message helper (used by Signal, Slack, Discord, Weixin, etc.)
and into a Telegram-specific _split_telegram_markdown function.
The shared split_message remains a plain-text chunker. The Telegram
channel now uses _split_telegram_markdown for its raw Markdown paths
that feed _markdown_to_telegram_html, preventing broken HTML rendering
when splits fall inside fenced code blocks.
Also fixes a regression where content beginning with whitespace before
a fence could emit a whitespace-only chunk.
Addresses review feedback on #4257.
When split_message splits a long message, it now checks whether the
split point falls inside a fenced code block. If so, it either moves
the split to before the opening fence or closes/reopens the fence
across chunks, preventing broken HTML rendering.
Addresses #4250
Drop the legacy unified-diff patch parameter and all related parsing/
generation logic (_parse_patch, _generate_patch, _apply_hunks, etc.).
The tool now accepts only the structured `edits` array, eliminating the
intermediate diff-string round-trip.
Also update file_edit_events tracking and tests to work exclusively
with edits.
Benchmark (zhipu glm-5.1, edits mode): 15/15 cases passed.
chengyongruandGitHub
chengyongruandchengyongru
Xubin Ren
chengyongruandXubin Ren
Kenneth ZhaoandGitHub
yorkhellenandXubin Ren
axelray-devandXubin Ren
ZhouandXubin Ren
yu-xin-candXubin Ren
w.antarandXubin Ren
Bayern4ever-dotandXubin Ren