Add _QWEN_THINKING_MODELS to _MODEL_THINKING_STYLES with enable_thinking style. Prevents Qwen 3.5/3.6/3.7 models from exposing raw reasoning content in chat responses. Closes#4934
Symptom
-------
LLM requests intermittently fail with:
'utf-8' codec can't encode characters in position N-N+1: surrogates not allowed
when messages contain emoji-heavy content (e.g. HTML with mixed emoji + JSON round-trips).
This blocks the affected session until the session file is quarantined.
Root cause
----------
Surrogate sanitization was only applied at the CLI entry point
(nanobot/cli/commands.py: _sanitize_surrogates). Requests entering
the LLM provider layer through other channels (Feishu, cron, webui,
tool results, memory injection) had no defensive cleaning, so any
message that happened to carry unpaired UTF-16 surrogates (from an
upstream JSON round-trip with ensure_ascii=True on ill-formed input,
memory rehydration, or third-party content) would blow up at
json.dumps -> HTTP encode time inside the provider client.
Fix
---
1. Extract sanitize_surrogates() and sanitize_surrogates_deep() into
nanobot/utils/helpers.py as the single source of truth. Both use
utf-16-le round-tripping with errors='surrogatepass' / 'replace',
so paired surrogates reconstruct back into their real code point
and lone surrogates collapse to U+FFFD.
2. Make nanobot/cli/commands.py:_sanitize_surrogates a thin wrapper
that re-exports the shared helper (backward compatible).
3. Add defense-in-depth at the LLM provider boundary in
nanobot/providers/base.py:_sanitize_empty_content by running
sanitize_surrogates_deep over each message and its content blocks
right before requests are serialized to JSON.
Non-goals
---------
- truncate_text() is intentionally left untouched. Python str slicing
cannot split a single code point into surrogate halves, so it is
not the source of lone surrogates.
- session/manager storage layer is untouched. Archived sessions
reproduced the failure only through the request path, not through
storage.
Verification
------------
- New regression suite tests/providers/test_sanitize_surrogates.py
covers: paired surrogate reconstruction, lone surrogate replacement,
identity return on clean input (zero allocation), deep recursion on
dict/list/tuple, provider _sanitize_empty_content integration, and
full utf-8 encodability of the sanitized request body.
- 14/14 new tests pass; full existing test module also green.
- Replayed 58 archived real session messages plus adversarial
lone-surrogate injection through the provider path with no encode
errors after the fix.
Impact
------
- No behaviour change for clean inputs (sanitize_surrogates_deep is
an identity return when no surrogate is present).
- Fails-safe: unpaired surrogates degrade to U+FFFD instead of
aborting the entire request.
jobs.json can store everyMs/atMs and next/last run timestamps as strings.
Loading left them as str, so _compute_next_run compared str to int and raised TypeError.
Coerce with an optional-int helper at from_store_dict, matching runHistory int() paths.
config.loader.load_config() intentionally returns the raw config with ${VAR}
references intact — env interpolation is a separate, explicit step
(resolve_config_env_vars) so that settings read/edit/save paths never
materialize secrets to disk or to the UI.
The transcription config path does not apply that step: both
channels/base.py (channel voice notes) and webui/transcription_ws.py (WebUI
recording) build their effective config via
resolve_transcription_config(load_config()). As a result a configured
api_key of "${GROQ_API_KEY}" (the documented way to reference secrets) is
passed to the provider verbatim, which fails with 401 Invalid API Key. No
amount of rotating the real key helps, because the literal placeholder
string is what gets sent.
Resolve the reference at the single choke point both callers share —
_resolve_transcription_api_key / _resolve_transcription_api_base — using a
new lenient loader.resolve_env_refs() helper (unset var -> empty string, so
a missing variable degrades to "not configured" rather than raising or
leaking). This fixes both entry points at once and cannot drift the way a
per-call-site fix does. Resolving inside load_config() was rejected: the
~20 settings-UI callers depend on it returning raw ${VAR} placeholders.
Literal keys are unaffected; the settings API only reads the derived
`configured` flag (never the key), which now reflects the resolved value.
Claude-Session: https://claude.ai/code/session_01Q3HuVaJAAQJA3kgVQVJ2Zt
save_config truncated config.json in place on crash mid-write.
Route through _write_text_atomic like the pairing store so a failed write leaves the prior file intact.
Maintainer edit: keep a top-level trailing '&' in the segment being matched so background execution cannot be checked as if the ampersand were absent. Redirection forms like 2>&1 and &> remain untouched.
Maintainer edit: single '&' backgrounds the preceding command and starts another top-level shell segment, so allowPatterns must split it the same way as ';', '|', '&&', and '||'. Keep fd redirections such as 2>&1 and &> intact.
Guard against shell-chain bypass where an attacker appends '&& malicious'
after an allowlisted prefix. The allowlist check now splits the command
on top-level chaining operators (&&, ||, ;, |) and requires every segment
to match at least one allowPattern independently.
Fixes#4521
Bind SessionManager saves to the existing raw archive path so SDK imports and other bypass saves cannot persist more than the file cap without archiving unconsolidated overflow.
Add an SDK regression test that exercises the real ingest path.
Refs #4787
The tool execution path caught BaseException, which includes
KeyboardInterrupt, SystemExit, MemoryError, and GeneratorExit.
These should never be caught and converted into conversational
error messages. CancelledError is already handled separately.
Change except BaseException to except Exception so fatal signals
propagate instead of being swallowed.
Adds parametrized regression test for KeyboardInterrupt and
SystemExit propagation.
Fixes#4788
The QQ channel's _run_bot() used a fixed 5-second reconnect interval with
no backoff. When the network is unavailable (e.g., DNS failure), this
produces excessive botpy SDK error tracebacks every 5 seconds, flooding
logs.
botpy's Client.bot_connect() catches ws_connect() exceptions internally
and calls BotWebSocket.on_error(), which logs a full traceback and
immediately re-queues the session. The outer _run_bot() except never
fires for the reported DNS failure path.
Override bot_connect() on the _Bot subclass to:
- Apply exponential backoff (5s -> 300s cap) before re-queuing the session
- Log network errors (ClientConnectorDNSError, ClientConnectorError,
OSError) compactly without traceback
- Reset backoff on successful connection
- Still call traceback.print_exc() for non-network errors
The outer _run_bot() loop retains exponential backoff as a fallback for
exceptions that escape start() entirely. The botpy library logging
redirect is elevated to ERROR to suppress redundant connection tracebacks.
Consistent with patterns already used in matrix.py and napcat.py.
Add 7 regression tests covering:
- DNS error applies backoff and re-queues session
- No traceback printed for network errors
- ClientConnectorError also triggers backoff
- Backoff doubles and caps at 300s
- Successful connection resets backoff
- Non-network errors still re-queue without backoff
- _is_network_error() classification
Fixes#4767
chengyongruandGitHub
axelray-devandchengyongru
chengyongruandchengyongru
Arthur K.andchengyongru
santhrealandchengyongru
Xubin Ren
yorkhellenandchengyongru
seteiroandGitHub
hamb1yandXubin Ren
yrkandXubin Ren
Kris LuandXubin Ren
Ben LenartsandXubin Ren
amplifierplusandchengyongru
michaelxerandXubin Ren
KDBandXubin Ren
sunpengcheng05andchengyongru
chengyongruandchengyongru
golaandchengyongru
Pei Futongandchengyongru