When the primary model (e.g. DeepSeek during peak hours) returns an empty
choices response with HTTP 200, the error carries no status code or
structured error metadata. The existing _FALLBACK_ERROR_TOKENS had no
matching token, so _should_fallback() returned False and fallback models
were never tried.
Changes:
- Add 'empty' token to _FALLBACK_ERROR_TOKENS so 'Error: API returned
empty choices.' text matches the fallback path
- Set error_kind='empty' in openai_compat_provider when returning
the empty-choices error, making the classification explicit
- Add test coverage for both text-only and error_kind matching paths
Fixes: glebov reported primary never falls back when DeepSeek returns
empty responses
maintainer edit: streamed timeout recovery was returning the retried response internally while the channel still treated the final outbound as already streamed. End the current stream segment before retry/fallback recovery so subsequent deltas are delivered in a new segment.
When a stream stalls mid-response, both the retry layer and
FallbackProvider blocked recovery because content had already been
emitted via on_content_delta. This left users with truncated replies
and no automatic recovery.
For error_kind="timeout" specifically:
- _run_with_retry now suppresses delta callbacks and retries the same
model instead of returning immediately
- FallbackProvider now allows failover to a different model with
delta callbacks suppressed
Non-timeout errors retain the original "skip retry/failover after
streamed content" behavior to avoid duplicate output.
Adds ProviderConfig.extra_query, threaded into AsyncOpenAI(default_query)
so that Azure-style gateways requiring query params like api-version can
be configured without URL hacks.
Also updates provider_signature to track extra_query changes so per-turn
refresh rebuilds the provider when the value changes.
Addresses the extra_query portion of #4204. The max_completion_tokens
model-awareness enhancement is intentionally left separate.
Resolve fallbackModels as preset references or explicit inline provider configs so failover uses complete model settings without exposing fallback logic to the agent loop.
Co-authored-by: Cursor <cursoragent@cursor.com>
Bind fallback model chains to the active model configuration so defaults and presets do not inherit or merge fallback behavior implicitly. Require explicit fallback providers while preserving per-fallback generation overrides and context-window safety.
Co-authored-by: Cursor <cursoragent@cursor.com>
When the primary model returns a non-transient error and no content
has been streamed yet, the runner now tries each model listed in the
active preset's fallback_models in order. Each fallback model may
reside on a different provider — a temporary provider instance is
created on-the-fly via make_provider(config, model=...).
Key design:
- Failover is request-scoped (does not affect subagents/dream/consolidator)
- Provider is restored via try/finally after each fallback attempt
- Skipped when content was already streamed to avoid duplicate output
- Recursive failover prevented by clearing fallback_models on fallback spec
- Circuit breaker trips open after 3 consecutive primary failures (60s cooldown)
- Cross-provider routing: fallback model prefix (e.g. groq/) determines provider
Fixes: cross-provider fallback was broken because the factory passed the
original preset (with provider forced to primary's provider) when creating
fallback providers. Now uses provider="auto" so the model string prefix
correctly routes to the right provider.
Also fixes: log messages now distinguish between primary-failed,
previous-fallback-failed, and circuit-open scenarios.
closes: https://github.com/HKUDS/nanobot/issues/3376
chengyongruandchengyongru
nanobot-contributorandXubin Ren
aiguozhi123456andXubin Ren
axelray-devandXubin Ren
outlook84andXubin Ren
Xubin RenandCursor