feat(runner): add model failover with fallback_models
When the primary model returns a non-transient error and no content has been streamed yet, the runner now tries each model listed in the active preset's fallback_models in order. Each fallback model may reside on a different provider — a temporary provider instance is created on-the-fly via make_provider(config, model=...). Key design: - Failover is request-scoped (does not affect subagents/dream/consolidator) - Provider is restored via try/finally after each fallback attempt - Skipped when content was already streamed to avoid duplicate output - Recursive failover prevented by clearing fallback_models on fallback spec - Circuit breaker trips open after 3 consecutive primary failures (60s cooldown) - Cross-provider routing: fallback model prefix (e.g. groq/) determines provider Fixes: cross-provider fallback was broken because the factory passed the original preset (with provider forced to primary's provider) when creating fallback providers. Now uses provider="auto" so the model string prefix correctly routes to the right provider. Also fixes: log messages now distinguish between primary-failed, previous-fallback-failed, and circuit-open scenarios. closes: https://github.com/HKUDS/nanobot/issues/3376
This commit is contained in:
@@ -82,6 +82,7 @@ class ModelPresetConfig(Base):
|
||||
context_window_tokens: int = 65_536
|
||||
temperature: float = 0.1
|
||||
reasoning_effort: str | None = None
|
||||
fallback_models: list[str] = Field(default_factory=list)
|
||||
|
||||
def to_generation_settings(self) -> Any:
|
||||
from nanobot.providers.base import GenerationSettings
|
||||
|
||||
Reference in New Issue
Block a user