feat(agent): make model presets session-scoped (#4866)

This commit is contained in:
chengyongru
2026-07-23 00:38:49 +08:00
committed by GitHub
parent 66690fdb0c
commit c22efb5f7a
41 changed files with 1140 additions and 155 deletions
+3 -3
View File
@@ -9,7 +9,7 @@ These commands work inside chat channels and interactive agent sessions:
| `/restart` | Restart the bot | | `/restart` | Restart the bot |
| `/status` | Show bot status | | `/status` | Show bot status |
| `/model` | Show the current model and available model presets | | `/model` | Show the current model and available model presets |
| `/model <preset>` | Switch the runtime model preset for future turns | | `/model <preset>` | Switch and persist the model preset for the current session |
| `/dream` | Run Dream memory consolidation now | | `/dream` | Run Dream memory consolidation now |
| `/dream-log` | Show the latest Dream memory change | | `/dream-log` | Show the latest Dream memory change |
| `/dream-log <sha>` | Show a specific Dream memory change | | `/dream-log <sha>` | Show a specific Dream memory change |
@@ -47,7 +47,7 @@ Use `/model` to inspect the current runtime model:
/model /model
``` ```
The response shows the current model, the current preset, and the available preset names. Named presets come from the top-level `modelPresets` config and are the recommended way to configure model choices. `default` is always available and represents the model settings from direct `agents.defaults.*` fields. The response shows the current session's model and preset, plus the available preset names. Named presets come from the top-level `modelPresets` config and are the recommended way to configure model choices. `default` is always available and represents the model settings from direct `agents.defaults.*` fields.
To switch presets for future turns: To switch presets for future turns:
@@ -57,7 +57,7 @@ To switch presets for future turns:
/model default /model default
``` ```
Preset names come from the top-level `modelPresets` config. Switching is runtime-only: it does not rewrite `config.json`, and an in-progress turn keeps using the model it started with. See [Configuration: Model presets](./configuration.md#model-presets) for setup details. Preset names come from the top-level `modelPresets` config. Switching affects only the current session and persists the selection in that session, so later turns keep using it across process restarts. It does not rewrite `config.json`, does not change other sessions, and does not alter an in-progress turn's captured model. Sessions without a saved selection follow `agents.defaults.modelPreset` (or the implicit `default` preset when it is omitted). See [Configuration: Model presets](./configuration.md#model-presets) for setup details.
## Local triggers ## Local triggers
+2 -2
View File
@@ -1298,7 +1298,7 @@ Contributor notes for adding new providers live in [`development.md`](./developm
## Model Presets ## Model Presets
Model presets let you name a complete model configuration and switch it at runtime with `/model <preset>`. They are the recommended way to configure models because the same names can be reused for startup selection, chat-command switching, and fallback chains. Model presets let you name a complete model configuration and select one per session with `/model <preset>`. They are the recommended way to configure models because the same names can be reused for new-session defaults, chat-command switching, and fallback chains.
Existing configs do not need to change. Direct `agents.defaults.model`, `provider`, `maxTokens`, `contextWindowTokens`, `temperature`, and `reasoningEffort` fields still define the implicit `default` preset. For new configs, prefer top-level `modelPresets` plus `agents.defaults.modelPreset`. Existing configs do not need to change. Direct `agents.defaults.model`, `provider`, `maxTokens`, `contextWindowTokens`, `temperature`, and `reasoningEffort` fields still define the implicit `default` preset. For new configs, prefer top-level `modelPresets` plus `agents.defaults.modelPreset`.
@@ -1362,7 +1362,7 @@ Existing configs do not need to change. Direct `agents.defaults.model`, `provide
`default` is reserved and always means the implicit preset built from direct `agents.defaults.*` fields; do not define `modelPresets.default`. Use `/model default` to switch back to those direct fields in an existing config. `default` is reserved and always means the implicit preset built from direct `agents.defaults.*` fields; do not define `modelPresets.default`. Use `/model default` to switch back to those direct fields in an existing config.
Set `agents.defaults.modelPreset` to choose the startup preset. When `modelPreset` is `null` or omitted, startup uses the implicit `default` preset from direct `agents.defaults.*` fields. Runtime changes made with `/model <preset>` are not written back to `config.json`; they affect future turns until the process restarts or another model/config change replaces them. Set `agents.defaults.modelPreset` to choose the preset followed by sessions that have no saved model selection. When `modelPreset` is `null` or omitted, such sessions follow the implicit `default` preset from direct `agents.defaults.*` fields. `/model <preset>` saves an override in the current session, so its future turns keep that preset across process restarts while other sessions remain unchanged. The command does not write the selection back to `config.json`.
### Model Fallbacks ### Model Fallbacks
+16 -15
View File
@@ -27,7 +27,8 @@ To allow the agent to set its configuration (e.g. switch models, adjust paramete
Legacy `tools.myEnabled` / `tools.mySet` keys are auto-migrated on load, and rewritten in-place the next time `nanobot onboard` refreshes the config. Legacy `tools.myEnabled` / `tools.mySet` keys are auto-migrated on load, and rewritten in-place the next time `nanobot onboard` refreshes the config.
All modifications are held in memory only — restart restores defaults. Most modifications are held in memory only. `model_preset` is the exception: it is
stored in the current session so the selection survives a restart.
--- ---
@@ -77,20 +78,18 @@ my(action="check", key="web_config.enable")
## set — Runtime tuning ## set — Runtime tuning
Changes take effect immediately, no restart required. Changes do not require a restart. `model_preset` is saved for the current session and
applies to its next turn; other writable runtime tuning takes effect immediately.
Direct `model` and `context_window_tokens` writes are rejected during an active session
because those setters change the shared instance default. Configure a named preset for
model or context-window changes instead.
```text ```text
my(action="set", key="max_iterations", value=80) my(action="set", key="max_iterations", value=80)
# → Bump iteration limit from 40 to 80 # → Bump iteration limit from 40 to 80
my(action="set", key="model_preset", value="fast") my(action="set", key="model_preset", value="fast")
# → Switch to a configured model preset # → Use a configured model preset for this session's next turn
my(action="set", key="model", value="fast-model")
# → Switch to a raw model and clear the active preset
my(action="set", key="context_window_tokens", value=262144)
# → Expand context window for long documents
``` ```
You can also store custom state in your scratchpad: You can also store custom state in your scratchpad:
@@ -109,9 +108,9 @@ These parameters have type and range validation — invalid values are rejected:
| Parameter | Type | Range | Purpose | | Parameter | Type | Range | Purpose |
|-----------|------|-------|---------| |-----------|------|-------|---------|
| `max_iterations` | int | 1100 | Max tool calls per conversation turn | | `max_iterations` | int | 1100 | Max tool calls per conversation turn |
| `context_window_tokens` | int | 4,0961,000,000 | Context window size | | `context_window_tokens` | int | 4,0961,000,000 | Instance default; during a session, select through a preset |
| `model` | str | non-empty | LLM model to use | | `model` | str | non-empty | Instance default; during a session, select through a preset |
| `model_preset` | str | configured preset name | Named preset to use | | `model_preset` | str | configured preset name | Current session's preset for its next turn |
Other parameters (e.g. `workspace`, `provider_retry_mode`, `max_tool_result_chars`) can be set freely, as long as the value is JSON-safe. Other parameters (e.g. `workspace`, `provider_retry_mode`, `max_tool_result_chars`) can be set freely, as long as the value is JSON-safe.
@@ -122,8 +121,8 @@ Other parameters (e.g. `workspace`, `provider_retry_mode`, `max_tool_result_char
### "This task is complex, I need more room" ### "This task is complex, I need more room"
```text ```text
Agent: This codebase is large, let me expand my context window to handle it. Agent: This codebase is large, let me switch this session to the configured deep preset.
→ my(action="set", key="context_window_tokens", value=262144) → my(action="set", key="model_preset", value="deep")
``` ```
### "Simple question, don't waste compute" ### "Simple question, don't waste compute"
@@ -180,7 +179,9 @@ Agent: The code review is progressing well. The test task hasn't started yet.
## Safety Mechanisms ## Safety Mechanisms
Core design principle: **All modifications live in memory only. Restart restores defaults.** The agent cannot cause persistent damage. Core design principle: **The tool does not rewrite `config.json`.** Instance-wide
changes live in memory only, while `model_preset` persists only as the current
session's selector.
### Off-limits (BLOCKED) ### Off-limits (BLOCKED)
+3 -1
View File
@@ -610,7 +610,9 @@ In chat:
/model fast /model fast
``` ```
`/model` switching is runtime-only. It does not rewrite `config.json`, and an in-progress turn keeps using the model it started with. `/model` stores the selection in the current session without rewriting `config.json`.
The selection survives restarts, does not affect other sessions, and an in-progress
turn keeps using the model it started with.
## Quick Failure Map ## Quick Failure Map
+7 -5
View File
@@ -494,8 +494,10 @@ Run the agent once and return a `RunResult`.
| `model` | `str \| None` | `None` | Override the model for this run only. | | `model` | `str \| None` | `None` | Override the model for this run only. |
| `model_preset` | `str \| None` | `None` | Override the model preset for this run only. | | `model_preset` | `str \| None` | `None` | Override the model preset for this run only. |
`model` and `model_preset` are per-run overrides and do not change Without an override, a run uses the preset saved in its session, or the configured
`bot.runtime.model` after the run completes. They are mutually exclusive. default when that session has no saved selection. `model` and `model_preset` are
mutually exclusive per-run overrides; they do not change the saved session selection
or `bot.runtime.model` after the run completes.
### `await bot.run_streamed(...)` ### `await bot.run_streamed(...)`
@@ -531,9 +533,9 @@ async for event in bot.stream("Generate a long answer"):
| `await cancel()` | Cancel the run and release stream resources. | | `await cancel()` | Cancel the run and release stream resources. |
| `await aclose()` | Close the stream; equivalent cleanup primitive for `async with` / manual lifecycle code. | | `await aclose()` | Close the stream; equivalent cleanup primitive for `async with` / manual lifecycle code. |
Normal SDK runs with different session keys may overlap. Runs that use per-run SDK runs with different session keys may overlap, including runs with per-run
`model` or `model_preset` overrides are exclusive while the override is active, `model` or `model_preset` overrides. Each run receives an immutable runtime without
because the current `AgentLoop` provider/model state is mutable. mutating the instance default. Runs sharing one session key remain serialized.
### `StreamEvent` ### `StreamEvent`
+6 -2
View File
@@ -152,7 +152,8 @@ All frames are JSON text. Each message has an `event` field.
Reasoning frames only flow when the channel's `showReasoning` is `true` (default) and the model returns reasoning content (DeepSeek-R1 / Kimi / MiMo / OpenAI reasoning models, Anthropic extended thinking, or inline `<think>` / `<thought>` tags). Models without reasoning produce zero `reasoning_delta` frames. Reasoning frames only flow when the channel's `showReasoning` is `true` (default) and the model returns reasoning content (DeepSeek-R1 / Kimi / MiMo / OpenAI reasoning models, Anthropic extended thinking, or inline `<think>` / `<thought>` tags). Models without reasoning produce zero `reasoning_delta` frames.
**`runtime_model_updated`** — broadcast when the gateway runtime model changes, for example after `/model <preset>`: **`runtime_model_updated`** — broadcast when the gateway default runtime changes or
when a config reload requires clients to refresh their model catalog:
```json ```json
{ {
@@ -162,7 +163,10 @@ Reasoning frames only flow when the channel's `showReasoning` is `true` (default
} }
``` ```
`model_preset` is omitted when no named preset is active. WebUI clients use this event to keep the displayed model badge in sync across slash commands, config reloads, and settings changes. `model_preset` is omitted when no named preset is active. WebUI clients use this event
to refresh model settings after default-runtime and config changes. `/model <preset>`
is session-scoped; its selection is reflected through `session_updated` and the
session row's `model_preset` field instead of this global event.
**`attached`** — confirmation for `new_chat` / `attach` inbound envelopes (see [Multi-chat multiplexing](#multi-chat-multiplexing)): **`attached`** — confirmation for `new_chat` / `attach` inbound envelopes (see [Multi-chat multiplexing](#multi-chat-multiplexing)):
+7 -2
View File
@@ -66,7 +66,7 @@ class AutoCompact:
def check_expired( def check_expired(
self, self,
schedule_background: Callable[[Coroutine], None], schedule_background: Callable[[Coroutine], None],
resolve_runtime: Callable[[], LLMRuntime], resolve_runtime: Callable[[Session], LLMRuntime],
active_session_keys: Collection[str] = (), active_session_keys: Collection[str] = (),
) -> None: ) -> None:
"""Schedule archival for idle sessions, skipping those with in-flight agent tasks.""" """Schedule archival for idle sessions, skipping those with in-flight agent tasks."""
@@ -79,7 +79,12 @@ class AutoCompact:
continue continue
updated_at = info.get("updated_at") updated_at = info.get("updated_at")
if self._is_expired(updated_at, now) and self._has_compactable_idle_tail(key): if self._is_expired(updated_at, now) and self._has_compactable_idle_tail(key):
runtime = resolve_runtime() session = self.sessions.get_or_create(key)
try:
runtime = resolve_runtime(session)
except (KeyError, ValueError):
# Invalid session selections remain recoverable through /model.
continue
self._archiving.add(key) self._archiving.add(key)
schedule_background(self._archive(key, runtime=runtime)) schedule_background(self._archive(key, runtime=runtime))
+46 -8
View File
@@ -79,6 +79,10 @@ from nanobot.session.manager import (
SessionManager, SessionManager,
replay_max_messages_for_context, replay_max_messages_for_context,
) )
from nanobot.session.model_selection import (
SESSION_MODEL_PRESET_METADATA_KEY,
model_preset_from_metadata,
)
from nanobot.triggers.local_turns import LocalTriggerTurnCoordinator from nanobot.triggers.local_turns import LocalTriggerTurnCoordinator
from nanobot.utils.cancellation import task_is_cancelling from nanobot.utils.cancellation import task_is_cancelling
from nanobot.utils.document import extract_documents, reference_non_image_attachments from nanobot.utils.document import extract_documents, reference_non_image_attachments
@@ -129,7 +133,7 @@ class TurnContext:
session_key: str session_key: str
state: TurnState state: TurnState
turn_id: str turn_id: str
runtime: LLMRuntime runtime: LLMRuntime | None
kind: TurnKind kind: TurnKind
delivery: TurnDelivery delivery: TurnDelivery
original_user_text: str | None = None original_user_text: str | None = None
@@ -155,6 +159,7 @@ class TurnContext:
on_progress: Callable[..., Awaitable[None]] | None = None on_progress: Callable[..., Awaitable[None]] | None = None
on_stream: Callable[[str], Awaitable[None]] | None = None on_stream: Callable[[str], Awaitable[None]] | None = None
on_stream_end: Callable[..., Awaitable[None]] | None = None on_stream_end: Callable[..., Awaitable[None]] | None = None
on_runtime_admitted: Callable[[LLMRuntime], Awaitable[None]] | None = None
on_retry_wait: Callable[[str], Awaitable[None]] | None = None on_retry_wait: Callable[[str], Awaitable[None]] | None = None
pending_queue: asyncio.Queue | None = None pending_queue: asyncio.Queue | None = None
@@ -282,6 +287,7 @@ class AgentLoop:
provider_snapshot_loader: Callable[..., ProviderSnapshot] | None = None, provider_snapshot_loader: Callable[..., ProviderSnapshot] | None = None,
provider_signature: tuple[object, ...] | None = None, provider_signature: tuple[object, ...] | None = None,
model_presets: dict[str, ModelPresetConfig] | None = None, model_presets: dict[str, ModelPresetConfig] | None = None,
preset_catalog_loader: preset_helpers.PresetCatalogLoader | None = None,
model_preset: str | None = None, model_preset: str | None = None,
preset_snapshot_loader: preset_helpers.PresetSnapshotLoader | None = None, preset_snapshot_loader: preset_helpers.PresetSnapshotLoader | None = None,
runtime_events: RuntimeEventBus | None = None, runtime_events: RuntimeEventBus | None = None,
@@ -331,6 +337,8 @@ class AgentLoop:
snapshot_signature=provider_signature, snapshot_signature=provider_signature,
), ),
model_presets=configured_presets, model_presets=configured_presets,
preset_catalog_loader=preset_catalog_loader,
configured_default_preset=model_preset,
provider_snapshot_loader=provider_snapshot_loader, provider_snapshot_loader=provider_snapshot_loader,
preset_snapshot_loader=preset_snapshot_loader, preset_snapshot_loader=preset_snapshot_loader,
) )
@@ -504,6 +512,28 @@ class AgentLoop:
"""Keep subagent runtime limits aligned with mutable loop settings.""" """Keep subagent runtime limits aligned with mutable loop settings."""
self.subagents.max_iterations = self.max_iterations self.subagents.max_iterations = self.max_iterations
def invalidate_runtime_config(self) -> None:
"""Invalidate runtime config and notify clients to refresh its catalog."""
self.runtime_resolver.invalidate()
self._publish_runtime_selection(self.runtime_resolver.runtime)
def runtime_for_session(self, session: Session) -> LLMRuntime:
"""Resolve the immutable runtime selected by one session."""
name = model_preset_from_metadata(session.metadata)
return self.llm_runtime() if name is None else self.runtime_resolver.resolve_preset(name)
def set_session_model_preset(
self,
session_key: str,
name: str,
) -> LLMRuntime:
"""Validate and persist one session's preset selection."""
runtime = self.runtime_resolver.resolve_preset(name)
session = self.sessions.get_or_create(session_key)
session.metadata[SESSION_MODEL_PRESET_METADATA_KEY] = runtime.model_preset
self.sessions.save(session)
return runtime
def _publish_runtime_selection( def _publish_runtime_selection(
self, self,
runtime: LLMRuntime, runtime: LLMRuntime,
@@ -983,7 +1013,7 @@ class AgentLoop:
except asyncio.TimeoutError: except asyncio.TimeoutError:
self.auto_compact.check_expired( self.auto_compact.check_expired(
self._schedule_background, self._schedule_background,
self.llm_runtime, self.runtime_for_session,
active_session_keys=self._pending_queues.keys(), active_session_keys=self._pending_queues.keys(),
) )
continue continue
@@ -1224,11 +1254,9 @@ class AgentLoop:
tools: ToolRegistry | None = None, tools: ToolRegistry | None = None,
runtime: LLMRuntime | None = None, runtime: LLMRuntime | None = None,
delivery: TurnDelivery | None = None, delivery: TurnDelivery | None = None,
on_runtime_admitted: Callable[[LLMRuntime], Awaitable[None]] | None = None,
) -> OutboundMessage | None: ) -> OutboundMessage | None:
"""Process a single inbound message and return the response.""" """Process a single inbound message and return the response."""
if runtime is None:
runtime = self.llm_runtime()
kind = TurnKind.SYSTEM if msg.channel == "system" else TurnKind.USER kind = TurnKind.SYSTEM if msg.channel == "system" else TurnKind.USER
if kind is TurnKind.SYSTEM: if kind is TurnKind.SYSTEM:
destination = ( destination = (
@@ -1268,6 +1296,7 @@ class AgentLoop:
on_progress=on_progress, on_progress=on_progress,
on_stream=on_stream, on_stream=on_stream,
on_stream_end=on_stream_end, on_stream_end=on_stream_end,
on_runtime_admitted=on_runtime_admitted,
pending_queue=pending_queue, pending_queue=pending_queue,
ephemeral=ephemeral, ephemeral=ephemeral,
run_extra_hooks_for_ephemeral=run_extra_hooks_for_ephemeral, run_extra_hooks_for_ephemeral=run_extra_hooks_for_ephemeral,
@@ -1452,13 +1481,19 @@ class AgentLoop:
return "dispatch" return "dispatch"
async def _state_build(self, ctx: TurnContext) -> str: async def _state_build(self, ctx: TurnContext) -> str:
runtime = ctx.runtime
if runtime is None:
runtime = self.runtime_for_session(ctx.session)
ctx.runtime = runtime
if ctx.on_runtime_admitted is not None:
await ctx.on_runtime_admitted(runtime)
replay_max_messages = replay_max_messages_for_context( replay_max_messages = replay_max_messages_for_context(
ctx.runtime.context_window_tokens runtime.context_window_tokens
) )
if not ctx.ephemeral: if not ctx.ephemeral:
await self.consolidator.maybe_consolidate_by_tokens( await self.consolidator.maybe_consolidate_by_tokens(
ctx.session, ctx.session,
runtime=ctx.runtime, runtime=runtime,
replay_max_messages=replay_max_messages, replay_max_messages=replay_max_messages,
) )
is_subagent = ctx.kind is TurnKind.SYSTEM and ctx.msg.sender_id == "subagent" is_subagent = ctx.kind is TurnKind.SYSTEM and ctx.msg.sender_id == "subagent"
@@ -1469,7 +1504,7 @@ class AgentLoop:
_hist_kwargs: dict[str, Any] = { _hist_kwargs: dict[str, Any] = {
"max_messages": replay_max_messages, "max_messages": replay_max_messages,
"max_tokens": self._replay_token_budget(ctx.runtime), "max_tokens": self._replay_token_budget(runtime),
"extend_to_user": is_subagent, "extend_to_user": is_subagent,
} }
ctx.history = ctx.session.get_history(**_hist_kwargs) ctx.history = ctx.session.get_history(**_hist_kwargs)
@@ -1864,6 +1899,7 @@ class AgentLoop:
tools: ToolRegistry | None = None, tools: ToolRegistry | None = None,
persist_user_message: bool = True, persist_user_message: bool = True,
runtime: LLMRuntime | None = None, runtime: LLMRuntime | None = None,
on_runtime_admitted: Callable[[LLMRuntime], Awaitable[None]] | None = None,
) -> OutboundMessage | None: ) -> OutboundMessage | None:
"""Process an external message directly and return the outbound payload.""" """Process an external message directly and return the outbound payload."""
if channel == "system": if channel == "system":
@@ -1897,6 +1933,8 @@ class AgentLoop:
kwargs["tools"] = tools kwargs["tools"] = tools
if runtime is not None: if runtime is not None:
kwargs["runtime"] = runtime kwargs["runtime"] = runtime
if on_runtime_admitted is not None:
kwargs["on_runtime_admitted"] = on_runtime_admitted
return await self._process_message( return await self._process_message(
msg, msg,
**kwargs, **kwargs,
+22 -4
View File
@@ -2,7 +2,9 @@
from __future__ import annotations from __future__ import annotations
from collections.abc import Callable from collections.abc import Callable, Mapping
from dataclasses import replace
from pathlib import Path
from typing import Any from typing import Any
from nanobot.config.schema import ModelPresetConfig from nanobot.config.schema import ModelPresetConfig
@@ -10,16 +12,31 @@ from nanobot.providers.base import LLMProvider
from nanobot.providers.factory import ProviderSnapshot, build_provider_snapshot from nanobot.providers.factory import ProviderSnapshot, build_provider_snapshot
PresetSnapshotLoader = Callable[[str], ProviderSnapshot] PresetSnapshotLoader = Callable[[str], ProviderSnapshot]
PresetCatalogLoader = Callable[[], Mapping[str, ModelPresetConfig]]
def default_selection_signature(signature: tuple[object, ...] | None) -> tuple[object, ...] | None: def default_selection_signature(
return signature[:2] if signature else None signature: tuple[object, ...] | None,
model_preset: str | None = None,
) -> tuple[object, ...] | None:
return (model_preset, *signature[:2]) if signature else None
def configured_model_presets(config: Any) -> dict[str, ModelPresetConfig]: def configured_model_presets(config: Any) -> dict[str, ModelPresetConfig]:
return {**config.model_presets, "default": config.resolve_default_preset()} return {**config.model_presets, "default": config.resolve_default_preset()}
def load_model_preset_catalog(
config_path: Path | None = None,
) -> dict[str, ModelPresetConfig]:
"""Load the current preset catalog from the configured file."""
from nanobot.config.loader import load_config, resolve_config_env_vars
return configured_model_presets(
resolve_config_env_vars(load_config(config_path)),
)
def make_preset_snapshot_loader( def make_preset_snapshot_loader(
config: Any, config: Any,
provider_snapshot_loader: Callable[..., ProviderSnapshot] | None, provider_snapshot_loader: Callable[..., ProviderSnapshot] | None,
@@ -40,6 +57,7 @@ def build_static_preset_snapshot(
context_window_tokens=preset.context_window_tokens, context_window_tokens=preset.context_window_tokens,
signature=("model_preset", name, preset.model_dump_json()), signature=("model_preset", name, preset.model_dump_json()),
generation=preset.to_generation_settings(), generation=preset.to_generation_settings(),
model_preset=name,
) )
@@ -51,7 +69,7 @@ def build_runtime_preset_snapshot(
loader: PresetSnapshotLoader | None, loader: PresetSnapshotLoader | None,
) -> ProviderSnapshot: ) -> ProviderSnapshot:
if loader is not None: if loader is not None:
return loader(name) return replace(loader(name), model_preset=name)
return build_static_preset_snapshot(provider, name, presets[name]) return build_static_preset_snapshot(provider, name, presets[name])
+40 -14
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
from collections.abc import Callable, Mapping from collections.abc import Callable, Mapping
from dataclasses import replace from dataclasses import replace
from types import MappingProxyType
from nanobot.agent import model_presets as preset_helpers from nanobot.agent import model_presets as preset_helpers
from nanobot.config.schema import Config, ModelPresetConfig from nanobot.config.schema import Config, ModelPresetConfig
@@ -24,17 +25,23 @@ class ModelRuntimeResolver:
initial_runtime: LLMRuntime, initial_runtime: LLMRuntime,
*, *,
model_presets: Mapping[str, ModelPresetConfig] | None = None, model_presets: Mapping[str, ModelPresetConfig] | None = None,
preset_catalog_loader: preset_helpers.PresetCatalogLoader | None = None,
configured_default_preset: str | None = None,
provider_snapshot_loader: Callable[[], ProviderSnapshot] | None = None, provider_snapshot_loader: Callable[[], ProviderSnapshot] | None = None,
preset_snapshot_loader: preset_helpers.PresetSnapshotLoader | None = None, preset_snapshot_loader: preset_helpers.PresetSnapshotLoader | None = None,
) -> None: ) -> None:
self._runtime = initial_runtime self._runtime = initial_runtime
self._model_presets = dict(model_presets or {}) self._model_presets = dict(model_presets or {})
self._preset_catalog_loader = preset_catalog_loader
self._preset_catalog_refresh_required = False
self._provider_snapshot_loader = provider_snapshot_loader self._provider_snapshot_loader = provider_snapshot_loader
self._preset_snapshot_loader = preset_snapshot_loader self._preset_snapshot_loader = preset_snapshot_loader
self._refresh_required = False self._refresh_required = False
self._resolved_presets: dict[str, LLMRuntime] = {}
self._tracks_provider_generation = initial_runtime.model_preset is None self._tracks_provider_generation = initial_runtime.model_preset is None
self._default_selection_signature = preset_helpers.default_selection_signature( self._default_selection_signature = preset_helpers.default_selection_signature(
initial_runtime.snapshot_signature initial_runtime.snapshot_signature,
configured_default_preset,
) )
@property @property
@@ -44,7 +51,11 @@ class ModelRuntimeResolver:
@property @property
def model_presets(self) -> Mapping[str, ModelPresetConfig]: def model_presets(self) -> Mapping[str, ModelPresetConfig]:
return self._model_presets self._refresh_preset_catalog()
return MappingProxyType({
name: preset.model_copy(deep=True)
for name, preset in self._model_presets.items()
})
@property @property
def model_preset(self) -> str | None: def model_preset(self) -> str | None:
@@ -71,41 +82,53 @@ class ModelRuntimeResolver:
def invalidate(self) -> None: def invalidate(self) -> None:
"""Refresh configured runtime state on the next admission.""" """Refresh configured runtime state on the next admission."""
self._refresh_required = True self._refresh_required = True
self._preset_catalog_refresh_required = True
self._resolved_presets.clear()
def _refresh_preset_catalog(self) -> None:
if not self._preset_catalog_refresh_required:
return
if self._preset_catalog_loader is not None:
self._model_presets = dict(self._preset_catalog_loader())
self._preset_catalog_refresh_required = False
def resolve_snapshot( def resolve_snapshot(
self, self,
snapshot: ProviderSnapshot, snapshot: ProviderSnapshot,
*,
model_preset: str | None = None,
) -> LLMRuntime: ) -> LLMRuntime:
"""Resolve a factory snapshot without changing the selected default.""" """Resolve a factory snapshot without changing the selected default."""
return runtime_from_provider_snapshot(snapshot, model_preset=model_preset) return runtime_from_provider_snapshot(snapshot)
def adopt_snapshot( def adopt_snapshot(
self, self,
snapshot: ProviderSnapshot, snapshot: ProviderSnapshot,
*,
model_preset: str | None = None,
) -> LLMRuntime: ) -> LLMRuntime:
"""Select a snapshot as the default for future turns.""" """Select a snapshot as the default for future turns."""
runtime = self.resolve_snapshot(snapshot, model_preset=model_preset) runtime = self.resolve_snapshot(snapshot)
self._runtime = runtime self._runtime = runtime
self._tracks_provider_generation = model_preset is None self._tracks_provider_generation = runtime.model_preset is None
self._default_selection_signature = preset_helpers.default_selection_signature( self._default_selection_signature = preset_helpers.default_selection_signature(
runtime.snapshot_signature runtime.snapshot_signature,
runtime.model_preset,
) )
return runtime return runtime
def resolve_preset(self, name: str | None) -> LLMRuntime: def resolve_preset(self, name: str | None) -> LLMRuntime:
"""Resolve a named preset without changing the selected default.""" """Resolve a named preset without changing the selected default."""
self._refresh_preset_catalog()
normalized = preset_helpers.normalize_preset_name(name, self._model_presets) normalized = preset_helpers.normalize_preset_name(name, self._model_presets)
cached = self._resolved_presets.get(normalized)
if cached is not None:
return cached
snapshot = preset_helpers.build_runtime_preset_snapshot( snapshot = preset_helpers.build_runtime_preset_snapshot(
name=normalized, name=normalized,
presets=self._model_presets, presets=self._model_presets,
provider=self._runtime.provider, provider=self._runtime.provider,
loader=self._preset_snapshot_loader, loader=self._preset_snapshot_loader,
) )
return self.resolve_snapshot(snapshot, model_preset=normalized) runtime = self.resolve_snapshot(snapshot)
self._resolved_presets[normalized] = runtime
return runtime
def select_preset(self, name: str | None) -> LLMRuntime: def select_preset(self, name: str | None) -> LLMRuntime:
"""Select a named preset as the default for future turns.""" """Select a named preset as the default for future turns."""
@@ -161,13 +184,16 @@ class ModelRuntimeResolver:
self._refresh_required = False self._refresh_required = False
return None return None
self._resolved_presets.clear()
snapshot = self._provider_snapshot_loader() snapshot = self._provider_snapshot_loader()
default_selection = preset_helpers.default_selection_signature(snapshot.signature) default_selection = preset_helpers.default_selection_signature(
snapshot.signature,
snapshot.model_preset,
)
active_preset = self._runtime.model_preset active_preset = self._runtime.model_preset
if active_preset and self._default_selection_signature in (None, default_selection): if active_preset and self._default_selection_signature in (None, default_selection):
runtime = self.resolve_preset(active_preset) runtime = self.resolve_preset(active_preset)
else: else:
active_preset = None
runtime = self.resolve_snapshot(snapshot) runtime = self.resolve_snapshot(snapshot)
unchanged = ( unchanged = (
@@ -184,7 +210,7 @@ class ModelRuntimeResolver:
self._default_selection_signature, self._default_selection_signature,
) = ( ) = (
runtime, runtime,
active_preset is None, runtime.model_preset is None,
default_selection, default_selection,
) )
return runtime return runtime
+6
View File
@@ -60,5 +60,11 @@ class RuntimeState(Protocol):
def set_runtime_context_window(self, context_window_tokens: int) -> Any: ... def set_runtime_context_window(self, context_window_tokens: int) -> Any: ...
def set_session_model_preset(
self,
session_key: str,
name: str,
) -> Any: ...
@property @property
def model_preset(self) -> str | None: ... def model_preset(self) -> str | None: ...
+32 -6
View File
@@ -3,12 +3,13 @@
from __future__ import annotations from __future__ import annotations
import time import time
from collections.abc import Mapping
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
from loguru import logger from loguru import logger
from nanobot.agent.tools.base import Tool, ToolResult from nanobot.agent.tools.base import Tool, ToolResult
from nanobot.agent.tools.context import current_request_context from nanobot.agent.tools.context import current_request_context, current_request_session_key
from nanobot.agent.tools.runtime_state import RuntimeState from nanobot.agent.tools.runtime_state import RuntimeState
from nanobot.config_base import Base from nanobot.config_base import Base
@@ -76,6 +77,7 @@ class MyTool(Tool):
"_current_iteration", # updated by runner only "_current_iteration", # updated by runner only
"exec_config", # inspect allowed (e.g. check sandbox), modify blocked "exec_config", # inspect allowed (e.g. check sandbox), modify blocked
"web_config", # inspect allowed (e.g. check enable), modify blocked "web_config", # inspect allowed (e.g. check enable), modify blocked
"model_presets", # config-derived catalog; changes require config reload
"workspace_sandbox", # read-only view of workspace enforcement level "workspace_sandbox", # read-only view of workspace enforcement level
"request", # current message routing metadata "request", # current message routing metadata
}) })
@@ -146,6 +148,8 @@ class MyTool(Tool):
"max_iterations - _current_iteration = remaining iterations.\n" "max_iterations - _current_iteration = remaining iterations.\n"
"Current routing metadata is available read-only via request.channel, " "Current routing metadata is available read-only via request.channel, "
"request.chat_id, and request.sender_id.\n" "request.chat_id, and request.sender_id.\n"
"Use model_preset for session-scoped model or context changes; direct "
"model/context_window_tokens writes are disabled during active sessions.\n"
"Note: web_config and exec_config are readable but read-only.\n" "Note: web_config and exec_config are readable but read-only.\n"
"\n" "\n"
"When to use:\n" "When to use:\n"
@@ -210,11 +214,11 @@ class MyTool(Tool):
if part.lower() in self._SENSITIVE_NAMES: if part.lower() in self._SENSITIVE_NAMES:
return None, f"'{part}' is not accessible" return None, f"'{part}' is not accessible"
try: try:
if isinstance(obj, dict): if isinstance(obj, Mapping):
if part in obj: if part in obj:
obj = obj[part] obj = obj[part]
else: else:
return None, f"'{part}' not found in dict" return None, f"'{part}' not found in mapping"
else: else:
obj = getattr(obj, part) obj = getattr(obj, part)
except (KeyError, AttributeError) as e: except (KeyError, AttributeError) as e:
@@ -257,7 +261,7 @@ class MyTool(Tool):
# SubagentManager: delegate to its _task_statuses dict # SubagentManager: delegate to its _task_statuses dict
if hasattr(val, "_task_statuses") and isinstance(val._task_statuses, dict): if hasattr(val, "_task_statuses") and isinstance(val._task_statuses, dict):
return MyTool._format_value(val._task_statuses, key) return MyTool._format_value(val._task_statuses, key)
if isinstance(val, dict) and val and _is_subagent_status(next(iter(val.values()))): if isinstance(val, Mapping) and val and _is_subagent_status(next(iter(val.values()))):
prefix = f"{key}: " if key else "" prefix = f"{key}: " if key else ""
lines = [f"{prefix}{len(val)} subagent(s):"] lines = [f"{prefix}{len(val)} subagent(s):"]
for tid, st in val.items(): for tid, st in val.items():
@@ -270,8 +274,8 @@ class MyTool(Tool):
if isinstance(val, (str, int, float, bool, type(None))): if isinstance(val, (str, int, float, bool, type(None))):
r = repr(val) r = repr(val)
return f"{key}: {r}" if key else r return f"{key}: {r}" if key else r
# Dict — small: show content; large: show keys for dot-path navigation # Mapping — small: show content; large: show keys for dot-path navigation
if isinstance(val, dict): if isinstance(val, Mapping):
ks = list(val.keys()) ks = list(val.keys())
if not ks: if not ks:
return f"{key}: {{}}" if key else "{}" return f"{key}: {{}}" if key else "{}"
@@ -447,6 +451,23 @@ class MyTool(Tool):
if not isinstance(value, str) or not value.strip(): if not isinstance(value, str) or not value.strip():
return ToolResult.error("Error: 'model_preset' must be a non-empty string") return ToolResult.error("Error: 'model_preset' must be a non-empty string")
name = value.strip() name = value.strip()
session_key = current_request_session_key()
if session_key:
try:
runtime = self._runtime_state.set_session_model_preset(
session_key,
name,
)
except (KeyError, ValueError) as exc:
message = str(exc.args[0]) if exc.args else str(exc)
punctuation = "" if message.endswith((".", "!", "?")) else "."
return ToolResult.error(f"Error: {message}{punctuation}")
self._audit("modify", f"model_preset = {name!r}")
return (
f"Set model_preset = {name!r} for the next turn; "
f"model will be {runtime.model!r}; "
f"context_window_tokens will be {runtime.context_window_tokens!r}"
)
result = self._modify_free("model_preset", name) result = self._modify_free("model_preset", name)
if isinstance(result, ToolResult) and result.is_error: if isinstance(result, ToolResult) and result.is_error:
return result if result.endswith((".", "!", "?")) else ToolResult.error(f"{result}.") return result if result.endswith((".", "!", "?")) else ToolResult.error(f"{result}.")
@@ -472,6 +493,11 @@ class MyTool(Tool):
return ToolResult.error(f"Error: '{key}' must be <= {spec['max']}") return ToolResult.error(f"Error: '{key}' must be <= {spec['max']}")
if "min_len" in spec and len(str(value)) < spec["min_len"]: if "min_len" in spec and len(str(value)) < spec["min_len"]:
return ToolResult.error(f"Error: '{key}' must be at least {spec['min_len']} characters") return ToolResult.error(f"Error: '{key}' must be at least {spec['min_len']} characters")
if key in {"model", "context_window_tokens"} and current_request_session_key():
return ToolResult.error(
f"Error: direct '{key}' changes are instance-wide and disabled "
"during an active session; use a configured model_preset"
)
if key == "model": if key == "model":
self._runtime_state.set_runtime_model(value) self._runtime_state.set_runtime_model(value)
elif key == "context_window_tokens": elif key == "context_window_tokens":
@@ -2942,6 +2942,7 @@ def test_sessions_list_includes_active_run_started_at(monkeypatch) -> None:
"updated_at": "2026-05-19T10:01:00Z", "updated_at": "2026-05-19T10:01:00Z",
"title": "Running", "title": "Running",
"preview": "work", "preview": "work",
"model_preset": "fast",
"path": "/private/path", "path": "/private/path",
}, },
{ {
@@ -2978,6 +2979,7 @@ def test_sessions_list_includes_active_run_started_at(monkeypatch) -> None:
"updated_at": "2026-05-19T10:01:00Z", "updated_at": "2026-05-19T10:01:00Z",
"title": "Running", "title": "Running",
"preview": "work", "preview": "work",
"model_preset": "fast",
"run_started_at": 1_700_000_000.0, "run_started_at": 1_700_000_000.0,
} }
] ]
+3 -1
View File
@@ -1605,6 +1605,7 @@ def _run_gateway(
health_server_enabled: bool = True, health_server_enabled: bool = True,
) -> None: ) -> None:
"""Shared gateway runtime; ``open_browser_url`` opens a tab once channels are up.""" """Shared gateway runtime; ``open_browser_url`` opens a tab once channels are up."""
from nanobot.agent.model_presets import load_model_preset_catalog
from nanobot.agent.tools.message import MessageTool from nanobot.agent.tools.message import MessageTool
from nanobot.agent.turn_delivery import TurnDeliveryFactory from nanobot.agent.turn_delivery import TurnDeliveryFactory
from nanobot.bus.queue import MessageBus from nanobot.bus.queue import MessageBus
@@ -1696,6 +1697,7 @@ def _run_gateway(
session_manager=session_manager, session_manager=session_manager,
image_generation_provider_configs=image_gen_provider_configs(config), image_generation_provider_configs=image_gen_provider_configs(config),
provider_snapshot_loader=load_provider_snapshot, provider_snapshot_loader=load_provider_snapshot,
preset_catalog_loader=load_model_preset_catalog,
runtime_events=runtime_events, runtime_events=runtime_events,
turn_delivery_factory=turn_delivery_factory, turn_delivery_factory=turn_delivery_factory,
provider_signature=provider_snapshot.signature, provider_signature=provider_snapshot.signature,
@@ -2086,7 +2088,7 @@ def _run_gateway(
asyncio.create_task( asyncio.create_task(
watch_config_file( watch_config_file(
Path(config_path), Path(config_path),
lambda: agent.runtime_resolver.invalidate(), lambda: agent.invalidate_runtime_config(),
), ),
name="nanobot-config-watcher", name="nanobot-config-watcher",
), ),
+19 -11
View File
@@ -237,7 +237,7 @@ async def cmd_status(ctx: CommandContext) -> OutboundMessage:
"""Build an outbound status message for a session.""" """Build an outbound status message for a session."""
loop = ctx.loop loop = ctx.loop
session = ctx.session or loop.sessions.get_or_create(ctx.key) session = ctx.session or loop.sessions.get_or_create(ctx.key)
runtime = ctx.runtime or loop.llm_runtime() runtime = ctx.runtime or loop.runtime_for_session(session)
ctx_est = 0 ctx_est = 0
with suppress(Exception): with suppress(Exception):
ctx_est, _ = loop.consolidator.estimate_session_prompt_tokens( ctx_est, _ = loop.consolidator.estimate_session_prompt_tokens(
@@ -286,11 +286,12 @@ async def cmd_new(ctx: CommandContext) -> OutboundMessage:
await loop._cancel_active_tasks(ctx.key) await loop._cancel_active_tasks(ctx.key)
session = ctx.session or loop.sessions.get_or_create(ctx.key) session = ctx.session or loop.sessions.get_or_create(ctx.key)
snapshot = session.messages[session.last_consolidated:] snapshot = session.messages[session.last_consolidated:]
if snapshot:
runtime = ctx.runtime or loop.runtime_for_session(session)
session.clear() session.clear()
loop.sessions.save(session) loop.sessions.save(session)
loop.sessions.invalidate(session.key) loop.sessions.invalidate(session.key)
if snapshot: if snapshot:
runtime = ctx.runtime or loop.llm_runtime()
loop._schedule_background( loop._schedule_background(
loop.consolidator.archive( loop.consolidator.archive(
snapshot, snapshot,
@@ -315,20 +316,25 @@ def _model_preset_names(loop) -> list[str]:
return ["default", *sorted(name for name in names if name != "default")] return ["default", *sorted(name for name in names if name != "default")]
def _active_model_preset_name(loop) -> str:
return loop.model_preset or "default"
def _command_error_message(exc: Exception) -> str: def _command_error_message(exc: Exception) -> str:
return str(exc.args[0]) if isinstance(exc, KeyError) and exc.args else str(exc) return str(exc.args[0]) if isinstance(exc, KeyError) and exc.args else str(exc)
def _model_command_status(loop) -> str: def _model_command_status(loop, session) -> str:
names = _model_preset_names(loop) names = _model_preset_names(loop)
active = _active_model_preset_name(loop) try:
runtime = loop.runtime_for_session(session)
except (KeyError, ValueError) as exc:
return "\n".join([
"## Model",
f"- Current selection error: {_command_error_message(exc)}",
f"- Available presets: {_format_preset_names(names)}",
"- Switch with `/model <preset>`.",
])
active = runtime.model_preset or "default"
return "\n".join([ return "\n".join([
"## Model", "## Model",
f"- Current model: `{loop.model}`", f"- Current model: `{runtime.model}`",
f"- Current preset: `{active}`", f"- Current preset: `{active}`",
f"- Available presets: {_format_preset_names(names)}", f"- Available presets: {_format_preset_names(names)}",
]) ])
@@ -341,10 +347,11 @@ async def cmd_model(ctx: CommandContext) -> OutboundMessage:
metadata = {**dict(ctx.msg.metadata or {}), "render_as": "text"} metadata = {**dict(ctx.msg.metadata or {}), "render_as": "text"}
if not args: if not args:
session = ctx.session or loop.sessions.get_or_create(ctx.key)
return OutboundMessage( return OutboundMessage(
channel=ctx.msg.channel, channel=ctx.msg.channel,
chat_id=ctx.msg.chat_id, chat_id=ctx.msg.chat_id,
content=_model_command_status(loop), content=_model_command_status(loop, session),
metadata=metadata, metadata=metadata,
) )
@@ -359,7 +366,7 @@ async def cmd_model(ctx: CommandContext) -> OutboundMessage:
name = parts[0] name = parts[0]
try: try:
runtime = loop.set_model_preset(name) runtime = loop.set_session_model_preset(ctx.key, name)
except (KeyError, ValueError) as exc: except (KeyError, ValueError) as exc:
names = _model_preset_names(loop) names = _model_preset_names(loop)
return OutboundMessage( return OutboundMessage(
@@ -375,6 +382,7 @@ async def cmd_model(ctx: CommandContext) -> OutboundMessage:
max_tokens = runtime.generation.max_tokens max_tokens = runtime.generation.max_tokens
lines = [ lines = [
f"Switched model preset to `{runtime.model_preset}`.", f"Switched model preset to `{runtime.model_preset}`.",
"- Scope: current session",
f"- Model: `{runtime.model}`", f"- Model: `{runtime.model}`",
f"- Context window: {runtime.context_window_tokens}", f"- Context window: {runtime.context_window_tokens}",
] ]
+32 -14
View File
@@ -37,6 +37,7 @@ from nanobot.sdk.types import (
StreamEventType, StreamEventType,
result_from_response, result_from_response,
) )
from nanobot.utils.llm_runtime import LLMRuntime
__all__ = [ __all__ = [
"Nanobot", "Nanobot",
@@ -192,16 +193,40 @@ class Nanobot:
model_preset: str | None = None, model_preset: str | None = None,
) -> RunStream: ) -> RunStream:
"""Start a streamed run and return a handle for events and final result.""" """Start a streamed run and return a handle for events and final result."""
runtime = self._loop.runtime_resolver.resolve_override( override_runtime = self._loop.runtime_resolver.resolve_override(
model=model, model=model,
model_preset=model_preset, model_preset=model_preset,
config=self._config, config=self._config,
) or self._loop.llm_runtime() )
queue: asyncio.Queue[StreamEvent | object] = asyncio.Queue(maxsize=256) queue: asyncio.Queue[StreamEvent | object] = asyncio.Queue(maxsize=256)
emitter = SDKStreamEmitter(queue) emitter = SDKStreamEmitter(queue)
stream_hook = SDKStreamingHook(emitter) stream_hook = SDKStreamingHook(emitter)
capture = SDKCaptureHook() capture = SDKCaptureHook()
per_run_hooks = [capture, stream_hook, *(hooks or [])] per_run_hooks = [capture, stream_hook, *(hooks or [])]
run_started = False
async def _emit_run_started(runtime: LLMRuntime | None = None) -> None:
nonlocal run_started
if run_started:
return
if runtime is None:
runtime = override_runtime
metadata: dict[str, Any] = {
"session_key": session_key,
"channel": channel,
"chat_id": chat_id,
"sender_id": sender_id,
}
if runtime is not None:
metadata.update({
"model": runtime.model,
"model_preset": runtime.model_preset,
})
await emitter.emit(StreamEvent(
type=STREAM_EVENT_RUN_STARTED,
metadata=metadata,
))
run_started = True
async def _on_stream(delta: str) -> None: async def _on_stream(delta: str) -> None:
await emitter.text_delta(delta) await emitter.text_delta(delta)
@@ -220,24 +245,16 @@ class Nanobot:
on_stream=_on_stream, on_stream=_on_stream,
on_stream_end=_on_stream_end, on_stream_end=_on_stream_end,
) )
kwargs["runtime"] = runtime kwargs["on_runtime_admitted"] = _emit_run_started
await emitter.emit(StreamEvent( if override_runtime is not None:
type=STREAM_EVENT_RUN_STARTED, kwargs["runtime"] = override_runtime
metadata={
"session_key": session_key,
"channel": channel,
"chat_id": chat_id,
"sender_id": sender_id,
"model": runtime.model,
"model_preset": runtime.model_preset,
},
))
try: try:
response = await self._loop.process_direct( response = await self._loop.process_direct(
message, message,
**kwargs, **kwargs,
hooks=per_run_hooks, hooks=per_run_hooks,
) )
await _emit_run_started()
await emitter.text_completed(resuming=False, force=False) await emitter.text_completed(resuming=False, force=False)
result = result_from_response(response, capture) result = result_from_response(response, capture)
await emitter.emit(StreamEvent( await emitter.emit(StreamEvent(
@@ -249,6 +266,7 @@ class Nanobot:
)) ))
return result return result
except Exception as exc: except Exception as exc:
await _emit_run_started()
await emitter.emit(StreamEvent( await emitter.emit(StreamEvent(
type=STREAM_EVENT_RUN_FAILED, type=STREAM_EVENT_RUN_FAILED,
error=str(exc), error=str(exc),
+7
View File
@@ -18,6 +18,7 @@ class ProviderSnapshot:
context_window_tokens: int context_window_tokens: int
signature: tuple[object, ...] signature: tuple[object, ...]
generation: GenerationSettings | None = None generation: GenerationSettings | None = None
model_preset: str | None = None
def _resolve_model_preset( def _resolve_model_preset(
@@ -261,6 +262,11 @@ def build_provider_snapshot(
preset: ModelPresetConfig | None = None, preset: ModelPresetConfig | None = None,
) -> ProviderSnapshot: ) -> ProviderSnapshot:
resolved = _resolve_model_preset(config, preset_name=preset_name, preset=preset) resolved = _resolve_model_preset(config, preset_name=preset_name, preset=preset)
selected_preset = (
config.agents.defaults.model_preset
if preset_name is None and preset is None
else preset_name
)
fallback_windows = [ fallback_windows = [
fallback.context_window_tokens fallback.context_window_tokens
for fallback in _resolve_fallback_presets(config, resolved) for fallback in _resolve_fallback_presets(config, resolved)
@@ -271,6 +277,7 @@ def build_provider_snapshot(
context_window_tokens=min([resolved.context_window_tokens, *fallback_windows]), context_window_tokens=min([resolved.context_window_tokens, *fallback_windows]),
signature=provider_signature(config, preset=resolved), signature=provider_signature(config, preset=resolved),
generation=resolved.to_generation_settings(), generation=resolved.to_generation_settings(),
model_preset=selected_preset,
) )
+3 -2
View File
@@ -196,7 +196,7 @@ class RuntimeClient:
async def compact_session(self, session_key: str) -> SessionSnapshot: async def compact_session(self, session_key: str) -> SessionSnapshot:
"""Run token/replay-window consolidation for one session.""" """Run token/replay-window consolidation for one session."""
session = self._loop.sessions.get_or_create(session_key) session = self._loop.sessions.get_or_create(session_key)
runtime = self._loop.llm_runtime() runtime = self._loop.runtime_for_session(session)
await self._loop.consolidator.maybe_consolidate_by_tokens( await self._loop.consolidator.maybe_consolidate_by_tokens(
session, session,
runtime=runtime, runtime=runtime,
@@ -208,7 +208,8 @@ class RuntimeClient:
async def compact_idle_session(self, session_key: str, *, max_suffix: int = 8) -> str | None: async def compact_idle_session(self, session_key: str, *, max_suffix: int = 8) -> str | None:
"""Run idle-session compaction for one session and return the summary.""" """Run idle-session compaction for one session and return the summary."""
runtime = self._loop.llm_runtime() session = self._loop.sessions.get_or_create(session_key)
runtime = self._loop.runtime_for_session(session)
return await self._loop.consolidator.compact_idle_session( return await self._loop.consolidator.compact_idle_session(
session_key, session_key,
runtime=runtime, runtime=runtime,
+20
View File
@@ -0,0 +1,20 @@
"""Session-scoped model preset metadata."""
from __future__ import annotations
from collections.abc import Mapping
# Session.metadata is public SDK data, so internal selectors use a reserved namespace.
SESSION_MODEL_PRESET_METADATA_KEY = "_nanobot_model_preset"
def model_preset_from_metadata(metadata: object) -> str | None:
"""Read the canonical session preset name from persisted metadata."""
if not isinstance(metadata, Mapping):
return None
if SESSION_MODEL_PRESET_METADATA_KEY not in metadata:
return None
value = metadata[SESSION_MODEL_PRESET_METADATA_KEY]
if not isinstance(value, str) or not value.strip():
raise ValueError("session model preset must be a non-empty string")
return value.strip()
+4 -5
View File
@@ -34,9 +34,8 @@ description: Inspect and optionally adjust the agent's runtime state. Use to che
| Situation | Command | | Situation | Command |
|-----------|---------| |-----------|---------|
| Large codebase analysis | `my(action="set", key="context_window_tokens", value=262144)` | | Check capacity for a large task | `my(action="check", key="context_window_tokens")` |
| Switch to a named model preset | `my(action="set", key="model_preset", value="<preset-name>")` | | Switch model or context capacity | `my(action="set", key="model_preset", value="<preset-name>")` |
| Repetitive simple tasks without a preset | `my(action="set", key="model", value="<fast-model>")` |
| Long multi-step task | `my(action="set", key="max_iterations", value=80)` | | Long multi-step task | `my(action="set", key="max_iterations", value=80)` |
**Tradeoff:** Bias toward stability. Only set when defaults are genuinely insufficient. **Tradeoff:** Bias toward stability. Only set when defaults are genuinely insufficient.
@@ -57,8 +56,8 @@ description: Inspect and optionally adjust the agent's runtime state. Use to che
## Constraints ## Constraints
- All modifications in-memory only — restart resets everything - `model_preset` is saved for the current session; other modifications are in-memory only
- Prefer `model_preset` for configured model choices. Direct `model` changes clear the active preset and should only be used when no preset exists. - Direct `model` and `context_window_tokens` writes are rejected during active sessions because they would change the shared instance default. Use a configured `model_preset` instead.
- Protected params have type/range validation: `max_iterations` (1100), `context_window_tokens` (40961M), `model` (non-empty str) - Protected params have type/range validation: `max_iterations` (1100), `context_window_tokens` (40961M), `model` (non-empty str)
- If `tools.my.allow_set` is false, check only - If `tools.my.allow_set` is false, check only
+5 -12
View File
@@ -34,23 +34,16 @@ Concrete scenarios showing when and how to use the my tool effectively.
``` ```
→ my(action="check") → my(action="check")
→ context_window_tokens: 200000 → context_window_tokens: 200000
→ my(action="set", key="context_window_tokens", value=262144) → my(action="set", key="model_preset", value="deep")
→ "Set context_window_tokens = 262144 (was 200000)" → "Set model_preset = 'deep' for the next turn; context_window_tokens will be 262144"
→ "I've expanded my context window to handle this large codebase." → "I've selected the configured deep preset for this session's next turn."
``` ```
### Switching to a configured model preset ### Switching to a configured model preset
``` ```
→ my(action="set", key="model_preset", value="fast") → my(action="set", key="model_preset", value="fast")
→ "Set model_preset = 'fast' (was 'deep'); model is now 'openai/gpt-4.1-mini'" → "Set model_preset = 'fast' for the next turn; model will be 'openai/gpt-4.1-mini'"
→ "Switched to the fast preset for these batch tasks." → "Selected the fast preset for this session's next turn."
```
### Switching to a raw model when no preset exists
```
→ my(action="set", key="model", value="anthropic/claude-haiku-4-5-20251001")
→ "Set model = 'anthropic/claude-haiku-4-5-20251001' (was 'anthropic/claude-sonnet-4-6')"
→ "Switched to a faster model for these batch tasks."
``` ```
## Cross-Turn Memory ## Cross-Turn Memory
+2 -4
View File
@@ -84,8 +84,6 @@ class LLMRuntime:
def runtime_from_provider_snapshot( def runtime_from_provider_snapshot(
snapshot: ProviderSnapshot, snapshot: ProviderSnapshot,
*,
model_preset: str | None = None,
) -> LLMRuntime: ) -> LLMRuntime:
"""Convert a provider factory snapshot into the canonical runtime value.""" """Convert a provider factory snapshot into the canonical runtime value."""
if snapshot.generation is not None: if snapshot.generation is not None:
@@ -94,13 +92,13 @@ def runtime_from_provider_snapshot(
model=snapshot.model, model=snapshot.model,
generation=snapshot.generation, generation=snapshot.generation,
context_window_tokens=snapshot.context_window_tokens, context_window_tokens=snapshot.context_window_tokens,
model_preset=model_preset, model_preset=snapshot.model_preset,
snapshot_signature=snapshot.signature, snapshot_signature=snapshot.signature,
) )
return LLMRuntime.capture( return LLMRuntime.capture(
snapshot.provider, snapshot.provider,
snapshot.model, snapshot.model,
context_window_tokens=snapshot.context_window_tokens, context_window_tokens=snapshot.context_window_tokens,
model_preset=model_preset, model_preset=snapshot.model_preset,
snapshot_signature=snapshot.signature, snapshot_signature=snapshot.signature,
) )
+6 -1
View File
@@ -25,9 +25,11 @@ from nanobot.session.manager import (
_message_preview_text, _message_preview_text,
_metadata_title, _metadata_title,
) )
from nanobot.session.model_selection import model_preset_from_metadata
_INDEX_VERSION = 2 _INDEX_VERSION = 4
_INDEX_FILENAME = ".webui_session_index.json" _INDEX_FILENAME = ".webui_session_index.json"
_MODEL_PRESET_FIELD = "model_preset"
_WEBUI_ACTIVITY_MTIME_NS = "webui_activity_mtime_ns" _WEBUI_ACTIVITY_MTIME_NS = "webui_activity_mtime_ns"
_WEBUI_ACTIVITY_SIZE = "webui_activity_size" _WEBUI_ACTIVITY_SIZE = "webui_activity_size"
_VISIBLE_TRANSCRIPT_ROLES = {"user", "assistant"} _VISIBLE_TRANSCRIPT_ROLES = {"user", "assistant"}
@@ -138,6 +140,7 @@ def _public_row(sessions_dir: Path, row: dict[str, Any]) -> dict[str, Any]:
"updated_at": row.get("updated_at"), "updated_at": row.get("updated_at"),
"title": row.get("title", ""), "title": row.get("title", ""),
"preview": row.get("preview", ""), "preview": row.get("preview", ""),
_MODEL_PRESET_FIELD: row.get(_MODEL_PRESET_FIELD),
"path": str(sessions_dir / str(row.get("file", ""))), "path": str(sessions_dir / str(row.get("file", ""))),
} }
@@ -256,6 +259,7 @@ def _indexed_row_for_session(session: Session, path: Path) -> dict[str, Any]:
), ),
"title": _metadata_title(session.metadata), "title": _metadata_title(session.metadata),
"preview": _preview_from_messages(session.messages), "preview": _preview_from_messages(session.messages),
_MODEL_PRESET_FIELD: model_preset_from_metadata(session.metadata),
"file": path.name, "file": path.name,
"mtime_ns": signature["mtime_ns"], "mtime_ns": signature["mtime_ns"],
"size": signature["size"], "size": signature["size"],
@@ -329,6 +333,7 @@ def _scan_session_row(session_manager: SessionManager, path: Path) -> dict[str,
), ),
"title": _metadata_title(data.get("metadata", {})), "title": _metadata_title(data.get("metadata", {})),
"preview": preview or fallback_preview, "preview": preview or fallback_preview,
_MODEL_PRESET_FIELD: model_preset_from_metadata(data.get("metadata", {})),
"file": path.name, "file": path.name,
"mtime_ns": signature["mtime_ns"], "mtime_ns": signature["mtime_ns"],
"size": signature["size"], "size": signature["size"],
+8
View File
@@ -805,6 +805,10 @@ def settings_payload(
"is_default": True, "is_default": True,
"model": defaults.model, "model": defaults.model,
"provider": defaults.provider, "provider": defaults.provider,
"resolved_provider": config.get_provider_name(
defaults.model,
preset=config.resolve_default_preset(),
),
"max_tokens": defaults.max_tokens, "max_tokens": defaults.max_tokens,
"context_window_tokens": defaults.context_window_tokens, "context_window_tokens": defaults.context_window_tokens,
"temperature": defaults.temperature, "temperature": defaults.temperature,
@@ -823,6 +827,10 @@ def settings_payload(
"is_default": False, "is_default": False,
"model": preset.model, "model": preset.model,
"provider": preset.provider, "provider": preset.provider,
"resolved_provider": config.get_provider_name(
preset.model,
preset=preset,
),
"max_tokens": preset.max_tokens, "max_tokens": preset.max_tokens,
"context_window_tokens": preset.context_window_tokens, "context_window_tokens": preset.context_window_tokens,
"temperature": preset.temperature, "temperature": preset.temperature,
+4 -4
View File
@@ -307,7 +307,7 @@ class TestAutoCompact:
loop.sessions.save(s2) loop.sessions.save(s2)
loop.consolidator.compact_idle_session = _make_fake_compact(loop) loop.consolidator.compact_idle_session = _make_fake_compact(loop)
loop.auto_compact.check_expired(loop._schedule_background, loop.llm_runtime) loop.auto_compact.check_expired(loop._schedule_background, loop.runtime_for_session)
await _drain_background_tasks(loop) await _drain_background_tasks(loop)
active_after = loop.sessions.get_or_create("cli:active") active_after = loop.sessions.get_or_create("cli:active")
@@ -776,7 +776,7 @@ class TestProactiveAutoCompact:
"""Helper: run check_expired via callback and wait for background tasks.""" """Helper: run check_expired via callback and wait for background tasks."""
loop.auto_compact.check_expired( loop.auto_compact.check_expired(
loop._schedule_background, loop._schedule_background,
loop.llm_runtime, loop.runtime_for_session,
active_session_keys=active_session_keys, active_session_keys=active_session_keys,
) )
await _drain_background_tasks(loop) await _drain_background_tasks(loop)
@@ -878,12 +878,12 @@ class TestProactiveAutoCompact:
loop.consolidator.compact_idle_session = _slow_compact loop.consolidator.compact_idle_session = _slow_compact
# First call starts archiving via callback # First call starts archiving via callback
loop.auto_compact.check_expired(loop._schedule_background, loop.llm_runtime) loop.auto_compact.check_expired(loop._schedule_background, loop.runtime_for_session)
await started.wait() await started.wait()
assert archive_count == 1 assert archive_count == 1
# Second call should skip (key is in _archiving) # Second call should skip (key is in _archiving)
loop.auto_compact.check_expired(loop._schedule_background, loop.llm_runtime) loop.auto_compact.check_expired(loop._schedule_background, loop.runtime_for_session)
assert archive_count == 1 assert archive_count == 1
# Clean up # Clean up
+51 -2
View File
@@ -9,7 +9,7 @@ from nanobot.agent.autocompact import AutoCompact
from nanobot.session.manager import Session, SessionManager from nanobot.session.manager import Session, SessionManager
def _runtime(): def _runtime(_session: Session | None = None):
return MagicMock(name="runtime") return MagicMock(name="runtime")
@@ -240,13 +240,62 @@ class TestCheckExpired:
resolve_runtime.return_value = replacement resolve_runtime.return_value = replacement
await scheduled[0] await scheduled[0]
resolve_runtime.assert_called_once_with() resolve_runtime.assert_called_once_with(session)
ac.consolidator.compact_idle_session.assert_awaited_once_with( ac.consolidator.compact_idle_session.assert_awaited_once_with(
"cli:old", "cli:old",
runtime=admitted, runtime=admitted,
max_suffix=ac._RECENT_SUFFIX_MESSAGES, max_suffix=ac._RECENT_SUFFIX_MESSAGES,
) )
@pytest.mark.parametrize("resolution_error", [KeyError, ValueError])
def test_invalid_preset_is_isolated_to_one_session(self, resolution_error):
ac = _make_autocompact(ttl=15)
old_dt = datetime.now() - timedelta(minutes=20)
sessions = {
key: _make_session(key, updated_at=old_dt)
for key in ("cli:removed", "cli:healthy")
}
for session in sessions.values():
_add_turns(session, 5)
ac.sessions.list_sessions.return_value = [
{"key": key, "updated_at": old_dt.isoformat()}
for key in sessions
]
ac.sessions.get_or_create.side_effect = sessions.__getitem__
healthy_runtime = _runtime()
def resolve_runtime(session: Session):
if session.key == "cli:removed":
raise resolution_error("model preset cannot be resolved")
return healthy_runtime
scheduled = []
def scheduler(coro):
scheduled.append(coro)
coro.close()
ac.check_expired(scheduler, resolve_runtime)
assert len(scheduled) == 1
assert ac._archiving == {"cli:healthy"}
def test_unexpected_runtime_resolution_failure_propagates(self):
ac = _make_autocompact(ttl=15)
old_dt = datetime.now() - timedelta(minutes=20)
session = _make_session("cli:old", updated_at=old_dt)
_add_turns(session, 5)
ac.sessions.list_sessions.return_value = [
{"key": session.key, "updated_at": old_dt.isoformat()}
]
ac.sessions.get_or_create.return_value = session
def fail(_session: Session):
raise RuntimeError("unexpected resolver failure")
with pytest.raises(RuntimeError, match="unexpected resolver failure"):
ac.check_expired(MagicMock(), fail)
def test_active_session_key_skips(self): def test_active_session_key_skips(self):
"""Session in active_session_keys should be skipped.""" """Session in active_session_keys should be skipped."""
ac = _make_autocompact(ttl=15) ac = _make_autocompact(ttl=15)
+97 -1
View File
@@ -52,9 +52,10 @@ def test_provider_snapshot_has_one_canonical_runtime_conversion() -> None:
model="snapshot-model", model="snapshot-model",
context_window_tokens=32_768, context_window_tokens=32_768,
signature=("snapshot-model", "openai"), signature=("snapshot-model", "openai"),
model_preset="fast",
) )
runtime = runtime_from_provider_snapshot(snapshot, model_preset="fast") runtime = runtime_from_provider_snapshot(snapshot)
assert runtime.provider is provider assert runtime.provider is provider
assert runtime.model == "snapshot-model" assert runtime.model == "snapshot-model"
@@ -94,6 +95,101 @@ def test_resolver_resolves_preset_without_mutating_selected_runtime() -> None:
assert resolved.generation == GenerationSettings(0.5, 512, None) assert resolved.generation == GenerationSettings(0.5, 512, None)
def test_resolver_reuses_preset_until_runtime_config_is_invalidated() -> None:
initial = _runtime()
preset = ModelPresetConfig(model="fast-model")
load_count = 0
preset_signature = ("fast-model", "auto", "initial")
def load_preset(_name: str) -> ProviderSnapshot:
nonlocal load_count
load_count += 1
return ProviderSnapshot(
provider=_provider(),
model="fast-model",
context_window_tokens=20_000,
signature=preset_signature,
)
resolver = ModelRuntimeResolver(
initial,
model_presets={"fast": preset},
preset_snapshot_loader=load_preset,
)
first = resolver.resolve_preset("fast")
second = resolver.resolve_preset("fast")
assert first is second
assert load_count == 1
preset_signature = ("fast-model", "auto", "new-credential")
resolver.invalidate()
refreshed = resolver.resolve_preset("fast")
assert refreshed is not first
assert load_count == 2
def test_resolver_refreshes_preset_catalog_after_invalidation() -> None:
provider = _provider()
catalog = {
"old": ModelPresetConfig(model="old-model", provider="openai"),
}
default_name = "old"
def load_preset(name: str) -> ProviderSnapshot:
preset = catalog[name]
return ProviderSnapshot(
provider=provider,
model=preset.model,
context_window_tokens=preset.context_window_tokens,
signature=(preset.model, preset.provider),
model_preset=name,
)
resolver = ModelRuntimeResolver(
runtime_from_provider_snapshot(load_preset("old")),
model_presets=catalog,
preset_catalog_loader=lambda: catalog,
configured_default_preset="old",
provider_snapshot_loader=lambda: load_preset(default_name),
preset_snapshot_loader=load_preset,
)
catalog["new"] = ModelPresetConfig(model="new-model", provider="openai")
default_name = "new"
resolver.invalidate()
assert resolver.admit().model_preset == "new"
assert set(resolver.model_presets) == {"old", "new"}
del catalog["old"]
resolver.invalidate()
assert resolver.admit().model_preset == "new"
assert set(resolver.model_presets) == {"new"}
def test_resolver_model_presets_are_read_only() -> None:
resolver = ModelRuntimeResolver(
_runtime(),
model_presets={"fast": ModelPresetConfig(model="fast-model")},
)
exposed = resolver.model_presets
with pytest.raises(TypeError):
exposed["other"] = ModelPresetConfig( # type: ignore[index]
model="other-model"
)
exposed["fast"].model = "mutated-model"
assert set(resolver.model_presets) == {"fast"}
assert resolver.model_presets["fast"].model == "fast-model"
assert resolver.resolve_preset("fast").model == "fast-model"
def test_resolver_model_override_is_derived_without_default_mutation() -> None: def test_resolver_model_override_is_derived_without_default_mutation() -> None:
initial = _runtime() initial = _runtime()
resolver = ModelRuntimeResolver(initial) resolver = ModelRuntimeResolver(initial)
+99
View File
@@ -1,3 +1,4 @@
import asyncio
from pathlib import Path from pathlib import Path
from types import SimpleNamespace from types import SimpleNamespace
from unittest.mock import MagicMock from unittest.mock import MagicMock
@@ -6,10 +7,15 @@ import pytest
from nanobot.agent.loop import AgentLoop from nanobot.agent.loop import AgentLoop
from nanobot.bus.queue import MessageBus from nanobot.bus.queue import MessageBus
from nanobot.bus.runtime_events import RuntimeModelChanged
from nanobot.config.loader import save_config from nanobot.config.loader import save_config
from nanobot.config.schema import Config, ModelPresetConfig from nanobot.config.schema import Config, ModelPresetConfig
from nanobot.providers.base import GenerationSettings from nanobot.providers.base import GenerationSettings
from nanobot.providers.factory import ProviderSnapshot, load_provider_snapshot from nanobot.providers.factory import ProviderSnapshot, load_provider_snapshot
from nanobot.session.model_selection import (
SESSION_MODEL_PRESET_METADATA_KEY,
model_preset_from_metadata,
)
from nanobot.webui.settings_api import update_agent_settings from nanobot.webui.settings_api import update_agent_settings
@@ -153,6 +159,99 @@ def test_same_snapshot_default_clears_preset_and_publishes_update(tmp_path: Path
assert published == [("fast-model", None)] assert published == [("fast-model", None)]
def test_named_default_refresh_is_used_by_sessions_without_override(tmp_path: Path) -> None:
provider = _provider("shared-model")
shared_signature = ("shared-model", "auto", "same-settings")
snapshots = {
"fast": ProviderSnapshot(
provider=provider,
model="shared-model",
context_window_tokens=16_000,
signature=shared_signature,
model_preset="fast",
),
"deep": ProviderSnapshot(
provider=provider,
model="shared-model",
context_window_tokens=16_000,
signature=shared_signature,
model_preset="deep",
),
}
default_snapshot = snapshots["deep"]
loop = AgentLoop(
bus=MessageBus(),
provider=provider,
workspace=tmp_path,
model="shared-model",
context_window_tokens=16_000,
provider_signature=shared_signature,
provider_snapshot_loader=lambda: default_snapshot,
model_presets={
name: ModelPresetConfig(model=snapshot.model)
for name, snapshot in snapshots.items()
},
model_preset="fast",
preset_snapshot_loader=snapshots.__getitem__,
)
loop.runtime_resolver.invalidate()
runtime = loop.llm_runtime()
session = loop.sessions.get_or_create("sdk:new-after-refresh")
assert runtime.model_preset == "deep"
assert loop.runtime_for_session(session).model_preset == "deep"
assert model_preset_from_metadata(session.metadata) is None
@pytest.mark.asyncio
async def test_config_invalidation_notifies_clients_before_session_runtime_refresh(
tmp_path: Path,
) -> None:
provider = _provider("model-a")
catalog = {"fast": ModelPresetConfig(model="model-a")}
current_model = "model-a"
published: list[RuntimeModelChanged] = []
def load_preset(_name: str) -> ProviderSnapshot:
return ProviderSnapshot(
provider=provider,
model=current_model,
context_window_tokens=16_000,
signature=(current_model, "auto"),
model_preset="fast",
)
loop = AgentLoop(
bus=MessageBus(),
provider=provider,
workspace=tmp_path,
model="model-a",
context_window_tokens=16_000,
provider_signature=("model-a", "auto"),
provider_snapshot_loader=lambda: load_preset("fast"),
model_presets=catalog,
preset_catalog_loader=lambda: catalog,
model_preset="fast",
preset_snapshot_loader=load_preset,
)
loop.runtime_events.subscribe(published.append, RuntimeModelChanged)
session = loop.sessions.get_or_create("websocket:chat")
session.metadata[SESSION_MODEL_PRESET_METADATA_KEY] = "fast"
current_model = "model-b"
catalog["fast"] = ModelPresetConfig(model="model-b")
loop.invalidate_runtime_config()
runtime = loop.runtime_for_session(session)
await asyncio.sleep(0)
assert [(event.model, event.model_preset) for event in published] == [
("model-a", "fast"),
]
assert runtime.model == "model-b"
assert loop.model_presets["fast"].model == "model-b"
def test_next_turn_captures_generation_changed_after_previous_admission( def test_next_turn_captures_generation_changed_after_previous_admission(
tmp_path: Path, tmp_path: Path,
) -> None: ) -> None:
+74
View File
@@ -4,10 +4,12 @@ from unittest.mock import MagicMock
import pytest import pytest
from nanobot.agent.loop import AgentLoop from nanobot.agent.loop import AgentLoop
from nanobot.agent.tools.context import RequestContext, request_context
from nanobot.agent.tools.self import MyTool from nanobot.agent.tools.self import MyTool
from nanobot.bus.queue import MessageBus from nanobot.bus.queue import MessageBus
from nanobot.config.schema import ModelPresetConfig from nanobot.config.schema import ModelPresetConfig
from nanobot.providers.factory import ProviderSnapshot from nanobot.providers.factory import ProviderSnapshot
from nanobot.session.model_selection import model_preset_from_metadata
def _provider(default_model: str, max_tokens: int = 123) -> MagicMock: def _provider(default_model: str, max_tokens: int = 123) -> MagicMock:
@@ -287,6 +289,78 @@ def test_self_tool_set_model_preset_unknown_lists_available(tmp_path) -> None:
assert loop.model == "base-model" assert loop.model == "base-model"
def test_self_tool_sets_model_preset_for_current_session(tmp_path) -> None:
presets = {
"default": ModelPresetConfig(model="base-model"),
"fast": ModelPresetConfig(model="openai/gpt-4.1"),
}
loop = _make_loop(tmp_path, presets=presets)
tool = MyTool(runtime_state=loop, modify_allowed=True)
with request_context(RequestContext(
channel="cli",
chat_id="one",
session_key="cli:one",
metadata={"source": "self-tool"},
)):
result = tool._modify("model_preset", "fast")
assert "for the next turn" in result
assert model_preset_from_metadata(
loop.sessions.get_or_create("cli:one").metadata
) == "fast"
assert loop.model_preset is None
assert loop.model == "base-model"
def test_self_tool_reports_session_preset_provider_configuration_error(tmp_path) -> None:
loop = _make_loop(tmp_path)
loop.set_session_model_preset = MagicMock(
side_effect=ValueError("No API key configured for provider 'openai'.")
)
tool = MyTool(runtime_state=loop, modify_allowed=True)
with request_context(RequestContext(
channel="cli",
chat_id="one",
session_key="cli:one",
)):
result = tool._modify("model_preset", "broken")
assert result == "Error: No API key configured for provider 'openai'."
@pytest.mark.parametrize(
("key", "value"),
[
("model", "other-model"),
("context_window_tokens", 8_192),
],
)
def test_self_tool_rejects_instance_runtime_changes_in_session(
tmp_path,
key: str,
value: object,
) -> None:
loop = _make_loop(tmp_path)
tool = MyTool(runtime_state=loop, modify_allowed=True)
session = loop.sessions.get_or_create("cli:one")
with request_context(RequestContext(
channel="cli",
chat_id="one",
session_key=session.key,
runtime=loop.runtime_for_session(session),
)):
result = tool._modify(key, value)
other_runtime = loop.runtime_for_session(loop.sessions.get_or_create("cli:two"))
assert "instance-wide and disabled" in result
assert "model_preset" in result
assert other_runtime.model == "base-model"
assert other_runtime.context_window_tokens == 1000
def test_self_tool_set_model_clears_active_preset(tmp_path) -> None: def test_self_tool_set_model_clears_active_preset(tmp_path) -> None:
presets = { presets = {
"fast": ModelPresetConfig(model="openai/gpt-4.1"), "fast": ModelPresetConfig(model="openai/gpt-4.1"),
+228
View File
@@ -0,0 +1,228 @@
import asyncio
import pytest
from nanobot.agent.loop import AgentLoop
from nanobot.bus.queue import MessageBus
from nanobot.config.schema import ModelPresetConfig
from nanobot.nanobot import Nanobot
from nanobot.providers.base import GenerationSettings, LLMProvider, LLMResponse
from nanobot.providers.factory import ProviderSnapshot
from nanobot.sdk.types import SessionSnapshot
from nanobot.session.model_selection import (
SESSION_MODEL_PRESET_METADATA_KEY,
model_preset_from_metadata,
)
from nanobot.utils.llm_runtime import LLMRuntime
class RecordingProvider(LLMProvider):
def __init__(self, name: str) -> None:
super().__init__()
self.name = name
self.generation = GenerationSettings(max_tokens=256, temperature=0.1)
self.calls: list[str | None] = []
async def chat(self, messages, tools=None, model=None, **kwargs):
await asyncio.sleep(0)
self.calls.append(model)
return LLMResponse(content=f"reply from {self.name}", finish_reason="stop")
def get_default_model(self) -> str:
return self.name
@pytest.mark.asyncio
async def test_sessions_run_concurrently_with_isolated_model_presets(tmp_path) -> None:
base = RecordingProvider("base-model")
fast = RecordingProvider("fast-model")
deep = RecordingProvider("deep-model")
providers = {"fast": fast, "deep": deep}
load_counts = {"fast": 0, "deep": 0}
presets = {
"default": ModelPresetConfig(model="base-model", context_window_tokens=8_000),
"fast": ModelPresetConfig(model="fast-model", context_window_tokens=16_000),
"deep": ModelPresetConfig(model="deep-model", context_window_tokens=32_000),
}
def load_preset(name: str) -> ProviderSnapshot:
load_counts[name] += 1
preset = presets[name]
provider = base if name == "default" else providers[name]
return ProviderSnapshot(
provider=provider,
model=preset.model,
context_window_tokens=preset.context_window_tokens,
signature=(name, preset.model),
)
loop = AgentLoop(
bus=MessageBus(),
provider=base,
workspace=tmp_path,
model="base-model",
context_window_tokens=8_000,
model_presets=presets,
preset_snapshot_loader=load_preset,
)
loop._schedule_background = lambda coro: coro.close() # type: ignore[method-assign]
loop.set_session_model_preset("sdk:fast", "fast")
loop.set_session_model_preset("sdk:deep", "deep")
fast_reply, deep_reply = await asyncio.gather(
loop.process_direct("hello", session_key="sdk:fast"),
loop.process_direct("hello", session_key="sdk:deep"),
)
assert fast_reply is not None and fast_reply.content == "reply from fast-model"
assert deep_reply is not None and deep_reply.content == "reply from deep-model"
assert fast.calls == ["fast-model"]
assert deep.calls == ["deep-model"]
assert base.calls == []
assert loop.provider is base
assert loop.model == "base-model"
assert load_counts == {"fast": 1, "deep": 1}
loop.sessions.invalidate("sdk:fast")
restored = loop.sessions.get_or_create("sdk:fast")
assert model_preset_from_metadata(restored.metadata) == "fast"
override = RecordingProvider("override-model")
override_runtime = LLMRuntime.capture(
override,
"override-model",
context_window_tokens=24_000,
)
override_reply = await loop.process_direct(
"hello",
session_key="sdk:fast",
runtime=override_runtime,
)
assert override_reply is not None
assert override_reply.content == "reply from override-model"
assert override.calls == ["override-model"]
assert fast.calls == ["fast-model"]
assert load_counts == {"fast": 1, "deep": 1}
@pytest.mark.asyncio
async def test_streamed_sdk_resolves_session_runtime_after_lock_admission(tmp_path) -> None:
base = RecordingProvider("base-model")
fast = RecordingProvider("fast-model")
deep = RecordingProvider("deep-model")
providers = {"fast": fast, "deep": deep}
presets = {
"fast": ModelPresetConfig(model="fast-model", context_window_tokens=16_000),
"deep": ModelPresetConfig(model="deep-model", context_window_tokens=32_000),
}
def load_preset(name: str) -> ProviderSnapshot:
preset = presets[name]
return ProviderSnapshot(
provider=providers[name],
model=preset.model,
context_window_tokens=preset.context_window_tokens,
signature=(name, preset.model),
)
loop = AgentLoop(
bus=MessageBus(),
provider=base,
workspace=tmp_path,
model="base-model",
context_window_tokens=8_000,
model_presets=presets,
preset_snapshot_loader=load_preset,
)
loop._schedule_background = lambda coro: coro.close() # type: ignore[method-assign]
session_key = "sdk:queued"
loop.set_session_model_preset(session_key, "fast")
lock = loop._session_locks.setdefault(session_key, asyncio.Lock())
await lock.acquire()
try:
run = await Nanobot(loop).run_streamed("hello", session_key=session_key)
loop.set_session_model_preset(session_key, "deep")
finally:
lock.release()
events = [event async for event in run.stream_events()]
result = await run.wait()
assert result.content == "reply from deep-model"
assert fast.calls == []
assert deep.calls == ["deep-model"]
assert events[0].type == "run.started"
assert events[0].metadata["model"] == "deep-model"
assert events[0].metadata["model_preset"] == "deep"
@pytest.mark.parametrize("custom_value", ["legacy-tag", 7])
@pytest.mark.asyncio
async def test_sdk_custom_model_preset_metadata_does_not_select_runtime(
tmp_path,
custom_value,
) -> None:
base = RecordingProvider("base-model")
loop = AgentLoop(
bus=MessageBus(),
provider=base,
workspace=tmp_path,
model="base-model",
context_window_tokens=8_000,
)
loop._schedule_background = lambda coro: coro.close() # type: ignore[method-assign]
bot = Nanobot(loop)
await bot.sessions.ingest(
"sdk:custom-metadata",
[],
metadata={"model_preset": custom_value},
)
ingested_result = await bot.run("hello", session_key="sdk:custom-metadata")
exported = bot.sessions.export("sdk:custom-metadata")
restored = await bot.sessions.restore(
SessionSnapshot(
key="sdk:restored-metadata",
messages=[],
metadata={"model_preset": custom_value},
)
)
restored_result = await bot.run("hello", session_key=restored.key)
assert ingested_result.content == "reply from base-model"
assert restored_result.content == "reply from base-model"
assert base.calls == ["base-model", "base-model"]
assert exported is not None
assert exported.metadata["model_preset"] == custom_value
assert restored.metadata["model_preset"] == custom_value
@pytest.mark.parametrize("invalid_value", [{"invalid": True}, " "])
@pytest.mark.asyncio
async def test_sdk_invalid_internal_model_preset_metadata_fails_explicitly(
tmp_path,
invalid_value,
) -> None:
base = RecordingProvider("base-model")
loop = AgentLoop(
bus=MessageBus(),
provider=base,
workspace=tmp_path,
model="base-model",
context_window_tokens=8_000,
)
loop._schedule_background = lambda coro: coro.close() # type: ignore[method-assign]
bot = Nanobot(loop)
await bot.sessions.ingest(
"sdk:invalid-internal-metadata",
[],
metadata={SESSION_MODEL_PRESET_METADATA_KEY: invalid_value},
)
with pytest.raises(ValueError, match="session model preset must be a non-empty string"):
await bot.run("hello", session_key="sdk:invalid-internal-metadata")
assert base.calls == []
+1 -1
View File
@@ -302,7 +302,7 @@ class TestCmdNewUnifiedSession:
sessions=sessions, sessions=sessions,
consolidator=SimpleNamespace(archive=AsyncMock(return_value=True)), consolidator=SimpleNamespace(archive=AsyncMock(return_value=True)),
_cancel_active_tasks=AsyncMock(return_value=0), _cancel_active_tasks=AsyncMock(return_value=0),
llm_runtime=MagicMock(return_value=MagicMock()), runtime_for_session=MagicMock(return_value=MagicMock()),
) )
loop._schedule_background = lambda coro: asyncio.ensure_future(coro) loop._schedule_background = lambda coro: asyncio.ensure_future(coro)
+28
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import time import time
from pathlib import Path from pathlib import Path
from types import MappingProxyType
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
import pytest import pytest
@@ -11,6 +12,7 @@ from pydantic import BaseModel
from nanobot.agent.tools.context import RequestContext, request_context from nanobot.agent.tools.context import RequestContext, request_context
from nanobot.agent.tools.self import MyTool from nanobot.agent.tools.self import MyTool
from nanobot.config.schema import ModelPresetConfig
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Helpers # Helpers
@@ -1078,6 +1080,32 @@ class TestSecurityAttributeProtection:
result = await tool.execute(action="set", key="web_config.enable", value=False) result = await tool.execute(action="set", key="web_config.enable", value=False)
assert "read-only" in result assert "read-only" in result
@pytest.mark.asyncio
async def test_modify_model_presets_dotpath_blocked(self):
"""The config-derived model preset catalog is inspectable but not mutable."""
presets = {"fast": {"model": "fast-model"}}
tool = _make_tool(runtime_state=_make_mock_loop(model_presets=presets))
result = await tool.execute(
action="set",
key="model_presets.other",
value={"model": "other-model"},
)
assert "read-only" in result
assert presets == {"fast": {"model": "fast-model"}}
@pytest.mark.asyncio
async def test_inspect_read_only_model_preset_dotpath(self):
presets = MappingProxyType({
"fast": ModelPresetConfig(model="fast-model"),
})
tool = _make_tool(runtime_state=_make_mock_loop(model_presets=presets))
result = await tool.execute(action="check", key="model_presets.fast.model")
assert result == "model_presets.fast.model: 'fast-model'"
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# current iteration count (Fix #2) # current iteration count (Fix #2)
+77 -10
View File
@@ -16,6 +16,10 @@ from nanobot.command.builtin import (
) )
from nanobot.command.router import CommandContext, CommandRouter from nanobot.command.router import CommandContext, CommandRouter
from nanobot.config.schema import ModelPresetConfig from nanobot.config.schema import ModelPresetConfig
from nanobot.session.model_selection import (
SESSION_MODEL_PRESET_METADATA_KEY,
model_preset_from_metadata,
)
def _provider(default_model: str, max_tokens: int = 123) -> MagicMock: def _provider(default_model: str, max_tokens: int = 123) -> MagicMock:
@@ -29,7 +33,7 @@ def _provider(default_model: str, max_tokens: int = 123) -> MagicMock:
return provider return provider
def _make_loop(tmp_path) -> AgentLoop: def _make_loop(tmp_path, *, preset_snapshot_loader=None) -> AgentLoop:
return AgentLoop( return AgentLoop(
bus=MessageBus(), bus=MessageBus(),
provider=_provider("base-model", max_tokens=123), provider=_provider("base-model", max_tokens=123),
@@ -48,6 +52,7 @@ def _make_loop(tmp_path) -> AgentLoop:
context_window_tokens=32_768, context_window_tokens=32_768,
), ),
}, },
preset_snapshot_loader=preset_snapshot_loader,
) )
@@ -64,6 +69,11 @@ def _ctx_session(loop: AgentLoop, raw: str, args: str = "") -> CommandContext:
) )
def _saved_model_preset(loop: AgentLoop, session_key: str = "cli:direct") -> str | None:
session = loop.sessions.get_or_create(session_key)
return model_preset_from_metadata(session.metadata)
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_model_command_lists_current_and_available_presets(tmp_path) -> None: async def test_model_command_lists_current_and_available_presets(tmp_path) -> None:
loop = _make_loop(tmp_path) loop = _make_loop(tmp_path)
@@ -84,23 +94,28 @@ async def test_model_command_switches_preset(tmp_path) -> None:
out = await cmd_model(_ctx(loop, "/model fast", args="fast")) out = await cmd_model(_ctx(loop, "/model fast", args="fast"))
assert "Switched model preset to `fast`." in out.content assert "Switched model preset to `fast`." in out.content
assert "Scope: current session" in out.content
assert "Model: `openai/gpt-4.1`" in out.content assert "Model: `openai/gpt-4.1`" in out.content
assert loop.model_preset == "fast" assert _saved_model_preset(loop) == "fast"
assert loop.model == "openai/gpt-4.1" assert loop.model_preset is None
assert not hasattr(loop.subagents, "model") assert loop.model == "base-model"
assert not hasattr(loop.consolidator, "model")
assert loop.llm_runtime().model == "openai/gpt-4.1" await loop.process_direct("/new", session_key="cli:direct")
assert _saved_model_preset(loop) == "fast"
status = await loop.process_direct("/status", session_key="cli:direct")
assert status is not None and "openai/gpt-4.1" in status.content
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_model_command_switches_back_to_default(tmp_path) -> None: async def test_model_command_switches_back_to_default(tmp_path) -> None:
loop = _make_loop(tmp_path) loop = _make_loop(tmp_path)
loop.set_model_preset("fast") await cmd_model(_ctx(loop, "/model fast", args="fast"))
out = await cmd_model(_ctx(loop, "/model default", args="default")) out = await cmd_model(_ctx(loop, "/model default", args="default"))
assert "Switched model preset to `default`." in out.content assert "Switched model preset to `default`." in out.content
assert loop.model_preset == "default" assert _saved_model_preset(loop) == "default"
assert loop.model_preset is None
assert loop.model == "base-model" assert loop.model == "base-model"
assert loop.context_window_tokens == 1000 assert loop.context_window_tokens == 1000
@@ -118,6 +133,24 @@ async def test_model_command_unknown_preset_keeps_old_state(tmp_path) -> None:
assert loop.model == "base-model" assert loop.model == "base-model"
@pytest.mark.asyncio
async def test_model_command_reports_provider_configuration_errors(tmp_path) -> None:
def fail_preset(_name: str):
raise ValueError("No API key configured for provider 'openai'.")
loop = _make_loop(tmp_path, preset_snapshot_loader=fail_preset)
switched = await cmd_model(_ctx(loop, "/model fast", args="fast"))
session = loop.sessions.get_or_create("cli:direct")
session.metadata[SESSION_MODEL_PRESET_METADATA_KEY] = "fast"
status = await cmd_model(_ctx(loop, "/model"))
assert "Could not switch model preset" in switched.content
assert "No API key configured for provider 'openai'." in switched.content
assert "Current selection error" in status.content
assert "No API key configured for provider 'openai'." in status.content
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_model_command_does_not_depend_on_my_allow_set(tmp_path) -> None: async def test_model_command_does_not_depend_on_my_allow_set(tmp_path) -> None:
loop = _make_loop(tmp_path) loop = _make_loop(tmp_path)
@@ -125,7 +158,7 @@ async def test_model_command_does_not_depend_on_my_allow_set(tmp_path) -> None:
await cmd_model(_ctx(loop, "/model fast", args="fast")) await cmd_model(_ctx(loop, "/model fast", args="fast"))
assert loop.model_preset == "fast" assert _saved_model_preset(loop) == "fast"
@pytest.mark.asyncio @pytest.mark.asyncio
@@ -142,11 +175,45 @@ async def test_model_command_registered_as_exact_and_prefix(tmp_path) -> None:
assert out.metadata == {"render_as": "text"} assert out.metadata == {"render_as": "text"}
assert out.content == "\n".join([ assert out.content == "\n".join([
"Switched model preset to `fast`.", "Switched model preset to `fast`.",
"- Scope: current session",
"- Model: `openai/gpt-4.1`", "- Model: `openai/gpt-4.1`",
"- Context window: 32768", "- Context window: 32768",
"- Max output tokens: 4096", "- Max output tokens: 4096",
]) ])
assert loop.model_preset == "fast" assert _saved_model_preset(loop) == "fast"
@pytest.mark.asyncio
async def test_model_command_does_not_change_another_session(tmp_path) -> None:
loop = _make_loop(tmp_path)
await cmd_model(_ctx(loop, "/model fast", args="fast"))
other = InboundMessage(channel="cli", sender_id="user", chat_id="other", content="/model")
out = await cmd_model(
CommandContext(msg=other, session=None, key=other.session_key, raw="/model", loop=loop)
)
assert "Current preset: `default`" in out.content
assert _saved_model_preset(loop) == "fast"
@pytest.mark.asyncio
async def test_model_command_reports_and_recovers_removed_session_preset(tmp_path) -> None:
loop = _make_loop(tmp_path)
session = loop.sessions.get_or_create("cli:direct")
session.metadata[SESSION_MODEL_PRESET_METADATA_KEY] = "removed"
loop.sessions.save(session)
status = await loop.process_direct("/model", session_key="cli:direct")
switched = await loop.process_direct("/model default", session_key="cli:direct")
assert status is not None
assert "model_preset 'removed' not found" in status.content
assert "Available presets: `default`, `fast`" in status.content
assert "Switch with `/model <preset>`" in status.content
assert switched is not None
assert "Switched model preset to `default`." in switched.content
assert _saved_model_preset(loop) == "default"
def test_model_command_in_help_and_palette() -> None: def test_model_command_in_help_and_palette() -> None:
+32 -9
View File
@@ -62,6 +62,13 @@ def _fake_provider(name: str, *, max_tokens: int = 8192) -> MagicMock:
return provider return provider
async def _admit_fake_runtime(bot, session_key, callback, runtime=None) -> None:
admitted = runtime or bot._loop.runtime_for_session(
bot._loop.sessions.get_or_create(session_key)
)
await callback(admitted)
def test_from_config_missing_file(): def test_from_config_missing_file():
with pytest.raises(FileNotFoundError): with pytest.raises(FileNotFoundError):
Nanobot.from_config("/nonexistent/config.json") Nanobot.from_config("/nonexistent/config.json")
@@ -633,8 +640,9 @@ async def test_run_model_preset_override_is_per_run(tmp_path):
model="openai/gpt-4.1-mini", model="openai/gpt-4.1-mini",
context_window_tokens=2048, context_window_tokens=2048,
signature=("preset", "fast"), signature=("preset", "fast"),
model_preset="fast",
) )
override_runtime = runtime_from_provider_snapshot(override, model_preset="fast") override_runtime = runtime_from_provider_snapshot(override)
bot._loop.runtime_resolver.resolve_override = MagicMock( bot._loop.runtime_resolver.resolve_override = MagicMock(
return_value=override_runtime return_value=override_runtime
) )
@@ -775,10 +783,12 @@ async def test_stream_yields_text_events_in_order(tmp_path):
bot = Nanobot.from_config(config_path, workspace=tmp_path) bot = Nanobot.from_config(config_path, workspace=tmp_path)
async def fake_process_direct( async def fake_process_direct(
message, *, session_key, on_stream, on_stream_end, hooks, runtime message, *, session_key, on_stream, on_stream_end, hooks, on_runtime_admitted,
runtime=None
): ):
assert message == "hi" assert message == "hi"
assert session_key == "sdk:default" assert session_key == "sdk:default"
await _admit_fake_runtime(bot, session_key, on_runtime_admitted, runtime)
await on_stream("Hel") await on_stream("Hel")
await on_stream("lo") await on_stream("lo")
await on_stream_end(resuming=False) await on_stream_end(resuming=False)
@@ -812,8 +822,10 @@ async def test_run_streamed_wait_returns_full_result_without_consuming_events(tm
bot = Nanobot.from_config(config_path, workspace=tmp_path) bot = Nanobot.from_config(config_path, workspace=tmp_path)
async def fake_process_direct( async def fake_process_direct(
message, *, session_key, on_stream, on_stream_end, hooks, runtime message, *, session_key, on_stream, on_stream_end, hooks, on_runtime_admitted,
runtime=None
): ):
await _admit_fake_runtime(bot, session_key, on_runtime_admitted, runtime)
await on_stream("done") await on_stream("done")
await on_stream_end(resuming=False) await on_stream_end(resuming=False)
ctx = AgentRunHookContext( ctx = AgentRunHookContext(
@@ -856,8 +868,10 @@ async def test_run_streamed_cancel_releases_full_queue_without_consuming(tmp_pat
bot = Nanobot.from_config(config_path, workspace=tmp_path) bot = Nanobot.from_config(config_path, workspace=tmp_path)
async def fake_process_direct( async def fake_process_direct(
message, *, session_key, on_stream, on_stream_end, hooks, runtime message, *, session_key, on_stream, on_stream_end, hooks, on_runtime_admitted,
runtime=None
): ):
await _admit_fake_runtime(bot, session_key, on_runtime_admitted, runtime)
for i in range(400): for i in range(400):
await on_stream(str(i)) await on_stream(str(i))
await on_stream_end(resuming=False) await on_stream_end(resuming=False)
@@ -921,7 +935,8 @@ async def test_run_streamed_forwards_runtime_options(tmp_path):
assert callable(kwargs["on_stream"]) assert callable(kwargs["on_stream"])
assert callable(kwargs["on_stream_end"]) assert callable(kwargs["on_stream_end"])
assert kwargs["hooks"] assert kwargs["hooks"]
assert kwargs["runtime"] is bot._loop.runtime_resolver.runtime assert "runtime" not in kwargs
assert callable(kwargs["on_runtime_admitted"])
@pytest.mark.asyncio @pytest.mark.asyncio
@@ -951,10 +966,12 @@ async def test_run_streamed_model_override_reports_admitted_runtime(tmp_path):
on_stream, on_stream,
on_stream_end, on_stream_end,
hooks, hooks,
on_runtime_admitted,
runtime, runtime,
): ):
assert runtime is override_runtime assert runtime is override_runtime
assert bot._loop.runtime_resolver.runtime is original_runtime assert bot._loop.runtime_resolver.runtime is original_runtime
await on_runtime_admitted(runtime)
await on_stream("ok") await on_stream("ok")
await on_stream_end(resuming=False) await on_stream_end(resuming=False)
return OutboundMessage(channel="cli", chat_id="direct", content="ok") return OutboundMessage(channel="cli", chat_id="direct", content="ok")
@@ -995,8 +1012,10 @@ async def test_run_streamed_emits_tool_events(tmp_path):
bot = Nanobot.from_config(config_path, workspace=tmp_path) bot = Nanobot.from_config(config_path, workspace=tmp_path)
async def fake_process_direct( async def fake_process_direct(
message, *, session_key, on_stream, on_stream_end, hooks, runtime message, *, session_key, on_stream, on_stream_end, hooks, on_runtime_admitted,
runtime=None
): ):
await _admit_fake_runtime(bot, session_key, on_runtime_admitted, runtime)
calls = [ calls = [
ToolCallRequest(id="call_ok", name="read_file", arguments={"path": "README.md"}), ToolCallRequest(id="call_ok", name="read_file", arguments={"path": "README.md"}),
ToolCallRequest(id="call_bad", name="exec", arguments={"cmd": "false"}), ToolCallRequest(id="call_bad", name="exec", arguments={"cmd": "false"}),
@@ -1043,8 +1062,10 @@ async def test_run_streamed_emits_reasoning_events(tmp_path):
bot = Nanobot.from_config(config_path, workspace=tmp_path) bot = Nanobot.from_config(config_path, workspace=tmp_path)
async def fake_process_direct( async def fake_process_direct(
message, *, session_key, on_stream, on_stream_end, hooks, runtime message, *, session_key, on_stream, on_stream_end, hooks, on_runtime_admitted,
runtime=None
): ):
await _admit_fake_runtime(bot, session_key, on_runtime_admitted, runtime)
for hook in hooks: for hook in hooks:
await hook.emit_reasoning("thinking") await hook.emit_reasoning("thinking")
await hook.emit_reasoning_end() await hook.emit_reasoning_end()
@@ -1072,8 +1093,10 @@ async def test_stream_generator_break_cancels_underlying_run(tmp_path):
cancelled = asyncio.Event() cancelled = asyncio.Event()
async def fake_process_direct( async def fake_process_direct(
message, *, session_key, on_stream, on_stream_end, hooks, runtime message, *, session_key, on_stream, on_stream_end, hooks, on_runtime_admitted,
runtime=None
): ):
await _admit_fake_runtime(bot, session_key, on_runtime_admitted, runtime)
try: try:
await on_stream("first") await on_stream("first")
await asyncio.sleep(10) await asyncio.sleep(10)
@@ -1387,7 +1410,7 @@ async def test_runtime_helpers_expose_model_workspace_and_compact(tmp_path):
bot = Nanobot.from_config(config_path, workspace=tmp_path) bot = Nanobot.from_config(config_path, workspace=tmp_path)
await bot.sessions.ingest("sdk:history", [{"role": "user", "content": "hello"}]) await bot.sessions.ingest("sdk:history", [{"role": "user", "content": "hello"}])
runtime = bot._loop.llm_runtime() runtime = bot._loop.llm_runtime()
bot._loop.llm_runtime = MagicMock(return_value=runtime) # type: ignore[method-assign] bot._loop.runtime_for_session = MagicMock(return_value=runtime) # type: ignore[method-assign]
bot._loop.consolidator.maybe_consolidate_by_tokens = AsyncMock() bot._loop.consolidator.maybe_consolidate_by_tokens = AsyncMock()
snapshot = await bot.runtime.compact_session("sdk:history") snapshot = await bot.runtime.compact_session("sdk:history")
+22
View File
@@ -4,11 +4,14 @@ import os
from datetime import datetime from datetime import datetime
from pathlib import Path from pathlib import Path
import pytest
import nanobot.webui.session_list_index as session_list_index import nanobot.webui.session_list_index as session_list_index
from nanobot.cron.session_turns import CRON_HISTORY_META from nanobot.cron.session_turns import CRON_HISTORY_META
from nanobot.session.automation_turns import AUTOMATION_HISTORY_META from nanobot.session.automation_turns import AUTOMATION_HISTORY_META
from nanobot.session.history_visibility import HIDDEN_HISTORY_META from nanobot.session.history_visibility import HIDDEN_HISTORY_META
from nanobot.session.manager import SessionManager from nanobot.session.manager import SessionManager
from nanobot.session.model_selection import SESSION_MODEL_PRESET_METADATA_KEY
def test_webui_session_list_reuses_valid_index_without_scanning_files( def test_webui_session_list_reuses_valid_index_without_scanning_files(
@@ -17,10 +20,12 @@ def test_webui_session_list_reuses_valid_index_without_scanning_files(
) -> None: ) -> None:
manager = SessionManager(tmp_path) manager = SessionManager(tmp_path)
session = manager.get_or_create("websocket:indexed") session = manager.get_or_create("websocket:indexed")
session.metadata[SESSION_MODEL_PRESET_METADATA_KEY] = "fast"
session.add_message("user", "indexed preview") session.add_message("user", "indexed preview")
manager.save(session) manager.save(session)
assert list_webui_sessions(manager)[0]["preview"] == "indexed preview" assert list_webui_sessions(manager)[0]["preview"] == "indexed preview"
assert list_webui_sessions(manager)[0]["model_preset"] == "fast"
def fail_scan(session_manager: SessionManager, path: Path) -> None: def fail_scan(session_manager: SessionManager, path: Path) -> None:
raise AssertionError(f"unexpected session file scan: {path}") raise AssertionError(f"unexpected session file scan: {path}")
@@ -31,6 +36,23 @@ def test_webui_session_list_reuses_valid_index_without_scanning_files(
assert rows[0]["key"] == "websocket:indexed" assert rows[0]["key"] == "websocket:indexed"
assert rows[0]["preview"] == "indexed preview" assert rows[0]["preview"] == "indexed preview"
assert rows[0]["model_preset"] == "fast"
def test_webui_session_list_rejects_invalid_internal_model_preset_metadata(
tmp_path: Path,
) -> None:
manager = SessionManager(tmp_path)
session = manager.get_or_create("websocket:custom-metadata")
session.metadata["model_preset"] = 7
session.metadata[SESSION_MODEL_PRESET_METADATA_KEY] = {"invalid": True}
session.add_message("user", "custom metadata")
manager.save(session)
with pytest.raises(ValueError, match="session model preset must be a non-empty string"):
list_webui_sessions(manager)
assert manager.get_or_create(session.key).metadata["model_preset"] == 7
def test_webui_session_list_rescans_only_changed_file(tmp_path: Path, monkeypatch) -> None: def test_webui_session_list_rescans_only_changed_file(tmp_path: Path, monkeypatch) -> None:
+20
View File
@@ -464,6 +464,26 @@ def test_settings_payload_includes_dynamic_custom_provider(
assert providers[DYNAMIC_PROVIDER_NAME]["api_base"] == DYNAMIC_PROVIDER_API_BASE assert providers[DYNAMIC_PROVIDER_NAME]["api_base"] == DYNAMIC_PROVIDER_API_BASE
def test_settings_payload_resolves_provider_for_each_auto_preset(
tmp_path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
config_path = tmp_path / "config.json"
config = _dynamic_provider_config()
config.model_presets["fast"] = ModelPresetConfig(
provider="auto",
model=f"{DYNAMIC_PROVIDER_NAME}/gpt-4",
)
save_config(config, config_path)
monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path)
payload = settings_payload()
presets = {row["name"]: row for row in payload["model_presets"]}
assert presets["fast"]["provider"] == "auto"
assert presets["fast"]["resolved_provider"] == DYNAMIC_PROVIDER_NAME
def test_settings_payload_groups_opencode_compatibility_alias(tmp_path, monkeypatch) -> None: def test_settings_payload_groups_opencode_compatibility_alias(tmp_path, monkeypatch) -> None:
config_path = tmp_path / "config.json" config_path = tmp_path / "config.json"
save_config(Config(), config_path) save_config(Config(), config_path)
+28 -15
View File
@@ -170,8 +170,14 @@ interface ModelBadgeInfo {
needsSetup: boolean; needsSetup: boolean;
} }
function activeModelPreset(settings: SettingsPayload | null): SettingsPayload["model_presets"][number] | null { function modelPresetForBadge(
settings: SettingsPayload | null,
scopedPreset: string | null,
): SettingsPayload["model_presets"][number] | null {
if (!settings) return null; if (!settings) return null;
if (scopedPreset) {
return settings.model_presets.find((preset) => preset.name === scopedPreset) ?? null;
}
const configured = settings.agent.model_preset || "default"; const configured = settings.agent.model_preset || "default";
return ( return (
settings.model_presets.find((preset) => preset.name === configured) settings.model_presets.find((preset) => preset.name === configured)
@@ -180,19 +186,25 @@ function activeModelPreset(settings: SettingsPayload | null): SettingsPayload["m
); );
} }
function resolvedModelProvider(settings: SettingsPayload | null, modelName: string | null): string | null { function toModelBadgeInfo(
const preset = activeModelPreset(settings); modelName: string | null,
const rawProvider = preset?.provider || settings?.agent.provider || null; settings: SettingsPayload | null,
if (rawProvider === "auto") { modelPreset: string | null = null,
return settings?.agent.resolved_provider || inferProviderFromModelName(modelName) || null; ): ModelBadgeInfo {
} const scopedPreset = modelPreset?.trim() || null;
return rawProvider || inferProviderFromModelName(modelName); const preset = modelPresetForBadge(settings, scopedPreset);
} const model = scopedPreset
? preset?.model || null
function toModelBadgeInfo(modelName: string | null, settings: SettingsPayload | null): ModelBadgeInfo { : modelName || settings?.agent.model || null;
const model = modelName || settings?.agent.model || null;
const label = toModelBadgeLabel(model); const label = toModelBadgeLabel(model);
const provider = resolvedModelProvider(settings, model); const rawProvider = preset?.provider
|| (!scopedPreset ? settings?.agent.provider : null)
|| null;
const provider = rawProvider === "auto"
? preset?.resolved_provider
|| (!scopedPreset ? settings?.agent.resolved_provider : null)
|| null
: rawProvider || inferProviderFromModelName(model);
const providerRow = provider const providerRow = provider
? settings?.providers.find((item) => item.name === provider) ? settings?.providers.find((item) => item.name === provider)
: null; : null;
@@ -459,9 +471,10 @@ export function ThreadShell({
const showHeroComposer = messages.length === 0 && !loading; const showHeroComposer = messages.length === 0 && !loading;
const wasShowingHeroComposerRef = useRef(showHeroComposer); const wasShowingHeroComposerRef = useRef(showHeroComposer);
const sessionModelPreset = session?.modelPreset?.trim() || null;
const modelBadge = useMemo( const modelBadge = useMemo(
() => toModelBadgeInfo(modelName, settings), () => toModelBadgeInfo(modelName, settings, sessionModelPreset),
[modelName, settings], [modelName, sessionModelPreset, settings],
); );
const modelBadgeLabel = modelBadge.needsSetup const modelBadgeLabel = modelBadge.needsSetup
? t("thread.composer.modelNotConfigured", { defaultValue: "Model not configured" }) ? t("thread.composer.modelNotConfigured", { defaultValue: "Model not configured" })
+2
View File
@@ -132,6 +132,7 @@ export async function listSessions(
updated_at: string | null; updated_at: string | null;
title?: string; title?: string;
preview?: string; preview?: string;
model_preset?: string | null;
run_started_at?: number | null; run_started_at?: number | null;
workspace_scope?: WorkspaceScopePayload | null; workspace_scope?: WorkspaceScopePayload | null;
}; };
@@ -148,6 +149,7 @@ export async function listSessions(
updatedAt: s.updated_at, updatedAt: s.updated_at,
title: s.title ?? "", title: s.title ?? "",
preview: s.preview ?? "", preview: s.preview ?? "",
modelPreset: s.model_preset ?? null,
runStartedAt: s.run_started_at ?? null, runStartedAt: s.run_started_at ?? null,
workspaceScope: s.workspace_scope ?? null, workspaceScope: s.workspace_scope ?? null,
})); }));
+3
View File
@@ -245,6 +245,8 @@ export interface ChatSummary {
updatedAt: string | null; updatedAt: string | null;
title?: string; title?: string;
preview: string; preview: string;
/** Model preset persisted for this session; null means it still follows the global default. */
modelPreset?: string | null;
/** Unix epoch seconds when this session currently has a turn in flight. */ /** Unix epoch seconds when this session currently has a turn in flight. */
runStartedAt?: number | null; runStartedAt?: number | null;
workspaceScope?: WorkspaceScopePayload | null; workspaceScope?: WorkspaceScopePayload | null;
@@ -406,6 +408,7 @@ export interface SettingsPayload {
is_default: boolean; is_default: boolean;
model: string; model: string;
provider: string; provider: string;
resolved_provider?: string | null;
max_tokens: number; max_tokens: number;
context_window_tokens: number; context_window_tokens: number;
temperature: number; temperature: number;
+2
View File
@@ -797,6 +797,7 @@ describe("webui API helpers", () => {
created_at: "2026-05-01T10:00:00", created_at: "2026-05-01T10:00:00",
updated_at: "2026-05-01T10:01:00", updated_at: "2026-05-01T10:01:00",
title: "优化 WebUI 标题", title: "优化 WebUI 标题",
model_preset: "fast",
run_started_at: 1_700_000_000, run_started_at: 1_700_000_000,
}, },
], ],
@@ -808,6 +809,7 @@ describe("webui API helpers", () => {
key: "websocket:chat-1", key: "websocket:chat-1",
title: "优化 WebUI 标题", title: "优化 WebUI 标题",
preview: "", preview: "",
modelPreset: "fast",
runStartedAt: 1_700_000_000, runStartedAt: 1_700_000_000,
}, },
]); ]);
+71 -1
View File
@@ -94,7 +94,7 @@ function expectSendMessageWithTurn(
); );
} }
function session(chatId: string) { function session(chatId: string, modelPreset?: string | null) {
return { return {
key: `websocket:${chatId}`, key: `websocket:${chatId}`,
channel: "websocket" as const, channel: "websocket" as const,
@@ -102,6 +102,7 @@ function session(chatId: string) {
createdAt: null, createdAt: null,
updatedAt: null, updatedAt: null,
preview: "", preview: "",
modelPreset,
}; };
} }
@@ -220,6 +221,20 @@ function modelSettings(model: string, provider: string): SettingsPayload {
}; };
} }
function settingsWithFastPreset(): SettingsPayload {
const settings = modelSettings("deepseek-v4-pro", "deepseek");
settings.model_presets.push({
...settings.model_presets[0]!,
name: "fast",
label: "Fast",
active: false,
is_default: false,
model: "openai-codex/gpt-5.5",
provider: "openai_codex",
});
return settings;
}
describe("ThreadShell", () => { describe("ThreadShell", () => {
beforeEach(() => { beforeEach(() => {
vi.stubGlobal( vi.stubGlobal(
@@ -341,6 +356,61 @@ describe("ThreadShell", () => {
expect(await screen.findByTestId("composer-model-logo-openai_codex")).toBeInTheDocument(); expect(await screen.findByTestId("composer-model-logo-openai_codex")).toBeInTheDocument();
}); });
it("resolves the composer model from the active session preset", async () => {
const client = makeClient();
render(
wrap(
client,
<ThreadShell
session={session("chat-fast", "fast")}
title="Fast session"
onToggleSidebar={() => {}}
settingsSnapshot={settingsWithFastPreset()}
/>,
"deepseek-v4-pro",
),
);
expect(await screen.findByTitle("gpt-5.5 · OpenAI Codex")).toBeInTheDocument();
expect(screen.queryByTitle("deepseek-v4-pro · DeepSeek")).not.toBeInTheDocument();
});
it("uses the backend-resolved provider for an auto session preset", async () => {
const client = makeClient();
const settings = modelSettings("deepseek-v4-pro", "deepseek");
settings.providers.push({
name: "companyproxy",
label: "Company Proxy",
configured: true,
});
settings.model_presets.push({
...settings.model_presets[0]!,
name: "fast",
label: "Fast",
active: false,
is_default: false,
model: "companyproxy/gpt-4",
provider: "auto",
resolved_provider: "companyproxy",
});
render(
wrap(
client,
<ThreadShell
session={session("chat-auto", "fast")}
title="Auto provider session"
onToggleSidebar={() => {}}
settingsSnapshot={settings}
/>,
"deepseek-v4-pro",
),
);
expect(await screen.findByTitle("gpt-4 · Company Proxy")).toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Model not configured" })).not.toBeInTheDocument();
});
it("opens model settings from the unconfigured model badge", async () => { it("opens model settings from the unconfigured model badge", async () => {
const client = makeClient(); const client = makeClient();
const settings = modelSettings("openai-codex/gpt-5.1-codex", "openai_codex"); const settings = modelSettings("openai-codex/gpt-5.1-codex", "openai_codex");