diff --git a/docs/chat-commands.md b/docs/chat-commands.md index 31468f8e..2e006243 100644 --- a/docs/chat-commands.md +++ b/docs/chat-commands.md @@ -9,7 +9,7 @@ These commands work inside chat channels and interactive agent sessions: | `/restart` | Restart the bot | | `/status` | Show bot status | | `/model` | Show the current model and available model presets | -| `/model ` | Switch the runtime model preset for future turns | +| `/model ` | Switch and persist the model preset for the current session | | `/dream` | Run Dream memory consolidation now | | `/dream-log` | Show the latest Dream memory change | | `/dream-log ` | Show a specific Dream memory change | @@ -47,7 +47,7 @@ Use `/model` to inspect the current runtime model: /model ``` -The response shows the current model, the current preset, and the available preset names. Named presets come from the top-level `modelPresets` config and are the recommended way to configure model choices. `default` is always available and represents the model settings from direct `agents.defaults.*` fields. +The response shows the current session's model and preset, plus the available preset names. Named presets come from the top-level `modelPresets` config and are the recommended way to configure model choices. `default` is always available and represents the model settings from direct `agents.defaults.*` fields. To switch presets for future turns: @@ -57,7 +57,7 @@ To switch presets for future turns: /model default ``` -Preset names come from the top-level `modelPresets` config. Switching is runtime-only: it does not rewrite `config.json`, and an in-progress turn keeps using the model it started with. See [Configuration: Model presets](./configuration.md#model-presets) for setup details. +Preset names come from the top-level `modelPresets` config. Switching affects only the current session and persists the selection in that session, so later turns keep using it across process restarts. It does not rewrite `config.json`, does not change other sessions, and does not alter an in-progress turn's captured model. Sessions without a saved selection follow `agents.defaults.modelPreset` (or the implicit `default` preset when it is omitted). See [Configuration: Model presets](./configuration.md#model-presets) for setup details. ## Local triggers diff --git a/docs/configuration.md b/docs/configuration.md index f42ec4ee..c69ad7d2 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1298,7 +1298,7 @@ Contributor notes for adding new providers live in [`development.md`](./developm ## Model Presets -Model presets let you name a complete model configuration and switch it at runtime with `/model `. They are the recommended way to configure models because the same names can be reused for startup selection, chat-command switching, and fallback chains. +Model presets let you name a complete model configuration and select one per session with `/model `. They are the recommended way to configure models because the same names can be reused for new-session defaults, chat-command switching, and fallback chains. Existing configs do not need to change. Direct `agents.defaults.model`, `provider`, `maxTokens`, `contextWindowTokens`, `temperature`, and `reasoningEffort` fields still define the implicit `default` preset. For new configs, prefer top-level `modelPresets` plus `agents.defaults.modelPreset`. @@ -1362,7 +1362,7 @@ Existing configs do not need to change. Direct `agents.defaults.model`, `provide `default` is reserved and always means the implicit preset built from direct `agents.defaults.*` fields; do not define `modelPresets.default`. Use `/model default` to switch back to those direct fields in an existing config. -Set `agents.defaults.modelPreset` to choose the startup preset. When `modelPreset` is `null` or omitted, startup uses the implicit `default` preset from direct `agents.defaults.*` fields. Runtime changes made with `/model ` are not written back to `config.json`; they affect future turns until the process restarts or another model/config change replaces them. +Set `agents.defaults.modelPreset` to choose the preset followed by sessions that have no saved model selection. When `modelPreset` is `null` or omitted, such sessions follow the implicit `default` preset from direct `agents.defaults.*` fields. `/model ` saves an override in the current session, so its future turns keep that preset across process restarts while other sessions remain unchanged. The command does not write the selection back to `config.json`. ### Model Fallbacks diff --git a/docs/my-tool.md b/docs/my-tool.md index 9cabc47e..e21466d1 100644 --- a/docs/my-tool.md +++ b/docs/my-tool.md @@ -27,7 +27,8 @@ To allow the agent to set its configuration (e.g. switch models, adjust paramete Legacy `tools.myEnabled` / `tools.mySet` keys are auto-migrated on load, and rewritten in-place the next time `nanobot onboard` refreshes the config. -All modifications are held in memory only — restart restores defaults. +Most modifications are held in memory only. `model_preset` is the exception: it is +stored in the current session so the selection survives a restart. --- @@ -77,20 +78,18 @@ my(action="check", key="web_config.enable") ## set — Runtime tuning -Changes take effect immediately, no restart required. +Changes do not require a restart. `model_preset` is saved for the current session and +applies to its next turn; other writable runtime tuning takes effect immediately. +Direct `model` and `context_window_tokens` writes are rejected during an active session +because those setters change the shared instance default. Configure a named preset for +model or context-window changes instead. ```text my(action="set", key="max_iterations", value=80) # → Bump iteration limit from 40 to 80 my(action="set", key="model_preset", value="fast") -# → Switch to a configured model preset - -my(action="set", key="model", value="fast-model") -# → Switch to a raw model and clear the active preset - -my(action="set", key="context_window_tokens", value=262144) -# → Expand context window for long documents +# → Use a configured model preset for this session's next turn ``` You can also store custom state in your scratchpad: @@ -109,9 +108,9 @@ These parameters have type and range validation — invalid values are rejected: | Parameter | Type | Range | Purpose | |-----------|------|-------|---------| | `max_iterations` | int | 1–100 | Max tool calls per conversation turn | -| `context_window_tokens` | int | 4,096–1,000,000 | Context window size | -| `model` | str | non-empty | LLM model to use | -| `model_preset` | str | configured preset name | Named preset to use | +| `context_window_tokens` | int | 4,096–1,000,000 | Instance default; during a session, select through a preset | +| `model` | str | non-empty | Instance default; during a session, select through a preset | +| `model_preset` | str | configured preset name | Current session's preset for its next turn | Other parameters (e.g. `workspace`, `provider_retry_mode`, `max_tool_result_chars`) can be set freely, as long as the value is JSON-safe. @@ -122,8 +121,8 @@ Other parameters (e.g. `workspace`, `provider_retry_mode`, `max_tool_result_char ### "This task is complex, I need more room" ```text -Agent: This codebase is large, let me expand my context window to handle it. -→ my(action="set", key="context_window_tokens", value=262144) +Agent: This codebase is large, let me switch this session to the configured deep preset. +→ my(action="set", key="model_preset", value="deep") ``` ### "Simple question, don't waste compute" @@ -180,7 +179,9 @@ Agent: The code review is progressing well. The test task hasn't started yet. ## Safety Mechanisms -Core design principle: **All modifications live in memory only. Restart restores defaults.** The agent cannot cause persistent damage. +Core design principle: **The tool does not rewrite `config.json`.** Instance-wide +changes live in memory only, while `model_preset` persists only as the current +session's selector. ### Off-limits (BLOCKED) diff --git a/docs/provider-cookbook.md b/docs/provider-cookbook.md index 742a2b33..f7a6b40d 100644 --- a/docs/provider-cookbook.md +++ b/docs/provider-cookbook.md @@ -610,7 +610,9 @@ In chat: /model fast ``` -`/model` switching is runtime-only. It does not rewrite `config.json`, and an in-progress turn keeps using the model it started with. +`/model` stores the selection in the current session without rewriting `config.json`. +The selection survives restarts, does not affect other sessions, and an in-progress +turn keeps using the model it started with. ## Quick Failure Map diff --git a/docs/python-sdk.md b/docs/python-sdk.md index ce3618ee..82660589 100644 --- a/docs/python-sdk.md +++ b/docs/python-sdk.md @@ -494,8 +494,10 @@ Run the agent once and return a `RunResult`. | `model` | `str \| None` | `None` | Override the model for this run only. | | `model_preset` | `str \| None` | `None` | Override the model preset for this run only. | -`model` and `model_preset` are per-run overrides and do not change -`bot.runtime.model` after the run completes. They are mutually exclusive. +Without an override, a run uses the preset saved in its session, or the configured +default when that session has no saved selection. `model` and `model_preset` are +mutually exclusive per-run overrides; they do not change the saved session selection +or `bot.runtime.model` after the run completes. ### `await bot.run_streamed(...)` @@ -531,9 +533,9 @@ async for event in bot.stream("Generate a long answer"): | `await cancel()` | Cancel the run and release stream resources. | | `await aclose()` | Close the stream; equivalent cleanup primitive for `async with` / manual lifecycle code. | -Normal SDK runs with different session keys may overlap. Runs that use per-run -`model` or `model_preset` overrides are exclusive while the override is active, -because the current `AgentLoop` provider/model state is mutable. +SDK runs with different session keys may overlap, including runs with per-run +`model` or `model_preset` overrides. Each run receives an immutable runtime without +mutating the instance default. Runs sharing one session key remain serialized. ### `StreamEvent` diff --git a/docs/websocket.md b/docs/websocket.md index df9104cc..41c78583 100644 --- a/docs/websocket.md +++ b/docs/websocket.md @@ -152,7 +152,8 @@ All frames are JSON text. Each message has an `event` field. Reasoning frames only flow when the channel's `showReasoning` is `true` (default) and the model returns reasoning content (DeepSeek-R1 / Kimi / MiMo / OpenAI reasoning models, Anthropic extended thinking, or inline `` / `` tags). Models without reasoning produce zero `reasoning_delta` frames. -**`runtime_model_updated`** — broadcast when the gateway runtime model changes, for example after `/model `: +**`runtime_model_updated`** — broadcast when the gateway default runtime changes or +when a config reload requires clients to refresh their model catalog: ```json { @@ -162,7 +163,10 @@ Reasoning frames only flow when the channel's `showReasoning` is `true` (default } ``` -`model_preset` is omitted when no named preset is active. WebUI clients use this event to keep the displayed model badge in sync across slash commands, config reloads, and settings changes. +`model_preset` is omitted when no named preset is active. WebUI clients use this event +to refresh model settings after default-runtime and config changes. `/model ` +is session-scoped; its selection is reflected through `session_updated` and the +session row's `model_preset` field instead of this global event. **`attached`** — confirmation for `new_chat` / `attach` inbound envelopes (see [Multi-chat multiplexing](#multi-chat-multiplexing)): diff --git a/nanobot/agent/autocompact.py b/nanobot/agent/autocompact.py index 6c9e580c..d73bf944 100644 --- a/nanobot/agent/autocompact.py +++ b/nanobot/agent/autocompact.py @@ -66,7 +66,7 @@ class AutoCompact: def check_expired( self, schedule_background: Callable[[Coroutine], None], - resolve_runtime: Callable[[], LLMRuntime], + resolve_runtime: Callable[[Session], LLMRuntime], active_session_keys: Collection[str] = (), ) -> None: """Schedule archival for idle sessions, skipping those with in-flight agent tasks.""" @@ -79,7 +79,12 @@ class AutoCompact: continue updated_at = info.get("updated_at") if self._is_expired(updated_at, now) and self._has_compactable_idle_tail(key): - runtime = resolve_runtime() + session = self.sessions.get_or_create(key) + try: + runtime = resolve_runtime(session) + except (KeyError, ValueError): + # Invalid session selections remain recoverable through /model. + continue self._archiving.add(key) schedule_background(self._archive(key, runtime=runtime)) diff --git a/nanobot/agent/loop.py b/nanobot/agent/loop.py index 9328e28b..29f946b2 100644 --- a/nanobot/agent/loop.py +++ b/nanobot/agent/loop.py @@ -79,6 +79,10 @@ from nanobot.session.manager import ( SessionManager, replay_max_messages_for_context, ) +from nanobot.session.model_selection import ( + SESSION_MODEL_PRESET_METADATA_KEY, + model_preset_from_metadata, +) from nanobot.triggers.local_turns import LocalTriggerTurnCoordinator from nanobot.utils.cancellation import task_is_cancelling from nanobot.utils.document import extract_documents, reference_non_image_attachments @@ -129,7 +133,7 @@ class TurnContext: session_key: str state: TurnState turn_id: str - runtime: LLMRuntime + runtime: LLMRuntime | None kind: TurnKind delivery: TurnDelivery original_user_text: str | None = None @@ -155,6 +159,7 @@ class TurnContext: on_progress: Callable[..., Awaitable[None]] | None = None on_stream: Callable[[str], Awaitable[None]] | None = None on_stream_end: Callable[..., Awaitable[None]] | None = None + on_runtime_admitted: Callable[[LLMRuntime], Awaitable[None]] | None = None on_retry_wait: Callable[[str], Awaitable[None]] | None = None pending_queue: asyncio.Queue | None = None @@ -282,6 +287,7 @@ class AgentLoop: provider_snapshot_loader: Callable[..., ProviderSnapshot] | None = None, provider_signature: tuple[object, ...] | None = None, model_presets: dict[str, ModelPresetConfig] | None = None, + preset_catalog_loader: preset_helpers.PresetCatalogLoader | None = None, model_preset: str | None = None, preset_snapshot_loader: preset_helpers.PresetSnapshotLoader | None = None, runtime_events: RuntimeEventBus | None = None, @@ -331,6 +337,8 @@ class AgentLoop: snapshot_signature=provider_signature, ), model_presets=configured_presets, + preset_catalog_loader=preset_catalog_loader, + configured_default_preset=model_preset, provider_snapshot_loader=provider_snapshot_loader, preset_snapshot_loader=preset_snapshot_loader, ) @@ -504,6 +512,28 @@ class AgentLoop: """Keep subagent runtime limits aligned with mutable loop settings.""" self.subagents.max_iterations = self.max_iterations + def invalidate_runtime_config(self) -> None: + """Invalidate runtime config and notify clients to refresh its catalog.""" + self.runtime_resolver.invalidate() + self._publish_runtime_selection(self.runtime_resolver.runtime) + + def runtime_for_session(self, session: Session) -> LLMRuntime: + """Resolve the immutable runtime selected by one session.""" + name = model_preset_from_metadata(session.metadata) + return self.llm_runtime() if name is None else self.runtime_resolver.resolve_preset(name) + + def set_session_model_preset( + self, + session_key: str, + name: str, + ) -> LLMRuntime: + """Validate and persist one session's preset selection.""" + runtime = self.runtime_resolver.resolve_preset(name) + session = self.sessions.get_or_create(session_key) + session.metadata[SESSION_MODEL_PRESET_METADATA_KEY] = runtime.model_preset + self.sessions.save(session) + return runtime + def _publish_runtime_selection( self, runtime: LLMRuntime, @@ -983,7 +1013,7 @@ class AgentLoop: except asyncio.TimeoutError: self.auto_compact.check_expired( self._schedule_background, - self.llm_runtime, + self.runtime_for_session, active_session_keys=self._pending_queues.keys(), ) continue @@ -1224,11 +1254,9 @@ class AgentLoop: tools: ToolRegistry | None = None, runtime: LLMRuntime | None = None, delivery: TurnDelivery | None = None, + on_runtime_admitted: Callable[[LLMRuntime], Awaitable[None]] | None = None, ) -> OutboundMessage | None: """Process a single inbound message and return the response.""" - if runtime is None: - runtime = self.llm_runtime() - kind = TurnKind.SYSTEM if msg.channel == "system" else TurnKind.USER if kind is TurnKind.SYSTEM: destination = ( @@ -1268,6 +1296,7 @@ class AgentLoop: on_progress=on_progress, on_stream=on_stream, on_stream_end=on_stream_end, + on_runtime_admitted=on_runtime_admitted, pending_queue=pending_queue, ephemeral=ephemeral, run_extra_hooks_for_ephemeral=run_extra_hooks_for_ephemeral, @@ -1452,13 +1481,19 @@ class AgentLoop: return "dispatch" async def _state_build(self, ctx: TurnContext) -> str: + runtime = ctx.runtime + if runtime is None: + runtime = self.runtime_for_session(ctx.session) + ctx.runtime = runtime + if ctx.on_runtime_admitted is not None: + await ctx.on_runtime_admitted(runtime) replay_max_messages = replay_max_messages_for_context( - ctx.runtime.context_window_tokens + runtime.context_window_tokens ) if not ctx.ephemeral: await self.consolidator.maybe_consolidate_by_tokens( ctx.session, - runtime=ctx.runtime, + runtime=runtime, replay_max_messages=replay_max_messages, ) is_subagent = ctx.kind is TurnKind.SYSTEM and ctx.msg.sender_id == "subagent" @@ -1469,7 +1504,7 @@ class AgentLoop: _hist_kwargs: dict[str, Any] = { "max_messages": replay_max_messages, - "max_tokens": self._replay_token_budget(ctx.runtime), + "max_tokens": self._replay_token_budget(runtime), "extend_to_user": is_subagent, } ctx.history = ctx.session.get_history(**_hist_kwargs) @@ -1864,6 +1899,7 @@ class AgentLoop: tools: ToolRegistry | None = None, persist_user_message: bool = True, runtime: LLMRuntime | None = None, + on_runtime_admitted: Callable[[LLMRuntime], Awaitable[None]] | None = None, ) -> OutboundMessage | None: """Process an external message directly and return the outbound payload.""" if channel == "system": @@ -1897,6 +1933,8 @@ class AgentLoop: kwargs["tools"] = tools if runtime is not None: kwargs["runtime"] = runtime + if on_runtime_admitted is not None: + kwargs["on_runtime_admitted"] = on_runtime_admitted return await self._process_message( msg, **kwargs, diff --git a/nanobot/agent/model_presets.py b/nanobot/agent/model_presets.py index d6484453..67b249a3 100644 --- a/nanobot/agent/model_presets.py +++ b/nanobot/agent/model_presets.py @@ -2,7 +2,9 @@ from __future__ import annotations -from collections.abc import Callable +from collections.abc import Callable, Mapping +from dataclasses import replace +from pathlib import Path from typing import Any from nanobot.config.schema import ModelPresetConfig @@ -10,16 +12,31 @@ from nanobot.providers.base import LLMProvider from nanobot.providers.factory import ProviderSnapshot, build_provider_snapshot PresetSnapshotLoader = Callable[[str], ProviderSnapshot] +PresetCatalogLoader = Callable[[], Mapping[str, ModelPresetConfig]] -def default_selection_signature(signature: tuple[object, ...] | None) -> tuple[object, ...] | None: - return signature[:2] if signature else None +def default_selection_signature( + signature: tuple[object, ...] | None, + model_preset: str | None = None, +) -> tuple[object, ...] | None: + return (model_preset, *signature[:2]) if signature else None def configured_model_presets(config: Any) -> dict[str, ModelPresetConfig]: return {**config.model_presets, "default": config.resolve_default_preset()} +def load_model_preset_catalog( + config_path: Path | None = None, +) -> dict[str, ModelPresetConfig]: + """Load the current preset catalog from the configured file.""" + from nanobot.config.loader import load_config, resolve_config_env_vars + + return configured_model_presets( + resolve_config_env_vars(load_config(config_path)), + ) + + def make_preset_snapshot_loader( config: Any, provider_snapshot_loader: Callable[..., ProviderSnapshot] | None, @@ -40,6 +57,7 @@ def build_static_preset_snapshot( context_window_tokens=preset.context_window_tokens, signature=("model_preset", name, preset.model_dump_json()), generation=preset.to_generation_settings(), + model_preset=name, ) @@ -51,7 +69,7 @@ def build_runtime_preset_snapshot( loader: PresetSnapshotLoader | None, ) -> ProviderSnapshot: if loader is not None: - return loader(name) + return replace(loader(name), model_preset=name) return build_static_preset_snapshot(provider, name, presets[name]) diff --git a/nanobot/agent/model_runtime.py b/nanobot/agent/model_runtime.py index 6c2be2a3..0a6ece82 100644 --- a/nanobot/agent/model_runtime.py +++ b/nanobot/agent/model_runtime.py @@ -4,6 +4,7 @@ from __future__ import annotations from collections.abc import Callable, Mapping from dataclasses import replace +from types import MappingProxyType from nanobot.agent import model_presets as preset_helpers from nanobot.config.schema import Config, ModelPresetConfig @@ -24,17 +25,23 @@ class ModelRuntimeResolver: initial_runtime: LLMRuntime, *, model_presets: Mapping[str, ModelPresetConfig] | None = None, + preset_catalog_loader: preset_helpers.PresetCatalogLoader | None = None, + configured_default_preset: str | None = None, provider_snapshot_loader: Callable[[], ProviderSnapshot] | None = None, preset_snapshot_loader: preset_helpers.PresetSnapshotLoader | None = None, ) -> None: self._runtime = initial_runtime self._model_presets = dict(model_presets or {}) + self._preset_catalog_loader = preset_catalog_loader + self._preset_catalog_refresh_required = False self._provider_snapshot_loader = provider_snapshot_loader self._preset_snapshot_loader = preset_snapshot_loader self._refresh_required = False + self._resolved_presets: dict[str, LLMRuntime] = {} self._tracks_provider_generation = initial_runtime.model_preset is None self._default_selection_signature = preset_helpers.default_selection_signature( - initial_runtime.snapshot_signature + initial_runtime.snapshot_signature, + configured_default_preset, ) @property @@ -44,7 +51,11 @@ class ModelRuntimeResolver: @property def model_presets(self) -> Mapping[str, ModelPresetConfig]: - return self._model_presets + self._refresh_preset_catalog() + return MappingProxyType({ + name: preset.model_copy(deep=True) + for name, preset in self._model_presets.items() + }) @property def model_preset(self) -> str | None: @@ -71,41 +82,53 @@ class ModelRuntimeResolver: def invalidate(self) -> None: """Refresh configured runtime state on the next admission.""" self._refresh_required = True + self._preset_catalog_refresh_required = True + self._resolved_presets.clear() + + def _refresh_preset_catalog(self) -> None: + if not self._preset_catalog_refresh_required: + return + if self._preset_catalog_loader is not None: + self._model_presets = dict(self._preset_catalog_loader()) + self._preset_catalog_refresh_required = False def resolve_snapshot( self, snapshot: ProviderSnapshot, - *, - model_preset: str | None = None, ) -> LLMRuntime: """Resolve a factory snapshot without changing the selected default.""" - return runtime_from_provider_snapshot(snapshot, model_preset=model_preset) + return runtime_from_provider_snapshot(snapshot) def adopt_snapshot( self, snapshot: ProviderSnapshot, - *, - model_preset: str | None = None, ) -> LLMRuntime: """Select a snapshot as the default for future turns.""" - runtime = self.resolve_snapshot(snapshot, model_preset=model_preset) + runtime = self.resolve_snapshot(snapshot) self._runtime = runtime - self._tracks_provider_generation = model_preset is None + self._tracks_provider_generation = runtime.model_preset is None self._default_selection_signature = preset_helpers.default_selection_signature( - runtime.snapshot_signature + runtime.snapshot_signature, + runtime.model_preset, ) return runtime def resolve_preset(self, name: str | None) -> LLMRuntime: """Resolve a named preset without changing the selected default.""" + self._refresh_preset_catalog() normalized = preset_helpers.normalize_preset_name(name, self._model_presets) + cached = self._resolved_presets.get(normalized) + if cached is not None: + return cached snapshot = preset_helpers.build_runtime_preset_snapshot( name=normalized, presets=self._model_presets, provider=self._runtime.provider, loader=self._preset_snapshot_loader, ) - return self.resolve_snapshot(snapshot, model_preset=normalized) + runtime = self.resolve_snapshot(snapshot) + self._resolved_presets[normalized] = runtime + return runtime def select_preset(self, name: str | None) -> LLMRuntime: """Select a named preset as the default for future turns.""" @@ -161,13 +184,16 @@ class ModelRuntimeResolver: self._refresh_required = False return None + self._resolved_presets.clear() snapshot = self._provider_snapshot_loader() - default_selection = preset_helpers.default_selection_signature(snapshot.signature) + default_selection = preset_helpers.default_selection_signature( + snapshot.signature, + snapshot.model_preset, + ) active_preset = self._runtime.model_preset if active_preset and self._default_selection_signature in (None, default_selection): runtime = self.resolve_preset(active_preset) else: - active_preset = None runtime = self.resolve_snapshot(snapshot) unchanged = ( @@ -184,7 +210,7 @@ class ModelRuntimeResolver: self._default_selection_signature, ) = ( runtime, - active_preset is None, + runtime.model_preset is None, default_selection, ) return runtime diff --git a/nanobot/agent/tools/runtime_state.py b/nanobot/agent/tools/runtime_state.py index 48245a8c..28898869 100644 --- a/nanobot/agent/tools/runtime_state.py +++ b/nanobot/agent/tools/runtime_state.py @@ -60,5 +60,11 @@ class RuntimeState(Protocol): def set_runtime_context_window(self, context_window_tokens: int) -> Any: ... + def set_session_model_preset( + self, + session_key: str, + name: str, + ) -> Any: ... + @property def model_preset(self) -> str | None: ... diff --git a/nanobot/agent/tools/self.py b/nanobot/agent/tools/self.py index 5050d8d6..ae60f96e 100644 --- a/nanobot/agent/tools/self.py +++ b/nanobot/agent/tools/self.py @@ -3,12 +3,13 @@ from __future__ import annotations import time +from collections.abc import Mapping from typing import TYPE_CHECKING, Any from loguru import logger from nanobot.agent.tools.base import Tool, ToolResult -from nanobot.agent.tools.context import current_request_context +from nanobot.agent.tools.context import current_request_context, current_request_session_key from nanobot.agent.tools.runtime_state import RuntimeState from nanobot.config_base import Base @@ -76,6 +77,7 @@ class MyTool(Tool): "_current_iteration", # updated by runner only "exec_config", # inspect allowed (e.g. check sandbox), modify blocked "web_config", # inspect allowed (e.g. check enable), modify blocked + "model_presets", # config-derived catalog; changes require config reload "workspace_sandbox", # read-only view of workspace enforcement level "request", # current message routing metadata }) @@ -146,6 +148,8 @@ class MyTool(Tool): "max_iterations - _current_iteration = remaining iterations.\n" "Current routing metadata is available read-only via request.channel, " "request.chat_id, and request.sender_id.\n" + "Use model_preset for session-scoped model or context changes; direct " + "model/context_window_tokens writes are disabled during active sessions.\n" "Note: web_config and exec_config are readable but read-only.\n" "\n" "When to use:\n" @@ -210,11 +214,11 @@ class MyTool(Tool): if part.lower() in self._SENSITIVE_NAMES: return None, f"'{part}' is not accessible" try: - if isinstance(obj, dict): + if isinstance(obj, Mapping): if part in obj: obj = obj[part] else: - return None, f"'{part}' not found in dict" + return None, f"'{part}' not found in mapping" else: obj = getattr(obj, part) except (KeyError, AttributeError) as e: @@ -257,7 +261,7 @@ class MyTool(Tool): # SubagentManager: delegate to its _task_statuses dict if hasattr(val, "_task_statuses") and isinstance(val._task_statuses, dict): return MyTool._format_value(val._task_statuses, key) - if isinstance(val, dict) and val and _is_subagent_status(next(iter(val.values()))): + if isinstance(val, Mapping) and val and _is_subagent_status(next(iter(val.values()))): prefix = f"{key}: " if key else "" lines = [f"{prefix}{len(val)} subagent(s):"] for tid, st in val.items(): @@ -270,8 +274,8 @@ class MyTool(Tool): if isinstance(val, (str, int, float, bool, type(None))): r = repr(val) return f"{key}: {r}" if key else r - # Dict — small: show content; large: show keys for dot-path navigation - if isinstance(val, dict): + # Mapping — small: show content; large: show keys for dot-path navigation + if isinstance(val, Mapping): ks = list(val.keys()) if not ks: return f"{key}: {{}}" if key else "{}" @@ -447,6 +451,23 @@ class MyTool(Tool): if not isinstance(value, str) or not value.strip(): return ToolResult.error("Error: 'model_preset' must be a non-empty string") name = value.strip() + session_key = current_request_session_key() + if session_key: + try: + runtime = self._runtime_state.set_session_model_preset( + session_key, + name, + ) + except (KeyError, ValueError) as exc: + message = str(exc.args[0]) if exc.args else str(exc) + punctuation = "" if message.endswith((".", "!", "?")) else "." + return ToolResult.error(f"Error: {message}{punctuation}") + self._audit("modify", f"model_preset = {name!r}") + return ( + f"Set model_preset = {name!r} for the next turn; " + f"model will be {runtime.model!r}; " + f"context_window_tokens will be {runtime.context_window_tokens!r}" + ) result = self._modify_free("model_preset", name) if isinstance(result, ToolResult) and result.is_error: return result if result.endswith((".", "!", "?")) else ToolResult.error(f"{result}.") @@ -472,6 +493,11 @@ class MyTool(Tool): return ToolResult.error(f"Error: '{key}' must be <= {spec['max']}") if "min_len" in spec and len(str(value)) < spec["min_len"]: return ToolResult.error(f"Error: '{key}' must be at least {spec['min_len']} characters") + if key in {"model", "context_window_tokens"} and current_request_session_key(): + return ToolResult.error( + f"Error: direct '{key}' changes are instance-wide and disabled " + "during an active session; use a configured model_preset" + ) if key == "model": self._runtime_state.set_runtime_model(value) elif key == "context_window_tokens": diff --git a/nanobot/channels/websocket/tests/test_websocket_channel.py b/nanobot/channels/websocket/tests/test_websocket_channel.py index 226301de..7898e581 100644 --- a/nanobot/channels/websocket/tests/test_websocket_channel.py +++ b/nanobot/channels/websocket/tests/test_websocket_channel.py @@ -2942,6 +2942,7 @@ def test_sessions_list_includes_active_run_started_at(monkeypatch) -> None: "updated_at": "2026-05-19T10:01:00Z", "title": "Running", "preview": "work", + "model_preset": "fast", "path": "/private/path", }, { @@ -2978,6 +2979,7 @@ def test_sessions_list_includes_active_run_started_at(monkeypatch) -> None: "updated_at": "2026-05-19T10:01:00Z", "title": "Running", "preview": "work", + "model_preset": "fast", "run_started_at": 1_700_000_000.0, } ] diff --git a/nanobot/cli/commands.py b/nanobot/cli/commands.py index baaac49c..9b35b68b 100644 --- a/nanobot/cli/commands.py +++ b/nanobot/cli/commands.py @@ -1605,6 +1605,7 @@ def _run_gateway( health_server_enabled: bool = True, ) -> None: """Shared gateway runtime; ``open_browser_url`` opens a tab once channels are up.""" + from nanobot.agent.model_presets import load_model_preset_catalog from nanobot.agent.tools.message import MessageTool from nanobot.agent.turn_delivery import TurnDeliveryFactory from nanobot.bus.queue import MessageBus @@ -1696,6 +1697,7 @@ def _run_gateway( session_manager=session_manager, image_generation_provider_configs=image_gen_provider_configs(config), provider_snapshot_loader=load_provider_snapshot, + preset_catalog_loader=load_model_preset_catalog, runtime_events=runtime_events, turn_delivery_factory=turn_delivery_factory, provider_signature=provider_snapshot.signature, @@ -2086,7 +2088,7 @@ def _run_gateway( asyncio.create_task( watch_config_file( Path(config_path), - lambda: agent.runtime_resolver.invalidate(), + lambda: agent.invalidate_runtime_config(), ), name="nanobot-config-watcher", ), diff --git a/nanobot/command/builtin.py b/nanobot/command/builtin.py index db788080..11d17ef3 100644 --- a/nanobot/command/builtin.py +++ b/nanobot/command/builtin.py @@ -237,7 +237,7 @@ async def cmd_status(ctx: CommandContext) -> OutboundMessage: """Build an outbound status message for a session.""" loop = ctx.loop session = ctx.session or loop.sessions.get_or_create(ctx.key) - runtime = ctx.runtime or loop.llm_runtime() + runtime = ctx.runtime or loop.runtime_for_session(session) ctx_est = 0 with suppress(Exception): ctx_est, _ = loop.consolidator.estimate_session_prompt_tokens( @@ -286,11 +286,12 @@ async def cmd_new(ctx: CommandContext) -> OutboundMessage: await loop._cancel_active_tasks(ctx.key) session = ctx.session or loop.sessions.get_or_create(ctx.key) snapshot = session.messages[session.last_consolidated:] + if snapshot: + runtime = ctx.runtime or loop.runtime_for_session(session) session.clear() loop.sessions.save(session) loop.sessions.invalidate(session.key) if snapshot: - runtime = ctx.runtime or loop.llm_runtime() loop._schedule_background( loop.consolidator.archive( snapshot, @@ -315,20 +316,25 @@ def _model_preset_names(loop) -> list[str]: return ["default", *sorted(name for name in names if name != "default")] -def _active_model_preset_name(loop) -> str: - return loop.model_preset or "default" - - def _command_error_message(exc: Exception) -> str: return str(exc.args[0]) if isinstance(exc, KeyError) and exc.args else str(exc) -def _model_command_status(loop) -> str: +def _model_command_status(loop, session) -> str: names = _model_preset_names(loop) - active = _active_model_preset_name(loop) + try: + runtime = loop.runtime_for_session(session) + except (KeyError, ValueError) as exc: + return "\n".join([ + "## Model", + f"- Current selection error: {_command_error_message(exc)}", + f"- Available presets: {_format_preset_names(names)}", + "- Switch with `/model `.", + ]) + active = runtime.model_preset or "default" return "\n".join([ "## Model", - f"- Current model: `{loop.model}`", + f"- Current model: `{runtime.model}`", f"- Current preset: `{active}`", f"- Available presets: {_format_preset_names(names)}", ]) @@ -341,10 +347,11 @@ async def cmd_model(ctx: CommandContext) -> OutboundMessage: metadata = {**dict(ctx.msg.metadata or {}), "render_as": "text"} if not args: + session = ctx.session or loop.sessions.get_or_create(ctx.key) return OutboundMessage( channel=ctx.msg.channel, chat_id=ctx.msg.chat_id, - content=_model_command_status(loop), + content=_model_command_status(loop, session), metadata=metadata, ) @@ -359,7 +366,7 @@ async def cmd_model(ctx: CommandContext) -> OutboundMessage: name = parts[0] try: - runtime = loop.set_model_preset(name) + runtime = loop.set_session_model_preset(ctx.key, name) except (KeyError, ValueError) as exc: names = _model_preset_names(loop) return OutboundMessage( @@ -375,6 +382,7 @@ async def cmd_model(ctx: CommandContext) -> OutboundMessage: max_tokens = runtime.generation.max_tokens lines = [ f"Switched model preset to `{runtime.model_preset}`.", + "- Scope: current session", f"- Model: `{runtime.model}`", f"- Context window: {runtime.context_window_tokens}", ] diff --git a/nanobot/nanobot.py b/nanobot/nanobot.py index aef50bac..9db9ab53 100644 --- a/nanobot/nanobot.py +++ b/nanobot/nanobot.py @@ -37,6 +37,7 @@ from nanobot.sdk.types import ( StreamEventType, result_from_response, ) +from nanobot.utils.llm_runtime import LLMRuntime __all__ = [ "Nanobot", @@ -192,16 +193,40 @@ class Nanobot: model_preset: str | None = None, ) -> RunStream: """Start a streamed run and return a handle for events and final result.""" - runtime = self._loop.runtime_resolver.resolve_override( + override_runtime = self._loop.runtime_resolver.resolve_override( model=model, model_preset=model_preset, config=self._config, - ) or self._loop.llm_runtime() + ) queue: asyncio.Queue[StreamEvent | object] = asyncio.Queue(maxsize=256) emitter = SDKStreamEmitter(queue) stream_hook = SDKStreamingHook(emitter) capture = SDKCaptureHook() per_run_hooks = [capture, stream_hook, *(hooks or [])] + run_started = False + + async def _emit_run_started(runtime: LLMRuntime | None = None) -> None: + nonlocal run_started + if run_started: + return + if runtime is None: + runtime = override_runtime + metadata: dict[str, Any] = { + "session_key": session_key, + "channel": channel, + "chat_id": chat_id, + "sender_id": sender_id, + } + if runtime is not None: + metadata.update({ + "model": runtime.model, + "model_preset": runtime.model_preset, + }) + await emitter.emit(StreamEvent( + type=STREAM_EVENT_RUN_STARTED, + metadata=metadata, + )) + run_started = True async def _on_stream(delta: str) -> None: await emitter.text_delta(delta) @@ -220,24 +245,16 @@ class Nanobot: on_stream=_on_stream, on_stream_end=_on_stream_end, ) - kwargs["runtime"] = runtime - await emitter.emit(StreamEvent( - type=STREAM_EVENT_RUN_STARTED, - metadata={ - "session_key": session_key, - "channel": channel, - "chat_id": chat_id, - "sender_id": sender_id, - "model": runtime.model, - "model_preset": runtime.model_preset, - }, - )) + kwargs["on_runtime_admitted"] = _emit_run_started + if override_runtime is not None: + kwargs["runtime"] = override_runtime try: response = await self._loop.process_direct( message, **kwargs, hooks=per_run_hooks, ) + await _emit_run_started() await emitter.text_completed(resuming=False, force=False) result = result_from_response(response, capture) await emitter.emit(StreamEvent( @@ -249,6 +266,7 @@ class Nanobot: )) return result except Exception as exc: + await _emit_run_started() await emitter.emit(StreamEvent( type=STREAM_EVENT_RUN_FAILED, error=str(exc), diff --git a/nanobot/providers/factory.py b/nanobot/providers/factory.py index 4b56f054..ead2e023 100644 --- a/nanobot/providers/factory.py +++ b/nanobot/providers/factory.py @@ -18,6 +18,7 @@ class ProviderSnapshot: context_window_tokens: int signature: tuple[object, ...] generation: GenerationSettings | None = None + model_preset: str | None = None def _resolve_model_preset( @@ -261,6 +262,11 @@ def build_provider_snapshot( preset: ModelPresetConfig | None = None, ) -> ProviderSnapshot: resolved = _resolve_model_preset(config, preset_name=preset_name, preset=preset) + selected_preset = ( + config.agents.defaults.model_preset + if preset_name is None and preset is None + else preset_name + ) fallback_windows = [ fallback.context_window_tokens for fallback in _resolve_fallback_presets(config, resolved) @@ -271,6 +277,7 @@ def build_provider_snapshot( context_window_tokens=min([resolved.context_window_tokens, *fallback_windows]), signature=provider_signature(config, preset=resolved), generation=resolved.to_generation_settings(), + model_preset=selected_preset, ) diff --git a/nanobot/sdk/clients.py b/nanobot/sdk/clients.py index be4bea86..04ee1181 100644 --- a/nanobot/sdk/clients.py +++ b/nanobot/sdk/clients.py @@ -196,7 +196,7 @@ class RuntimeClient: async def compact_session(self, session_key: str) -> SessionSnapshot: """Run token/replay-window consolidation for one session.""" session = self._loop.sessions.get_or_create(session_key) - runtime = self._loop.llm_runtime() + runtime = self._loop.runtime_for_session(session) await self._loop.consolidator.maybe_consolidate_by_tokens( session, runtime=runtime, @@ -208,7 +208,8 @@ class RuntimeClient: async def compact_idle_session(self, session_key: str, *, max_suffix: int = 8) -> str | None: """Run idle-session compaction for one session and return the summary.""" - runtime = self._loop.llm_runtime() + session = self._loop.sessions.get_or_create(session_key) + runtime = self._loop.runtime_for_session(session) return await self._loop.consolidator.compact_idle_session( session_key, runtime=runtime, diff --git a/nanobot/session/model_selection.py b/nanobot/session/model_selection.py new file mode 100644 index 00000000..bf5be146 --- /dev/null +++ b/nanobot/session/model_selection.py @@ -0,0 +1,20 @@ +"""Session-scoped model preset metadata.""" + +from __future__ import annotations + +from collections.abc import Mapping + +# Session.metadata is public SDK data, so internal selectors use a reserved namespace. +SESSION_MODEL_PRESET_METADATA_KEY = "_nanobot_model_preset" + + +def model_preset_from_metadata(metadata: object) -> str | None: + """Read the canonical session preset name from persisted metadata.""" + if not isinstance(metadata, Mapping): + return None + if SESSION_MODEL_PRESET_METADATA_KEY not in metadata: + return None + value = metadata[SESSION_MODEL_PRESET_METADATA_KEY] + if not isinstance(value, str) or not value.strip(): + raise ValueError("session model preset must be a non-empty string") + return value.strip() diff --git a/nanobot/skills/my/SKILL.md b/nanobot/skills/my/SKILL.md index b420c1bf..930bd3a9 100644 --- a/nanobot/skills/my/SKILL.md +++ b/nanobot/skills/my/SKILL.md @@ -34,9 +34,8 @@ description: Inspect and optionally adjust the agent's runtime state. Use to che | Situation | Command | |-----------|---------| -| Large codebase analysis | `my(action="set", key="context_window_tokens", value=262144)` | -| Switch to a named model preset | `my(action="set", key="model_preset", value="")` | -| Repetitive simple tasks without a preset | `my(action="set", key="model", value="")` | +| Check capacity for a large task | `my(action="check", key="context_window_tokens")` | +| Switch model or context capacity | `my(action="set", key="model_preset", value="")` | | Long multi-step task | `my(action="set", key="max_iterations", value=80)` | **Tradeoff:** Bias toward stability. Only set when defaults are genuinely insufficient. @@ -57,8 +56,8 @@ description: Inspect and optionally adjust the agent's runtime state. Use to che ## Constraints -- All modifications in-memory only — restart resets everything -- Prefer `model_preset` for configured model choices. Direct `model` changes clear the active preset and should only be used when no preset exists. +- `model_preset` is saved for the current session; other modifications are in-memory only +- Direct `model` and `context_window_tokens` writes are rejected during active sessions because they would change the shared instance default. Use a configured `model_preset` instead. - Protected params have type/range validation: `max_iterations` (1–100), `context_window_tokens` (4096–1M), `model` (non-empty str) - If `tools.my.allow_set` is false, check only diff --git a/nanobot/skills/my/references/examples.md b/nanobot/skills/my/references/examples.md index 9ef73943..961813d9 100644 --- a/nanobot/skills/my/references/examples.md +++ b/nanobot/skills/my/references/examples.md @@ -34,23 +34,16 @@ Concrete scenarios showing when and how to use the my tool effectively. ``` → my(action="check") → context_window_tokens: 200000 -→ my(action="set", key="context_window_tokens", value=262144) - → "Set context_window_tokens = 262144 (was 200000)" -→ "I've expanded my context window to handle this large codebase." +→ my(action="set", key="model_preset", value="deep") + → "Set model_preset = 'deep' for the next turn; context_window_tokens will be 262144" +→ "I've selected the configured deep preset for this session's next turn." ``` ### Switching to a configured model preset ``` → my(action="set", key="model_preset", value="fast") - → "Set model_preset = 'fast' (was 'deep'); model is now 'openai/gpt-4.1-mini'" -→ "Switched to the fast preset for these batch tasks." -``` - -### Switching to a raw model when no preset exists -``` -→ my(action="set", key="model", value="anthropic/claude-haiku-4-5-20251001") - → "Set model = 'anthropic/claude-haiku-4-5-20251001' (was 'anthropic/claude-sonnet-4-6')" -→ "Switched to a faster model for these batch tasks." + → "Set model_preset = 'fast' for the next turn; model will be 'openai/gpt-4.1-mini'" +→ "Selected the fast preset for this session's next turn." ``` ## Cross-Turn Memory diff --git a/nanobot/utils/llm_runtime.py b/nanobot/utils/llm_runtime.py index 89fdf7f9..405b96b1 100644 --- a/nanobot/utils/llm_runtime.py +++ b/nanobot/utils/llm_runtime.py @@ -84,8 +84,6 @@ class LLMRuntime: def runtime_from_provider_snapshot( snapshot: ProviderSnapshot, - *, - model_preset: str | None = None, ) -> LLMRuntime: """Convert a provider factory snapshot into the canonical runtime value.""" if snapshot.generation is not None: @@ -94,13 +92,13 @@ def runtime_from_provider_snapshot( model=snapshot.model, generation=snapshot.generation, context_window_tokens=snapshot.context_window_tokens, - model_preset=model_preset, + model_preset=snapshot.model_preset, snapshot_signature=snapshot.signature, ) return LLMRuntime.capture( snapshot.provider, snapshot.model, context_window_tokens=snapshot.context_window_tokens, - model_preset=model_preset, + model_preset=snapshot.model_preset, snapshot_signature=snapshot.signature, ) diff --git a/nanobot/webui/session_list_index.py b/nanobot/webui/session_list_index.py index 545173c2..c0689a43 100644 --- a/nanobot/webui/session_list_index.py +++ b/nanobot/webui/session_list_index.py @@ -25,9 +25,11 @@ from nanobot.session.manager import ( _message_preview_text, _metadata_title, ) +from nanobot.session.model_selection import model_preset_from_metadata -_INDEX_VERSION = 2 +_INDEX_VERSION = 4 _INDEX_FILENAME = ".webui_session_index.json" +_MODEL_PRESET_FIELD = "model_preset" _WEBUI_ACTIVITY_MTIME_NS = "webui_activity_mtime_ns" _WEBUI_ACTIVITY_SIZE = "webui_activity_size" _VISIBLE_TRANSCRIPT_ROLES = {"user", "assistant"} @@ -138,6 +140,7 @@ def _public_row(sessions_dir: Path, row: dict[str, Any]) -> dict[str, Any]: "updated_at": row.get("updated_at"), "title": row.get("title", ""), "preview": row.get("preview", ""), + _MODEL_PRESET_FIELD: row.get(_MODEL_PRESET_FIELD), "path": str(sessions_dir / str(row.get("file", ""))), } @@ -256,6 +259,7 @@ def _indexed_row_for_session(session: Session, path: Path) -> dict[str, Any]: ), "title": _metadata_title(session.metadata), "preview": _preview_from_messages(session.messages), + _MODEL_PRESET_FIELD: model_preset_from_metadata(session.metadata), "file": path.name, "mtime_ns": signature["mtime_ns"], "size": signature["size"], @@ -329,6 +333,7 @@ def _scan_session_row(session_manager: SessionManager, path: Path) -> dict[str, ), "title": _metadata_title(data.get("metadata", {})), "preview": preview or fallback_preview, + _MODEL_PRESET_FIELD: model_preset_from_metadata(data.get("metadata", {})), "file": path.name, "mtime_ns": signature["mtime_ns"], "size": signature["size"], diff --git a/nanobot/webui/settings_api.py b/nanobot/webui/settings_api.py index 7d3880a2..113b26e1 100644 --- a/nanobot/webui/settings_api.py +++ b/nanobot/webui/settings_api.py @@ -805,6 +805,10 @@ def settings_payload( "is_default": True, "model": defaults.model, "provider": defaults.provider, + "resolved_provider": config.get_provider_name( + defaults.model, + preset=config.resolve_default_preset(), + ), "max_tokens": defaults.max_tokens, "context_window_tokens": defaults.context_window_tokens, "temperature": defaults.temperature, @@ -823,6 +827,10 @@ def settings_payload( "is_default": False, "model": preset.model, "provider": preset.provider, + "resolved_provider": config.get_provider_name( + preset.model, + preset=preset, + ), "max_tokens": preset.max_tokens, "context_window_tokens": preset.context_window_tokens, "temperature": preset.temperature, diff --git a/tests/agent/test_auto_compact.py b/tests/agent/test_auto_compact.py index 34bbf04d..35d96186 100644 --- a/tests/agent/test_auto_compact.py +++ b/tests/agent/test_auto_compact.py @@ -307,7 +307,7 @@ class TestAutoCompact: loop.sessions.save(s2) loop.consolidator.compact_idle_session = _make_fake_compact(loop) - loop.auto_compact.check_expired(loop._schedule_background, loop.llm_runtime) + loop.auto_compact.check_expired(loop._schedule_background, loop.runtime_for_session) await _drain_background_tasks(loop) active_after = loop.sessions.get_or_create("cli:active") @@ -776,7 +776,7 @@ class TestProactiveAutoCompact: """Helper: run check_expired via callback and wait for background tasks.""" loop.auto_compact.check_expired( loop._schedule_background, - loop.llm_runtime, + loop.runtime_for_session, active_session_keys=active_session_keys, ) await _drain_background_tasks(loop) @@ -878,12 +878,12 @@ class TestProactiveAutoCompact: loop.consolidator.compact_idle_session = _slow_compact # First call starts archiving via callback - loop.auto_compact.check_expired(loop._schedule_background, loop.llm_runtime) + loop.auto_compact.check_expired(loop._schedule_background, loop.runtime_for_session) await started.wait() assert archive_count == 1 # Second call should skip (key is in _archiving) - loop.auto_compact.check_expired(loop._schedule_background, loop.llm_runtime) + loop.auto_compact.check_expired(loop._schedule_background, loop.runtime_for_session) assert archive_count == 1 # Clean up diff --git a/tests/agent/test_autocompact_unit.py b/tests/agent/test_autocompact_unit.py index ec610aba..75565368 100644 --- a/tests/agent/test_autocompact_unit.py +++ b/tests/agent/test_autocompact_unit.py @@ -9,7 +9,7 @@ from nanobot.agent.autocompact import AutoCompact from nanobot.session.manager import Session, SessionManager -def _runtime(): +def _runtime(_session: Session | None = None): return MagicMock(name="runtime") @@ -240,13 +240,62 @@ class TestCheckExpired: resolve_runtime.return_value = replacement await scheduled[0] - resolve_runtime.assert_called_once_with() + resolve_runtime.assert_called_once_with(session) ac.consolidator.compact_idle_session.assert_awaited_once_with( "cli:old", runtime=admitted, max_suffix=ac._RECENT_SUFFIX_MESSAGES, ) + @pytest.mark.parametrize("resolution_error", [KeyError, ValueError]) + def test_invalid_preset_is_isolated_to_one_session(self, resolution_error): + ac = _make_autocompact(ttl=15) + old_dt = datetime.now() - timedelta(minutes=20) + sessions = { + key: _make_session(key, updated_at=old_dt) + for key in ("cli:removed", "cli:healthy") + } + for session in sessions.values(): + _add_turns(session, 5) + ac.sessions.list_sessions.return_value = [ + {"key": key, "updated_at": old_dt.isoformat()} + for key in sessions + ] + ac.sessions.get_or_create.side_effect = sessions.__getitem__ + healthy_runtime = _runtime() + + def resolve_runtime(session: Session): + if session.key == "cli:removed": + raise resolution_error("model preset cannot be resolved") + return healthy_runtime + + scheduled = [] + + def scheduler(coro): + scheduled.append(coro) + coro.close() + + ac.check_expired(scheduler, resolve_runtime) + + assert len(scheduled) == 1 + assert ac._archiving == {"cli:healthy"} + + def test_unexpected_runtime_resolution_failure_propagates(self): + ac = _make_autocompact(ttl=15) + old_dt = datetime.now() - timedelta(minutes=20) + session = _make_session("cli:old", updated_at=old_dt) + _add_turns(session, 5) + ac.sessions.list_sessions.return_value = [ + {"key": session.key, "updated_at": old_dt.isoformat()} + ] + ac.sessions.get_or_create.return_value = session + + def fail(_session: Session): + raise RuntimeError("unexpected resolver failure") + + with pytest.raises(RuntimeError, match="unexpected resolver failure"): + ac.check_expired(MagicMock(), fail) + def test_active_session_key_skips(self): """Session in active_session_keys should be skipped.""" ac = _make_autocompact(ttl=15) diff --git a/tests/agent/test_model_runtime_resolver.py b/tests/agent/test_model_runtime_resolver.py index b6454af0..d3aa7dac 100644 --- a/tests/agent/test_model_runtime_resolver.py +++ b/tests/agent/test_model_runtime_resolver.py @@ -52,9 +52,10 @@ def test_provider_snapshot_has_one_canonical_runtime_conversion() -> None: model="snapshot-model", context_window_tokens=32_768, signature=("snapshot-model", "openai"), + model_preset="fast", ) - runtime = runtime_from_provider_snapshot(snapshot, model_preset="fast") + runtime = runtime_from_provider_snapshot(snapshot) assert runtime.provider is provider assert runtime.model == "snapshot-model" @@ -94,6 +95,101 @@ def test_resolver_resolves_preset_without_mutating_selected_runtime() -> None: assert resolved.generation == GenerationSettings(0.5, 512, None) +def test_resolver_reuses_preset_until_runtime_config_is_invalidated() -> None: + initial = _runtime() + preset = ModelPresetConfig(model="fast-model") + load_count = 0 + preset_signature = ("fast-model", "auto", "initial") + + def load_preset(_name: str) -> ProviderSnapshot: + nonlocal load_count + load_count += 1 + return ProviderSnapshot( + provider=_provider(), + model="fast-model", + context_window_tokens=20_000, + signature=preset_signature, + ) + + resolver = ModelRuntimeResolver( + initial, + model_presets={"fast": preset}, + preset_snapshot_loader=load_preset, + ) + + first = resolver.resolve_preset("fast") + second = resolver.resolve_preset("fast") + + assert first is second + assert load_count == 1 + + preset_signature = ("fast-model", "auto", "new-credential") + resolver.invalidate() + refreshed = resolver.resolve_preset("fast") + + assert refreshed is not first + assert load_count == 2 + + +def test_resolver_refreshes_preset_catalog_after_invalidation() -> None: + provider = _provider() + catalog = { + "old": ModelPresetConfig(model="old-model", provider="openai"), + } + default_name = "old" + + def load_preset(name: str) -> ProviderSnapshot: + preset = catalog[name] + return ProviderSnapshot( + provider=provider, + model=preset.model, + context_window_tokens=preset.context_window_tokens, + signature=(preset.model, preset.provider), + model_preset=name, + ) + + resolver = ModelRuntimeResolver( + runtime_from_provider_snapshot(load_preset("old")), + model_presets=catalog, + preset_catalog_loader=lambda: catalog, + configured_default_preset="old", + provider_snapshot_loader=lambda: load_preset(default_name), + preset_snapshot_loader=load_preset, + ) + + catalog["new"] = ModelPresetConfig(model="new-model", provider="openai") + default_name = "new" + resolver.invalidate() + + assert resolver.admit().model_preset == "new" + assert set(resolver.model_presets) == {"old", "new"} + + del catalog["old"] + resolver.invalidate() + + assert resolver.admit().model_preset == "new" + assert set(resolver.model_presets) == {"new"} + + +def test_resolver_model_presets_are_read_only() -> None: + resolver = ModelRuntimeResolver( + _runtime(), + model_presets={"fast": ModelPresetConfig(model="fast-model")}, + ) + + exposed = resolver.model_presets + with pytest.raises(TypeError): + exposed["other"] = ModelPresetConfig( # type: ignore[index] + model="other-model" + ) + + exposed["fast"].model = "mutated-model" + + assert set(resolver.model_presets) == {"fast"} + assert resolver.model_presets["fast"].model == "fast-model" + assert resolver.resolve_preset("fast").model == "fast-model" + + def test_resolver_model_override_is_derived_without_default_mutation() -> None: initial = _runtime() resolver = ModelRuntimeResolver(initial) diff --git a/tests/agent/test_runtime_refresh.py b/tests/agent/test_runtime_refresh.py index e18eba9e..c722824e 100644 --- a/tests/agent/test_runtime_refresh.py +++ b/tests/agent/test_runtime_refresh.py @@ -1,3 +1,4 @@ +import asyncio from pathlib import Path from types import SimpleNamespace from unittest.mock import MagicMock @@ -6,10 +7,15 @@ import pytest from nanobot.agent.loop import AgentLoop from nanobot.bus.queue import MessageBus +from nanobot.bus.runtime_events import RuntimeModelChanged from nanobot.config.loader import save_config from nanobot.config.schema import Config, ModelPresetConfig from nanobot.providers.base import GenerationSettings from nanobot.providers.factory import ProviderSnapshot, load_provider_snapshot +from nanobot.session.model_selection import ( + SESSION_MODEL_PRESET_METADATA_KEY, + model_preset_from_metadata, +) from nanobot.webui.settings_api import update_agent_settings @@ -153,6 +159,99 @@ def test_same_snapshot_default_clears_preset_and_publishes_update(tmp_path: Path assert published == [("fast-model", None)] +def test_named_default_refresh_is_used_by_sessions_without_override(tmp_path: Path) -> None: + provider = _provider("shared-model") + shared_signature = ("shared-model", "auto", "same-settings") + snapshots = { + "fast": ProviderSnapshot( + provider=provider, + model="shared-model", + context_window_tokens=16_000, + signature=shared_signature, + model_preset="fast", + ), + "deep": ProviderSnapshot( + provider=provider, + model="shared-model", + context_window_tokens=16_000, + signature=shared_signature, + model_preset="deep", + ), + } + default_snapshot = snapshots["deep"] + loop = AgentLoop( + bus=MessageBus(), + provider=provider, + workspace=tmp_path, + model="shared-model", + context_window_tokens=16_000, + provider_signature=shared_signature, + provider_snapshot_loader=lambda: default_snapshot, + model_presets={ + name: ModelPresetConfig(model=snapshot.model) + for name, snapshot in snapshots.items() + }, + model_preset="fast", + preset_snapshot_loader=snapshots.__getitem__, + ) + + loop.runtime_resolver.invalidate() + runtime = loop.llm_runtime() + session = loop.sessions.get_or_create("sdk:new-after-refresh") + + assert runtime.model_preset == "deep" + assert loop.runtime_for_session(session).model_preset == "deep" + assert model_preset_from_metadata(session.metadata) is None + + +@pytest.mark.asyncio +async def test_config_invalidation_notifies_clients_before_session_runtime_refresh( + tmp_path: Path, +) -> None: + provider = _provider("model-a") + catalog = {"fast": ModelPresetConfig(model="model-a")} + current_model = "model-a" + published: list[RuntimeModelChanged] = [] + + def load_preset(_name: str) -> ProviderSnapshot: + return ProviderSnapshot( + provider=provider, + model=current_model, + context_window_tokens=16_000, + signature=(current_model, "auto"), + model_preset="fast", + ) + + loop = AgentLoop( + bus=MessageBus(), + provider=provider, + workspace=tmp_path, + model="model-a", + context_window_tokens=16_000, + provider_signature=("model-a", "auto"), + provider_snapshot_loader=lambda: load_preset("fast"), + model_presets=catalog, + preset_catalog_loader=lambda: catalog, + model_preset="fast", + preset_snapshot_loader=load_preset, + ) + loop.runtime_events.subscribe(published.append, RuntimeModelChanged) + session = loop.sessions.get_or_create("websocket:chat") + session.metadata[SESSION_MODEL_PRESET_METADATA_KEY] = "fast" + current_model = "model-b" + catalog["fast"] = ModelPresetConfig(model="model-b") + + loop.invalidate_runtime_config() + runtime = loop.runtime_for_session(session) + await asyncio.sleep(0) + + assert [(event.model, event.model_preset) for event in published] == [ + ("model-a", "fast"), + ] + assert runtime.model == "model-b" + assert loop.model_presets["fast"].model == "model-b" + + def test_next_turn_captures_generation_changed_after_previous_admission( tmp_path: Path, ) -> None: diff --git a/tests/agent/test_self_model_preset.py b/tests/agent/test_self_model_preset.py index 59423b17..f0a1e107 100644 --- a/tests/agent/test_self_model_preset.py +++ b/tests/agent/test_self_model_preset.py @@ -4,10 +4,12 @@ from unittest.mock import MagicMock import pytest from nanobot.agent.loop import AgentLoop +from nanobot.agent.tools.context import RequestContext, request_context from nanobot.agent.tools.self import MyTool from nanobot.bus.queue import MessageBus from nanobot.config.schema import ModelPresetConfig from nanobot.providers.factory import ProviderSnapshot +from nanobot.session.model_selection import model_preset_from_metadata def _provider(default_model: str, max_tokens: int = 123) -> MagicMock: @@ -287,6 +289,78 @@ def test_self_tool_set_model_preset_unknown_lists_available(tmp_path) -> None: assert loop.model == "base-model" +def test_self_tool_sets_model_preset_for_current_session(tmp_path) -> None: + presets = { + "default": ModelPresetConfig(model="base-model"), + "fast": ModelPresetConfig(model="openai/gpt-4.1"), + } + loop = _make_loop(tmp_path, presets=presets) + tool = MyTool(runtime_state=loop, modify_allowed=True) + + with request_context(RequestContext( + channel="cli", + chat_id="one", + session_key="cli:one", + metadata={"source": "self-tool"}, + )): + result = tool._modify("model_preset", "fast") + + assert "for the next turn" in result + assert model_preset_from_metadata( + loop.sessions.get_or_create("cli:one").metadata + ) == "fast" + assert loop.model_preset is None + assert loop.model == "base-model" + + +def test_self_tool_reports_session_preset_provider_configuration_error(tmp_path) -> None: + loop = _make_loop(tmp_path) + loop.set_session_model_preset = MagicMock( + side_effect=ValueError("No API key configured for provider 'openai'.") + ) + tool = MyTool(runtime_state=loop, modify_allowed=True) + + with request_context(RequestContext( + channel="cli", + chat_id="one", + session_key="cli:one", + )): + result = tool._modify("model_preset", "broken") + + assert result == "Error: No API key configured for provider 'openai'." + + +@pytest.mark.parametrize( + ("key", "value"), + [ + ("model", "other-model"), + ("context_window_tokens", 8_192), + ], +) +def test_self_tool_rejects_instance_runtime_changes_in_session( + tmp_path, + key: str, + value: object, +) -> None: + loop = _make_loop(tmp_path) + tool = MyTool(runtime_state=loop, modify_allowed=True) + session = loop.sessions.get_or_create("cli:one") + + with request_context(RequestContext( + channel="cli", + chat_id="one", + session_key=session.key, + runtime=loop.runtime_for_session(session), + )): + result = tool._modify(key, value) + + other_runtime = loop.runtime_for_session(loop.sessions.get_or_create("cli:two")) + assert "instance-wide and disabled" in result + assert "model_preset" in result + assert other_runtime.model == "base-model" + assert other_runtime.context_window_tokens == 1000 + + def test_self_tool_set_model_clears_active_preset(tmp_path) -> None: presets = { "fast": ModelPresetConfig(model="openai/gpt-4.1"), diff --git a/tests/agent/test_session_model_runtime.py b/tests/agent/test_session_model_runtime.py new file mode 100644 index 00000000..aff77abe --- /dev/null +++ b/tests/agent/test_session_model_runtime.py @@ -0,0 +1,228 @@ +import asyncio + +import pytest + +from nanobot.agent.loop import AgentLoop +from nanobot.bus.queue import MessageBus +from nanobot.config.schema import ModelPresetConfig +from nanobot.nanobot import Nanobot +from nanobot.providers.base import GenerationSettings, LLMProvider, LLMResponse +from nanobot.providers.factory import ProviderSnapshot +from nanobot.sdk.types import SessionSnapshot +from nanobot.session.model_selection import ( + SESSION_MODEL_PRESET_METADATA_KEY, + model_preset_from_metadata, +) +from nanobot.utils.llm_runtime import LLMRuntime + + +class RecordingProvider(LLMProvider): + def __init__(self, name: str) -> None: + super().__init__() + self.name = name + self.generation = GenerationSettings(max_tokens=256, temperature=0.1) + self.calls: list[str | None] = [] + + async def chat(self, messages, tools=None, model=None, **kwargs): + await asyncio.sleep(0) + self.calls.append(model) + return LLMResponse(content=f"reply from {self.name}", finish_reason="stop") + + def get_default_model(self) -> str: + return self.name + + +@pytest.mark.asyncio +async def test_sessions_run_concurrently_with_isolated_model_presets(tmp_path) -> None: + base = RecordingProvider("base-model") + fast = RecordingProvider("fast-model") + deep = RecordingProvider("deep-model") + providers = {"fast": fast, "deep": deep} + load_counts = {"fast": 0, "deep": 0} + presets = { + "default": ModelPresetConfig(model="base-model", context_window_tokens=8_000), + "fast": ModelPresetConfig(model="fast-model", context_window_tokens=16_000), + "deep": ModelPresetConfig(model="deep-model", context_window_tokens=32_000), + } + + def load_preset(name: str) -> ProviderSnapshot: + load_counts[name] += 1 + preset = presets[name] + provider = base if name == "default" else providers[name] + return ProviderSnapshot( + provider=provider, + model=preset.model, + context_window_tokens=preset.context_window_tokens, + signature=(name, preset.model), + ) + + loop = AgentLoop( + bus=MessageBus(), + provider=base, + workspace=tmp_path, + model="base-model", + context_window_tokens=8_000, + model_presets=presets, + preset_snapshot_loader=load_preset, + ) + loop._schedule_background = lambda coro: coro.close() # type: ignore[method-assign] + loop.set_session_model_preset("sdk:fast", "fast") + loop.set_session_model_preset("sdk:deep", "deep") + + fast_reply, deep_reply = await asyncio.gather( + loop.process_direct("hello", session_key="sdk:fast"), + loop.process_direct("hello", session_key="sdk:deep"), + ) + + assert fast_reply is not None and fast_reply.content == "reply from fast-model" + assert deep_reply is not None and deep_reply.content == "reply from deep-model" + assert fast.calls == ["fast-model"] + assert deep.calls == ["deep-model"] + assert base.calls == [] + assert loop.provider is base + assert loop.model == "base-model" + assert load_counts == {"fast": 1, "deep": 1} + + loop.sessions.invalidate("sdk:fast") + restored = loop.sessions.get_or_create("sdk:fast") + assert model_preset_from_metadata(restored.metadata) == "fast" + + override = RecordingProvider("override-model") + override_runtime = LLMRuntime.capture( + override, + "override-model", + context_window_tokens=24_000, + ) + override_reply = await loop.process_direct( + "hello", + session_key="sdk:fast", + runtime=override_runtime, + ) + + assert override_reply is not None + assert override_reply.content == "reply from override-model" + assert override.calls == ["override-model"] + assert fast.calls == ["fast-model"] + assert load_counts == {"fast": 1, "deep": 1} + + +@pytest.mark.asyncio +async def test_streamed_sdk_resolves_session_runtime_after_lock_admission(tmp_path) -> None: + base = RecordingProvider("base-model") + fast = RecordingProvider("fast-model") + deep = RecordingProvider("deep-model") + providers = {"fast": fast, "deep": deep} + presets = { + "fast": ModelPresetConfig(model="fast-model", context_window_tokens=16_000), + "deep": ModelPresetConfig(model="deep-model", context_window_tokens=32_000), + } + + def load_preset(name: str) -> ProviderSnapshot: + preset = presets[name] + return ProviderSnapshot( + provider=providers[name], + model=preset.model, + context_window_tokens=preset.context_window_tokens, + signature=(name, preset.model), + ) + + loop = AgentLoop( + bus=MessageBus(), + provider=base, + workspace=tmp_path, + model="base-model", + context_window_tokens=8_000, + model_presets=presets, + preset_snapshot_loader=load_preset, + ) + loop._schedule_background = lambda coro: coro.close() # type: ignore[method-assign] + session_key = "sdk:queued" + loop.set_session_model_preset(session_key, "fast") + + lock = loop._session_locks.setdefault(session_key, asyncio.Lock()) + await lock.acquire() + try: + run = await Nanobot(loop).run_streamed("hello", session_key=session_key) + loop.set_session_model_preset(session_key, "deep") + finally: + lock.release() + + events = [event async for event in run.stream_events()] + result = await run.wait() + + assert result.content == "reply from deep-model" + assert fast.calls == [] + assert deep.calls == ["deep-model"] + assert events[0].type == "run.started" + assert events[0].metadata["model"] == "deep-model" + assert events[0].metadata["model_preset"] == "deep" + + +@pytest.mark.parametrize("custom_value", ["legacy-tag", 7]) +@pytest.mark.asyncio +async def test_sdk_custom_model_preset_metadata_does_not_select_runtime( + tmp_path, + custom_value, +) -> None: + base = RecordingProvider("base-model") + loop = AgentLoop( + bus=MessageBus(), + provider=base, + workspace=tmp_path, + model="base-model", + context_window_tokens=8_000, + ) + loop._schedule_background = lambda coro: coro.close() # type: ignore[method-assign] + bot = Nanobot(loop) + + await bot.sessions.ingest( + "sdk:custom-metadata", + [], + metadata={"model_preset": custom_value}, + ) + ingested_result = await bot.run("hello", session_key="sdk:custom-metadata") + exported = bot.sessions.export("sdk:custom-metadata") + restored = await bot.sessions.restore( + SessionSnapshot( + key="sdk:restored-metadata", + messages=[], + metadata={"model_preset": custom_value}, + ) + ) + restored_result = await bot.run("hello", session_key=restored.key) + + assert ingested_result.content == "reply from base-model" + assert restored_result.content == "reply from base-model" + assert base.calls == ["base-model", "base-model"] + assert exported is not None + assert exported.metadata["model_preset"] == custom_value + assert restored.metadata["model_preset"] == custom_value + + +@pytest.mark.parametrize("invalid_value", [{"invalid": True}, " "]) +@pytest.mark.asyncio +async def test_sdk_invalid_internal_model_preset_metadata_fails_explicitly( + tmp_path, + invalid_value, +) -> None: + base = RecordingProvider("base-model") + loop = AgentLoop( + bus=MessageBus(), + provider=base, + workspace=tmp_path, + model="base-model", + context_window_tokens=8_000, + ) + loop._schedule_background = lambda coro: coro.close() # type: ignore[method-assign] + bot = Nanobot(loop) + + await bot.sessions.ingest( + "sdk:invalid-internal-metadata", + [], + metadata={SESSION_MODEL_PRESET_METADATA_KEY: invalid_value}, + ) + + with pytest.raises(ValueError, match="session model preset must be a non-empty string"): + await bot.run("hello", session_key="sdk:invalid-internal-metadata") + + assert base.calls == [] diff --git a/tests/agent/test_unified_session.py b/tests/agent/test_unified_session.py index 8ee5e723..febd4c26 100644 --- a/tests/agent/test_unified_session.py +++ b/tests/agent/test_unified_session.py @@ -302,7 +302,7 @@ class TestCmdNewUnifiedSession: sessions=sessions, consolidator=SimpleNamespace(archive=AsyncMock(return_value=True)), _cancel_active_tasks=AsyncMock(return_value=0), - llm_runtime=MagicMock(return_value=MagicMock()), + runtime_for_session=MagicMock(return_value=MagicMock()), ) loop._schedule_background = lambda coro: asyncio.ensure_future(coro) diff --git a/tests/agent/tools/test_self_tool.py b/tests/agent/tools/test_self_tool.py index 01c18dde..449c5149 100644 --- a/tests/agent/tools/test_self_tool.py +++ b/tests/agent/tools/test_self_tool.py @@ -4,6 +4,7 @@ from __future__ import annotations import time from pathlib import Path +from types import MappingProxyType from unittest.mock import MagicMock, patch import pytest @@ -11,6 +12,7 @@ from pydantic import BaseModel from nanobot.agent.tools.context import RequestContext, request_context from nanobot.agent.tools.self import MyTool +from nanobot.config.schema import ModelPresetConfig # --------------------------------------------------------------------------- # Helpers @@ -1078,6 +1080,32 @@ class TestSecurityAttributeProtection: result = await tool.execute(action="set", key="web_config.enable", value=False) assert "read-only" in result + @pytest.mark.asyncio + async def test_modify_model_presets_dotpath_blocked(self): + """The config-derived model preset catalog is inspectable but not mutable.""" + presets = {"fast": {"model": "fast-model"}} + tool = _make_tool(runtime_state=_make_mock_loop(model_presets=presets)) + + result = await tool.execute( + action="set", + key="model_presets.other", + value={"model": "other-model"}, + ) + + assert "read-only" in result + assert presets == {"fast": {"model": "fast-model"}} + + @pytest.mark.asyncio + async def test_inspect_read_only_model_preset_dotpath(self): + presets = MappingProxyType({ + "fast": ModelPresetConfig(model="fast-model"), + }) + tool = _make_tool(runtime_state=_make_mock_loop(model_presets=presets)) + + result = await tool.execute(action="check", key="model_presets.fast.model") + + assert result == "model_presets.fast.model: 'fast-model'" + # --------------------------------------------------------------------------- # current iteration count (Fix #2) diff --git a/tests/command/test_model_command.py b/tests/command/test_model_command.py index b2366ce8..ceef123c 100644 --- a/tests/command/test_model_command.py +++ b/tests/command/test_model_command.py @@ -16,6 +16,10 @@ from nanobot.command.builtin import ( ) from nanobot.command.router import CommandContext, CommandRouter from nanobot.config.schema import ModelPresetConfig +from nanobot.session.model_selection import ( + SESSION_MODEL_PRESET_METADATA_KEY, + model_preset_from_metadata, +) def _provider(default_model: str, max_tokens: int = 123) -> MagicMock: @@ -29,7 +33,7 @@ def _provider(default_model: str, max_tokens: int = 123) -> MagicMock: return provider -def _make_loop(tmp_path) -> AgentLoop: +def _make_loop(tmp_path, *, preset_snapshot_loader=None) -> AgentLoop: return AgentLoop( bus=MessageBus(), provider=_provider("base-model", max_tokens=123), @@ -48,6 +52,7 @@ def _make_loop(tmp_path) -> AgentLoop: context_window_tokens=32_768, ), }, + preset_snapshot_loader=preset_snapshot_loader, ) @@ -64,6 +69,11 @@ def _ctx_session(loop: AgentLoop, raw: str, args: str = "") -> CommandContext: ) +def _saved_model_preset(loop: AgentLoop, session_key: str = "cli:direct") -> str | None: + session = loop.sessions.get_or_create(session_key) + return model_preset_from_metadata(session.metadata) + + @pytest.mark.asyncio async def test_model_command_lists_current_and_available_presets(tmp_path) -> None: loop = _make_loop(tmp_path) @@ -84,23 +94,28 @@ async def test_model_command_switches_preset(tmp_path) -> None: out = await cmd_model(_ctx(loop, "/model fast", args="fast")) assert "Switched model preset to `fast`." in out.content + assert "Scope: current session" in out.content assert "Model: `openai/gpt-4.1`" in out.content - assert loop.model_preset == "fast" - assert loop.model == "openai/gpt-4.1" - assert not hasattr(loop.subagents, "model") - assert not hasattr(loop.consolidator, "model") - assert loop.llm_runtime().model == "openai/gpt-4.1" + assert _saved_model_preset(loop) == "fast" + assert loop.model_preset is None + assert loop.model == "base-model" + + await loop.process_direct("/new", session_key="cli:direct") + assert _saved_model_preset(loop) == "fast" + status = await loop.process_direct("/status", session_key="cli:direct") + assert status is not None and "openai/gpt-4.1" in status.content @pytest.mark.asyncio async def test_model_command_switches_back_to_default(tmp_path) -> None: loop = _make_loop(tmp_path) - loop.set_model_preset("fast") + await cmd_model(_ctx(loop, "/model fast", args="fast")) out = await cmd_model(_ctx(loop, "/model default", args="default")) assert "Switched model preset to `default`." in out.content - assert loop.model_preset == "default" + assert _saved_model_preset(loop) == "default" + assert loop.model_preset is None assert loop.model == "base-model" assert loop.context_window_tokens == 1000 @@ -118,6 +133,24 @@ async def test_model_command_unknown_preset_keeps_old_state(tmp_path) -> None: assert loop.model == "base-model" +@pytest.mark.asyncio +async def test_model_command_reports_provider_configuration_errors(tmp_path) -> None: + def fail_preset(_name: str): + raise ValueError("No API key configured for provider 'openai'.") + + loop = _make_loop(tmp_path, preset_snapshot_loader=fail_preset) + + switched = await cmd_model(_ctx(loop, "/model fast", args="fast")) + session = loop.sessions.get_or_create("cli:direct") + session.metadata[SESSION_MODEL_PRESET_METADATA_KEY] = "fast" + status = await cmd_model(_ctx(loop, "/model")) + + assert "Could not switch model preset" in switched.content + assert "No API key configured for provider 'openai'." in switched.content + assert "Current selection error" in status.content + assert "No API key configured for provider 'openai'." in status.content + + @pytest.mark.asyncio async def test_model_command_does_not_depend_on_my_allow_set(tmp_path) -> None: loop = _make_loop(tmp_path) @@ -125,7 +158,7 @@ async def test_model_command_does_not_depend_on_my_allow_set(tmp_path) -> None: await cmd_model(_ctx(loop, "/model fast", args="fast")) - assert loop.model_preset == "fast" + assert _saved_model_preset(loop) == "fast" @pytest.mark.asyncio @@ -142,11 +175,45 @@ async def test_model_command_registered_as_exact_and_prefix(tmp_path) -> None: assert out.metadata == {"render_as": "text"} assert out.content == "\n".join([ "Switched model preset to `fast`.", + "- Scope: current session", "- Model: `openai/gpt-4.1`", "- Context window: 32768", "- Max output tokens: 4096", ]) - assert loop.model_preset == "fast" + assert _saved_model_preset(loop) == "fast" + + +@pytest.mark.asyncio +async def test_model_command_does_not_change_another_session(tmp_path) -> None: + loop = _make_loop(tmp_path) + + await cmd_model(_ctx(loop, "/model fast", args="fast")) + other = InboundMessage(channel="cli", sender_id="user", chat_id="other", content="/model") + out = await cmd_model( + CommandContext(msg=other, session=None, key=other.session_key, raw="/model", loop=loop) + ) + + assert "Current preset: `default`" in out.content + assert _saved_model_preset(loop) == "fast" + + +@pytest.mark.asyncio +async def test_model_command_reports_and_recovers_removed_session_preset(tmp_path) -> None: + loop = _make_loop(tmp_path) + session = loop.sessions.get_or_create("cli:direct") + session.metadata[SESSION_MODEL_PRESET_METADATA_KEY] = "removed" + loop.sessions.save(session) + + status = await loop.process_direct("/model", session_key="cli:direct") + switched = await loop.process_direct("/model default", session_key="cli:direct") + + assert status is not None + assert "model_preset 'removed' not found" in status.content + assert "Available presets: `default`, `fast`" in status.content + assert "Switch with `/model `" in status.content + assert switched is not None + assert "Switched model preset to `default`." in switched.content + assert _saved_model_preset(loop) == "default" def test_model_command_in_help_and_palette() -> None: diff --git a/tests/test_nanobot_facade.py b/tests/test_nanobot_facade.py index c7c1017e..d66c4155 100644 --- a/tests/test_nanobot_facade.py +++ b/tests/test_nanobot_facade.py @@ -62,6 +62,13 @@ def _fake_provider(name: str, *, max_tokens: int = 8192) -> MagicMock: return provider +async def _admit_fake_runtime(bot, session_key, callback, runtime=None) -> None: + admitted = runtime or bot._loop.runtime_for_session( + bot._loop.sessions.get_or_create(session_key) + ) + await callback(admitted) + + def test_from_config_missing_file(): with pytest.raises(FileNotFoundError): Nanobot.from_config("/nonexistent/config.json") @@ -633,8 +640,9 @@ async def test_run_model_preset_override_is_per_run(tmp_path): model="openai/gpt-4.1-mini", context_window_tokens=2048, signature=("preset", "fast"), + model_preset="fast", ) - override_runtime = runtime_from_provider_snapshot(override, model_preset="fast") + override_runtime = runtime_from_provider_snapshot(override) bot._loop.runtime_resolver.resolve_override = MagicMock( return_value=override_runtime ) @@ -775,10 +783,12 @@ async def test_stream_yields_text_events_in_order(tmp_path): bot = Nanobot.from_config(config_path, workspace=tmp_path) async def fake_process_direct( - message, *, session_key, on_stream, on_stream_end, hooks, runtime + message, *, session_key, on_stream, on_stream_end, hooks, on_runtime_admitted, + runtime=None ): assert message == "hi" assert session_key == "sdk:default" + await _admit_fake_runtime(bot, session_key, on_runtime_admitted, runtime) await on_stream("Hel") await on_stream("lo") await on_stream_end(resuming=False) @@ -812,8 +822,10 @@ async def test_run_streamed_wait_returns_full_result_without_consuming_events(tm bot = Nanobot.from_config(config_path, workspace=tmp_path) async def fake_process_direct( - message, *, session_key, on_stream, on_stream_end, hooks, runtime + message, *, session_key, on_stream, on_stream_end, hooks, on_runtime_admitted, + runtime=None ): + await _admit_fake_runtime(bot, session_key, on_runtime_admitted, runtime) await on_stream("done") await on_stream_end(resuming=False) ctx = AgentRunHookContext( @@ -856,8 +868,10 @@ async def test_run_streamed_cancel_releases_full_queue_without_consuming(tmp_pat bot = Nanobot.from_config(config_path, workspace=tmp_path) async def fake_process_direct( - message, *, session_key, on_stream, on_stream_end, hooks, runtime + message, *, session_key, on_stream, on_stream_end, hooks, on_runtime_admitted, + runtime=None ): + await _admit_fake_runtime(bot, session_key, on_runtime_admitted, runtime) for i in range(400): await on_stream(str(i)) await on_stream_end(resuming=False) @@ -921,7 +935,8 @@ async def test_run_streamed_forwards_runtime_options(tmp_path): assert callable(kwargs["on_stream"]) assert callable(kwargs["on_stream_end"]) assert kwargs["hooks"] - assert kwargs["runtime"] is bot._loop.runtime_resolver.runtime + assert "runtime" not in kwargs + assert callable(kwargs["on_runtime_admitted"]) @pytest.mark.asyncio @@ -951,10 +966,12 @@ async def test_run_streamed_model_override_reports_admitted_runtime(tmp_path): on_stream, on_stream_end, hooks, + on_runtime_admitted, runtime, ): assert runtime is override_runtime assert bot._loop.runtime_resolver.runtime is original_runtime + await on_runtime_admitted(runtime) await on_stream("ok") await on_stream_end(resuming=False) return OutboundMessage(channel="cli", chat_id="direct", content="ok") @@ -995,8 +1012,10 @@ async def test_run_streamed_emits_tool_events(tmp_path): bot = Nanobot.from_config(config_path, workspace=tmp_path) async def fake_process_direct( - message, *, session_key, on_stream, on_stream_end, hooks, runtime + message, *, session_key, on_stream, on_stream_end, hooks, on_runtime_admitted, + runtime=None ): + await _admit_fake_runtime(bot, session_key, on_runtime_admitted, runtime) calls = [ ToolCallRequest(id="call_ok", name="read_file", arguments={"path": "README.md"}), ToolCallRequest(id="call_bad", name="exec", arguments={"cmd": "false"}), @@ -1043,8 +1062,10 @@ async def test_run_streamed_emits_reasoning_events(tmp_path): bot = Nanobot.from_config(config_path, workspace=tmp_path) async def fake_process_direct( - message, *, session_key, on_stream, on_stream_end, hooks, runtime + message, *, session_key, on_stream, on_stream_end, hooks, on_runtime_admitted, + runtime=None ): + await _admit_fake_runtime(bot, session_key, on_runtime_admitted, runtime) for hook in hooks: await hook.emit_reasoning("thinking") await hook.emit_reasoning_end() @@ -1072,8 +1093,10 @@ async def test_stream_generator_break_cancels_underlying_run(tmp_path): cancelled = asyncio.Event() async def fake_process_direct( - message, *, session_key, on_stream, on_stream_end, hooks, runtime + message, *, session_key, on_stream, on_stream_end, hooks, on_runtime_admitted, + runtime=None ): + await _admit_fake_runtime(bot, session_key, on_runtime_admitted, runtime) try: await on_stream("first") await asyncio.sleep(10) @@ -1387,7 +1410,7 @@ async def test_runtime_helpers_expose_model_workspace_and_compact(tmp_path): bot = Nanobot.from_config(config_path, workspace=tmp_path) await bot.sessions.ingest("sdk:history", [{"role": "user", "content": "hello"}]) runtime = bot._loop.llm_runtime() - bot._loop.llm_runtime = MagicMock(return_value=runtime) # type: ignore[method-assign] + bot._loop.runtime_for_session = MagicMock(return_value=runtime) # type: ignore[method-assign] bot._loop.consolidator.maybe_consolidate_by_tokens = AsyncMock() snapshot = await bot.runtime.compact_session("sdk:history") diff --git a/tests/webui/test_session_list_index.py b/tests/webui/test_session_list_index.py index 3dd6e1ac..6712c712 100644 --- a/tests/webui/test_session_list_index.py +++ b/tests/webui/test_session_list_index.py @@ -4,11 +4,14 @@ import os from datetime import datetime from pathlib import Path +import pytest + import nanobot.webui.session_list_index as session_list_index from nanobot.cron.session_turns import CRON_HISTORY_META from nanobot.session.automation_turns import AUTOMATION_HISTORY_META from nanobot.session.history_visibility import HIDDEN_HISTORY_META from nanobot.session.manager import SessionManager +from nanobot.session.model_selection import SESSION_MODEL_PRESET_METADATA_KEY def test_webui_session_list_reuses_valid_index_without_scanning_files( @@ -17,10 +20,12 @@ def test_webui_session_list_reuses_valid_index_without_scanning_files( ) -> None: manager = SessionManager(tmp_path) session = manager.get_or_create("websocket:indexed") + session.metadata[SESSION_MODEL_PRESET_METADATA_KEY] = "fast" session.add_message("user", "indexed preview") manager.save(session) assert list_webui_sessions(manager)[0]["preview"] == "indexed preview" + assert list_webui_sessions(manager)[0]["model_preset"] == "fast" def fail_scan(session_manager: SessionManager, path: Path) -> None: raise AssertionError(f"unexpected session file scan: {path}") @@ -31,6 +36,23 @@ def test_webui_session_list_reuses_valid_index_without_scanning_files( assert rows[0]["key"] == "websocket:indexed" assert rows[0]["preview"] == "indexed preview" + assert rows[0]["model_preset"] == "fast" + + +def test_webui_session_list_rejects_invalid_internal_model_preset_metadata( + tmp_path: Path, +) -> None: + manager = SessionManager(tmp_path) + session = manager.get_or_create("websocket:custom-metadata") + session.metadata["model_preset"] = 7 + session.metadata[SESSION_MODEL_PRESET_METADATA_KEY] = {"invalid": True} + session.add_message("user", "custom metadata") + manager.save(session) + + with pytest.raises(ValueError, match="session model preset must be a non-empty string"): + list_webui_sessions(manager) + + assert manager.get_or_create(session.key).metadata["model_preset"] == 7 def test_webui_session_list_rescans_only_changed_file(tmp_path: Path, monkeypatch) -> None: diff --git a/tests/webui/test_settings_api.py b/tests/webui/test_settings_api.py index 42f628d7..13cdc9ff 100644 --- a/tests/webui/test_settings_api.py +++ b/tests/webui/test_settings_api.py @@ -464,6 +464,26 @@ def test_settings_payload_includes_dynamic_custom_provider( assert providers[DYNAMIC_PROVIDER_NAME]["api_base"] == DYNAMIC_PROVIDER_API_BASE +def test_settings_payload_resolves_provider_for_each_auto_preset( + tmp_path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + config_path = tmp_path / "config.json" + config = _dynamic_provider_config() + config.model_presets["fast"] = ModelPresetConfig( + provider="auto", + model=f"{DYNAMIC_PROVIDER_NAME}/gpt-4", + ) + save_config(config, config_path) + monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path) + + payload = settings_payload() + presets = {row["name"]: row for row in payload["model_presets"]} + + assert presets["fast"]["provider"] == "auto" + assert presets["fast"]["resolved_provider"] == DYNAMIC_PROVIDER_NAME + + def test_settings_payload_groups_opencode_compatibility_alias(tmp_path, monkeypatch) -> None: config_path = tmp_path / "config.json" save_config(Config(), config_path) diff --git a/webui/src/components/thread/ThreadShell.tsx b/webui/src/components/thread/ThreadShell.tsx index 5d5d6188..c95cd83b 100644 --- a/webui/src/components/thread/ThreadShell.tsx +++ b/webui/src/components/thread/ThreadShell.tsx @@ -170,8 +170,14 @@ interface ModelBadgeInfo { needsSetup: boolean; } -function activeModelPreset(settings: SettingsPayload | null): SettingsPayload["model_presets"][number] | null { +function modelPresetForBadge( + settings: SettingsPayload | null, + scopedPreset: string | null, +): SettingsPayload["model_presets"][number] | null { if (!settings) return null; + if (scopedPreset) { + return settings.model_presets.find((preset) => preset.name === scopedPreset) ?? null; + } const configured = settings.agent.model_preset || "default"; return ( settings.model_presets.find((preset) => preset.name === configured) @@ -180,19 +186,25 @@ function activeModelPreset(settings: SettingsPayload | null): SettingsPayload["m ); } -function resolvedModelProvider(settings: SettingsPayload | null, modelName: string | null): string | null { - const preset = activeModelPreset(settings); - const rawProvider = preset?.provider || settings?.agent.provider || null; - if (rawProvider === "auto") { - return settings?.agent.resolved_provider || inferProviderFromModelName(modelName) || null; - } - return rawProvider || inferProviderFromModelName(modelName); -} - -function toModelBadgeInfo(modelName: string | null, settings: SettingsPayload | null): ModelBadgeInfo { - const model = modelName || settings?.agent.model || null; +function toModelBadgeInfo( + modelName: string | null, + settings: SettingsPayload | null, + modelPreset: string | null = null, +): ModelBadgeInfo { + const scopedPreset = modelPreset?.trim() || null; + const preset = modelPresetForBadge(settings, scopedPreset); + const model = scopedPreset + ? preset?.model || null + : modelName || settings?.agent.model || null; const label = toModelBadgeLabel(model); - const provider = resolvedModelProvider(settings, model); + const rawProvider = preset?.provider + || (!scopedPreset ? settings?.agent.provider : null) + || null; + const provider = rawProvider === "auto" + ? preset?.resolved_provider + || (!scopedPreset ? settings?.agent.resolved_provider : null) + || null + : rawProvider || inferProviderFromModelName(model); const providerRow = provider ? settings?.providers.find((item) => item.name === provider) : null; @@ -459,9 +471,10 @@ export function ThreadShell({ const showHeroComposer = messages.length === 0 && !loading; const wasShowingHeroComposerRef = useRef(showHeroComposer); + const sessionModelPreset = session?.modelPreset?.trim() || null; const modelBadge = useMemo( - () => toModelBadgeInfo(modelName, settings), - [modelName, settings], + () => toModelBadgeInfo(modelName, settings, sessionModelPreset), + [modelName, sessionModelPreset, settings], ); const modelBadgeLabel = modelBadge.needsSetup ? t("thread.composer.modelNotConfigured", { defaultValue: "Model not configured" }) diff --git a/webui/src/lib/api.ts b/webui/src/lib/api.ts index d650ea54..378bec84 100644 --- a/webui/src/lib/api.ts +++ b/webui/src/lib/api.ts @@ -132,6 +132,7 @@ export async function listSessions( updated_at: string | null; title?: string; preview?: string; + model_preset?: string | null; run_started_at?: number | null; workspace_scope?: WorkspaceScopePayload | null; }; @@ -148,6 +149,7 @@ export async function listSessions( updatedAt: s.updated_at, title: s.title ?? "", preview: s.preview ?? "", + modelPreset: s.model_preset ?? null, runStartedAt: s.run_started_at ?? null, workspaceScope: s.workspace_scope ?? null, })); diff --git a/webui/src/lib/types.ts b/webui/src/lib/types.ts index 899abfae..1d2f0e3f 100644 --- a/webui/src/lib/types.ts +++ b/webui/src/lib/types.ts @@ -245,6 +245,8 @@ export interface ChatSummary { updatedAt: string | null; title?: string; preview: string; + /** Model preset persisted for this session; null means it still follows the global default. */ + modelPreset?: string | null; /** Unix epoch seconds when this session currently has a turn in flight. */ runStartedAt?: number | null; workspaceScope?: WorkspaceScopePayload | null; @@ -406,6 +408,7 @@ export interface SettingsPayload { is_default: boolean; model: string; provider: string; + resolved_provider?: string | null; max_tokens: number; context_window_tokens: number; temperature: number; diff --git a/webui/src/tests/api.test.ts b/webui/src/tests/api.test.ts index 914f92fe..0bae3fd2 100644 --- a/webui/src/tests/api.test.ts +++ b/webui/src/tests/api.test.ts @@ -797,6 +797,7 @@ describe("webui API helpers", () => { created_at: "2026-05-01T10:00:00", updated_at: "2026-05-01T10:01:00", title: "优化 WebUI 标题", + model_preset: "fast", run_started_at: 1_700_000_000, }, ], @@ -808,6 +809,7 @@ describe("webui API helpers", () => { key: "websocket:chat-1", title: "优化 WebUI 标题", preview: "", + modelPreset: "fast", runStartedAt: 1_700_000_000, }, ]); diff --git a/webui/src/tests/thread-shell.test.tsx b/webui/src/tests/thread-shell.test.tsx index 03e88495..8563e166 100644 --- a/webui/src/tests/thread-shell.test.tsx +++ b/webui/src/tests/thread-shell.test.tsx @@ -94,7 +94,7 @@ function expectSendMessageWithTurn( ); } -function session(chatId: string) { +function session(chatId: string, modelPreset?: string | null) { return { key: `websocket:${chatId}`, channel: "websocket" as const, @@ -102,6 +102,7 @@ function session(chatId: string) { createdAt: null, updatedAt: null, preview: "", + modelPreset, }; } @@ -220,6 +221,20 @@ function modelSettings(model: string, provider: string): SettingsPayload { }; } +function settingsWithFastPreset(): SettingsPayload { + const settings = modelSettings("deepseek-v4-pro", "deepseek"); + settings.model_presets.push({ + ...settings.model_presets[0]!, + name: "fast", + label: "Fast", + active: false, + is_default: false, + model: "openai-codex/gpt-5.5", + provider: "openai_codex", + }); + return settings; +} + describe("ThreadShell", () => { beforeEach(() => { vi.stubGlobal( @@ -341,6 +356,61 @@ describe("ThreadShell", () => { expect(await screen.findByTestId("composer-model-logo-openai_codex")).toBeInTheDocument(); }); + it("resolves the composer model from the active session preset", async () => { + const client = makeClient(); + render( + wrap( + client, + {}} + settingsSnapshot={settingsWithFastPreset()} + />, + "deepseek-v4-pro", + ), + ); + + expect(await screen.findByTitle("gpt-5.5 · OpenAI Codex")).toBeInTheDocument(); + expect(screen.queryByTitle("deepseek-v4-pro · DeepSeek")).not.toBeInTheDocument(); + }); + + it("uses the backend-resolved provider for an auto session preset", async () => { + const client = makeClient(); + const settings = modelSettings("deepseek-v4-pro", "deepseek"); + settings.providers.push({ + name: "companyproxy", + label: "Company Proxy", + configured: true, + }); + settings.model_presets.push({ + ...settings.model_presets[0]!, + name: "fast", + label: "Fast", + active: false, + is_default: false, + model: "companyproxy/gpt-4", + provider: "auto", + resolved_provider: "companyproxy", + }); + + render( + wrap( + client, + {}} + settingsSnapshot={settings} + />, + "deepseek-v4-pro", + ), + ); + + expect(await screen.findByTitle("gpt-4 · Company Proxy")).toBeInTheDocument(); + expect(screen.queryByRole("button", { name: "Model not configured" })).not.toBeInTheDocument(); + }); + it("opens model settings from the unconfigured model badge", async () => { const client = makeClient(); const settings = modelSettings("openai-codex/gpt-5.1-codex", "openai_codex");