diff --git a/docs/cli-reference.md b/docs/cli-reference.md index b9a4899e..1db77c3b 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -20,7 +20,7 @@ Use this page when you know what you want to run and need the command shape. For | Check chat channel setup | `nanobot channels status` | Useful before starting `nanobot gateway` | | Manage optional features | `nanobot plugins list` | Shows channels and optional capabilities you can turn on | | Log in to QR/OAuth-style channels | `nanobot channels login ` | Used by channels such as WhatsApp and WeChat | -| Log in to OAuth model providers | `nanobot provider login ` | Used by OAuth providers such as OpenAI Codex and GitHub Copilot | +| Log in to OAuth model providers | `nanobot provider login ` | Used by OpenAI Codex, xAI subscription, and GitHub Copilot providers | ## Global @@ -287,8 +287,10 @@ remain accepted as no-op compatibility aliases. | Command | Description | |---|---| | `nanobot provider login openai-codex --set-main` | Authenticate Codex and select its current default model | +| `nanobot provider login xai-grok --set-main` | Authenticate an eligible X Premium / Grok subscription and select Grok 4.5; hosted X Search is enabled for models that advertise support | | `nanobot provider login github-copilot --set-main` | Authenticate GitHub Copilot and select its current default model | | `nanobot provider logout openai-codex` | Remove OpenAI Codex OAuth state | +| `nanobot provider logout xai-grok --config ` | Remove the selected nanobot instance's xAI OAuth state | | `nanobot provider logout github-copilot` | Remove GitHub Copilot OAuth state | See [`providers.md`](./providers.md#oauth-providers) for when OAuth providers need explicit provider/model selection. diff --git a/docs/configuration.md b/docs/configuration.md index c69ad7d2..404cb772 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -260,7 +260,7 @@ Tracing covers the providers that go through nanobot's OpenAI-compatible client > - **Xiaomi MiMo thinking mode**: MiMo models (e.g. `mimo-v2.5-pro`) default to enabled thinking. Use `agents.defaults.reasoningEffort: "none"` to disable it, or `"low"` / `"medium"` / `"high"` to keep it on. Omitting the field preserves the provider's per-model default. > - **Xiaomi MiMo Token Plan**: If you're on MiMo's token plan, set `"apiBase": "https://token-plan-sgp.xiaomimimo.com/v1"` in your xiaomi_mimo provider config. > - **Custom OpenAI-compatible providers**: Besides the built-in `custom` provider, any extra key under `providers` can define its own OpenAI-compatible endpoint. For example, `providers.companyProxy.apiBase` plus `modelPresets.primary.provider: "companyProxy"` creates a separate custom provider. Set `apiBase`; set `apiKey` only when the endpoint requires it. This named-custom path uses the OpenAI-compatible request format only. For Anthropic-compatible proxies, use `providers.anthropic.apiBase` with `provider: "anthropic"`. -> - **Provider-scoped proxy**: `providers..proxy` routes only that provider through an HTTP proxy. It is supported for OpenAI-compatible providers and `openai_codex`. Native provider backends such as `anthropic`, `bedrock`, `azure_openai`, and `github_copilot` reject `proxy`. +> - **Provider-scoped proxy**: `providers..proxy` routes only that provider through an HTTP proxy. It is supported for OpenAI-compatible providers, `openai_codex`, and `xai_grok`. Native provider backends such as `anthropic`, `bedrock`, `azure_openai`, and `github_copilot` reject `proxy`. | Provider | Purpose | Get API Key | |----------|---------|-------------| @@ -305,6 +305,7 @@ Tracing covers the providers that go through nanobot's OpenAI-compatible client | `vllm` | LLM (local, any OpenAI-compatible server) | — | | `nvidia` | LLM (NVIDIA NIM) | [build.nvidia.com](https://build.nvidia.com/) | | `openai_codex` | LLM (Codex, OAuth) | `nanobot provider login openai-codex --set-main` | +| `xai_grok` | LLM (Grok, OAuth) | `nanobot provider login xai-grok --set-main` | | `github_copilot` | LLM (GitHub Copilot, OAuth) | `nanobot provider login github-copilot` | | `qianfan` | LLM (Baidu Qianfan) | [cloud.baidu.com](https://cloud.baidu.com/doc/qianfan/s/Hmh4suq26) | @@ -702,6 +703,51 @@ For proxy, remote/headless login, model-name, or config-key errors, see [`troubl +
+xAI Grok (OAuth) + +Use an eligible X Premium / Grok subscription without putting an API key in +`config.json`: + +```bash +nanobot provider login xai-grok --set-main +nanobot agent -m "Hello from Grok." +``` + +The default model is `xai-grok/grok-4.5` with a 500,000-token context window. +The provider reads xAI's model catalog and includes the server-hosted `x_search` +tool only when the selected model advertises `supportsBackendSearch`. Models +without that capability continue normally without hosted X Search. When enabled, +searches run inside xAI's Responses API and citations arrive as inline links. + +This is xAI subscription OAuth, not X Developer OAuth. nanobot follows the +public OAuth client and proxy contract used by +[Grok Build](https://github.com/xai-org/grok-build/blob/main/crates/codegen/xai-grok-pager/docs/user-guide/02-authentication.md). +The browser flow uses a random loopback callback and PKCE. The resulting token +is stored in the active instance's `auth/xai.json` (normally +`~/.nanobot/auth/xai.json`), separately from Grok Build so rotating refresh +tokens cannot invalidate one another. + +To use a provider-specific proxy, merge this into `config.json` before login: + +```json +{ + "providers": { + "xaiGrok": { + "proxy": "http://127.0.0.1:7890" + } + } +} +``` + +The proxy applies to OAuth discovery, token exchange/refresh, model-catalog +lookups, and subscription model requests. Because this integration depends on +xAI's public Grok Build client contract, an upstream contract change may require +a nanobot update. + +
+ +
GitHub Copilot (OAuth) diff --git a/docs/providers.md b/docs/providers.md index 9ce3135e..2d2ea452 100644 --- a/docs/providers.md +++ b/docs/providers.md @@ -63,11 +63,11 @@ These fields answer different questions: | `model` | `modelPresets..model` | The model ID expected by that provider or gateway. | | `apiKey` | `providers..apiKey` | Credential for that provider. Use `${ENV_VAR}` for secrets. | | `apiBase` | `providers..apiBase` | HTTP base URL of the provider endpoint. | -| `proxy` | `providers..proxy` | Optional HTTP proxy for this provider only. Supported for OpenAI-compatible providers and OpenAI Codex. | +| `proxy` | `providers..proxy` | Optional HTTP proxy for this provider only. Supported for OpenAI-compatible providers, OpenAI Codex, and xAI OAuth. | You usually omit `apiBase` for hosted built-in providers such as OpenRouter, Anthropic direct, OpenAI direct, Groq, or Bedrock because nanobot knows their default endpoints. Set `apiBase` for `custom`, local OpenAI-compatible servers, provider proxies, regional endpoints, or subscription endpoints. Include the API version path when the endpoint requires it, for example `https://api.example.com/v1` or `http://localhost:11434/v1`. -Use `proxy` when one provider must send HTTP traffic through a proxy without changing process-wide `HTTP_PROXY` / `HTTPS_PROXY`. This is supported for providers that use nanobot's OpenAI-compatible client, including `openai`, `custom`, named custom providers, OpenRouter-style gateways, local OpenAI-compatible servers, and similar registry entries. It is also supported for `openai_codex`, including Codex OAuth token exchange/refresh and Codex Responses API requests. Native provider backends such as `anthropic`, `bedrock`, `azure_openai`, and `github_copilot` reject `proxy`; use their endpoint-specific configuration instead. +Use `proxy` when one provider must send HTTP traffic through a proxy without changing process-wide `HTTP_PROXY` / `HTTPS_PROXY`. This is supported for providers that use nanobot's OpenAI-compatible client, including `openai`, `custom`, named custom providers, OpenRouter-style gateways, local OpenAI-compatible servers, and similar registry entries. It is also supported for `openai_codex` and `xai_grok`, including OAuth token exchange/refresh and model requests. Native provider backends such as `anthropic`, `bedrock`, `azure_openai`, and `github_copilot` reject `proxy`; use their endpoint-specific configuration instead. ## Common Provider Patterns @@ -433,6 +433,25 @@ For OpenAI Codex: nanobot provider login openai-codex --set-main ``` +For an eligible X Premium / Grok subscription: + +```bash +nanobot provider login xai-grok --set-main +``` + +This selects `xai-grok/grok-4.5`. The provider reads xAI's model catalog and +exposes the hosted `x_search` tool only when the selected model advertises +`supportsBackendSearch`; otherwise the model runs without hosted X Search. +When enabled, Grok can search current X posts and return inline source links +without invoking a local nanobot tool. Credentials are stored under the +active instance's `auth/xai.json` (normally `~/.nanobot/auth/xai.json`), not in +`config.json` and not in Grok Build's credential file. + +The login is xAI subscription OAuth, not X Developer OAuth. It follows the +public client contract documented and implemented by +[Grok Build](https://github.com/xai-org/grok-build/blob/main/crates/codegen/xai-grok-pager/docs/user-guide/02-authentication.md); +xAI may change that upstream contract independently of nanobot. + For GitHub Copilot: ```bash diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index ba89dadc..c3b9d977 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -135,12 +135,17 @@ If you need a known-good snippet instead of diagnosis, use [`provider-cookbook.m | Provider cannot be inferred | Pin `modelPresets..provider` in the active preset instead of using `"auto"`. For legacy direct configs, pin `agents.defaults.provider`. | | Local model connection refused | Ollama, vLLM, LM Studio, or another local server is not running, or `apiBase` points to the wrong port. | | Bedrock validation error | Check AWS region, credentials, model access, model ID, and whether the model supports Converse. | -| OAuth provider fails | Run `nanobot provider login openai-codex --set-main` or `nanobot provider login github-copilot --set-main`. | +| OAuth provider fails | Run the matching login command: `openai-codex`, `xai-grok`, or `github-copilot`, normally with `--set-main`. | | Codex OAuth needs a proxy | Set `providers.openaiCodex.proxy` before running the login command. The proxy applies to login, token refresh, and Codex API requests. | | Codex login runs on a remote/headless machine | Open the printed URL in a local browser, then paste the final `http://localhost:1455/auth/callback?...` URL back into the terminal. | | Codex login runs in Docker | Start the container with `docker run -it` so the OAuth flow has an interactive terminal. | | Codex says a model is not supported with a ChatGPT account | Use provider `openai_codex` with a Codex model such as `openai-codex/gpt-5.6-sol`. Do not use the direct-API `openai/...` prefix with Codex OAuth. | | Config says `providers.openai_codex` conflicts with the built-in provider | Under `providers`, keep only the canonical `openaiCodex` settings key and remove a duplicate `openai_codex` key. A model preset's `provider` value remains `openai_codex`. | +| xAI OAuth needs a proxy | Set `providers.xaiGrok.proxy` before login. It applies to OAuth discovery, token exchange/refresh, and Grok subscription requests. | +| xAI login runs on a remote/headless machine | In the WebUI, finish sign-in in your local browser; if the loopback redirect cannot reach the server, copy the final URL from the address bar into the WebUI dialog. From the CLI, run `nanobot provider login xai-grok` interactively, open the printed URL elsewhere, and paste the final callback URL or authorization code when prompted. | +| xAI returns 403 or subscription access denied | Confirm the signed-in account has an eligible X Premium / Grok subscription, then run `nanobot provider login xai-grok` again. This provider does not use an xAI API key or X Developer OAuth. | +| xAI returns 400 `invalid-argument` | Read the bounded `Response body` appended to the provider error. Hosted `x_search` is sent only when xAI's model catalog advertises `supportsBackendSearch`; the model ID `grok-4.5` itself is valid. | +| xAI model or X Search stops working after an upstream release | The integration follows Grok Build's public OAuth/proxy client contract. Update nanobot if xAI changes that contract. | ## Langfuse Problems diff --git a/nanobot/cli/commands.py b/nanobot/cli/commands.py index 9b35b68b..d96c92c8 100644 --- a/nanobot/cli/commands.py +++ b/nanobot/cli/commands.py @@ -2662,11 +2662,13 @@ _LOGOUT_HANDLERS: dict[str, Callable[[], None]] = {} _PROVIDER_DISPLAY: dict[str, str] = { "openai_codex": "OpenAI Codex", + "xai_grok": "xAI Grok", "github_copilot": "GitHub Copilot", } _OAUTH_PROVIDER_DEFAULT_MODELS: dict[str, str] = { "openai_codex": "openai-codex/gpt-5.6-sol", + "xai_grok": "xai-grok/grok-4.5", "github_copilot": "github-copilot/gpt-5.4-mini", } @@ -2720,6 +2722,8 @@ def _set_oauth_provider_as_main( config.agents.defaults.model_preset = None config.agents.defaults.provider = provider_name config.agents.defaults.model = selected_model + if provider_name == "xai_grok" and selected_model == "xai-grok/grok-4.5": + config.agents.defaults.context_window_tokens = 500_000 save_config(config, resolved_config_path) saved_path = resolved_config_path or get_config_path() @@ -2732,7 +2736,10 @@ def _set_oauth_provider_as_main( @provider_app.command("login") def provider_login( - provider: str = typer.Argument(..., help="OAuth provider (e.g. 'openai-codex', 'github-copilot')"), + provider: str = typer.Argument( + ..., + help="OAuth provider (e.g. 'openai-codex', 'xai-grok', 'github-copilot')", + ), set_main: bool = typer.Option( False, "--set-main", @@ -2770,7 +2777,11 @@ def provider_login( @provider_app.command("logout") def provider_logout( - provider: str = typer.Argument(..., help="OAuth provider (e.g. 'openai-codex', 'github-copilot')"), + provider: str = typer.Argument( + ..., + help="OAuth provider (e.g. 'openai-codex', 'xai-grok', 'github-copilot')", + ), + config: str | None = typer.Option(None, "--config", "-c", help="Path to config file"), ): """Log out from an OAuth provider.""" spec = _resolve_oauth_provider(provider) @@ -2780,6 +2791,13 @@ def provider_logout( console.print(f"[red]Logout not implemented for {spec.label}[/red]") raise typer.Exit(1) + if config: + from nanobot.config.loader import set_config_path + + resolved_config_path = Path(config).expanduser().resolve() + set_config_path(resolved_config_path) + console.print(f"[dim]Using config: {resolved_config_path}[/dim]") + console.print(f"{__logo__} OAuth Logout - {spec.label}\n") handler() @@ -2830,6 +2848,55 @@ def _logout_openai_codex() -> None: _delete_oauth_files(storage.get_token_path(), _PROVIDER_DISPLAY["openai_codex"]) +@_register_login("xai_grok") +def _login_xai_grok() -> None: + """Authenticate with xAI using the Grok subscription OAuth contract.""" + from nanobot.config.loader import load_config, resolve_config_env_vars + from nanobot.providers.xai_oauth import get_xai_oauth_token, login_xai_oauth + + try: + proxy = resolve_config_env_vars(load_config()).providers.xai_grok.proxy or None + except ValueError as exc: + console.print(f"[red]{exc}[/red]") + raise typer.Exit(1) from exc + + token = None + with suppress(Exception): + token = get_xai_oauth_token(proxy=proxy) + if not (token and token.access): + console.print( + "[cyan]Starting xAI browser sign-in for your X Premium / Grok subscription...[/cyan]\n" + ) + try: + token = login_xai_oauth( + print_fn=lambda message: console.print(message), + prompt_fn=lambda prompt: typer.prompt(prompt), + proxy=proxy, + ) + except Exception as exc: + console.print(f"[red]Authentication error: {exc}[/red]") + raise typer.Exit(1) from exc + account = token.account_id or "xAI account" + console.print(f"[green]✓ Authenticated with xAI[/green] [dim]{account}[/dim]") + console.print( + "[dim]Hosted X Search is enabled automatically when the selected model supports it.[/dim]" + ) + + +@_register_logout("xai_grok") +def _logout_xai_grok() -> None: + """Clear local xAI OAuth credentials for this nanobot instance.""" + from nanobot.providers.xai_oauth import get_xai_oauth_storage_path, logout_xai_oauth + + token_path = get_xai_oauth_storage_path() + provider_label = _PROVIDER_DISPLAY["xai_grok"] + if logout_xai_oauth(): + console.print(f"[green]✓ Logged out from {provider_label}[/green]") + console.print(f"[dim]Removed: {token_path}[/dim]") + else: + console.print(f"[yellow]! No local OAuth credentials found for {provider_label}[/yellow]") + + @_register_logout("github_copilot") def _logout_github_copilot() -> None: """Clear local OAuth credentials for GitHub Copilot.""" diff --git a/nanobot/config/loader.py b/nanobot/config/loader.py index 743ab8a0..347c1c29 100644 --- a/nanobot/config/loader.py +++ b/nanobot/config/loader.py @@ -81,10 +81,20 @@ def save_config(config: Config, config_path: Path | None = None) -> None: path.parent.mkdir(parents=True, exist_ok=True) data = config.model_dump(mode="json", by_alias=True) - if config.providers.openai_codex.proxy is not None: - data.setdefault("providers", {})["openaiCodex"] = { - "proxy": config.providers.openai_codex.proxy, - } + # OAuth credentials live in dedicated token stores. Persist only the + # non-credential request settings consumed by these provider backends. + for alias, provider in ( + ("openaiCodex", config.providers.openai_codex), + ("xaiGrok", config.providers.xai_grok), + ): + settings = provider.model_dump( + mode="json", + by_alias=True, + include={"proxy", "extra_body"}, + exclude_none=True, + ) + if settings: + data.setdefault("providers", {})[alias] = settings # Temp + replace so a crash mid-write cannot leave a truncated config.json. _write_text_atomic(path, json.dumps(data, indent=2, ensure_ascii=False)) diff --git a/nanobot/config/schema.py b/nanobot/config/schema.py index 95e7d68c..47c04ae2 100644 --- a/nanobot/config/schema.py +++ b/nanobot/config/schema.py @@ -269,6 +269,7 @@ class ProvidersConfig(Base): byteplus: ProviderConfig = Field(default_factory=ProviderConfig) # BytePlus (VolcEngine international) byteplus_coding_plan: ProviderConfig = Field(default_factory=ProviderConfig) # BytePlus Coding Plan openai_codex: ProviderConfig = Field(default_factory=ProviderConfig, exclude=True) # OpenAI Codex (OAuth) + xai_grok: ProviderConfig = Field(default_factory=ProviderConfig, exclude=True) # xAI Grok (OAuth) github_copilot: ProviderConfig = Field(default_factory=ProviderConfig, exclude=True) # Github Copilot (OAuth) qianfan: ProviderConfig = Field(default_factory=ProviderConfig) # Qianfan (百度千帆) nvidia: ProviderConfig = Field(default_factory=ProviderConfig) # NVIDIA NIM (nvapi- keys) diff --git a/nanobot/providers/__init__.py b/nanobot/providers/__init__.py index f5834b73..4eea5d3d 100644 --- a/nanobot/providers/__init__.py +++ b/nanobot/providers/__init__.py @@ -13,6 +13,7 @@ __all__ = [ "AnthropicProvider", "OpenAICompatProvider", "OpenAICodexProvider", + "XAIGrokProvider", "GitHubCopilotProvider", "AzureOpenAIProvider", "BedrockProvider", @@ -22,6 +23,7 @@ _LAZY_IMPORTS = { "AnthropicProvider": ".anthropic_provider", "OpenAICompatProvider": ".openai_compat_provider", "OpenAICodexProvider": ".openai_codex_provider", + "XAIGrokProvider": ".xai_grok_provider", "GitHubCopilotProvider": ".github_copilot_provider", "AzureOpenAIProvider": ".azure_openai_provider", "BedrockProvider": ".bedrock_provider", @@ -34,6 +36,7 @@ if TYPE_CHECKING: from nanobot.providers.github_copilot_provider import GitHubCopilotProvider from nanobot.providers.openai_codex_provider import OpenAICodexProvider from nanobot.providers.openai_compat_provider import OpenAICompatProvider + from nanobot.providers.xai_grok_provider import XAIGrokProvider def __getattr__(name: str): diff --git a/nanobot/providers/factory.py b/nanobot/providers/factory.py index ead2e023..7fdec1a3 100644 --- a/nanobot/providers/factory.py +++ b/nanobot/providers/factory.py @@ -60,10 +60,10 @@ def _make_provider_core( if spec and spec.is_transcription_only: raise ValueError(f"Provider '{provider_name}' only supports transcription.") backend = spec.backend if spec else "openai_compat" - if p and p.proxy and backend not in {"openai_compat", "openai_codex"}: + if p and p.proxy and backend not in {"openai_compat", "openai_codex", "xai_grok"}: raise ValueError( f"providers.{provider_name}.proxy is only supported for " - "OpenAI-compatible providers and OpenAI Codex." + "OpenAI-compatible providers, OpenAI Codex, and xAI Grok." ) if backend == "azure_openai": @@ -91,6 +91,14 @@ def _make_provider_core( proxy=getattr(p, "proxy", None) if p else None, extra_body=p.extra_body if p else None, ) + elif backend == "xai_grok": + from nanobot.providers.xai_grok_provider import XAIGrokProvider + + provider = XAIGrokProvider( + default_model=model, + proxy=getattr(p, "proxy", None) if p else None, + extra_body=p.extra_body if p else None, + ) elif backend == "azure_openai": from nanobot.providers.azure_openai_provider import AzureOpenAIProvider diff --git a/nanobot/providers/registry.py b/nanobot/providers/registry.py index 59db09a4..d5acd0de 100644 --- a/nanobot/providers/registry.py +++ b/nanobot/providers/registry.py @@ -47,7 +47,8 @@ class ProviderSpec: settings_alias_for: str = "" # compatibility alias grouped under this provider in Settings # which provider implementation to use - # "openai_compat" | "anthropic" | "azure_openai" | "openai_codex" | "github_copilot" | "bedrock" + # "openai_compat" | "anthropic" | "azure_openai" | "openai_codex" | "xai_grok" + # | "github_copilot" | "bedrock" backend: str = "openai_compat" # extra env vars / request headers supplied by the provider integration. @@ -420,6 +421,25 @@ PROVIDERS: tuple[ProviderSpec, ...] = ( default_api_base="https://chatgpt.com/backend-api", is_oauth=True, ), + # xAI subscription: OAuth-based, with capability-gated server-hosted X Search. + ProviderSpec( + name="xai_grok", + keywords=("xai-grok", "xai_grok"), + env_key="", + display_name="xAI Grok", + model_catalog="builtin", + builtin_models=( + ProviderModelSpec( + id="xai-grok/grok-4.5", + label="Grok 4.5", + description="Grok via xAI subscription; X Search is enabled when supported.", + context_window=500000, + ), + ), + backend="xai_grok", + default_api_base="https://cli-chat-proxy.grok.com/v1", + is_oauth=True, + ), # GitHub Copilot: OAuth-based ProviderSpec( name="github_copilot", diff --git a/nanobot/providers/xai_grok_provider.py b/nanobot/providers/xai_grok_provider.py new file mode 100644 index 00000000..c503d89d --- /dev/null +++ b/nanobot/providers/xai_grok_provider.py @@ -0,0 +1,545 @@ +"""xAI subscription provider with capability-gated hosted X Search.""" + +from __future__ import annotations + +import asyncio +import base64 +import json +import re +import time +import uuid +from collections.abc import Awaitable, Callable +from typing import Any + +import httpx +from loguru import logger + +from nanobot import __version__ +from nanobot.providers.base import ( + LLMProvider, + LLMResponse, + ToolCallRequest, + resolve_stream_idle_timeout_s, +) +from nanobot.providers.openai_responses import ( + consume_sse_with_reasoning, + convert_messages, + convert_tools, +) +from nanobot.providers.xai_oauth import ( + XAI_CLIENT_VERSION, + XAIToken, + get_xai_oauth_token, +) + +DEFAULT_XAI_GROK_URL = "https://cli-chat-proxy.grok.com/v1/responses" +DEFAULT_XAI_GROK_MODELS_URL = "https://cli-chat-proxy.grok.com/v1/models" +DEFAULT_XAI_GROK_MODEL = "xai-grok/grok-4.5" +_MODEL_CAPABILITIES_TTL_S = 5 * 60 +_MAX_ERROR_BODY_CHARS = 1000 +_SENSITIVE_ERROR_KEYS = { + "accesstoken", + "apikey", + "authorization", + "idtoken", + "refreshtoken", +} + + +class XAIGrokProvider(LLMProvider): + """Call xAI's subscription proxy and expose supported hosted tools.""" + + supports_progress_deltas = True + + def __init__( + self, + default_model: str = DEFAULT_XAI_GROK_MODEL, + proxy: str | None = None, + extra_body: dict[str, Any] | None = None, + ): + super().__init__(api_key=None, api_base=None) + self.default_model = default_model + self.proxy = proxy or None + self._extra_body = dict(extra_body or {}) + self._model_capabilities: dict[str, bool] | None = None + self._model_capabilities_fetched_at = 0.0 + + async def _supports_backend_search(self, token: XAIToken, model: str) -> bool: + now = time.monotonic() + capabilities = self._model_capabilities + if ( + capabilities is None + or now - self._model_capabilities_fetched_at >= _MODEL_CAPABILITIES_TTL_S + ): + try: + capabilities = await _fetch_xai_model_capabilities( + DEFAULT_XAI_GROK_MODELS_URL, + _build_model_headers(token), + proxy=self.proxy, + ) + except Exception as exc: + logger.warning( + "xAI model capability lookup failed; hosted X Search disabled for model {}: " + "type={} error={}", + model, + type(exc).__name__, + str(exc).strip() or "unexpected error", + ) + capabilities = {} + self._model_capabilities = capabilities + self._model_capabilities_fetched_at = now + else: + self._model_capabilities = capabilities + self._model_capabilities_fetched_at = now + return capabilities.get(model, False) + + async def _call_xai( + self, + messages: list[dict[str, Any]], + tools: list[dict[str, Any]] | None, + model: str | None, + max_tokens: int, + temperature: float, + reasoning_effort: str | None, + tool_choice: str | dict[str, Any] | None, + on_content_delta: Callable[[str], Awaitable[None]] | None = None, + on_thinking_delta: Callable[[str], Awaitable[None]] | None = None, + on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None, + ) -> LLMResponse: + wire_model = _strip_model_prefix(model or self.default_model) + system_prompt, input_items = convert_messages(messages) + + stage = "oauth_token" + try: + token = await asyncio.to_thread(get_xai_oauth_token, proxy=self.proxy) + stage = "model_capabilities" + supports_backend_search = await self._supports_backend_search(token, wire_model) + converted_tools = convert_tools(tools or []) + if supports_backend_search: + converted_tools = [ + tool for tool in converted_tools if tool.get("name") != "x_search" + ] + converted_tools.append({"type": "x_search"}) + + body: dict[str, Any] = { + "model": wire_model, + "store": False, + "stream": True, + "instructions": system_prompt, + "input": input_items, + "include": ["reasoning.encrypted_content"], + "tools": converted_tools, + "tool_choice": tool_choice or "auto", + "parallel_tool_calls": True, + "stream_tool_calls": True, + "max_output_tokens": max_tokens, + "temperature": temperature, + "reasoning": _build_reasoning_options(reasoning_effort), + } + if self._extra_body: + body.update(self._extra_body) + + headers = _build_headers(token.access, wire_model) + stage = "xai_request" + try: + result = await _request_xai( + DEFAULT_XAI_GROK_URL, + headers, + body, + proxy=self.proxy, + on_content_delta=on_content_delta, + on_thinking_delta=on_thinking_delta, + on_tool_call_delta=on_tool_call_delta, + ) + except _XAIHTTPError as exc: + if exc.status_code != 401: + raise + stage = "oauth_refresh" + token = await asyncio.to_thread( + get_xai_oauth_token, + proxy=self.proxy, + force_refresh=True, + ) + self._model_capabilities = None + self._model_capabilities_fetched_at = 0.0 + headers = _build_headers(token.access, wire_model) + stage = "xai_request_retry" + result = await _request_xai( + DEFAULT_XAI_GROK_URL, + headers, + body, + proxy=self.proxy, + on_content_delta=on_content_delta, + on_thinking_delta=on_thinking_delta, + on_tool_call_delta=on_tool_call_delta, + ) + + content, tool_calls, finish_reason, usage, reasoning_content = result + return LLMResponse( + content=content, + tool_calls=tool_calls, + finish_reason=finish_reason, + usage=usage, + reasoning_content=reasoning_content, + ) + except Exception as exc: + response = _xai_error_response(exc) + logger.warning( + "xAI subscription request failed: stage={} type={} retryable={} status={} " + "error_type={} error_code={} response_body={}", + stage, + type(exc).__name__, + response.error_should_retry, + response.error_status_code, + response.error_type, + response.error_code, + getattr(exc, "response_body", None), + ) + return response + + async def chat( + self, + messages: list[dict[str, Any]], + tools: list[dict[str, Any]] | None = None, + model: str | None = None, + max_tokens: int = 4096, + temperature: float = 0.7, + reasoning_effort: str | None = None, + tool_choice: str | dict[str, Any] | None = None, + ) -> LLMResponse: + return await self._call_xai( + messages, tools, model, max_tokens, temperature, reasoning_effort, tool_choice + ) + + async def chat_stream( + self, + messages: list[dict[str, Any]], + tools: list[dict[str, Any]] | None = None, + model: str | None = None, + max_tokens: int = 4096, + temperature: float = 0.7, + reasoning_effort: str | None = None, + tool_choice: str | dict[str, Any] | None = None, + on_content_delta: Callable[[str], Awaitable[None]] | None = None, + on_thinking_delta: Callable[[str], Awaitable[None]] | None = None, + on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None, + ) -> LLMResponse: + return await self._call_xai( + messages, + tools, + model, + max_tokens, + temperature, + reasoning_effort, + tool_choice, + on_content_delta, + on_thinking_delta, + on_tool_call_delta, + ) + + def get_default_model(self) -> str: + return self.default_model + + +def _strip_model_prefix(model: str) -> str: + if model.startswith("xai-grok/") or model.startswith("xai_grok/"): + return model.split("/", 1)[1] + return model + + +def _build_reasoning_options(reasoning_effort: str | None) -> dict[str, str]: + options = {"summary": "concise"} + if reasoning_effort and reasoning_effort.lower() != "none": + options["effort"] = reasoning_effort + return options + + +def _build_headers(token: str, model: str) -> dict[str, str]: + conversation_id = str(uuid.uuid4()) + return { + "Authorization": f"Bearer {token}", + "X-XAI-Token-Auth": "xai-grok-cli", + "x-authenticateresponse": "authenticate-response", + "x-grok-client-version": XAI_CLIENT_VERSION, + "x-grok-client-identifier": "nanobot", + "x-grok-client-mode": "headless", + "x-grok-conv-id": conversation_id, + "x-grok-req-id": str(uuid.uuid4()), + "x-grok-model-override": model, + "x-grok-session-id": conversation_id, + "x-grok-agent-id": str(uuid.uuid4()), + "User-Agent": f"nanobot/{__version__} (python)", + "accept": "text/event-stream", + "content-type": "application/json", + } + + +def _build_model_headers(token: XAIToken) -> dict[str, str]: + headers = { + "Authorization": f"Bearer {token.access}", + "X-XAI-Token-Auth": "xai-grok-cli", + "x-grok-client-version": XAI_CLIENT_VERSION, + "x-grok-client-identifier": "nanobot", + "x-grok-client-mode": "headless", + "User-Agent": f"nanobot/{__version__} (python)", + "accept": "application/json", + } + claims = _decode_access_token_claims(token.access) + user_id = claims.get("sub") + if claims.get("principal_type") == "Team": + user_id = claims.get("principal_id") or user_id + if isinstance(user_id, str) and user_id: + headers["x-userid"] = user_id + email = claims.get("email") + if not isinstance(email, str) or "@" not in email: + email = token.account_id if token.account_id and "@" in token.account_id else None + if email: + headers["x-email"] = email + return headers + + +def _decode_access_token_claims(token: str) -> dict[str, Any]: + """Read identity hints from the signed token; the server still authenticates it.""" + parts = token.split(".") + if len(parts) < 2 or not parts[1]: + return {} + payload = parts[1] + try: + decoded = base64.urlsafe_b64decode(payload + "=" * (-len(payload) % 4)) + claims = json.loads(decoded) + except (ValueError, TypeError): + return {} + return claims if isinstance(claims, dict) else {} + + +class _XAIHTTPError(RuntimeError): + def __init__( + self, + message: str, + *, + status_code: int, + retry_after: float | None = None, + error_type: str | None = None, + error_code: str | None = None, + should_retry: bool | None = None, + response_body: str | None = None, + ): + super().__init__(message) + self.status_code = status_code + self.retry_after = retry_after + self.error_type = error_type + self.error_code = error_code + self.should_retry = should_retry + self.response_body = response_body + + +async def _fetch_xai_model_capabilities( + url: str, + headers: dict[str, str], + *, + proxy: str | None = None, +) -> dict[str, bool]: + client_kwargs: dict[str, Any] = {"timeout": 10.0, "follow_redirects": False} + if proxy: + client_kwargs.update(proxy=proxy, trust_env=False) + async with httpx.AsyncClient(**client_kwargs) as client: + response = await client.get(url, headers=headers) + if response.status_code != 200: + raw = response.content.decode("utf-8", "ignore") + raise _build_xai_http_error(response.status_code, response.headers, raw) + try: + payload = response.json() + except ValueError as exc: + raise RuntimeError("xAI model catalog returned invalid JSON.") from exc + return _parse_xai_model_capabilities(payload) + + +def _parse_xai_model_capabilities(payload: Any) -> dict[str, bool]: + if isinstance(payload, dict): + rows = payload.get("data") + if not isinstance(rows, list): + rows = payload.get("models") + else: + rows = payload + if not isinstance(rows, list): + return {} + + capabilities: dict[str, bool] = {} + for row in rows: + if not isinstance(row, dict): + continue + meta = row.get("_meta") if isinstance(row.get("_meta"), dict) else {} + support_value = row.get("supportsBackendSearch") + if not isinstance(support_value, bool): + support_value = row.get("supports_backend_search") + if not isinstance(support_value, bool): + support_value = meta.get("supportsBackendSearch") + if not isinstance(support_value, bool): + support_value = meta.get("supports_backend_search") + supports_backend_search = support_value if isinstance(support_value, bool) else False + + identifiers = ( + row.get("model"), + row.get("modelId"), + row.get("id"), + meta.get("model"), + meta.get("modelId"), + ) + for identifier in identifiers: + if isinstance(identifier, str) and identifier.strip(): + capabilities[_strip_model_prefix(identifier.strip())] = supports_backend_search + return capabilities + + +async def _request_xai( + url: str, + headers: dict[str, str], + body: dict[str, Any], + *, + proxy: str | None = None, + on_content_delta: Callable[[str], Awaitable[None]] | None = None, + on_thinking_delta: Callable[[str], Awaitable[None]] | None = None, + on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None, +) -> tuple[str, list[ToolCallRequest], str, dict[str, int], str | None]: + client_kwargs: dict[str, Any] = {"timeout": resolve_stream_idle_timeout_s()} + if proxy: + client_kwargs.update(proxy=proxy, trust_env=False) + async with httpx.AsyncClient(**client_kwargs) as client: + async with client.stream("POST", url, headers=headers, json=body) as response: + if response.status_code != 200: + content = await response.aread() + raw = content.decode("utf-8", "ignore") + raise _build_xai_http_error(response.status_code, response.headers, raw) + return await consume_sse_with_reasoning( + response, + on_content_delta=on_content_delta, + on_tool_call_delta=on_tool_call_delta, + on_reasoning_delta=on_thinking_delta, + ) + + +def _build_xai_http_error( + status_code: int, + headers: httpx.Headers, + raw: str, +) -> _XAIHTTPError: + retry_after = LLMProvider._extract_retry_after_from_headers(headers) + error_type, error_code = LLMProvider._extract_error_type_code(raw) + response_body = _bounded_error_body(raw) + return _XAIHTTPError( + _friendly_error(status_code, response_body), + status_code=status_code, + retry_after=retry_after, + error_type=error_type, + error_code=error_code, + should_retry=_should_retry_status(status_code, error_type, error_code, raw), + response_body=response_body, + ) + + +def _bounded_error_body(raw: str) -> str | None: + text = raw.strip() + if not text: + return None + + try: + payload = json.loads(text) + except (TypeError, ValueError): + pass + else: + text = json.dumps( + _redact_error_payload(payload), + ensure_ascii=False, + separators=(",", ":"), + ) + + text = re.sub(r"(?i)(bearer\s+)[a-z0-9._~+/=-]+", r"\1[REDACTED]", text) + text = " ".join(text.split()) + if len(text) > _MAX_ERROR_BODY_CHARS: + return f"{text[:_MAX_ERROR_BODY_CHARS]}…" + return text + + +def _redact_error_payload(payload: Any) -> Any: + if isinstance(payload, dict): + return { + key: "[REDACTED]" if _is_sensitive_error_key(key) else _redact_error_payload(value) + for key, value in payload.items() + } + if isinstance(payload, list): + return [_redact_error_payload(value) for value in payload] + return payload + + +def _is_sensitive_error_key(key: str) -> bool: + normalized = re.sub(r"[^a-z0-9]", "", key.casefold()) + return normalized in _SENSITIVE_ERROR_KEYS + + +def _friendly_error(status_code: int, response_body: str | None = None) -> str: + if status_code == 401: + message = "xAI rejected the login. Sign in again with `nanobot provider login xai-grok`." + elif status_code == 403: + message = "This xAI account or subscription cannot access the Grok subscription endpoint." + elif status_code == 426: + message = "xAI requires a newer Grok client version. Update nanobot and try again." + elif status_code == 429: + message = "xAI usage quota or rate limit reached. Please try again later." + else: + message = f"xAI subscription endpoint returned HTTP {status_code}." + if response_body: + return f"{message} Response body: {response_body}" + return message + + +def _xai_error_response(exc: Exception) -> LLMResponse: + status_code = getattr(exc, "status_code", None) + should_retry = getattr(exc, "should_retry", None) + error_kind: str | None = None + if isinstance(exc, (httpx.TimeoutException, asyncio.TimeoutError)): + error_kind = "timeout" + should_retry = True if should_retry is None else should_retry + elif isinstance(exc, (httpx.NetworkError, httpx.TransportError)): + error_kind = "connection" + should_retry = True if should_retry is None else should_retry + elif isinstance(exc, _XAIHTTPError): + error_kind = "http" + if status_code is not None and should_retry is None: + should_retry = _should_retry_status( + int(status_code), + getattr(exc, "error_type", None), + getattr(exc, "error_code", None), + None, + ) + message = str(exc).strip() or "unexpected error" + retry_after = getattr(exc, "retry_after", None) + return LLMResponse( + content=f"Error calling xAI ({type(exc).__name__}): {message}", + finish_reason="error", + retry_after=retry_after, + error_status_code=int(status_code) if status_code is not None else None, + error_kind=error_kind, + error_type=getattr(exc, "error_type", None), + error_code=getattr(exc, "error_code", None), + error_retry_after_s=retry_after, + error_should_retry=should_retry, + ) + + +def _should_retry_status( + status_code: int, + error_type: str | None, + error_code: str | None, + content: str | None, +) -> bool: + if status_code == 429: + return LLMProvider._is_retryable_429_response( + LLMResponse( + content=content or "", + finish_reason="error", + error_status_code=status_code, + error_type=error_type, + error_code=error_code, + ) + ) + return status_code in LLMProvider._RETRYABLE_STATUS_CODES or status_code >= 500 diff --git a/nanobot/providers/xai_oauth.py b/nanobot/providers/xai_oauth.py new file mode 100644 index 00000000..ca5e0dda --- /dev/null +++ b/nanobot/providers/xai_oauth.py @@ -0,0 +1,745 @@ +"""xAI subscription OAuth (Authorization Code + PKCE) support. + +This integration follows the public OAuth client contract used by Grok Build. +Credentials are stored separately from Grok Build so rotating refresh tokens are +never shared between the two applications. +""" + +from __future__ import annotations + +import base64 +import hashlib +import hmac +import json +import os +import queue +import secrets +import threading +import time +import webbrowser +from collections.abc import Callable +from contextlib import suppress +from dataclasses import asdict, dataclass +from http import HTTPStatus +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path +from typing import Any +from urllib.parse import parse_qs, urlencode, urlsplit + +import httpx +from filelock import FileLock +from loguru import logger + +from nanobot.config.paths import get_data_dir +from nanobot.utils.helpers import _write_text_atomic + +XAI_OAUTH_ISSUER = "https://auth.x.ai" +XAI_CLIENT_ID = "b1a00492-073a-47ea-816f-4c329264a828" +XAI_CLIENT_VERSION = "0.2.109" +XAI_ALLOWED_CALLBACK_ORIGIN = "https://accounts.x.ai" +XAI_OAUTH_SCOPES = ( + "openid", + "profile", + "email", + "offline_access", + "grok-cli:access", + "api:access", + "conversations:read", + "conversations:write", + "workspaces:read", + "workspaces:write", +) + +_DISCOVERY_URL = f"{XAI_OAUTH_ISSUER}/.well-known/openid-configuration" +_TOKEN_REFRESH_MARGIN_MS = 5 * 60 * 1000 +_DEFAULT_TOKEN_TTL_S = 60 * 60 +_HTTP_TIMEOUT_S = 15.0 + + +class XAIOAuthError(RuntimeError): + """An actionable xAI OAuth failure with no credential material.""" + + +@dataclass(frozen=True) +class XAIToken: + """Persisted xAI OAuth token material.""" + + access: str + refresh: str | None + expires: int + account_id: str | None = None + + @classmethod + def from_dict(cls, value: Any) -> XAIToken | None: + if not isinstance(value, dict): + return None + access = value.get("access") + if not isinstance(access, str) or not access: + return None + refresh = value.get("refresh") + if not isinstance(refresh, str) or not refresh: + refresh = None + try: + expires = int(value.get("expires") or 0) + except (TypeError, ValueError): + expires = 0 + account_id = value.get("account_id") + if not isinstance(account_id, str) or not account_id: + account_id = None + return cls(access=access, refresh=refresh, expires=expires, account_id=account_id) + + +@dataclass(frozen=True) +class _Discovery: + authorization_endpoint: str + token_endpoint: str + userinfo_endpoint: str | None + + +@dataclass(frozen=True) +class _CallbackResult: + code: str | None = None + state: str | None = None + error: str | None = None + + +class XAIOAuthLoginFlow: + """Pending xAI OAuth login that can finish through loopback or an authorization code.""" + + def __init__( + self, + *, + authorization_url: str, + redirect_uri: str, + verifier: str, + state: str, + discovery: _Discovery, + proxy: str | None, + result_queue: queue.Queue[_CallbackResult], + server: ThreadingHTTPServer, + timeout_s: float, + ) -> None: + self.authorization_url = authorization_url + self.redirect_uri = redirect_uri + self._verifier = verifier + self._state = state + self._discovery = discovery + self._proxy = proxy + self._result_queue = result_queue + self._server = server + self._expires_at = time.monotonic() + timeout_s + self._lock = threading.Lock() + self._stop_event = threading.Event() + self._token: XAIToken | None = None + self._error: Exception | None = None + self._closed = False + self._server_thread = threading.Thread( + target=_serve_callback_server, + args=(server, self._stop_event), + name="nanobot-xai-grok-oauth-callback", + daemon=True, + ) + self._server_thread.start() + self._timeout_timer = threading.Timer(timeout_s, self._expire) + self._timeout_timer.daemon = True + self._timeout_timer.start() + + @property + def expired(self) -> bool: + return time.monotonic() >= self._expires_at + + @property + def remaining_seconds(self) -> int: + return max(0, int(self._expires_at - time.monotonic())) + + def complete(self, authorization_code: str | None = None) -> XAIToken | None: + """Complete this flow, or return ``None`` while loopback is still pending.""" + with self._lock: + if self._token is not None: + return self._token + self._raise_if_finished() + + callback: _CallbackResult | None + if authorization_code is not None: + callback = _CallbackResult(code=authorization_code.strip()) + else: + try: + callback = self._result_queue.get_nowait() + except queue.Empty: + callback = None + if callback is None: + with self._lock: + if self._token is not None: + return self._token + self._raise_if_finished() + if self.expired: + self._expire_locked() + self._raise_if_finished() + return None + return self._finish(callback) + + def wait(self, timeout_s: float) -> XAIToken: + """Wait for the loopback callback and complete this flow.""" + with self._lock: + if self._token is not None: + return self._token + self._raise_if_finished() + try: + callback = self._result_queue.get(timeout=timeout_s) + except queue.Empty as exc: + with self._lock: + if self._error is not None: + raise self._error + raise XAIOAuthError( + "Timed out waiting for xAI sign-in. Run " + "`nanobot provider login xai-grok` to try again." + ) from exc + return self._finish(callback) + + def cancel(self) -> None: + """Stop the callback listener for an abandoned flow.""" + with self._lock: + if self._token is None and self._error is None: + self._error = XAIOAuthError("xAI sign-in was cancelled.") + self._close_locked() + + def _finish(self, callback: _CallbackResult) -> XAIToken: + with self._lock: + if self._token is not None: + return self._token + self._raise_if_finished() + self._close_locked() + try: + token = _exchange_callback( + callback, + expected_state=self._state, + discovery=self._discovery, + verifier=self._verifier, + redirect_uri=self.redirect_uri, + proxy=self._proxy, + ) + with _token_lock(): + _write_token(token) + except Exception as exc: + self._error = exc + raise + self._token = token + return token + + def _expire(self) -> None: + with self._lock: + if self._token is not None or self._error is not None: + return + self._expire_locked() + + def _expire_locked(self) -> None: + self._error = XAIOAuthError("xAI sign-in expired. Start a new sign-in flow.") + self._close_locked() + + def _raise_if_finished(self) -> None: + if self._token is not None: + return + if self._error is not None: + raise self._error + + def _close_locked(self) -> None: + if self._closed: + return + self._closed = True + self._timeout_timer.cancel() + self._stop_event.set() + if threading.current_thread() is not self._server_thread: + self._server_thread.join(timeout=2) + + +def get_xai_oauth_storage_path() -> Path: + """Return the instance-scoped xAI OAuth credential path.""" + return get_data_dir() / "auth" / "xai.json" + + +def get_xai_oauth_login_status() -> XAIToken | None: + """Return locally stored login state without making a network request.""" + return _load_token() + + +def logout_xai_oauth() -> bool: + """Remove this instance's credentials while excluding token refreshes.""" + path = get_xai_oauth_storage_path() + with _token_lock(): + try: + path.unlink() + except FileNotFoundError: + return False + return True + + +def login_xai_oauth( + *, + print_fn: Callable[[str], None] = print, + prompt_fn: Callable[[str], str] | None = None, + proxy: str | None = None, + callback_timeout_s: float = 600, + browser_opener: Callable[[str], bool] = webbrowser.open, +) -> XAIToken: + """Run xAI's browser-based OAuth flow and persist the resulting token. + + ``prompt_fn`` is used only when no local browser could be opened. It lets a + headless user paste either the final callback URL or its authorization code. + """ + flow = start_xai_oauth_login(proxy=proxy, timeout_s=callback_timeout_s) + + try: + print_fn("Opening xAI sign-in in your browser...") + print_fn(f"If it does not open automatically, visit:\n{flow.authorization_url}") + opened = False + with suppress(Exception): + opened = bool(browser_opener(flow.authorization_url)) + + if not opened and prompt_fn is not None: + pasted = prompt_fn("Paste the final callback URL (or authorization code)") + token = flow.complete(pasted) + if token is None: # pragma: no cover - pasted input always resolves a callback + raise XAIOAuthError("xAI sign-in returned no authorization code.") + return token + return flow.wait(callback_timeout_s) + finally: + flow.cancel() + + +def start_xai_oauth_login( + *, + proxy: str | None = None, + timeout_s: float = 600, +) -> XAIOAuthLoginFlow: + """Create a non-blocking OAuth flow for browser or pasted-callback completion.""" + discovery = _discover(proxy) + verifier, challenge = _generate_pkce() + state = secrets.token_urlsafe(32) + nonce = secrets.token_urlsafe(32) + result_queue: queue.Queue[_CallbackResult] = queue.Queue(maxsize=1) + server = _make_callback_server(state, result_queue) + redirect_uri = f"http://127.0.0.1:{server.server_port}/callback" + authorization_url = _build_authorize_url( + discovery.authorization_endpoint, + redirect_uri=redirect_uri, + challenge=challenge, + state=state, + nonce=nonce, + ) + return XAIOAuthLoginFlow( + authorization_url=authorization_url, + redirect_uri=redirect_uri, + verifier=verifier, + state=state, + discovery=discovery, + proxy=proxy, + result_queue=result_queue, + server=server, + timeout_s=timeout_s, + ) + + +def complete_xai_oauth_login( + flow: XAIOAuthLoginFlow, + authorization_code: str | None = None, +) -> XAIToken | None: + """Complete a pending login from loopback state or a pasted authorization code.""" + return flow.complete(authorization_code) + + +def get_xai_oauth_token( + *, + proxy: str | None = None, + min_ttl_ms: int = _TOKEN_REFRESH_MARGIN_MS, + force_refresh: bool = False, +) -> XAIToken: + """Load a usable token, refreshing it under an inter-process lock when needed.""" + token = _load_token() + if token is None: + raise XAIOAuthError( + "xAI is not signed in. Run `nanobot provider login xai-grok` first." + ) + if not force_refresh and _token_is_fresh(token, min_ttl_ms): + return token + if not token.refresh: + if not force_refresh and token.expires > _now_ms(): + return token + raise XAIOAuthError( + "The xAI login has expired and cannot be refreshed. " + "Run `nanobot provider login xai-grok` again." + ) + + with _token_lock(): + latest = _load_token() + if latest is None: + raise XAIOAuthError( + "xAI is not signed in. Run `nanobot provider login xai-grok` first." + ) + if not force_refresh and _token_is_fresh(latest, min_ttl_ms): + return latest + if not latest.refresh: + raise XAIOAuthError( + "The xAI login has expired and cannot be refreshed. " + "Run `nanobot provider login xai-grok` again." + ) + refreshed = _refresh_token(latest, proxy) + _write_token(refreshed) + return refreshed + + +def _discover(proxy: str | None) -> _Discovery: + try: + with _http_client(proxy) as client: + response = client.get(_DISCOVERY_URL) + except httpx.HTTPError as exc: + raise XAIOAuthError(f"Could not reach xAI sign-in: {type(exc).__name__}.") from exc + if response.status_code != HTTPStatus.OK: + raise _oauth_http_error(response, "discovery") + try: + payload = response.json() + except ValueError as exc: + raise XAIOAuthError("xAI sign-in discovery returned invalid JSON.") from exc + if payload.get("issuer", "").rstrip("/") != XAI_OAUTH_ISSUER: + raise XAIOAuthError("xAI sign-in discovery returned an unexpected issuer.") + authorization_endpoint = _validate_xai_endpoint( + payload.get("authorization_endpoint"), "authorization" + ) + token_endpoint = _validate_xai_endpoint(payload.get("token_endpoint"), "token") + userinfo_raw = payload.get("userinfo_endpoint") + userinfo_endpoint = ( + _validate_xai_endpoint(userinfo_raw, "userinfo") if userinfo_raw else None + ) + return _Discovery(authorization_endpoint, token_endpoint, userinfo_endpoint) + + +def _validate_xai_endpoint(value: Any, label: str) -> str: + if not isinstance(value, str): + raise XAIOAuthError(f"xAI sign-in discovery omitted the {label} endpoint.") + parsed = urlsplit(value) + try: + port = parsed.port + except ValueError as exc: + raise XAIOAuthError( + f"xAI sign-in discovery returned an unsafe {label} endpoint." + ) from exc + if ( + parsed.scheme != "https" + or parsed.hostname != "auth.x.ai" + or parsed.username is not None + or parsed.password is not None + or port not in (None, 443) + ): + raise XAIOAuthError(f"xAI sign-in discovery returned an unsafe {label} endpoint.") + return value + + +def _generate_pkce() -> tuple[str, str]: + verifier = _base64url(secrets.token_bytes(32)) + challenge = _base64url(hashlib.sha256(verifier.encode("ascii")).digest()) + return verifier, challenge + + +def _base64url(value: bytes) -> str: + return base64.urlsafe_b64encode(value).rstrip(b"=").decode("ascii") + + +def _build_authorize_url( + endpoint: str, + *, + redirect_uri: str, + challenge: str, + state: str, + nonce: str, +) -> str: + params = { + "response_type": "code", + "client_id": XAI_CLIENT_ID, + "redirect_uri": redirect_uri, + "scope": " ".join(XAI_OAUTH_SCOPES), + "code_challenge": challenge, + "code_challenge_method": "S256", + "state": state, + "nonce": nonce, + "referrer": "nanobot", + } + return f"{endpoint}?{urlencode(params)}" + + +def _make_callback_server( + expected_state: str, + result_queue: queue.Queue[_CallbackResult], +) -> ThreadingHTTPServer: + class CallbackHandler(BaseHTTPRequestHandler): + def do_OPTIONS(self) -> None: # noqa: N802 + if self.path.split("?", 1)[0] != "/callback": + self.send_error(HTTPStatus.NOT_FOUND) + return + self.send_response(HTTPStatus.NO_CONTENT) + self._send_cors_headers() + self.send_header("Access-Control-Allow-Methods", "GET, OPTIONS") + self.send_header("Access-Control-Allow-Headers", "Content-Type") + self.send_header("Access-Control-Allow-Private-Network", "true") + self.end_headers() + + def do_GET(self) -> None: # noqa: N802 + parsed = urlsplit(self.path) + if parsed.path != "/callback": + self.send_error(HTTPStatus.NOT_FOUND) + return + params = parse_qs(parsed.query) + code = _first(params, "code") + received_state = _first(params, "state") + error = _first(params, "error_description") or _first(params, "error") + if code and received_state and hmac.compare_digest(received_state, expected_state): + result = _CallbackResult(code=code, state=received_state) + title = "Signed in to xAI" + message = "You can close this tab and return to nanobot." + elif code: + result = _CallbackResult(error="OAuth state mismatch") + title = "Sign-in failed" + message = "The sign-in response could not be verified. Return to nanobot and retry." + else: + result = _CallbackResult(error=error or "access denied") + title = "Access denied" + message = "Return to nanobot and try signing in again." + with suppress(queue.Full): + result_queue.put_nowait(result) + body = _callback_page(title, message) + encoded = body.encode("utf-8") + self.send_response(HTTPStatus.OK) + self._send_cors_headers() + self.send_header("Content-Type", "text/html; charset=utf-8") + self.send_header("Content-Length", str(len(encoded))) + self.send_header("Cache-Control", "no-store") + self.end_headers() + self.wfile.write(encoded) + + def _send_cors_headers(self) -> None: + origin = self.headers.get("Origin") + if origin == XAI_ALLOWED_CALLBACK_ORIGIN: + self.send_header("Access-Control-Allow-Origin", origin) + self.send_header("Vary", "Origin") + self.send_header("Access-Control-Allow-Private-Network", "true") + + def log_message(self, _format: str, *_args: Any) -> None: + # Callback query strings contain an authorization code. + return + + return ThreadingHTTPServer(("127.0.0.1", 0), CallbackHandler) + + +def _serve_callback_server( + server: ThreadingHTTPServer, + stop_event: threading.Event, +) -> None: + server.timeout = 0.2 + try: + while not stop_event.is_set(): + server.handle_request() + finally: + server.server_close() + + +def _first(params: dict[str, list[str]], key: str) -> str | None: + values = params.get(key) + return values[0] if values else None + + +def _callback_page(title: str, message: str) -> str: + return f""" + +{title}

{title}

{message}

""" + + +def _exchange_callback( + callback: _CallbackResult, + *, + expected_state: str, + discovery: _Discovery, + verifier: str, + redirect_uri: str, + proxy: str | None, +) -> XAIToken: + if callback.error: + raise XAIOAuthError(f"xAI sign-in was not completed: {callback.error}") + if not callback.code: + raise XAIOAuthError("xAI sign-in returned no authorization code.") + if callback.state and not hmac.compare_digest(callback.state, expected_state): + raise XAIOAuthError("xAI sign-in failed because the OAuth state did not match.") + + payload = _exchange_code( + discovery.token_endpoint, + code=callback.code, + verifier=verifier, + redirect_uri=redirect_uri, + proxy=proxy, + ) + account_id = _fetch_account(discovery.userinfo_endpoint, payload["access_token"], proxy) + return _token_from_response(payload, account_id=account_id) + + +def _exchange_code( + token_endpoint: str, + *, + code: str, + verifier: str, + redirect_uri: str, + proxy: str | None, +) -> dict[str, Any]: + data = { + "grant_type": "authorization_code", + "code": code, + "redirect_uri": redirect_uri, + "client_id": XAI_CLIENT_ID, + "code_verifier": verifier, + } + try: + with _http_client(proxy) as client: + response = client.post( + token_endpoint, + data=data, + headers={"x-grok-client-version": XAI_CLIENT_VERSION}, + ) + except httpx.HTTPError as exc: + raise XAIOAuthError(f"Could not exchange the xAI sign-in code: {type(exc).__name__}.") from exc + if not response.is_success: + raise _oauth_http_error(response, "token exchange") + return _token_payload(response) + + +def _refresh_token(token: XAIToken, proxy: str | None) -> XAIToken: + data = { + "grant_type": "refresh_token", + "refresh_token": token.refresh or "", + "client_id": XAI_CLIENT_ID, + } + try: + with _http_client(proxy) as client: + response = client.post( + f"{XAI_OAUTH_ISSUER}/oauth2/token", + data=data, + headers={"x-grok-client-version": XAI_CLIENT_VERSION}, + ) + except httpx.HTTPError as exc: + raise XAIOAuthError(f"Could not refresh the xAI login: {type(exc).__name__}.") from exc + if not response.is_success: + raise _oauth_http_error(response, "token refresh") + payload = _token_payload(response) + return _token_from_response( + payload, + account_id=token.account_id, + previous_refresh=token.refresh, + ) + + +def _token_payload(response: httpx.Response) -> dict[str, Any]: + try: + payload = response.json() + except ValueError as exc: + raise XAIOAuthError("xAI sign-in returned an invalid token response.") from exc + if not isinstance(payload, dict) or not isinstance(payload.get("access_token"), str): + raise XAIOAuthError("xAI sign-in returned no access token.") + return payload + + +def _token_from_response( + payload: dict[str, Any], + *, + account_id: str | None, + previous_refresh: str | None = None, +) -> XAIToken: + try: + expires_in = max(1, int(payload.get("expires_in") or _DEFAULT_TOKEN_TTL_S)) + except (TypeError, ValueError): + expires_in = _DEFAULT_TOKEN_TTL_S + refresh = payload.get("refresh_token") + if not isinstance(refresh, str) or not refresh: + refresh = previous_refresh + return XAIToken( + access=payload["access_token"], + refresh=refresh, + expires=_now_ms() + expires_in * 1000, + account_id=account_id, + ) + + +def _fetch_account(endpoint: str | None, access_token: str, proxy: str | None) -> str | None: + if not endpoint: + return None + try: + with _http_client(proxy) as client: + response = client.get(endpoint, headers={"Authorization": f"Bearer {access_token}"}) + if not response.is_success: + return None + payload = response.json() + except (httpx.HTTPError, ValueError): + return None + if not isinstance(payload, dict): + return None + for key in ("email", "preferred_username", "name", "sub"): + value = payload.get(key) + if isinstance(value, str) and value: + return value + return None + + +def _oauth_http_error(response: httpx.Response, action: str) -> XAIOAuthError: + code: str | None = None + description: str | None = None + with suppress(ValueError): + payload = response.json() + if isinstance(payload, dict): + raw_code = payload.get("error") + raw_description = payload.get("error_description") or payload.get("message") + code = raw_code[:80] if isinstance(raw_code, str) else None + description = raw_description[:200] if isinstance(raw_description, str) else None + detail = ": ".join(value for value in (code, description) if value) + suffix = f" ({detail})" if detail else "" + return XAIOAuthError(f"xAI OAuth {action} failed with HTTP {response.status_code}{suffix}.") + + +def _http_client(proxy: str | None) -> httpx.Client: + kwargs: dict[str, Any] = {"timeout": _HTTP_TIMEOUT_S} + if proxy: + kwargs.update(proxy=proxy, trust_env=False) + return httpx.Client(**kwargs) + + +def _token_lock() -> FileLock: + path = get_xai_oauth_storage_path() + path.parent.mkdir(parents=True, exist_ok=True) + return FileLock(str(path.with_suffix(".lock")), timeout=15) + + +def _load_token() -> XAIToken | None: + path = get_xai_oauth_storage_path() + try: + payload = json.loads(path.read_text(encoding="utf-8")) + except FileNotFoundError: + return None + except (OSError, ValueError, TypeError) as exc: + logger.warning("Could not read xAI OAuth credentials: {}", type(exc).__name__) + return None + return XAIToken.from_dict(payload) + + +def _write_token(token: XAIToken) -> None: + path = get_xai_oauth_storage_path() + path.parent.mkdir(parents=True, exist_ok=True) + with suppress(OSError): + os.chmod(path.parent, 0o700) + _write_text_atomic(path, json.dumps(asdict(token), indent=2, ensure_ascii=False)) + with suppress(OSError): + os.chmod(path, 0o600) + + +def _token_is_fresh(token: XAIToken, min_ttl_ms: int) -> bool: + return bool(token.access and token.expires > _now_ms() + max(0, min_ttl_ms)) + + +def _now_ms() -> int: + return int(time.time() * 1000) diff --git a/nanobot/webui/settings_api.py b/nanobot/webui/settings_api.py index 113b26e1..152e5a0c 100644 --- a/nanobot/webui/settings_api.py +++ b/nanobot/webui/settings_api.py @@ -8,6 +8,8 @@ from __future__ import annotations import os import re +import secrets +import threading import time from contextlib import suppress from typing import Any, Literal @@ -121,7 +123,12 @@ _IMAGE_GENERATION_ASPECT_RATIOS = { "2:3", "21:9", } -_CONTEXT_WINDOW_TOKEN_OPTIONS = {65_536, 200_000, 262_144, 1_048_576} +_CONTEXT_WINDOW_TOKEN_OPTIONS = {65_536, 200_000, 262_144, 500_000, 1_048_576} +_OAUTH_PROXY_PROVIDERS = {"openai_codex", "xai_grok"} +_XAI_WEBUI_OAUTH_TIMEOUT_S = 600 +_XAI_WEBUI_OAUTH_MAX_FLOWS = 8 +_xai_webui_oauth_flows: dict[str, Any] = {} +_xai_webui_oauth_flows_lock = threading.Lock() _MODEL_CONFIGURATION_SLUG_RE = re.compile(r"[^a-z0-9_-]+") _ENV_REF_RE = re.compile(r"\$\{([A-Za-z_][A-Za-z0-9_]*)\}") @@ -304,6 +311,32 @@ def _oauth_provider_status(spec: Any) -> dict[str, Any]: "login_supported": True, } + if spec.name == "xai_grok": + try: + from nanobot.providers.xai_oauth import get_xai_oauth_login_status + except Exception: + return { + "configured": False, + "account": None, + "expires_at": None, + "login_supported": False, + } + token = None + with suppress(Exception): + token = get_xai_oauth_login_status() + expires_at = getattr(token, "expires", None) if token else None + now_ms = int(time.time() * 1000) + return { + "configured": bool( + token + and token.access + and (getattr(token, "refresh", None) or (expires_at and expires_at > now_ms)) + ), + "account": getattr(token, "account_id", None) if token else None, + "expires_at": expires_at, + "login_supported": True, + } + return {"configured": False, "account": None, "expires_at": None, "login_supported": False} @@ -374,6 +407,8 @@ def _provider_settings_row( row["oauth_account"] = oauth_status["account"] row["oauth_expires_at"] = oauth_status["expires_at"] row["oauth_login_supported"] = oauth_status["login_supported"] + if spec.name in _OAUTH_PROXY_PROVIDERS: + row["proxy"] = provider_config.proxy if spec.name == "openai": row["api_type"] = provider_config.api_type return row @@ -635,7 +670,7 @@ def _parse_context_window_tokens(value: str | None) -> int | None: raise WebUISettingsError("context_window_tokens must be an integer") from None if parsed not in _CONTEXT_WINDOW_TOKEN_OPTIONS: raise WebUISettingsError( - "context_window_tokens must be 65536, 200000, 262144, or 1048576" + "context_window_tokens must be 65536, 200000, 262144, 500000, or 1048576" ) return parsed @@ -1171,7 +1206,23 @@ def update_provider_settings(query: QueryParams) -> dict[str, Any]: raise WebUISettingsError("unknown provider") spec, provider_key, provider_config = resolved_provider if spec.is_oauth: - raise WebUISettingsError("unknown provider") + if spec.name not in _OAUTH_PROXY_PROVIDERS: + raise WebUISettingsError("unknown provider") + if any( + key in query + for key in ("api_key", "apiKey", "api_base", "apiBase", "api_type") + ): + raise WebUISettingsError("OAuth provider only supports proxy settings") + + changed = False + if "proxy" in query: + proxy = (_query_first(query, "proxy") or "").strip() or None + if provider_config.proxy != proxy: + provider_config.proxy = proxy + changed = True + if changed: + save_config(config) + return settings_payload() changed = False if "api_key" in query or "apiKey" in query: @@ -1263,9 +1314,68 @@ def login_oauth_provider(query: QueryParams) -> dict[str, Any]: raise WebUISettingsError("OAuth login failed", status=401) return settings_payload() + if spec.name == "xai_grok": + from nanobot.providers.xai_oauth import start_xai_oauth_login + + try: + proxy = resolve_config_env_vars(load_config()).providers.xai_grok.proxy or None + except ValueError as e: + raise WebUISettingsError(str(e), status=400) from e + try: + flow = start_xai_oauth_login( + proxy=proxy, + timeout_s=_XAI_WEBUI_OAUTH_TIMEOUT_S, + ) + except Exception as e: + raise WebUISettingsError(f"xAI OAuth login failed: {e}", status=502) from e + flow_id = secrets.token_urlsafe(24) + _register_xai_webui_oauth_flow(flow_id, flow) + return { + "status": "authorization_required", + "provider": spec.name, + "flow_id": flow_id, + "authorization_url": flow.authorization_url, + "expires_in": flow.remaining_seconds, + } + raise WebUISettingsError("OAuth login is not supported for this provider") +def complete_oauth_provider( + query: QueryParams, + authorization_code: str | None = None, +) -> dict[str, Any]: + provider_name = (_query_first(query, "provider") or "").strip() + flow_id = (_query_first(query, "flow_id") or "").strip() + spec = find_by_name(provider_name) + if spec is None or spec.name != "xai_grok": + raise WebUISettingsError("OAuth completion is not supported for this provider") + if not flow_id: + raise WebUISettingsError("flow_id is required") + + flow = _get_xai_webui_oauth_flow(flow_id) + if flow is None: + raise WebUISettingsError("xAI sign-in expired. Start again.", status=410) + + from nanobot.providers.xai_oauth import complete_xai_oauth_login + + try: + token = complete_xai_oauth_login(flow, authorization_code) + except Exception as e: + _remove_xai_webui_oauth_flow(flow_id, flow) + raise WebUISettingsError(f"xAI OAuth login failed: {e}", status=502) from e + if token is None: + return { + "status": "pending", + "provider": spec.name, + "flow_id": flow_id, + } + _remove_xai_webui_oauth_flow(flow_id, flow, cancel=False) + if not token.access: + raise WebUISettingsError("OAuth login failed", status=401) + return settings_payload() + + def logout_oauth_provider(query: QueryParams) -> dict[str, Any]: provider_name = (_query_first(query, "provider") or "").strip() if not provider_name: @@ -1291,6 +1401,12 @@ def logout_oauth_provider(query: QueryParams) -> dict[str, Any]: "oauth_cli_kit not installed. Run: pip install oauth-cli-kit", status=500 ) from None token_path = get_storage().get_token_path() + elif spec.name == "xai_grok": + from nanobot.providers.xai_oauth import logout_xai_oauth + + _clear_xai_webui_oauth_flows() + logout_xai_oauth() + return settings_payload() else: raise WebUISettingsError("OAuth logout is not supported for this provider") @@ -1300,6 +1416,51 @@ def logout_oauth_provider(query: QueryParams) -> dict[str, Any]: return settings_payload() +def _register_xai_webui_oauth_flow(flow_id: str, flow: Any) -> None: + discarded: list[Any] = [] + with _xai_webui_oauth_flows_lock: + for existing_id, existing in list(_xai_webui_oauth_flows.items()): + if existing.expired: + discarded.append(_xai_webui_oauth_flows.pop(existing_id)) + while len(_xai_webui_oauth_flows) >= _XAI_WEBUI_OAUTH_MAX_FLOWS: + oldest_id = next(iter(_xai_webui_oauth_flows)) + discarded.append(_xai_webui_oauth_flows.pop(oldest_id)) + _xai_webui_oauth_flows[flow_id] = flow + for existing in discarded: + existing.cancel() + + +def _get_xai_webui_oauth_flow(flow_id: str) -> Any | None: + with _xai_webui_oauth_flows_lock: + flow = _xai_webui_oauth_flows.get(flow_id) + if flow is None or not flow.expired: + return flow + _xai_webui_oauth_flows.pop(flow_id, None) + flow.cancel() + return None + + +def _remove_xai_webui_oauth_flow( + flow_id: str, + flow: Any, + *, + cancel: bool = True, +) -> None: + with _xai_webui_oauth_flows_lock: + if _xai_webui_oauth_flows.get(flow_id) is flow: + _xai_webui_oauth_flows.pop(flow_id) + if cancel: + flow.cancel() + + +def _clear_xai_webui_oauth_flows() -> None: + with _xai_webui_oauth_flows_lock: + flows = list(_xai_webui_oauth_flows.values()) + _xai_webui_oauth_flows.clear() + for flow in flows: + flow.cancel() + + def update_network_safety_settings(query: QueryParams) -> dict[str, Any]: raw_allow = ( _query_first_alias(query, "webui_allow_local_service_access", "webuiAllowLocalServiceAccess") diff --git a/nanobot/webui/settings_routes.py b/nanobot/webui/settings_routes.py index 1464b8bb..1060f0f2 100644 --- a/nanobot/webui/settings_routes.py +++ b/nanobot/webui/settings_routes.py @@ -37,6 +37,7 @@ from nanobot.optional_features import ( ) from nanobot.pairing import approve_code, deny_code, list_pending from nanobot.webui.cli_apps_api import cli_apps_action, cli_apps_payload +from nanobot.webui.http_utils import case_insensitive_header from nanobot.webui.http_utils import is_local_browser_request as _is_local_browser_request from nanobot.webui.http_utils import query_first as _query_first from nanobot.webui.mcp_presets_api import mcp_presets_settings_action @@ -47,6 +48,7 @@ from nanobot.webui.nanobot_features_api import ( ) from nanobot.webui.settings_api import ( WebUISettingsError, + complete_oauth_provider, create_model_configuration, decorate_settings_payload, login_oauth_provider, @@ -73,6 +75,8 @@ _CHANNEL_VALUES_HEADER = "X-Nanobot-Channel-Values" _CHANNEL_VALUES_HEADER_MAX_BYTES = 64 * 1024 _API_SERVICE_VALUES_HEADER = "X-Nanobot-API-Service-Values" _API_SERVICE_VALUES_HEADER_MAX_BYTES = 8 * 1024 +_OAUTH_CODE_HEADER = "X-Nanobot-OAuth-Code" +_OAUTH_CODE_HEADER_MAX_BYTES = 8 * 1024 _SKIP_FIELD = object() _CHANNEL_CONNECT_ACTIONS = frozenset({"start", "poll", "cancel"}) @@ -146,6 +150,8 @@ class WebUISettingsRouter: return await self._handle_settings_provider_models(request) if path == "/api/settings/provider/oauth-login": return await self._handle_settings_provider_oauth(request, "login") + if path == "/api/settings/provider/oauth-login/complete": + return await self._handle_settings_provider_oauth(request, "complete") if path == "/api/settings/provider/oauth-logout": return await self._handle_settings_provider_oauth(request, "logout") if path == "/api/settings/web-search/update": @@ -378,10 +384,24 @@ class WebUISettingsRouter: try: if action == "login": payload = await asyncio.to_thread(login_oauth_provider, query) + elif action == "complete": + authorization_code = case_insensitive_header( + request.headers, + _OAUTH_CODE_HEADER, + ) + if len(authorization_code.encode("utf-8")) > _OAUTH_CODE_HEADER_MAX_BYTES: + raise WebUISettingsError("OAuth authorization code is too large") + payload = await asyncio.to_thread( + complete_oauth_provider, + query, + authorization_code or None, + ) else: payload = await asyncio.to_thread(logout_oauth_provider, query) except WebUISettingsError as e: return self._error_response(e.status, e.message) + if payload.get("status") in {"authorization_required", "pending"}: + return self._json_response(payload) return self._json_response(self._with_restart_state(payload)) def _handle_settings_web_search_update(self, request: WsRequest) -> Response: diff --git a/tests/cli/test_commands.py b/tests/cli/test_commands.py index 9843f6d2..0b9ba4d8 100644 --- a/tests/cli/test_commands.py +++ b/tests/cli/test_commands.py @@ -478,6 +478,7 @@ def test_config_dump_excludes_oauth_provider_blocks(): providers = config.model_dump(by_alias=True)["providers"] assert "openaiCodex" not in providers + assert "xaiGrok" not in providers assert "githubCopilot" not in providers @@ -541,6 +542,47 @@ def test_provider_logout_openai_codex_succeeds_when_no_local_oauth_file(monkeypa assert "No local OAuth credentials found for OpenAI Codex" in result.stdout +def test_provider_logout_xai_grok_removes_instance_credentials(tmp_path, monkeypatch): + token_path = tmp_path / "auth" / "xai.json" + lock_path = token_path.with_suffix(".lock") + token_path.parent.mkdir(parents=True, exist_ok=True) + token_path.write_text("{}", encoding="utf-8") + lock_path.write_text("", encoding="utf-8") + monkeypatch.setattr( + "nanobot.providers.xai_oauth.get_xai_oauth_storage_path", + lambda: token_path, + ) + + result = runner.invoke(app, ["provider", "logout", "xai-grok"]) + + assert result.exit_code == 0 + assert not token_path.exists() + assert "Logged out from xAI Grok" in result.stdout + + +def test_provider_logout_xai_grok_uses_explicit_config_path(tmp_path, monkeypatch): + from nanobot.config import loader + + default_config = tmp_path / "default" / "config.json" + selected_config = tmp_path / "selected" / "config.json" + default_token = default_config.parent / "auth" / "xai.json" + selected_token = selected_config.parent / "auth" / "xai.json" + for token_path in (default_token, selected_token): + token_path.parent.mkdir(parents=True, exist_ok=True) + token_path.write_text("{}", encoding="utf-8") + monkeypatch.setattr(loader, "_current_config_path", default_config) + + result = runner.invoke( + app, + ["provider", "logout", "xai-grok", "--config", str(selected_config)], + ) + + assert result.exit_code == 0 + assert default_token.exists() + assert not selected_token.exists() + assert "Using config:" in result.stdout + + def test_provider_logout_github_copilot_removes_local_oauth_files(tmp_path, monkeypatch): token_path = tmp_path / "auth" / "github-copilot.json" lock_path = token_path.with_suffix(".lock") @@ -579,12 +621,17 @@ def test_provider_logout_paths_resolve_to_expected_files(): from oauth_cli_kit.storage import FileTokenStorage from nanobot.providers.github_copilot_provider import get_storage + from nanobot.providers.xai_oauth import get_xai_oauth_storage_path codex_storage = FileTokenStorage(token_filename=OPENAI_CODEX_PROVIDER.token_filename) codex_path = codex_storage.get_token_path() assert codex_path.name == "codex.json" assert codex_path.parent.name == "auth" + xai_path = get_xai_oauth_storage_path() + assert xai_path.name == "xai.json" + assert xai_path.parent.name == "auth" + gh_storage = get_storage() gh_path = gh_storage.get_token_path() assert gh_path.name == "github-copilot.json" @@ -663,6 +710,36 @@ def test_provider_login_can_set_github_copilot_as_main_provider(tmp_path): assert make_provider(saved).__class__.__name__ == "GitHubCopilotProvider" +def test_provider_login_can_set_xai_grok_as_main_provider(tmp_path): + config_path = tmp_path / "config.json" + original = cli_commands._LOGIN_HANDLERS["xai_grok"] + cli_commands._LOGIN_HANDLERS["xai_grok"] = lambda: None + try: + result = runner.invoke( + app, + [ + "provider", + "login", + "xai-grok", + "--set-main", + "--config", + str(config_path), + ], + ) + finally: + cli_commands._LOGIN_HANDLERS["xai_grok"] = original + + assert result.exit_code == 0 + assert "Set xai-grok as the main provider" in result.stdout + + saved = Config.model_validate(json.loads(config_path.read_text(encoding="utf-8"))) + assert saved.agents.defaults.provider == "xai_grok" + assert saved.agents.defaults.model == "xai-grok/grok-4.5" + assert saved.agents.defaults.context_window_tokens == 500_000 + assert saved.agents.defaults.model_preset is None + assert make_provider(saved).__class__.__name__ == "XAIGrokProvider" + + def test_provider_login_model_implies_set_main_provider(tmp_path): config_path = tmp_path / "config.json" original = cli_commands._LOGIN_HANDLERS["github_copilot"] @@ -792,6 +869,33 @@ def test_provider_login_openai_codex_resolves_proxy_env_ref(monkeypatch): assert captured["proxy"] == proxy +def test_provider_login_xai_grok_runs_browser_flow_with_configured_proxy(monkeypatch): + proxy = "http://127.0.0.1:23458" + monkeypatch.setattr( + "nanobot.config.loader.load_config", + lambda: Config.model_validate({"providers": {"xaiGrok": {"proxy": proxy}}}), + ) + monkeypatch.setattr( + "nanobot.providers.xai_oauth.get_xai_oauth_token", + lambda **_kwargs: (_ for _ in ()).throw(RuntimeError("not signed in")), + ) + captured: dict[str, object] = {} + + def fake_login(*, print_fn, prompt_fn, proxy=None): + captured.update(print_fn=print_fn, prompt_fn=prompt_fn, proxy=proxy) + return SimpleNamespace(access="access-token", account_id="user@example.com") + + monkeypatch.setattr("nanobot.providers.xai_oauth.login_xai_oauth", fake_login) + + result = runner.invoke(app, ["provider", "login", "xai-grok"]) + + assert result.exit_code == 0 + assert captured["proxy"] == proxy + assert callable(captured["print_fn"]) + assert callable(captured["prompt_fn"]) + assert "Hosted X Search is enabled automatically when the selected model supports it" in result.stdout + + def test_config_matches_explicit_ollama_prefix_without_api_key(): config = Config() config.agents.defaults.model = "ollama/llama3.2" diff --git a/tests/config/test_env_interpolation.py b/tests/config/test_env_interpolation.py index 329010cc..fb01bda9 100644 --- a/tests/config/test_env_interpolation.py +++ b/tests/config/test_env_interpolation.py @@ -111,6 +111,7 @@ class TestResolveConfig: "agents": {"defaults": {"dream": {"cron": "0 */4 * * *"}}}, "providers": { "openaiCodex": {"apiKey": "codex-secret"}, + "xaiGrok": {"apiKey": "xai-secret"}, "githubCopilot": {"apiKey": "copilot-secret"}, "groq": {"apiKey": "groq-secret"}, }, @@ -125,6 +126,7 @@ class TestResolveConfig: saved = json.loads(config_path.read_text(encoding="utf-8")) assert saved["agents"]["defaults"]["dream"]["cron"] == "0 */4 * * *" assert "openaiCodex" not in saved["providers"] + assert "xaiGrok" not in saved["providers"] assert "githubCopilot" not in saved["providers"] assert saved["providers"]["groq"]["apiKey"] == "groq-secret" @@ -137,6 +139,7 @@ class TestResolveConfig: "openaiCodex": { "apiKey": "codex-secret", "proxy": proxy, + "extraBody": {"service_tier": "priority"}, }, "groq": {"apiKey": "groq-secret"}, } @@ -146,13 +149,77 @@ class TestResolveConfig: save_config(config, config_path) saved = json.loads(config_path.read_text(encoding="utf-8")) - assert saved["providers"]["openaiCodex"] == {"proxy": proxy} + assert saved["providers"]["openaiCodex"] == { + "extraBody": {"service_tier": "priority"}, + "proxy": proxy, + } assert saved["providers"]["groq"]["apiKey"] == "groq-secret" reloaded = load_config(config_path) assert reloaded.providers.openai_codex.proxy == proxy + assert reloaded.providers.openai_codex.extra_body == {"service_tier": "priority"} assert reloaded.providers.openai_codex.api_key is None + def test_save_preserves_xai_grok_proxy_but_never_credentials(self, tmp_path): + config_path = tmp_path / "config.json" + proxy = "http://127.0.0.1:23458" + config = Config.model_validate( + { + "providers": { + "xaiGrok": { + "apiKey": "must-not-be-saved", + "proxy": proxy, + "extraBody": {"parallel_tool_calls": False}, + } + } + } + ) + + save_config(config, config_path) + + saved = json.loads(config_path.read_text(encoding="utf-8")) + assert saved["providers"]["xaiGrok"] == { + "extraBody": {"parallel_tool_calls": False}, + "proxy": proxy, + } + assert "must-not-be-saved" not in config_path.read_text(encoding="utf-8") + + reloaded = load_config(config_path) + assert reloaded.providers.xai_grok.proxy == proxy + assert reloaded.providers.xai_grok.extra_body == {"parallel_tool_calls": False} + assert reloaded.providers.xai_grok.api_key is None + + def test_save_preserves_settings_across_oauth_provider_blocks(self, tmp_path): + config_path = tmp_path / "config.json" + config = Config.model_validate( + { + "providers": { + "openaiCodex": { + "apiKey": "codex-secret", + "extraBody": {"service_tier": "${CODEX_SERVICE_TIER}"}, + }, + "xaiGrok": { + "apiKey": "xai-secret", + "proxy": "http://127.0.0.1:7890", + "extraBody": {"parallel_tool_calls": False}, + }, + } + } + ) + + save_config(config, config_path) + + saved = json.loads(config_path.read_text(encoding="utf-8")) + assert saved["providers"]["openaiCodex"] == { + "extraBody": {"service_tier": "${CODEX_SERVICE_TIER}"} + } + assert saved["providers"]["xaiGrok"] == { + "extraBody": {"parallel_tool_calls": False}, + "proxy": "http://127.0.0.1:7890", + } + assert "codex-secret" not in config_path.read_text(encoding="utf-8") + assert "xai-secret" not in config_path.read_text(encoding="utf-8") + def test_preserves_excluded_fields_when_no_env_refs(self, tmp_path): """Regression: fields with ``exclude=True`` (e.g. ProviderConfig.openai_codex) must survive ``resolve_config_env_vars`` when the config has no diff --git a/tests/providers/test_providers_init.py b/tests/providers/test_providers_init.py index 707a3f68..f3c3b51c 100644 --- a/tests/providers/test_providers_init.py +++ b/tests/providers/test_providers_init.py @@ -7,43 +7,61 @@ import sys def test_importing_providers_package_is_lazy(monkeypatch) -> None: + original_package = sys.modules["nanobot.providers"] monkeypatch.delitem(sys.modules, "nanobot.providers", raising=False) monkeypatch.delitem(sys.modules, "nanobot.providers.anthropic_provider", raising=False) monkeypatch.delitem(sys.modules, "nanobot.providers.openai_compat_provider", raising=False) monkeypatch.delitem(sys.modules, "nanobot.providers.openai_codex_provider", raising=False) + monkeypatch.delitem(sys.modules, "nanobot.providers.xai_oauth", raising=False) + monkeypatch.delitem(sys.modules, "nanobot.providers.xai_grok_provider", raising=False) monkeypatch.delitem(sys.modules, "nanobot.providers.github_copilot_provider", raising=False) monkeypatch.delitem(sys.modules, "nanobot.providers.azure_openai_provider", raising=False) monkeypatch.delitem(sys.modules, "nanobot.providers.bedrock_provider", raising=False) - providers = importlib.import_module("nanobot.providers") + try: + providers = importlib.import_module("nanobot.providers") - assert "nanobot.providers.anthropic_provider" not in sys.modules - assert "nanobot.providers.openai_compat_provider" not in sys.modules - assert "nanobot.providers.openai_codex_provider" not in sys.modules - assert "nanobot.providers.github_copilot_provider" not in sys.modules - assert "nanobot.providers.azure_openai_provider" not in sys.modules - assert "nanobot.providers.bedrock_provider" not in sys.modules - assert providers.__all__ == [ - "LLMProvider", - "LLMResponse", - "AnthropicProvider", - "OpenAICompatProvider", - "OpenAICodexProvider", - "GitHubCopilotProvider", - "AzureOpenAIProvider", - "BedrockProvider", - ] + assert "nanobot.providers.anthropic_provider" not in sys.modules + assert "nanobot.providers.openai_compat_provider" not in sys.modules + assert "nanobot.providers.openai_codex_provider" not in sys.modules + assert "nanobot.providers.xai_oauth" not in sys.modules + assert "nanobot.providers.xai_grok_provider" not in sys.modules + assert "nanobot.providers.github_copilot_provider" not in sys.modules + assert "nanobot.providers.azure_openai_provider" not in sys.modules + assert "nanobot.providers.bedrock_provider" not in sys.modules + assert providers.__all__ == [ + "LLMProvider", + "LLMResponse", + "AnthropicProvider", + "OpenAICompatProvider", + "OpenAICodexProvider", + "XAIGrokProvider", + "GitHubCopilotProvider", + "AzureOpenAIProvider", + "BedrockProvider", + ] + finally: + # Importing a replacement subpackage also replaces nanobot.providers on the + # parent package. Restore both views so this isolation test cannot pollute + # later tests that resolve a module through a dotted monkeypatch target. + monkeypatch.undo() + setattr(sys.modules["nanobot"], "providers", original_package) def test_explicit_provider_import_still_works(monkeypatch) -> None: + original_package = sys.modules["nanobot.providers"] monkeypatch.delitem(sys.modules, "nanobot.providers", raising=False) monkeypatch.delitem(sys.modules, "nanobot.providers.anthropic_provider", raising=False) - namespace: dict[str, object] = {} - exec("from nanobot.providers import AnthropicProvider", namespace) + try: + namespace: dict[str, object] = {} + exec("from nanobot.providers import AnthropicProvider", namespace) - assert namespace["AnthropicProvider"].__name__ == "AnthropicProvider" - assert "nanobot.providers.anthropic_provider" in sys.modules + assert namespace["AnthropicProvider"].__name__ == "AnthropicProvider" + assert "nanobot.providers.anthropic_provider" in sys.modules + finally: + monkeypatch.undo() + setattr(sys.modules["nanobot"], "providers", original_package) def test_openai_codex_supports_progress_deltas() -> None: diff --git a/tests/providers/test_xai_grok_provider.py b/tests/providers/test_xai_grok_provider.py new file mode 100644 index 00000000..46fc6255 --- /dev/null +++ b/tests/providers/test_xai_grok_provider.py @@ -0,0 +1,526 @@ +from __future__ import annotations + +import base64 +import json +import time +from types import SimpleNamespace +from typing import Any + +import httpx +import pytest + +from nanobot.config.schema import Config +from nanobot.providers.factory import make_provider +from nanobot.providers.registry import find_by_name +from nanobot.providers.xai_grok_provider import ( + DEFAULT_XAI_GROK_MODEL, + DEFAULT_XAI_GROK_MODELS_URL, + XAIGrokProvider, + _bounded_error_body, + _build_headers, + _build_model_headers, + _build_reasoning_options, + _build_xai_http_error, + _fetch_xai_model_capabilities, + _parse_xai_model_capabilities, + _request_xai, + _xai_error_response, + _XAIHTTPError, +) + + +def _token(access: str = "subscription-token") -> SimpleNamespace: + return SimpleNamespace( + access=access, + refresh="refresh-token", + expires=int(time.time() * 1000) + 3_600_000, + account_id="account", + ) + + +def _mock_token(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr( + "nanobot.providers.xai_grok_provider.get_xai_oauth_token", + lambda **_kwargs: _token(), + ) + + +def _mock_model_capabilities( + monkeypatch: pytest.MonkeyPatch, + *, + supports_backend_search: bool, +) -> None: + async def fake_fetch(*_args, **_kwargs): + return {"grok-4.5": supports_backend_search} + + monkeypatch.setattr( + "nanobot.providers.xai_grok_provider._fetch_xai_model_capabilities", + fake_fetch, + ) + + +def test_xai_grok_registry_exposes_curated_x_search_model() -> None: + spec = find_by_name("xai_grok") + + assert spec is not None + assert spec.is_oauth is True + assert spec.backend == "xai_grok" + assert spec.builtin_models[0].id == DEFAULT_XAI_GROK_MODEL + assert spec.builtin_models[0].context_window == 500000 + assert "when supported" in spec.builtin_models[0].description + + +def test_reasoning_options_omit_disabled_effort() -> None: + assert _build_reasoning_options("none") == {"summary": "concise"} + + +@pytest.mark.asyncio +async def test_provider_injects_hosted_x_search_and_required_proxy_headers(monkeypatch) -> None: + _mock_token(monkeypatch) + _mock_model_capabilities(monkeypatch, supports_backend_search=True) + calls: list[tuple[str, dict[str, str], dict[str, Any]]] = [] + + async def fake_request(url, headers, body, **_kwargs): + calls.append((url, headers, body)) + return "answer [[1]](https://x.com/example/status/1)", [], "stop", {}, None + + monkeypatch.setattr("nanobot.providers.xai_grok_provider._request_xai", fake_request) + provider = XAIGrokProvider() + tools = [ + { + "type": "function", + "function": { + "name": "read_file", + "description": "Read a file", + "parameters": {"type": "object"}, + }, + }, + { + "type": "function", + "function": { + "name": "x_search", + "description": "A colliding local tool", + "parameters": {"type": "object"}, + }, + }, + ] + + response = await provider.chat( + [{"role": "user", "content": "What is happening on X?"}], + tools=tools, + max_tokens=1234, + temperature=0.2, + reasoning_effort="high", + ) + + assert response.content == "answer [[1]](https://x.com/example/status/1)" + url, headers, body = calls[0] + assert url == "https://cli-chat-proxy.grok.com/v1/responses" + assert body["model"] == "grok-4.5" + assert body["tools"] == [ + { + "type": "function", + "name": "read_file", + "description": "Read a file", + "parameters": {"type": "object"}, + }, + {"type": "x_search"}, + ] + assert body["max_output_tokens"] == 1234 + assert body["temperature"] == 0.2 + assert body["stream_tool_calls"] is True + assert body["reasoning"] == {"summary": "concise", "effort": "high"} + assert body["store"] is False + assert headers["Authorization"] == "Bearer subscription-token" + assert headers["X-XAI-Token-Auth"] == "xai-grok-cli" + assert headers["x-authenticateresponse"] == "authenticate-response" + assert headers["x-grok-client-identifier"] == "nanobot" + assert headers["x-grok-client-mode"] == "headless" + assert headers["x-grok-model-override"] == "grok-4.5" + + +@pytest.mark.asyncio +async def test_provider_keeps_local_x_search_when_model_does_not_support_hosted_search( + monkeypatch, +) -> None: + _mock_token(monkeypatch) + _mock_model_capabilities(monkeypatch, supports_backend_search=False) + bodies: list[dict[str, Any]] = [] + + async def fake_request(_url, _headers, body, **_kwargs): + bodies.append(body) + return "ok", [], "stop", {}, None + + monkeypatch.setattr("nanobot.providers.xai_grok_provider._request_xai", fake_request) + provider = XAIGrokProvider() + tools = [ + { + "type": "function", + "function": { + "name": "x_search", + "description": "A local search fallback", + "parameters": {"type": "object"}, + }, + } + ] + + response = await provider.chat([{"role": "user", "content": "search"}], tools=tools) + + assert response.content == "ok" + assert bodies[0]["tools"] == [ + { + "type": "function", + "name": "x_search", + "description": "A local search fallback", + "parameters": {"type": "object"}, + } + ] + + +@pytest.mark.asyncio +async def test_provider_fails_closed_and_caches_model_catalog_failure(monkeypatch) -> None: + _mock_token(monkeypatch) + fetch_calls = 0 + bodies: list[dict[str, Any]] = [] + + async def failing_fetch(*_args, **_kwargs): + nonlocal fetch_calls + fetch_calls += 1 + raise httpx.ConnectError("catalog unavailable") + + async def fake_request(_url, _headers, body, **_kwargs): + bodies.append(body) + return "ok", [], "stop", {}, None + + monkeypatch.setattr( + "nanobot.providers.xai_grok_provider._fetch_xai_model_capabilities", + failing_fetch, + ) + monkeypatch.setattr("nanobot.providers.xai_grok_provider._request_xai", fake_request) + provider = XAIGrokProvider() + + await provider.chat([{"role": "user", "content": "first"}]) + await provider.chat([{"role": "user", "content": "second"}]) + + assert fetch_calls == 1 + assert all({"type": "x_search"} not in body["tools"] for body in bodies) + + +@pytest.mark.asyncio +async def test_provider_refreshes_and_retries_exactly_once_after_401(monkeypatch) -> None: + _mock_model_capabilities(monkeypatch, supports_backend_search=False) + token_calls: list[tuple[str | None, bool]] = [] + + def fake_token(*, proxy=None, force_refresh=False): + token_calls.append((proxy, force_refresh)) + return _token("fresh-token" if force_refresh else "stale-token") + + monkeypatch.setattr( + "nanobot.providers.xai_grok_provider.get_xai_oauth_token", + fake_token, + ) + request_tokens: list[str] = [] + + async def fake_request(_url, headers, _body, **_kwargs): + request_tokens.append(headers["Authorization"]) + if len(request_tokens) == 1: + raise _XAIHTTPError("unauthorized", status_code=401, should_retry=False) + return "ok", [], "stop", {}, None + + monkeypatch.setattr("nanobot.providers.xai_grok_provider._request_xai", fake_request) + provider = XAIGrokProvider(proxy="http://127.0.0.1:7890") + + response = await provider.chat([{"role": "user", "content": "hello"}]) + + assert response.content == "ok" + assert token_calls == [ + ("http://127.0.0.1:7890", False), + ("http://127.0.0.1:7890", True), + ] + assert request_tokens == ["Bearer stale-token", "Bearer fresh-token"] + + +@pytest.mark.asyncio +async def test_second_401_is_non_retryable_and_prompts_reauthentication(monkeypatch) -> None: + _mock_token(monkeypatch) + _mock_model_capabilities(monkeypatch, supports_backend_search=False) + + async def always_unauthorized(*_args, **_kwargs): + raise _XAIHTTPError( + "xAI rejected the login. Sign in again with `nanobot provider login xai-grok`.", + status_code=401, + should_retry=False, + ) + + monkeypatch.setattr( + "nanobot.providers.xai_grok_provider._request_xai", + always_unauthorized, + ) + provider = XAIGrokProvider() + + response = await provider.chat([{"role": "user", "content": "hello"}]) + + assert response.finish_reason == "error" + assert response.error_status_code == 401 + assert response.error_kind == "http" + assert response.error_should_retry is False + assert "nanobot provider login xai-grok" in (response.content or "") + + +@pytest.mark.asyncio +async def test_factory_builds_xai_provider_and_applies_explicit_body_overrides(monkeypatch) -> None: + _mock_token(monkeypatch) + _mock_model_capabilities(monkeypatch, supports_backend_search=True) + bodies: list[dict[str, Any]] = [] + + async def fake_request(_url, _headers, body, **_kwargs): + bodies.append(body) + return "ok", [], "stop", {}, None + + monkeypatch.setattr("nanobot.providers.xai_grok_provider._request_xai", fake_request) + config = Config.model_validate( + { + "agents": { + "defaults": { + "model": "xai-grok/grok-4.5", + "provider": "xai_grok", + } + }, + "providers": { + "xaiGrok": { + "proxy": "http://127.0.0.1:7890", + "extraBody": {"parallel_tool_calls": False}, + } + }, + } + ) + + provider = make_provider(config) + response = await provider.chat([{"role": "user", "content": "hello"}]) + + assert isinstance(provider, XAIGrokProvider) + assert provider.proxy == "http://127.0.0.1:7890" + assert response.content == "ok" + assert bodies[0]["parallel_tool_calls"] is False + assert {"type": "x_search"} in bodies[0]["tools"] + + +@pytest.mark.asyncio +async def test_raw_response_request_streams_text_usage_and_inline_citations(monkeypatch) -> None: + original_client = httpx.AsyncClient + captured: dict[str, Any] = {} + events = [ + {"type": "response.output_text.delta", "delta": "Live result "}, + { + "type": "response.output_text.delta", + "delta": "[[1]](https://x.com/example/status/1)", + }, + { + "type": "response.completed", + "response": { + "status": "completed", + "usage": {"input_tokens": 8, "output_tokens": 4, "total_tokens": 12}, + }, + }, + ] + content = "".join(f"data: {json.dumps(event)}\n\n" for event in events) + + def handler(request: httpx.Request) -> httpx.Response: + captured["request"] = request + captured["json"] = json.loads(request.content) + return httpx.Response(200, content=content, request=request) + + def fake_client(**kwargs) -> httpx.AsyncClient: + captured["kwargs"] = kwargs + return original_client( + transport=httpx.MockTransport(handler), + timeout=kwargs["timeout"], + ) + + monkeypatch.setattr("nanobot.providers.xai_grok_provider.httpx.AsyncClient", fake_client) + deltas: list[str] = [] + + result = await _request_xai( + "https://cli-chat-proxy.grok.com/v1/responses", + _build_headers("secret", "grok-4.5"), + {"model": "grok-4.5", "tools": [{"type": "x_search"}]}, + on_content_delta=lambda delta: _append(deltas, delta), + ) + + assert result[0] == "Live result [[1]](https://x.com/example/status/1)" + assert result[2] == "stop" + assert result[3] == {"prompt_tokens": 8, "completion_tokens": 4, "total_tokens": 12} + assert deltas == ["Live result ", "[[1]](https://x.com/example/status/1)"] + assert captured["json"]["tools"] == [{"type": "x_search"}] + + +def test_model_capabilities_follow_upstream_aliases_and_default_to_disabled() -> None: + capabilities = _parse_xai_model_capabilities( + { + "data": [ + {"id": "grok-4.5", "supportsBackendSearch": False}, + { + "model": "grok-search", + "supports_backend_search": True, + }, + { + "modelId": "grok-meta", + "_meta": {"supportsBackendSearch": True}, + }, + {"id": "grok-unknown"}, + ] + } + ) + + assert capabilities == { + "grok-4.5": False, + "grok-search": True, + "grok-meta": True, + "grok-unknown": False, + } + + +@pytest.mark.asyncio +async def test_model_capability_request_uses_subscription_headers(monkeypatch) -> None: + original_client = httpx.AsyncClient + captured: dict[str, Any] = {} + + def handler(request: httpx.Request) -> httpx.Response: + captured["request"] = request + return httpx.Response( + 200, + json={"data": [{"id": "grok-search", "supportsBackendSearch": True}]}, + request=request, + ) + + def fake_client(**kwargs) -> httpx.AsyncClient: + captured["kwargs"] = kwargs + return original_client( + transport=httpx.MockTransport(handler), + timeout=kwargs["timeout"], + follow_redirects=kwargs["follow_redirects"], + ) + + monkeypatch.setattr("nanobot.providers.xai_grok_provider.httpx.AsyncClient", fake_client) + payload = base64.urlsafe_b64encode( + json.dumps({"sub": "user-42", "email": "user@example.com"}).encode() + ).decode().rstrip("=") + access_token = f"header.{payload}.signature" + headers = _build_model_headers(_token(access_token)) + + capabilities = await _fetch_xai_model_capabilities( + DEFAULT_XAI_GROK_MODELS_URL, + headers, + ) + + request = captured["request"] + assert isinstance(request, httpx.Request) + assert request.method == "GET" + assert str(request.url) == DEFAULT_XAI_GROK_MODELS_URL + assert request.headers["Authorization"] == f"Bearer {access_token}" + assert request.headers["X-XAI-Token-Auth"] == "xai-grok-cli" + assert request.headers["x-userid"] == "user-42" + assert request.headers["x-email"] == "user@example.com" + assert captured["kwargs"] == {"timeout": 10.0, "follow_redirects": False} + assert capabilities == {"grok-search": True} + + +@pytest.mark.asyncio +async def test_raw_response_error_preserves_bounded_redacted_body(monkeypatch) -> None: + original_client = httpx.AsyncClient + raw = json.dumps( + { + "code": "invalid-argument", + "message": "Hosted x_search is not supported by grok-4.5", + "access_token": "must-not-leak", + } + ) + + def handler(request: httpx.Request) -> httpx.Response: + return httpx.Response(400, content=raw, request=request) + + def fake_client(**kwargs) -> httpx.AsyncClient: + return original_client( + transport=httpx.MockTransport(handler), + timeout=kwargs["timeout"], + ) + + monkeypatch.setattr("nanobot.providers.xai_grok_provider.httpx.AsyncClient", fake_client) + + with pytest.raises(_XAIHTTPError) as caught: + await _request_xai( + "https://cli-chat-proxy.grok.com/v1/responses", + _build_headers("secret", "grok-4.5"), + {"model": "grok-4.5"}, + ) + + error = caught.value + assert error.status_code == 400 + assert error.error_code == "invalid-argument" + assert error.should_retry is False + assert error.response_body == ( + '{"code":"invalid-argument","message":"Hosted x_search is not supported by ' + 'grok-4.5","access_token":"[REDACTED]"}' + ) + assert f"Response body: {error.response_body}" in str(error) + assert "must-not-leak" not in str(error) + + provider_response = _xai_error_response(error) + assert provider_response.error_status_code == 400 + assert provider_response.error_code == "invalid-argument" + assert error.response_body in (provider_response.content or "") + + +def test_plain_error_body_is_single_line_and_bounded() -> None: + detail = _bounded_error_body("Bearer secret-token\n" + "x" * 1100) + + assert detail is not None + assert detail.startswith("Bearer [REDACTED] ") + assert detail.endswith("…") + assert len(detail) == 1001 + + +def test_client_version_rejection_explains_update_and_preserves_body() -> None: + raw = json.dumps( + { + "code": "upgrade-required", + "message": "Client version 0.2.109 is no longer supported", + } + ) + + error = _build_xai_http_error(426, httpx.Headers(), raw) + response = _xai_error_response(error) + + assert error.status_code == 426 + assert error.should_retry is False + assert error.response_body == ( + '{"code":"upgrade-required","message":"Client version 0.2.109 is no longer supported"}' + ) + assert "xAI requires a newer Grok client version. Update nanobot and try again." in str(error) + assert error.response_body in str(error) + assert response.error_status_code == 426 + assert error.response_body in (response.content or "") + + +def test_large_json_error_body_redacts_camel_case_credentials_before_bounding() -> None: + detail = _bounded_error_body( + json.dumps( + { + "accessToken": "access-must-not-leak", + "refresh-token": "refresh-must-not-leak", + "padding": "x" * 33_000, + } + ) + ) + + assert detail is not None + assert '"accessToken":"[REDACTED]"' in detail + assert '"refresh-token":"[REDACTED]"' in detail + assert "access-must-not-leak" not in detail + assert "refresh-must-not-leak" not in detail + assert detail.endswith("…") + assert len(detail) == 1001 + + +async def _append(target: list[str], value: str) -> None: + target.append(value) diff --git a/tests/providers/test_xai_oauth.py b/tests/providers/test_xai_oauth.py new file mode 100644 index 00000000..1da7d478 --- /dev/null +++ b/tests/providers/test_xai_oauth.py @@ -0,0 +1,340 @@ +from __future__ import annotations + +import json +import queue +import threading +import time +from urllib.parse import parse_qs, urlencode, urlsplit + +import httpx +import pytest + +import nanobot.providers.xai_oauth as xai_oauth +from nanobot.providers.xai_oauth import ( + XAI_CLIENT_ID, + XAI_OAUTH_SCOPES, + XAIOAuthError, + XAIToken, + _build_authorize_url, + _CallbackResult, + _Discovery, + _generate_pkce, + _make_callback_server, + _validate_xai_endpoint, + complete_xai_oauth_login, + get_xai_oauth_login_status, + get_xai_oauth_storage_path, + get_xai_oauth_token, + login_xai_oauth, + logout_xai_oauth, + start_xai_oauth_login, +) + + +def _use_temp_credentials(monkeypatch: pytest.MonkeyPatch, tmp_path) -> None: + monkeypatch.setattr(xai_oauth, "get_data_dir", lambda: tmp_path) + + +def test_authorize_url_uses_pkce_and_frozen_xai_scope_contract() -> None: + verifier, challenge = _generate_pkce() + + url = _build_authorize_url( + "https://auth.x.ai/oauth2/authorize", + redirect_uri="http://127.0.0.1:54321/callback", + challenge=challenge, + state="state-value", + nonce="nonce-value", + ) + + params = parse_qs(urlsplit(url).query) + assert len(verifier) >= 43 + assert params == { + "response_type": ["code"], + "client_id": [XAI_CLIENT_ID], + "redirect_uri": ["http://127.0.0.1:54321/callback"], + "scope": [" ".join(XAI_OAUTH_SCOPES)], + "code_challenge": [challenge], + "code_challenge_method": ["S256"], + "state": ["state-value"], + "nonce": ["nonce-value"], + "referrer": ["nanobot"], + } + + +@pytest.mark.parametrize( + "endpoint", + [ + "http://auth.x.ai/oauth2/token", + "https://evil.example/oauth2/token", + "https://auth.x.ai:444/oauth2/token", + "https://user@auth.x.ai/oauth2/token", + "https://auth.x.ai:invalid/oauth2/token", + ], +) +def test_discovery_rejects_unsafe_endpoints(endpoint: str) -> None: + with pytest.raises(XAIOAuthError, match="unsafe token endpoint"): + _validate_xai_endpoint(endpoint, "token") + + +def test_callback_server_accepts_only_matching_state_and_allows_accounts_origin() -> None: + results: queue.Queue[_CallbackResult] = queue.Queue(maxsize=1) + server = _make_callback_server("expected-state", results) + thread = threading.Thread(target=server.serve_forever, daemon=True) + thread.start() + try: + response = httpx.get( + f"http://127.0.0.1:{server.server_port}/callback", + params={"code": "one-time-code", "state": "expected-state"}, + headers={"Origin": "https://accounts.x.ai"}, + ) + result = results.get(timeout=1) + finally: + server.shutdown() + server.server_close() + thread.join(timeout=2) + + assert response.status_code == 200 + assert response.headers["access-control-allow-origin"] == "https://accounts.x.ai" + assert response.headers["access-control-allow-private-network"] == "true" + assert result == _CallbackResult(code="one-time-code", state="expected-state") + assert "one-time-code" not in response.text + + +def test_login_uses_random_loopback_callback_and_saves_separate_credentials( + monkeypatch: pytest.MonkeyPatch, + tmp_path, +) -> None: + _use_temp_credentials(monkeypatch, tmp_path) + discovery = _Discovery( + authorization_endpoint="https://auth.x.ai/oauth2/authorize", + token_endpoint="https://auth.x.ai/oauth2/token", + userinfo_endpoint="https://auth.x.ai/oauth2/userinfo", + ) + monkeypatch.setattr(xai_oauth, "_discover", lambda _proxy: discovery) + exchanged: dict[str, str] = {} + + def fake_exchange(endpoint: str, **kwargs): + exchanged.update(endpoint=endpoint, **kwargs) + return { + "access_token": "access-secret", + "refresh_token": "refresh-secret", + "expires_in": 3600, + } + + monkeypatch.setattr(xai_oauth, "_exchange_code", fake_exchange) + monkeypatch.setattr( + xai_oauth, + "_fetch_account", + lambda endpoint, access, proxy: "user@example.com", + ) + opened_urls: list[str] = [] + + def complete_in_browser(authorize_url: str) -> bool: + opened_urls.append(authorize_url) + params = parse_qs(urlsplit(authorize_url).query) + callback_url = params["redirect_uri"][0] + callback_query = urlencode({"code": "auth-code", "state": params["state"][0]}) + response = httpx.get(f"{callback_url}?{callback_query}") + assert response.status_code == 200 + return True + + token = login_xai_oauth( + print_fn=lambda _message: None, + browser_opener=complete_in_browser, + callback_timeout_s=1, + ) + + assert token.account_id == "user@example.com" + assert exchanged["code"] == "auth-code" + assert exchanged["redirect_uri"].startswith("http://127.0.0.1:") + assert exchanged["verifier"] + assert opened_urls + assert get_xai_oauth_storage_path() == tmp_path / "auth" / "xai.json" + saved = json.loads(get_xai_oauth_storage_path().read_text(encoding="utf-8")) + assert saved["access"] == "access-secret" + assert saved["refresh"] == "refresh-secret" + assert get_xai_oauth_login_status() == token + + +def test_pending_login_accepts_authorization_code_from_remote_browser( + monkeypatch: pytest.MonkeyPatch, + tmp_path, +) -> None: + _use_temp_credentials(monkeypatch, tmp_path) + discovery = _Discovery( + authorization_endpoint="https://auth.x.ai/oauth2/authorize", + token_endpoint="https://auth.x.ai/oauth2/token", + userinfo_endpoint=None, + ) + monkeypatch.setattr(xai_oauth, "_discover", lambda _proxy: discovery) + exchanged: dict[str, str] = {} + + def fake_exchange(endpoint: str, **kwargs): + exchanged.update(endpoint=endpoint, **kwargs) + return {"access_token": "remote-access", "expires_in": 3600} + + monkeypatch.setattr(xai_oauth, "_exchange_code", fake_exchange) + + flow = start_xai_oauth_login(timeout_s=5) + try: + params = parse_qs(urlsplit(flow.authorization_url).query) + callback_url = params["redirect_uri"][0] + + token = complete_xai_oauth_login(flow, "remote-code") + finally: + flow.cancel() + + assert token is not None + assert token.access == "remote-access" + assert exchanged["code"] == "remote-code" + assert exchanged["redirect_uri"] == callback_url + assert get_xai_oauth_login_status() == token + + +def test_expired_token_refreshes_once_and_persists_rotated_refresh_token( + monkeypatch: pytest.MonkeyPatch, + tmp_path, +) -> None: + _use_temp_credentials(monkeypatch, tmp_path) + expired = XAIToken( + access="old-access", + refresh="old-refresh", + expires=int(time.time() * 1000) - 1, + account_id="account", + ) + xai_oauth._write_token(expired) + calls: list[tuple[XAIToken, str | None]] = [] + + def fake_refresh(token: XAIToken, proxy: str | None) -> XAIToken: + calls.append((token, proxy)) + return XAIToken( + access="new-access", + refresh="new-refresh", + expires=int(time.time() * 1000) + 3_600_000, + account_id=token.account_id, + ) + + monkeypatch.setattr(xai_oauth, "_refresh_token", fake_refresh) + + refreshed = get_xai_oauth_token(proxy="http://127.0.0.1:7890") + + assert refreshed.access == "new-access" + assert refreshed.refresh == "new-refresh" + assert calls == [(expired, "http://127.0.0.1:7890")] + assert get_xai_oauth_login_status() == refreshed + + +def test_missing_credentials_returns_actionable_login_command( + monkeypatch: pytest.MonkeyPatch, + tmp_path, +) -> None: + _use_temp_credentials(monkeypatch, tmp_path) + + with pytest.raises(XAIOAuthError, match="nanobot provider login xai-grok"): + get_xai_oauth_token() + + +def test_logout_waits_for_inflight_refresh_and_removes_rotated_token( + monkeypatch: pytest.MonkeyPatch, + tmp_path, +) -> None: + _use_temp_credentials(monkeypatch, tmp_path) + expired = XAIToken( + access="old-access", + refresh="old-refresh", + expires=int(time.time() * 1000) - 1, + ) + xai_oauth._write_token(expired) + refresh_started = threading.Event() + finish_refresh = threading.Event() + logout_started = threading.Event() + logout_finished = threading.Event() + refresh_errors: list[Exception] = [] + logout_results: list[bool] = [] + + def fake_refresh(token: XAIToken, _proxy: str | None) -> XAIToken: + refresh_started.set() + assert finish_refresh.wait(timeout=2) + return XAIToken( + access="new-access", + refresh=token.refresh, + expires=int(time.time() * 1000) + 3_600_000, + ) + + def refresh() -> None: + try: + get_xai_oauth_token() + except Exception as exc: # pragma: no cover - asserted below + refresh_errors.append(exc) + + def logout() -> None: + logout_started.set() + logout_results.append(logout_xai_oauth()) + logout_finished.set() + + monkeypatch.setattr(xai_oauth, "_refresh_token", fake_refresh) + refresh_thread = threading.Thread(target=refresh) + refresh_thread.start() + assert refresh_started.wait(timeout=2) + + logout_thread = threading.Thread(target=logout) + logout_thread.start() + assert logout_started.wait(timeout=2) + assert not logout_finished.wait(timeout=0.05) + finish_refresh.set() + refresh_thread.join(timeout=2) + logout_thread.join(timeout=2) + + assert not refresh_thread.is_alive() + assert not logout_thread.is_alive() + assert refresh_errors == [] + assert logout_results == [True] + assert not get_xai_oauth_storage_path().exists() + + +def test_refresh_does_not_restore_credentials_removed_after_its_initial_read( + monkeypatch: pytest.MonkeyPatch, + tmp_path, +) -> None: + _use_temp_credentials(monkeypatch, tmp_path) + expired = XAIToken( + access="old-access", + refresh="old-refresh", + expires=int(time.time() * 1000) - 1, + ) + xai_oauth._write_token(expired) + real_load_token = xai_oauth._load_token + initial_read_finished = threading.Event() + continue_refresh = threading.Event() + refresh_errors: list[Exception] = [] + first_load = True + + def gated_load_token() -> XAIToken | None: + nonlocal first_load + token = real_load_token() + if first_load: + first_load = False + initial_read_finished.set() + assert continue_refresh.wait(timeout=2) + return token + + def refresh() -> None: + try: + get_xai_oauth_token() + except Exception as exc: + refresh_errors.append(exc) + + monkeypatch.setattr(xai_oauth, "_load_token", gated_load_token) + refresh_thread = threading.Thread(target=refresh) + refresh_thread.start() + assert initial_read_finished.wait(timeout=2) + + assert logout_xai_oauth() is True + continue_refresh.set() + refresh_thread.join(timeout=2) + + assert not refresh_thread.is_alive() + assert len(refresh_errors) == 1 + assert isinstance(refresh_errors[0], XAIOAuthError) + assert "not signed in" in str(refresh_errors[0]) + assert not get_xai_oauth_storage_path().exists() diff --git a/tests/webui/test_settings_api.py b/tests/webui/test_settings_api.py index 13cdc9ff..9c8ec2e9 100644 --- a/tests/webui/test_settings_api.py +++ b/tests/webui/test_settings_api.py @@ -16,8 +16,10 @@ from nanobot.webui.settings_api import ( _model_catalog_kind, _oauth_provider_status, _reasoning_effort_values_for, + complete_oauth_provider, create_model_configuration, login_oauth_provider, + logout_oauth_provider, provider_models_payload, settings_payload, settings_usage_payload, @@ -344,6 +346,63 @@ def test_update_provider_settings_updates_dynamic_custom_provider( assert dynamic_provider.api_key == "sk-test" +@pytest.mark.parametrize( + ("provider_name", "config_attr"), + [ + ("openai_codex", "openai_codex"), + ("xai_grok", "xai_grok"), + ], +) +def test_update_provider_settings_updates_and_clears_oauth_proxy( + provider_name: str, + config_attr: str, + tmp_path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + config_path = tmp_path / "config.json" + config = Config() + getattr(config.providers, config_attr).proxy = "http://127.0.0.1:7000" + save_config(config, config_path) + monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path) + monkeypatch.setattr( + "nanobot.webui.settings_api._oauth_provider_status", + lambda _spec: { + "configured": False, + "account": None, + "expires_at": None, + "login_supported": True, + }, + ) + + payload = update_provider_settings( + {"provider": [provider_name], "proxy": [" http://127.0.0.1:7890 "]} + ) + + providers = {row["name"]: row for row in payload["providers"]} + assert providers[provider_name]["proxy"] == "http://127.0.0.1:7890" + assert getattr(load_config(config_path).providers, config_attr).proxy == ( + "http://127.0.0.1:7890" + ) + + cleared = update_provider_settings({"provider": [provider_name], "proxy": [" "]}) + + providers = {row["name"]: row for row in cleared["providers"]} + assert providers[provider_name]["proxy"] is None + assert getattr(load_config(config_path).providers, config_attr).proxy is None + + +def test_update_provider_settings_keeps_oauth_credentials_read_only( + tmp_path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + config_path = tmp_path / "config.json" + save_config(Config(), config_path) + monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path) + + with pytest.raises(WebUISettingsError, match="only supports proxy settings"): + update_provider_settings({"provider": ["openai_codex"], "apiKey": ["not-allowed"]}) + + def test_update_agent_settings_accepts_context_window_options( tmp_path, monkeypatch: pytest.MonkeyPatch, @@ -396,7 +455,7 @@ def test_update_context_window_rejects_unknown_values( with pytest.raises( WebUISettingsError, - match="context_window_tokens must be 65536, 200000, 262144, or 1048576", + match="context_window_tokens must be 65536, 200000, 262144, 500000, or 1048576", ): update_agent_settings({"context_window_tokens": ["128000"]}) @@ -1020,6 +1079,30 @@ def test_openai_codex_oauth_status_rejects_unavailable_token( assert status["account"] is None +def test_xai_grok_status_accepts_refreshable_login( + monkeypatch: pytest.MonkeyPatch, +) -> None: + token = SimpleNamespace( + access="access-token", + refresh="refresh-token", + expires=1, + account_id="user@example.com", + ) + monkeypatch.setattr( + "nanobot.providers.xai_oauth.get_xai_oauth_login_status", + lambda: token, + ) + + status = _oauth_provider_status(find_by_name("xai_grok")) + + assert status == { + "configured": True, + "account": "user@example.com", + "expires_at": 1, + "login_supported": True, + } + + def test_openai_codex_oauth_login_passes_configured_proxy( tmp_path, monkeypatch: pytest.MonkeyPatch, @@ -1089,6 +1172,118 @@ def test_github_copilot_oauth_login_reports_missing_oauth_cli_kit( assert "oauth_cli_kit not installed. Run: pip install oauth-cli-kit" in str(exc.value) +def test_xai_grok_login_starts_fresh_browser_flow_with_proxy( + tmp_path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + proxy = "http://127.0.0.1:23458" + config_path = tmp_path / "config.json" + save_config(Config.model_validate({"providers": {"xaiGrok": {"proxy": proxy}}}), config_path) + monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path) + captured: dict[str, object] = {} + + class FakeFlow: + authorization_url = "https://auth.x.ai/oauth2/authorize?state=test" + remaining_seconds = 600 + expired = False + + def cancel(self) -> None: + captured["cancelled"] = True + + def fake_start(*, proxy=None, timeout_s=None): + captured.update(proxy=proxy, timeout_s=timeout_s) + return FakeFlow() + + monkeypatch.setattr("nanobot.providers.xai_oauth.start_xai_oauth_login", fake_start) + + payload = login_oauth_provider({"provider": ["xai-grok"]}) + + assert captured["proxy"] == proxy + assert captured["timeout_s"] == 600 + assert payload["status"] == "authorization_required" + assert payload["provider"] == "xai_grok" + assert payload["authorization_url"] == FakeFlow.authorization_url + assert payload["flow_id"] + + callbacks: list[str | None] = [] + + def fake_complete(_flow, callback): + callbacks.append(callback) + if callback is None: + return None + return SimpleNamespace(access="access-token") + + monkeypatch.setattr( + "nanobot.providers.xai_oauth.complete_xai_oauth_login", + fake_complete, + ) + monkeypatch.setattr( + "nanobot.webui.settings_api.settings_payload", + lambda: {"settings": "ready"}, + ) + + pending = complete_oauth_provider( + {"provider": ["xai-grok"], "flow_id": [payload["flow_id"]]}, + ) + completed = complete_oauth_provider( + {"provider": ["xai-grok"], "flow_id": [payload["flow_id"]]}, + "secret", + ) + + assert pending == { + "status": "pending", + "provider": "xai_grok", + "flow_id": payload["flow_id"], + } + assert completed == {"settings": "ready"} + assert callbacks == [None, "secret"] + + +def test_xai_grok_login_reports_upstream_failure_as_bad_gateway( + tmp_path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + config_path = tmp_path / "config.json" + save_config(Config(), config_path) + monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path) + failure = RuntimeError("Could not reach xAI sign-in: ConnectError.") + + def fake_start(**_kwargs): + raise failure + + monkeypatch.setattr("nanobot.providers.xai_oauth.start_xai_oauth_login", fake_start) + + with pytest.raises(WebUISettingsError) as exc: + login_oauth_provider({"provider": ["xai-grok"]}) + + assert exc.value.status == 502 + assert str(exc.value) == ( + "xAI OAuth login failed: Could not reach xAI sign-in: ConnectError." + ) + assert exc.value.__cause__ is failure + + +def test_xai_grok_logout_removes_token_through_shared_lock( + tmp_path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + config_path = tmp_path / "config.json" + save_config(Config(), config_path) + monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path) + token_path = tmp_path / "auth" / "xai.json" + token_path.parent.mkdir(parents=True) + token_path.write_text("{}", encoding="utf-8") + token_path.with_suffix(".lock").write_text("", encoding="utf-8") + monkeypatch.setattr( + "nanobot.providers.xai_oauth.get_xai_oauth_storage_path", + lambda: token_path, + ) + + logout_oauth_provider({"provider": ["xai-grok"]}) + + assert not token_path.exists() + + def test_provider_models_payload_fetches_openai_compatible_models( tmp_path, monkeypatch: pytest.MonkeyPatch, @@ -1144,6 +1339,22 @@ def test_provider_models_payload_returns_curated_openai_codex_models() -> None: ] +def test_provider_models_payload_returns_xai_grok_model() -> None: + payload = provider_models_payload({"provider": ["xai_grok"]}) + + assert payload["status"] == "available" + assert payload["catalog_kind"] == "builtin" + assert payload["models"] == [ + { + "id": "xai-grok/grok-4.5", + "label": "Grok 4.5", + "description": "Grok via xAI subscription; X Search is enabled when supported.", + "owned_by": "xAI Grok", + "context_window": 500000, + } + ] + + def test_provider_models_payload_fetches_dynamic_custom_provider_models( tmp_path, monkeypatch: pytest.MonkeyPatch, diff --git a/tests/webui/test_settings_routes.py b/tests/webui/test_settings_routes.py new file mode 100644 index 00000000..9a87abc3 --- /dev/null +++ b/tests/webui/test_settings_routes.py @@ -0,0 +1,72 @@ +from __future__ import annotations + +import json +from types import SimpleNamespace +from urllib.parse import parse_qs, urlsplit + +import pytest +from websockets.datastructures import Headers + +from nanobot.webui.http_utils import http_json_response +from nanobot.webui.settings_routes import WebUISettingsRouter + + +@pytest.mark.asyncio +async def test_xai_oauth_completion_reads_code_from_private_header(monkeypatch) -> None: + captured: dict[str, object] = {} + + def complete(query, authorization_code=None): + captured.update(query=query, authorization_code=authorization_code) + return { + "status": "pending", + "provider": "xai_grok", + "flow_id": "flow-123", + } + + monkeypatch.setattr("nanobot.webui.settings_routes.complete_oauth_provider", complete) + router = WebUISettingsRouter( + bus=SimpleNamespace(), + logger=SimpleNamespace(exception=lambda *_args: None), + check_api_token=lambda _request: True, + parse_query=lambda path: parse_qs(urlsplit(path).query), + json_response=http_json_response, + error_response=lambda status, message: http_json_response( + {"error": message}, + status=status, + ), + runtime_surface="browser", + runtime_capabilities={}, + ) + request = SimpleNamespace( + path=( + "/api/settings/provider/oauth-login/complete" + "?provider=xai_grok&flow_id=flow-123" + ), + headers=Headers( + [ + ( + "X-Nanobot-OAuth-Code", + "secret", + ) + ] + ), + ) + + response = await router.dispatch( + None, + request, + "/api/settings/provider/oauth-login/complete", + ) + + assert response is not None + assert response.status_code == 200 + assert json.loads(response.body) == { + "status": "pending", + "provider": "xai_grok", + "flow_id": "flow-123", + } + assert captured == { + "query": {"provider": ["xai_grok"], "flow_id": ["flow-123"]}, + "authorization_code": "secret", + } + assert "secret" not in request.path diff --git a/webui/src/components/settings/SettingsView.tsx b/webui/src/components/settings/SettingsView.tsx index 24810cea..2a4a0f7e 100644 --- a/webui/src/components/settings/SettingsView.tsx +++ b/webui/src/components/settings/SettingsView.tsx @@ -98,6 +98,7 @@ import { Textarea } from "@/components/ui/textarea"; import { isLoopbackHost } from "@/lib/network"; import { checkVersion, + completeProviderOAuth, createModelConfiguration, disableNanobotFeature, enableNanobotFeature, @@ -166,6 +167,10 @@ import type { NanobotFeaturesPayload, NetworkSafetySettingsUpdate, ProviderModelsPayload, + ProviderOAuthAuthorizationRequired, + ProviderOAuthCompletionResult, + ProviderOAuthLoginResult, + ProviderOAuthPending, SessionAutomationJob, SettingsPayload, SkillSummary, @@ -188,6 +193,18 @@ export type SettingsSectionKey = | "runtime" | "advanced"; +function isProviderOAuthAuthorizationRequired( + payload: ProviderOAuthLoginResult, +): payload is ProviderOAuthAuthorizationRequired { + return (payload as ProviderOAuthAuthorizationRequired).status === "authorization_required"; +} + +function isProviderOAuthPending( + payload: ProviderOAuthCompletionResult, +): payload is ProviderOAuthPending { + return (payload as ProviderOAuthPending).status === "pending"; +} + type AppsKindFilter = "ready" | "cli" | "mcp"; type AutomationFilter = "all" | "active" | "paused" | "failed" | "system"; type AutomationSort = "next" | "last" | "updated" | "name"; @@ -223,10 +240,16 @@ type RestartAwarePayload = { runtime_capabilities?: SettingsPayload["runtime_capabilities"]; }; type ProviderApiType = "auto" | "chat_completions" | "responses"; -type ProviderForm = { apiKey: string; apiBase: string; apiType: ProviderApiType }; +type ProviderForm = { + apiKey: string; + apiBase: string; + apiType: ProviderApiType; + proxy: string; +}; type CustomMcpTransport = "stdio" | "streamableHttp" | "sse"; -const CONTEXT_WINDOW_TOKEN_OPTIONS = [65_536, 200_000, 262_144, 1_048_576] as const; +const CONTEXT_WINDOW_TOKEN_OPTIONS = [65_536, 200_000, 262_144, 500_000, 1_048_576] as const; +const OAUTH_PROXY_PROVIDERS = new Set(["openai_codex", "xai_grok"]); const DEFERRED_MODEL_LIST_PROVIDERS = new Set([ "aihubmix", "atomic_chat", @@ -541,6 +564,8 @@ export function SettingsView({ const { t } = useTranslation(); const { token } = useClient(); const pageVisible = usePageVisibility(); + const remoteBrowserAccess = + typeof window !== "undefined" && !isLoopbackHost(window.location.hostname); const [settings, setSettings] = useState(() => initialSettings); const [cliApps, setCliApps] = useState(null); const [nanobotFeatures, setNanobotFeatures] = useState(null); @@ -565,6 +590,11 @@ export function SettingsView({ const [nanobotFeatureConfirm, setNanobotFeatureConfirm] = useState(null); const [mcpPresetAction, setMcpPresetAction] = useState(null); const [providerSaving, setProviderSaving] = useState(null); + const [xaiOAuthFlow, setXaiOAuthFlow] = + useState(null); + const xaiOAuthFlowRef = useRef(null); + const [xaiOAuthCode, setXaiOAuthCode] = useState(""); + const [xaiOAuthCompleting, setXaiOAuthCompleting] = useState(false); const [webSearchSaving, setWebSearchSaving] = useState(false); const [imageGenerationSaving, setImageGenerationSaving] = useState(false); const [transcriptionSaving, setTranscriptionSaving] = useState(false); @@ -658,6 +688,46 @@ export function SettingsView({ onSettingsChange?.(payload); }, [onSettingsChange]); + const closeXaiOAuthFlow = useCallback(() => { + xaiOAuthFlowRef.current = null; + setXaiOAuthFlow(null); + setXaiOAuthCode(""); + setXaiOAuthCompleting(false); + }, []); + + useEffect(() => { + if (!xaiOAuthFlow) return; + let cancelled = false; + let timer: number | null = null; + const poll = async () => { + try { + const payload = await completeProviderOAuth( + token, + xaiOAuthFlow.provider, + xaiOAuthFlow.flow_id, + ); + if (cancelled || xaiOAuthFlowRef.current?.flow_id !== xaiOAuthFlow.flow_id) return; + if (isProviderOAuthPending(payload)) { + timer = window.setTimeout(() => void poll(), 1000); + return; + } + applyPayload(payload); + setExpandedProvider(xaiOAuthFlow.provider); + setError(null); + closeXaiOAuthFlow(); + } catch (err) { + if (cancelled || xaiOAuthFlowRef.current?.flow_id !== xaiOAuthFlow.flow_id) return; + setError((err as Error).message); + closeXaiOAuthFlow(); + } + }; + timer = window.setTimeout(() => void poll(), 1000); + return () => { + cancelled = true; + if (timer !== null) window.clearTimeout(timer); + }; + }, [applyPayload, closeXaiOAuthFlow, token, xaiOAuthFlow]); + useEffect(() => { if (!initialSettings || settings !== null) return; applyPayload(initialSettings); @@ -882,6 +952,7 @@ export function SettingsView({ apiKey: next[provider.name]?.apiKey ?? "", apiBase: next[provider.name]?.apiBase ?? provider.api_base ?? provider.default_api_base ?? "", apiType: next[provider.name]?.apiType ?? provider.api_type ?? "auto", + proxy: next[provider.name]?.proxy ?? provider.proxy ?? "", }; } return next; @@ -1229,11 +1300,16 @@ export function SettingsView({ if (providerSaving) return; const provider = settings?.providers.find((item) => item.name === providerName); if (!provider) return; - if (provider.auth_type === "oauth") return; - const providerForm = providerForms[providerName] ?? { apiKey: "", apiBase: "", apiType: "auto" }; + const isOauthProvider = provider.auth_type === "oauth"; + const providerForm = providerForms[providerName] ?? { + apiKey: "", + apiBase: "", + apiType: "auto", + proxy: provider.proxy ?? "", + }; const apiKey = providerForm.apiKey.trim(); const apiKeyRequired = provider.api_key_required ?? true; - if (!provider.configured && apiKeyRequired && !apiKey) { + if (!isOauthProvider && !provider.configured && apiKeyRequired && !apiKey) { setError(t("settings.byok.apiKeyRequired")); return; } @@ -1245,12 +1321,20 @@ export function SettingsView({ ? "azure" : null; if (supportName && !(await installCapabilities([supportName]))) return; - const payload = await updateProviderSettings(token, { - provider: providerName, - apiKey: apiKey || undefined, - apiBase: providerForm.apiBase.trim(), - apiType: providerForm.apiType, - }); + const payload = await updateProviderSettings( + token, + isOauthProvider + ? { + provider: providerName, + proxy: providerForm.proxy.trim(), + } + : { + provider: providerName, + apiKey: apiKey || undefined, + apiBase: providerForm.apiBase.trim(), + apiType: providerForm.apiType, + }, + ); applyPayload(payload); if (payload.requires_restart) { setPendingRestartSections((prev) => ({ ...prev, image: true })); @@ -1262,6 +1346,7 @@ export function SettingsView({ apiKey: "", apiBase: providerForm.apiBase.trim(), apiType: providerForm.apiType, + proxy: providerForm.proxy.trim(), }, })); setVisibleProviderKeys((prev) => ({ ...prev, [providerName]: false })); @@ -1276,22 +1361,75 @@ export function SettingsView({ const runProviderOAuth = async (providerName: string, action: "login" | "logout") => { if (providerSaving) return; + let popup: Window | null = null; + if (action === "login" && providerName === "xai_grok" && !remoteBrowserAccess) { + try { + popup = window.open("about:blank", "_blank"); + if (popup) popup.opener = null; + } catch { + popup = null; + } + } setProviderSaving(providerName); try { const payload = action === "login" ? await loginProviderOAuth(token, providerName) : await logoutProviderOAuth(token, providerName); + if (isProviderOAuthAuthorizationRequired(payload)) { + try { + if (popup && !popup.closed) popup.location.href = payload.authorization_url; + } catch { + // The dialog keeps the authorization link available when the popup was closed. + } + xaiOAuthFlowRef.current = payload; + setXaiOAuthFlow(payload); + setXaiOAuthCode(""); + setExpandedProvider(providerName); + setError(null); + return; + } + popup?.close(); + closeXaiOAuthFlow(); applyPayload(payload); setExpandedProvider(providerName); setError(null); } catch (err) { + popup?.close(); setError((err as Error).message); } finally { setProviderSaving(null); } }; + const completeXaiOAuth = async () => { + const flow = xaiOAuthFlowRef.current; + const authorizationCode = xaiOAuthCode.trim(); + if (!flow || !authorizationCode || xaiOAuthCompleting) return; + setXaiOAuthCompleting(true); + try { + const payload = await completeProviderOAuth( + token, + flow.provider, + flow.flow_id, + authorizationCode, + ); + if (xaiOAuthFlowRef.current?.flow_id !== flow.flow_id) return; + if (isProviderOAuthPending(payload)) return; + applyPayload(payload); + setExpandedProvider(flow.provider); + setError(null); + closeXaiOAuthFlow(); + } catch (err) { + if (xaiOAuthFlowRef.current?.flow_id === flow.flow_id) { + setError((err as Error).message); + closeXaiOAuthFlow(); + } + } finally { + setXaiOAuthCompleting(false); + } + }; + const saveWebSearch = async () => { if (!settings || webSearchSaving) return; const provider = settings.web_search.providers.find((item) => item.name === webSearchForm.provider); @@ -1364,6 +1502,7 @@ export function SettingsView({ apiKey: "", apiBase: provider.api_base ?? provider.default_api_base ?? "", apiType: provider.api_type ?? "auto", + proxy: provider.proxy ?? "", }, })); setVisibleProviderKeys((prev) => ({ ...prev, [providerName]: false })); @@ -1418,6 +1557,7 @@ export function SettingsView({ apiKey: "", apiBase: forms[providerName]?.apiBase ?? "", apiType: forms[providerName]?.apiType ?? "auto", + proxy: forms[providerName]?.proxy ?? "", }, })); setVisibleProviderKeys((visible) => ({ ...visible, [providerName]: false })); @@ -1671,6 +1811,7 @@ export function SettingsView({ providerSaving={providerSaving} query={providerQuery} showBrandLogos={localPrefs.brandLogos} + remoteBrowserAccess={remoteBrowserAccess} onQueryChange={setProviderQuery} onToggleProvider={handleToggleProvider} onToggleProviderKey={toggleProviderKeyVisibility} @@ -1682,6 +1823,7 @@ export function SettingsView({ apiKey: prev[provider]?.apiKey ?? "", apiBase: prev[provider]?.apiBase ?? "", apiType: prev[provider]?.apiType ?? "auto", + proxy: prev[provider]?.proxy ?? "", ...value, }, })) @@ -1917,6 +2059,21 @@ export function SettingsView({ onSave={handleCreateModelConfiguration} /> + { + if (!xaiOAuthFlow) return; + const opened = window.open(xaiOAuthFlow.authorization_url, "_blank", "noopener,noreferrer"); + if (opened) opened.opener = null; + }} + onComplete={() => void completeXaiOAuth()} + onClose={closeXaiOAuthFlow} + /> + void; + onOpenAuthorization: () => void; + onComplete: () => void; + onClose: () => void; +}) { + const { t } = useTranslation(); + + return ( + { + if (!open) onClose(); + }} + > + +
{ + event.preventDefault(); + onComplete(); + }} + > + + xAI Grok + + {remoteBrowserAccess + ? t("settings.oauth.remoteCodeHelp") + : t("settings.oauth.localCodeHelp")} + + +
+ + onAuthorizationCodeChange(event.target.value)} + placeholder={t("settings.oauth.authorizationCode")} + aria-label={t("settings.oauth.authorizationCode")} + autoComplete="off" + spellCheck={false} + /> +
+ + + + +
+
+
+ ); +} + function NewModelConfigurationDialog({ open, draft, @@ -2827,11 +3060,13 @@ function ModelsSettings({ label: tokens === 1_048_576 ? "1M" - : tokens === 262_144 - ? "256K" - : tokens === 200_000 - ? "200K" - : "64K", + : tokens === 500_000 + ? "500K" + : tokens === 262_144 + ? "256K" + : tokens === 200_000 + ? "200K" + : "64K", }))} onChange={(value) => setForm((prev) => ({ @@ -2871,6 +3106,7 @@ function ProvidersSettings({ providerSaving, query, showBrandLogos, + remoteBrowserAccess, onQueryChange, onToggleProvider, onToggleProviderKey, @@ -2895,6 +3131,7 @@ function ProvidersSettings({ providerSaving: string | null; query: string; showBrandLogos: boolean; + remoteBrowserAccess: boolean; onQueryChange: (query: string) => void; onToggleProvider: (provider: string) => void; onToggleProviderKey: (provider: string) => void; @@ -2923,14 +3160,20 @@ function ProvidersSettings({ apiKey: "", apiBase: provider.api_base ?? provider.default_api_base ?? "", apiType: provider.api_type ?? "auto", + proxy: provider.proxy ?? "", }; const saving = providerSaving === provider.name; const isOauthProvider = provider.auth_type === "oauth"; + const supportsOauthProxy = isOauthProvider && OAUTH_PROXY_PROVIDERS.has(provider.name); const keyVisible = !!visibleProviderKeys[provider.name]; const editingKey = !provider.configured || !!editingProviderKeys[provider.name]; const apiKeyRequired = provider.api_key_required ?? true; const apiKey = form.apiKey.trim(); const apiBase = form.apiBase.trim(); + const proxy = form.proxy.trim(); + const oauthProxyDirty = supportsOauthProxy && proxy !== (provider.proxy ?? "").trim(); + const oauthProxySaving = saving && oauthProxyDirty; + const oauthActionBusy = saving && !oauthProxySaving; const missingRequiredApiKey = !isOauthProvider && apiKeyRequired && !provider.configured && !apiKey; const missingOptionalCredential = !isOauthProvider && !apiKeyRequired && !provider.configured && !apiKey && !apiBase; @@ -2990,48 +3233,120 @@ function ProvidersSettings({

{capabilityError}

) : null} {isOauthProvider ? ( -
-
-

- {tx("settings.oauth.authentication", "OAuth authentication")} -

-

- {provider.configured - ? t("settings.oauth.signedInAs", { - account: provider.oauth_account || provider.label, - defaultValue: "Signed in as {{account}}", - }) - : tx("settings.oauth.signInHelp", "Sign in from this device; no API key is stored in config.")} -

-
-
- {provider.configured ? ( + <> +
+
+

+ {tx("settings.oauth.authentication", "OAuth authentication")} +

+

+ {provider.configured + ? t("settings.oauth.signedInAs", { + account: provider.oauth_account || provider.label, + defaultValue: "Signed in as {{account}}", + }) + : provider.name === "xai_grok" && remoteBrowserAccess + ? tx( + "settings.oauth.remoteSignInHelp", + "Select Sign in to open xAI on your computer, then paste the authorization code shown after login.", + ) + : tx("settings.oauth.signInHelp", "Sign in from this device; no API key is stored in config.")} +

+
+
+ {provider.configured ? ( + + ) : null} - ) : null} - +
-
+ {supportsOauthProxy ? ( +
+
+ + + + +
+
+ + onChangeProviderForm(provider.name, { proxy: event.target.value }) + } + placeholder="http://127.0.0.1:7890" + autoCapitalize="none" + autoComplete="off" + autoCorrect="off" + spellCheck={false} + className="h-9 min-w-0 flex-1 rounded-full font-mono text-[12px]" + /> +
+ + +
+
+
+ ) : null} + ) : ( <>