Commit Graph
26 Commits
Author SHA1 Message Date
chengyongruandchengyongru 39348dfafe refactor(agent): remove dead lifecycle scaffolding 2026-07-27 12:00:06 +08:00
Xubin Ren 1cfa48ad4a fix(web-search): return structured Serper errors 2026-07-06 12:11:31 +08:00
franciscomaestreandXubin Ren 8a42a9c73a feat(web-search): add Serper.dev (Google Search API) provider
Add 'serper' as a web search backend, following the existing provider
pattern (keenable/exa): POST to https://google.serper.dev/search with the
X-API-KEY header, map the 'organic' results into the shared result format,
and fall back to DuckDuckGo when no key is configured.

- key resolved from config.api_key or SERPER_API_KEY env var
- 429 handled with a rate-limit message; other HTTP errors surfaced
- tests cover success, env-key, no-key fallback, HTTP error and rate limit
- docs: add Serper config example and list it in tools.web.search providers
2026-07-06 12:11:31 +08:00
hyoukadevandXubin Ren 9c6eaf0bed test: deduplicate proxy value and construct tool via constructor
- Use a local variable for the proxy URL instead of hardcoding it twice
- Pass proxy through the WebSearchTool constructor instead of mutating
  after instantiation (matches real usage path)
- Add assertion that timeout is still forwarded correctly
- Use generic mock data instead of test-specific strings
2026-06-24 15:45:44 +08:00
hyoukadevandXubin Ren c66a0217d2 fix(web): pass proxy to DDGS client
The DuckDuckGo search provider instantiated DDGS(timeout=10) without
passing the configured proxy, making web_search unusable in environments
that require a proxy (e.g. behind GFW). DDGS supports a proxy parameter
and the proxy value is already available as self.proxy — it was simply
not forwarded.

Add a test verifying the proxy kwarg is forwarded to DDGS.
2026-06-24 15:45:44 +08:00
74daa81a1b feat(web): allow Keenable search without an API key
Keenable's public endpoint serves the free tier (1000 req/hour) without
auth. Route to /v1/search/public with the X-Keenable-Title header when no
key is configured, instead of falling back to DuckDuckGo; keep the
authenticated /v1/search path when an apiKey or KEENABLE_API_KEY is set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 15:22:13 +08:00
d7280da17c fix(web): require API key for Keenable, fall back to DuckDuckGo
Manual testing against the live API showed the Keenable REST endpoint
(/v1/search) returns 401 without a key — the keyless "free tier" applies
only to the CLI, not the HTTP API. Treat Keenable like every other
key-based provider: fall back to DuckDuckGo when no key is configured,
and drop the now-inaccurate free-tier wording from the docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 00:08:52 +08:00
092b07c7aa feat(web): use shared user-agent and send X-Keenable-Title
Drop the bespoke nanobot/<version> User-Agent in favor of self.user_agent
for consistency with every other search provider, and add an
X-Keenable-Title: nanobot header so Keenable can attribute traffic.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 00:08:52 +08:00
83cb296cdc style(test): match existing single-line result-dict style
Keep the Keenable search test consistent with the surrounding mocks
(test_tavily/test_brave) rather than introducing a multi-line outlier.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 00:08:52 +08:00
fa3e902ee8 feat(web): register Keenable provider in WebUI, docs, and tests
Bring the Keenable search provider in line with the established
multi-file provider pattern so it surfaces everywhere the others do:

- settings_api.py: register in the web-search provider options so it
  appears in the WebUI settings dropdown (credential: api_key, optional).
- WebUI provider-brand: add keenable brand entry + brand test.
- docs/configuration.md: provider table row + config example.
- Harden _search_keenable: honor config.timeout and return explicit
  messages on HTTP status errors (429 / other), matching peer providers.
- Add env-key and HTTP-error tests; add the websocket settings whitelist
  assertion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 00:08:52 +08:00
4f6e5e9cb8 feat(web): send honest nanobot UA for Keenable
Keenable is a first-party API, so identify as nanobot/<version> instead
of the shared spoofed-browser User-Agent used by scraping providers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 00:08:52 +08:00
6fcf65a8e3 chore(web): drop redundant comments from Keenable provider
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 00:08:52 +08:00
a630a78941 feat(web): add Keenable search provider
Add Keenable (https://keenable.ai) as a web_search backend, modeled on
the existing httpx-based providers. Unlike key-gated providers, Keenable
has a no-login free tier, so it resolves to itself even without an API
key instead of falling back to DuckDuckGo; the X-API-Key header is only
sent when a key is configured (config api_key or KEENABLE_API_KEY env).

Maps result snippet (falling back to description) into the shared
content field. Covered by tests for keyed/anonymous search and the
no-fallback concurrency behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 00:08:52 +08:00
MoranandXubin Ren 9c492143b4 search: add Bocha web search provider 2026-06-10 15:51:15 +08:00
erikmackinnonandXubin Ren 31bfec58d0 Add Exa web search provider 2026-06-10 15:02:07 +08:00
Xubin Ren edf34d857a search: add Volcengine web search provider 2026-06-02 13:55:12 +08:00
A.G. BocsardiandXubin Ren 172ec4d4c4 fix(web): update Kagi search API integration
Use Kagi's documented v1 Search API shape from the OpenAPI spec: POST /search, Bearer auth, JSON query payload, and data.search results.
2026-05-26 12:27:01 +08:00
hanyuanlingandXubin Ren b2ac609bb5 fix(web): back off Brave search rate limits 2026-05-16 00:12:50 +08:00
28f9bbff31 feat(web_search): add olostep provider
Adds Olostep (https://www.olostep.com) as an optional web_search backend
using the official olostep Python SDK (client.answers.create()).

Changes:
- pyproject.toml: adds olostep>=0.1.0 optional dependency
- schema.py: adds olostep to provider comment in WebSearchConfig
- web.py: adds _search_olostep() with lazy import and provider branching
- docs/configuration.md: documents Olostep setup under web search config
- tests: unit tests for the new provider

Backward compatible: existing users see no behavior change unless they
opt into provider: "olostep". No hard dependency at runtime path.

Co-authored-by: umerkay <umerkk164@gmail.com>
2026-04-28 19:09:38 +08:00
Xubin Ren f4d8783f5e test(web): cover configurable fetch behavior
Ensure custom user agents are applied to direct web requests and disabling Jina Reader forces the local readability path.

Made-with: Cursor
2026-04-28 07:25:47 +00:00
yeyitechandXubin Ren ee061f0595 fix(web): serialize duckduckgo search calls 2026-04-14 14:10:06 +08:00
Xubin RenandXubin Ren b959ae6d89 test(web): cover Kagi search provider
Add focused coverage for the Kagi web search provider, including the request format and the DuckDuckGo fallback when no API key is configured.
2026-04-11 16:53:05 +08:00
Xubin RenandXubin Ren 4b4d8b506d test: add regression test for DuckDuckGo asyncio.wait_for timeout guard
Made-with: Cursor
2026-04-06 02:21:51 +08:00
Xubin RenandXubin Ren 90caf5ce51 test: remove duplicate test_jina_422_falls_back_to_duckduckgo
The same test function name appeared twice; Python silently shadows the
first definition so it never ran.  Keep the version that also asserts
the request URL contains "s.jina.ai".

Made-with: Cursor
2026-04-06 02:06:00 +08:00
KimGLeeandXubin Ren f422de8084 fix(web-search): fix Jina search format and fallback 2026-04-06 02:06:00 +08:00
chengyongruandXubin Ren 72acba5d27 refactor(tests): optimize unit test structure 2026-03-24 15:12:22 +08:00