Grayscale/nanobot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(ollama): explain tool prompt cache reuse

Browse Source
This commit is contained in:
chengyongru
2026-07-20 17:47:04 +08:00
committed by chengyongru
parent 8423cf3eeb
commit 9d830fb6b6
4 changed files with 254 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/provider-cookbook.md
+7 -1
View File
@@ -431,7 +431,13 @@ curl -sS http://localhost:11434/v1/models
nanobot agent -m "Hello!"
```
If you see `connection refused`, Ollama is not running or `apiBase` points to the wrong port. If the response is very slow, try a smaller local model or lower `contextWindowTokens`.
If you see `connection refused`, Ollama is not running or `apiBase` points to the wrong port. If every response is slow, try a smaller local model or lower `contextWindowTokens`.
If direct Ollama responses are fast but tool-using nanobot turns repeatedly evaluate
thousands of prompt tokens, the model's chat template may be moving its tool
definitions between requests. See
[Improve Ollama Tool-Calling Prompt Cache Reuse](./guides/configure-ollama-prompt-cache.md)
for a diagnostic procedure and an optional model-specific workaround.
## Recipe: vLLM or LM Studio