docs: establish helper-first retrieval as default entry point
Make overview() + memories() + search() the standard first pass for broad retrieval tasks, with sql() positioned as an escalation path for exact joins/aggregations. Add a Default First Pass section to SKILL.md, a copyable first-pass pattern to query-patterns.md, and update retrieval-semantics.md to reinforce the helper-before-sql principle.
This commit is contained in:
@@ -47,14 +47,39 @@ The query file runs inside `(async () => { ... })()`. Use `return` to emit JSON.
|
|||||||
Query scripts are read-only: `remember()` is not available, and `sql()` only
|
Query scripts are read-only: `remember()` is not available, and `sql()` only
|
||||||
accepts read-only SELECT/WITH queries.
|
accepts read-only SELECT/WITH queries.
|
||||||
|
|
||||||
|
## Default First Pass
|
||||||
|
|
||||||
|
Start with helpers, not raw SQL. For the first Obelisk query in a task, normally
|
||||||
|
call `overview({ limit: 6 })` unless the user already gave an exact
|
||||||
|
`session_id`, message `uuid`, or absolute file path.
|
||||||
|
|
||||||
|
For semantic or synthesis tasks, combine orientation, memory recall, and raw
|
||||||
|
session evidence before deciding whether a detail pass is needed:
|
||||||
|
|
||||||
|
```js
|
||||||
|
const map = overview({ limit: 6 });
|
||||||
|
const project = map.current.project?.project;
|
||||||
|
const topic = 'topic terms from the user request';
|
||||||
|
|
||||||
|
return {
|
||||||
|
orientation: map.current_project,
|
||||||
|
prior_memories: memories({ project, query: topic, limit: 5 }),
|
||||||
|
session_evidence: search(topic.replace(/[-_]/g, ' '), { project, limit: 8 }),
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `sql()` only as an escalation path for exact joins, aggregations, or schema
|
||||||
|
questions that helpers cannot express cleanly. Do not use raw SQL as a generic
|
||||||
|
fallback for broad retrieval.
|
||||||
|
|
||||||
## Query Routing
|
## Query Routing
|
||||||
|
|
||||||
Before writing a query, classify the task. Progressive disclosure is useful, but
|
Before writing a query, classify the task. Progressive disclosure is useful, but
|
||||||
skipping the relevant reference usually costs extra query rounds.
|
skipping the relevant reference usually costs extra query rounds.
|
||||||
|
|
||||||
- Read `references/schema.md` before raw `sql()` unless the needed table/column relationship is already explicit here. Do this before running the SQL, not after a missing-column error.
|
- Read `references/query-patterns.md` before the first query for broad synthesis, progress summaries, design history, weekly/monthly reviews, or questions that ask what the user did, learned, decided, tried, or abandoned. Start from the first-pass or one-shot synthesis pattern, then run a faceted detail pass if needed.
|
||||||
- Read `references/retrieval-semantics.md` before multi-step retrieval, scoped project/file/session searches, or synthesis/conclusion/history questions. It defines the query design frame.
|
- Read `references/retrieval-semantics.md` before multi-step retrieval, scoped project/file/session searches, or synthesis/conclusion/history questions. It defines the query design frame.
|
||||||
- Read `references/query-patterns.md` when you need copyable query scripts: one-shot synthesis, learned detail passes, workflow trees, failure groups, file history, summaries, subagents, raw windows, or empty results.
|
- Read `references/schema.md` before raw `sql()` unless the needed table/column relationship is already explicit here. Do this before running the SQL, not after a missing-column error. Do not start with raw SQL for broad synthesis unless helpers cannot express the needed aggregation or join.
|
||||||
- Read `references/pitfalls.md` after an error or when helper fields, FTS syntax, aliases, or row shapes are unclear.
|
- Read `references/pitfalls.md` after an error or when helper fields, FTS syntax, aliases, or row shapes are unclear.
|
||||||
|
|
||||||
If a helper row shape is unclear, first run a tiny scoped query and return
|
If a helper row shape is unclear, first run a tiny scoped query and return
|
||||||
@@ -100,7 +125,9 @@ expand vertically from one evidence point without dumping the whole session.
|
|||||||
|
|
||||||
### `sql(query, ...params)`
|
### `sql(query, ...params)`
|
||||||
|
|
||||||
Read-only SQL SELECT/WITH with `?` placeholders. Returns array rows.
|
Read-only SQL SELECT/WITH with `?` placeholders. Returns array rows. SQL is an
|
||||||
|
escape hatch for exact structured joins and aggregations after the helper-first
|
||||||
|
surface is insufficient; it is not the default retrieval entry point.
|
||||||
|
|
||||||
Before writing non-trivial SQL, read `references/schema.md`. Common safe joins:
|
Before writing non-trivial SQL, read `references/schema.md`. Common safe joins:
|
||||||
|
|
||||||
@@ -115,8 +142,8 @@ Tables: `sessions`, `messages`, `tool_calls`, `tool_results`, `summaries`,
|
|||||||
## Structured Helpers
|
## Structured Helpers
|
||||||
|
|
||||||
These helpers are convenience accessors over the same SQLite structure. They do
|
These helpers are convenience accessors over the same SQLite structure. They do
|
||||||
not replace `sql()`; use `sql()` when you need an exact aggregation or a join
|
not replace `sql()`, but they are the default first-pass surface. Use `sql()`
|
||||||
the helper does not expose.
|
when you need an exact aggregation or a join the helper does not expose.
|
||||||
|
|
||||||
All list helpers accept a bounded `limit`. Many also accept:
|
All list helpers accept a bounded `limit`. Many also accept:
|
||||||
`{ project, after, before, sessionId, sessions, branch }`. Check the schema or a
|
`{ project, after, before, sessionId, sessions, branch }`. Check the schema or a
|
||||||
@@ -141,7 +168,8 @@ tiny sample before relying on less common filters.
|
|||||||
Keep queries scoped, bounded, and structural.
|
Keep queries scoped, bounded, and structural.
|
||||||
|
|
||||||
- Scope First: classify the locator as scope, artifact, or semantic. Use the narrowest structural locator before FTS; empty scoped results are valid unless the user asks to broaden.
|
- Scope First: classify the locator as scope, artifact, or semantic. Use the narrowest structural locator before FTS; empty scoped results are valid unless the user asks to broaden.
|
||||||
- Orient When Needed: use `overview()` when the current project or available scopes are unclear. It is a navigation map; confirm facts with `memories()`, `search()`, helpers, or `sql()`.
|
- Orient First: for a new task, normally call `overview({ limit: 6 })` before deeper retrieval unless the user gave an exact session/message/file locator. It is a navigation map; confirm facts with `memories()`, `search()`, helpers, or, only when needed, `sql()`.
|
||||||
|
- Helper First: prefer `overview()`, `memories()`, `search()`, `sessions()`, `summaries()`, `fileHistory()`, and other helpers for first-pass retrieval. Escalate to raw `sql()` only when helpers cannot express the needed join, grouping, or exact schema-level check.
|
||||||
- Plan Before Probe: for conclusion, broad history, failure investigation, or file evolution, write a bounded retrieval script instead of spending turns on intermediate results.
|
- Plan Before Probe: for conclusion, broad history, failure investigation, or file evolution, write a bounded retrieval script instead of spending turns on intermediate results.
|
||||||
- Structure Before Text: compute counts, joins, grouping, dedupe, and projection in SQL or JS; keep runtime JSON compact, ideally under 10k-12k chars for synthesis tasks.
|
- Structure Before Text: compute counts, joins, grouping, dedupe, and projection in SQL or JS; keep runtime JSON compact, ideally under 10k-12k chars for synthesis tasks.
|
||||||
- Evidence Before Conclusion: return compact evidence with stable IDs (`session_id`, `uuid`, `tool_call_id`, `run_id`, `agent_id`) and short snippets, then synthesize in the final answer.
|
- Evidence Before Conclusion: return compact evidence with stable IDs (`session_id`, `uuid`, `tool_call_id`, `run_id`, `agent_id`) and short snippets, then synthesize in the final answer.
|
||||||
|
|||||||
@@ -4,6 +4,67 @@ These are copyable CodeAct patterns for `runtime.mjs --query` scripts, plus one
|
|||||||
`--remember` registration pattern. They are not new APIs. Adapt them to the
|
`--remember` registration pattern. They are not new APIs. Adapt them to the
|
||||||
user's scope and return compact evidence.
|
user's scope and return compact evidence.
|
||||||
|
|
||||||
|
Read this before the first query for broad synthesis, progress summaries,
|
||||||
|
design history, weekly/monthly reviews, or questions that ask what the user did,
|
||||||
|
learned, decided, tried, or abandoned. Start with a helper-first pass; use raw
|
||||||
|
`sql()` only when the helper surface cannot express the needed join or
|
||||||
|
aggregation.
|
||||||
|
|
||||||
|
## First Pass: Overview + Recall + Evidence
|
||||||
|
|
||||||
|
Use this for broad synthesis before writing custom SQL. It gives the agent a
|
||||||
|
map, prior notes, and raw session evidence in one bounded result. Then run a
|
||||||
|
faceted detail pass if the first pass reveals useful projects, sessions, files,
|
||||||
|
or terms.
|
||||||
|
|
||||||
|
```js
|
||||||
|
const topic = 'topic terms from the user request';
|
||||||
|
const map = overview({ limit: 6 });
|
||||||
|
const project = map.current.project?.project;
|
||||||
|
const scoped = project ? { project } : {};
|
||||||
|
|
||||||
|
return {
|
||||||
|
query_plan: {
|
||||||
|
mode: 'first_pass',
|
||||||
|
topic,
|
||||||
|
project: project || null,
|
||||||
|
limits: { sessions: 6, memories: 5, search: 8 },
|
||||||
|
},
|
||||||
|
orientation: map.current_project && {
|
||||||
|
project: map.current_project.project,
|
||||||
|
session_total: map.current_project.session_total,
|
||||||
|
sessions: map.current_project.sessions.map(s => ({
|
||||||
|
id: s.id,
|
||||||
|
title: s.title,
|
||||||
|
branch: s.git_branch,
|
||||||
|
ended_at: s.ended_at,
|
||||||
|
})),
|
||||||
|
memory_total: map.current_project.memory_total,
|
||||||
|
memories: map.current_project.memories.map(m => ({
|
||||||
|
id: m.id,
|
||||||
|
path: m.path,
|
||||||
|
summary: m.summary?.slice(0, 240),
|
||||||
|
})),
|
||||||
|
},
|
||||||
|
prior_memories: memories({ ...scoped, query: topic, limit: 5 }).map(m => ({
|
||||||
|
id: m.id,
|
||||||
|
path: m.path,
|
||||||
|
session_id: m.session_id,
|
||||||
|
created_at: m.created_at,
|
||||||
|
summary: m.summary?.slice(0, 260),
|
||||||
|
})),
|
||||||
|
session_evidence: search(topic.replace(/[-_]/g, ' '), { ...scoped, limit: 8 })
|
||||||
|
.slice(0, 6)
|
||||||
|
.map(h => ({
|
||||||
|
session_id: h.session.id,
|
||||||
|
session_title: h.session.title,
|
||||||
|
uuid: h.message.uuid,
|
||||||
|
timestamp: h.message.timestamp,
|
||||||
|
snippet: h.message.text?.slice(0, 220),
|
||||||
|
})),
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
## Orient Before Retrieval
|
## Orient Before Retrieval
|
||||||
|
|
||||||
Use `overview()` when the current project or available scopes are unclear. Treat
|
Use `overview()` when the current project or available scopes are unclear. Treat
|
||||||
|
|||||||
@@ -21,6 +21,11 @@ project counts, and recent current-project session/memory entry points. Use it
|
|||||||
when scope is unclear, then query the memory or raw session layer for evidence.
|
when scope is unclear, then query the memory or raw session layer for evidence.
|
||||||
It does not guess the current session.
|
It does not guess the current session.
|
||||||
|
|
||||||
|
For a new task, the first pass normally starts with `overview({ limit: 6 })`
|
||||||
|
unless the user gave an exact session ID, message UUID, or absolute file path.
|
||||||
|
Broad synthesis and progress-summary tasks should start from
|
||||||
|
`references/query-patterns.md`, not raw SQL.
|
||||||
|
|
||||||
One-shot retrieval is not all-shot retrieval. A query script may perform
|
One-shot retrieval is not all-shot retrieval. A query script may perform
|
||||||
multiple steps, but the first locator should be the narrowest semantic fit. If a
|
multiple steps, but the first locator should be the narrowest semantic fit. If a
|
||||||
scope locator finds the relevant project/session/file, do not also run broad FTS
|
scope locator finds the relevant project/session/file, do not also run broad FTS
|
||||||
@@ -34,9 +39,11 @@ Project-like fields are distinct:
|
|||||||
- `messages.cwd`: working directory at message time.
|
- `messages.cwd`: working directory at message time.
|
||||||
- helper `project`: SQL `LIKE` over `sessions.project`, not exact membership.
|
- helper `project`: SQL `LIKE` over `sessions.project`, not exact membership.
|
||||||
|
|
||||||
For exact project membership, use `sql()` with `s.project = ?` or
|
For exact project membership, prefer helper filters or a scoped first pass when
|
||||||
`s.project_path = ?`. Empty or tiny scoped results are valid results; do not
|
they are expressive enough; use `sql()` with `s.project = ?` or
|
||||||
broaden unless the user asks or your `query_plan` explicitly marks a fallback.
|
`s.project_path = ?` when you need exact membership across a join or
|
||||||
|
aggregation. Empty or tiny scoped results are valid results; do not broaden
|
||||||
|
unless the user asks or your `query_plan` explicitly marks a fallback.
|
||||||
|
|
||||||
### Plan Before Probe
|
### Plan Before Probe
|
||||||
|
|
||||||
@@ -58,7 +65,9 @@ If vocabulary is still unclear, use a small filtered window and say so in
|
|||||||
|
|
||||||
### Structure Before Text
|
### Structure Before Text
|
||||||
|
|
||||||
Use the database shape before asking the model to read text.
|
Use the database shape before asking the model to read text. This means
|
||||||
|
structured helpers and compact JS shaping first; raw SQL only when it expresses
|
||||||
|
the needed join, grouping, or exact schema-level check better than helpers.
|
||||||
|
|
||||||
- Count and aggregate in SQL or JS (`GROUP BY`, `COUNT`, `MAX`, `ORDER BY`, `LIMIT`).
|
- Count and aggregate in SQL or JS (`GROUP BY`, `COUNT`, `MAX`, `ORDER BY`, `LIMIT`).
|
||||||
- Join metadata from the owner table instead of inventing fields.
|
- Join metadata from the owner table instead of inventing fields.
|
||||||
|
|||||||
Reference in New Issue
Block a user