feat(query): add overview() for project-aware session/memory discovery

Resolves current project from cwd, lists all known projects with session
  and memory counts, and returns the current project's recent sessions and
  memories in one call. Enables the agent to orient itself at the start of
  a retrieval without multiple exploratory queries.
This commit is contained in:
tommy0103
2026-06-10 03:04:11 +08:00
parent 34f3a164ab
commit 807e5141fa
6 changed files with 312 additions and 4 deletions
+3 -3
View File
@@ -108,9 +108,9 @@ references only when the question needs them:
- `context(uuid)` — full story around a message (parent chain, subagent/workflow metadata) - `context(uuid)` — full story around a message (parent chain, subagent/workflow metadata)
- `sql(query, ...params)` — read-only SQL for structured queries - `sql(query, ...params)` — read-only SQL for structured queries
**Structured shortcuts** — session, memory, summary, subagent, workflow, **Structured shortcuts** overview, session, memory, summary, subagent,
file-history, failure, raw-window, and parent-chain helpers over the same workflow, file-history, failure, raw-window, and parent-chain helpers over the
SQLite data. same SQLite data.
**References** — agent reads on demand when the task needs deeper structure: **References** — agent reads on demand when the task needs deeper structure:
+9
View File
@@ -122,6 +122,7 @@ All list helpers accept a bounded `limit`. Many also accept:
`{ project, after, before, sessionId, sessions, branch }`. Check the schema or a `{ project, after, before, sessionId, sessions, branch }`. Check the schema or a
tiny sample before relying on less common filters. tiny sample before relying on less common filters.
- `overview(opts?)` -- compact orientation map. Returns current cwd/project if knowable, global project counts, and current-project recent sessions plus memory records. It is a map, not evidence.
- `sessions(opts?)` -- session rows, newest first. `project` is a SQL `LIKE` pattern. - `sessions(opts?)` -- session rows, newest first. `project` is a SQL `LIKE` pattern.
- `recent(n?)` -- shorthand for recent sessions. - `recent(n?)` -- shorthand for recent sessions.
- `summaries(opts?)` -- summary rows, newest first: `{ id, session_id, timestamp, source, content, session_title, project }`. - `summaries(opts?)` -- summary rows, newest first: `{ id, session_id, timestamp, source, content, session_title, project }`.
@@ -140,9 +141,11 @@ tiny sample before relying on less common filters.
Keep queries scoped, bounded, and structural. Keep queries scoped, bounded, and structural.
- Scope First: classify the locator as scope, artifact, or semantic. Use the narrowest structural locator before FTS; empty scoped results are valid unless the user asks to broaden. - Scope First: classify the locator as scope, artifact, or semantic. Use the narrowest structural locator before FTS; empty scoped results are valid unless the user asks to broaden.
- Orient When Needed: use `overview()` when the current project or available scopes are unclear. It is a navigation map; confirm facts with `memories()`, `search()`, helpers, or `sql()`.
- Plan Before Probe: for conclusion, broad history, failure investigation, or file evolution, write a bounded retrieval script instead of spending turns on intermediate results. - Plan Before Probe: for conclusion, broad history, failure investigation, or file evolution, write a bounded retrieval script instead of spending turns on intermediate results.
- Structure Before Text: compute counts, joins, grouping, dedupe, and projection in SQL or JS; keep runtime JSON compact, ideally under 10k-12k chars for synthesis tasks. - Structure Before Text: compute counts, joins, grouping, dedupe, and projection in SQL or JS; keep runtime JSON compact, ideally under 10k-12k chars for synthesis tasks.
- Evidence Before Conclusion: return compact evidence with stable IDs (`session_id`, `uuid`, `tool_call_id`, `run_id`, `agent_id`) and short snippets, then synthesize in the final answer. - Evidence Before Conclusion: return compact evidence with stable IDs (`session_id`, `uuid`, `tool_call_id`, `run_id`, `agent_id`) and short snippets, then synthesize in the final answer.
- Persist Durable Conclusions: after answering, if retrieval produced a durable conclusion that future sessions are likely to reuse and `memories()` does not already cover it, explicitly offer to write a memory. Keep the offer brief. Do not write the markdown file or run `--remember` until the user approves.
If field, context, ordering, FTS, or helper semantics affect the query, read If field, context, ordering, FTS, or helper semantics affect the query, read
`references/retrieval-semantics.md` before coding. If a query errors, read `references/retrieval-semantics.md` before coding. If a query errors, read
@@ -163,6 +166,12 @@ prior conclusions relevant to the current task. Like other list helpers,
passing a string is treated as `sessionId`, and passing a number is treated as passing a string is treated as `sessionId`, and passing a number is treated as
`limit`. Read the file at `path` for full content. `limit`. Read the file at `path` for full content.
Good memory candidates include design decisions, project conventions, abandoned
alternatives, repeated failure causes, workflow patterns, and conclusions
synthesized across multiple raw evidence points. Do not propose memory for
one-off lookups, uncertain findings, or conclusions already covered by existing
memories.
**Writing memories:** after a retrieval produces a conclusion worth persisting, **Writing memories:** after a retrieval produces a conclusion worth persisting,
propose writing a memory file. The user must approve. Flow: propose writing a memory file. The user must approve. Flow:
+31
View File
@@ -4,6 +4,37 @@ These are copyable CodeAct patterns for `runtime.mjs --query` scripts, plus one
`--remember` registration pattern. They are not new APIs. Adapt them to the `--remember` registration pattern. They are not new APIs. Adapt them to the
user's scope and return compact evidence. user's scope and return compact evidence.
## Orient Before Retrieval
Use `overview()` when the current project or available scopes are unclear. Treat
the result as a map, not evidence; follow up with `memories()`, `search()`,
helpers, or `sql()` for facts.
```js
const map = overview({ limit: 6 });
return {
current: map.current,
current_project: map.current_project && {
project: map.current_project.project,
session_total: map.current_project.session_total,
sessions: map.current_project.sessions.map(s => ({
id: s.id,
title: s.title,
branch: s.git_branch,
ended_at: s.ended_at,
})),
memory_total: map.current_project.memory_total,
memories: map.current_project.memories.map(m => ({
id: m.id,
path: m.path,
summary: m.summary?.slice(0, 240),
})),
},
projects: map.projects.slice(0, 8),
totals: map.totals,
};
```
## Bounded Search To Context ## Bounded Search To Context
Use `search()` to locate candidates, then expand only the strongest hits. Use `search()` to locate candidates, then expand only the strongest hits.
+15
View File
@@ -11,10 +11,16 @@ Classify the user's request before choosing tools.
| User signal | Locator mode | Start with | Avoid first | | User signal | Locator mode | Start with | Avoid first |
|-------------|--------------|------------|-------------| |-------------|--------------|------------|-------------|
| unclear project/session landscape | orientation | `overview()` | treating overview rows as evidence |
| project name/path, session, cwd, file, time range | scope | `sessions()`, exact SQL on `project_path`, `sessionId`, `fileHistory()` | broad FTS | | project name/path, session, cwd, file, time range | scope | `sessions()`, exact SQL on `project_path`, `sessionId`, `fileHistory()` | broad FTS |
| workflow, subagent, tool call, summary, edit | artifact | `workflows()`, `subagents()`, `summaries()`, `tool_calls`, `tool_results` | all-session search | | workflow, subagent, tool call, summary, edit | artifact | `workflows()`, `subagents()`, `summaries()`, `tool_calls`, `tool_results` | all-session search |
| concept, conclusion, design history, vague memory | semantic | `memories({ query })`, `search()`, summaries, bounded facet sweep | session dumps | | concept, conclusion, design history, vague memory | semantic | `memories({ query })`, `search()`, summaries, bounded facet sweep | session dumps |
`overview()` is a navigation map: current cwd/project if knowable, global
project counts, and recent current-project session/memory entry points. Use it
when scope is unclear, then query the memory or raw session layer for evidence.
It does not guess the current session.
One-shot retrieval is not all-shot retrieval. A query script may perform One-shot retrieval is not all-shot retrieval. A query script may perform
multiple steps, but the first locator should be the narrowest semantic fit. If a multiple steps, but the first locator should be the narrowest semantic fit. If a
scope locator finds the relevant project/session/file, do not also run broad FTS scope locator finds the relevant project/session/file, do not also run broad FTS
@@ -94,6 +100,15 @@ For semantic questions, build a task-local evidence view:
Then synthesize the conclusion in the final answer. Do not pretend the raw Then synthesize the conclusion in the final answer. Do not pretend the raw
evidence view is itself a stored Obelisk entity. evidence view is itself a stored Obelisk entity.
After synthesis, check whether the conclusion should become a memory. Offer to
write one when the result is durable, likely to help future sessions, and not
already covered by `prior_memories`. Good candidates include design decisions,
project conventions, abandoned alternatives, repeated failure causes, workflow
patterns, and conclusions synthesized across multiple raw evidence points. Do
not propose memory for one-off lookups, uncertain findings, or duplicate
coverage. The offer is only a proposal: write the markdown file and run
`--remember` only after user approval.
## Text Search Semantics ## Text Search Semantics
`search(text)` passes text to SQLite FTS5 `MATCH`. `search(text)` passes text to SQLite FTS5 `MATCH`.
+79
View File
@@ -416,6 +416,85 @@ const last5 = recent(5);
return last5.map(s => ({ title: s.title, project: s.project_path, ended: s.ended_at })); return last5.map(s => ({ title: s.title, project: s.project_path, ended: s.ended_at }));
``` ```
#### `overview(opts?)`
Compact orientation map for choosing the next retrieval scope. It is not an
evidence helper: it does not return snippets, full messages, or markdown file
contents. Passing a string is treated as `project`, and passing a number is
treated as the current-project session `limit`.
| Param | Type | Description |
|-------|------|-------------|
| `opts.project` | `string` | Project slug or SQL `LIKE` pattern to use as the current project scope |
| `opts.limit` | `number` | Max recent sessions in `current_project.sessions` (default 8) |
| `opts.projectLimit` | `number` | Max rows in the global `projects` map (default 20) |
| `opts.memoryLimit` | `number` | Max memory records in `current_project.memories` (default 100) |
If `opts.project` is absent, `overview()` tries to identify the current project
from `process.cwd()` against `sessions.project_path`, then from exact
`messages.cwd` matches. It does not guess the current session.
**Returns:**
```js
{
current: {
cwd,
project: {
project,
project_path,
source: 'opts' | 'cwd_project_path' | 'cwd_messages',
confidence: 'exact' | 'inferred' | 'unknown'
} | null
},
current_project: {
project,
project_path,
session_total,
sessions: [
{ id, title, project, project_path, started_at, ended_at, git_branch, message_count }
],
memory_total,
memories: [
{ id, path, summary, session_id, project, created_at }
]
} | null,
projects: [
{
project,
project_path,
session_count,
memory_count,
last_session_at,
last_memory_at,
recent_branches
}
],
totals: { projects, sessions, memories }
}
```
Use `current_project.sessions` as recent entry points only. `session_total` and
`memory_total` tell you whether the returned arrays are complete enough for the
task. Confirm facts with `memories()`, `search()`, other helpers, or `sql()`.
```js
const map = overview({ limit: 5 });
return {
current: map.current,
sessions: map.current_project?.sessions.map(s => ({
id: s.id,
title: s.title,
ended_at: s.ended_at,
})),
memories: map.current_project?.memories.map(m => ({
id: m.id,
path: m.path,
summary: m.summary,
})),
};
```
#### `sessions(opts?)` #### `sessions(opts?)`
Query sessions with filters. For backward compatibility, passing a number is treated as `limit`. Query sessions with filters. For backward compatibility, passing a number is treated as `limit`.
+175 -1
View File
@@ -40,6 +40,13 @@ function createQueryApi(db) {
return db.prepare(sql).all(...p); return db.prepare(sql).all(...p);
}; };
const normalizeOverviewOpts = (optsOrScalar) => {
if (optsOrScalar == null) return {};
if (typeof optsOrScalar === 'string') return { project: optsOrScalar };
if (typeof optsOrScalar === 'number') return { limit: optsOrScalar };
return optsOrScalar;
};
const search = (text, opts = {}) => { const search = (text, opts = {}) => {
const { limit = 20, sessionId, project, after, before, cwd } = opts; const { limit = 20, sessionId, project, after, before, cwd } = opts;
let where = 'WHERE mf.text MATCH ?'; let where = 'WHERE mf.text MATCH ?';
@@ -181,6 +188,173 @@ function createQueryApi(db) {
return db.prepare(`SELECT su.*, s.title as session_title, s.project FROM summaries su LEFT JOIN sessions s ON s.id=su.session_id WHERE ${where} ORDER BY su.timestamp DESC LIMIT ?`).all(...params); return db.prepare(`SELECT su.*, s.title as session_title, s.project FROM summaries su LEFT JOIN sessions s ON s.id=su.session_id WHERE ${where} ORDER BY su.timestamp DESC LIMIT ?`).all(...params);
}; };
const overview = (optsOrScalar) => {
const opts = normalizeOverviewOpts(optsOrScalar);
const cwd = process.cwd();
const sessionLimit = opts.limit ?? 8;
const projectLimit = opts.projectLimit ?? 20;
const memoryLimit = opts.memoryLimit ?? 100;
const projectDescriptor = (row, source, confidence) => row ? ({
project: row.project,
project_path: row.project_path || null,
source,
confidence,
}) : null;
const latestProjectByPattern = (pattern) => {
const fromSessions = db.prepare(`
SELECT project, project_path
FROM sessions
WHERE project LIKE ?
ORDER BY COALESCE(ended_at, started_at) DESC
LIMIT 1
`).get(pattern);
if (fromSessions) return fromSessions;
return db.prepare(`
SELECT project, NULL AS project_path
FROM memories
WHERE project LIKE ?
ORDER BY created_at DESC
LIMIT 1
`).get(pattern);
};
const resolveCurrentProject = () => {
if (opts.project) {
const row = latestProjectByPattern(opts.project);
const confidence = row ? (/[%_]/.test(opts.project) ? 'inferred' : 'exact') : 'unknown';
return projectDescriptor(row || { project: opts.project, project_path: null }, 'opts', confidence);
}
const paths = db.prepare(`
SELECT project, project_path, MAX(COALESCE(ended_at, started_at)) AS last_seen
FROM sessions
WHERE project IS NOT NULL AND project_path IS NOT NULL AND project_path != ''
GROUP BY project, project_path
`).all();
const byProjectPath = paths
.filter(r => cwd === r.project_path || cwd.startsWith(r.project_path + path.sep))
.sort((a, b) => b.project_path.length - a.project_path.length || String(b.last_seen || '').localeCompare(String(a.last_seen || '')))[0];
if (byProjectPath) return projectDescriptor(byProjectPath, 'cwd_project_path', 'exact');
const byMessageCwd = db.prepare(`
SELECT s.project, s.project_path, MAX(m.timestamp) AS last_seen
FROM messages m
LEFT JOIN sessions s ON s.id=m.session_id
WHERE m.cwd = ? AND s.project IS NOT NULL
GROUP BY s.project, s.project_path
ORDER BY last_seen DESC
LIMIT 1
`).get(cwd);
if (byMessageCwd) return projectDescriptor(byMessageCwd, 'cwd_messages', 'inferred');
return null;
};
const projects = db.prepare(`
WITH names AS (
SELECT project FROM sessions WHERE project IS NOT NULL GROUP BY project
UNION
SELECT project FROM memories WHERE project IS NOT NULL GROUP BY project
),
session_stats AS (
SELECT project, COUNT(*) AS session_count, MAX(COALESCE(ended_at, started_at)) AS last_session_at
FROM sessions
WHERE project IS NOT NULL
GROUP BY project
),
memory_stats AS (
SELECT project, COUNT(*) AS memory_count, MAX(created_at) AS last_memory_at
FROM memories
WHERE project IS NOT NULL
GROUP BY project
)
SELECT
n.project,
(
SELECT s2.project_path
FROM sessions s2
WHERE s2.project = n.project AND s2.project_path IS NOT NULL
ORDER BY COALESCE(s2.ended_at, s2.started_at) DESC
LIMIT 1
) AS project_path,
COALESCE(ss.session_count, 0) AS session_count,
COALESCE(ms.memory_count, 0) AS memory_count,
ss.last_session_at,
ms.last_memory_at
FROM names n
LEFT JOIN session_stats ss ON ss.project = n.project
LEFT JOIN memory_stats ms ON ms.project = n.project
ORDER BY COALESCE(ss.last_session_at, ms.last_memory_at) DESC
LIMIT ?
`).all(projectLimit).map(row => {
const branches = db.prepare(`
SELECT git_branch
FROM sessions
WHERE project = ? AND git_branch IS NOT NULL AND git_branch != ''
GROUP BY git_branch
ORDER BY MAX(COALESCE(ended_at, started_at)) DESC
LIMIT 5
`).all(row.project).map(r => r.git_branch);
return { ...row, recent_branches: branches };
});
const currentProject = resolveCurrentProject();
let current_project = null;
if (currentProject?.project) {
const sessionTotal = db.prepare('SELECT COUNT(*) AS c FROM sessions WHERE project = ?').get(currentProject.project)?.c || 0;
const sessionsForProject = db.prepare(`
SELECT id, title, project, project_path, started_at, ended_at, git_branch, message_count
FROM sessions
WHERE project = ?
ORDER BY COALESCE(ended_at, started_at) DESC
LIMIT ?
`).all(currentProject.project, sessionLimit);
const memoryTotal = db.prepare('SELECT COUNT(*) AS c FROM memories WHERE project = ?').get(currentProject.project)?.c || 0;
const memoriesForProject = db.prepare(`
SELECT id, path, summary, session_id, project, created_at
FROM memories
WHERE project = ?
ORDER BY created_at DESC
LIMIT ?
`).all(currentProject.project, memoryLimit);
current_project = {
project: currentProject.project,
project_path: currentProject.project_path,
session_total: sessionTotal,
sessions: sessionsForProject,
memory_total: memoryTotal,
memories: memoriesForProject,
};
}
const totalProjects = db.prepare(`
SELECT COUNT(*) AS c
FROM (
SELECT project FROM sessions WHERE project IS NOT NULL GROUP BY project
UNION
SELECT project FROM memories WHERE project IS NOT NULL GROUP BY project
)
`).get()?.c || 0;
const totalSessions = db.prepare('SELECT COUNT(*) AS c FROM sessions').get()?.c || 0;
const totalMemories = db.prepare('SELECT COUNT(*) AS c FROM memories').get()?.c || 0;
return {
current: {
cwd,
project: currentProject,
},
current_project,
projects,
totals: {
projects: totalProjects,
sessions: totalSessions,
memories: totalMemories,
},
};
};
const resolveJsonlPath = (messageUuid) => { const resolveJsonlPath = (messageUuid) => {
const msg = db.prepare('SELECT session_id, agent_id FROM messages WHERE uuid=?').get(messageUuid); const msg = db.prepare('SELECT session_id, agent_id FROM messages WHERE uuid=?').get(messageUuid);
if (!msg) return null; if (!msg) return null;
@@ -251,7 +425,7 @@ function createQueryApi(db) {
return db.prepare(`SELECT mem.* FROM memories mem ${join} WHERE ${where} ORDER BY mem.created_at DESC LIMIT ?`).all(...params); return db.prepare(`SELECT mem.* FROM memories mem ${join} WHERE ${where} ORDER BY mem.created_at DESC LIMIT ?`).all(...params);
}; };
return { sql: q, search, context, trace, thread, subagents, workflows, workflowTree, fileHistory, failures, sessions, recent, summaries, raw, memories }; return { sql: q, search, context, trace, thread, subagents, workflows, workflowTree, fileHistory, failures, sessions, recent, summaries, raw, memories, overview };
} }
function createRememberApi(db) { function createRememberApi(db) {