feat(query): add overview() for project-aware session/memory discovery
Resolves current project from cwd, lists all known projects with session and memory counts, and returns the current project's recent sessions and memories in one call. Enables the agent to orient itself at the start of a retrieval without multiple exploratory queries.
This commit is contained in:
@@ -108,9 +108,9 @@ references only when the question needs them:
|
||||
- `context(uuid)` — full story around a message (parent chain, subagent/workflow metadata)
|
||||
- `sql(query, ...params)` — read-only SQL for structured queries
|
||||
|
||||
**Structured shortcuts** — session, memory, summary, subagent, workflow,
|
||||
file-history, failure, raw-window, and parent-chain helpers over the same
|
||||
SQLite data.
|
||||
**Structured shortcuts** — overview, session, memory, summary, subagent,
|
||||
workflow, file-history, failure, raw-window, and parent-chain helpers over the
|
||||
same SQLite data.
|
||||
|
||||
**References** — agent reads on demand when the task needs deeper structure:
|
||||
|
||||
|
||||
@@ -122,6 +122,7 @@ All list helpers accept a bounded `limit`. Many also accept:
|
||||
`{ project, after, before, sessionId, sessions, branch }`. Check the schema or a
|
||||
tiny sample before relying on less common filters.
|
||||
|
||||
- `overview(opts?)` -- compact orientation map. Returns current cwd/project if knowable, global project counts, and current-project recent sessions plus memory records. It is a map, not evidence.
|
||||
- `sessions(opts?)` -- session rows, newest first. `project` is a SQL `LIKE` pattern.
|
||||
- `recent(n?)` -- shorthand for recent sessions.
|
||||
- `summaries(opts?)` -- summary rows, newest first: `{ id, session_id, timestamp, source, content, session_title, project }`.
|
||||
@@ -140,9 +141,11 @@ tiny sample before relying on less common filters.
|
||||
Keep queries scoped, bounded, and structural.
|
||||
|
||||
- Scope First: classify the locator as scope, artifact, or semantic. Use the narrowest structural locator before FTS; empty scoped results are valid unless the user asks to broaden.
|
||||
- Orient When Needed: use `overview()` when the current project or available scopes are unclear. It is a navigation map; confirm facts with `memories()`, `search()`, helpers, or `sql()`.
|
||||
- Plan Before Probe: for conclusion, broad history, failure investigation, or file evolution, write a bounded retrieval script instead of spending turns on intermediate results.
|
||||
- Structure Before Text: compute counts, joins, grouping, dedupe, and projection in SQL or JS; keep runtime JSON compact, ideally under 10k-12k chars for synthesis tasks.
|
||||
- Evidence Before Conclusion: return compact evidence with stable IDs (`session_id`, `uuid`, `tool_call_id`, `run_id`, `agent_id`) and short snippets, then synthesize in the final answer.
|
||||
- Persist Durable Conclusions: after answering, if retrieval produced a durable conclusion that future sessions are likely to reuse and `memories()` does not already cover it, explicitly offer to write a memory. Keep the offer brief. Do not write the markdown file or run `--remember` until the user approves.
|
||||
|
||||
If field, context, ordering, FTS, or helper semantics affect the query, read
|
||||
`references/retrieval-semantics.md` before coding. If a query errors, read
|
||||
@@ -163,6 +166,12 @@ prior conclusions relevant to the current task. Like other list helpers,
|
||||
passing a string is treated as `sessionId`, and passing a number is treated as
|
||||
`limit`. Read the file at `path` for full content.
|
||||
|
||||
Good memory candidates include design decisions, project conventions, abandoned
|
||||
alternatives, repeated failure causes, workflow patterns, and conclusions
|
||||
synthesized across multiple raw evidence points. Do not propose memory for
|
||||
one-off lookups, uncertain findings, or conclusions already covered by existing
|
||||
memories.
|
||||
|
||||
**Writing memories:** after a retrieval produces a conclusion worth persisting,
|
||||
propose writing a memory file. The user must approve. Flow:
|
||||
|
||||
|
||||
@@ -4,6 +4,37 @@ These are copyable CodeAct patterns for `runtime.mjs --query` scripts, plus one
|
||||
`--remember` registration pattern. They are not new APIs. Adapt them to the
|
||||
user's scope and return compact evidence.
|
||||
|
||||
## Orient Before Retrieval
|
||||
|
||||
Use `overview()` when the current project or available scopes are unclear. Treat
|
||||
the result as a map, not evidence; follow up with `memories()`, `search()`,
|
||||
helpers, or `sql()` for facts.
|
||||
|
||||
```js
|
||||
const map = overview({ limit: 6 });
|
||||
return {
|
||||
current: map.current,
|
||||
current_project: map.current_project && {
|
||||
project: map.current_project.project,
|
||||
session_total: map.current_project.session_total,
|
||||
sessions: map.current_project.sessions.map(s => ({
|
||||
id: s.id,
|
||||
title: s.title,
|
||||
branch: s.git_branch,
|
||||
ended_at: s.ended_at,
|
||||
})),
|
||||
memory_total: map.current_project.memory_total,
|
||||
memories: map.current_project.memories.map(m => ({
|
||||
id: m.id,
|
||||
path: m.path,
|
||||
summary: m.summary?.slice(0, 240),
|
||||
})),
|
||||
},
|
||||
projects: map.projects.slice(0, 8),
|
||||
totals: map.totals,
|
||||
};
|
||||
```
|
||||
|
||||
## Bounded Search To Context
|
||||
|
||||
Use `search()` to locate candidates, then expand only the strongest hits.
|
||||
|
||||
@@ -11,10 +11,16 @@ Classify the user's request before choosing tools.
|
||||
|
||||
| User signal | Locator mode | Start with | Avoid first |
|
||||
|-------------|--------------|------------|-------------|
|
||||
| unclear project/session landscape | orientation | `overview()` | treating overview rows as evidence |
|
||||
| project name/path, session, cwd, file, time range | scope | `sessions()`, exact SQL on `project_path`, `sessionId`, `fileHistory()` | broad FTS |
|
||||
| workflow, subagent, tool call, summary, edit | artifact | `workflows()`, `subagents()`, `summaries()`, `tool_calls`, `tool_results` | all-session search |
|
||||
| concept, conclusion, design history, vague memory | semantic | `memories({ query })`, `search()`, summaries, bounded facet sweep | session dumps |
|
||||
|
||||
`overview()` is a navigation map: current cwd/project if knowable, global
|
||||
project counts, and recent current-project session/memory entry points. Use it
|
||||
when scope is unclear, then query the memory or raw session layer for evidence.
|
||||
It does not guess the current session.
|
||||
|
||||
One-shot retrieval is not all-shot retrieval. A query script may perform
|
||||
multiple steps, but the first locator should be the narrowest semantic fit. If a
|
||||
scope locator finds the relevant project/session/file, do not also run broad FTS
|
||||
@@ -94,6 +100,15 @@ For semantic questions, build a task-local evidence view:
|
||||
Then synthesize the conclusion in the final answer. Do not pretend the raw
|
||||
evidence view is itself a stored Obelisk entity.
|
||||
|
||||
After synthesis, check whether the conclusion should become a memory. Offer to
|
||||
write one when the result is durable, likely to help future sessions, and not
|
||||
already covered by `prior_memories`. Good candidates include design decisions,
|
||||
project conventions, abandoned alternatives, repeated failure causes, workflow
|
||||
patterns, and conclusions synthesized across multiple raw evidence points. Do
|
||||
not propose memory for one-off lookups, uncertain findings, or duplicate
|
||||
coverage. The offer is only a proposal: write the markdown file and run
|
||||
`--remember` only after user approval.
|
||||
|
||||
## Text Search Semantics
|
||||
|
||||
`search(text)` passes text to SQLite FTS5 `MATCH`.
|
||||
|
||||
@@ -416,6 +416,85 @@ const last5 = recent(5);
|
||||
return last5.map(s => ({ title: s.title, project: s.project_path, ended: s.ended_at }));
|
||||
```
|
||||
|
||||
#### `overview(opts?)`
|
||||
|
||||
Compact orientation map for choosing the next retrieval scope. It is not an
|
||||
evidence helper: it does not return snippets, full messages, or markdown file
|
||||
contents. Passing a string is treated as `project`, and passing a number is
|
||||
treated as the current-project session `limit`.
|
||||
|
||||
| Param | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `opts.project` | `string` | Project slug or SQL `LIKE` pattern to use as the current project scope |
|
||||
| `opts.limit` | `number` | Max recent sessions in `current_project.sessions` (default 8) |
|
||||
| `opts.projectLimit` | `number` | Max rows in the global `projects` map (default 20) |
|
||||
| `opts.memoryLimit` | `number` | Max memory records in `current_project.memories` (default 100) |
|
||||
|
||||
If `opts.project` is absent, `overview()` tries to identify the current project
|
||||
from `process.cwd()` against `sessions.project_path`, then from exact
|
||||
`messages.cwd` matches. It does not guess the current session.
|
||||
|
||||
**Returns:**
|
||||
|
||||
```js
|
||||
{
|
||||
current: {
|
||||
cwd,
|
||||
project: {
|
||||
project,
|
||||
project_path,
|
||||
source: 'opts' | 'cwd_project_path' | 'cwd_messages',
|
||||
confidence: 'exact' | 'inferred' | 'unknown'
|
||||
} | null
|
||||
},
|
||||
current_project: {
|
||||
project,
|
||||
project_path,
|
||||
session_total,
|
||||
sessions: [
|
||||
{ id, title, project, project_path, started_at, ended_at, git_branch, message_count }
|
||||
],
|
||||
memory_total,
|
||||
memories: [
|
||||
{ id, path, summary, session_id, project, created_at }
|
||||
]
|
||||
} | null,
|
||||
projects: [
|
||||
{
|
||||
project,
|
||||
project_path,
|
||||
session_count,
|
||||
memory_count,
|
||||
last_session_at,
|
||||
last_memory_at,
|
||||
recent_branches
|
||||
}
|
||||
],
|
||||
totals: { projects, sessions, memories }
|
||||
}
|
||||
```
|
||||
|
||||
Use `current_project.sessions` as recent entry points only. `session_total` and
|
||||
`memory_total` tell you whether the returned arrays are complete enough for the
|
||||
task. Confirm facts with `memories()`, `search()`, other helpers, or `sql()`.
|
||||
|
||||
```js
|
||||
const map = overview({ limit: 5 });
|
||||
return {
|
||||
current: map.current,
|
||||
sessions: map.current_project?.sessions.map(s => ({
|
||||
id: s.id,
|
||||
title: s.title,
|
||||
ended_at: s.ended_at,
|
||||
})),
|
||||
memories: map.current_project?.memories.map(m => ({
|
||||
id: m.id,
|
||||
path: m.path,
|
||||
summary: m.summary,
|
||||
})),
|
||||
};
|
||||
```
|
||||
|
||||
#### `sessions(opts?)`
|
||||
|
||||
Query sessions with filters. For backward compatibility, passing a number is treated as `limit`.
|
||||
|
||||
+175
-1
@@ -40,6 +40,13 @@ function createQueryApi(db) {
|
||||
return db.prepare(sql).all(...p);
|
||||
};
|
||||
|
||||
const normalizeOverviewOpts = (optsOrScalar) => {
|
||||
if (optsOrScalar == null) return {};
|
||||
if (typeof optsOrScalar === 'string') return { project: optsOrScalar };
|
||||
if (typeof optsOrScalar === 'number') return { limit: optsOrScalar };
|
||||
return optsOrScalar;
|
||||
};
|
||||
|
||||
const search = (text, opts = {}) => {
|
||||
const { limit = 20, sessionId, project, after, before, cwd } = opts;
|
||||
let where = 'WHERE mf.text MATCH ?';
|
||||
@@ -181,6 +188,173 @@ function createQueryApi(db) {
|
||||
return db.prepare(`SELECT su.*, s.title as session_title, s.project FROM summaries su LEFT JOIN sessions s ON s.id=su.session_id WHERE ${where} ORDER BY su.timestamp DESC LIMIT ?`).all(...params);
|
||||
};
|
||||
|
||||
const overview = (optsOrScalar) => {
|
||||
const opts = normalizeOverviewOpts(optsOrScalar);
|
||||
const cwd = process.cwd();
|
||||
const sessionLimit = opts.limit ?? 8;
|
||||
const projectLimit = opts.projectLimit ?? 20;
|
||||
const memoryLimit = opts.memoryLimit ?? 100;
|
||||
|
||||
const projectDescriptor = (row, source, confidence) => row ? ({
|
||||
project: row.project,
|
||||
project_path: row.project_path || null,
|
||||
source,
|
||||
confidence,
|
||||
}) : null;
|
||||
|
||||
const latestProjectByPattern = (pattern) => {
|
||||
const fromSessions = db.prepare(`
|
||||
SELECT project, project_path
|
||||
FROM sessions
|
||||
WHERE project LIKE ?
|
||||
ORDER BY COALESCE(ended_at, started_at) DESC
|
||||
LIMIT 1
|
||||
`).get(pattern);
|
||||
if (fromSessions) return fromSessions;
|
||||
return db.prepare(`
|
||||
SELECT project, NULL AS project_path
|
||||
FROM memories
|
||||
WHERE project LIKE ?
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 1
|
||||
`).get(pattern);
|
||||
};
|
||||
|
||||
const resolveCurrentProject = () => {
|
||||
if (opts.project) {
|
||||
const row = latestProjectByPattern(opts.project);
|
||||
const confidence = row ? (/[%_]/.test(opts.project) ? 'inferred' : 'exact') : 'unknown';
|
||||
return projectDescriptor(row || { project: opts.project, project_path: null }, 'opts', confidence);
|
||||
}
|
||||
|
||||
const paths = db.prepare(`
|
||||
SELECT project, project_path, MAX(COALESCE(ended_at, started_at)) AS last_seen
|
||||
FROM sessions
|
||||
WHERE project IS NOT NULL AND project_path IS NOT NULL AND project_path != ''
|
||||
GROUP BY project, project_path
|
||||
`).all();
|
||||
const byProjectPath = paths
|
||||
.filter(r => cwd === r.project_path || cwd.startsWith(r.project_path + path.sep))
|
||||
.sort((a, b) => b.project_path.length - a.project_path.length || String(b.last_seen || '').localeCompare(String(a.last_seen || '')))[0];
|
||||
if (byProjectPath) return projectDescriptor(byProjectPath, 'cwd_project_path', 'exact');
|
||||
|
||||
const byMessageCwd = db.prepare(`
|
||||
SELECT s.project, s.project_path, MAX(m.timestamp) AS last_seen
|
||||
FROM messages m
|
||||
LEFT JOIN sessions s ON s.id=m.session_id
|
||||
WHERE m.cwd = ? AND s.project IS NOT NULL
|
||||
GROUP BY s.project, s.project_path
|
||||
ORDER BY last_seen DESC
|
||||
LIMIT 1
|
||||
`).get(cwd);
|
||||
if (byMessageCwd) return projectDescriptor(byMessageCwd, 'cwd_messages', 'inferred');
|
||||
|
||||
return null;
|
||||
};
|
||||
|
||||
const projects = db.prepare(`
|
||||
WITH names AS (
|
||||
SELECT project FROM sessions WHERE project IS NOT NULL GROUP BY project
|
||||
UNION
|
||||
SELECT project FROM memories WHERE project IS NOT NULL GROUP BY project
|
||||
),
|
||||
session_stats AS (
|
||||
SELECT project, COUNT(*) AS session_count, MAX(COALESCE(ended_at, started_at)) AS last_session_at
|
||||
FROM sessions
|
||||
WHERE project IS NOT NULL
|
||||
GROUP BY project
|
||||
),
|
||||
memory_stats AS (
|
||||
SELECT project, COUNT(*) AS memory_count, MAX(created_at) AS last_memory_at
|
||||
FROM memories
|
||||
WHERE project IS NOT NULL
|
||||
GROUP BY project
|
||||
)
|
||||
SELECT
|
||||
n.project,
|
||||
(
|
||||
SELECT s2.project_path
|
||||
FROM sessions s2
|
||||
WHERE s2.project = n.project AND s2.project_path IS NOT NULL
|
||||
ORDER BY COALESCE(s2.ended_at, s2.started_at) DESC
|
||||
LIMIT 1
|
||||
) AS project_path,
|
||||
COALESCE(ss.session_count, 0) AS session_count,
|
||||
COALESCE(ms.memory_count, 0) AS memory_count,
|
||||
ss.last_session_at,
|
||||
ms.last_memory_at
|
||||
FROM names n
|
||||
LEFT JOIN session_stats ss ON ss.project = n.project
|
||||
LEFT JOIN memory_stats ms ON ms.project = n.project
|
||||
ORDER BY COALESCE(ss.last_session_at, ms.last_memory_at) DESC
|
||||
LIMIT ?
|
||||
`).all(projectLimit).map(row => {
|
||||
const branches = db.prepare(`
|
||||
SELECT git_branch
|
||||
FROM sessions
|
||||
WHERE project = ? AND git_branch IS NOT NULL AND git_branch != ''
|
||||
GROUP BY git_branch
|
||||
ORDER BY MAX(COALESCE(ended_at, started_at)) DESC
|
||||
LIMIT 5
|
||||
`).all(row.project).map(r => r.git_branch);
|
||||
return { ...row, recent_branches: branches };
|
||||
});
|
||||
|
||||
const currentProject = resolveCurrentProject();
|
||||
let current_project = null;
|
||||
if (currentProject?.project) {
|
||||
const sessionTotal = db.prepare('SELECT COUNT(*) AS c FROM sessions WHERE project = ?').get(currentProject.project)?.c || 0;
|
||||
const sessionsForProject = db.prepare(`
|
||||
SELECT id, title, project, project_path, started_at, ended_at, git_branch, message_count
|
||||
FROM sessions
|
||||
WHERE project = ?
|
||||
ORDER BY COALESCE(ended_at, started_at) DESC
|
||||
LIMIT ?
|
||||
`).all(currentProject.project, sessionLimit);
|
||||
const memoryTotal = db.prepare('SELECT COUNT(*) AS c FROM memories WHERE project = ?').get(currentProject.project)?.c || 0;
|
||||
const memoriesForProject = db.prepare(`
|
||||
SELECT id, path, summary, session_id, project, created_at
|
||||
FROM memories
|
||||
WHERE project = ?
|
||||
ORDER BY created_at DESC
|
||||
LIMIT ?
|
||||
`).all(currentProject.project, memoryLimit);
|
||||
current_project = {
|
||||
project: currentProject.project,
|
||||
project_path: currentProject.project_path,
|
||||
session_total: sessionTotal,
|
||||
sessions: sessionsForProject,
|
||||
memory_total: memoryTotal,
|
||||
memories: memoriesForProject,
|
||||
};
|
||||
}
|
||||
|
||||
const totalProjects = db.prepare(`
|
||||
SELECT COUNT(*) AS c
|
||||
FROM (
|
||||
SELECT project FROM sessions WHERE project IS NOT NULL GROUP BY project
|
||||
UNION
|
||||
SELECT project FROM memories WHERE project IS NOT NULL GROUP BY project
|
||||
)
|
||||
`).get()?.c || 0;
|
||||
const totalSessions = db.prepare('SELECT COUNT(*) AS c FROM sessions').get()?.c || 0;
|
||||
const totalMemories = db.prepare('SELECT COUNT(*) AS c FROM memories').get()?.c || 0;
|
||||
|
||||
return {
|
||||
current: {
|
||||
cwd,
|
||||
project: currentProject,
|
||||
},
|
||||
current_project,
|
||||
projects,
|
||||
totals: {
|
||||
projects: totalProjects,
|
||||
sessions: totalSessions,
|
||||
memories: totalMemories,
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
const resolveJsonlPath = (messageUuid) => {
|
||||
const msg = db.prepare('SELECT session_id, agent_id FROM messages WHERE uuid=?').get(messageUuid);
|
||||
if (!msg) return null;
|
||||
@@ -251,7 +425,7 @@ function createQueryApi(db) {
|
||||
return db.prepare(`SELECT mem.* FROM memories mem ${join} WHERE ${where} ORDER BY mem.created_at DESC LIMIT ?`).all(...params);
|
||||
};
|
||||
|
||||
return { sql: q, search, context, trace, thread, subagents, workflows, workflowTree, fileHistory, failures, sessions, recent, summaries, raw, memories };
|
||||
return { sql: q, search, context, trace, thread, subagents, workflows, workflowTree, fileHistory, failures, sessions, recent, summaries, raw, memories, overview };
|
||||
}
|
||||
|
||||
function createRememberApi(db) {
|
||||
|
||||
Reference in New Issue
Block a user