feat: evolve retrieval layer with content_type, memory FTS, forget(), and recap references

Extract schema DDL into scripts/schema.sql. Add content_type and is_meta
  columns to messages for transcript control-plane filtering. Introduce
  FTS5-backed memory recall with safe tokenization, memory soft-delete via
  forget() through the renamed --attune runtime, and anchors on memory
  records. Expand query helpers (includeMeta, thread opts, overview
  project-path awareness). Add per-card recap retrieval and writing
  references under references/recap/.
This commit is contained in:
tommy0103
2026-06-15 02:25:21 +08:00
parent b52f57b538
commit 3822bda89e
24 changed files with 1290 additions and 157 deletions
+3
View File
@@ -2,3 +2,6 @@
plans/ plans/
.skillopt-backups .skillopt-backups
tests/ tests/
node_modules/
dist-renderer/
release/
+47 -5
View File
@@ -91,7 +91,14 @@ Reads the JSON result, answers you in natural language
When a retrieval produces a memory worth keeping, the agent proposes a markdown When a retrieval produces a memory worth keeping, the agent proposes a markdown
memory file. After user approval, it registers that file with the narrow memory file. After user approval, it registers that file with the narrow
`runtime.mjs --remember <script>` runtime, which exposes only `remember()`. `runtime.mjs --attune <script>` runtime, which exposes only memory mutation
helpers such as `remember()` and `forget()`.
Memory is a synthesis cache, not a replacement for raw sessions. The agent can
decide whether to use, ignore, or verify a memory during an answer. Persistent
changes still require human approval, but explicit corrections count: if you say
a memory is wrong, outdated, or should be replaced, the agent can archive or
update the exact matching record without a second confirmation.
**The core idea: don't make humans browse, tag, or organize sessions.** **The core idea: don't make humans browse, tag, or organize sessions.**
Don't invent a rigid query DSL either. Don't invent a rigid query DSL either.
@@ -104,7 +111,7 @@ references only when the question needs them:
**Core primitives** — the main CodeAct surface: **Core primitives** — the main CodeAct surface:
- `search(text)` — FTS5 full-text search, returns matches with surrounding context - `search(text)` — FTS5 full-text search, returns matches with surrounding context plus message `content_type` and `is_meta`
- `context(uuid)` — full story around a message (parent chain, subagent/workflow metadata) - `context(uuid)` — full story around a message (parent chain, subagent/workflow metadata)
- `sql(query, ...params)` — read-only SQL for structured queries - `sql(query, ...params)` — read-only SQL for structured queries
@@ -116,11 +123,26 @@ same SQLite data.
- `references/schema.md` — full SQLite schema and API reference - `references/schema.md` — full SQLite schema and API reference
- `references/query-patterns.md` — copyable CodeAct recipes for common retrieval tasks - `references/query-patterns.md` — copyable CodeAct recipes for common retrieval tasks
- `references/retrieval-semantics.md` — query design frame for scoped and synthesis retrieval
- `references/recap/overview.md` — optional `/obelisk recap` card-by-card entrypoint
- `references/recap/pattern1-cover.md` and `references/recap/writing1-cover.md`
- `references/recap/pattern2-thinking.md` and `references/recap/writing2-thinking.md`
- `references/recap/pattern3-vibe.md` and `references/recap/writing3-vibe.md`
- `references/recap/pattern4-workflow.md` and `references/recap/writing4-workflow.md`
- `references/recap/pattern5-closing.md` and `references/recap/writing5-closing.md`
- `references/pitfalls.md` — scope, FTS, ordering, compact/raw, and field-name traps - `references/pitfalls.md` — scope, FTS, ordering, compact/raw, and field-name traps
The executable SQLite schema lives in `scripts/schema.sql`; `references/schema.md`
is the human/agent explanation of that contract.
The design is progressive disclosure with guardrails: the main skill keeps the The design is progressive disclosure with guardrails: the main skill keeps the
core contract and high-risk pitfalls visible, while longer recipes and the full core contract and high-risk pitfalls visible, while longer recipes and the full
schema stay out of the first prompt until the agent needs them. schema stay out of the first prompt until the agent needs them.
The optional recap references are only for the explicit `/obelisk recap` intent;
they are not part of the ordinary retrieval path. `references/recap/overview.md`
drives a card-by-card loop: read one card's retrieval pattern, gather that
card's evidence, read its writing reference, update the JSON, then continue.
This keeps schema, taste, and query planning from competing in one large prompt.
## What gets indexed ## What gets indexed
@@ -132,9 +154,9 @@ schema stay out of the first prompt until the agent needs them.
| **Subagents** | `subagents/agent-<id>.jsonl` | Agent type, description, full conversation | | **Subagents** | `subagents/agent-<id>.jsonl` | Agent type, description, full conversation |
| **Workflows** | `workflows/wf_<runId>.json` | Script, structured result, agent count | | **Workflows** | `workflows/wf_<runId>.json` | Script, structured result, agent count |
| **Workflow agents** | `subagents/workflows/wf_<runId>/` | Per-agent transcripts linked to workflow | | **Workflow agents** | `subagents/workflows/wf_<runId>/` | Per-agent transcripts linked to workflow |
| **Memories** | markdown files registered by the agent after user approval | Prior conclusions linked to source sessions/messages | | **Memories** | markdown files registered by the agent after user approval | Prior conclusions linked to source sessions/messages and optional anchors |
Full-text search via FTS5 covers message text across every layer, while the SQLite tables preserve the structure agents need for investigation. Full-text search via FTS5 covers message text across every session layer and ranked memory recall over registered memory summaries, while the SQLite tables preserve the structure agents need for investigation.
## Structure ## Structure
@@ -142,16 +164,36 @@ Full-text search via FTS5 covers message text across every layer, while the SQLi
.claude/skills/obelisk/ .claude/skills/obelisk/
├── SKILL.md # Skill definition + simple API + examples ├── SKILL.md # Skill definition + simple API + examples
├── scripts/ ├── scripts/
── runtime.mjs # Indexer + query runtime (400 lines, zero deps) ── schema.sql # Executable SQLite schema
│ └── runtime.mjs # Indexer + query runtime (zero deps)
└── references/ └── references/
├── schema.md # Full table schema + advanced API reference ├── schema.md # Full table schema + advanced API reference
├── query-patterns.md # Copyable retrieval recipes ├── query-patterns.md # Copyable retrieval recipes
├── retrieval-semantics.md # Query design frame for retrieval semantics
├── recap-patterns.md # Compatibility pointer to references/recap/overview.md
├── recap-writing.md # Compatibility pointer to per-card recap writing docs
├── recap/
│ ├── overview.md
│ ├── pattern1-cover.md
│ ├── writing1-cover.md
│ ├── pattern2-thinking.md
│ ├── writing2-thinking.md
│ ├── pattern3-vibe.md
│ ├── writing3-vibe.md
│ ├── pattern4-workflow.md
│ ├── writing4-workflow.md
│ ├── pattern5-closing.md
│ └── writing5-closing.md
└── pitfalls.md # Scope, FTS, ordering, and compactness traps └── pitfalls.md # Scope, FTS, ordering, and compactness traps
``` ```
## Implementation Notes ## Implementation Notes
The index rebuilds incrementally — only new or modified JSONL files are re-parsed. The index rebuilds incrementally — only new or modified JSONL files are re-parsed.
When the optional app is running, it is the active indexer: it watches Claude
project files, builds in a worker thread, writes `__app_heartbeat__` plus
`__app_last_successful_build__` into `index_state`, and the skill-side lazy
build skips work only while both markers are fresh.
Zero npm dependencies. Uses Node 22's built-in node:sqlite with FTS5. The entire runtime is ~400 lines. Zero npm dependencies. Uses Node 22's built-in node:sqlite with FTS5. The entire runtime is ~400 lines.
+102 -17
View File
@@ -44,8 +44,8 @@ Custom query:
3. Parse JSON stdout and answer with concise evidence. 3. Parse JSON stdout and answer with concise evidence.
The query file runs inside `(async () => { ... })()`. Use `return` to emit JSON. The query file runs inside `(async () => { ... })()`. Use `return` to emit JSON.
Query scripts are read-only: `remember()` is not available, and `sql()` only Query scripts are read-only: `remember()` and `forget()` are not available, and
accepts read-only SELECT/WITH queries. `sql()` only accepts read-only SELECT/WITH queries.
## Default First Pass ## Default First Pass
@@ -72,12 +72,40 @@ Use `sql()` only as an escalation path for exact joins, aggregations, or schema
questions that helpers cannot express cleanly. Do not use raw SQL as a generic questions that helpers cannot express cleanly. Do not use raw SQL as a generic
fallback for broad retrieval. fallback for broad retrieval.
## Intent Routing
Obelisk supports a small intent prefix layer after `/obelisk`. This is for
output intent, not retrieval architecture.
| Intent | Description | Reference |
|---|---|---|
| `recap [target]` | Generate weekly/monthly recap card content for app handoff or share-style output. | `references/recap/overview.md` |
Routing rules:
1. If the first word is `recap`, read `references/recap/overview.md` before the
first query. Everything after `recap` is the recap target.
Common app-generated prompts include `/obelisk recap this week`,
`/obelisk recap last week`, `/obelisk recap this month`, and
`/obelisk recap last month`; interpret these as natural period targets
relative to the current date and timezone.
2. `recap` does not create a separate retrieval layer. It still uses
`overview()`, `memories()`, helpers, and `sql()` only when needed.
3. Follow the overview's card-by-card sequence. Each card has its own retrieval
pattern and writing file; retrieve that card's evidence, read that card's
writing file, update the JSON, then move to the next card. Do not preload all
recap references before the current card is written.
4. If the first word is not `recap`, do not load
`references/recap/overview.md`. Continue with Query Routing below. Do not
infer recap from broad requests for weekly/monthly summaries, charts,
rankings, shareable cards, or playlist-style metaphors.
## Query Routing ## Query Routing
Before writing a query, classify the task. Progressive disclosure is useful, but Before writing a query, classify the task. Progressive disclosure is useful, but
skipping the relevant reference usually costs extra query rounds. skipping the relevant reference usually costs extra query rounds.
- Read `references/query-patterns.md` before the first query for broad synthesis, progress summaries, design history, weekly/monthly reviews, or questions that ask what the user did, learned, decided, tried, or abandoned. Start from the first-pass or one-shot synthesis pattern, then run a faceted detail pass if needed. - Read `references/query-patterns.md` before the first query for broad synthesis, progress summaries, design history, ordinary weekly/monthly reviews, or questions that ask what the user did, learned, decided, tried, or abandoned. Start from the first-pass or one-shot synthesis pattern, then run a faceted detail pass if needed.
- Read `references/retrieval-semantics.md` before multi-step retrieval, scoped project/file/session searches, or synthesis/conclusion/history questions. It defines the query design frame. - Read `references/retrieval-semantics.md` before multi-step retrieval, scoped project/file/session searches, or synthesis/conclusion/history questions. It defines the query design frame.
- Read `references/schema.md` before raw `sql()` unless the needed table/column relationship is already explicit here. Do this before running the SQL, not after a missing-column error. Do not start with raw SQL for broad synthesis unless helpers cannot express the needed aggregation or join. - Read `references/schema.md` before raw `sql()` unless the needed table/column relationship is already explicit here. Do this before running the SQL, not after a missing-column error. Do not start with raw SQL for broad synthesis unless helpers cannot express the needed aggregation or join.
- Read `references/pitfalls.md` after an error or when helper fields, FTS syntax, aliases, or row shapes are unclear. - Read `references/pitfalls.md` after an error or when helper fields, FTS syntax, aliases, or row shapes are unclear.
@@ -95,7 +123,7 @@ messages.
Returns: Returns:
```js ```js
[{ message: { uuid, text, role, timestamp, model, cwd }, [{ message: { uuid, text, content_type, is_meta, role, timestamp, model, cwd },
session: { id, title, project, started_at }, session: { id, title, project, started_at },
rank, rank,
context }] context }]
@@ -105,7 +133,22 @@ Returns:
timestamp. It is not the parent chain. Use `context(uuid)` or `trace(uuid)` for timestamp. It is not the parent chain. Use `context(uuid)` or `trace(uuid)` for
causal/parent-chain context. causal/parent-chain context.
Opts: `{ limit, sessionId, project, after, before, cwd }`. Use `message.content_type` to keep evidence boundaries intact:
`text` is user/assistant visible language, `thinking` is trace/debug material,
`tool_use` marks a tool-call message whose details live in `tool_calls`, and
`tool_result` marks a tool-result message whose details live in `tool_results`.
`unknown` is a conservative fallback. Do not treat `thinking` as a user-visible
assistant conclusion. Real user input is `type='user'` plus `content_type='text'`;
do not invent a separate `user_message` content type.
Use `message.is_meta` to separate transcript control-plane material from
conversation evidence. `is_meta=1` marks injected caveats, command envelopes, or
other messages that entered the transcript as user-role content but should not
be treated as the user's request by default. `search()` and `thread()` omit meta
messages unless `includeMeta: true` is passed; `context()` and `trace()` preserve
the original chain and expose `is_meta` on rows.
Opts: `{ limit, sessionId, project, after, before, cwd, includeMeta }`.
`project` is a SQL `LIKE` filter over `sessions.project`, not an exact project `project` is a SQL `LIKE` filter over `sessions.project`, not an exact project
identity. Results are already ordered by FTS5 rank; lower rank sorts earlier. identity. Results are already ordered by FTS5 rank; lower rank sorts earlier.
@@ -129,7 +172,9 @@ Read-only SQL SELECT/WITH with `?` placeholders. Returns array rows. SQL is an
escape hatch for exact structured joins and aggregations after the helper-first escape hatch for exact structured joins and aggregations after the helper-first
surface is insufficient; it is not the default retrieval entry point. surface is insufficient; it is not the default retrieval entry point.
Before writing non-trivial SQL, read `references/schema.md`. Common safe joins: Before writing non-trivial SQL, read `references/schema.md`. The executable DDL
lives in `scripts/schema.sql`; use the reference for query semantics and the SQL
file for schema-source alignment. Common safe joins:
- `tool_calls` does not have timestamps. Join `messages m ON m.uuid = tc.message_uuid`. - `tool_calls` does not have timestamps. Join `messages m ON m.uuid = tc.message_uuid`.
- `tool_results` does not have timestamps. Join `messages m ON m.uuid = tr.message_uuid`. - `tool_results` does not have timestamps. Join `messages m ON m.uuid = tr.message_uuid`.
@@ -159,9 +204,9 @@ tiny sample before relying on less common filters.
- `fileHistory(filePath, opts?)` -- Read/Edit/Write tool calls for a file, oldest first; includes many `Read` rows. - `fileHistory(filePath, opts?)` -- Read/Edit/Write tool calls for a file, oldest first; includes many `Read` rows.
- `failures(opts?)` -- failed tool results with tool/session context, newest first. - `failures(opts?)` -- failed tool results with tool/session context, newest first.
- `trace(uuid)` -- parent chain from root to message. - `trace(uuid)` -- parent chain from root to message.
- `thread(sessionId)` -- full session messages; last resort only. - `thread(sessionId, opts?)` -- session messages ordered by timestamp, omitting meta messages by default. Pass `{ includeMeta: true }` when investigating injected context or command envelopes.
- `raw(uuid, opts?)` -- windowed access to the original JSONL line. - `raw(uuid, opts?)` -- windowed access to the original JSONL line.
- `memories(opts?)` -- recall memory layer, newest first. opts: `{ query, project, sessionId, sessions, after, before, branch, limit }`. `query` filters summary/path by English terms. Returns registered memory records (id, path, summary, project, session_id, created_at). Read the file at `path` for full content. - `memories(opts?)` -- recall memory layer. opts: `{ query, project, sessionId, sessions, after, before, branch, limit }`. Without `query`, returns active memory records newest first. With `query`, searches `summary`/`path` through safe FTS5 tokenization and returns `rank`; lower rank sorts earlier. Records may include nullable JSON `anchors` for explicit recall surfaces such as files. Read the file at `path` for full content.
## Retrieval Contract ## Retrieval Contract
@@ -173,7 +218,8 @@ Keep queries scoped, bounded, and structural.
- Plan Before Probe: for conclusion, broad history, failure investigation, or file evolution, write a bounded retrieval script instead of spending turns on intermediate results. - Plan Before Probe: for conclusion, broad history, failure investigation, or file evolution, write a bounded retrieval script instead of spending turns on intermediate results.
- Structure Before Text: compute counts, joins, grouping, dedupe, and projection in SQL or JS; keep runtime JSON compact, ideally under 10k-12k chars for synthesis tasks. - Structure Before Text: compute counts, joins, grouping, dedupe, and projection in SQL or JS; keep runtime JSON compact, ideally under 10k-12k chars for synthesis tasks.
- Evidence Before Conclusion: return compact evidence with stable IDs (`session_id`, `uuid`, `tool_call_id`, `run_id`, `agent_id`) and short snippets, then synthesize in the final answer. - Evidence Before Conclusion: return compact evidence with stable IDs (`session_id`, `uuid`, `tool_call_id`, `run_id`, `agent_id`) and short snippets, then synthesize in the final answer.
- Persist Durable Conclusions: after answering, if retrieval produced a durable conclusion that future sessions are likely to reuse and `memories()` does not already cover it, explicitly offer to write a memory. Keep the offer brief. Do not write the markdown file or run `--remember` until the user approves. - Exclude Meta By Default: `is_meta=1` rows are injected/control-plane transcript material. Helpers hide them by default; raw SQL for ordinary conversation evidence should include `COALESCE(m.is_meta,0)=0` unless meta rows are the investigation target.
- Persist Durable Conclusions: after answering, if retrieval produced a durable conclusion that future sessions are likely to reuse and `memories()` does not already cover it, explicitly offer to write a memory. Keep the offer brief. Do not write the markdown file or run `--attune` until the user approves.
If field, context, ordering, FTS, or helper semantics affect the query, read If field, context, ordering, FTS, or helper semantics affect the query, read
`references/retrieval-semantics.md` before coding. If a query errors, read `references/retrieval-semantics.md` before coding. If a query errors, read
@@ -196,9 +242,13 @@ obvious CJK text in memory queries and summaries as a guardrail.
**Recall:** query `memories({ query: 'English topic terms', project: '...' })` **Recall:** query `memories({ query: 'English topic terms', project: '...' })`
to find prior conclusions relevant to the current task. Translate non-English to find prior conclusions relevant to the current task. Translate non-English
user requests into concise English query terms before calling `memories()`. Like user requests into concise English query terms before calling `memories()`.
other list helpers, passing a string is treated as `sessionId`, and passing a Memory recall uses safe FTS5 tokenization over `summary` and `path`, so
number is treated as `limit`. Read the file at `path` for full content. hyphens/punctuation are tokenized instead of causing raw `MATCH` syntax errors.
Like other list helpers, passing a string is treated as `sessionId`, and passing
a number is treated as `limit`. Read the file at `path` for full content.
`memories()` returns active memories only. An archived memory is
management/audit data, not recall data.
Good memory candidates include design decisions, project conventions, abandoned Good memory candidates include design decisions, project conventions, abandoned
alternatives, repeated failure causes, workflow patterns, and conclusions alternatives, repeated failure causes, workflow patterns, and conclusions
@@ -206,6 +256,14 @@ synthesized across multiple raw evidence points. Do not propose memory for
one-off lookups, uncertain findings, or conclusions already covered by existing one-off lookups, uncertain findings, or conclusions already covered by existing
memories. memories.
**Mutation approvals:** judging whether to use a memory in the current answer is
an agent decision and does not require approval. Persistent memory changes do.
If the user explicitly says a memory is wrong, outdated, should be forgotten, or
should now say something else, that request is the approval to archive or update
the exact matching memory. Do not ask for a second confirmation unless multiple
memories could match. If you notice a possible conflict yourself, explain it
briefly and ask before changing memory state.
**Writing memories:** after a retrieval produces a conclusion worth persisting, **Writing memories:** after a retrieval produces a conclusion worth persisting,
propose writing a memory file. The user must approve. Flow: propose writing a memory file. The user must approve. Flow:
@@ -218,6 +276,7 @@ return remember({
session_id: 'current-session-id', session_id: 'current-session-id',
message_start: 'uuid-of-first-relevant-msg', message_start: 'uuid-of-first-relevant-msg',
message_end: 'uuid-of-last-relevant-msg', message_end: 'uuid-of-last-relevant-msg',
anchors: [{ kind: 'file', path: 'src/path/to/file.ts' }],
summary: 'Detailed summary: what was decided, why, what alternatives were considered, and what constraints drove the choice.' summary: 'Detailed summary: what was decided, why, what alternatives were considered, and what constraints drove the choice.'
}) })
``` ```
@@ -225,17 +284,21 @@ return remember({
Run the registration script with: Run the registration script with:
```bash ```bash
node $SKILL_DIR/scripts/runtime.mjs --remember /tmp/register-memory.mjs node $SKILL_DIR/scripts/runtime.mjs --attune /tmp/register-memory.mjs
``` ```
`--remember` exposes only `remember()`. It does not expose `search()`, `sql()`, `--attune` exposes only memory mutation helpers: `remember()` and `forget()`.
`memories()`, or other retrieval helpers. If you need source IDs, find them It does not expose `search()`, `sql()`, `memories()`, or other retrieval
first with a normal `--query` script. helpers. If you need source IDs or memory IDs, find them first with a normal
`--query` script.
`remember()` validates that `path` already exists and points to a file. Relative `remember()` validates that `path` already exists and points to a file. Relative
paths are resolved against the source session's `project_path` when paths are resolved against the source session's `project_path` when
`session_id` is provided, then stored as normalized absolute paths. Prefer `session_id` is provided, then stored as normalized absolute paths. Prefer
project-relative paths such as `.obelisk/memories/...` plus `session_id`. project-relative paths such as `.obelisk/memories/...` plus `session_id`.
Optional `anchors` must be an array of objects and is stored as nullable JSON
text. Use it only for explicit recall surfaces, such as files associated with
the memory.
`summary` must be English and detailed enough that `memories()` results alone `summary` must be English and detailed enough that `memories()` results alone
can judge relevance without reading the file. Include the decision, the can judge relevance without reading the file. Include the decision, the
@@ -244,7 +307,29 @@ reasoning, and the key constraints — not just a title.
The `message_start`/`message_end` range marks where in the conversation this The `message_start`/`message_end` range marks where in the conversation this
conclusion was drawn. Use it later to trace back to the original evidence. conclusion was drawn. Use it later to trace back to the original evidence.
Memory records survive index rebuilds. They are never auto-deleted. **Forgetting memories:** if the user says a memory is outdated, wrong, or should
be forgotten, use normal recall first to identify the exact memory ID. If there
is exactly one clear candidate, the user's request is approval to archive it. If
multiple memories could match, ask which one to forget. Then run an `--attune`
script:
```js
return forget({
id: 'mem-id-to-delete',
reason: 'Outdated by newer project guidance.',
});
```
`forget()` archives the memory record by setting `deleted_at` and
`deleted_reason`. It removes the record from active recall but does not delete
the markdown file. Memory records survive index rebuilds and are never changed
automatically.
**Updating memories:** updating memory is one user-approved operation:
archive the old memory with `forget()`, then write and register a replacement
markdown memory with `remember()`. If the user explicitly corrected the memory,
that correction is approval for the combined archive-plus-write flow. If you
discovered the mismatch yourself, ask first.
## Minimal Patterns ## Minimal Patterns
+59 -5
View File
@@ -1,7 +1,7 @@
# Obelisk Query Patterns # Obelisk Query Patterns
These are copyable CodeAct patterns for `runtime.mjs --query` scripts, plus one These are copyable CodeAct patterns for `runtime.mjs --query` scripts plus
`--remember` registration pattern. They are not new APIs. Adapt them to the `--attune` memory mutation patterns. They are not new APIs. Adapt them to the
user's scope and return compact evidence. user's scope and return compact evidence.
Read this before the first query for broad synthesis, progress summaries, Read this before the first query for broad synthesis, progress summaries,
@@ -43,14 +43,17 @@ return {
memories: map.current_project.memories.map(m => ({ memories: map.current_project.memories.map(m => ({
id: m.id, id: m.id,
path: m.path, path: m.path,
anchors: m.anchors,
summary: m.summary?.slice(0, 240), summary: m.summary?.slice(0, 240),
})), })),
}, },
prior_memories: memories({ ...scoped, query: topic, limit: 5 }).map(m => ({ prior_memories: memories({ ...scoped, query: topic, limit: 5 }).map(m => ({
id: m.id, id: m.id,
path: m.path, path: m.path,
anchors: m.anchors,
session_id: m.session_id, session_id: m.session_id,
created_at: m.created_at, created_at: m.created_at,
rank: m.rank,
summary: m.summary?.slice(0, 260), summary: m.summary?.slice(0, 260),
})), })),
session_evidence: search(topic.replace(/[-_]/g, ' '), { ...scoped, limit: 8 }) session_evidence: search(topic.replace(/[-_]/g, ' '), { ...scoped, limit: 8 })
@@ -88,6 +91,7 @@ return {
memories: map.current_project.memories.map(m => ({ memories: map.current_project.memories.map(m => ({
id: m.id, id: m.id,
path: m.path, path: m.path,
anchors: m.anchors,
summary: m.summary?.slice(0, 240), summary: m.summary?.slice(0, 240),
})), })),
}, },
@@ -138,11 +142,13 @@ const prior_memories = memories({
}).map(m => ({ }).map(m => ({
id: m.id, id: m.id,
path: m.path, path: m.path,
anchors: m.anchors,
session_id: m.session_id, session_id: m.session_id,
message_start: m.message_start, message_start: m.message_start,
message_end: m.message_end, message_end: m.message_end,
created_at: m.created_at, created_at: m.created_at,
summary: m.summary?.slice(0, 260), summary: m.summary?.slice(0, 260),
rank: m.rank,
})); }));
const session_evidence = search(ftsTopic, { project, limit: 8 }) const session_evidence = search(ftsTopic, { project, limit: 8 })
@@ -167,14 +173,14 @@ return {
}; };
``` ```
## Register Approved Memory ## Attune Approved Memory
Use this only after the user approves writing memory and the markdown file Use this only after the user approves writing memory and the markdown file
already exists. `remember()` validates the file and stores a normalized absolute already exists. `remember()` validates the file and stores a normalized absolute
path, so keep the script small and return the registered record. path, so keep the script small and return the registered record.
Run this script with `runtime.mjs --remember <script>`. The `--remember` runtime Run this script with `runtime.mjs --attune <script>`. The `--attune` runtime
exposes only `remember()`, not retrieval helpers. exposes only `remember()` and `forget()`, not retrieval helpers.
```js ```js
return remember({ return remember({
@@ -182,6 +188,7 @@ return remember({
session_id: 'source-session-id', session_id: 'source-session-id',
message_start: 'first-message-uuid', message_start: 'first-message-uuid',
message_end: 'last-message-uuid', message_end: 'last-message-uuid',
anchors: [{ kind: 'file', path: 'SKILL.md' }],
summary: [ summary: [
'Decision: Obelisk uses one user-facing entry that queries both memory and raw sessions.', 'Decision: Obelisk uses one user-facing entry that queries both memory and raw sessions.',
'Memory records are prior notes and must be identified naturally when they influence an answer.', 'Memory records are prior notes and must be identified naturally when they influence an answer.',
@@ -190,6 +197,53 @@ return remember({
}); });
``` ```
## Forget Approved Memory
Use this only after the user asks to archive an outdated or wrong memory. Identify
the exact memory ID in a normal `--query` script first. If one candidate clearly
matches the user's request, that request is approval to archive it; if several
candidates match, ask which one to forget.
Run the mutation with `runtime.mjs --attune <script>`:
```js
return forget({
id: 'mem-id-to-delete',
reason: 'Outdated by newer project guidance.',
});
```
`forget()` archives the record. Active recall through `memories()` will omit it,
and the markdown file at `path` is left in place.
## Update Approved Memory
Use this when the user explicitly corrects an existing memory, or after the
agent proposes a replacement and the user approves. An update is one combined
operation: archive the old record and register the replacement markdown file.
The new markdown file must already exist before running `--attune`.
```js
const archived = forget({
id: 'old-memory-id',
reason: 'Replaced by updated memory from the current session.',
});
const created = remember({
path: '.obelisk/memories/updated-memory.md',
session_id: 'current-session-id',
message_start: 'first-message-uuid',
message_end: 'last-message-uuid',
anchors: [{ kind: 'file', path: 'src/path/to/file.ts' }],
summary: 'Updated summary: concise English retrieval surface for the replacement memory.',
});
return { archived, created };
```
If the agent only suspects a memory is stale, do not run this pattern yet.
Answer from current evidence and ask whether to archive or replace the memory.
## One-Shot Retrieval For Synthesis ## One-Shot Retrieval For Synthesis
Use this for conclusion, broad history, failure investigation, or file evolution Use this for conclusion, broad history, failure investigation, or file evolution
+9
View File
@@ -0,0 +1,9 @@
# Obelisk Recap Retrieval Patterns
Compatibility pointer.
The `/obelisk recap` flow now starts at `references/recap/overview.md`.
Do not use this as an all-in-one retrieval document. The current flow is
card-by-card: read the overview, then for each card read its `patternN-*.md`,
retrieve that card's evidence, read its `writingN-*.md`, and update the JSON
before moving on.
+8
View File
@@ -0,0 +1,8 @@
# Obelisk Recap Writing
Compatibility pointer.
The `/obelisk recap` writing contract now lives in the per-card writing files
under `references/recap/`, coordinated by `references/recap/overview.md`.
Read the overview first. Then use each per-card writing file immediately after
that card's retrieval pattern, rather than loading one large writing prompt.
+74
View File
@@ -0,0 +1,74 @@
# Obelisk Recap Overview
Use this only when the first word after `/obelisk` is `recap`. Everything after
`recap` is the target period or style hint.
## Highest Priority: Phase Loop
This workflow is sequential. Do not preload all recap files. Do not gather all
evidence first and write all cards at the end.
Follow this loop exactly:
1. Resolve the target period from the user's phrase.
2. Run only a tiny orientation pass such as `overview({ limit: 6 })`.
3. For Card 1, read `pattern1-cover.md`.
4. Retrieve only Card 1 evidence.
5. Read `writing1-cover.md`.
6. Update/write the JSON for Card 1 now.
7. Only after the JSON is updated, move to Card 2 and repeat.
Card order:
| card | retrieve | write |
|---|---|---|
| 1 cover | `pattern1-cover.md` | `writing1-cover.md` |
| 2 thinking | `pattern2-thinking.md` | `writing2-thinking.md` |
| 3 vibe | `pattern3-vibe.md` | `writing3-vibe.md` |
| 4 workflow | `pattern4-workflow.md` | `writing4-workflow.md` |
| 5 closing | `pattern5-closing.md` | `writing5-closing.md` |
The per-card files own retrieval details, JSON field duties, and card-specific
taste. Do not move those concerns back into this file.
## Period Targets
- `this week`, `last week`: calendar week in the user's runtime timezone.
- `this month`, `last month`: calendar month in the user's runtime timezone.
Do not infer timezone from examples, UTC suffixes, or file timestamps when
runtime/session timezone is available.
## Overall Deck Taste
This is a Spotify Wrapped-like set of personal share cards: concise, designed,
slightly playful, and built to make the user's work feel seen.
Do not criticize the user. Do not scold, diagnose, rank their personality, or
turn friction into a performance review.
Use designed English chrome where it feels like card UI: week/month labels,
archetype labels, compact stats, verdict seals, and signoffs. Preserve the
user's own language for prompts, quotes, catchphrases, and reactions. This is
not a translation task.
The deck should feel like a small artifact from someone who noticed the week,
not a report generated from a database.
## Archetypes
Choose one dominant archetype from the period's dominant attention, not from the
current recap-generation session.
| archetype | when it fits | tone baseline |
|---|---|---|
| `architect` | structure, boundaries, schema, systems | matter-of-fact structural pride |
| `debugger` | symptoms, false positives, root-cause loops | wry and bug-comfortable |
| `shipper` | dense implementation cadence | energetic but not breathless |
| `curator` | organization, memory, docs, refinement | reflective and precise |
| `director` | workflows, subagents, orchestration | observant from a slight remove |
| `cartographer` | moving boundaries and redrawing maps | patient and surveyor-like |
| `wanderer` | many projects without one center | gentle, exploratory |
If two fit, pick the one that describes what the user spent more thinking time
on, not what shipped.
+36
View File
@@ -0,0 +1,36 @@
# Card 1 Cover Retrieval
Goal: choose the recap's dominant claim, persona, activity shape, and compact
footer. The cover is not a topic inventory; it is one glanceable claim about
what the period felt like.
Use the period from `overview.md`. Start from `overview({ limit: 6 })`, then
look at in-period sessions, summaries, memories, and any obvious project scope.
If the user asked for a project, keep that scope; otherwise prefer the current
project only when the evidence makes it the clear center.
Prefer helpers first. If you need custom SQL for activity, token/message counts,
or source-session scope, read `references/schema.md` before writing the SQL.
Retrieve:
- dominant claim: one thing that defined the period, supported by raw evidence;
- persona: which archetype best matches the user's attention;
- source sessions and memories used by this cover claim;
- activity: weekly day intensities or monthly day intensities when supported;
- footer: compact public metric such as sessions, messages, or tokens.
Avoid:
- a claim that lists three topics;
- an archetype chosen from the recap-generation session itself;
- footer caveats like excluded projects, exact SQL filters, or long session names;
- making the cover a workflow metric when the week was really about a decision.
Read this card's writing file immediately after the cover evidence is stable:
`references/recap/writing1-cover.md`. Then update the JSON fields
`period`, `source`, `metrics`, `persona`, and the first `cards[]` entry.
Do not read `pattern2-thinking.md` until this JSON update is done.
Stop when the cover has one evidence-backed dominant claim, one chosen persona,
one metric scope, and at least one `evidence` anchor.
+33
View File
@@ -0,0 +1,33 @@
# Card 2 Thinking Retrieval
Goal: find turning points. This card is not a project timeline and not an implementation log. It is the record of what changed in the user's mind.
Retrieve 3-6 turns. A turn needs both sides:
- the user question, friction, doubt, or request that started the turn;
- the later decision, reframing, finding, or constraint that replaced the earlier
state.
Useful searches:
- user questions in the period: "为什么", "是不是", "怎么", "我觉得", "不应该";
- places where the user corrected the direction and then approved a new frame;
- summaries that name decisions, followed by `context()` or `thread()` for the
user's actual words;
- memory records only as hints; raw session evidence must provide the prompt and
turn.
Prefer helpers first. If you need custom SQL for message windows or user-turn
counts, read `references/schema.md` before writing the SQL.
Do not use workflow names, feature names, or agent task labels as prompts when
the user had their own wording. Do not use counts, "5 rounds", "13 agents", or
implementation effort as turns unless that count is the turn itself.
Read this card's writing file immediately after the turns are chosen:
`references/recap/writing2-thinking.md`. Then update the JSON `thinking_path`
card and add evidence for each item.
Do not read `pattern3-vibe.md` until this JSON update is done.
Stop when each item has a source-language prompt label, a short changed-state
prompt, turn, and an `evidence` anchor.
+30
View File
@@ -0,0 +1,30 @@
# Card 3 Vibe Retrieval
Goal: find small human signals in visible user messages. Vibe is not a correction log, not bracketed runtime text, and not a psychological profile.
Look for:
- catchphrases and repeated tiny reactions;
- unusually blunt praise or rejection;
- late-night disbelief, jokes, or rituals;
- one quotable sentence that captures the period's character.
Only count visible user messages. Helper APIs omit meta by default, but custom
SQL for phrase counts must filter user text with `COALESCE(m.is_meta,0)=0` and
`m.content_type='text'`. Do not count tool results, injected command envelopes,
UI labels, or bracketed runtime strings.
Useful retrieval:
- targeted phrase counts after you notice a likely catchphrase;
- `thread(sessionId)` around high-energy moments;
- `search()` for exact phrases, then `context()` for timing;
- a bounded SQL count only after reading `references/schema.md`.
Read this card's writing file immediately after you have the small user signals:
`references/recap/writing3-vibe.md`. Then update the JSON `vibe` card and add
evidence for every quote, count, and timestamp.
Do not read `pattern4-workflow.md` until this JSON update is done.
Stop when every observation is either exact user words or a tiny label backed by
exact user words.
+41
View File
@@ -0,0 +1,41 @@
# Card 4 Workflow Retrieval
Goal: find actual workflow runs and how the user received them. Card 4 is about
orchestration as experienced by the user, not an agent performance table.
Workflow rows have their own `workflows.timestamp`. During this card's retrieval,
call `workflows({ project: projectLike, after, before })` before concluding the
period had zero workflows.
Prefer helpers first. If you need custom SQL for workflow joins, timestamps, or
message reactions, read `references/schema.md` before writing the SQL.
Do not derive workflow counts only from `sessions({ after, before })`: long
sessions can start before the period and still contain workflow runs inside the
period. Do not scope workflow lookup by exact `project_path`; nested cwd values
can belong to the same Claude project slug.
For each candidate workflow:
- get the actual workflow_name from `workflows()` or `workflowTree()`;
- collect run id, timestamp, project, agent count, and compact result for stats
and evidence only;
- search the parent session for the user message immediately following the workflow completion;
- use that user reaction as `items[].reaction`.
Rank rows by the strength of the user reaction, not by agent count, workflow
size, duration, or implementation importance. A small workflow with "完美" is a
better row than a large workflow with no visible response.
Do not use architecture topics, memory-system milestones, app modules, or recap
feature work as workflow rows unless they are actual workflow_name values.
Do not make a row for a workflow with no visible user reaction; keep it only in
`stats`, `metrics`, or `evidence`.
Read this card's writing file immediately after workflow evidence is stable:
`references/recap/writing4-workflow.md`. Then update the JSON `workflow` card,
top-level workflow metrics, and source session ids for workflow evidence.
Do not read `pattern5-closing.md` until this JSON update is done.
Stop when every displayed row has an actual workflow name and a visible user
reaction.
+35
View File
@@ -0,0 +1,35 @@
# Card 5 Closing Retrieval
Goal: close with a small personal receipt. Use the same period and source scope
as the recap, or explicitly record a wider metric in `evidence`.
Retrieve:
- one consistent metric that can stand alone, such as streak, active days,
sessions, messages, or workflows;
- one or two compact receipts;
- most said phrase, only if a real repeated user phrase is supported;
- signoff material from the period's mood, not a second summary.
For phrase counts, count only non-meta visible user text. For streaks and active
days, define whether the scope is all Obelisk data, the current project, or the
selected evidence sessions. Keep the scope consistent with the cover footer
unless the evidence explicitly says otherwise.
Prefer helpers first. If you need custom SQL for phrase counts, active days, or
streaks, read `references/schema.md` before writing the SQL.
Avoid:
- naked numbers without units;
- project report bullets;
- internal session names;
- token audits;
- slogans, advice, or next-action commands.
Read this card's writing file immediately after the closing receipt is chosen:
`references/recap/writing5-closing.md`. Then update the JSON `closing` card and
add evidence for counts and phrases.
This is the final card; save the completed JSON before replying.
Stop when the closing can end the deck without explaining the whole week again.
+83
View File
@@ -0,0 +1,83 @@
# Card 1 Cover Writing
The cover should be readable in one glance: badge, persona, one plain claim,
activity, footer. Before writing, say the claim to the user in a chat bubble.
If it sounds like a topic list or report heading, shrink it.
## Mock taste anchor
```json
{
"type": "cover",
"badge": "Week 24",
"title": "The Architect",
"claim": "从零设计了一个完整的 memory 系统。",
"activity": [0.85, 0.92, 0.72, 0.45, 0.88, 0.30, 0],
"footer": "12 sessions · 2.4M tokens"
}
```
This works because `从零设计了一个完整的 memory 系统。` is one plain claim, in
the user's language, and can be read in one breath. `The Architect` is English
chrome; it gives the card a designed surface without translating the user's
actual work.
## JSON Shape
```ts
type CoverCard = {
type: "cover";
badge: string;
title: string;
claim: string;
activity: number[];
footer: string;
evidence_refs?: string[];
};
```
Field duties:
- `badge`: compact period chrome, such as `Week 24`.
- `title`: persona label, usually `The Architect`, `The Debugger`, etc.
- `claim`: one plain claim; not a topic list, project inventory, colon-led
tagline, or clever English that hides the user's language.
- `activity`: period intensity values from retrieval.
- `footer`: public metric line with no internal filter notes.
After writing, check that `persona.claim` and `cover.claim` tell the same
story, and attach `evidence_refs` to the claim or metric if it is surprisingly
specific.
## First JSON Write
After Card 1, create or update the recap JSON file. Do this before reading
`pattern2-thinking.md`.
Use this top-level shape:
```ts
type Recap = {
schema_version: "obelisk.recap.v1";
kind: "weekly" | "monthly";
generated_at: string;
period: { label: string; start: string; end: string; timezone: string };
source: { project?: string; session_ids: string[]; memory_ids?: string[] };
metrics: {
sessions?: number;
messages?: number;
tokens?: number;
active_days?: number[];
streak_days?: number;
workflows?: number;
workflow_agents?: number;
corrections?: number;
};
persona: { archetype: string; title: string; claim: string; tone: string };
cards: [CoverCard, { type: "thinking_path" }, { type: "vibe" }, { type: "workflow" }, { type: "closing" }];
evidence?: Array<{ id: string; session_id?: string; message_uuid?: string; memory_id?: string; summary?: string }>;
};
```
For app handoff, write JSON under `~/.obelisk/recap/`. Weekly filenames are
`recap-{YYYY}-W{WW}.json`; monthly filenames are `recap-{YYYY}-{MM}.json`.
+49
View File
@@ -0,0 +1,49 @@
# Card 2 Thinking Writing
Thinking Path should feel like a few bends in the user's reasoning, not a
weekly changelog. Before writing, test each row by asking: "what changed here?"
## Mock taste anchor
```json
{
"type": "thinking_path",
"title": "Five questions, five turns.",
"items": [
{ "day": "Mon", "prompt": "为什么要把 session 编译成 wiki", "turn": "raw SQLite, no wiki" },
{ "day": "Tue", "prompt": "buildWhere 是什么", "turn": "unified filter opts, not DSL" },
{ "day": "Wed", "prompt": "failures() 90% 误报", "turn": "is_error in JSONL" },
{ "day": "Thu", "prompt": "memory 层需要清理机制吗", "turn": "soft-delete, human-only" },
{ "day": "Fri", "prompt": "热力图不选中默认显示本月", "turn": "GitHub-style activity timeline" }
]
}
```
The prompts stay close to the user's words. Each turn is a short decision
fragment, not a full explanation.
## JSON Shape
```ts
type ThinkingPathCard = {
type: "thinking_path";
title: string;
items: Array<{
day: string;
prompt: string;
turn: string;
evidence_refs?: string[];
}>;
};
```
Field duties:
- `title`: designed deck line, not `本周路径`, not a research-paper heading.
- `prompt`: user's compact question, friction, or task. Use source language.
- `turn`: short decision fragment, finding, or shift; usually under 10 words.
Compact English fragments are allowed when they work as designed chrome.
After writing, remove any row whose prompt is a workflow name or whose turn
describes implementation rather than changed thinking.
Update the JSON now before reading `pattern3-vibe.md`.
+73
View File
@@ -0,0 +1,73 @@
# Card 3 Vibe Writing
Vibe is affectionate observation. It should make the user recognize themselves
without feeling evaluated. Before writing, remove anything that reads like a
correction audit, behavior label, diagnosis, or complaint ledger.
## Mock taste anchor
```json
{
"type": "vibe",
"title": "A short character study.",
"voice_lines": [
{ "label": "catchphrase", "text": "这太丑了", "count": 4 },
{ "label": "highest praise", "text": "可以" },
{ "label": "late night", "text": "你在干什么", "time": "02:47 AM" }
],
"meter": {
"label": "patience",
"value": 0.78,
"caption": "saint"
},
"quote": {
"text": "若无必要,勿增实体。",
"caption": "your most philosophical moment"
}
}
```
The humor comes from exact small lines. `可以` is funnier and truer than
"approval signal".
## JSON Shape
```ts
type VibeCard = {
type: "vibe";
title: string;
voice_lines: Array<{
label: string;
text: string;
count?: number;
time?: string;
evidence_refs?: string[];
}>;
meter?: {
label: string;
value: number;
caption: string;
};
quote?: {
text: string;
caption?: string;
evidence_refs?: string[];
};
};
```
Field duties:
- `title`: light character-study line, not a scorecard.
- `voice_lines[].text`: exact user words; no paraphrase, translation,
ellipsized half-quote, meta text, or correction log.
- `voice_lines[].label`: designed chrome can be English; the quoted user text
stays in source language.
- `meter`: meter is not a diagnosis. Keep the caption one or two words and
affectionate, never punitive.
- `quote.text`: one exact user phrase or sentence.
Do not use `[Request interrupted by user]`, tool output, injected context, or
UI status text as vibe. After writing, verify every `voice_lines[].text` and
`quote.text` can be traced to a non-meta user message.
Update the JSON now before reading `pattern4-workflow.md`.
+68
View File
@@ -0,0 +1,68 @@
# Card 4 Workflow Writing
Workflow is the orchestration card. It should show the strongest few workflow
runs and the user's reaction to them. Before writing, remove any row whose
reaction is not traceable to a visible user reaction.
## Mock taste anchor
```json
{
"type": "workflow",
"title": "Three workflows. Forty-two agents.",
"deck": "你召唤了机器军团。结果各有不同。",
"stats": "3 workflows · 42 agents",
"items": [
{ "name": "hono-plugin-review", "reaction": "完美" },
{ "name": "vue-migration", "reaction": "你这页面完全和之前的不一样…" },
{ "name": "split-render-js", "reaction": "可以" }
],
"verdict": "Mostly tolerated."
}
```
The row reactions are user reactions. The title carries the metric; the verdict
is a small English seal.
## JSON Shape
```ts
type WorkflowCard = {
type: "workflow";
title: string;
deck?: string;
stats?: string;
items: Array<{
name: string;
reaction: string;
evidence_refs?: string[];
}>;
verdict: string;
};
```
Field duties:
- `title`: human story line or compact metric line.
- `deck`: optional second line; do not repeat stats mechanically.
- `stats`: compact count line.
- `items[].name`: actual workflow name, command name, or run-id prefix.
- `items[].reaction`: exact or lightly trimmed user reaction. Preserve source
language. No feature description, implementation summary, agent count,
duration, "framework switch", "modularization", "theming landed", or other
internal progress label.
- `verdict`: compact seal based on the row reactions, often 3-6 words.
Agent counts belong only in `title` or `stats`, never in `items[].reaction`.
These row values are invalid because they are implementation labels, not user
reactions:
- `13 agents, the big build` is invalid.
- `9 agents, framework switch` is invalid.
- `6 agents, modularization` is invalid.
- `theming landed` is invalid.
If no user reaction exists, omit the row rather than write an implementation result.
After writing, check that every row name maps to retrieval evidence and every
reaction can be read as quoted user verdict text.
Update the JSON now before reading `pattern5-closing.md`.
+54
View File
@@ -0,0 +1,54 @@
# Card 5 Closing Writing
Closing is a receipt, not a second summary. Before writing, read the headline
alone. If it does not mean anything without the rest of the card, add the unit
or choose a better line.
## Mock taste anchor
```json
{
"type": "closing",
"headline": "19 days",
"receipts": ["847 messages exchanged", "12 corrections · 47 approvals"],
"most_said_phrase": "好的开始做吧",
"signoff": "See you next week."
}
```
This works because `19 days` has a unit, the receipts feel like a small receipt,
and `See you next week.` is a quiet goodbye instead of a slogan.
## JSON Shape
```ts
type ClosingCard = {
type: "closing";
headline: string;
receipts: string[];
most_said_phrase?: string;
signoff: string;
evidence_refs?: string[];
};
```
Field duties:
- `headline`: compact stat or phrase with its unit; not a naked number.
- `receipts`: at most two `receipts`, compact and personal.
- `most_said_phrase`: complete phrase the user actually said, or omit it.
- `signoff`: short and earned; quiet goodbye, not advice or a brand slogan.
English signoff chrome such as `See you next week.` is allowed.
After writing, remove internal scope notes from visible fields and put them in
`evidence`. The final card should feel like the deck ending, not the report
continuing.
Final save rules:
- The file contains only the JSON object: no Markdown fence, no prose.
- Keep exactly five cards in this order: cover, thinking_path, vibe, workflow,
closing.
- Keep private SQL, raw tool output, secrets, long paths, and source caveats out
of visible card text; put traceability in `evidence`.
- After saving, reply briefly with the saved path and important evidence caveats.
+34 -4
View File
@@ -78,7 +78,8 @@ the needed join, grouping, or exact schema-level check better than helpers.
Ordering and context are semantic: Ordering and context are semantic:
- `sessions()`, `memories()`, `summaries()`, `workflows()`, and `failures()` are newest first. - `sessions()`, `summaries()`, `workflows()`, and `failures()` are newest first.
- `memories()` without `query` is newest first; `memories({ query })` is FTS-ranked over memory `summary`/`path`, with lower rank sorting earlier.
- `fileHistory()` is oldest first. - `fileHistory()` is oldest first.
- `search().context` is temporal neighbors in one session, not causal context. - `search().context` is temporal neighbors in one session, not causal context.
- `context(uuid)` and `trace(uuid)` are for parent-chain/causal expansion. - `context(uuid)` and `trace(uuid)` are for parent-chain/causal expansion.
@@ -97,19 +98,44 @@ For semantic questions, build a task-local evidence view:
{ {
query_plan: { mode, scope, facets, limits }, query_plan: { mode, scope, facets, limits },
prior_memories: [ prior_memories: [
{ id, path, session_id, created_at, summary } { id, path, anchors, session_id, created_at, summary }
], ],
evidence: [ evidence: [
{ type, id, session_id, timestamp, facet, snippet } { type, id, session_id, timestamp, content_type, is_meta, facet, snippet }
], ],
omitted: 0 omitted: 0
} }
``` ```
For message evidence, preserve `content_type` when projecting snippets.
`text` can support user-visible claims; `thinking` is only trace/debug context;
`tool_use` means follow `tool_calls` for structured details; `tool_result`
means follow `tool_results` for structured output. Mixed or unfamiliar message
surfaces remain `unknown`.
Preserve `is_meta` separately from `content_type`. Default message evidence
should exclude `is_meta=1` rows because they are transcript control-plane
content, not ordinary user intent or assistant conclusions. Include them only
when investigating injected caveats, command envelopes, or transcript structure.
When writing raw SQL for ordinary conversation evidence, add
`COALESCE(m.is_meta,0)=0` to message filters unless meta rows are the subject of
the investigation.
Memory recall is English-indexed: translate non-English user requests into Memory recall is English-indexed: translate non-English user requests into
concise English query terms before calling `memories({ query })`. Memory concise English query terms before calling `memories({ query })`. Memory
summaries registered with `remember()` are also English, regardless of the summaries registered with `remember()` are also English, regardless of the
conversation language. conversation language.
`memories({ query })` uses safe FTS5 tokenization over memory `summary` and
`path`, so hyphens and punctuation do not need raw `MATCH` escaping.
`memories()` returns active memories only. For raw SQL memory recall, include
`deleted_at IS NULL`; archived memory records are management/audit data.
The agent may decide whether to use, ignore, or verify a recalled memory for the
current answer without user approval because no persistent state changes. If a
user explicitly says a memory is wrong, outdated, should be forgotten, or should
be replaced, that request is approval to mutate the exact matching memory. If
the agent discovers the conflict without an explicit user request, it should
answer from current evidence and ask before archiving or replacing the memory.
Then synthesize the conclusion in the final answer. Do not pretend the raw Then synthesize the conclusion in the final answer. Do not pretend the raw
evidence view is itself a stored Obelisk entity. evidence view is itself a stored Obelisk entity.
@@ -121,7 +147,11 @@ project conventions, abandoned alternatives, repeated failure causes, workflow
patterns, and conclusions synthesized across multiple raw evidence points. Do patterns, and conclusions synthesized across multiple raw evidence points. Do
not propose memory for one-off lookups, uncertain findings, or duplicate not propose memory for one-off lookups, uncertain findings, or duplicate
coverage. The offer is only a proposal: write the markdown file and run coverage. The offer is only a proposal: write the markdown file and run
`--remember` only after user approval. `--attune` only after user approval.
Memory updates are archive-plus-write, not in-place edits: run `forget()` on the
old record and `remember()` the replacement markdown file under the same user
approval.
## Text Search Semantics ## Text Search Semantics
+199 -27
View File
@@ -3,6 +3,9 @@
Advanced reference for the obelisk database. Advanced reference for the obelisk database.
Read this when `search()`, `context()`, or `sql()` are not enough. Read this when `search()`, `context()`, or `sql()` are not enough.
Executable schema source: `scripts/schema.sql`. This document explains that
contract for agents and humans; it is not the runtime source of truth.
--- ---
## 1. Database Schema ## 1. Database Schema
@@ -41,6 +44,8 @@ CREATE TABLE messages (
timestamp TEXT, -- ISO 8601 timestamp TEXT, -- ISO 8601
role TEXT, -- "user" or "assistant" (from message payload) role TEXT, -- "user" or "assistant" (from message payload)
text TEXT, -- extracted text content (thinking + text blocks, truncated to 10k chars) text TEXT, -- extracted text content (thinking + text blocks, truncated to 10k chars)
content_type TEXT, -- "text", "thinking", "tool_use", "tool_result", or "unknown"
is_meta INTEGER DEFAULT 0, -- 1 for injected/control-plane transcript messages
model TEXT, -- model name (e.g. "claude-opus-4-6-20250529"), NULL for user messages model TEXT, -- model name (e.g. "claude-opus-4-6-20250529"), NULL for user messages
is_sidechain INTEGER DEFAULT 0, -- 1 if this message is on a sidechain (retry/branch) is_sidechain INTEGER DEFAULT 0, -- 1 if this message is on a sidechain (retry/branch)
agent_id TEXT, -- subagent or workflow agent UUID (NULL for main conversation) agent_id TEXT, -- subagent or workflow agent UUID (NULL for main conversation)
@@ -54,6 +59,22 @@ CREATE TABLE messages (
Indexes: `idx_messages_session(session_id)`, `idx_messages_agent(agent_id)`, `idx_messages_ts(session_id, timestamp)`. Indexes: `idx_messages_session(session_id)`, `idx_messages_agent(agent_id)`, `idx_messages_ts(session_id, timestamp)`.
`content_type` preserves the top-level Claude Code content block shape for the
message row. Treat `text` as user/assistant visible language, `thinking` as
trace/debug material, and `tool_use` as a marker that the assistant message
contains tool calls. `tool_result` marks a tool-result message, but the
structured payload remains in `tool_results`. Tool-call details remain in
`tool_calls`. Messages whose top-level content is not one of these four raw
message surfaces are `unknown`. Real user input is represented by `type='user'`
and `content_type='text'`, not by a separate `user_message` content type.
`is_meta` marks transcript control-plane content: injected caveats, command
envelopes such as `<command-name>/exit</command-name>`, and similar messages
that may appear as user-role text but are not ordinary user intent. It is
separate from `type`, `role`, and `content_type`. Default helpers hide meta
messages from ordinary recall; use `includeMeta: true` or explicit SQL when
investigating injected context, command messages, or transcript structure.
### messages_fts ### messages_fts
FTS5 virtual table for full-text search over message text. FTS5 virtual table for full-text search over message text.
@@ -68,7 +89,28 @@ CREATE VIRTUAL TABLE messages_fts USING fts5(
); );
``` ```
Queried via `MATCH` syntax. Rebuilt on each index pass. Queried via `MATCH` syntax. The table is kept in sync by the `messages_fts_*`
triggers above; rebuild it manually only when repairing FTS state.
### memories_fts
FTS5 virtual table for ranked memory recall over registered memory summaries
and paths.
```sql
CREATE VIRTUAL TABLE memories_fts USING fts5(
id UNINDEXED, -- memory record ID, carried for inspection
path, -- searchable memory file path
summary, -- searchable compact memory summary
content=memories,
content_rowid=rowid,
tokenize='unicode61 remove_diacritics 1'
);
```
`memories({ query })` queries this table with safe tokenization and joins back to
`memories`, omitting archived rows. It is rebuilt during index finalization;
`remember()` also inserts the new memory row into FTS immediately.
### tool_calls ### tool_calls
@@ -177,6 +219,14 @@ CREATE TABLE index_state (
); );
``` ```
Sentinel rows use synthetic `jsonl_path` keys:
`__last_build__` stores the last completed build time, `__app_heartbeat__`
stores the optional app indexer's liveness heartbeat,
`__app_last_successful_build__` stores the app's last successful index build,
`__indexer_owner_app__` marks app ownership, and `__last_source_mtime__` records
the newest indexed source mtime. Skill-side lazy builds skip work only while the
app heartbeat and app successful-build marker are both fresh.
### memories ### memories
Human-approved markdown memory records registered in Obelisk. The markdown Human-approved markdown memory records registered in Obelisk. The markdown
@@ -191,14 +241,22 @@ CREATE TABLE memories (
message_start TEXT, -- first relevant message UUID, if known message_start TEXT, -- first relevant message UUID, if known
message_end TEXT, -- last relevant message UUID, if known message_end TEXT, -- last relevant message UUID, if known
path TEXT, -- normalized absolute markdown memory file path path TEXT, -- normalized absolute markdown memory file path
anchors TEXT, -- optional JSON array of recall anchors
summary TEXT, -- retrieval summary of the memory summary TEXT, -- retrieval summary of the memory
created_at TEXT -- ISO 8601 registration time created_at TEXT, -- ISO 8601 registration time
deleted_at TEXT, -- ISO 8601 archive time, if forgotten
deleted_reason TEXT -- human/agent deletion reason, if forgotten
); );
``` ```
Indexes: `idx_memories_project(project)`, Indexes: `idx_memories_project(project)`,
`idx_memories_session(session_id)`, `idx_memories_created(created_at)`. `idx_memories_session(session_id)`, `idx_memories_created(created_at)`.
Active memory means `deleted_at IS NULL`. Recall helpers return active memories
only. Archived memories are management/audit data, not recall data. Query recall
uses `memories_fts` joined back to `memories`; when using raw SQL for memory
recall, include `deleted_at IS NULL`.
### Key Relationships ### Key Relationships
``` ```
@@ -221,8 +279,8 @@ workflows.run_id <-- workflow_agents.run_id
## 2. Query API Reference ## 2. Query API Reference
Read helpers are available as globals inside `--query` scripts. Memory write Read helpers are available as globals inside `--query` scripts. Memory mutation
helpers are available only inside `--remember` scripts. Scripts run in an async helpers are available only inside `--attune` scripts. Scripts run in an async
IIFE with a 30-second timeout. IIFE with a 30-second timeout.
### Simple Layer ### Simple Layer
@@ -240,6 +298,7 @@ Full-text search across all message text using FTS5.
| `opts.after` | `string` | ISO 8601 lower bound on timestamp | | `opts.after` | `string` | ISO 8601 lower bound on timestamp |
| `opts.before` | `string` | ISO 8601 upper bound on timestamp | | `opts.before` | `string` | ISO 8601 upper bound on timestamp |
| `opts.cwd` | `string` | Filter by working directory (supports LIKE) | | `opts.cwd` | `string` | Filter by working directory (supports LIKE) |
| `opts.includeMeta` | `boolean` | Include injected/control-plane messages (default `false`) |
**Scope note:** `sessions.project` is the stored Claude Code project slug, **Scope note:** `sessions.project` is the stored Claude Code project slug,
`sessions.project_path` is the absolute session path derived from message `cwd` `sessions.project_path` is the absolute session path derived from message `cwd`
@@ -248,15 +307,22 @@ Helper `project` filters are fuzzy `LIKE` filters over `sessions.project`. For
exact project membership, use `sql()` with `s.project = ?` or exact project membership, use `sql()` with `s.project = ?` or
`s.project_path = ?`. `s.project_path = ?`.
**Returns:** `Array<{ message, session, rank, context }>` where `context` is the **Returns:** `Array<{ message, session, rank, context }>` where `message`
6 nearest messages by timestamp in the same session. It is temporal neighbor includes `{ uuid, text, content_type, is_meta, role, timestamp, model, cwd }`
context, not a parent chain. `rank` is the FTS5 relevance score used by and `context` is the 6 nearest non-meta messages by timestamp in the same
`ORDER BY rank`; lower values sort earlier, so treat the returned order as the session unless `includeMeta: true` is passed. It is temporal neighbor context,
relevance order unless you are deliberately using FTS5 ranking details. not a parent chain. `rank` is the FTS5 relevance score used by `ORDER BY rank`;
lower values sort earlier, so treat the returned order as the relevance order
unless you are deliberately using FTS5 ranking details.
```js ```js
const hits = search('MCTS exploration'); const hits = search('MCTS exploration');
return hits.map(h => ({ title: h.session.title, text: h.message.text?.slice(0, 200) })); return hits.map(h => ({
title: h.session.title,
content_type: h.message.content_type,
is_meta: h.message.is_meta,
text: h.message.text?.slice(0, 200),
}));
``` ```
#### `context(uuid)` #### `context(uuid)`
@@ -285,8 +351,8 @@ Read-only SQL with parameterized bindings. Returns an array of row objects.
**Returns:** `Array<Object>` -- each row as `{ column: value }`. **Returns:** `Array<Object>` -- each row as `{ column: value }`.
Write statements are rejected. Use `--remember` and `remember()` for memory Write statements are rejected. Use `--attune` with `remember()` or `forget()`
registration after user approval. for memory mutation after user approval.
```js ```js
const rows = sql('SELECT id, title FROM sessions WHERE project = ? ORDER BY ended_at DESC LIMIT 5', 'Users-tomiya-Code-quiet-zero'); const rows = sql('SELECT id, title FROM sessions WHERE project = ? ORDER BY ended_at DESC LIMIT 5', 'Users-tomiya-Code-quiet-zero');
@@ -306,9 +372,11 @@ const chain = trace('some-uuid');
return chain.map(m => ({ role: m.role, text: m.text?.slice(0, 100) })); return chain.map(m => ({ role: m.role, text: m.text?.slice(0, 100) }));
``` ```
#### `thread(sessionId)` #### `thread(sessionId, opts?)`
All messages in a session, ordered by timestamp. Session messages ordered by timestamp. Meta messages are omitted by default;
pass `{ includeMeta: true }` to include injected caveats, command envelopes, and
other control-plane transcript rows.
**Returns:** `Array<message>`. **Returns:** `Array<message>`.
@@ -317,6 +385,31 @@ const msgs = thread('session-uuid');
return { count: msgs.length, first: msgs[0]?.text?.slice(0, 100) }; return { count: msgs.length, first: msgs[0]?.text?.slice(0, 100) };
``` ```
#### `raw(uuid, opts?)`
Windowed access to the original JSONL line for a message. Use this when indexed
text, tool inputs, or tool results were truncated and you need the raw source.
It resolves main-session, subagent, and workflow-agent JSONL paths from the
indexed message metadata.
| Param | Type | Description |
|-------|------|-------------|
| `uuid` | `string` | Message UUID |
| `opts.offset` | `number` | Character offset into the JSONL line (default 0) |
| `opts.limit` | `number` | Max characters to return (default 10000) |
**Returns:** `{ text, totalLength, offset, limit, hasMore }` or `null` if the
source line cannot be found.
```js
const line = raw('message-uuid', { offset: 0, limit: 4000 });
return {
preview: line?.text,
has_more: line?.hasMore,
total: line?.totalLength,
};
```
#### `subagents(opts?)` #### `subagents(opts?)`
All subagent spawns, with message counts. For backward compatibility, passing a string is treated as `sessionId`. All subagent spawns, with message counts. For backward compatibility, passing a string is treated as `sessionId`.
@@ -417,6 +510,34 @@ const last5 = recent(5);
return last5.map(s => ({ title: s.title, project: s.project_path, ended: s.ended_at })); return last5.map(s => ({ title: s.title, project: s.project_path, ended: s.ended_at }));
``` ```
#### `summaries(opts?)`
Session summary rows, newest first. For backward compatibility, passing a
string is treated as `sessionId`, and passing a number is treated as `limit`.
| Param | Type | Description |
|-------|------|-------------|
| `opts.sessionId` | `string` | Restrict to one session |
| `opts.sessions` | `string[]` | Restrict to a set of session IDs |
| `opts.project` | `string` | SQL `LIKE` pattern over `sessions.project` |
| `opts.after` | `string` | ISO 8601 lower bound on summary timestamp |
| `opts.before` | `string` | ISO 8601 upper bound on summary timestamp |
| `opts.branch` | `string` | Filter by source session git branch (exact match) |
| `opts.limit` | `number` | Max results (default 100) |
**Returns:** `Array<summary_row & { session_title, project }>` ordered by
`timestamp` descending.
```js
const rows = summaries({ project: '%quiet-zero%', limit: 5 });
return rows.map(s => ({
session: s.session_title,
source: s.source,
summary: s.content?.slice(0, 240),
timestamp: s.timestamp,
}));
```
#### `overview(opts?)` #### `overview(opts?)`
Compact orientation map for choosing the next retrieval scope. It is not an Compact orientation map for choosing the next retrieval scope. It is not an
@@ -457,7 +578,7 @@ from `process.cwd()` against `sessions.project_path`, then from exact
], ],
memory_total, memory_total,
memories: [ memories: [
{ id, path, summary, session_id, project, created_at } { id, path, anchors, summary, session_id, project, created_at }
] ]
} | null, } | null,
projects: [ projects: [
@@ -491,6 +612,7 @@ return {
memories: map.current_project?.memories.map(m => ({ memories: map.current_project?.memories.map(m => ({
id: m.id, id: m.id,
path: m.path, path: m.path,
anchors: m.anchors,
summary: m.summary, summary: m.summary,
})), })),
}; };
@@ -522,12 +644,12 @@ return qz.map(s => ({ title: s.title, branch: s.git_branch, ended: s.ended_at })
#### `memories(opts?)` #### `memories(opts?)`
Registered markdown memory records. Like other list helpers, passing a string Active registered markdown memory records. Like other list helpers, passing a
is treated as `sessionId`, and passing a number is treated as `limit`. string is treated as `sessionId`, and passing a number is treated as `limit`.
| Param | Type | Description | | Param | Type | Description |
|-------|------|-------------| |-------|------|-------------|
| `opts.query` | `string` | English term filter over `summary` and `path`; hyphens/underscores are treated as spaces | | `opts.query` | `string` | English FTS recall query over `summary` and `path`; hyphens/underscores/punctuation are safely tokenized |
| `opts.project` | `string` | SQL `LIKE` pattern over `memories.project` | | `opts.project` | `string` | SQL `LIKE` pattern over `memories.project` |
| `opts.sessionId` | `string` | Restrict to one source session | | `opts.sessionId` | `string` | Restrict to one source session |
| `opts.sessions` | `string[]` | Restrict to a set of source session IDs | | `opts.sessions` | `string[]` | Restrict to a set of source session IDs |
@@ -536,9 +658,13 @@ is treated as `sessionId`, and passing a number is treated as `limit`.
| `opts.branch` | `string` | Filter by source session git branch (exact match) | | `opts.branch` | `string` | Filter by source session git branch (exact match) |
| `opts.limit` | `number` | Max results (default 50) | | `opts.limit` | `number` | Max results (default 50) |
**Returns:** `Array<memory_row>` ordered by `created_at` descending. **Returns:** `Array<memory_row & { rank?: number }>` with archived memories
omitted. Without `query`, results are ordered by `created_at` descending. With
`query`, results are ordered by FTS rank first, then `created_at` descending;
lower rank sorts earlier.
`query` is a lightweight English term filter, not FTS5 ranking. Translate `query` uses safe FTS5 tokenization rather than raw `MATCH`, so punctuation-only
queries return no rows instead of broadening into all memories. Translate
non-English user requests into concise English query terms before calling non-English user requests into concise English query terms before calling
`memories()`. Use it to avoid pulling all recent memories, then read the `memories()`. Use it to avoid pulling all recent memories, then read the
markdown file at `path` when a memory looks relevant. The runtime rejects markdown file at `path` when a memory looks relevant. The runtime rejects
@@ -553,6 +679,7 @@ const prior = memories({
return prior.map(m => ({ return prior.map(m => ({
id: m.id, id: m.id,
path: m.path, path: m.path,
anchors: m.anchors,
session_id: m.session_id, session_id: m.session_id,
summary: m.summary?.slice(0, 240), summary: m.summary?.slice(0, 240),
})); }));
@@ -562,11 +689,11 @@ return prior.map(m => ({
Register a human-approved markdown memory file. This is a write helper, not a Register a human-approved markdown memory file. This is a write helper, not a
recall helper; use it only after the user has approved writing memory. It is recall helper; use it only after the user has approved writing memory. It is
available only in scripts run with `runtime.mjs --remember`. available only in scripts run with `runtime.mjs --attune`.
`--remember` exposes only `remember()`, not `search()`, `sql()`, `memories()`, `--attune` exposes only `remember()` and `forget()`, not `search()`, `sql()`,
or other retrieval helpers. If source IDs are unknown, find them first with a `memories()`, or other retrieval helpers. If source IDs or memory IDs are
normal `--query` script. unknown, find them first with a normal `--query` script.
| Param | Type | Description | | Param | Type | Description |
|-------|------|-------------| |-------|------|-------------|
@@ -576,12 +703,14 @@ normal `--query` script.
| `record.message_start` | `string` | First relevant source message UUID, if known | | `record.message_start` | `string` | First relevant source message UUID, if known |
| `record.message_end` | `string` | Last relevant source message UUID, if known | | `record.message_end` | `string` | Last relevant source message UUID, if known |
| `record.project` | `string` | Project slug override. Defaults from `sessions.project` for `session_id` | | `record.project` | `string` | Project slug override. Defaults from `sessions.project` for `session_id` |
| `record.anchors` | `array` or JSON `string` | Optional recall anchors stored as JSON text. Expected shape is an array of objects, such as `{ kind: 'file', path: 'src/index/builder.ts' }` |
`remember()` validates that `path` exists and is a regular file, and rejects `remember()` validates that `path` exists and is a regular file, and rejects
obvious CJK text in `summary`. It stores the normalized absolute path in obvious CJK text in `summary`. It stores the normalized absolute path in
`memories.path`. `memories.path`. `anchors` is nullable; omit it or pass an empty array when the
memory has no explicit file or object anchors.
**Returns:** `{ id, path, project, created_at }`. **Returns:** `{ id, path, project, anchors, created_at }`.
```js ```js
return remember({ return remember({
@@ -589,10 +718,53 @@ return remember({
session_id: 'source-session-id', session_id: 'source-session-id',
message_start: 'first-message-uuid', message_start: 'first-message-uuid',
message_end: 'last-message-uuid', message_end: 'last-message-uuid',
anchors: [{ kind: 'file', path: 'src/index/builder.ts' }],
summary: 'Decision: keep Obelisk as one user-facing entry that queries both memory and raw session evidence. Memory is prior notes, not final authority.', summary: 'Decision: keep Obelisk as one user-facing entry that queries both memory and raw session evidence. Memory is prior notes, not final authority.',
}); });
``` ```
#### `forget(record)`
Archive a human-approved memory record. Use it when the user says a memory is
outdated, wrong, or should be forgotten. It is available only in scripts run
with `runtime.mjs --attune`.
`forget()` requires a precise memory ID. Do not pass a query string and let the
helper choose. If the ID is unknown, first use a normal `--query` script with
`memories()` to identify candidates. If exactly one candidate clearly matches
the user's request, the request is approval to archive it. If multiple memories
could match, ask the user which one to forget.
| Param | Type | Description |
|-------|------|-------------|
| `record.id` | `string` | Memory record ID to archive |
| `record.reason` | `string` | Required reason for audit and future management views |
`forget()` sets `deleted_at` and `deleted_reason`. It does not delete the
markdown file at `path`. Active recall helpers omit archived memories.
**Returns:** `{ id, deleted_at, deleted_reason }`, or the same fields plus
`already_deleted: true` if the record had already been forgotten.
```js
return forget({
id: 'mem-20260610-example',
reason: 'Outdated by newer project guidance.',
});
```
#### Memory Mutation Approval
Agents may decide whether to use, ignore, or verify memory in a single answer
without approval. Approval is required only for persistent memory mutations.
When the user explicitly says a memory is wrong, outdated, should be forgotten,
or should say something else, that utterance is approval to mutate the exact
matching memory. If multiple memories could match, ask the user to choose.
Updating is not an in-place edit. Archive the old record with `forget()`, then
write and register a replacement markdown file with `remember()` under the same
approval.
--- ---
## 3. Common Query Patterns ## 3. Common Query Patterns
@@ -667,7 +839,7 @@ const wfs = workflows();
for (const wf of wfs.slice(0, 3)) { for (const wf of wfs.slice(0, 3)) {
const tree = workflowTree(wf.run_id); const tree = workflowTree(wf.run_id);
wf.agent_details = tree?.agents.map(a => ({ wf.agent_details = tree?.agents.map(a => ({
type: a.agent_type, desc: a.description, msgs: a.messages.length, type: a.agent_type, desc: a.description, msgs: a.messageCount,
})); }));
} }
return wfs.slice(0, 3); return wfs.slice(0, 3);
+44 -55
View File
@@ -8,69 +8,34 @@ const { DatabaseSync } = require('node:sqlite');
const CLAUDE_DIR = path.join(os.homedir(), '.claude'); const CLAUDE_DIR = path.join(os.homedir(), '.claude');
const DB_PATH = path.join(CLAUDE_DIR, 'obelisk.sqlite'); const DB_PATH = path.join(CLAUDE_DIR, 'obelisk.sqlite');
const TEXT_LIMIT = 10000; const TEXT_LIMIT = 10000;
const SCHEMA = fs.readFileSync(new URL('./schema.sql', import.meta.url), 'utf8');
const SCHEMA = `
CREATE TABLE IF NOT EXISTS sessions (
id TEXT PRIMARY KEY, title TEXT, project TEXT, project_path TEXT,
started_at TEXT, ended_at TEXT, git_branch TEXT, version TEXT,
message_count INTEGER DEFAULT 0, jsonl_path TEXT);
CREATE TABLE IF NOT EXISTS messages (
uuid TEXT PRIMARY KEY, session_id TEXT, type TEXT, parent_uuid TEXT,
timestamp TEXT, role TEXT, text TEXT, model TEXT,
is_sidechain INTEGER DEFAULT 0, agent_id TEXT,
input_tokens INTEGER, output_tokens INTEGER,
cwd TEXT, skill TEXT, turn_duration_ms INTEGER);
CREATE TABLE IF NOT EXISTS tool_calls (
id TEXT PRIMARY KEY, message_uuid TEXT, session_id TEXT,
name TEXT, input_json TEXT, file_path TEXT);
CREATE TABLE IF NOT EXISTS tool_results (
tool_use_id TEXT PRIMARY KEY, message_uuid TEXT, session_id TEXT,
content TEXT, file_path TEXT, is_error INTEGER DEFAULT 0);
CREATE TABLE IF NOT EXISTS subagents (
agent_id TEXT PRIMARY KEY, session_id TEXT, parent_tool_use_id TEXT,
agent_type TEXT, description TEXT, duration_ms INTEGER, total_tokens INTEGER);
CREATE TABLE IF NOT EXISTS workflows (
run_id TEXT PRIMARY KEY, session_id TEXT, task_id TEXT,
script TEXT, result_json TEXT, timestamp TEXT, agent_count INTEGER DEFAULT 0,
duration_ms INTEGER, total_tokens INTEGER, status TEXT, workflow_name TEXT);
CREATE TABLE IF NOT EXISTS workflow_agents (
agent_id TEXT PRIMARY KEY, run_id TEXT, session_id TEXT,
agent_type TEXT, description TEXT,
phase TEXT, label TEXT, model TEXT, state TEXT,
duration_ms INTEGER, tokens INTEGER, tool_calls INTEGER);
CREATE TABLE IF NOT EXISTS index_state (
jsonl_path TEXT PRIMARY KEY, mtime REAL, lines_processed INTEGER);
CREATE TABLE IF NOT EXISTS summaries (
id TEXT PRIMARY KEY, session_id TEXT, timestamp TEXT,
source TEXT, content TEXT);
CREATE VIRTUAL TABLE IF NOT EXISTS messages_fts USING fts5(
uuid UNINDEXED, session_id UNINDEXED, text, content=messages, content_rowid=rowid);
CREATE INDEX IF NOT EXISTS idx_messages_session ON messages(session_id);
CREATE INDEX IF NOT EXISTS idx_messages_agent ON messages(agent_id);
CREATE INDEX IF NOT EXISTS idx_messages_ts ON messages(session_id, timestamp);
CREATE INDEX IF NOT EXISTS idx_tc_session_name ON tool_calls(session_id, name);
CREATE INDEX IF NOT EXISTS idx_tc_file ON tool_calls(file_path);
CREATE INDEX IF NOT EXISTS idx_sa_session ON subagents(session_id);
CREATE INDEX IF NOT EXISTS idx_wf_session ON workflows(session_id);
CREATE INDEX IF NOT EXISTS idx_wa_run ON workflow_agents(run_id);
CREATE INDEX IF NOT EXISTS idx_summaries_session ON summaries(session_id);
CREATE TABLE IF NOT EXISTS memories (
id TEXT PRIMARY KEY, session_id TEXT, project TEXT,
message_start TEXT, message_end TEXT,
path TEXT, summary TEXT, created_at TEXT);
CREATE INDEX IF NOT EXISTS idx_memories_project ON memories(project);
CREATE INDEX IF NOT EXISTS idx_memories_session ON memories(session_id);
CREATE INDEX IF NOT EXISTS idx_memories_created ON memories(created_at);
`;
function openDb() { function openDb() {
const db = new DatabaseSync(DB_PATH); const db = new DatabaseSync(DB_PATH);
db.exec('PRAGMA journal_mode=WAL'); db.exec('PRAGMA journal_mode=WAL');
db.exec('PRAGMA synchronous=NORMAL'); db.exec('PRAGMA synchronous=NORMAL');
db.exec(SCHEMA); db.exec(SCHEMA);
migrateDb(db);
return db; return db;
} }
function ensureColumn(db, table, column, definition) {
const columns = db.prepare(`PRAGMA table_info(${table})`).all().map(c => c.name);
if (!columns.includes(column)) db.exec(`ALTER TABLE ${table} ADD COLUMN ${column} ${definition}`);
}
function migrateDb(db) {
ensureColumn(db, 'messages', 'content_type', 'TEXT');
ensureColumn(db, 'messages', 'is_meta', 'INTEGER DEFAULT 0');
ensureColumn(db, 'memories', 'anchors', 'TEXT');
ensureColumn(db, 'memories', 'deleted_at', 'TEXT');
ensureColumn(db, 'memories', 'deleted_reason', 'TEXT');
}
function rebuildMemoryFts(db) {
db.exec("INSERT INTO memories_fts(memories_fts) VALUES('rebuild')");
}
function trunc(s) { function trunc(s) {
return typeof s === 'string' && s.length > TEXT_LIMIT ? s.slice(0, TEXT_LIMIT) : s; return typeof s === 'string' && s.length > TEXT_LIMIT ? s.slice(0, TEXT_LIMIT) : s;
} }
@@ -101,6 +66,30 @@ function extractText(content) {
return parts.length ? trunc(parts.join('\n')) : null; return parts.length ? trunc(parts.join('\n')) : null;
} }
function extractContentType(content) {
if (typeof content === 'string') return 'text';
if (!Array.isArray(content) || !content.length) return 'unknown';
const types = new Set();
let sawUnknown = false;
for (const b of content) {
if (!b || typeof b !== 'object') { sawUnknown = true; continue; }
if (b.type === 'text') types.add('text');
else if (b.type === 'thinking') types.add('thinking');
else if (b.type === 'tool_use') types.add('tool_use');
else if (b.type === 'tool_result') types.add('tool_result');
else sawUnknown = true;
}
return !sawUnknown && types.size === 1 ? [...types][0] : 'unknown';
}
const COMMAND_ENVELOPE_RE = /^\s*(<command-name>[^<]+<\/command-name>|<(?:task-notification|system-reminder)\b|<local-command(?:\b|-))/;
function extractMessageIsMeta(record, text = extractText(record?.message?.content)) {
const msg = record?.message || {};
if (record?.isMeta === true || msg.isMeta === true) return 1;
return typeof text === 'string' && COMMAND_ENVELOPE_RE.test(text) ? 1 : 0;
}
function filePath(name, input) { function filePath(name, input) {
if (!input) return null; if (!input) return null;
return ['Read', 'Edit', 'Write', 'NotebookEdit'].includes(name) ? (input.file_path || null) : null; return ['Read', 'Edit', 'Write', 'NotebookEdit'].includes(name) ? (input.file_path || null) : null;
@@ -129,4 +118,4 @@ function readLines(filePath, callback) {
} }
} }
export { CLAUDE_DIR, DB_PATH, TEXT_LIMIT, openDb, trunc, truncJson, extractText, filePath, isDir, readLines, fs, path, os }; export { CLAUDE_DIR, DB_PATH, TEXT_LIMIT, openDb, rebuildMemoryFts, trunc, truncJson, extractText, extractContentType, extractMessageIsMeta, filePath, isDir, readLines, fs, path, os };
+30 -6
View File
@@ -1,4 +1,4 @@
import { CLAUDE_DIR, openDb, trunc, truncJson, extractText, filePath, isDir, readLines, fs, path } from './db.mjs'; import { CLAUDE_DIR, openDb, rebuildMemoryFts, trunc, truncJson, extractText, extractContentType, extractMessageIsMeta, filePath, isDir, readLines, fs, path } from './db.mjs';
const PROJECTS_DIR = path.join(CLAUDE_DIR, 'projects'); const PROJECTS_DIR = path.join(CLAUDE_DIR, 'projects');
const HISTORY_PATH = path.join(CLAUDE_DIR, 'history.jsonl'); const HISTORY_PATH = path.join(CLAUDE_DIR, 'history.jsonl');
@@ -82,7 +82,7 @@ function indexJsonl(db, fi) {
const ins = { const ins = {
ses: db.prepare('INSERT OR REPLACE INTO sessions (id,title,project,project_path,started_at,ended_at,git_branch,version,message_count,jsonl_path) VALUES (?,?,?,?,?,?,?,?,?,?)'), ses: db.prepare('INSERT OR REPLACE INTO sessions (id,title,project,project_path,started_at,ended_at,git_branch,version,message_count,jsonl_path) VALUES (?,?,?,?,?,?,?,?,?,?)'),
msg: db.prepare('INSERT OR REPLACE INTO messages (uuid,session_id,type,parent_uuid,timestamp,role,text,model,is_sidechain,agent_id,input_tokens,output_tokens,cwd,skill) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?)'), msg: db.prepare('INSERT OR REPLACE INTO messages (uuid,session_id,type,parent_uuid,timestamp,role,text,content_type,is_meta,model,is_sidechain,agent_id,input_tokens,output_tokens,cwd,skill) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)'),
tc: db.prepare('INSERT OR REPLACE INTO tool_calls (id,message_uuid,session_id,name,input_json,file_path) VALUES (?,?,?,?,?,?)'), tc: db.prepare('INSERT OR REPLACE INTO tool_calls (id,message_uuid,session_id,name,input_json,file_path) VALUES (?,?,?,?,?,?)'),
tr: db.prepare('INSERT OR REPLACE INTO tool_results (tool_use_id,message_uuid,session_id,content,file_path,is_error) VALUES (?,?,?,?,?,?)'), tr: db.prepare('INSERT OR REPLACE INTO tool_results (tool_use_id,message_uuid,session_id,content,file_path,is_error) VALUES (?,?,?,?,?,?)'),
sum: db.prepare('INSERT OR REPLACE INTO summaries (id,session_id,timestamp,source,content) VALUES (?,?,?,?,?)'), sum: db.prepare('INSERT OR REPLACE INTO summaries (id,session_id,timestamp,source,content) VALUES (?,?,?,?,?)'),
@@ -129,12 +129,14 @@ function indexJsonl(db, fi) {
const msg = obj.message || {}; const msg = obj.message || {};
const text = extractText(msg.content); const text = extractText(msg.content);
const contentType = extractContentType(msg.content);
const isMeta = extractMessageIsMeta(obj, text);
const usage = msg.usage || {}; const usage = msg.usage || {};
const aid = fi.isSubagent ? fi.agentId : (obj.agentId || null); const aid = fi.isSubagent ? fi.agentId : (obj.agentId || null);
if (obj.uuid) { if (obj.uuid) {
ins.msg.run(obj.uuid, sid, obj.type, obj.parentUuid || null, ts, ins.msg.run(obj.uuid, sid, obj.type, obj.parentUuid || null, ts,
msg.role || obj.type, text, msg.model || null, msg.role || obj.type, text, contentType, isMeta, msg.model || null,
obj.isSidechain ? 1 : 0, aid, usage.input_tokens || null, usage.output_tokens || null, obj.isSidechain ? 1 : 0, aid, usage.input_tokens || null, usage.output_tokens || null,
obj.cwd || null, obj.attributionSkill || null); obj.cwd || null, obj.attributionSkill || null);
} }
@@ -244,12 +246,33 @@ function indexHistory(db) {
} }
const BUILD_DEBOUNCE_MS = 30000; const BUILD_DEBOUNCE_MS = 30000;
const APP_HEARTBEAT_FRESH_MS = 60000;
function shouldSkipBuild(db, { now = Date.now() } = {}) {
const appHeartbeat = db.prepare("SELECT mtime FROM index_state WHERE jsonl_path='__app_heartbeat__'").get();
const appSuccessfulBuild = db.prepare("SELECT mtime FROM index_state WHERE jsonl_path='__app_last_successful_build__'").get();
if (
appHeartbeat && now - appHeartbeat.mtime < APP_HEARTBEAT_FRESH_MS &&
appSuccessfulBuild && now - appSuccessfulBuild.mtime < APP_HEARTBEAT_FRESH_MS
) {
return { skip: true, reason: 'app_successful_build' };
}
const last = db.prepare("SELECT mtime FROM index_state WHERE jsonl_path='__last_build__'").get();
if (last && now - last.mtime < BUILD_DEBOUNCE_MS) {
return { skip: true, reason: 'recent_build' };
}
return { skip: false };
}
function buildIndex({ force = false } = {}) { function buildIndex({ force = false } = {}) {
const db = openDb(); const db = openDb();
if (!force) { if (!force) {
const last = db.prepare("SELECT mtime FROM index_state WHERE jsonl_path='__last_build__'").get(); const skip = shouldSkipBuild(db);
if (last && Date.now() - last.mtime < BUILD_DEBOUNCE_MS) { db.close(); return; } if (skip.skip) { db.close(); return; }
}
if (force) {
db.prepare("DELETE FROM index_state WHERE jsonl_path != '__last_build__'").run();
} }
const files = discoverJsonlFiles(); const files = discoverJsonlFiles();
@@ -270,6 +293,7 @@ function buildIndex({ force = false } = {}) {
refreshSessionProjectPaths(db); refreshSessionProjectPaths(db);
indexHistory(db); indexHistory(db);
db.exec("INSERT INTO messages_fts(messages_fts) VALUES('rebuild')"); db.exec("INSERT INTO messages_fts(messages_fts) VALUES('rebuild')");
rebuildMemoryFts(db);
db.prepare("INSERT OR REPLACE INTO index_state (jsonl_path, mtime, lines_processed) VALUES ('__last_build__', ?, 0)").run(Date.now()); db.prepare("INSERT OR REPLACE INTO index_state (jsonl_path, mtime, lines_processed) VALUES ('__last_build__', ?, 0)").run(Date.now());
db.exec('COMMIT'); db.exec('COMMIT');
} catch (e) { } catch (e) {
@@ -279,4 +303,4 @@ function buildIndex({ force = false } = {}) {
db.close(); db.close();
} }
export { buildIndex, inferProjectPath, refreshSessionProjectPaths }; export { buildIndex, inferProjectPath, refreshSessionProjectPaths, shouldSkipBuild };
+87 -31
View File
@@ -1,4 +1,4 @@
import { openDb, readLines, fs, path } from './db.mjs'; import { readLines, fs, path } from './db.mjs';
function normalizeOpts(optsOrScalar, scalarKey = 'sessionId') { function normalizeOpts(optsOrScalar, scalarKey = 'sessionId') {
if (optsOrScalar == null) return {}; if (optsOrScalar == null) return {};
@@ -45,6 +45,14 @@ function assertEnglishMemoryText(value, label) {
} }
} }
function buildSafeFtsQuery(text) {
const tokens = String(text || '').match(/[\p{Letter}\p{Number}]+/gu) || [];
return tokens
.slice(0, 12)
.map(token => `"${token}"`)
.join(' ');
}
function createQueryApi(db) { function createQueryApi(db) {
const q = (sql, ...p) => { const q = (sql, ...p) => {
assertReadOnlySql(sql); assertReadOnlySql(sql);
@@ -59,7 +67,7 @@ function createQueryApi(db) {
}; };
const search = (text, opts = {}) => { const search = (text, opts = {}) => {
const { limit = 20, sessionId, project, after, before, cwd } = opts; const { limit = 20, sessionId, project, after, before, cwd, includeMeta = false } = opts;
let where = 'WHERE mf.text MATCH ?'; let where = 'WHERE mf.text MATCH ?';
const p = [text]; const p = [text];
if (sessionId) { where += ' AND mf.session_id=?'; p.push(sessionId); } if (sessionId) { where += ' AND mf.session_id=?'; p.push(sessionId); }
@@ -67,19 +75,21 @@ function createQueryApi(db) {
if (after) { where += ' AND m.timestamp>?'; p.push(after); } if (after) { where += ' AND m.timestamp>?'; p.push(after); }
if (before) { where += ' AND m.timestamp<?'; p.push(before); } if (before) { where += ' AND m.timestamp<?'; p.push(before); }
if (cwd) { where += ' AND m.cwd LIKE ?'; p.push(cwd); } if (cwd) { where += ' AND m.cwd LIKE ?'; p.push(cwd); }
if (!includeMeta) where += ' AND COALESCE(m.is_meta,0)=0';
p.push(limit); p.push(limit);
const rows = db.prepare(` const rows = db.prepare(`
SELECT m.uuid,m.session_id,m.text,m.role,m.timestamp,m.model,m.cwd, SELECT m.uuid,m.session_id,m.text,m.content_type,m.is_meta,m.role,m.timestamp,m.model,m.cwd,
s.id as s_id,s.title as s_title,s.project as s_project,s.started_at as s_started, s.id as s_id,s.title as s_title,s.project as s_project,s.started_at as s_started,
rank rank
FROM messages_fts mf JOIN messages m ON m.uuid=mf.uuid LEFT JOIN sessions s ON s.id=m.session_id FROM messages_fts mf JOIN messages m ON m.uuid=mf.uuid LEFT JOIN sessions s ON s.id=m.session_id
${where} ORDER BY rank LIMIT ?`).all(...p); ${where} ORDER BY rank LIMIT ?`).all(...p);
return rows.map(r => { return rows.map(r => {
const metaClause = includeMeta ? '' : 'AND COALESCE(is_meta,0)=0';
const ctx = db.prepare( const ctx = db.prepare(
'SELECT uuid,text,role,timestamp,model FROM messages WHERE session_id=? AND uuid!=? ORDER BY ABS(JULIANDAY(timestamp)-JULIANDAY(?)) LIMIT 6' `SELECT uuid,text,content_type,is_meta,role,timestamp,model FROM messages WHERE session_id=? AND uuid!=? ${metaClause} ORDER BY ABS(JULIANDAY(timestamp)-JULIANDAY(?)) LIMIT 6`
).all(r.session_id, r.uuid, r.timestamp).sort((a,b) => a.timestamp < b.timestamp ? -1 : 1); ).all(r.session_id, r.uuid, r.timestamp).sort((a,b) => a.timestamp < b.timestamp ? -1 : 1);
return { return {
message: { uuid: r.uuid, text: r.text, role: r.role, timestamp: r.timestamp, model: r.model, cwd: r.cwd }, message: { uuid: r.uuid, text: r.text, content_type: r.content_type, is_meta: r.is_meta || 0, role: r.role, timestamp: r.timestamp, model: r.model, cwd: r.cwd },
session: { id: r.s_id, title: r.s_title, project: r.s_project, started_at: r.s_started }, session: { id: r.s_id, title: r.s_title, project: r.s_project, started_at: r.s_started },
rank: r.rank, rank: r.rank,
context: ctx, context: ctx,
@@ -110,7 +120,11 @@ function createQueryApi(db) {
return chain; return chain;
}; };
const thread = (sid) => db.prepare('SELECT * FROM messages WHERE session_id=? ORDER BY timestamp').all(sid); const thread = (sid, opts = {}) => {
const includeMeta = opts?.includeMeta === true;
const metaClause = includeMeta ? '' : 'AND COALESCE(is_meta,0)=0';
return db.prepare(`SELECT * FROM messages WHERE session_id=? ${metaClause} ORDER BY timestamp`).all(sid);
};
const subagents = (optsOrSid) => { const subagents = (optsOrSid) => {
const opts = normalizeOpts(optsOrSid); const opts = normalizeOpts(optsOrSid);
@@ -267,7 +281,7 @@ function createQueryApi(db) {
WITH names AS ( WITH names AS (
SELECT project FROM sessions WHERE project IS NOT NULL GROUP BY project SELECT project FROM sessions WHERE project IS NOT NULL GROUP BY project
UNION UNION
SELECT project FROM memories WHERE project IS NOT NULL GROUP BY project SELECT project FROM memories WHERE project IS NOT NULL AND deleted_at IS NULL GROUP BY project
), ),
session_stats AS ( session_stats AS (
SELECT project, COUNT(*) AS session_count, MAX(COALESCE(ended_at, started_at)) AS last_session_at SELECT project, COUNT(*) AS session_count, MAX(COALESCE(ended_at, started_at)) AS last_session_at
@@ -278,7 +292,7 @@ function createQueryApi(db) {
memory_stats AS ( memory_stats AS (
SELECT project, COUNT(*) AS memory_count, MAX(created_at) AS last_memory_at SELECT project, COUNT(*) AS memory_count, MAX(created_at) AS last_memory_at
FROM memories FROM memories
WHERE project IS NOT NULL WHERE project IS NOT NULL AND deleted_at IS NULL
GROUP BY project GROUP BY project
) )
SELECT SELECT
@@ -322,11 +336,11 @@ function createQueryApi(db) {
ORDER BY COALESCE(ended_at, started_at) DESC ORDER BY COALESCE(ended_at, started_at) DESC
LIMIT ? LIMIT ?
`).all(currentProject.project, sessionLimit); `).all(currentProject.project, sessionLimit);
const memoryTotal = db.prepare('SELECT COUNT(*) AS c FROM memories WHERE project = ?').get(currentProject.project)?.c || 0; const memoryTotal = db.prepare('SELECT COUNT(*) AS c FROM memories WHERE project = ? AND deleted_at IS NULL').get(currentProject.project)?.c || 0;
const memoriesForProject = db.prepare(` const memoriesForProject = db.prepare(`
SELECT id, path, summary, session_id, project, created_at SELECT id, path, anchors, summary, session_id, project, created_at
FROM memories FROM memories
WHERE project = ? WHERE project = ? AND deleted_at IS NULL
ORDER BY created_at DESC ORDER BY created_at DESC
LIMIT ? LIMIT ?
`).all(currentProject.project, memoryLimit); `).all(currentProject.project, memoryLimit);
@@ -345,11 +359,11 @@ function createQueryApi(db) {
FROM ( FROM (
SELECT project FROM sessions WHERE project IS NOT NULL GROUP BY project SELECT project FROM sessions WHERE project IS NOT NULL GROUP BY project
UNION UNION
SELECT project FROM memories WHERE project IS NOT NULL GROUP BY project SELECT project FROM memories WHERE project IS NOT NULL AND deleted_at IS NULL GROUP BY project
) )
`).get()?.c || 0; `).get()?.c || 0;
const totalSessions = db.prepare('SELECT COUNT(*) AS c FROM sessions').get()?.c || 0; const totalSessions = db.prepare('SELECT COUNT(*) AS c FROM sessions').get()?.c || 0;
const totalMemories = db.prepare('SELECT COUNT(*) AS c FROM memories').get()?.c || 0; const totalMemories = db.prepare('SELECT COUNT(*) AS c FROM memories WHERE deleted_at IS NULL').get()?.c || 0;
return { return {
current: { current: {
@@ -422,25 +436,32 @@ function createQueryApi(db) {
timestamp: 'mem.created_at', timestamp: 'mem.created_at',
branch: 's.git_branch', branch: 's.git_branch',
}); });
const terms = String(query || '') let where = baseWhere + ' AND mem.deleted_at IS NULL';
.trim()
.replace(/[-_]/g, ' ')
.split(/\s+/)
.filter(Boolean);
let where = baseWhere;
for (const term of terms) {
where += " AND lower(coalesce(mem.summary,'') || ' ' || coalesce(mem.path,'')) LIKE ?";
params.push(`%${term.toLowerCase()}%`);
}
params.push(limit);
const join = needsJoin ? 'LEFT JOIN sessions s ON s.id=mem.session_id' : ''; const join = needsJoin ? 'LEFT JOIN sessions s ON s.id=mem.session_id' : '';
const hasQuery = String(query || '').trim().length > 0;
const ftsQuery = buildSafeFtsQuery(query);
if (!hasQuery) {
params.push(limit);
return db.prepare(`SELECT mem.* FROM memories mem ${join} WHERE ${where} ORDER BY mem.created_at DESC LIMIT ?`).all(...params); return db.prepare(`SELECT mem.* FROM memories mem ${join} WHERE ${where} ORDER BY mem.created_at DESC LIMIT ?`).all(...params);
}
if (!ftsQuery) return [];
params.unshift(ftsQuery);
params.push(limit);
return db.prepare(`
SELECT mem.*, mf.rank AS rank
FROM memories_fts mf
JOIN memories mem ON mem.rowid = mf.rowid
${join}
WHERE memories_fts MATCH ? AND ${where}
ORDER BY mf.rank, mem.created_at DESC
LIMIT ?
`).all(...params);
}; };
return { sql: q, search, context, trace, thread, subagents, workflows, workflowTree, fileHistory, failures, sessions, recent, summaries, raw, memories, overview }; return { sql: q, search, context, trace, thread, subagents, workflows, workflowTree, fileHistory, failures, sessions, recent, summaries, raw, memories, overview };
} }
function createRememberApi(db) { function createAttuneApi(db) {
const resolveMemoryPath = (memoryPath, sessionId) => { const resolveMemoryPath = (memoryPath, sessionId) => {
let base = null; let base = null;
if (sessionId) { if (sessionId) {
@@ -459,19 +480,54 @@ function createRememberApi(db) {
return resolved; return resolved;
}; };
const remember = ({ path: memoryPath, session_id, message_start, message_end, summary, project }) => { const normalizeAnchors = (anchors) => {
if (anchors == null) return null;
let parsed = anchors;
if (typeof anchors === 'string') {
const trimmed = anchors.trim();
if (!trimmed) return null;
try {
parsed = JSON.parse(trimmed);
} catch {
throw new Error('remember() anchors must be a JSON array');
}
}
if (!Array.isArray(parsed)) throw new Error('remember() anchors must be an array');
for (const anchor of parsed) {
if (!anchor || typeof anchor !== 'object' || Array.isArray(anchor)) {
throw new Error('remember() anchors entries must be objects');
}
}
return parsed.length ? JSON.stringify(parsed) : null;
};
const remember = ({ path: memoryPath, session_id, message_start, message_end, summary, project, anchors }) => {
if (!memoryPath || !summary) throw new Error('remember() requires path and summary'); if (!memoryPath || !summary) throw new Error('remember() requires path and summary');
assertEnglishMemoryText(summary, 'remember() summary'); assertEnglishMemoryText(summary, 'remember() summary');
const normalizedPath = resolveMemoryPath(memoryPath, session_id); const normalizedPath = resolveMemoryPath(memoryPath, session_id);
const normalizedAnchors = normalizeAnchors(anchors);
const id = `mem-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`; const id = `mem-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
const proj = project || db.prepare('SELECT project FROM sessions WHERE id=?').get(session_id)?.project || null; const proj = project || db.prepare('SELECT project FROM sessions WHERE id=?').get(session_id)?.project || null;
const created_at = new Date().toISOString(); const created_at = new Date().toISOString();
db.prepare('INSERT OR REPLACE INTO memories (id, session_id, project, message_start, message_end, path, summary, created_at) VALUES (?,?,?,?,?,?,?,?)').run( db.prepare('INSERT OR REPLACE INTO memories (id, session_id, project, message_start, message_end, path, anchors, summary, created_at) VALUES (?,?,?,?,?,?,?,?,?)').run(
id, session_id || null, proj, message_start || null, message_end || null, normalizedPath, summary, created_at); id, session_id || null, proj, message_start || null, message_end || null, normalizedPath, normalizedAnchors, summary, created_at);
return { id, path: normalizedPath, project: proj, created_at }; return { id, path: normalizedPath, project: proj, anchors: normalizedAnchors, created_at };
}; };
return { remember }; const forget = ({ id, reason }) => {
const deletionReason = String(reason || '').trim();
if (!id || !deletionReason) throw new Error('forget() requires id and reason');
const row = db.prepare('SELECT id, deleted_at, deleted_reason FROM memories WHERE id=?').get(id);
if (!row) throw new Error(`forget() memory not found: ${id}`);
if (row.deleted_at) {
return { id, deleted_at: row.deleted_at, deleted_reason: row.deleted_reason, already_deleted: true };
}
const deleted_at = new Date().toISOString();
db.prepare('UPDATE memories SET deleted_at=?, deleted_reason=? WHERE id=?').run(deleted_at, deletionReason, id);
return { id, deleted_at, deleted_reason: deletionReason };
};
return { remember, forget };
} }
export { createQueryApi, createRememberApi }; export { createQueryApi, createAttuneApi };
+6 -6
View File
@@ -7,7 +7,7 @@ const vm = require('node:vm');
import { DB_PATH, openDb } from './db.mjs'; import { DB_PATH, openDb } from './db.mjs';
import { buildIndex } from './indexer.mjs'; import { buildIndex } from './indexer.mjs';
import { createQueryApi, createRememberApi } from './query.mjs'; import { createQueryApi, createAttuneApi } from './query.mjs';
function executeScript(api, scriptContent) { function executeScript(api, scriptContent) {
const sandbox = { const sandbox = {
@@ -22,8 +22,8 @@ function executeQuery(db, scriptContent) {
return executeScript(createQueryApi(db), scriptContent); return executeScript(createQueryApi(db), scriptContent);
} }
function executeRemember(db, scriptContent) { function executeAttune(db, scriptContent) {
return executeScript(createRememberApi(db), scriptContent); return executeScript(createAttuneApi(db), scriptContent);
} }
function main() { function main() {
@@ -49,16 +49,16 @@ function main() {
.catch(e => { process.stdout.write(JSON.stringify({ error: e.message, stack: e.stack }) + '\n'); db.close(); process.exitCode = 1; }); .catch(e => { process.stdout.write(JSON.stringify({ error: e.message, stack: e.stack }) + '\n'); db.close(); process.exitCode = 1; });
return; return;
} }
if (args[0] === '--remember' && args[1]) { if (args[0] === '--attune' && args[1]) {
buildIndex(); buildIndex();
const db = openDb(); const db = openDb();
const script = fs.readFileSync(path.resolve(args[1]), 'utf8'); const script = fs.readFileSync(path.resolve(args[1]), 'utf8');
executeRemember(db, script) executeAttune(db, script)
.then(r => { process.stdout.write(JSON.stringify(r, null, 2) + '\n'); db.close(); }) .then(r => { process.stdout.write(JSON.stringify(r, null, 2) + '\n'); db.close(); })
.catch(e => { process.stdout.write(JSON.stringify({ error: e.message, stack: e.stack }) + '\n'); db.close(); process.exitCode = 1; }); .catch(e => { process.stdout.write(JSON.stringify({ error: e.message, stack: e.stack }) + '\n'); db.close(); process.exitCode = 1; });
return; return;
} }
process.stderr.write('Usage:\n node runtime.mjs --build\n node runtime.mjs --search "text"\n node runtime.mjs --query <file.js>\n node runtime.mjs --remember <file.js>\n'); process.stderr.write('Usage:\n node runtime.mjs --build\n node runtime.mjs --search "text"\n node runtime.mjs --query <file.js>\n node runtime.mjs --attune <file.js>\n');
process.exitCode = 1; process.exitCode = 1;
} }
+85
View File
@@ -0,0 +1,85 @@
CREATE TABLE IF NOT EXISTS sessions (
id TEXT PRIMARY KEY, title TEXT, project TEXT, project_path TEXT,
started_at TEXT, ended_at TEXT, git_branch TEXT, version TEXT,
message_count INTEGER DEFAULT 0, jsonl_path TEXT);
CREATE TABLE IF NOT EXISTS messages (
uuid TEXT PRIMARY KEY, session_id TEXT, type TEXT, parent_uuid TEXT,
timestamp TEXT, role TEXT, text TEXT, content_type TEXT,
is_meta INTEGER DEFAULT 0, model TEXT,
is_sidechain INTEGER DEFAULT 0, agent_id TEXT,
input_tokens INTEGER, output_tokens INTEGER,
cwd TEXT, skill TEXT, turn_duration_ms INTEGER);
CREATE TABLE IF NOT EXISTS tool_calls (
id TEXT PRIMARY KEY, message_uuid TEXT, session_id TEXT,
name TEXT, input_json TEXT, file_path TEXT);
CREATE TABLE IF NOT EXISTS tool_results (
tool_use_id TEXT PRIMARY KEY, message_uuid TEXT, session_id TEXT,
content TEXT, file_path TEXT, is_error INTEGER DEFAULT 0);
CREATE TABLE IF NOT EXISTS subagents (
agent_id TEXT PRIMARY KEY, session_id TEXT, parent_tool_use_id TEXT,
agent_type TEXT, description TEXT, duration_ms INTEGER, total_tokens INTEGER);
CREATE TABLE IF NOT EXISTS workflows (
run_id TEXT PRIMARY KEY, session_id TEXT, task_id TEXT,
script TEXT, result_json TEXT, timestamp TEXT, agent_count INTEGER DEFAULT 0,
duration_ms INTEGER, total_tokens INTEGER, status TEXT, workflow_name TEXT);
CREATE TABLE IF NOT EXISTS workflow_agents (
agent_id TEXT PRIMARY KEY, run_id TEXT, session_id TEXT,
agent_type TEXT, description TEXT,
phase TEXT, label TEXT, model TEXT, state TEXT,
duration_ms INTEGER, tokens INTEGER, tool_calls INTEGER);
CREATE TABLE IF NOT EXISTS index_state (
jsonl_path TEXT PRIMARY KEY, mtime REAL, lines_processed INTEGER);
CREATE TABLE IF NOT EXISTS summaries (
id TEXT PRIMARY KEY, session_id TEXT, timestamp TEXT,
source TEXT, content TEXT);
CREATE VIRTUAL TABLE IF NOT EXISTS messages_fts USING fts5(
uuid UNINDEXED, session_id UNINDEXED, text, content=messages, content_rowid=rowid);
CREATE TRIGGER IF NOT EXISTS messages_fts_ai AFTER INSERT ON messages BEGIN
INSERT INTO messages_fts(rowid, uuid, session_id, text)
VALUES (new.rowid, new.uuid, new.session_id, new.text);
END;
CREATE TRIGGER IF NOT EXISTS messages_fts_ad AFTER DELETE ON messages BEGIN
INSERT INTO messages_fts(messages_fts, rowid, uuid, session_id, text)
VALUES ('delete', old.rowid, old.uuid, old.session_id, old.text);
END;
CREATE TRIGGER IF NOT EXISTS messages_fts_au AFTER UPDATE ON messages BEGIN
INSERT INTO messages_fts(messages_fts, rowid, uuid, session_id, text)
VALUES ('delete', old.rowid, old.uuid, old.session_id, old.text);
INSERT INTO messages_fts(rowid, uuid, session_id, text)
VALUES (new.rowid, new.uuid, new.session_id, new.text);
END;
CREATE INDEX IF NOT EXISTS idx_messages_session ON messages(session_id);
CREATE INDEX IF NOT EXISTS idx_messages_agent ON messages(agent_id);
CREATE INDEX IF NOT EXISTS idx_messages_ts ON messages(session_id, timestamp);
CREATE INDEX IF NOT EXISTS idx_tc_session_name ON tool_calls(session_id, name);
CREATE INDEX IF NOT EXISTS idx_tc_file ON tool_calls(file_path);
CREATE INDEX IF NOT EXISTS idx_sa_session ON subagents(session_id);
CREATE INDEX IF NOT EXISTS idx_wf_session ON workflows(session_id);
CREATE INDEX IF NOT EXISTS idx_wa_run ON workflow_agents(run_id);
CREATE INDEX IF NOT EXISTS idx_summaries_session ON summaries(session_id);
CREATE TABLE IF NOT EXISTS memories (
id TEXT PRIMARY KEY, session_id TEXT, project TEXT,
message_start TEXT, message_end TEXT,
path TEXT, anchors TEXT, summary TEXT, created_at TEXT,
deleted_at TEXT, deleted_reason TEXT);
CREATE VIRTUAL TABLE IF NOT EXISTS memories_fts USING fts5(
id UNINDEXED, path, summary,
content=memories, content_rowid=rowid,
tokenize='unicode61 remove_diacritics 1');
CREATE TRIGGER IF NOT EXISTS memories_fts_ai AFTER INSERT ON memories BEGIN
INSERT INTO memories_fts(rowid, id, path, summary)
VALUES (new.rowid, new.id, new.path, new.summary);
END;
CREATE TRIGGER IF NOT EXISTS memories_fts_ad AFTER DELETE ON memories BEGIN
INSERT INTO memories_fts(memories_fts, rowid, id, path, summary)
VALUES ('delete', old.rowid, old.id, old.path, old.summary);
END;
CREATE TRIGGER IF NOT EXISTS memories_fts_au AFTER UPDATE ON memories BEGIN
INSERT INTO memories_fts(memories_fts, rowid, id, path, summary)
VALUES ('delete', old.rowid, old.id, old.path, old.summary);
INSERT INTO memories_fts(rowid, id, path, summary)
VALUES (new.rowid, new.id, new.path, new.summary);
END;
CREATE INDEX IF NOT EXISTS idx_memories_project ON memories(project);
CREATE INDEX IF NOT EXISTS idx_memories_session ON memories(session_id);
CREATE INDEX IF NOT EXISTS idx_memories_created ON memories(created_at);