feat: evolve retrieval layer with content_type, memory FTS, forget(), and recap references

Extract schema DDL into scripts/schema.sql. Add content_type and is_meta
  columns to messages for transcript control-plane filtering. Introduce
  FTS5-backed memory recall with safe tokenization, memory soft-delete via
  forget() through the renamed --attune runtime, and anchors on memory
  records. Expand query helpers (includeMeta, thread opts, overview
  project-path awareness). Add per-card recap retrieval and writing
  references under references/recap/.
This commit is contained in:
tommy0103
2026-06-15 02:25:21 +08:00
parent b52f57b538
commit 3822bda89e
24 changed files with 1290 additions and 157 deletions
+34 -4
View File
@@ -78,7 +78,8 @@ the needed join, grouping, or exact schema-level check better than helpers.
Ordering and context are semantic:
- `sessions()`, `memories()`, `summaries()`, `workflows()`, and `failures()` are newest first.
- `sessions()`, `summaries()`, `workflows()`, and `failures()` are newest first.
- `memories()` without `query` is newest first; `memories({ query })` is FTS-ranked over memory `summary`/`path`, with lower rank sorting earlier.
- `fileHistory()` is oldest first.
- `search().context` is temporal neighbors in one session, not causal context.
- `context(uuid)` and `trace(uuid)` are for parent-chain/causal expansion.
@@ -97,19 +98,44 @@ For semantic questions, build a task-local evidence view:
{
query_plan: { mode, scope, facets, limits },
prior_memories: [
{ id, path, session_id, created_at, summary }
{ id, path, anchors, session_id, created_at, summary }
],
evidence: [
{ type, id, session_id, timestamp, facet, snippet }
{ type, id, session_id, timestamp, content_type, is_meta, facet, snippet }
],
omitted: 0
}
```
For message evidence, preserve `content_type` when projecting snippets.
`text` can support user-visible claims; `thinking` is only trace/debug context;
`tool_use` means follow `tool_calls` for structured details; `tool_result`
means follow `tool_results` for structured output. Mixed or unfamiliar message
surfaces remain `unknown`.
Preserve `is_meta` separately from `content_type`. Default message evidence
should exclude `is_meta=1` rows because they are transcript control-plane
content, not ordinary user intent or assistant conclusions. Include them only
when investigating injected caveats, command envelopes, or transcript structure.
When writing raw SQL for ordinary conversation evidence, add
`COALESCE(m.is_meta,0)=0` to message filters unless meta rows are the subject of
the investigation.
Memory recall is English-indexed: translate non-English user requests into
concise English query terms before calling `memories({ query })`. Memory
summaries registered with `remember()` are also English, regardless of the
conversation language.
`memories({ query })` uses safe FTS5 tokenization over memory `summary` and
`path`, so hyphens and punctuation do not need raw `MATCH` escaping.
`memories()` returns active memories only. For raw SQL memory recall, include
`deleted_at IS NULL`; archived memory records are management/audit data.
The agent may decide whether to use, ignore, or verify a recalled memory for the
current answer without user approval because no persistent state changes. If a
user explicitly says a memory is wrong, outdated, should be forgotten, or should
be replaced, that request is approval to mutate the exact matching memory. If
the agent discovers the conflict without an explicit user request, it should
answer from current evidence and ask before archiving or replacing the memory.
Then synthesize the conclusion in the final answer. Do not pretend the raw
evidence view is itself a stored Obelisk entity.
@@ -121,7 +147,11 @@ project conventions, abandoned alternatives, repeated failure causes, workflow
patterns, and conclusions synthesized across multiple raw evidence points. Do
not propose memory for one-off lookups, uncertain findings, or duplicate
coverage. The offer is only a proposal: write the markdown file and run
`--remember` only after user approval.
`--attune` only after user approval.
Memory updates are archive-plus-write, not in-place edits: run `forget()` on the
old record and `remember()` the replacement markdown file under the same user
approval.
## Text Search Semantics