feat: index Codex sessions alongside Claude Code with source tagging

Add `source` column to sessions and messages ('claude' | 'codex').
  Discover and parse Codex JSONL files from ~/.codex/sessions/, mapping
  Codex thread/item structures to the same schema (messages, tool_calls,
  tool_results, subagents). Move DB to ~/.obelisk/ with legacy migration.
  Add rebuild-to-temp-then-swap for safe full rebuilds. On the app side:
  source filter toggle, collapsible untitled session fold, configurable
  codexDir in Settings, and a dev script. Update SKILL.md and query
  helpers to expose source fields and accept source filter opt.
This commit is contained in:
tommy0103
2026-06-17 23:40:36 +08:00
parent e3ca1735b9
commit 12407227cc
22 changed files with 2259 additions and 244 deletions
+21 -10
View File
@@ -1,7 +1,7 @@
--- ---
name: obelisk name: obelisk
description: > description: >
Search and query past Claude Code session history. Search and query past Claude Code and Codex session history.
Reactive: when the user asks "how did I fix X", "what did we do last time", "find the session where", "上次怎么修的", "之前的session", "历史记录". Reactive: when the user asks "how did I fix X", "what did we do last time", "find the session where", "上次怎么修的", "之前的session", "历史记录".
Proactive: when the user references past work you lack context for, when you're about to modify a file with complex edit history, when the user says "继续之前的" or "continue where we left off", or when understanding prior decisions would improve your current response. Proactive: when the user references past work you lack context for, when you're about to modify a file with complex edit history, when the user says "继续之前的" or "continue where we left off", or when understanding prior decisions would improve your current response.
Memory: when the user says "记住这个", "remember this", "写入记忆", "save this conclusion", or when you determine a retrieval result contains a conclusion worth persisting. Memory: when the user says "记住这个", "remember this", "写入记忆", "save this conclusion", or when you determine a retrieval result contains a conclusion worth persisting.
@@ -13,11 +13,19 @@ allowed-tools:
# obelisk # obelisk
Search and query Claude Code session history stored in `~/.claude/`. Search and query Claude Code and Codex session history stored in `~/.claude/`
and `~/.codex/`.
Obelisk indexes sessions, messages, tool calls, tool results, summaries, Obelisk indexes sessions, messages, tool calls, tool results, summaries,
subagents, workflows, workflow agents, parent chains, and raw JSONL lines into subagents, workflows, workflow agents, parent chains, and raw JSONL lines into
SQLite + FTS5. SQLite + FTS5.
Obelisk has two transcript sources. Treat both as ordinary sessions by default:
Claude rows use `source='claude'`; Codex rows use `source='codex'` and IDs
prefixed with `codex:`. Use `source` only when provenance matters or the user
asks to scope to one provider. Codex subagent child threads are mapped to the
same `subagents` table; Codex workflow rows may be absent because Codex does not
emit Claude-style workflow metadata.
Obelisk is a CodeAct memory layer: write a small JS query, run it locally, read Obelisk is a CodeAct memory layer: write a small JS query, run it locally, read
the JSON, then answer. Do not turn history into a flat document or browse entire the JSON, then answer. Do not turn history into a flat document or browse entire
sessions by default. sessions by default.
@@ -123,8 +131,8 @@ messages.
Returns: Returns:
```js ```js
[{ message: { uuid, text, content_type, is_meta, role, timestamp, model, cwd }, [{ message: { uuid, text, content_type, is_meta, role, timestamp, model, cwd, source },
session: { id, title, project, started_at }, session: { id, title, project, started_at, source },
rank, rank,
context }] context }]
``` ```
@@ -148,13 +156,16 @@ be treated as the user's request by default. `search()` and `thread()` omit meta
messages unless `includeMeta: true` is passed; `context()` and `trace()` preserve messages unless `includeMeta: true` is passed; `context()` and `trace()` preserve
the original chain and expose `is_meta` on rows. the original chain and expose `is_meta` on rows.
Opts: `{ limit, sessionId, project, after, before, cwd, includeMeta }`. Opts: `{ limit, sessionId, project, after, before, cwd, source, includeMeta }`.
`project` is a SQL `LIKE` filter over `sessions.project`, not an exact project `project` is a SQL `LIKE` filter over `sessions.project`, not an exact project
identity. Results are already ordered by FTS5 rank; lower rank sorts earlier. identity. Results are already ordered by FTS5 rank; lower rank sorts earlier.
Prefer returned order over manually interpreting numeric rank unless you are Prefer returned order over manually interpreting numeric rank unless you are
deliberately using FTS5 semantics. deliberately using FTS5 semantics.
`source` can be `'claude'`, `'codex'`, or omitted. Omitted means search all
indexed sources.
### `context(uuid)` ### `context(uuid)`
Returns the full story around one indexed message: Returns the full story around one indexed message:
@@ -191,13 +202,13 @@ not replace `sql()`, but they are the default first-pass surface. Use `sql()`
when you need an exact aggregation or a join the helper does not expose. when you need an exact aggregation or a join the helper does not expose.
All list helpers accept a bounded `limit`. Many also accept: All list helpers accept a bounded `limit`. Many also accept:
`{ project, after, before, sessionId, sessions, branch }`. Check the schema or a `{ project, after, before, sessionId, sessions, branch, source }`. Check the
tiny sample before relying on less common filters. schema or a tiny sample before relying on less common filters.
- `overview(opts?)` -- compact orientation map. Returns current cwd/project if knowable, global project counts, and current-project recent sessions plus memory records. It is a map, not evidence. - `overview(opts?)` -- compact orientation map. Returns current cwd/project if knowable, global project/source counts, and current-project recent sessions plus memory records. It is a map, not evidence.
- `sessions(opts?)` -- session rows, newest first. `project` is a SQL `LIKE` pattern. - `sessions(opts?)` -- session rows, newest first. `project` is a SQL `LIKE` pattern.
- `recent(n?)` -- shorthand for recent sessions. - `recent(n?)` -- shorthand for recent sessions.
- `summaries(opts?)` -- summary rows, newest first: `{ id, session_id, timestamp, source, content, session_title, project }`. - `summaries(opts?)` -- summary rows, newest first: `{ id, session_id, timestamp, source, content, session_title, project }`; here `source` is the summary kind, not the transcript provider.
- `subagents(opts?)` -- subagent metadata plus `messageCount`. - `subagents(opts?)` -- subagent metadata plus `messageCount`.
- `workflows(opts?)` -- workflow runs, newest first. - `workflows(opts?)` -- workflow runs, newest first.
- `workflowTree(runId)` -- workflow row plus parsed `result` and `agents`; may include bulky `script` and `result_json`, so project compact fields. - `workflowTree(runId)` -- workflow row plus parsed `result` and `agents`; may include bulky `script` and `result_json`, so project compact fields.
@@ -372,6 +383,6 @@ See `references/query-patterns.md` for longer recipes.
## Notes ## Notes
- First run builds the index. Later runs update incrementally. - First run builds the index. Later runs update incrementally.
- DB location: `~/.claude/obelisk.sqlite`. - DB location: `~/.obelisk/obelisk.sqlite`; old `~/.claude/obelisk.sqlite` is copied forward if needed.
- Query scripts run in a sandboxed VM with no filesystem or network access from inside the script. - Query scripts run in a sandboxed VM with no filesystem or network access from inside the script.
- Indexed text and stored tool inputs/results are truncated to 10k chars. Use `raw(uuid, { offset, limit })` for specific JSONL windows. - Indexed text and stored tool inputs/results are truncated to 10k chars. Use `raw(uuid, { offset, limit })` for specific JSONL windows.
+32 -20
View File
@@ -10,6 +10,7 @@ const DEFAULT_WATCH_RETRY_MS = 5000;
function createIndexerService({ function createIndexerService({
projectsDir = DEFAULT_PROJECTS_DIR, projectsDir = DEFAULT_PROJECTS_DIR,
watchDirs = [projectsDir],
debounceMs = DEFAULT_DEBOUNCE_MS, debounceMs = DEFAULT_DEBOUNCE_MS,
stabilityMs = DEFAULT_STABILITY_MS, stabilityMs = DEFAULT_STABILITY_MS,
heartbeatMs = DEFAULT_HEARTBEAT_MS, heartbeatMs = DEFAULT_HEARTBEAT_MS,
@@ -28,31 +29,42 @@ function createIndexerService({
} = {}) { } = {}) {
if (typeof buildIndex !== 'function') throw new Error('createIndexerService() requires buildIndex'); if (typeof buildIndex !== 'function') throw new Error('createIndexerService() requires buildIndex');
const watch = watchProjects || ((onChange) => { const watch = watchProjects || ((onChange) => {
if (!fs.existsSync(projectsDir)) return null; const roots = [...new Set((Array.isArray(watchDirs) ? watchDirs : [watchDirs]).filter(Boolean))];
const watcher = (chokidar || require('chokidar')).watch(projectsDir, { const existingRoots = roots.filter(root => fs.existsSync(root));
cwd: projectsDir, if (!existingRoots.length) return null;
ignoreInitial: true, const watchers = [];
awaitWriteFinish: {
stabilityThreshold: Math.max(stabilityMs, 500),
pollInterval: 100,
},
ignored: (targetPath, stats) => {
if (stats?.isDirectory()) return false;
if (!stats) return false;
return !String(targetPath).endsWith('.jsonl') && !String(targetPath).endsWith('.json');
},
});
const onFileChange = (filename) => { const onFileChange = (filename) => {
const name = filename ? String(filename) : ''; const name = filename ? String(filename) : '';
if (!name || name.endsWith('.jsonl') || name.endsWith('.json')) onChange(name); if (!name || name.endsWith('.jsonl') || name.endsWith('.json')) onChange(name);
}; };
return watcher for (const root of existingRoots) {
.on('add', onFileChange) const watcher = (chokidar || require('chokidar')).watch(root, {
.on('change', onFileChange) cwd: root,
.on('unlink', onFileChange) ignoreInitial: true,
.on('error', (error) => { awaitWriteFinish: {
logger.warn?.(`Obelisk watcher failed: ${error.message}`); stabilityThreshold: Math.max(stabilityMs, 500),
pollInterval: 100,
},
ignored: (targetPath, stats) => {
if (stats?.isDirectory()) return false;
if (!stats) return false;
return !String(targetPath).endsWith('.jsonl') && !String(targetPath).endsWith('.json');
},
}); });
watcher
.on('add', onFileChange)
.on('change', onFileChange)
.on('unlink', onFileChange)
.on('error', (error) => {
logger.warn?.(`Obelisk watcher failed: ${error.message}`);
});
watchers.push(watcher);
}
return {
close() {
return Promise.all(watchers.map(w => Promise.resolve(w.close?.())));
},
};
}); });
let buildTimer = null; let buildTimer = null;
+2 -1
View File
@@ -49,8 +49,9 @@ function createWorkerBuildIndex({
const stop = () => { const stop = () => {
const current = worker; const current = worker;
worker = null; worker = null;
if (current?.terminate) current.terminate(); const termination = current?.terminate ? Promise.resolve(current.terminate()) : Promise.resolve();
rejectPending(new Error('Indexer worker stopped')); rejectPending(new Error('Indexer worker stopped'));
return termination;
}; };
return { buildIndex, stop }; return { buildIndex, stop };
+627 -15
View File
@@ -5,7 +5,9 @@ const Database = require('better-sqlite3');
const TEXT_LIMIT = 10000; const TEXT_LIMIT = 10000;
const DEFAULT_CLAUDE_DIR = path.join(os.homedir(), '.claude'); const DEFAULT_CLAUDE_DIR = path.join(os.homedir(), '.claude');
const DEFAULT_DB_PATH = path.join(DEFAULT_CLAUDE_DIR, 'obelisk.sqlite'); const DEFAULT_CODEX_DIR = path.join(os.homedir(), '.codex');
const DEFAULT_OBELISK_DIR = path.join(os.homedir(), '.obelisk');
const DEFAULT_DB_PATH = path.join(DEFAULT_OBELISK_DIR, 'obelisk.sqlite');
const DEFAULT_PROJECTS_DIR = path.join(DEFAULT_CLAUDE_DIR, 'projects'); const DEFAULT_PROJECTS_DIR = path.join(DEFAULT_CLAUDE_DIR, 'projects');
const DEFAULT_HISTORY_PATH = path.join(DEFAULT_CLAUDE_DIR, 'history.jsonl'); const DEFAULT_HISTORY_PATH = path.join(DEFAULT_CLAUDE_DIR, 'history.jsonl');
@@ -20,13 +22,17 @@ function resolveSchemaPath() {
return found; return found;
} }
function installSchema(db, schemaPath = resolveSchemaPath()) {
db.exec(fs.readFileSync(schemaPath, 'utf8'));
migrateDb(db);
}
function openIndexDb({ dbPath = DEFAULT_DB_PATH, schemaPath = resolveSchemaPath(), DatabaseImpl = Database } = {}) { function openIndexDb({ dbPath = DEFAULT_DB_PATH, schemaPath = resolveSchemaPath(), DatabaseImpl = Database } = {}) {
fs.mkdirSync(path.dirname(dbPath), { recursive: true }); fs.mkdirSync(path.dirname(dbPath), { recursive: true });
const db = new DatabaseImpl(dbPath); const db = new DatabaseImpl(dbPath);
db.pragma('journal_mode = WAL'); db.pragma('journal_mode = WAL');
db.pragma('synchronous = NORMAL'); db.pragma('synchronous = NORMAL');
db.exec(fs.readFileSync(schemaPath, 'utf8')); installSchema(db, schemaPath);
migrateDb(db);
return db; return db;
} }
@@ -36,13 +42,54 @@ function ensureColumn(db, table, column, definition) {
} }
function migrateDb(db) { function migrateDb(db) {
ensureColumn(db, 'sessions', 'source', "TEXT DEFAULT 'claude'");
ensureColumn(db, 'messages', 'content_type', 'TEXT'); ensureColumn(db, 'messages', 'content_type', 'TEXT');
ensureColumn(db, 'messages', 'is_meta', 'INTEGER DEFAULT 0'); ensureColumn(db, 'messages', 'is_meta', 'INTEGER DEFAULT 0');
ensureColumn(db, 'messages', 'source', "TEXT DEFAULT 'claude'");
ensureColumn(db, 'memories', 'anchors', 'TEXT'); ensureColumn(db, 'memories', 'anchors', 'TEXT');
ensureColumn(db, 'memories', 'deleted_at', 'TEXT'); ensureColumn(db, 'memories', 'deleted_at', 'TEXT');
ensureColumn(db, 'memories', 'deleted_reason', 'TEXT'); ensureColumn(db, 'memories', 'deleted_reason', 'TEXT');
} }
function copyMemoriesFromDb(db, sourceDbPath) {
if (!sourceDbPath || !fs.existsSync(sourceDbPath)) return false;
db.prepare('ATTACH DATABASE ? AS previous_obelisk').run(sourceDbPath);
try {
const hasMemories = db.prepare(`
SELECT name FROM previous_obelisk.sqlite_master
WHERE type='table' AND name='memories'
`).get();
if (!hasMemories) return false;
const sourceColumns = new Set(
db.prepare('PRAGMA previous_obelisk.table_info(memories)').all().map(column => column.name),
);
const targetColumns = [
'id',
'session_id',
'project',
'message_start',
'message_end',
'path',
'anchors',
'summary',
'created_at',
'deleted_at',
'deleted_reason',
];
const selectList = targetColumns
.map(column => sourceColumns.has(column) ? column : `NULL AS ${column}`)
.join(',');
db.exec(`
INSERT OR REPLACE INTO memories (${targetColumns.join(',')})
SELECT ${selectList} FROM previous_obelisk.memories
`);
return true;
} finally {
db.exec('DETACH DATABASE previous_obelisk');
}
}
function trunc(s) { function trunc(s) {
return typeof s === 'string' && s.length > TEXT_LIMIT ? s.slice(0, TEXT_LIMIT) : s; return typeof s === 'string' && s.length > TEXT_LIMIT ? s.slice(0, TEXT_LIMIT) : s;
} }
@@ -128,6 +175,12 @@ function normalizeObservedCwd(cwd) {
return path.normalize(cwd); return path.normalize(cwd);
} }
function projectSlugFromPath(projectPath) {
const normalized = normalizeObservedCwd(projectPath);
if (!normalized) return null;
return '-' + normalized.replace(/^[\\/]+/, '').replace(/[\\/]+/g, '-');
}
function inferProjectPath(project, observedCwds = []) { function inferProjectPath(project, observedCwds = []) {
const byPath = new Map(); const byPath = new Map();
for (const cwd of observedCwds) { for (const cwd of observedCwds) {
@@ -158,6 +211,7 @@ function normalizeChangedPath(projectsDir, changedPath) {
function jsonlFileInfoFromPath(projectsDir, changedPath) { function jsonlFileInfoFromPath(projectsDir, changedPath) {
const fp = normalizeChangedPath(projectsDir, changedPath); const fp = normalizeChangedPath(projectsDir, changedPath);
if (!fp || !fp.endsWith('.jsonl')) return null; if (!fp || !fp.endsWith('.jsonl')) return null;
if (!fs.existsSync(fp)) return null;
const rel = path.relative(projectsDir, fp); const rel = path.relative(projectsDir, fp);
if (!rel || rel.startsWith('..') || path.isAbsolute(rel)) return null; if (!rel || rel.startsWith('..') || path.isAbsolute(rel)) return null;
const parts = rel.split(path.sep); const parts = rel.split(path.sep);
@@ -191,8 +245,10 @@ function sessionIdFromChangedPath(projectsDir, changedPath) {
const rel = path.relative(projectsDir, fp); const rel = path.relative(projectsDir, fp);
if (!rel || rel.startsWith('..') || path.isAbsolute(rel)) return null; if (!rel || rel.startsWith('..') || path.isAbsolute(rel)) return null;
const parts = rel.split(path.sep); const parts = rel.split(path.sep);
if (parts.length === 2 && parts[1].endsWith('.jsonl')) return parts[1].slice(0, -6); if (parts.length === 2 && parts[1].endsWith('.jsonl')) {
if (parts.length >= 3) return parts[1] || null; return fs.existsSync(fp) ? parts[1].slice(0, -6) : null;
}
if (parts.length >= 3) return fs.existsSync(fp) ? parts[1] || null : null;
return null; return null;
} }
@@ -253,6 +309,70 @@ function discoverJsonlFilesFull({ projectsDir = DEFAULT_PROJECTS_DIR } = {}) {
return files; return files;
} }
function discoverCodexJsonlFiles({ codexDir = DEFAULT_CODEX_DIR, changedPaths = undefined } = {}) {
if (Array.isArray(changedPaths) && changedPaths.length) {
const changedFiles = discoverCodexJsonlFilesForChanges({ codexDir, changedPaths });
if (changedFiles.length) return changedFiles;
return [];
}
return discoverCodexJsonlFilesFull({ codexDir });
}
function codexSessionsDir(codexDir = DEFAULT_CODEX_DIR) {
return path.join(codexDir, 'sessions');
}
function normalizeChangedPathForRoot(rootDir, changedPath) {
if (!changedPath) return null;
const raw = String(changedPath);
return path.isAbsolute(raw) ? path.normalize(raw) : path.normalize(path.join(rootDir, raw));
}
function isPathInside(rootDir, candidate) {
if (!rootDir || !candidate) return false;
const rel = path.relative(rootDir, candidate);
return !!rel && !rel.startsWith('..') && !path.isAbsolute(rel);
}
function discoverCodexJsonlFilesForChanges({ codexDir = DEFAULT_CODEX_DIR, changedPaths = [] } = {}) {
const files = [];
const sessionsDir = codexSessionsDir(codexDir);
for (const changedPath of changedPaths) {
const rootRelativePath = normalizeChangedPathForRoot(codexDir, changedPath);
if (!rootRelativePath) continue;
if (path.normalize(rootRelativePath) === path.join(codexDir, 'session_index.jsonl')) {
return discoverCodexJsonlFilesFull({ codexDir });
}
const sessionRelativePath = normalizeChangedPathForRoot(sessionsDir, changedPath);
const fp = isPathInside(sessionsDir, rootRelativePath) ? rootRelativePath : sessionRelativePath;
if (!fp.endsWith('.jsonl') || !isPathInside(sessionsDir, fp)) continue;
if (!fs.existsSync(fp)) continue;
files.push({ path: fp, source: 'codex' });
}
return dedupeFileInfos(files);
}
function discoverCodexJsonlFilesFull({ codexDir = DEFAULT_CODEX_DIR } = {}) {
const root = codexSessionsDir(codexDir);
const files = [];
if (!fs.existsSync(root)) return files;
const stack = [root];
while (stack.length) {
const current = stack.pop();
let entries;
try { entries = fs.readdirSync(current, { withFileTypes: true }); } catch { continue; }
for (const entry of entries) {
const fp = path.join(current, entry.name);
if (entry.isDirectory()) {
stack.push(fp);
} else if (entry.isFile() && entry.name.endsWith('.jsonl')) {
files.push({ path: fp, source: 'codex' });
}
}
}
return files.sort((a, b) => a.path.localeCompare(b.path));
}
function needsReindex(db, fp) { function needsReindex(db, fp) {
const mt = fs.statSync(fp).mtimeMs; const mt = fs.statSync(fp).mtimeMs;
const row = db.prepare('SELECT mtime, lines_processed FROM index_state WHERE jsonl_path = ?').get(fp); const row = db.prepare('SELECT mtime, lines_processed FROM index_state WHERE jsonl_path = ?').get(fp);
@@ -264,10 +384,10 @@ function indexJsonl(db, fi) {
const { needed, skip, mtime } = needsReindex(db, fi.path); const { needed, skip, mtime } = needsReindex(db, fi.path);
if (!needed) return; if (!needed) return;
const ins = { const ins = {
ses: db.prepare('INSERT OR REPLACE INTO sessions (id,title,project,project_path,started_at,ended_at,git_branch,version,message_count,jsonl_path) VALUES (?,?,?,?,?,?,?,?,?,?)'), ses: db.prepare('INSERT OR REPLACE INTO sessions (id,title,project,project_path,started_at,ended_at,git_branch,version,message_count,jsonl_path,source) VALUES (?,?,?,?,?,?,?,?,?,?,?)'),
msg: db.prepare(` msg: db.prepare(`
INSERT INTO messages (uuid,session_id,type,parent_uuid,timestamp,role,text,content_type,is_meta,model,is_sidechain,agent_id,input_tokens,output_tokens,cwd,skill) INSERT INTO messages (uuid,session_id,type,parent_uuid,timestamp,role,text,content_type,is_meta,model,is_sidechain,agent_id,input_tokens,output_tokens,cwd,skill,source)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
ON CONFLICT(uuid) DO UPDATE SET ON CONFLICT(uuid) DO UPDATE SET
session_id=excluded.session_id, session_id=excluded.session_id,
type=excluded.type, type=excluded.type,
@@ -283,7 +403,8 @@ function indexJsonl(db, fi) {
input_tokens=excluded.input_tokens, input_tokens=excluded.input_tokens,
output_tokens=excluded.output_tokens, output_tokens=excluded.output_tokens,
cwd=excluded.cwd, cwd=excluded.cwd,
skill=excluded.skill skill=excluded.skill,
source=excluded.source
`), `),
tc: db.prepare('INSERT OR REPLACE INTO tool_calls (id,message_uuid,session_id,name,input_json,file_path) VALUES (?,?,?,?,?,?)'), tc: db.prepare('INSERT OR REPLACE INTO tool_calls (id,message_uuid,session_id,name,input_json,file_path) VALUES (?,?,?,?,?,?)'),
tr: db.prepare('INSERT OR REPLACE INTO tool_results (tool_use_id,message_uuid,session_id,content,file_path,is_error) VALUES (?,?,?,?,?,?)'), tr: db.prepare('INSERT OR REPLACE INTO tool_results (tool_use_id,message_uuid,session_id,content,file_path,is_error) VALUES (?,?,?,?,?,?)'),
@@ -336,7 +457,7 @@ function indexJsonl(db, fi) {
ins.msg.run(obj.uuid, sid, obj.type, obj.parentUuid || null, ts, ins.msg.run(obj.uuid, sid, obj.type, obj.parentUuid || null, ts,
msg.role || obj.type, text, contentType, isMeta, msg.model || null, msg.role || obj.type, text, contentType, isMeta, msg.model || null,
obj.isSidechain ? 1 : 0, aid, usage.input_tokens || null, usage.output_tokens || null, obj.isSidechain ? 1 : 0, aid, usage.input_tokens || null, usage.output_tokens || null,
obj.cwd || null, obj.attributionSkill || null); obj.cwd || null, obj.attributionSkill || null, 'claude');
} }
if (obj.type === 'assistant' && Array.isArray(msg.content)) { if (obj.type === 'assistant' && Array.isArray(msg.content)) {
for (const b of msg.content) { for (const b of msg.content) {
@@ -356,12 +477,464 @@ function indexJsonl(db, fi) {
if (!fi.isSubagent) { if (!fi.isSubagent) {
const pp = inferProjectPath(fi.project, sm.cwds); const pp = inferProjectPath(fi.project, sm.cwds);
ins.ses.run(fi.sessionId, sm.title, fi.project, pp, sm.started_at, sm.ended_at, sm.git_branch, sm.version, sm.n, fi.path); ins.ses.run(fi.sessionId, sm.title, fi.project, pp, sm.started_at, sm.ended_at, sm.git_branch, sm.version, sm.n, fi.path, 'claude');
} }
ins.idx.run(fi.path, mtime, lineNum); ins.idx.run(fi.path, mtime, lineNum);
return { sessionId: fi.sessionId, path: fi.path }; return { sessionId: fi.sessionId, path: fi.path };
} }
function codexDbId(id) {
if (!id) return null;
const raw = String(id).replace(/^codex:/, '');
return `codex:${raw}`;
}
function codexRawId(id) {
return id ? String(id).replace(/^codex:/, '') : null;
}
function codexLineUuid(threadId, lineNum) {
return `codex:${codexRawId(threadId)}:${String(lineNum).padStart(6, '0')}`;
}
function codexCallId(callId) {
if (!callId) return null;
return `codex:${String(callId).replace(/^codex:/, '')}`;
}
function codexParentThreadId(meta) {
const subagent = meta?.source?.subagent;
return subagent?.thread_spawn?.parent_thread_id
|| meta?.forked_from_id
|| subagent?.parent_thread_id
|| null;
}
function codexIsGuardianThread(meta, records = []) {
const subagent = meta?.source?.subagent;
if (subagent?.other === 'guardian') return true;
if (meta?.thread_source !== 'subagent') return false;
return records.some(({ obj }) => obj?.payload?.model === 'codex-auto-review' || obj?.model === 'codex-auto-review');
}
function deleteCodexThreadRows(db, threadRawId) {
const threadId = codexDbId(threadRawId);
if (!threadId) return;
db.prepare(`
DELETE FROM tool_results
WHERE session_id = ?
OR message_uuid IN (SELECT uuid FROM messages WHERE session_id = ? OR agent_id = ?)
`).run(threadId, threadId, threadId);
db.prepare(`
DELETE FROM tool_calls
WHERE session_id = ?
OR message_uuid IN (SELECT uuid FROM messages WHERE session_id = ? OR agent_id = ?)
`).run(threadId, threadId, threadId);
db.prepare('DELETE FROM messages WHERE session_id = ? OR agent_id = ?').run(threadId, threadId);
db.prepare('DELETE FROM subagents WHERE agent_id = ? OR session_id = ?').run(threadId, threadId);
db.prepare('DELETE FROM summaries WHERE session_id = ?').run(threadId);
db.prepare('DELETE FROM sessions WHERE id = ?').run(threadId);
}
function readCodexGuardianThreadInfo(filePath) {
const records = [];
let metaRecord = null;
let lineNum = 0;
readLines(filePath, (line) => {
lineNum++;
let obj;
try {
obj = JSON.parse(line);
} catch {
return;
}
records.push({ lineNum, obj });
if (obj?.type === 'session_meta' && obj.payload?.id) {
metaRecord = { lineNum, obj };
if (obj.payload?.source?.subagent?.other === 'guardian') return false;
if (obj.payload?.thread_source !== 'subagent') return false;
}
if (metaRecord && codexIsGuardianThread(metaRecord.obj.payload, records)) return false;
});
const meta = metaRecord?.obj?.payload;
if (!meta || !codexIsGuardianThread(meta, records)) return null;
return { threadRawId: codexRawId(meta.id), lineNum };
}
function codexAgentNickname(meta) {
return meta?.agent_nickname
|| meta?.source?.subagent?.thread_spawn?.agent_nickname
|| null;
}
function codexAgentRole(meta) {
return meta?.agent_role
|| meta?.source?.subagent?.thread_spawn?.agent_role
|| null;
}
function parseCodexJsonInput(value) {
if (value === null || value === undefined || value === '') return {};
if (typeof value !== 'string') return value;
try { return JSON.parse(value); } catch { return value; }
}
function codexUsage(payload) {
const usage = payload?.info?.last_token_usage || payload?.info?.total_token_usage || payload?.last_token_usage || null;
if (!usage) return {};
return {
inputTokens: usage.input_tokens ?? null,
outputTokens: usage.output_tokens ?? null,
};
}
function codexEventText(payload) {
if (typeof payload?.message === 'string') return payload.message;
if (Array.isArray(payload?.text_elements) && payload.text_elements.length) {
const parts = payload.text_elements.map(item => typeof item === 'string' ? item : item?.text).filter(Boolean);
if (parts.length) return parts.join('\n');
}
if (typeof payload?.text === 'string') return payload.text;
return null;
}
function codexMessagePayloadText(payload) {
if (!Array.isArray(payload?.content)) return null;
const parts = [];
for (const block of payload.content) {
if (typeof block?.text === 'string') parts.push(block.text);
}
return parts.length ? parts.join('\n') : null;
}
function codexVisibleMessageKey(role, text) {
return `${role || ''}\u0000${text || ''}`;
}
function codexToolInput(payload) {
if (payload?.type === 'custom_tool_call') return parseCodexJsonInput(payload.input);
if (payload?.type === 'tool_search_call') return parseCodexJsonInput(payload.arguments);
if (payload?.type === 'web_search_call') return { action: payload.action || null };
return parseCodexJsonInput(payload?.arguments);
}
function codexToolOutput(payload) {
if (typeof payload?.output === 'string') return payload.output;
if (payload?.output !== undefined) return JSON.stringify(payload.output);
if (payload?.tools !== undefined) return JSON.stringify(payload.tools);
if (payload?.execution !== undefined) return JSON.stringify(payload.execution);
return null;
}
function upsertCodexSubagent(db, {
agentId,
sessionId,
parentToolUseId = null,
agentType = null,
description = null,
durationMs = null,
totalTokens = null,
} = {}) {
if (!agentId || !sessionId) return;
db.prepare(`
INSERT INTO subagents (agent_id,session_id,parent_tool_use_id,agent_type,description,duration_ms,total_tokens)
VALUES (?,?,?,?,?,?,?)
ON CONFLICT(agent_id) DO UPDATE SET
session_id=excluded.session_id,
parent_tool_use_id=COALESCE(excluded.parent_tool_use_id, subagents.parent_tool_use_id),
agent_type=COALESCE(excluded.agent_type, subagents.agent_type),
description=COALESCE(excluded.description, subagents.description),
duration_ms=COALESCE(excluded.duration_ms, subagents.duration_ms),
total_tokens=COALESCE(excluded.total_tokens, subagents.total_tokens)
`).run(agentId, sessionId, parentToolUseId, agentType, description, durationMs, totalTokens);
}
function indexCodexJsonl(db, fi) {
const state = needsReindex(db, fi.path);
if (!state.needed) {
const guardian = readCodexGuardianThreadInfo(fi.path);
if (guardian) deleteCodexThreadRows(db, guardian.threadRawId);
return null;
}
const mtime = state.mtime;
const records = [];
let lineNum = 0;
readLines(fi.path, (line) => {
lineNum++;
try {
records.push({ lineNum, obj: JSON.parse(line) });
} catch {}
});
const metaRecord = records.find(r => r.obj?.type === 'session_meta' && r.obj.payload?.id);
if (!metaRecord) {
db.prepare('INSERT OR REPLACE INTO index_state (jsonl_path,mtime,lines_processed) VALUES (?,?,?)').run(fi.path, mtime, lineNum);
return null;
}
const meta = metaRecord.obj.payload;
const threadRawId = codexRawId(meta.id);
if (codexIsGuardianThread(meta, records)) {
deleteCodexThreadRows(db, threadRawId);
db.prepare('INSERT OR REPLACE INTO index_state (jsonl_path,mtime,lines_processed) VALUES (?,?,?)').run(fi.path, mtime, lineNum);
return null;
}
const parentRawId = codexParentThreadId(meta);
const sessionId = codexDbId(parentRawId || threadRawId);
const agentId = parentRawId ? codexDbId(threadRawId) : null;
const isSidechain = agentId ? 1 : 0;
const projectPath = normalizeObservedCwd(meta.cwd);
const project = projectSlugFromPath(projectPath);
const sm = {
started_at: meta.timestamp || metaRecord.obj.timestamp || null,
ended_at: meta.timestamp || metaRecord.obj.timestamp || null,
git_branch: meta.git?.branch || null,
version: meta.cli_version || null,
title: null,
n: 0,
cwds: projectPath ? [projectPath] : [],
lastMessageUuid: null,
lastTextAssistantUuid: null,
totalInputTokens: 0,
totalOutputTokens: 0,
};
const ins = {
ses: db.prepare('INSERT OR REPLACE INTO sessions (id,title,project,project_path,started_at,ended_at,git_branch,version,message_count,jsonl_path,source) VALUES (?,?,?,?,?,?,?,?,?,?,?)'),
msg: db.prepare(`
INSERT INTO messages (uuid,session_id,type,parent_uuid,timestamp,role,text,content_type,is_meta,model,is_sidechain,agent_id,input_tokens,output_tokens,cwd,skill,source)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
ON CONFLICT(uuid) DO UPDATE SET
session_id=excluded.session_id,
type=excluded.type,
parent_uuid=excluded.parent_uuid,
timestamp=excluded.timestamp,
role=excluded.role,
text=excluded.text,
content_type=excluded.content_type,
is_meta=excluded.is_meta,
model=excluded.model,
is_sidechain=excluded.is_sidechain,
agent_id=excluded.agent_id,
input_tokens=excluded.input_tokens,
output_tokens=excluded.output_tokens,
cwd=excluded.cwd,
skill=excluded.skill,
source=excluded.source
`),
tc: db.prepare('INSERT OR REPLACE INTO tool_calls (id,message_uuid,session_id,name,input_json,file_path) VALUES (?,?,?,?,?,?)'),
tr: db.prepare('INSERT OR REPLACE INTO tool_results (tool_use_id,message_uuid,session_id,content,file_path,is_error) VALUES (?,?,?,?,?,?)'),
idx: db.prepare('INSERT OR REPLACE INTO index_state (jsonl_path,mtime,lines_processed) VALUES (?,?,?)'),
dur: db.prepare('UPDATE messages SET turn_duration_ms=? WHERE uuid=?'),
usage: db.prepare('UPDATE messages SET input_tokens=?, output_tokens=? WHERE uuid=?'),
};
let currentCwd = projectPath;
let currentModel = null;
const eventMessageKeys = new Set();
const callMessageUuids = new Map();
for (const { obj } of records) {
if (obj?.type !== 'event_msg') continue;
const payload = obj.payload || {};
if (payload.type !== 'user_message' && payload.type !== 'agent_message') continue;
const text = codexEventText(payload);
if (text === null) continue;
eventMessageKeys.add(codexVisibleMessageKey(payload.type === 'user_message' ? 'user' : 'assistant', text));
}
const updateBounds = (ts) => {
if (!ts) return;
if (!sm.started_at || ts < sm.started_at) sm.started_at = ts;
if (!sm.ended_at || ts > sm.ended_at) sm.ended_at = ts;
};
const insertMessage = ({ uuid, type, role, text = null, contentType = 'text', timestamp, isMeta = 0 }) => {
ins.msg.run(
uuid,
sessionId,
type,
sm.lastMessageUuid,
timestamp || null,
role,
trunc(text),
contentType,
isMeta,
currentModel,
isSidechain,
agentId,
null,
null,
currentCwd,
null,
'codex',
);
sm.lastMessageUuid = uuid;
if (!agentId) sm.n++;
if (type === 'assistant' && contentType === 'text') sm.lastTextAssistantUuid = uuid;
updateBounds(timestamp);
return uuid;
};
for (const { lineNum: currentLine, obj } of records) {
const ts = obj.timestamp || null;
if (obj.type === 'session_meta') {
if (obj.payload?.cwd) {
currentCwd = normalizeObservedCwd(obj.payload.cwd) || currentCwd;
if (currentCwd) sm.cwds.push(currentCwd);
}
if (obj.payload?.git?.branch) sm.git_branch = obj.payload.git.branch;
if (obj.payload?.cli_version) sm.version = obj.payload.cli_version;
updateBounds(obj.payload?.timestamp || ts);
continue;
}
if (obj.type === 'turn_context') {
currentCwd = normalizeObservedCwd(obj.payload?.cwd) || currentCwd;
currentModel = obj.payload?.model || currentModel;
if (currentCwd) sm.cwds.push(currentCwd);
updateBounds(ts);
continue;
}
if (obj.type === 'event_msg') {
const payload = obj.payload || {};
if (payload.type === 'user_message' || payload.type === 'agent_message' || payload.type === 'agent_reasoning') {
const text = codexEventText(payload);
if (text === null) continue;
const isReasoning = payload.type === 'agent_reasoning';
insertMessage({
uuid: codexLineUuid(threadRawId, currentLine),
type: payload.type === 'user_message' ? 'user' : 'assistant',
role: payload.type === 'user_message' ? 'user' : 'assistant',
text,
contentType: isReasoning ? 'thinking' : 'text',
timestamp: ts,
});
continue;
}
if (payload.type === 'collab_agent_spawn_end' && payload.call_id && payload.new_thread_id) {
const uuid = insertMessage({
uuid: codexLineUuid(threadRawId, currentLine),
type: 'assistant',
role: 'assistant',
text: null,
contentType: 'tool_use',
timestamp: ts,
});
const toolId = codexCallId(payload.call_id);
const description = payload.new_agent_nickname || payload.new_agent_role || 'Agent';
const input = {
description,
subagent_type: payload.new_agent_role || 'Agent',
prompt: payload.prompt || '',
new_thread_id: payload.new_thread_id,
model: payload.model || null,
reasoning_effort: payload.reasoning_effort || null,
};
ins.tc.run(toolId, uuid, sessionId, 'Agent', truncJson(input), null);
callMessageUuids.set(toolId, uuid);
upsertCodexSubagent(db, {
agentId: codexDbId(payload.new_thread_id),
sessionId,
parentToolUseId: toolId,
agentType: payload.new_agent_role || null,
description,
});
continue;
}
if (payload.type === 'task_complete') {
if (sm.lastTextAssistantUuid && payload.duration_ms !== undefined) {
ins.dur.run(payload.duration_ms || null, sm.lastTextAssistantUuid);
}
updateBounds(ts);
continue;
}
if (payload.type === 'token_count') {
const usage = codexUsage(payload);
if (usage.inputTokens !== null) sm.totalInputTokens = usage.inputTokens;
if (usage.outputTokens !== null) sm.totalOutputTokens = usage.outputTokens;
if (sm.lastTextAssistantUuid && (usage.inputTokens !== null || usage.outputTokens !== null)) {
ins.usage.run(usage.inputTokens, usage.outputTokens, sm.lastTextAssistantUuid);
}
continue;
}
if (payload.type === 'thread_name_updated' && payload.thread_name) {
sm.title = payload.thread_name;
}
continue;
}
if (obj.type !== 'response_item') continue;
const payload = obj.payload || {};
if (payload.type === 'message' && payload.role !== 'developer') {
const text = codexMessagePayloadText(payload);
const role = payload.role || 'assistant';
if (text !== null && !eventMessageKeys.has(codexVisibleMessageKey(role, text))) {
insertMessage({
uuid: codexLineUuid(threadRawId, currentLine),
type: role === 'user' ? 'user' : 'assistant',
role,
text,
contentType: 'text',
timestamp: ts,
});
}
continue;
}
if (['function_call', 'custom_tool_call', 'tool_search_call', 'web_search_call'].includes(payload.type) && payload.call_id) {
const uuid = insertMessage({
uuid: codexLineUuid(threadRawId, currentLine),
type: 'assistant',
role: 'assistant',
text: null,
contentType: 'tool_use',
timestamp: ts,
});
const name = payload.name || payload.tool || payload.type.replace(/_call$/, '');
const toolId = codexCallId(payload.call_id);
ins.tc.run(toolId, uuid, sessionId, name, truncJson(codexToolInput(payload)), null);
callMessageUuids.set(toolId, uuid);
continue;
}
if (['function_call_output', 'custom_tool_call_output', 'tool_search_output'].includes(payload.type) && payload.call_id) {
const toolId = codexCallId(payload.call_id);
ins.tr.run(toolId, callMessageUuids.get(toolId) || null, sessionId, trunc(codexToolOutput(payload) || ''), null, payload.is_error ? 1 : 0);
}
}
if (agentId) {
const tokenTotal = (sm.totalInputTokens || 0) + (sm.totalOutputTokens || 0);
const started = sm.started_at ? new Date(sm.started_at).getTime() : null;
const ended = sm.ended_at ? new Date(sm.ended_at).getTime() : null;
upsertCodexSubagent(db, {
agentId,
sessionId,
agentType: codexAgentRole(meta),
description: codexAgentNickname(meta),
durationMs: started && ended ? ended - started : null,
totalTokens: tokenTotal || null,
});
} else {
const pp = inferProjectPath(project, sm.cwds);
ins.ses.run(sessionId, sm.title, project, pp, sm.started_at, sm.ended_at, sm.git_branch, sm.version, sm.n, fi.path, 'codex');
}
ins.idx.run(fi.path, mtime, lineNum);
return { sessionId, path: fi.path };
}
function indexCodexSessionIndex(db, { codexDir = DEFAULT_CODEX_DIR } = {}) {
const indexPath = path.join(codexDir, 'session_index.jsonl');
if (!fs.existsSync(indexPath)) return;
readLines(indexPath, (line) => {
try {
const item = JSON.parse(line);
if (!item.id || !item.thread_name) return;
db.prepare('UPDATE sessions SET title=COALESCE(title, ?), ended_at=COALESCE(ended_at, ?) WHERE id=? AND source=?')
.run(item.thread_name, item.updated_at || null, codexDbId(item.id), 'codex');
} catch (error) {
console.warn(`Warning: malformed Codex session index line: ${error.message}`);
}
});
}
function refreshSessionProjectPaths(db) { function refreshSessionProjectPaths(db) {
const sessions = db.prepare('SELECT id, project FROM sessions').all(); const sessions = db.prepare('SELECT id, project FROM sessions').all();
const cwdStmt = db.prepare(` const cwdStmt = db.prepare(`
@@ -452,6 +1025,24 @@ function rebuildFts(db) {
db.exec("INSERT INTO memories_fts(memories_fts) VALUES('rebuild')"); db.exec("INSERT INTO memories_fts(memories_fts) VALUES('rebuild')");
} }
function checkpointDb(db) {
try {
db.pragma('wal_checkpoint(TRUNCATE)');
} catch {}
}
const MESSAGE_FTS_TRIGGERS = [
'messages_fts_ai',
'messages_fts_ad',
'messages_fts_au',
];
function dropMessageFtsTriggers(db) {
for (const trigger of MESSAGE_FTS_TRIGGERS) {
db.exec(`DROP TRIGGER IF EXISTS ${trigger}`);
}
}
function ensureFtsReady(db, { force = false } = {}) { function ensureFtsReady(db, { force = false } = {}) {
const marker = '__fts_triggers_ready__'; const marker = '__fts_triggers_ready__';
const ready = db.prepare('SELECT jsonl_path FROM index_state WHERE jsonl_path = ?').get(marker); const ready = db.prepare('SELECT jsonl_path FROM index_state WHERE jsonl_path = ?').get(marker);
@@ -477,16 +1068,25 @@ function writeHeartbeat({ dbPath = DEFAULT_DB_PATH, DatabaseImpl = Database } =
function buildIndex({ function buildIndex({
claudeDir = DEFAULT_CLAUDE_DIR, claudeDir = DEFAULT_CLAUDE_DIR,
codexDir = path.join(path.dirname(claudeDir), '.codex'),
projectsDir = path.join(claudeDir, 'projects'), projectsDir = path.join(claudeDir, 'projects'),
historyPath = path.join(claudeDir, 'history.jsonl'), historyPath = path.join(claudeDir, 'history.jsonl'),
dbPath = path.join(claudeDir, 'obelisk.sqlite'), dbPath = DEFAULT_DB_PATH,
schemaPath = resolveSchemaPath(), schemaPath = resolveSchemaPath(),
DatabaseImpl = Database, DatabaseImpl = Database,
force = false, force = false,
changedPaths = undefined, changedPaths = undefined,
preserveDbPath = null,
} = {}) { } = {}) {
const db = openIndexDb({ dbPath, schemaPath, DatabaseImpl }); const db = openIndexDb({ dbPath, schemaPath, DatabaseImpl });
const files = discoverJsonlFiles({ projectsDir, changedPaths: force ? undefined : changedPaths }); let messageFtsTriggersDropped = false;
if (preserveDbPath && path.resolve(preserveDbPath) !== path.resolve(dbPath)) {
copyMemoriesFromDb(db, preserveDbPath);
}
const files = [
...discoverJsonlFiles({ projectsDir, changedPaths: force ? undefined : changedPaths }),
...discoverCodexJsonlFiles({ codexDir, changedPaths: force ? undefined : changedPaths }),
];
const latestSourceMtime = files.reduce((latest, file) => { const latestSourceMtime = files.reduce((latest, file) => {
try { try {
return Math.max(latest, fs.statSync(file.path).mtimeMs); return Math.max(latest, fs.statSync(file.path).mtimeMs);
@@ -497,6 +1097,8 @@ function buildIndex({
try { try {
if (force) { if (force) {
dropMessageFtsTriggers(db);
messageFtsTriggersDropped = true;
db.prepare("DELETE FROM index_state WHERE substr(jsonl_path, 1, 2) != '__'").run(); db.prepare("DELETE FROM index_state WHERE substr(jsonl_path, 1, 2) != '__'").run();
db.prepare("DELETE FROM messages").run(); db.prepare("DELETE FROM messages").run();
db.prepare("DELETE FROM tool_calls").run(); db.prepare("DELETE FROM tool_calls").run();
@@ -517,9 +1119,9 @@ function buildIndex({
for (const file of files) { for (const file of files) {
db.exec('BEGIN'); db.exec('BEGIN');
try { try {
const indexed = indexJsonl(db, file); const indexed = file.source === 'codex' ? indexCodexJsonl(db, file) : indexJsonl(db, file);
if (indexed?.sessionId) affectedSessionIds.add(indexed.sessionId); if (indexed?.sessionId) affectedSessionIds.add(indexed.sessionId);
indexSubagentMeta(db, file); if (file.source !== 'codex') indexSubagentMeta(db, file);
db.exec('COMMIT'); db.exec('COMMIT');
} catch (error) { } catch (error) {
db.exec('ROLLBACK'); db.exec('ROLLBACK');
@@ -532,6 +1134,8 @@ function buildIndex({
indexWorkflows(db, { projectsDir }); indexWorkflows(db, { projectsDir });
refreshSessionProjectPaths(db); refreshSessionProjectPaths(db);
indexHistory(db, { historyPath }); indexHistory(db, { historyPath });
indexCodexSessionIndex(db, { codexDir });
if (messageFtsTriggersDropped) installSchema(db, schemaPath);
ftsRebuilt = ensureFtsReady(db, { force }); ftsRebuilt = ensureFtsReady(db, { force });
writeIndexMarker(db, '__last_build__'); writeIndexMarker(db, '__last_build__');
writeIndexMarker(db, '__app_heartbeat__'); writeIndexMarker(db, '__app_heartbeat__');
@@ -545,6 +1149,14 @@ function buildIndex({
} }
return { files: files.length, latestSourceMtime, affectedSessionIds: [...affectedSessionIds], ftsRebuilt }; return { files: files.length, latestSourceMtime, affectedSessionIds: [...affectedSessionIds], ftsRebuilt };
} finally { } finally {
if (messageFtsTriggersDropped) {
try {
installSchema(db, schemaPath);
} catch (error) {
console.warn(`Warning: failed to restore message FTS triggers: ${error.message}`);
}
}
checkpointDb(db);
db.close(); db.close();
} }
} }
+262 -40
View File
@@ -31,6 +31,7 @@ function detectClaudeDir() {
} }
const DEFAULT_CLAUDE_DIR = detectClaudeDir(); const DEFAULT_CLAUDE_DIR = detectClaudeDir();
const DEFAULT_CODEX_DIR = path.join(os.homedir(), '.codex');
let db; let db;
let indexerService; let indexerService;
@@ -41,14 +42,106 @@ function getConfiguredClaudeDir() {
return persisted.claudeDir || DEFAULT_CLAUDE_DIR; return persisted.claudeDir || DEFAULT_CLAUDE_DIR;
} }
function getPathsForClaudeDir(claudeDir = getConfiguredClaudeDir()) { function getConfiguredCodexDir() {
const persisted = loadPersistedSettings();
return persisted.codexDir || DEFAULT_CODEX_DIR;
}
function getPathsForClaudeDir(claudeDir = getConfiguredClaudeDir(), codexDir = getConfiguredCodexDir()) {
return { return {
claudeDir, claudeDir,
dbPath: path.join(claudeDir, 'obelisk.sqlite'), codexDir,
dbPath: path.join(OBELISK_DIR, 'obelisk.sqlite'),
projectsDir: path.join(claudeDir, 'projects'), projectsDir: path.join(claudeDir, 'projects'),
}; };
} }
function migrateLegacyDbIfNeeded(paths = getPathsForClaudeDir()) {
if (fs.existsSync(paths.dbPath)) return;
const legacyDbPath = path.join(paths.claudeDir, 'obelisk.sqlite');
if (!fs.existsSync(legacyDbPath)) return;
try {
fs.mkdirSync(path.dirname(paths.dbPath), { recursive: true });
fs.copyFileSync(legacyDbPath, paths.dbPath);
} catch (error) {
console.warn?.(`Obelisk legacy DB migration skipped: ${error.message}`);
}
}
function rebuildTempDbPath(dbPath) {
return path.join(
path.dirname(dbPath),
`${path.basename(dbPath)}.rebuild-${process.pid}-${Date.now()}.tmp`,
);
}
function dbFileSet(dbPath) {
return [dbPath, `${dbPath}-wal`, `${dbPath}-shm`];
}
function cleanupDbFiles(dbPath) {
for (const filePath of dbFileSet(dbPath)) {
try {
fs.rmSync(filePath, { force: true });
} catch {}
}
}
function replaceDbWithTemp(tempDbPath, dbPath) {
fs.mkdirSync(path.dirname(dbPath), { recursive: true });
for (const sidecar of [`${dbPath}-wal`, `${dbPath}-shm`]) {
try {
fs.rmSync(sidecar, { force: true });
} catch {}
}
fs.renameSync(tempDbPath, dbPath);
for (const suffix of ['-wal', '-shm']) {
const tempSidecar = `${tempDbPath}${suffix}`;
if (!fs.existsSync(tempSidecar)) continue;
fs.renameSync(tempSidecar, `${dbPath}${suffix}`);
}
}
function resolveSchemaPath() {
const candidates = [
path.join(__dirname, 'schema.sql'),
path.join(__dirname, '..', 'scripts', 'schema.sql'),
process.resourcesPath ? path.join(process.resourcesPath, 'scripts', 'schema.sql') : null,
].filter(Boolean);
return candidates.find(p => fs.existsSync(p));
}
function ensureColumn(db, table, column, definition) {
const columns = db.prepare(`PRAGMA table_info(${table})`).all().map(c => c.name);
if (!columns.includes(column)) db.exec(`ALTER TABLE ${table} ADD COLUMN ${column} ${definition}`);
}
function tableExists(db, table) {
return Boolean(db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name=?").get(table));
}
function migrateExistingColumns(db) {
if (tableExists(db, 'sessions')) ensureColumn(db, 'sessions', 'source', "TEXT DEFAULT 'claude'");
if (tableExists(db, 'messages')) {
ensureColumn(db, 'messages', 'content_type', 'TEXT');
ensureColumn(db, 'messages', 'is_meta', 'INTEGER DEFAULT 0');
ensureColumn(db, 'messages', 'source', "TEXT DEFAULT 'claude'");
}
if (tableExists(db, 'memories')) {
ensureColumn(db, 'memories', 'anchors', 'TEXT');
ensureColumn(db, 'memories', 'deleted_at', 'TEXT');
ensureColumn(db, 'memories', 'deleted_reason', 'TEXT');
}
}
function migrateDb(db) {
if (typeof db.exec !== 'function' || typeof db.prepare !== 'function') return;
migrateExistingColumns(db);
const schemaPath = resolveSchemaPath();
if (schemaPath) db.exec(fs.readFileSync(schemaPath, 'utf8'));
migrateExistingColumns(db);
}
function closeDb() { function closeDb() {
if (db) db.close(); if (db) db.close();
db = null; db = null;
@@ -59,6 +152,7 @@ function openDb(dbPath = getPathsForClaudeDir().dbPath) {
if (!fs.existsSync(dbPath)) return null; if (!fs.existsSync(dbPath)) return null;
db = new Database(dbPath, { readonly: false }); db = new Database(dbPath, { readonly: false });
db.pragma('journal_mode = WAL'); db.pragma('journal_mode = WAL');
migrateDb(db);
return db; return db;
} }
@@ -75,15 +169,30 @@ function notifyIndexUpdated(result = {}) {
} }
} }
function sourceWhereClause(opts = {}, column = 'source') {
if (opts.includeCodex || opts.source === 'all') return { sql: '', params: [] };
if (opts.source) return { sql: `COALESCE(${column}, 'claude') = ?`, params: [opts.source] };
return { sql: `COALESCE(${column}, 'claude') = 'claude'`, params: [] };
}
function appendWhere(sql, params, clause) {
if (!clause) return sql;
return `${sql}${sql.includes(' WHERE ') ? ' AND ' : ' WHERE '}${clause}`;
}
function startIndexerService({ buildOnStart = false } = {}) { function startIndexerService({ buildOnStart = false } = {}) {
const paths = getPathsForClaudeDir(); const paths = getPathsForClaudeDir();
migrateLegacyDbIfNeeded(paths);
const codexSessionsDir = path.join(paths.codexDir, 'sessions');
indexerService = createIndexerService({ indexerService = createIndexerService({
projectsDir: paths.projectsDir, projectsDir: paths.projectsDir,
watchDirs: [paths.projectsDir, codexSessionsDir],
buildIndex: async ({ reason, changedPaths }) => { buildIndex: async ({ reason, changedPaths }) => {
const result = await indexerWorker.buildIndex({ const result = await indexerWorker.buildIndex({
reason, reason,
changedPaths, changedPaths,
claudeDir: paths.claudeDir, claudeDir: paths.claudeDir,
codexDir: paths.codexDir,
projectsDir: paths.projectsDir, projectsDir: paths.projectsDir,
dbPath: paths.dbPath, dbPath: paths.dbPath,
}); });
@@ -99,7 +208,9 @@ function startIndexerService({ buildOnStart = false } = {}) {
function startBackgroundResources({ runStartupBuild = false } = {}) { function startBackgroundResources({ runStartupBuild = false } = {}) {
if (!indexerWorker) indexerWorker = createWorkerBuildIndex(); if (!indexerWorker) indexerWorker = createWorkerBuildIndex();
openDb(); const paths = getPathsForClaudeDir();
migrateLegacyDbIfNeeded(paths);
openDb(paths.dbPath);
if (!indexerService) { if (!indexerService) {
const service = startIndexerService({ buildOnStart: false }); const service = startIndexerService({ buildOnStart: false });
if (runStartupBuild) service.runBuildNow('startup'); if (runStartupBuild) service.runBuildNow('startup');
@@ -107,11 +218,11 @@ function startBackgroundResources({ runStartupBuild = false } = {}) {
if (!obeliskWatcher) startObeliskWatcher(); if (!obeliskWatcher) startObeliskWatcher();
} }
async function stopIndexerServiceAndWait() { async function stopIndexerServiceAndWait({ waitForIdle = true } = {}) {
const service = indexerService; const service = indexerService;
if (!service) return; if (!service) return;
service.stop(); service.stop();
if (typeof service.idle === 'function') await service.idle(); if (waitForIdle && typeof service.idle === 'function') await service.idle();
if (indexerService === service) indexerService = null; if (indexerService === service) indexerService = null;
} }
@@ -157,7 +268,7 @@ function createWindow() {
}); });
if (isDev) { if (isDev) {
win.loadURL('http://localhost:5173'); win.loadURL(process.env.OBELISK_DEV_SERVER_URL || 'http://localhost:5173');
if (shouldOpenDevTools) { if (shouldOpenDevTools) {
win.webContents.openDevTools(); win.webContents.openDevTools();
} }
@@ -225,9 +336,14 @@ app.on('window-all-closed', () => {
ipcMain.handle('db:getSessions', (_, opts = {}) => { ipcMain.handle('db:getSessions', (_, opts = {}) => {
if (!db) return []; if (!db) return [];
const { project, limit = 200 } = opts; const { project, limit = 200 } = opts;
let sql = `SELECT id, title, project, project_path, started_at, ended_at, git_branch, version, message_count, jsonl_path FROM sessions`; let sql = `SELECT id, title, project, project_path, started_at, ended_at, git_branch, version, message_count, jsonl_path, source FROM sessions`;
const params = []; const params = [];
if (project) { sql += ` WHERE project LIKE ?`; params.push(project); } const sourceFilter = sourceWhereClause(opts);
if (sourceFilter.sql) {
sql = appendWhere(sql, params, sourceFilter.sql);
params.push(...sourceFilter.params);
}
if (project) { sql = appendWhere(sql, params, `project LIKE ?`); params.push(project); }
sql += ` ORDER BY COALESCE(ended_at, started_at) DESC LIMIT ?`; sql += ` ORDER BY COALESCE(ended_at, started_at) DESC LIMIT ?`;
params.push(limit); params.push(limit);
return db.prepare(sql).all(...params); return db.prepare(sql).all(...params);
@@ -238,8 +354,8 @@ ipcMain.handle('db:getSessionMessages', (_, sessionId) => {
return db.prepare(` return db.prepare(`
SELECT m.uuid, m.session_id, m.type, m.parent_uuid, m.timestamp, m.role, m.text, m.model, SELECT m.uuid, m.session_id, m.type, m.parent_uuid, m.timestamp, m.role, m.text, m.model,
m.is_sidechain, m.agent_id, m.input_tokens, m.output_tokens, m.cwd, m.skill, m.turn_duration_ms, m.is_sidechain, m.agent_id, m.input_tokens, m.output_tokens, m.cwd, m.skill, m.turn_duration_ms,
m.content_type, m.is_meta m.content_type, m.is_meta, m.source
FROM messages m WHERE m.session_id = ? AND m.agent_id IS NULL ORDER BY m.timestamp FROM messages m WHERE m.session_id = ? AND m.agent_id IS NULL ORDER BY m.timestamp, m.uuid
`).all(sessionId); `).all(sessionId);
}); });
@@ -272,8 +388,8 @@ ipcMain.handle('db:getSubagentMessages', (_, agentId) => {
return db.prepare(` return db.prepare(`
SELECT m.uuid, m.session_id, m.type, m.parent_uuid, m.timestamp, m.role, m.text, m.model, SELECT m.uuid, m.session_id, m.type, m.parent_uuid, m.timestamp, m.role, m.text, m.model,
m.is_sidechain, m.agent_id, m.input_tokens, m.output_tokens, m.cwd, m.skill, m.turn_duration_ms, m.is_sidechain, m.agent_id, m.input_tokens, m.output_tokens, m.cwd, m.skill, m.turn_duration_ms,
m.content_type, m.is_meta m.content_type, m.is_meta, m.source
FROM messages m WHERE m.agent_id = ? ORDER BY m.timestamp FROM messages m WHERE m.agent_id = ? ORDER BY m.timestamp, m.uuid
`).all(agentId); `).all(agentId);
}); });
@@ -310,9 +426,45 @@ ipcMain.handle('db:getMemories', () => {
ipcMain.handle('db:getMessageFullText', (_, uuid) => { ipcMain.handle('db:getMessageFullText', (_, uuid) => {
if (!db) return null; if (!db) return null;
const msg = db.prepare('SELECT session_id, agent_id FROM messages WHERE uuid=?').get(uuid); const msg = db.prepare('SELECT session_id, agent_id, source FROM messages WHERE uuid=?').get(uuid);
if (!msg) return null; if (!msg) return null;
if (msg.source === 'codex' || String(uuid).startsWith('codex:')) {
const match = /^codex:([^:]+):(\d+)$/.exec(String(uuid));
if (!match) return null;
const rawThreadId = match[1];
const targetLine = Number(match[2]);
let jsonlPath = null;
if (!msg.agent_id) {
jsonlPath = db.prepare('SELECT jsonl_path FROM sessions WHERE id=?').get(msg.session_id)?.jsonl_path || null;
}
if (!jsonlPath) {
jsonlPath = db.prepare(`
SELECT jsonl_path FROM index_state
WHERE jsonl_path LIKE ? AND jsonl_path LIKE '%.jsonl'
ORDER BY length(jsonl_path) ASC
LIMIT 1
`).get(`%${rawThreadId}.jsonl`)?.jsonl_path || null;
}
if (!jsonlPath || !fs.existsSync(jsonlPath)) return null;
const lines = fs.readFileSync(jsonlPath, 'utf-8').split('\n').filter(Boolean);
const line = lines[targetLine - 1];
if (!line) return null;
try {
const obj = JSON.parse(line);
const payload = obj.payload || {};
if (obj.type === 'event_msg') {
if (typeof payload.message === 'string') return payload.message;
if (typeof payload.text === 'string') return payload.text;
}
if (obj.type === 'response_item' && payload.type === 'message' && Array.isArray(payload.content)) {
const parts = payload.content.map(b => b.text).filter(Boolean);
return parts.join('\n') || null;
}
} catch {}
return null;
}
// Resolve JSONL path // Resolve JSONL path
let jsonlPath = null; let jsonlPath = null;
if (msg.agent_id) { if (msg.agent_id) {
@@ -378,58 +530,68 @@ ipcMain.handle('db:restoreMemory', (_, id) => {
return true; return true;
}); });
ipcMain.handle('db:getProjects', () => { ipcMain.handle('db:getProjects', (_, opts = {}) => {
if (!db) return []; if (!db) return [];
const sourceFilter = sourceWhereClause(opts);
const where = sourceFilter.sql ? `WHERE ${sourceFilter.sql}` : '';
return db.prepare(` return db.prepare(`
SELECT project, project_path, COUNT(*) as session_count, SELECT project, project_path, COUNT(*) as session_count,
MAX(COALESCE(ended_at, started_at)) as last_active MAX(COALESCE(ended_at, started_at)) as last_active
FROM sessions WHERE project IS NOT NULL FROM sessions ${where ? `${where} AND` : 'WHERE'} project IS NOT NULL
GROUP BY project ORDER BY last_active DESC GROUP BY project ORDER BY last_active DESC
`).all(); `).all(...sourceFilter.params);
}); });
ipcMain.handle('db:getStats', () => { ipcMain.handle('db:getStats', (_, opts = {}) => {
if (!db) return { sessions: 0, memories: 0, memoriesArchived: 0 }; if (!db) return { sessions: 0, memories: 0, memoriesArchived: 0 };
const sessions = db.prepare('SELECT COUNT(*) as c FROM sessions').get()?.c || 0; const sourceFilter = sourceWhereClause(opts);
const where = sourceFilter.sql ? `WHERE ${sourceFilter.sql}` : '';
const sessions = db.prepare(`SELECT COUNT(*) as c FROM sessions ${where}`).get(...sourceFilter.params)?.c || 0;
const memories = db.prepare('SELECT COUNT(*) as c FROM memories WHERE deleted_at IS NULL').get()?.c || 0; const memories = db.prepare('SELECT COUNT(*) as c FROM memories WHERE deleted_at IS NULL').get()?.c || 0;
const memoriesArchived = db.prepare('SELECT COUNT(*) as c FROM memories WHERE deleted_at IS NOT NULL').get()?.c || 0; const memoriesArchived = db.prepare('SELECT COUNT(*) as c FROM memories WHERE deleted_at IS NOT NULL').get()?.c || 0;
return { sessions, memories, memoriesArchived }; return { sessions, memories, memoriesArchived };
}); });
ipcMain.handle('db:getUsageStats', () => { ipcMain.handle('db:getUsageStats', (_, opts = {}) => {
if (!db) return { daily: [], totalTokens: 0, peakDay: null, longestTurn: null }; if (!db) return { daily: [], totalTokens: 0, peakDay: null, longestTurn: null };
const sourceFilter = sourceWhereClause(opts, 'source');
const sourceSql = sourceFilter.sql ? `AND ${sourceFilter.sql}` : '';
const daily = db.prepare(` const daily = db.prepare(`
SELECT DATE(timestamp) as day, SELECT DATE(timestamp) as day,
SUM(COALESCE(input_tokens,0) + COALESCE(output_tokens,0)) as tokens SUM(COALESCE(input_tokens,0) + COALESCE(output_tokens,0)) as tokens
FROM messages FROM messages
WHERE timestamp IS NOT NULL AND (input_tokens IS NOT NULL OR output_tokens IS NOT NULL) WHERE timestamp IS NOT NULL AND (input_tokens IS NOT NULL OR output_tokens IS NOT NULL)
${sourceSql}
GROUP BY DATE(timestamp) GROUP BY DATE(timestamp)
ORDER BY day ORDER BY day
`).all(); `).all(...sourceFilter.params);
const totalTokens = db.prepare(` const totalTokens = db.prepare(`
SELECT SUM(COALESCE(input_tokens,0) + COALESCE(output_tokens,0)) as total SELECT SUM(COALESCE(input_tokens,0) + COALESCE(output_tokens,0)) as total
FROM messages FROM messages
`).get()?.total || 0; ${sourceFilter.sql ? `WHERE ${sourceFilter.sql}` : ''}
`).get(...sourceFilter.params)?.total || 0;
const peakDay = db.prepare(` const peakDay = db.prepare(`
SELECT DATE(timestamp) as day, SELECT DATE(timestamp) as day,
SUM(COALESCE(input_tokens,0) + COALESCE(output_tokens,0)) as tokens SUM(COALESCE(input_tokens,0) + COALESCE(output_tokens,0)) as tokens
FROM messages FROM messages
WHERE timestamp IS NOT NULL AND (input_tokens IS NOT NULL OR output_tokens IS NOT NULL) WHERE timestamp IS NOT NULL AND (input_tokens IS NOT NULL OR output_tokens IS NOT NULL)
${sourceSql}
GROUP BY DATE(timestamp) GROUP BY DATE(timestamp)
ORDER BY tokens DESC ORDER BY tokens DESC
LIMIT 1 LIMIT 1
`).get() || null; `).get(...sourceFilter.params) || null;
const longestTurn = db.prepare(` const longestTurn = db.prepare(`
SELECT turn_duration_ms, uuid, session_id, timestamp SELECT turn_duration_ms, uuid, session_id, timestamp
FROM messages FROM messages
WHERE turn_duration_ms IS NOT NULL WHERE turn_duration_ms IS NOT NULL
${sourceSql}
ORDER BY turn_duration_ms DESC ORDER BY turn_duration_ms DESC
LIMIT 1 LIMIT 1
`).get() || null; `).get(...sourceFilter.params) || null;
return { daily, totalTokens, peakDay, longestTurn }; return { daily, totalTokens, peakDay, longestTurn };
}); });
@@ -539,32 +701,66 @@ function savePersistedSettings(settings) {
ipcMain.handle('settings:get', () => { ipcMain.handle('settings:get', () => {
const persisted = loadPersistedSettings(); const persisted = loadPersistedSettings();
const { claudeDir, dbPath: dbFile } = getPathsForClaudeDir(persisted.claudeDir || DEFAULT_CLAUDE_DIR); const { claudeDir, codexDir, dbPath: dbFile } = getPathsForClaudeDir(
persisted.claudeDir || DEFAULT_CLAUDE_DIR,
persisted.codexDir || DEFAULT_CODEX_DIR,
);
const recapDir = persisted.recapDir || RECAP_DIR; const recapDir = persisted.recapDir || RECAP_DIR;
const exists = fs.existsSync(claudeDir); const claudeExists = fs.existsSync(claudeDir);
let sessionCount = 0; const codexExists = fs.existsSync(codexDir);
let claudeSessionCount = 0;
let codexSessionCount = 0;
let memoryCount = 0; let memoryCount = 0;
let lastIndexed = ''; let claudeLastIndexed = '';
let codexLastIndexed = '';
if (db) { if (db) {
try { try {
sessionCount = db.prepare('SELECT COUNT(*) as c FROM sessions').get()?.c || 0; claudeSessionCount = db.prepare("SELECT COUNT(*) as c FROM sessions WHERE COALESCE(source, 'claude') = 'claude'").get()?.c || 0;
codexSessionCount = db.prepare("SELECT COUNT(*) as c FROM sessions WHERE source = 'codex'").get()?.c || 0;
memoryCount = db.prepare('SELECT COUNT(*) as c FROM memories WHERE deleted_at IS NULL').get()?.c || 0; memoryCount = db.prepare('SELECT COUNT(*) as c FROM memories WHERE deleted_at IS NULL').get()?.c || 0;
const latest = db.prepare('SELECT MAX(started_at) as t FROM sessions').get(); const claudeLatest = db.prepare("SELECT MAX(started_at) as t FROM sessions WHERE COALESCE(source, 'claude') = 'claude'").get();
lastIndexed = latest?.t || ''; claudeLastIndexed = claudeLatest?.t || '';
const codexLatest = db.prepare("SELECT MAX(started_at) as t FROM sessions WHERE source = 'codex'").get();
codexLastIndexed = codexLatest?.t || '';
} catch {} } catch {}
} }
return { return {
claudeDir, claudeDir,
codexDir,
dbPath: dbFile, dbPath: dbFile,
recapDir, recapDir,
autoRefresh: persisted.autoRefresh !== false, autoRefresh: persisted.autoRefresh !== false,
sessionCount, sources: [
{
id: 'claude',
name: 'Claude Code',
vendor: 'Anthropic',
path: claudeDir,
exists: claudeExists,
sessionCount: claudeSessionCount,
lastIndexed: claudeLastIndexed,
status: claudeExists ? 'ok' : 'error',
statusText: claudeExists ? 'Connected' : 'Folder not found',
},
{
id: 'codex',
name: 'Codex',
vendor: 'OpenAI',
path: codexDir,
exists: codexExists,
sessionCount: codexSessionCount,
lastIndexed: codexLastIndexed,
status: codexExists ? (codexSessionCount > 0 ? 'ok' : 'warn') : 'error',
statusText: codexExists ? (codexSessionCount > 0 ? 'Connected' : 'No sessions found') : 'Folder not found',
},
],
memoryCount, memoryCount,
lastIndexed, sessionCount: claudeSessionCount + codexSessionCount,
status: exists ? 'ok' : 'error', lastIndexed: claudeLastIndexed,
statusText: exists ? 'Connected' : 'Folder not found', status: claudeExists ? 'ok' : 'error',
statusText: claudeExists ? 'Connected' : 'Folder not found',
}; };
}); });
@@ -586,9 +782,14 @@ ipcMain.handle('settings:set', async (_, key, value) => {
} }
} }
if (key === 'claudeDir') { if (key === 'claudeDir' || key === 'codexDir') {
await stopIndexerServiceAndWait(); await stopIndexerServiceAndWait();
openDb(); const paths = getPathsForClaudeDir(
persisted.claudeDir || DEFAULT_CLAUDE_DIR,
persisted.codexDir || DEFAULT_CODEX_DIR,
);
migrateLegacyDbIfNeeded(paths);
openDb(paths.dbPath);
if (persisted.autoRefresh !== false) { if (persisted.autoRefresh !== false) {
startIndexerService({ buildOnStart: true }); startIndexerService({ buildOnStart: true });
} }
@@ -615,22 +816,43 @@ ipcMain.handle('settings:revealPath', (_, p) => {
ipcMain.handle('settings:rebuildIndex', async () => { ipcMain.handle('settings:rebuildIndex', async () => {
if (!indexerWorker) return null; if (!indexerWorker) return null;
const persisted = loadPersistedSettings(); const persisted = loadPersistedSettings();
const paths = getPathsForClaudeDir(persisted.claudeDir || DEFAULT_CLAUDE_DIR); const paths = getPathsForClaudeDir(
persisted.claudeDir || DEFAULT_CLAUDE_DIR,
persisted.codexDir || DEFAULT_CODEX_DIR,
);
const tempDbPath = rebuildTempDbPath(paths.dbPath);
const shouldRestartWatcher = persisted.autoRefresh !== false; const shouldRestartWatcher = persisted.autoRefresh !== false;
await stopIndexerServiceAndWait(); await stopIndexerServiceAndWait({ waitForIdle: false });
closeDb(); if (indexerWorker) {
await Promise.resolve(indexerWorker.stop());
indexerWorker = createWorkerBuildIndex();
}
cleanupDbFiles(tempDbPath);
try { try {
migrateLegacyDbIfNeeded(paths);
const result = await indexerWorker.buildIndex({ const result = await indexerWorker.buildIndex({
reason: 'manual-rebuild', reason: 'manual-rebuild',
force: true, force: true,
claudeDir: paths.claudeDir, claudeDir: paths.claudeDir,
codexDir: paths.codexDir,
projectsDir: paths.projectsDir, projectsDir: paths.projectsDir,
dbPath: paths.dbPath, dbPath: tempDbPath,
preserveDbPath: fs.existsSync(paths.dbPath) ? paths.dbPath : null,
}); });
closeDb();
replaceDbWithTemp(tempDbPath, paths.dbPath);
openDb(paths.dbPath); openDb(paths.dbPath);
notifyIndexUpdated(result); notifyIndexUpdated(result);
return result; return result;
} finally { } finally {
cleanupDbFiles(tempDbPath);
if (!db) {
try {
openDb(paths.dbPath);
} catch (error) {
console.warn?.(`Obelisk DB reopen after rebuild failed: ${error.message}`);
}
}
if (shouldRestartWatcher) startIndexerService({ buildOnStart: false }); if (shouldRestartWatcher) startIndexerService({ buildOnStart: false });
} }
}); });
+1 -1
View File
@@ -5,7 +5,7 @@
"main": "main.js", "main": "main.js",
"scripts": { "scripts": {
"start": "electron .", "start": "electron .",
"dev": "electron . --dev", "dev": "node scripts/dev.js",
"dev:renderer": "vite renderer", "dev:renderer": "vite renderer",
"build:renderer": "vite build renderer", "build:renderer": "vite build renderer",
"build": "npm run build:renderer && electron-builder", "build": "npm run build:renderer && electron-builder",
+109 -2
View File
@@ -47,6 +47,11 @@ const sidebarProjectsForCurrentScope = (search = state.projectSearch) => buildSi
const sidebarProjects = computed(() => sidebarProjectsForCurrentScope()); const sidebarProjects = computed(() => sidebarProjectsForCurrentScope());
const NOISE_PROJECT_RE = /^(od-conn-test|[0-9a-f]{6,})/i;
const normalProjects = computed(() => sidebarProjects.value.filter(p => p.count > 1 || !NOISE_PROJECT_RE.test(p.label)));
const noiseProjects = computed(() => sidebarProjects.value.filter(p => p.count <= 1 && NOISE_PROJECT_RE.test(p.label)));
const showNoiseProjects = ref(false);
const totalProjectCount = computed(() => { const totalProjectCount = computed(() => {
return sidebarProjectsForCurrentScope('').length; return sidebarProjectsForCurrentScope('').length;
}); });
@@ -158,11 +163,36 @@ const keepAliveIncludes = ['SessionDetail'];
const isExportRoute = computed(() => route.name === 'RecapExport'); const isExportRoute = computed(() => route.name === 'RecapExport');
// --- Source health dots ---
const sourceDots = ref([]);
const sourceDetails = ref([]);
const showSourcePopover = ref(false);
async function loadSourceDots() {
if (!window.obelisk?.getSettings) return;
const s = await window.obelisk.getSettings();
sourceDots.value = (s.sources || []).map(src => ({ id: src.id, status: src.status }));
sourceDetails.value = s.sources || [];
}
loadSourceDots();
// --- Recap --- // --- Recap ---
const recapGenerateOpen = ref(false); const recapGenerateOpen = ref(false);
function setRecapKind(k) { function setRecapKind(k) {
router.replace({ path: '/recap', query: { kind: k } }); router.replace({ path: '/recap', query: { kind: k } });
} }
// --- Source filter ---
const showSourceFilter = ref(false);
const sourceFilterActive = computed(() => state.sourceFilter !== 'all' && state.sourceFilter !== undefined);
const sourceFilterLabel = computed(() => {
if (!state.sourceFilter || state.sourceFilter === 'all') return 'All sources';
return state.sourceFilter === 'claude' ? 'Claude Code' : 'Codex';
});
function toggleSourceFilter() { showSourceFilter.value = !showSourceFilter.value; }
function setSourceFilter(id) {
state.sourceFilter = id;
showSourceFilter.value = false;
}
provide('recapGenerateOpen', recapGenerateOpen); provide('recapGenerateOpen', recapGenerateOpen);
</script> </script>
@@ -203,6 +233,24 @@ provide('recapGenerateOpen', recapGenerateOpen);
<rect x="15.5" y="33" width="9" height="1.6" rx="0.3" fill="#0f172a"/> <rect x="15.5" y="33" width="9" height="1.6" rx="0.3" fill="#0f172a"/>
</svg> </svg>
<span class="name">Obelisk</span> <span class="name">Obelisk</span>
<button class="source-health" title="Connected sources" @click="showSourcePopover = !showSourcePopover">
<span v-for="src in sourceDots" :key="src.id" class="h-dot" :class="src.id + '-' + src.status"></span>
</button>
<div class="sources-popover" :class="{ show: showSourcePopover }">
<div class="sp-head">Connected sources</div>
<div class="sp-list">
<button v-for="src in sourceDetails" :key="src.id" class="sp-row" @click="router.push('/settings')">
<span class="sp-dot" :class="src.id"></span>
<div class="sp-body">
<div class="sp-name">{{ src.name }} <span class="sp-count" v-if="src.sessionCount">{{ src.sessionCount }} sessions</span></div>
<div class="sp-meta" :class="src.status">{{ src.statusText }}</div>
</div>
</button>
</div>
<div class="sp-foot">
<button @click="router.push('/settings'); showSourcePopover = false">Manage in Settings </button>
</div>
</div>
</div> </div>
<div class="sidebar-section"> <div class="sidebar-section">
@@ -283,7 +331,15 @@ provide('recapGenerateOpen', recapGenerateOpen);
</div> </div>
<div class="sidebar-section projects" v-if="currentRouteType === 'sessions' || currentRouteType === 'memory'"> <div class="sidebar-section projects" v-if="currentRouteType === 'sessions' || currentRouteType === 'memory'">
<div class="sidebar-section-title"><span>Projects</span></div> <div class="sidebar-section-title">
<span>Projects</span>
<button v-if="noiseProjects.length" class="filter-toggle" :class="{ active: showNoiseProjects }" @click.stop="showNoiseProjects = !showNoiseProjects">
<svg viewBox="0 0 12 12" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round">
<path d="M2 6h8M2 3h8M2 9h5"/>
</svg>
{{ showNoiseProjects ? 'hide noise' : 'show all' }}
</button>
</div>
<div class="sidebar-search" v-if="totalProjectCount >= 6"> <div class="sidebar-search" v-if="totalProjectCount >= 6">
<svg class="sidebar-search-icon" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.6"> <svg class="sidebar-search-icon" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.6">
<circle cx="7" cy="7" r="5"/> <circle cx="7" cy="7" r="5"/>
@@ -299,7 +355,7 @@ provide('recapGenerateOpen', recapGenerateOpen);
</div> </div>
<div class="sidebar-list" id="sidebar-projects"> <div class="sidebar-list" id="sidebar-projects">
<button <button
v-for="p in sidebarProjects" v-for="p in normalProjects"
:key="p.slug" :key="p.slug"
class="sidebar-item" class="sidebar-item"
:class="{ active: state.projectFilter === p.slug }" :class="{ active: state.projectFilter === p.slug }"
@@ -311,6 +367,28 @@ provide('recapGenerateOpen', recapGenerateOpen);
<span class="label">{{ p.label }}</span> <span class="label">{{ p.label }}</span>
<span class="badge">{{ p.count }}</span> <span class="badge">{{ p.count }}</span>
</button> </button>
<!-- Noise projects fold -->
<button v-if="noiseProjects.length" class="project-fold" :class="{ expanded: showNoiseProjects }" @click="showNoiseProjects = !showNoiseProjects">
<svg class="chev" viewBox="0 0 12 12" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round"><path d="M4 2.5l3 3.5-3 3.5"/></svg>
<span class="label">{{ noiseProjects.length }} test projects hidden</span>
<span class="count">{{ noiseProjects.length }}</span>
</button>
<template v-if="showNoiseProjects">
<button
v-for="p in noiseProjects"
:key="p.slug"
class="sidebar-item noise"
:class="{ active: state.projectFilter === p.slug }"
@click="handleSidebarProject(p.slug)"
>
<svg class="icon" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linejoin="round">
<path d="M2 5.5V12a1.5 1.5 0 0 0 1.5 1.5h9A1.5 1.5 0 0 0 14 12V6.5A1.5 1.5 0 0 0 12.5 5H8.3L7 3.5H3.5A1.5 1.5 0 0 0 2 5z"/>
</svg>
<span class="label">{{ p.label }}</span>
<span class="badge">{{ p.count }}</span>
</button>
</template>
</div> </div>
</div> </div>
@@ -403,6 +481,35 @@ provide('recapGenerateOpen', recapGenerateOpen);
</button> </button>
</template> </template>
<!-- Source filter (session list only, multi-source) -->
<div v-if="showToolbar && route.name === 'SessionList' && sourceDots.length > 1" class="source-filter-wrap">
<button class="filter-btn" :class="{ active: sourceFilterActive }" @click="toggleSourceFilter">
<svg viewBox="0 0 12 12" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round">
<path d="M2 3h8M3.5 6h5M5 9h2"/>
</svg>
<span class="filter-label">{{ sourceFilterLabel }}</span>
</button>
<div class="filter-dropdown" :class="{ show: showSourceFilter }">
<div
v-for="src in sourceDots" :key="src.id"
class="fd-row" :class="{ checked: state.sourceFilter === 'all' || state.sourceFilter === src.id }"
@click.stop="setSourceFilter(src.id)"
>
<div class="fd-check">
<svg viewBox="0 0 12 12" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.5 6l2.5 2.5 4.5-5"/></svg>
</div>
<span class="fd-name">{{ src.id === 'claude' ? 'Claude Code' : 'Codex' }}</span>
</div>
<div class="fd-divider"></div>
<div class="fd-row" :class="{ checked: state.sourceFilter === 'all' }" @click.stop="setSourceFilter('all')">
<div class="fd-check">
<svg viewBox="0 0 12 12" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.5 6l2.5 2.5 4.5-5"/></svg>
</div>
<span class="fd-name" style="color: var(--accent-2);">All sources</span>
</div>
</div>
</div>
<div class="toolbar-search" id="search-wrap" v-if="showToolbar"> <div class="toolbar-search" id="search-wrap" v-if="showToolbar">
<svg class="toolbar-search-icon" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.6"> <svg class="toolbar-search-icon" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.6">
<circle cx="7" cy="7" r="5"/> <circle cx="7" cy="7" r="5"/>
+18 -1
View File
@@ -11,7 +11,7 @@ import { state, clearUndo } from './store.js';
export async function loadInitialData() { export async function loadInitialData() {
const [rawMemories, rawSessions, stats, projects] = await Promise.all([ const [rawMemories, rawSessions, stats, projects] = await Promise.all([
window.obelisk.getMemories(), window.obelisk.getMemories(),
window.obelisk.getSessions(), window.obelisk.getSessions({ source: 'all', limit: 1000 }),
window.obelisk.getStats(), window.obelisk.getStats(),
window.obelisk.getProjects() window.obelisk.getProjects()
]); ]);
@@ -213,6 +213,23 @@ export async function loadSessionDetail(sessionId) {
} else { } else {
const out = { ...msg }; const out = { ...msg };
if (msg._thinking) out._thinking = msg._thinking; if (msg._thinking) out._thinking = msg._thinking;
// For text assistant messages, absorb following tool_use messages (Codex pattern)
if (msg.type === 'assistant' && msg.content_type !== 'tool_use' && msg.content_type !== 'thinking') {
if (!out.tool_calls) out.tool_calls = [];
let j = i + 1;
while (j < rawAssembled.length) {
const next = rawAssembled[j];
if (next.content_type === 'tool_result') { j++; continue; }
if (next.type === 'assistant' && next.content_type === 'tool_use') {
if (next.tool_calls) out.tool_calls.push(...next.tool_calls);
j++;
continue;
}
break;
}
if (!out.tool_calls.length) delete out.tool_calls;
i = j - 1;
}
assembledMessages.push(out); assembledMessages.push(out);
} }
} }
+1
View File
@@ -17,6 +17,7 @@ export const state = reactive({
pendingFocusUuid: null, pendingFocusUuid: null,
query: '', query: '',
projectFilter: 'all', projectFilter: 'all',
sourceFilter: 'all',
projectSearch: '', projectSearch: '',
sortDesc: true, sortDesc: true,
includeMessageBodies: false, includeMessageBodies: false,
+4
View File
@@ -615,6 +615,10 @@ function getToolCallParsedInput(tc) {
<span class="project-name">{{ formatProjectLabel(session.project) }}</span> <span class="project-name">{{ formatProjectLabel(session.project) }}</span>
<span class="sep">&middot;</span> <span class="sep">&middot;</span>
<span class="project-path">{{ session.project_path || '' }}</span> <span class="project-path">{{ session.project_path || '' }}</span>
<span class="via">
<span class="via-dot" :class="session.source || 'claude'"></span>
via {{ (session.source || 'claude') === 'codex' ? 'Codex' : 'Claude Code' }}
</span>
</div> </div>
<div class="session-title">{{ session.title || '(untitled)' }}</div> <div class="session-title">{{ session.title || '(untitled)' }}</div>
<div class="session-meta-inline"> <div class="session-meta-inline">
+113 -1
View File
@@ -23,6 +23,7 @@ const visibleSessions = computed(() => {
const q = state.query.trim().toLowerCase(); const q = state.query.trim().toLowerCase();
return state.sessions return state.sessions
.filter(s => state.projectFilter === 'all' || s.project === state.projectFilter) .filter(s => state.projectFilter === 'all' || s.project === state.projectFilter)
.filter(s => state.sourceFilter === 'all' || (s.source || 'claude') === state.sourceFilter)
.map(s => { .map(s => {
if (!q) return { ...s, messageHit: null }; if (!q) return { ...s, messageHit: null };
const topMatch = (s.title || '').toLowerCase().includes(q) || const topMatch = (s.title || '').toLowerCase().includes(q) ||
@@ -40,6 +41,14 @@ const visibleSessions = computed(() => {
}); });
const showProjectPrefix = computed(() => state.projectFilter === 'all'); const showProjectPrefix = computed(() => state.projectFilter === 'all');
const showNoise = ref(false);
function isNoise(s) {
return !s.title;
}
const normalSessions = computed(() => visibleSessions.value.filter(s => !isNoise(s)));
const noiseSessions = computed(() => visibleSessions.value.filter(s => isNoise(s)));
function titleHTML(session) { function titleHTML(session) {
return highlightPlain(session.title || '(untitled)', state.query.trim()); return highlightPlain(session.title || '(untitled)', state.query.trim());
@@ -135,7 +144,7 @@ function obeliskStyle(session) {
<div v-else class="session-list"> <div v-else class="session-list">
<div <div
v-for="s in visibleSessions" v-for="s in normalSessions"
:key="s.id" :key="s.id"
class="srow" class="srow"
:class="{ cursor: state.cursorId === s.id }" :class="{ cursor: state.cursorId === s.id }"
@@ -155,6 +164,48 @@ function obeliskStyle(session) {
</div> </div>
<div class="srow-right">{{ timeLabel(s) }}</div> <div class="srow-right">{{ timeLabel(s) }}</div>
</div> </div>
<!-- Noise fold banner -->
<div v-if="noiseSessions.length && !state.query" class="fold-banner" :class="{ expanded: showNoise }" @click="showNoise = !showNoise">
<svg class="chev" viewBox="0 0 12 12" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round">
<path d="M4 2.5l3 3.5-3 3.5"/>
</svg>
<div class="body">
<strong>{{ noiseSessions.length }}</strong> quiet sessions hidden untitled, likely tests or incomplete runs.
</div>
<span v-if="!showNoise" class="reveal-link">Show all</span>
</div>
<!-- Noise sessions (collapsed by default) -->
<div v-if="showNoise && noiseSessions.length" class="noise-group">
<div class="noise-group-head">
{{ noiseSessions.length }} sessions · untitled
</div>
<div
v-for="s in noiseSessions"
:key="s.id"
class="srow noise"
@click="openSession(s)"
>
<div class="srow-body">
<div class="srow-title">(untitled)</div>
<div class="srow-meta">
<template v-if="showProjectPrefix">
<span class="project-tag" v-html="projectLabel(s)"></span>
<span class="dot"></span>
</template>
<span>{{ s.message_count || 0 }} msg</span>
</div>
</div>
<div class="srow-right">{{ timeLabel(s) }}</div>
</div>
<button class="noise-fold-bottom" @click.stop="showNoise = false">
<svg class="chev" viewBox="0 0 12 12" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round">
<path d="M4 2.5l3 3.5-3 3.5"/>
</svg>
Collapse
</button>
</div>
</div> </div>
</div> </div>
</template> </template>
@@ -333,4 +384,65 @@ function obeliskStyle(session) {
font-family: var(--font-mono); color: var(--fg-2); font-family: var(--font-mono); color: var(--fg-2);
background: rgba(0,0,0,0.3); padding: 1px 6px; border-radius: 3px; background: rgba(0,0,0,0.3); padding: 1px 6px; border-radius: 3px;
} }
/* Noise fold */
.fold-banner {
display: flex; align-items: center; gap: 12px;
padding: 10px 22px;
background: rgba(255,255,255,0.015);
border-top: 1px solid var(--hairline);
border-bottom: 1px solid var(--hairline);
font-size: 12.5px; color: var(--muted);
cursor: pointer; transition: all 0.1s;
}
.fold-banner:hover { background: rgba(255,255,255,0.03); color: var(--fg-2); }
.fold-banner.expanded { color: var(--fg-3); background: rgba(255,255,255,0.02); }
.fold-banner .chev {
width: 10px; height: 10px; color: var(--muted-2);
transition: transform 0.15s; flex-shrink: 0;
}
.fold-banner.expanded .chev { transform: rotate(90deg); color: var(--accent-2); }
.fold-banner .body { flex: 1; }
.fold-banner .body strong {
color: var(--fg-2); font-weight: 500;
font-variant-numeric: tabular-nums;
font-family: var(--font-mono); font-size: 11.5px;
}
.fold-banner .reveal-link {
font-size: 11.5px; color: var(--accent-2);
text-decoration: none; border-bottom: 1px solid rgba(167,139,250,0.4);
padding-bottom: 1px; transition: all 0.12s; flex-shrink: 0;
}
.fold-banner:hover .reveal-link { color: var(--accent); border-bottom-color: var(--accent); }
.noise-group {
border-bottom: 1px solid var(--hairline-strong);
background: rgba(0,0,0,0.15);
}
.noise-group-head {
padding: 6px 22px;
font-family: var(--font-mono); font-size: 10px; color: var(--muted-2);
letter-spacing: 0.06em; text-transform: uppercase;
background: rgba(0,0,0,0.1); border-bottom: 1px solid var(--hairline);
}
.srow.noise { padding: 8px 22px 8px 18px; }
.srow.noise .srow-title {
color: var(--muted); font-style: italic;
font-size: 13px; font-weight: 400;
}
.srow.noise .srow-meta { color: var(--muted-2); }
.noise-fold-bottom {
padding: 8px 22px; background: rgba(0,0,0,0.2);
font-family: var(--font-mono); font-size: 11px; color: var(--muted);
cursor: pointer; transition: all 0.1s;
display: flex; align-items: center; gap: 8px;
border-top: 1px solid var(--hairline);
border: none; width: 100%; text-align: left;
}
.noise-fold-bottom:hover { background: rgba(0,0,0,0.3); color: var(--fg-2); }
.noise-fold-bottom .chev {
width: 9px; height: 9px; color: var(--muted-2);
transform: rotate(-90deg);
}
</style> </style>
+109 -88
View File
@@ -1,17 +1,13 @@
<script setup> <script setup>
import { ref, onMounted, watch } from 'vue'; import { ref, onMounted, nextTick } from 'vue';
defineOptions({ name: 'Settings' }); defineOptions({ name: 'Settings' });
const claudePath = ref(''); const sources = ref([]);
const dbPath = ref(''); const dbPath = ref('');
const recapPath = ref(''); const recapPath = ref('');
const autoRefresh = ref(true); const autoRefresh = ref(true);
const status = ref('ok');
const statusText = ref('Connected');
const sessionCount = ref(0);
const memoryCount = ref(0); const memoryCount = ref(0);
const lastIndexed = ref('');
const rebuilding = ref(false); const rebuilding = ref(false);
const version = ref('0.1.0'); const version = ref('0.1.0');
@@ -22,23 +18,19 @@ onMounted(async () => {
async function loadSettings() { async function loadSettings() {
if (!window.obelisk?.getSettings) return; if (!window.obelisk?.getSettings) return;
const s = await window.obelisk.getSettings(); const s = await window.obelisk.getSettings();
claudePath.value = s.claudeDir || '~/.claude'; sources.value = s.sources || [];
dbPath.value = s.dbPath || ''; dbPath.value = s.dbPath || '';
recapPath.value = s.recapDir || '~/.obelisk/recap'; recapPath.value = s.recapDir || '~/.obelisk/recap';
autoRefresh.value = s.autoRefresh !== false; autoRefresh.value = s.autoRefresh !== false;
sessionCount.value = s.sessionCount || 0;
memoryCount.value = s.memoryCount || 0; memoryCount.value = s.memoryCount || 0;
lastIndexed.value = s.lastIndexed || '';
status.value = s.status || 'ok';
statusText.value = s.statusText || 'Connected';
} }
async function browsePath() { async function browseSourcePath(source) {
if (!window.obelisk?.browseFolder) return; if (!window.obelisk?.browseFolder) return;
const result = await window.obelisk.browseFolder(); const result = await window.obelisk.browseFolder();
if (result) { if (result) {
claudePath.value = result; const key = source.id === 'claude' ? 'claudeDir' : 'codexDir';
await saveSetting('claudeDir', result); await saveSetting(key, result);
await loadSettings(); await loadSettings();
} }
} }
@@ -52,11 +44,6 @@ async function browseRecapPath() {
} }
} }
async function resetPath() {
await saveSetting('claudeDir', null);
await loadSettings();
}
async function toggleAutoRefresh() { async function toggleAutoRefresh() {
autoRefresh.value = !autoRefresh.value; autoRefresh.value = !autoRefresh.value;
await saveSetting('autoRefresh', autoRefresh.value); await saveSetting('autoRefresh', autoRefresh.value);
@@ -68,11 +55,6 @@ async function saveSetting(key, value) {
} }
} }
async function commitClaudePath() {
await saveSetting('claudeDir', claudePath.value);
await loadSettings();
}
async function commitRecapPath() { async function commitRecapPath() {
await saveSetting('recapDir', recapPath.value); await saveSetting('recapDir', recapPath.value);
} }
@@ -80,7 +62,8 @@ async function commitRecapPath() {
async function rebuildIndex() { async function rebuildIndex() {
if (rebuilding.value || !window.obelisk?.rebuildIndex) return; if (rebuilding.value || !window.obelisk?.rebuildIndex) return;
rebuilding.value = true; rebuilding.value = true;
statusText.value = 'Rebuilding…'; await nextTick();
await new Promise(resolve => requestAnimationFrame(resolve));
try { try {
await window.obelisk.rebuildIndex(); await window.obelisk.rebuildIndex();
await loadSettings(); await loadSettings();
@@ -111,86 +94,79 @@ function fmtRelative(iso) {
<div class="settings-wrap"> <div class="settings-wrap">
<div class="settings-content"> <div class="settings-content">
<!-- Data Source --> <!-- Data Sources -->
<section class="settings-section"> <section class="settings-section">
<div class="settings-section-head"> <div class="settings-section-head">
<h2>Data Source</h2> <h2>Data Sources</h2>
<p>Where Obelisk reads your Claude Code session history.</p> <p>Where Obelisk reads your agent session history.</p>
</div> </div>
<div class="form-row"> <div
<div> v-for="src in sources" :key="src.id"
<div class="form-label">Claude Code path</div> class="source-card"
<div class="form-label-hint">Default <code>~/.claude</code> on macOS &amp; Linux.</div> :class="{ error: src.status === 'error', warn: src.status === 'warn' }"
>
<div class="source-card-head">
<div class="source-card-mark" :class="src.id">
<span class="mark-dot"></span>
</div>
<div class="source-card-info">
<div class="source-card-name">
{{ src.name }}
<span class="vendor">by {{ src.vendor }}</span>
</div>
<div class="source-card-status">
<span class="stat-dot" :class="src.status"></span>
<span class="stat-text" :class="src.status">{{ src.statusText }}</span>
<template v-if="src.lastIndexed">
<span class="sep">·</span>
<span>last read <strong>{{ fmtRelative(src.lastIndexed) }}</strong></span>
</template>
<template v-if="src.sessionCount">
<span class="sep">·</span>
<span><strong>{{ src.sessionCount }}</strong> sessions</span>
</template>
</div>
</div>
</div> </div>
<div class="form-control"> <div class="source-card-body">
<div class="path-input"> <div class="path-input">
<input <input class="path-field" :class="{ error: src.status === 'error' }" type="text" :value="src.path" spellcheck="false" readonly/>
class="path-field" <button class="btn" @click="browseSourcePath(src)">
:class="{ error: status === 'error' }"
type="text"
v-model="claudePath"
spellcheck="false"
@keydown.enter="commitClaudePath"
@blur="commitClaudePath"
/>
<button class="btn" @click="browsePath">
<svg viewBox="0 0 14 14" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linejoin="round"> <svg viewBox="0 0 14 14" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linejoin="round">
<path d="M2.5 3.5h3.5l1.2 1.2h4.3a1 1 0 0 1 1 1V11a1 1 0 0 1-1 1H2.5a1 1 0 0 1-1-1V4.5a1 1 0 0 1 1-1z"/> <path d="M2.5 3.5h3.5l1.2 1.2h4.3a1 1 0 0 1 1 1V11a1 1 0 0 1-1 1H2.5a1 1 0 0 1-1-1V4.5a1 1 0 0 1 1-1z"/>
</svg> </svg>
Browse Browse
</button> </button>
<button class="btn subtle" @click="resetPath" title="Reset to default">
<svg viewBox="0 0 14 14" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round">
<path d="M12.5 6.5A5 5 0 1 0 12 9.5"/>
<path d="M12.5 2v4.5h-4.5"/>
</svg>
</button>
</div>
<div class="status-row" :class="status">
<span class="status-dot" :class="status"></span>
<span class="status-text">{{ statusText }}</span>
<div class="status-meta" v-if="sessionCount || lastIndexed">
<template v-if="lastIndexed">
<span>last read <strong>{{ fmtRelative(lastIndexed) }}</strong></span>
<span class="sep">·</span>
</template>
<span><strong>{{ sessionCount }}</strong> sessions</span>
<span class="sep">·</span>
<span><strong>{{ memoryCount }}</strong> memories</span>
</div>
</div> </div>
</div> </div>
</div> </div>
</section>
<div class="form-row"> <!-- Index -->
<div> <section class="settings-section">
<div class="form-label">Index location</div> <div class="settings-section-head">
<div class="form-label-hint">SQLite database where Obelisk caches the session index.</div> <h2>Index location</h2>
</div> <p>SQLite database where Obelisk caches the unified session index.</p>
<div class="form-control">
<div class="path-input">
<input class="path-field" type="text" :value="dbPath" spellcheck="false" readonly/>
<button class="btn" @click="revealDb">Reveal</button>
</div>
</div>
</div> </div>
<div class="path-input" style="max-width: 480px;">
<input class="path-field" type="text" :value="dbPath" spellcheck="false" readonly/>
<button class="btn" @click="revealDb">Reveal</button>
</div>
</section>
<div class="form-row"> <!-- Auto-refresh -->
<div> <section class="settings-section">
<div class="form-label">Auto-refresh</div> <div class="settings-section-head">
<div class="form-label-hint">Obelisk re-reads when new session files appear.</div> <h2>Auto-refresh</h2>
</div> <p>Obelisk re-reads when new session files appear.</p>
<div class="form-control">
<label class="toggle-label" @click.prevent="toggleAutoRefresh">
<span class="toggle-track" :class="{ on: autoRefresh }">
<span class="toggle-thumb"></span>
</span>
<span class="toggle-text">Watch <code>.claude</code> for changes</span>
</label>
</div>
</div> </div>
<label class="toggle-label" @click.prevent="toggleAutoRefresh">
<span class="toggle-track" :class="{ on: autoRefresh }">
<span class="toggle-thumb"></span>
</span>
<span class="toggle-text">Watch data sources for changes</span>
</label>
</section> </section>
<!-- Recap --> <!-- Recap -->
@@ -241,7 +217,7 @@ function fmtRelative(iso) {
</button> </button>
</div> </div>
<div class="reset-hint"> <div class="reset-hint">
Rebuilding only re-reads your Claude Code data. It does not delete memories or recaps. Rebuilding re-reads your coding agent session data. It does not delete memories or recaps.
</div> </div>
</div> </div>
</div> </div>
@@ -269,6 +245,51 @@ function fmtRelative(iso) {
font-size: 13px; color: var(--muted); font-size: 13px; color: var(--muted);
} }
/* Source cards */
.source-card {
padding: 18px; border: 1px solid var(--hairline); border-radius: 8px;
background: rgba(0,0,0,0.18); margin-bottom: 12px;
transition: border-color 0.15s;
}
.source-card:hover { border-color: var(--hairline-strong); }
.source-card.error { border-color: rgba(248,113,113,0.25); }
.source-card.warn { border-color: rgba(251,191,36,0.20); }
.source-card-head { display: flex; align-items: center; gap: 10px; margin-bottom: 14px; }
.source-card-mark {
width: 28px; height: 28px; border-radius: 6px;
background: rgba(0,0,0,0.4); border: 1px solid var(--hairline-strong);
display: grid; place-items: center; flex-shrink: 0;
}
.source-card-mark .mark-dot { width: 8px; height: 8px; border-radius: 50%; }
.source-card-mark.claude .mark-dot { background: #d97757; box-shadow: 0 0 6px rgba(217,119,87,0.5); }
.source-card-mark.codex .mark-dot { background: #10a37f; box-shadow: 0 0 6px rgba(16,163,127,0.5); }
.source-card-info { flex: 1; min-width: 0; }
.source-card-name {
font-size: 14px; color: var(--fg); font-weight: 600; letter-spacing: -0.005em;
display: flex; align-items: baseline; gap: 8px;
}
.source-card-name .vendor { font-size: 11.5px; color: var(--muted); font-weight: 400; }
.source-card-status {
font-family: var(--font-mono); font-size: 10.5px; color: var(--muted);
margin-top: 3px; display: flex; align-items: center; gap: 8px;
}
.source-card-status .stat-dot { width: 6px; height: 6px; border-radius: 50%; position: relative; }
.source-card-status .stat-dot.ok { background: #34d399; box-shadow: 0 0 5px rgba(52,211,153,0.5); }
.source-card-status .stat-dot.warn { background: #fbbf24; box-shadow: 0 0 5px rgba(251,191,36,0.5); }
.source-card-status .stat-dot.error { background: #f87171; box-shadow: 0 0 5px rgba(248,113,113,0.5); }
.source-card-status .stat-dot.ok::before {
content: ''; position: absolute; inset: -2.5px; border-radius: 50%;
border: 1px solid #34d399; opacity: 0.5; animation: src-pulse 1.6s ease-out infinite;
}
@keyframes src-pulse { 0% { transform: scale(0.8); opacity: 0.5; } 100% { transform: scale(1.8); opacity: 0; } }
.source-card-status .stat-text { color: var(--fg-2); }
.source-card-status .stat-text.ok { color: #34d399; }
.source-card-status .stat-text.warn { color: #fbbf24; }
.source-card-status .stat-text.error { color: #f87171; }
.source-card-status .sep { color: var(--muted-3); }
.source-card-status strong { color: var(--fg-2); font-weight: 500; }
.source-card-body { display: flex; flex-direction: column; gap: 10px; }
.form-row { .form-row {
display: grid; grid-template-columns: 180px 1fr; display: grid; grid-template-columns: 180px 1fr;
gap: 24px; padding: 14px 0; align-items: start; gap: 24px; padding: 14px 0; align-items: start;
+11
View File
@@ -425,6 +425,17 @@
.session-eyebrow .project-name { color: var(--fg-2); font-weight: 500; } .session-eyebrow .project-name { color: var(--fg-2); font-weight: 500; }
.session-eyebrow .project-path { font-family: var(--font-mono); color: var(--muted); } .session-eyebrow .project-path { font-family: var(--font-mono); color: var(--muted); }
.session-eyebrow .sep { color: var(--muted-2); margin: 0 2px; } .session-eyebrow .sep { color: var(--muted-2); margin: 0 2px; }
.session-eyebrow .via {
display: inline-flex; align-items: center; gap: 5px;
font-family: var(--font-mono); font-size: 10.5px; color: var(--muted);
letter-spacing: 0.02em;
padding: 1px 7px; background: rgba(255,255,255,0.04);
border: 1px solid var(--hairline); border-radius: 3px;
margin-left: 6px;
}
.session-eyebrow .via .via-dot { width: 4px; height: 4px; border-radius: 50%; }
.session-eyebrow .via .via-dot.claude { background: #d97757; box-shadow: 0 0 4px rgba(217,119,87,0.5); }
.session-eyebrow .via .via-dot.codex { background: #10a37f; box-shadow: 0 0 4px rgba(16,163,127,0.5); }
.session-title { .session-title {
font-size: 22px; font-weight: 600; color: var(--fg); font-size: 22px; font-weight: 600; color: var(--fg);
line-height: 1.3; margin-bottom: 14px; letter-spacing: -0.01em; line-height: 1.3; margin-bottom: 14px; letter-spacing: -0.01em;
+106 -3
View File
@@ -2,11 +2,12 @@
border-right: 1px solid var(--hairline-strong); border-right: 1px solid var(--hairline-strong);
background: rgba(0,0,0,0.2); background: rgba(0,0,0,0.2);
display: flex; flex-direction: column; display: flex; flex-direction: column;
min-height: 0; min-height: 0; min-width: 0; overflow: hidden;
} }
.sidebar-brand { .sidebar-brand {
display: flex; align-items: center; gap: 8px; display: flex; align-items: center; gap: 8px;
padding: 0 14px; height: 36px; padding: 0 14px; height: 36px;
position: relative;
border-bottom: 1px solid var(--hairline); border-bottom: 1px solid var(--hairline);
flex-shrink: 0; flex-shrink: 0;
} }
@@ -17,13 +18,115 @@
.sidebar-section + .sidebar-section { border-top: 1px solid var(--hairline); } .sidebar-section + .sidebar-section { border-top: 1px solid var(--hairline); }
.sidebar-spacer { flex: 1; min-height: 0; } .sidebar-spacer { flex: 1; min-height: 0; }
.sidebar-bottom { margin-top: auto; } .sidebar-bottom { margin-top: auto; }
/* Source health dots — each dot = one source, colored by brand + status */
.source-health {
display: inline-flex; align-items: center; gap: 3px;
padding: 4px 6px; border-radius: 4px; margin-left: auto;
cursor: pointer; transition: background 0.1s;
}
.source-health:hover { background: var(--surface-strong); }
.source-health .h-dot {
width: 5px; height: 5px; border-radius: 50%; flex-shrink: 0;
}
.source-health .h-dot.claude-ok { background: #d97757; box-shadow: 0 0 4px rgba(217,119,87,0.6); }
.source-health .h-dot.claude-warn { background: rgba(217,119,87,0.4); }
.source-health .h-dot.claude-error { background: rgba(217,119,87,0.25); }
.source-health .h-dot.codex-ok { background: #10a37f; box-shadow: 0 0 4px rgba(16,163,127,0.6); }
.source-health .h-dot.codex-warn { background: rgba(16,163,127,0.4); }
.source-health .h-dot.codex-error { background: rgba(16,163,127,0.25); }
.source-health .h-dot.off { background: var(--muted-3); }
/* Sources popover */
.sources-popover {
position: absolute; top: 100%; left: 0; margin-top: 6px;
width: 260px; background: rgba(20, 22, 38, 0.98);
backdrop-filter: blur(24px); -webkit-backdrop-filter: blur(24px);
border: 1px solid var(--hairline-strong); border-radius: 8px;
box-shadow: 0 12px 40px rgba(0,0,0,0.6), inset 0 1px 0 rgba(255,255,255,0.05);
opacity: 0; transform: translateY(-4px);
pointer-events: none; transition: all 0.15s; z-index: 200; overflow: hidden;
}
.sources-popover.show { opacity: 1; transform: translateY(0); pointer-events: auto; }
.sp-head {
padding: 10px 14px 8px; border-bottom: 1px solid var(--hairline);
font-size: 11.5px; color: var(--muted);
}
.sp-list { padding: 6px 0; }
.sp-row {
display: flex; align-items: center; gap: 10px;
padding: 8px 14px; cursor: pointer; transition: background 0.08s;
width: 100%; text-align: left; border: none; background: none; color: inherit;
}
.sp-row:hover { background: rgba(255,255,255,0.03); }
.sp-dot { width: 7px; height: 7px; border-radius: 50%; flex-shrink: 0; }
.sp-dot.claude { background: #d97757; box-shadow: 0 0 6px rgba(217,119,87,0.5); }
.sp-dot.codex { background: #10a37f; box-shadow: 0 0 6px rgba(16,163,127,0.5); }
.sp-dot.off { background: var(--muted-3); }
.sp-body { flex: 1; min-width: 0; }
.sp-name { font-size: 12.5px; color: var(--fg); font-weight: 500; display: flex; align-items: baseline; gap: 6px; }
.sp-name .sp-count { font-family: var(--font-mono); font-size: 10.5px; color: var(--muted); font-weight: 400; }
.sp-meta { font-family: var(--font-mono); font-size: 10.5px; color: var(--muted); margin-top: 2px; }
.sp-meta.warn { color: #fbbf24; }
.sp-meta.error { color: #f87171; }
.sp-foot {
padding: 8px 14px; border-top: 1px solid var(--hairline); background: rgba(0,0,0,0.2);
}
.sp-foot button {
font-size: 11.5px; color: var(--accent-2); border: none; background: none;
cursor: pointer; border-bottom: 1px solid rgba(167,139,250,0.4); padding-bottom: 1px;
transition: all 0.12s;
}
.sp-foot button:hover { color: var(--accent); border-bottom-color: var(--accent); }
/* Project noise fold */
.project-fold {
display: flex; align-items: center; gap: 8px;
padding: 0 10px; height: 26px; border-radius: 5px;
color: var(--muted); font-size: 12px;
cursor: pointer; user-select: none; transition: all 0.08s;
width: 100%; text-align: left; border: none; background: none;
}
.project-fold:hover { background: var(--surface-strong); color: var(--fg-2); }
.project-fold.expanded { color: var(--fg-3); }
.project-fold .chev {
width: 9px; height: 9px; color: var(--muted-2);
transition: transform 0.15s; flex-shrink: 0;
}
.project-fold.expanded .chev { transform: rotate(90deg); color: var(--muted); }
.project-fold .label { flex: 1; }
.project-fold .count {
font-family: var(--font-mono); font-size: 10px; color: var(--muted-2);
font-variant-numeric: tabular-nums; letter-spacing: 0.02em;
}
.sidebar-item.noise { opacity: 0.6; }
.sidebar-item.noise .icon { color: var(--muted-2); }
.sidebar-item.noise .label {
font-family: var(--font-mono); font-size: 11.5px;
color: var(--muted); letter-spacing: 0.005em;
}
.sidebar-item.noise:hover { opacity: 1; }
.sidebar-section-title { .sidebar-section-title {
padding: 4px 10px 6px; padding: 4px 10px 6px;
font-size: 10.5px; color: var(--muted); font-size: 10.5px; color: var(--muted);
font-weight: 500; letter-spacing: 0.04em; font-weight: 500; letter-spacing: 0.04em;
display: flex; justify-content: space-between; display: flex; align-items: center; justify-content: space-between;
flex-shrink: 0;
} }
.sidebar-section-title .filter-toggle {
display: inline-flex; align-items: center; gap: 4px;
font-family: var(--font-mono); font-size: 10px; color: var(--muted);
letter-spacing: 0.02em; cursor: pointer;
padding: 2px 6px; border-radius: 3px;
text-transform: lowercase; transition: all 0.1s;
white-space: nowrap; flex-shrink: 0;
background: none; border: 1px solid transparent;
width: auto; max-width: none;
}
.sidebar-section-title .filter-toggle svg { width: 10px; height: 10px; flex-shrink: 0; }
.sidebar-section-title .filter-toggle:hover { color: var(--fg-2); background: var(--surface); border-color: var(--hairline-strong); }
.sidebar-section-title .filter-toggle.active { color: var(--accent-2); background: var(--accent-soft); border-color: rgba(167,139,250,0.35); }
.sidebar-search { position: relative; padding: 0 6px 6px; flex-shrink: 0; } .sidebar-search { position: relative; padding: 0 6px 6px; flex-shrink: 0; }
.sidebar-search input { .sidebar-search input {
width: 100%; height: 24px; width: 100%; height: 24px;
+47
View File
@@ -7,6 +7,7 @@
background: rgba(0,0,0,0.15); background: rgba(0,0,0,0.15);
backdrop-filter: blur(20px); backdrop-filter: blur(20px);
-webkit-backdrop-filter: blur(20px); -webkit-backdrop-filter: blur(20px);
position: relative; z-index: 50;
} }
.breadcrumb { display: flex; align-items: center; gap: 6px; min-width: 0; } .breadcrumb { display: flex; align-items: center; gap: 6px; min-width: 0; }
.crumb { .crumb {
@@ -96,6 +97,52 @@
.tab-group button:hover { background: var(--surface); color: var(--fg-2); } .tab-group button:hover { background: var(--surface); color: var(--fg-2); }
.tab-group button.active { background: var(--accent-soft); color: var(--accent-2); } .tab-group button.active { background: var(--accent-soft); color: var(--accent-2); }
/* Source filter */
.source-filter-wrap { position: relative; }
.filter-btn {
display: inline-flex; align-items: center; gap: 6px;
height: 26px; padding: 0 10px;
border: 1px solid var(--hairline-strong); border-radius: 5px;
background: var(--surface); color: var(--fg-2);
font-size: 11.5px; font-weight: 500; cursor: pointer;
transition: all 0.12s;
}
.filter-btn:hover { background: var(--surface-strong); color: var(--fg); border-color: var(--hairline-vivid); }
.filter-btn.active { border-color: rgba(167,139,250,0.35); background: var(--accent-soft); color: var(--accent-2); }
.filter-btn svg { width: 11px; height: 11px; }
.filter-btn .filter-label { font-family: var(--font-mono); font-size: 10.5px; color: var(--muted); letter-spacing: 0.04em; }
.filter-btn.active .filter-label { color: var(--accent); }
.filter-dropdown {
position: absolute; top: calc(100% + 6px); right: 0;
width: 220px; background: rgba(20, 22, 38, 0.98);
backdrop-filter: blur(24px); -webkit-backdrop-filter: blur(24px);
border: 1px solid var(--hairline-strong); border-radius: 8px;
box-shadow: 0 12px 40px rgba(0,0,0,0.6), inset 0 1px 0 rgba(255,255,255,0.05);
opacity: 0; transform: translateY(-4px);
pointer-events: none; transition: all 0.15s;
z-index: 100; padding: 6px;
}
.filter-dropdown.show { opacity: 1; transform: translateY(0); pointer-events: auto; }
.fd-row {
display: flex; align-items: center; gap: 10px;
padding: 8px 10px; border-radius: 5px; cursor: pointer;
transition: background 0.08s;
}
.fd-row:hover { background: rgba(255,255,255,0.03); }
.fd-row .fd-check {
width: 14px; height: 14px;
border: 1.5px solid var(--muted-2); border-radius: 3px;
flex-shrink: 0; display: grid; place-items: center;
transition: all 0.1s;
}
.fd-row.checked .fd-check { background: var(--accent); border-color: var(--accent); box-shadow: 0 0 6px var(--accent-glow); }
.fd-row .fd-check svg { width: 10px; height: 10px; color: var(--bg); opacity: 0; }
.fd-row.checked .fd-check svg { opacity: 1; }
.fd-row .fd-name { font-size: 12.5px; color: var(--fg-2); flex: 1; }
.fd-row.checked .fd-name { color: var(--fg); }
.fd-divider { height: 1px; background: var(--hairline); margin: 4px 6px; }
.toolbar-action-primary { .toolbar-action-primary {
display: inline-flex; align-items: center; gap: 5px; display: inline-flex; align-items: center; gap: 5px;
height: 26px; padding: 0 12px; height: 26px; padding: 0 12px;
+138
View File
@@ -0,0 +1,138 @@
const { spawn } = require('child_process');
const http = require('http');
const net = require('net');
const path = require('path');
const appDir = path.join(__dirname, '..');
const binExt = process.platform === 'win32' ? '.cmd' : '';
const viteBin = path.join(appDir, 'node_modules', '.bin', `vite${binExt}`);
const electronBin = path.join(appDir, 'node_modules', '.bin', `electron${binExt}`);
const DEFAULT_DEV_PORT = Number(process.env.OBELISK_DEV_SERVER_PORT || 5173);
let viteProcess = null;
let electronProcess = null;
let shuttingDown = false;
function spawnLocal(command, args, extraEnv = {}) {
return spawn(command, args, {
cwd: appDir,
stdio: 'inherit',
env: { ...process.env, ...extraEnv },
shell: process.platform === 'win32',
});
}
function waitForDevServer(url, timeoutMs = 20000) {
const started = Date.now();
return new Promise((resolve, reject) => {
const poll = () => {
const req = http.get(url, (res) => {
res.resume();
if (res.statusCode >= 200 && res.statusCode < 400) {
resolve();
} else {
if (Date.now() - started >= timeoutMs) {
reject(new Error(`Timed out waiting for ${url}`));
} else {
setTimeout(poll, 250);
}
}
});
req.on('error', () => {
if (Date.now() - started >= timeoutMs) {
reject(new Error(`Timed out waiting for ${url}`));
return;
}
setTimeout(poll, 250);
});
req.setTimeout(1000, () => {
req.destroy();
});
};
poll();
});
}
function isDevServerRunning(url) {
return new Promise((resolve) => {
const req = http.get(url, (res) => {
res.resume();
resolve(res.statusCode >= 200 && res.statusCode < 400);
});
req.on('error', () => resolve(false));
req.setTimeout(1000, () => {
req.destroy();
resolve(false);
});
});
}
function isPortAvailable(port) {
return new Promise((resolve) => {
const server = net.createServer();
server.once('error', () => resolve(false));
server.once('listening', () => {
server.close(() => resolve(true));
});
server.listen(port, '127.0.0.1');
});
}
async function findAvailablePort(startPort) {
for (let port = startPort; port < startPort + 20; port++) {
if (await isPortAvailable(port)) return port;
}
throw new Error(`No available port found from ${startPort} to ${startPort + 19}`);
}
function stopChild(child) {
if (!child || child.killed) return;
child.kill(process.platform === 'win32' ? undefined : 'SIGTERM');
}
function shutdown(exitCode = 0) {
if (shuttingDown) return;
shuttingDown = true;
stopChild(electronProcess);
stopChild(viteProcess);
process.exit(exitCode);
}
async function main() {
let devPort = DEFAULT_DEV_PORT;
let devUrl = `http://127.0.0.1:${devPort}`;
const existingServer = await isDevServerRunning(devUrl);
if (existingServer) {
console.log(`Using existing Vite dev server at ${devUrl}`);
} else {
devPort = await findAvailablePort(DEFAULT_DEV_PORT);
devUrl = `http://127.0.0.1:${devPort}`;
viteProcess = spawnLocal(viteBin, ['renderer', '--host', '127.0.0.1', '--port', String(devPort), '--strictPort']);
viteProcess.on('exit', (code, signal) => {
if (!shuttingDown && !electronProcess) shutdown(code || (signal ? 1 : 0));
});
try {
await waitForDevServer(devUrl);
} catch (error) {
console.error(error.message);
shutdown(1);
return;
}
}
electronProcess = spawnLocal(electronBin, ['.', '--dev', ...process.argv.slice(2)], {
OBELISK_DEV_SERVER_URL: devUrl,
});
electronProcess.on('exit', (code, signal) => {
shutdown(code || (signal ? 1 : 0));
});
}
process.on('SIGINT', () => shutdown(0));
process.on('SIGTERM', () => shutdown(0));
main().catch((error) => {
console.error(error);
shutdown(1);
});
+3 -1
View File
@@ -33,11 +33,13 @@ unless scoped evidence is insufficient and `query_plan` says why.
Project-like fields are distinct: Project-like fields are distinct:
- `sessions.project`: stored Claude Code project slug. - `sessions.project`: provider-normalized project slug.
- `memories.project`: stored project slug copied onto registered memory records. - `memories.project`: stored project slug copied onto registered memory records.
- `sessions.project_path`: absolute session path derived from message `cwd` when available; slug decoding is only a fallback. - `sessions.project_path`: absolute session path derived from message `cwd` when available; slug decoding is only a fallback.
- `messages.cwd`: working directory at message time. - `messages.cwd`: working directory at message time.
- `sessions.source` / `messages.source`: transcript provider, currently `claude` or `codex`.
- helper `project`: SQL `LIKE` over `sessions.project`, not exact membership. - helper `project`: SQL `LIKE` over `sessions.project`, not exact membership.
- helper `source`: optional provider filter. Omit it unless provenance matters.
For exact project membership, prefer helper filters or a scoped first pass when For exact project membership, prefer helper filters or a scoped first pass when
they are expressive enough; use `sql()` with `s.project = ?` or they are expressive enough; use `sql()` with `s.project = ?` or
+57 -27
View File
@@ -10,24 +10,35 @@ contract for agents and humans; it is not the runtime source of truth.
## 1. Database Schema ## 1. Database Schema
Database location: `~/.claude/obelisk.sqlite` Database location: `~/.obelisk/obelisk.sqlite`. Older
`~/.claude/obelisk.sqlite` databases are copied forward on first open when the
new database does not exist.
Obelisk indexes two transcript sources into the same schema: Claude Code rows
use `source='claude'`; Codex rows use `source='codex'` and synthetic IDs
prefixed with `codex:`. Query helpers search all sources by default. Use helper
`source` filters or raw SQL on `sessions.source` / `messages.source` only when
provider provenance matters.
### sessions ### sessions
One row per Claude Code session. One row per root session. Claude session IDs match JSONL filenames. Codex root
session IDs are prefixed with `codex:`; Codex child threads are attached through
`subagents` instead of becoming separate `sessions` rows.
```sql ```sql
CREATE TABLE sessions ( CREATE TABLE sessions (
id TEXT PRIMARY KEY, -- session UUID (matches JSONL filename) id TEXT PRIMARY KEY, -- Claude UUID or "codex:<thread-id>"
title TEXT, -- AI-generated session title (may be NULL) title TEXT, -- AI-generated/session title (may be NULL)
project TEXT, -- Claude project slug (e.g. "-Users-tomiya-Code-quiet-zero") project TEXT, -- provider-normalized project slug (e.g. "-Users-tomiya-Code-quiet-zero")
project_path TEXT, -- absolute session cwd-derived path, with slug fallback (e.g. "/Users/tomiya/Code/quiet-zero") project_path TEXT, -- absolute session cwd-derived path, with slug fallback (e.g. "/Users/tomiya/Code/quiet-zero")
started_at TEXT, -- ISO 8601 timestamp of first message started_at TEXT, -- ISO 8601 timestamp of first message
ended_at TEXT, -- ISO 8601 timestamp of last message ended_at TEXT, -- ISO 8601 timestamp of last message
git_branch TEXT, -- git branch active during session (if any) git_branch TEXT, -- git branch active during session (if any)
version TEXT, -- Claude Code version string version TEXT, -- provider CLI/app version string
message_count INTEGER DEFAULT 0, -- total user + assistant messages message_count INTEGER DEFAULT 0, -- total user + assistant messages
jsonl_path TEXT -- absolute path to source JSONL file jsonl_path TEXT, -- absolute path to source JSONL file
source TEXT DEFAULT 'claude' -- "claude" or "codex"
); );
``` ```
@@ -53,18 +64,19 @@ CREATE TABLE messages (
output_tokens INTEGER, -- token usage (assistant messages only) output_tokens INTEGER, -- token usage (assistant messages only)
cwd TEXT, -- working directory at message time (may differ from session project_path) cwd TEXT, -- working directory at message time (may differ from session project_path)
skill TEXT, -- skill that generated this response (e.g. "obelisk"), NULL if none skill TEXT, -- skill that generated this response (e.g. "obelisk"), NULL if none
turn_duration_ms INTEGER -- wall-clock duration of the turn ending at this message (from system turn_duration event) turn_duration_ms INTEGER, -- wall-clock duration of the turn ending at this message
source TEXT DEFAULT 'claude' -- "claude" or "codex"
); );
``` ```
Indexes: `idx_messages_session(session_id)`, `idx_messages_agent(agent_id)`, `idx_messages_ts(session_id, timestamp)`. Indexes: `idx_messages_session(session_id)`, `idx_messages_agent(agent_id)`, `idx_messages_ts(session_id, timestamp)`.
`content_type` preserves the top-level Claude Code content block shape for the `content_type` preserves the normalized transcript surface. Treat `text` as
message row. Treat `text` as user/assistant visible language, `thinking` as user/assistant visible language, `thinking` as trace/debug material, and
trace/debug material, and `tool_use` as a marker that the assistant message `tool_use` as a marker that the assistant message contains tool calls.
contains tool calls. `tool_result` marks a tool-result message, but the `tool_result` marks a tool-result message when the provider emits one as a
structured payload remains in `tool_results`. Tool-call details remain in message; structured payloads remain in `tool_results`. Tool-call details remain
`tool_calls`. Messages whose top-level content is not one of these four raw in `tool_calls`. Messages whose top-level content is not one of these raw
message surfaces are `unknown`. Real user input is represented by `type='user'` message surfaces are `unknown`. Real user input is represented by `type='user'`
and `content_type='text'`, not by a separate `user_message` content type. and `content_type='text'`, not by a separate `user_message` content type.
@@ -298,27 +310,30 @@ Full-text search across all message text using FTS5.
| `opts.after` | `string` | ISO 8601 lower bound on timestamp | | `opts.after` | `string` | ISO 8601 lower bound on timestamp |
| `opts.before` | `string` | ISO 8601 upper bound on timestamp | | `opts.before` | `string` | ISO 8601 upper bound on timestamp |
| `opts.cwd` | `string` | Filter by working directory (supports LIKE) | | `opts.cwd` | `string` | Filter by working directory (supports LIKE) |
| `opts.source` | `string` | Optional provider filter: `"claude"` or `"codex"` |
| `opts.includeMeta` | `boolean` | Include injected/control-plane messages (default `false`) | | `opts.includeMeta` | `boolean` | Include injected/control-plane messages (default `false`) |
**Scope note:** `sessions.project` is the stored Claude Code project slug, **Scope note:** `sessions.project` is the provider-normalized project slug,
`sessions.project_path` is the absolute session path derived from message `cwd` `sessions.project_path` is the absolute session path derived from message `cwd`
when available, and `messages.cwd` is the working directory at message time. when available, `messages.cwd` is the working directory at message time, and
Helper `project` filters are fuzzy `LIKE` filters over `sessions.project`. For `source` is the provider. Helper `project` filters are fuzzy `LIKE` filters over
exact project membership, use `sql()` with `s.project = ?` or `sessions.project`. For exact project membership, use `sql()` with
`s.project_path = ?`. `s.project = ?` or `s.project_path = ?`.
**Returns:** `Array<{ message, session, rank, context }>` where `message` **Returns:** `Array<{ message, session, rank, context }>` where `message`
includes `{ uuid, text, content_type, is_meta, role, timestamp, model, cwd }` includes `{ uuid, text, content_type, is_meta, role, timestamp, model, cwd,
and `context` is the 6 nearest non-meta messages by timestamp in the same source }`, `session` includes `source`, and `context` is the 6 nearest non-meta
session unless `includeMeta: true` is passed. It is temporal neighbor context, messages by timestamp in the same session unless `includeMeta: true` is passed.
not a parent chain. `rank` is the FTS5 relevance score used by `ORDER BY rank`; It is temporal neighbor context, not a parent chain. `rank` is the FTS5
lower values sort earlier, so treat the returned order as the relevance order relevance score used by `ORDER BY rank`; lower values sort earlier, so treat the
unless you are deliberately using FTS5 ranking details. returned order as the relevance order unless you are deliberately using FTS5
ranking details.
```js ```js
const hits = search('MCTS exploration'); const hits = search('MCTS exploration');
return hits.map(h => ({ return hits.map(h => ({
title: h.session.title, title: h.session.title,
source: h.session.source,
content_type: h.message.content_type, content_type: h.message.content_type,
is_meta: h.message.is_meta, is_meta: h.message.is_meta,
text: h.message.text?.slice(0, 200), text: h.message.text?.slice(0, 200),
@@ -418,6 +433,7 @@ All subagent spawns, with message counts. For backward compatibility, passing a
|-------|------|-------------| |-------|------|-------------|
| `opts.sessionId` | `string` | Restrict to one session | | `opts.sessionId` | `string` | Restrict to one session |
| `opts.project` | `string` | SQL `LIKE` pattern over `sessions.project` | | `opts.project` | `string` | SQL `LIKE` pattern over `sessions.project` |
| `opts.source` | `string` | Optional provider filter: `"claude"` or `"codex"` |
| `opts.limit` | `number` | Max results (default 100) | | `opts.limit` | `number` | Max results (default 100) |
**Returns:** `Array<{ ...subagent_row, messageCount }>`. **Returns:** `Array<{ ...subagent_row, messageCount }>`.
@@ -437,6 +453,7 @@ Workflow executions. For backward compatibility, passing a string is treated as
| `opts.project` | `string` | SQL `LIKE` pattern over `sessions.project` | | `opts.project` | `string` | SQL `LIKE` pattern over `sessions.project` |
| `opts.after` | `string` | ISO 8601 lower bound on timestamp | | `opts.after` | `string` | ISO 8601 lower bound on timestamp |
| `opts.before` | `string` | ISO 8601 upper bound on timestamp | | `opts.before` | `string` | ISO 8601 upper bound on timestamp |
| `opts.source` | `string` | Optional provider filter: `"claude"` or `"codex"` |
| `opts.limit` | `number` | Max results (default 100) | | `opts.limit` | `number` | Max results (default 100) |
**Returns:** `Array<workflow_row>`. **Returns:** `Array<workflow_row>`.
@@ -466,6 +483,7 @@ All tool calls that touched a specific file, across every session.
| `filePath` | `string` | Absolute file path (required) | | `filePath` | `string` | Absolute file path (required) |
| `opts.after` | `string` | ISO 8601 lower bound | | `opts.after` | `string` | ISO 8601 lower bound |
| `opts.before` | `string` | ISO 8601 upper bound | | `opts.before` | `string` | ISO 8601 upper bound |
| `opts.source` | `string` | Optional provider filter: `"claude"` or `"codex"` |
| `opts.limit` | `number` | Max results (default 200) | | `opts.limit` | `number` | Max results (default 200) |
**Returns:** `Array<{ toolCall, session, timestamp }>`. **Returns:** `Array<{ toolCall, session, timestamp }>`.
@@ -488,6 +506,7 @@ Tool calls whose results contain error patterns (`Error`, `ENOENT`, `failed`, `p
| `opts.project` | `string` | SQL `LIKE` pattern over `sessions.project` | | `opts.project` | `string` | SQL `LIKE` pattern over `sessions.project` |
| `opts.after` | `string` | ISO 8601 lower bound | | `opts.after` | `string` | ISO 8601 lower bound |
| `opts.before` | `string` | ISO 8601 upper bound | | `opts.before` | `string` | ISO 8601 upper bound |
| `opts.source` | `string` | Optional provider filter: `"claude"` or `"codex"` |
| `opts.limit` | `number` | Max results (default 50) | | `opts.limit` | `number` | Max results (default 50) |
**Returns:** `Array<{ toolCall, result, session, nextMessages }>`. **Returns:** `Array<{ toolCall, result, session, nextMessages }>`.
@@ -523,11 +542,15 @@ string is treated as `sessionId`, and passing a number is treated as `limit`.
| `opts.after` | `string` | ISO 8601 lower bound on summary timestamp | | `opts.after` | `string` | ISO 8601 lower bound on summary timestamp |
| `opts.before` | `string` | ISO 8601 upper bound on summary timestamp | | `opts.before` | `string` | ISO 8601 upper bound on summary timestamp |
| `opts.branch` | `string` | Filter by source session git branch (exact match) | | `opts.branch` | `string` | Filter by source session git branch (exact match) |
| `opts.source` | `string` | Optional provider filter: `"claude"` or `"codex"` |
| `opts.limit` | `number` | Max results (default 100) | | `opts.limit` | `number` | Max results (default 100) |
**Returns:** `Array<summary_row & { session_title, project }>` ordered by **Returns:** `Array<summary_row & { session_title, project }>` ordered by
`timestamp` descending. `timestamp` descending.
Note: `summaries.source` is the summary kind such as `away_summary`; provider
filtering uses the joined session's `source`.
```js ```js
const rows = summaries({ project: '%quiet-zero%', limit: 5 }); const rows = summaries({ project: '%quiet-zero%', limit: 5 });
return rows.map(s => ({ return rows.map(s => ({
@@ -574,7 +597,7 @@ from `process.cwd()` against `sessions.project_path`, then from exact
project_path, project_path,
session_total, session_total,
sessions: [ sessions: [
{ id, title, project, project_path, started_at, ended_at, git_branch, message_count } { id, title, project, project_path, started_at, ended_at, git_branch, message_count, source }
], ],
memory_total, memory_total,
memories: [ memories: [
@@ -592,7 +615,12 @@ from `process.cwd()` against `sessions.project_path`, then from exact
recent_branches recent_branches
} }
], ],
totals: { projects, sessions, memories } totals: {
projects,
sessions,
memories,
sources: [{ source: 'claude' | 'codex', session_count, last_session_at }]
}
} }
``` ```
@@ -629,6 +657,7 @@ Query sessions with filters. For backward compatibility, passing a number is tre
| `opts.before` | `string` | ISO 8601 upper bound on `started_at` | | `opts.before` | `string` | ISO 8601 upper bound on `started_at` |
| `opts.limit` | `number` | Max results (default 50) | | `opts.limit` | `number` | Max results (default 50) |
| `opts.branch` | `string` | Filter by git branch (exact match) | | `opts.branch` | `string` | Filter by git branch (exact match) |
| `opts.source` | `string` | Optional provider filter: `"claude"` or `"codex"` |
| `opts.sessionId` | `string` | Restrict to one session | | `opts.sessionId` | `string` | Restrict to one session |
| `opts.sessions` | `string[]` | Restrict to a set of session IDs | | `opts.sessions` | `string[]` | Restrict to a set of session IDs |
@@ -656,6 +685,7 @@ string is treated as `sessionId`, and passing a number is treated as `limit`.
| `opts.after` | `string` | ISO 8601 lower bound on `created_at` | | `opts.after` | `string` | ISO 8601 lower bound on `created_at` |
| `opts.before` | `string` | ISO 8601 upper bound on `created_at` | | `opts.before` | `string` | ISO 8601 upper bound on `created_at` |
| `opts.branch` | `string` | Filter by source session git branch (exact match) | | `opts.branch` | `string` | Filter by source session git branch (exact match) |
| `opts.source` | `string` | Optional provider filter through the source session: `"claude"` or `"codex"` |
| `opts.limit` | `number` | Max results (default 50) | | `opts.limit` | `number` | Max results (default 50) |
**Returns:** `Array<memory_row & { rank?: number }>` with archived memories **Returns:** `Array<memory_row & { rank?: number }>` with archived memories
+34 -7
View File
@@ -6,14 +6,27 @@ const os = require('node:os');
const { DatabaseSync } = require('node:sqlite'); const { DatabaseSync } = require('node:sqlite');
const CLAUDE_DIR = path.join(os.homedir(), '.claude'); const CLAUDE_DIR = path.join(os.homedir(), '.claude');
const DB_PATH = path.join(CLAUDE_DIR, 'obelisk.sqlite'); const CODEX_DIR = path.join(os.homedir(), '.codex');
const OBELISK_DIR = path.join(os.homedir(), '.obelisk');
const LEGACY_DB_PATH = path.join(CLAUDE_DIR, 'obelisk.sqlite');
const DB_PATH = path.join(OBELISK_DIR, 'obelisk.sqlite');
const TEXT_LIMIT = 10000; const TEXT_LIMIT = 10000;
const SCHEMA = fs.readFileSync(new URL('./schema.sql', import.meta.url), 'utf8'); const SCHEMA = fs.readFileSync(new URL('./schema.sql', import.meta.url), 'utf8');
function migrateLegacyDbIfNeeded() {
if (fs.existsSync(DB_PATH)) return;
if (!fs.existsSync(LEGACY_DB_PATH)) return;
fs.mkdirSync(path.dirname(DB_PATH), { recursive: true });
fs.copyFileSync(LEGACY_DB_PATH, DB_PATH);
}
function openDb() { function openDb() {
migrateLegacyDbIfNeeded();
fs.mkdirSync(path.dirname(DB_PATH), { recursive: true });
const db = new DatabaseSync(DB_PATH); const db = new DatabaseSync(DB_PATH);
db.exec('PRAGMA journal_mode=WAL'); db.exec('PRAGMA journal_mode=WAL');
db.exec('PRAGMA synchronous=NORMAL'); db.exec('PRAGMA synchronous=NORMAL');
migrateExistingColumns(db);
db.exec(SCHEMA); db.exec(SCHEMA);
migrateDb(db); migrateDb(db);
return db; return db;
@@ -24,12 +37,26 @@ function ensureColumn(db, table, column, definition) {
if (!columns.includes(column)) db.exec(`ALTER TABLE ${table} ADD COLUMN ${column} ${definition}`); if (!columns.includes(column)) db.exec(`ALTER TABLE ${table} ADD COLUMN ${column} ${definition}`);
} }
function tableExists(db, table) {
return Boolean(db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name=?").get(table));
}
function migrateExistingColumns(db) {
if (tableExists(db, 'sessions')) ensureColumn(db, 'sessions', 'source', "TEXT DEFAULT 'claude'");
if (tableExists(db, 'messages')) {
ensureColumn(db, 'messages', 'content_type', 'TEXT');
ensureColumn(db, 'messages', 'is_meta', 'INTEGER DEFAULT 0');
ensureColumn(db, 'messages', 'source', "TEXT DEFAULT 'claude'");
}
if (tableExists(db, 'memories')) {
ensureColumn(db, 'memories', 'anchors', 'TEXT');
ensureColumn(db, 'memories', 'deleted_at', 'TEXT');
ensureColumn(db, 'memories', 'deleted_reason', 'TEXT');
}
}
function migrateDb(db) { function migrateDb(db) {
ensureColumn(db, 'messages', 'content_type', 'TEXT'); migrateExistingColumns(db);
ensureColumn(db, 'messages', 'is_meta', 'INTEGER DEFAULT 0');
ensureColumn(db, 'memories', 'anchors', 'TEXT');
ensureColumn(db, 'memories', 'deleted_at', 'TEXT');
ensureColumn(db, 'memories', 'deleted_reason', 'TEXT');
} }
function rebuildMemoryFts(db) { function rebuildMemoryFts(db) {
@@ -118,4 +145,4 @@ function readLines(filePath, callback) {
} }
} }
export { CLAUDE_DIR, DB_PATH, TEXT_LIMIT, openDb, rebuildMemoryFts, trunc, truncJson, extractText, extractContentType, extractMessageIsMeta, filePath, isDir, readLines, fs, path, os }; export { CLAUDE_DIR, CODEX_DIR, OBELISK_DIR, DB_PATH, TEXT_LIMIT, openDb, rebuildMemoryFts, trunc, truncJson, extractText, extractContentType, extractMessageIsMeta, filePath, isDir, readLines, fs, path, os };
+493 -8
View File
@@ -1,7 +1,8 @@
import { CLAUDE_DIR, openDb, rebuildMemoryFts, trunc, truncJson, extractText, extractContentType, extractMessageIsMeta, filePath, isDir, readLines, fs, path } from './db.mjs'; import { CLAUDE_DIR, CODEX_DIR, openDb, rebuildMemoryFts, trunc, truncJson, extractText, extractContentType, extractMessageIsMeta, filePath, isDir, readLines, fs, path } from './db.mjs';
const PROJECTS_DIR = path.join(CLAUDE_DIR, 'projects'); const PROJECTS_DIR = path.join(CLAUDE_DIR, 'projects');
const HISTORY_PATH = path.join(CLAUDE_DIR, 'history.jsonl'); const HISTORY_PATH = path.join(CLAUDE_DIR, 'history.jsonl');
const CODEX_SESSIONS_DIR = path.join(CODEX_DIR, 'sessions');
function legacyProjectPathFromSlug(project) { function legacyProjectPathFromSlug(project) {
if (!project) return null; if (!project) return null;
@@ -13,6 +14,12 @@ function normalizeObservedCwd(cwd) {
return path.normalize(cwd); return path.normalize(cwd);
} }
function projectSlugFromPath(projectPath) {
const normalized = normalizeObservedCwd(projectPath);
if (!normalized) return null;
return '-' + normalized.replace(/^[\\/]+/, '').replace(/[\\/]+/g, '-');
}
function inferProjectPath(project, observedCwds = []) { function inferProjectPath(project, observedCwds = []) {
const byPath = new Map(); const byPath = new Map();
for (const cwd of observedCwds) { for (const cwd of observedCwds) {
@@ -68,6 +75,25 @@ function discoverJsonlFiles() {
return files; return files;
} }
function discoverCodexJsonlFiles() {
const files = [];
if (!fs.existsSync(CODEX_SESSIONS_DIR)) return files;
const walk = (dir) => {
let entries;
try { entries = fs.readdirSync(dir, { withFileTypes: true }); } catch { return; }
for (const entry of entries) {
const fp = path.join(dir, entry.name);
if (entry.isDirectory()) {
walk(fp);
} else if (entry.isFile() && entry.name.endsWith('.jsonl')) {
files.push({ path: fp, source: 'codex' });
}
}
};
walk(CODEX_SESSIONS_DIR);
return files;
}
function needsReindex(db, fp) { function needsReindex(db, fp) {
const mt = fs.statSync(fp).mtimeMs; const mt = fs.statSync(fp).mtimeMs;
const row = db.prepare('SELECT mtime, lines_processed FROM index_state WHERE jsonl_path = ?').get(fp); const row = db.prepare('SELECT mtime, lines_processed FROM index_state WHERE jsonl_path = ?').get(fp);
@@ -81,8 +107,8 @@ function indexJsonl(db, fi) {
const mt = fs.statSync(fi.path).mtimeMs; const mt = fs.statSync(fi.path).mtimeMs;
const ins = { const ins = {
ses: db.prepare('INSERT OR REPLACE INTO sessions (id,title,project,project_path,started_at,ended_at,git_branch,version,message_count,jsonl_path) VALUES (?,?,?,?,?,?,?,?,?,?)'), ses: db.prepare('INSERT OR REPLACE INTO sessions (id,title,project,project_path,started_at,ended_at,git_branch,version,message_count,jsonl_path,source) VALUES (?,?,?,?,?,?,?,?,?,?,?)'),
msg: db.prepare('INSERT OR REPLACE INTO messages (uuid,session_id,type,parent_uuid,timestamp,role,text,content_type,is_meta,model,is_sidechain,agent_id,input_tokens,output_tokens,cwd,skill) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)'), msg: db.prepare('INSERT OR REPLACE INTO messages (uuid,session_id,type,parent_uuid,timestamp,role,text,content_type,is_meta,model,is_sidechain,agent_id,input_tokens,output_tokens,cwd,skill,source) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)'),
tc: db.prepare('INSERT OR REPLACE INTO tool_calls (id,message_uuid,session_id,name,input_json,file_path) VALUES (?,?,?,?,?,?)'), tc: db.prepare('INSERT OR REPLACE INTO tool_calls (id,message_uuid,session_id,name,input_json,file_path) VALUES (?,?,?,?,?,?)'),
tr: db.prepare('INSERT OR REPLACE INTO tool_results (tool_use_id,message_uuid,session_id,content,file_path,is_error) VALUES (?,?,?,?,?,?)'), tr: db.prepare('INSERT OR REPLACE INTO tool_results (tool_use_id,message_uuid,session_id,content,file_path,is_error) VALUES (?,?,?,?,?,?)'),
sum: db.prepare('INSERT OR REPLACE INTO summaries (id,session_id,timestamp,source,content) VALUES (?,?,?,?,?)'), sum: db.prepare('INSERT OR REPLACE INTO summaries (id,session_id,timestamp,source,content) VALUES (?,?,?,?,?)'),
@@ -138,7 +164,7 @@ function indexJsonl(db, fi) {
ins.msg.run(obj.uuid, sid, obj.type, obj.parentUuid || null, ts, ins.msg.run(obj.uuid, sid, obj.type, obj.parentUuid || null, ts,
msg.role || obj.type, text, contentType, isMeta, msg.model || null, msg.role || obj.type, text, contentType, isMeta, msg.model || null,
obj.isSidechain ? 1 : 0, aid, usage.input_tokens || null, usage.output_tokens || null, obj.isSidechain ? 1 : 0, aid, usage.input_tokens || null, usage.output_tokens || null,
obj.cwd || null, obj.attributionSkill || null); obj.cwd || null, obj.attributionSkill || null, 'claude');
} }
if (obj.type === 'assistant' && Array.isArray(msg.content)) { if (obj.type === 'assistant' && Array.isArray(msg.content)) {
@@ -160,11 +186,462 @@ function indexJsonl(db, fi) {
if (!fi.isSubagent) { if (!fi.isSubagent) {
const pp = inferProjectPath(fi.project, sm.cwds); const pp = inferProjectPath(fi.project, sm.cwds);
ins.ses.run(fi.sessionId, sm.title, fi.project, pp, sm.started_at, sm.ended_at, sm.git_branch, sm.version, sm.n, fi.path); ins.ses.run(fi.sessionId, sm.title, fi.project, pp, sm.started_at, sm.ended_at, sm.git_branch, sm.version, sm.n, fi.path, 'claude');
} }
ins.idx.run(fi.path, mt, lineNum); ins.idx.run(fi.path, mt, lineNum);
} }
function codexDbId(id) {
if (!id) return null;
const raw = String(id).replace(/^codex:/, '');
return `codex:${raw}`;
}
function codexRawId(id) {
return id ? String(id).replace(/^codex:/, '') : null;
}
function codexLineUuid(threadId, lineNum) {
return `codex:${codexRawId(threadId)}:${String(lineNum).padStart(6, '0')}`;
}
function codexCallId(callId) {
if (!callId) return null;
return `codex:${String(callId).replace(/^codex:/, '')}`;
}
function codexParentThreadId(meta) {
const subagent = meta?.source?.subagent;
return subagent?.thread_spawn?.parent_thread_id
|| meta?.forked_from_id
|| subagent?.parent_thread_id
|| null;
}
function codexIsGuardianThread(meta, records = []) {
const subagent = meta?.source?.subagent;
if (subagent?.other === 'guardian') return true;
if (meta?.thread_source !== 'subagent') return false;
return records.some(({ obj }) => obj?.payload?.model === 'codex-auto-review' || obj?.model === 'codex-auto-review');
}
function deleteCodexThreadRows(db, threadRawId) {
const threadId = codexDbId(threadRawId);
if (!threadId) return;
db.prepare(`
DELETE FROM tool_results
WHERE session_id = ?
OR message_uuid IN (SELECT uuid FROM messages WHERE session_id = ? OR agent_id = ?)
`).run(threadId, threadId, threadId);
db.prepare(`
DELETE FROM tool_calls
WHERE session_id = ?
OR message_uuid IN (SELECT uuid FROM messages WHERE session_id = ? OR agent_id = ?)
`).run(threadId, threadId, threadId);
db.prepare('DELETE FROM messages WHERE session_id = ? OR agent_id = ?').run(threadId, threadId);
db.prepare('DELETE FROM subagents WHERE agent_id = ? OR session_id = ?').run(threadId, threadId);
db.prepare('DELETE FROM summaries WHERE session_id = ?').run(threadId);
db.prepare('DELETE FROM sessions WHERE id = ?').run(threadId);
}
function readCodexGuardianThreadInfo(filePath) {
const records = [];
let metaRecord = null;
let lineNum = 0;
readLines(filePath, (line) => {
lineNum++;
let obj;
try {
obj = JSON.parse(line);
} catch {
return;
}
records.push({ lineNum, obj });
if (obj?.type === 'session_meta' && obj.payload?.id) {
metaRecord = { lineNum, obj };
if (obj.payload?.source?.subagent?.other === 'guardian') return false;
if (obj.payload?.thread_source !== 'subagent') return false;
}
if (metaRecord && codexIsGuardianThread(metaRecord.obj.payload, records)) return false;
});
const meta = metaRecord?.obj?.payload;
if (!meta || !codexIsGuardianThread(meta, records)) return null;
return { threadRawId: codexRawId(meta.id), lineNum };
}
function codexAgentNickname(meta) {
return meta?.agent_nickname
|| meta?.source?.subagent?.thread_spawn?.agent_nickname
|| null;
}
function codexAgentRole(meta) {
return meta?.agent_role
|| meta?.source?.subagent?.thread_spawn?.agent_role
|| null;
}
function parseCodexJsonInput(value) {
if (value === null || value === undefined || value === '') return {};
if (typeof value !== 'string') return value;
try { return JSON.parse(value); } catch { return value; }
}
function codexUsage(payload) {
const usage = payload?.info?.last_token_usage || payload?.info?.total_token_usage || payload?.last_token_usage || null;
if (!usage) return {};
return {
inputTokens: usage.input_tokens ?? null,
outputTokens: usage.output_tokens ?? null,
};
}
function codexEventText(payload) {
if (typeof payload?.message === 'string') return payload.message;
if (Array.isArray(payload?.text_elements) && payload.text_elements.length) {
const parts = payload.text_elements.map(item => typeof item === 'string' ? item : item?.text).filter(Boolean);
if (parts.length) return parts.join('\n');
}
if (typeof payload?.text === 'string') return payload.text;
return null;
}
function codexMessagePayloadText(payload) {
if (!Array.isArray(payload?.content)) return null;
const parts = [];
for (const block of payload.content) {
if (typeof block?.text === 'string') parts.push(block.text);
}
return parts.length ? parts.join('\n') : null;
}
function codexVisibleMessageKey(role, text) {
return `${role || ''}\u0000${text || ''}`;
}
function codexToolInput(payload) {
if (payload?.type === 'custom_tool_call') return parseCodexJsonInput(payload.input);
if (payload?.type === 'tool_search_call') return parseCodexJsonInput(payload.arguments);
if (payload?.type === 'web_search_call') return { action: payload.action || null };
return parseCodexJsonInput(payload?.arguments);
}
function codexToolOutput(payload) {
if (typeof payload?.output === 'string') return payload.output;
if (payload?.output !== undefined) return JSON.stringify(payload.output);
if (payload?.tools !== undefined) return JSON.stringify(payload.tools);
if (payload?.execution !== undefined) return JSON.stringify(payload.execution);
return null;
}
function upsertCodexSubagent(db, {
agentId,
sessionId,
parentToolUseId = null,
agentType = null,
description = null,
durationMs = null,
totalTokens = null,
} = {}) {
if (!agentId || !sessionId) return;
db.prepare(`
INSERT INTO subagents (agent_id,session_id,parent_tool_use_id,agent_type,description,duration_ms,total_tokens)
VALUES (?,?,?,?,?,?,?)
ON CONFLICT(agent_id) DO UPDATE SET
session_id=excluded.session_id,
parent_tool_use_id=COALESCE(excluded.parent_tool_use_id, subagents.parent_tool_use_id),
agent_type=COALESCE(excluded.agent_type, subagents.agent_type),
description=COALESCE(excluded.description, subagents.description),
duration_ms=COALESCE(excluded.duration_ms, subagents.duration_ms),
total_tokens=COALESCE(excluded.total_tokens, subagents.total_tokens)
`).run(agentId, sessionId, parentToolUseId, agentType, description, durationMs, totalTokens);
}
function indexCodexJsonl(db, fi) {
const state = needsReindex(db, fi.path);
if (!state.needed) {
const guardian = readCodexGuardianThreadInfo(fi.path);
if (guardian) deleteCodexThreadRows(db, guardian.threadRawId);
return;
}
const mt = fs.statSync(fi.path).mtimeMs;
const records = [];
let lineNum = 0;
readLines(fi.path, (line) => {
lineNum++;
try {
records.push({ lineNum, obj: JSON.parse(line) });
} catch {}
});
const metaRecord = records.find(r => r.obj?.type === 'session_meta' && r.obj.payload?.id);
if (!metaRecord) {
db.prepare('INSERT OR REPLACE INTO index_state (jsonl_path,mtime,lines_processed) VALUES (?,?,?)').run(fi.path, mt, lineNum);
return;
}
const meta = metaRecord.obj.payload;
const threadRawId = codexRawId(meta.id);
if (codexIsGuardianThread(meta, records)) {
deleteCodexThreadRows(db, threadRawId);
db.prepare('INSERT OR REPLACE INTO index_state (jsonl_path,mtime,lines_processed) VALUES (?,?,?)').run(fi.path, mt, lineNum);
return;
}
const parentRawId = codexParentThreadId(meta);
const sessionId = codexDbId(parentRawId || threadRawId);
const agentId = parentRawId ? codexDbId(threadRawId) : null;
const isSidechain = agentId ? 1 : 0;
const projectPath = normalizeObservedCwd(meta.cwd);
const project = projectSlugFromPath(projectPath);
const sm = {
started_at: meta.timestamp || metaRecord.obj.timestamp || null,
ended_at: meta.timestamp || metaRecord.obj.timestamp || null,
git_branch: meta.git?.branch || null,
version: meta.cli_version || null,
title: null,
n: 0,
cwds: projectPath ? [projectPath] : [],
lastMessageUuid: null,
lastTextAssistantUuid: null,
totalInputTokens: 0,
totalOutputTokens: 0,
};
const ins = {
ses: db.prepare('INSERT OR REPLACE INTO sessions (id,title,project,project_path,started_at,ended_at,git_branch,version,message_count,jsonl_path,source) VALUES (?,?,?,?,?,?,?,?,?,?,?)'),
msg: db.prepare(`
INSERT INTO messages (uuid,session_id,type,parent_uuid,timestamp,role,text,content_type,is_meta,model,is_sidechain,agent_id,input_tokens,output_tokens,cwd,skill,source)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
ON CONFLICT(uuid) DO UPDATE SET
session_id=excluded.session_id,
type=excluded.type,
parent_uuid=excluded.parent_uuid,
timestamp=excluded.timestamp,
role=excluded.role,
text=excluded.text,
content_type=excluded.content_type,
is_meta=excluded.is_meta,
model=excluded.model,
is_sidechain=excluded.is_sidechain,
agent_id=excluded.agent_id,
input_tokens=excluded.input_tokens,
output_tokens=excluded.output_tokens,
cwd=excluded.cwd,
skill=excluded.skill,
source=excluded.source
`),
tc: db.prepare('INSERT OR REPLACE INTO tool_calls (id,message_uuid,session_id,name,input_json,file_path) VALUES (?,?,?,?,?,?)'),
tr: db.prepare('INSERT OR REPLACE INTO tool_results (tool_use_id,message_uuid,session_id,content,file_path,is_error) VALUES (?,?,?,?,?,?)'),
idx: db.prepare('INSERT OR REPLACE INTO index_state (jsonl_path,mtime,lines_processed) VALUES (?,?,?)'),
dur: db.prepare('UPDATE messages SET turn_duration_ms=? WHERE uuid=?'),
usage: db.prepare('UPDATE messages SET input_tokens=?, output_tokens=? WHERE uuid=?'),
};
let currentCwd = projectPath;
let currentModel = null;
const eventMessageKeys = new Set();
const callMessageUuids = new Map();
for (const { obj } of records) {
if (obj?.type !== 'event_msg') continue;
const payload = obj.payload || {};
if (payload.type !== 'user_message' && payload.type !== 'agent_message') continue;
const text = codexEventText(payload);
if (text === null) continue;
eventMessageKeys.add(codexVisibleMessageKey(payload.type === 'user_message' ? 'user' : 'assistant', text));
}
const updateBounds = (ts) => {
if (!ts) return;
if (!sm.started_at || ts < sm.started_at) sm.started_at = ts;
if (!sm.ended_at || ts > sm.ended_at) sm.ended_at = ts;
};
const insertMessage = ({ uuid, type, role, text = null, contentType = 'text', timestamp, isMeta = 0 }) => {
ins.msg.run(
uuid,
sessionId,
type,
sm.lastMessageUuid,
timestamp || null,
role,
trunc(text),
contentType,
isMeta,
currentModel,
isSidechain,
agentId,
null,
null,
currentCwd,
null,
'codex',
);
sm.lastMessageUuid = uuid;
if (!agentId) sm.n++;
if (type === 'assistant' && contentType === 'text') sm.lastTextAssistantUuid = uuid;
updateBounds(timestamp);
return uuid;
};
for (const { lineNum: currentLine, obj } of records) {
const ts = obj.timestamp || null;
if (obj.type === 'session_meta') {
if (obj.payload?.cwd) {
currentCwd = normalizeObservedCwd(obj.payload.cwd) || currentCwd;
if (currentCwd) sm.cwds.push(currentCwd);
}
if (obj.payload?.git?.branch) sm.git_branch = obj.payload.git.branch;
if (obj.payload?.cli_version) sm.version = obj.payload.cli_version;
updateBounds(obj.payload?.timestamp || ts);
continue;
}
if (obj.type === 'turn_context') {
currentCwd = normalizeObservedCwd(obj.payload?.cwd) || currentCwd;
currentModel = obj.payload?.model || currentModel;
if (currentCwd) sm.cwds.push(currentCwd);
updateBounds(ts);
continue;
}
if (obj.type === 'event_msg') {
const payload = obj.payload || {};
if (payload.type === 'user_message' || payload.type === 'agent_message' || payload.type === 'agent_reasoning') {
const text = codexEventText(payload);
if (text === null) continue;
const isReasoning = payload.type === 'agent_reasoning';
insertMessage({
uuid: codexLineUuid(threadRawId, currentLine),
type: payload.type === 'user_message' ? 'user' : 'assistant',
role: payload.type === 'user_message' ? 'user' : 'assistant',
text,
contentType: isReasoning ? 'thinking' : 'text',
timestamp: ts,
});
continue;
}
if (payload.type === 'collab_agent_spawn_end' && payload.call_id && payload.new_thread_id) {
const uuid = insertMessage({
uuid: codexLineUuid(threadRawId, currentLine),
type: 'assistant',
role: 'assistant',
text: null,
contentType: 'tool_use',
timestamp: ts,
});
const toolId = codexCallId(payload.call_id);
const description = payload.new_agent_nickname || payload.new_agent_role || 'Agent';
const input = {
description,
subagent_type: payload.new_agent_role || 'Agent',
prompt: payload.prompt || '',
new_thread_id: payload.new_thread_id,
model: payload.model || null,
reasoning_effort: payload.reasoning_effort || null,
};
ins.tc.run(toolId, uuid, sessionId, 'Agent', truncJson(input), null);
callMessageUuids.set(toolId, uuid);
upsertCodexSubagent(db, {
agentId: codexDbId(payload.new_thread_id),
sessionId,
parentToolUseId: toolId,
agentType: payload.new_agent_role || null,
description,
});
continue;
}
if (payload.type === 'task_complete') {
if (sm.lastTextAssistantUuid && payload.duration_ms !== undefined) {
ins.dur.run(payload.duration_ms || null, sm.lastTextAssistantUuid);
}
updateBounds(ts);
continue;
}
if (payload.type === 'token_count') {
const usage = codexUsage(payload);
if (usage.inputTokens !== null) sm.totalInputTokens = usage.inputTokens;
if (usage.outputTokens !== null) sm.totalOutputTokens = usage.outputTokens;
if (sm.lastTextAssistantUuid && (usage.inputTokens !== null || usage.outputTokens !== null)) {
ins.usage.run(usage.inputTokens, usage.outputTokens, sm.lastTextAssistantUuid);
}
continue;
}
if (payload.type === 'thread_name_updated' && payload.thread_name) {
sm.title = payload.thread_name;
}
continue;
}
if (obj.type !== 'response_item') continue;
const payload = obj.payload || {};
if (payload.type === 'message' && payload.role !== 'developer') {
const text = codexMessagePayloadText(payload);
const role = payload.role || 'assistant';
if (text !== null && !eventMessageKeys.has(codexVisibleMessageKey(role, text))) {
insertMessage({
uuid: codexLineUuid(threadRawId, currentLine),
type: role === 'user' ? 'user' : 'assistant',
role,
text,
contentType: 'text',
timestamp: ts,
});
}
continue;
}
if (['function_call', 'custom_tool_call', 'tool_search_call', 'web_search_call'].includes(payload.type) && payload.call_id) {
const uuid = insertMessage({
uuid: codexLineUuid(threadRawId, currentLine),
type: 'assistant',
role: 'assistant',
text: null,
contentType: 'tool_use',
timestamp: ts,
});
const name = payload.name || payload.tool || payload.type.replace(/_call$/, '');
const toolId = codexCallId(payload.call_id);
ins.tc.run(toolId, uuid, sessionId, name, truncJson(codexToolInput(payload)), null);
callMessageUuids.set(toolId, uuid);
continue;
}
if (['function_call_output', 'custom_tool_call_output', 'tool_search_output'].includes(payload.type) && payload.call_id) {
const toolId = codexCallId(payload.call_id);
ins.tr.run(toolId, callMessageUuids.get(toolId) || null, sessionId, trunc(codexToolOutput(payload) || ''), null, payload.is_error ? 1 : 0);
}
}
if (agentId) {
const tokenTotal = (sm.totalInputTokens || 0) + (sm.totalOutputTokens || 0);
const started = sm.started_at ? new Date(sm.started_at).getTime() : null;
const ended = sm.ended_at ? new Date(sm.ended_at).getTime() : null;
upsertCodexSubagent(db, {
agentId,
sessionId,
agentType: codexAgentRole(meta),
description: codexAgentNickname(meta),
durationMs: started && ended ? ended - started : null,
totalTokens: tokenTotal || null,
});
} else {
const pp = inferProjectPath(project, sm.cwds);
ins.ses.run(sessionId, sm.title, project, pp, sm.started_at, sm.ended_at, sm.git_branch, sm.version, sm.n, fi.path, 'codex');
}
ins.idx.run(fi.path, mt, lineNum);
}
function indexCodexSessionIndex(db) {
const indexPath = path.join(CODEX_DIR, 'session_index.jsonl');
if (!fs.existsSync(indexPath)) return;
readLines(indexPath, (line) => {
try {
const item = JSON.parse(line);
if (!item.id || !item.thread_name) return;
db.prepare('UPDATE sessions SET title=COALESCE(title, ?), ended_at=COALESCE(ended_at, ?) WHERE id=? AND source=?')
.run(item.thread_name, item.updated_at || null, codexDbId(item.id), 'codex');
} catch (e) {
process.stderr.write(`Warning: malformed Codex session index line: ${e.message}\n`);
}
});
}
function refreshSessionProjectPaths(db) { function refreshSessionProjectPaths(db) {
const sessions = db.prepare('SELECT id, project FROM sessions').all(); const sessions = db.prepare('SELECT id, project FROM sessions').all();
const cwdStmt = db.prepare(` const cwdStmt = db.prepare(`
@@ -275,12 +752,19 @@ function buildIndex({ force = false } = {}) {
db.prepare("DELETE FROM index_state WHERE jsonl_path != '__last_build__'").run(); db.prepare("DELETE FROM index_state WHERE jsonl_path != '__last_build__'").run();
} }
const files = discoverJsonlFiles(); const files = [
...discoverJsonlFiles(),
...discoverCodexJsonlFiles(),
];
for (const f of files) { for (const f of files) {
db.exec('BEGIN'); db.exec('BEGIN');
try { try {
indexJsonl(db, f); if (f.source === 'codex') {
indexSubagentMeta(db, f); indexCodexJsonl(db, f);
} else {
indexJsonl(db, f);
indexSubagentMeta(db, f);
}
db.exec('COMMIT'); db.exec('COMMIT');
} catch (e) { } catch (e) {
db.exec('ROLLBACK'); db.exec('ROLLBACK');
@@ -292,6 +776,7 @@ function buildIndex({ force = false } = {}) {
indexWorkflows(db); indexWorkflows(db);
refreshSessionProjectPaths(db); refreshSessionProjectPaths(db);
indexHistory(db); indexHistory(db);
indexCodexSessionIndex(db);
db.exec("INSERT INTO messages_fts(messages_fts) VALUES('rebuild')"); db.exec("INSERT INTO messages_fts(messages_fts) VALUES('rebuild')");
rebuildMemoryFts(db); rebuildMemoryFts(db);
db.prepare("INSERT OR REPLACE INTO index_state (jsonl_path, mtime, lines_processed) VALUES ('__last_build__', ?, 0)").run(Date.now()); db.prepare("INSERT OR REPLACE INTO index_state (jsonl_path, mtime, lines_processed) VALUES ('__last_build__', ?, 0)").run(Date.now());
+66 -17
View File
@@ -19,6 +19,10 @@ function buildWhere(opts, aliases) {
if (opts.after) { clauses.push(`${aliases.timestamp} > ?`); params.push(opts.after); } if (opts.after) { clauses.push(`${aliases.timestamp} > ?`); params.push(opts.after); }
if (opts.before) { clauses.push(`${aliases.timestamp} < ?`); params.push(opts.before); } if (opts.before) { clauses.push(`${aliases.timestamp} < ?`); params.push(opts.before); }
if (opts.branch) { clauses.push(`${aliases.branch} = ?`); params.push(opts.branch); } if (opts.branch) { clauses.push(`${aliases.branch} = ?`); params.push(opts.branch); }
if (opts.source && opts.source !== 'all' && aliases.source) {
clauses.push(`COALESCE(${aliases.source}, 'claude') = ?`);
params.push(opts.source);
}
return { where: clauses.length ? clauses.join(' AND ') : '1=1', params }; return { where: clauses.length ? clauses.join(' AND ') : '1=1', params };
} }
@@ -67,7 +71,7 @@ function createQueryApi(db) {
}; };
const search = (text, opts = {}) => { const search = (text, opts = {}) => {
const { limit = 20, sessionId, project, after, before, cwd, includeMeta = false } = opts; const { limit = 20, sessionId, project, after, before, cwd, source, includeMeta = false } = opts;
let where = 'WHERE mf.text MATCH ?'; let where = 'WHERE mf.text MATCH ?';
const p = [text]; const p = [text];
if (sessionId) { where += ' AND mf.session_id=?'; p.push(sessionId); } if (sessionId) { where += ' AND mf.session_id=?'; p.push(sessionId); }
@@ -75,22 +79,25 @@ function createQueryApi(db) {
if (after) { where += ' AND m.timestamp>?'; p.push(after); } if (after) { where += ' AND m.timestamp>?'; p.push(after); }
if (before) { where += ' AND m.timestamp<?'; p.push(before); } if (before) { where += ' AND m.timestamp<?'; p.push(before); }
if (cwd) { where += ' AND m.cwd LIKE ?'; p.push(cwd); } if (cwd) { where += ' AND m.cwd LIKE ?'; p.push(cwd); }
if (source && source !== 'all') { where += " AND COALESCE(m.source, s.source, 'claude')=?"; p.push(source); }
if (!includeMeta) where += ' AND COALESCE(m.is_meta,0)=0'; if (!includeMeta) where += ' AND COALESCE(m.is_meta,0)=0';
p.push(limit); p.push(limit);
const rows = db.prepare(` const rows = db.prepare(`
SELECT m.uuid,m.session_id,m.text,m.content_type,m.is_meta,m.role,m.timestamp,m.model,m.cwd, SELECT m.uuid,m.session_id,m.text,m.content_type,m.is_meta,m.role,m.timestamp,m.model,m.cwd,m.source as m_source,
s.id as s_id,s.title as s_title,s.project as s_project,s.started_at as s_started, s.id as s_id,s.title as s_title,s.project as s_project,s.started_at as s_started,
s.source as s_source,
rank rank
FROM messages_fts mf JOIN messages m ON m.uuid=mf.uuid LEFT JOIN sessions s ON s.id=m.session_id FROM messages_fts mf JOIN messages m ON m.uuid=mf.uuid LEFT JOIN sessions s ON s.id=m.session_id
${where} ORDER BY rank LIMIT ?`).all(...p); ${where} ORDER BY rank LIMIT ?`).all(...p);
return rows.map(r => { return rows.map(r => {
const metaClause = includeMeta ? '' : 'AND COALESCE(is_meta,0)=0'; const metaClause = includeMeta ? '' : 'AND COALESCE(is_meta,0)=0';
const ctx = db.prepare( const ctx = db.prepare(
`SELECT uuid,text,content_type,is_meta,role,timestamp,model FROM messages WHERE session_id=? AND uuid!=? ${metaClause} ORDER BY ABS(JULIANDAY(timestamp)-JULIANDAY(?)) LIMIT 6` `SELECT uuid,text,content_type,is_meta,role,timestamp,model,COALESCE(source, 'claude') as source FROM messages WHERE session_id=? AND uuid!=? ${metaClause} ORDER BY ABS(JULIANDAY(timestamp)-JULIANDAY(?)) LIMIT 6`
).all(r.session_id, r.uuid, r.timestamp).sort((a,b) => a.timestamp < b.timestamp ? -1 : 1); ).all(r.session_id, r.uuid, r.timestamp).sort((a,b) => a.timestamp < b.timestamp ? -1 : 1);
const sourceValue = r.m_source || r.s_source || 'claude';
return { return {
message: { uuid: r.uuid, text: r.text, content_type: r.content_type, is_meta: r.is_meta || 0, role: r.role, timestamp: r.timestamp, model: r.model, cwd: r.cwd }, message: { uuid: r.uuid, text: r.text, content_type: r.content_type, is_meta: r.is_meta || 0, role: r.role, timestamp: r.timestamp, model: r.model, cwd: r.cwd, source: sourceValue },
session: { id: r.s_id, title: r.s_title, project: r.s_project, started_at: r.s_started }, session: { id: r.s_id, title: r.s_title, project: r.s_project, started_at: r.s_started, source: r.s_source || sourceValue },
rank: r.rank, rank: r.rank,
context: ctx, context: ctx,
}; };
@@ -129,8 +136,8 @@ function createQueryApi(db) {
const subagents = (optsOrSid) => { const subagents = (optsOrSid) => {
const opts = normalizeOpts(optsOrSid); const opts = normalizeOpts(optsOrSid);
const { limit = 100 } = opts; const { limit = 100 } = opts;
const needsJoin = opts.project || opts.branch; const needsJoin = opts.project || opts.branch || opts.source;
const { where, params } = buildWhere(opts, { sessionId: 'sa.session_id', project: 's.project', timestamp: 'sa.session_id', branch: 's.git_branch' }); const { where, params } = buildWhere(opts, { sessionId: 'sa.session_id', project: 's.project', timestamp: 'sa.session_id', branch: 's.git_branch', source: 's.source' });
params.push(limit); params.push(limit);
const join = needsJoin ? 'LEFT JOIN sessions s ON s.id=sa.session_id' : ''; const join = needsJoin ? 'LEFT JOIN sessions s ON s.id=sa.session_id' : '';
return db.prepare(`SELECT sa.* FROM subagents sa ${join} WHERE ${where} LIMIT ?`).all(...params).map(r => { return db.prepare(`SELECT sa.* FROM subagents sa ${join} WHERE ${where} LIMIT ?`).all(...params).map(r => {
@@ -142,8 +149,8 @@ function createQueryApi(db) {
const workflows = (optsOrSid) => { const workflows = (optsOrSid) => {
const opts = normalizeOpts(optsOrSid); const opts = normalizeOpts(optsOrSid);
const { limit = 100 } = opts; const { limit = 100 } = opts;
const needsJoin = opts.project || opts.branch; const needsJoin = opts.project || opts.branch || opts.source;
const { where, params } = buildWhere(opts, { sessionId: 'w.session_id', project: 's.project', timestamp: 'w.timestamp', branch: 's.git_branch' }); const { where, params } = buildWhere(opts, { sessionId: 'w.session_id', project: 's.project', timestamp: 'w.timestamp', branch: 's.git_branch', source: 's.source' });
params.push(limit); params.push(limit);
const join = needsJoin ? 'LEFT JOIN sessions s ON s.id=w.session_id' : ''; const join = needsJoin ? 'LEFT JOIN sessions s ON s.id=w.session_id' : '';
return db.prepare(`SELECT w.* FROM workflows w ${join} WHERE ${where} ORDER BY w.timestamp DESC LIMIT ?`).all(...params); return db.prepare(`SELECT w.* FROM workflows w ${join} WHERE ${where} ORDER BY w.timestamp DESC LIMIT ?`).all(...params);
@@ -162,11 +169,12 @@ function createQueryApi(db) {
}; };
const fileHistory = (fp, opts = {}) => { const fileHistory = (fp, opts = {}) => {
const { limit = 200, after, before } = opts; const { limit = 200, after, before, source } = opts;
let where = 'tc.file_path=?'; let where = 'tc.file_path=?';
const params = [fp]; const params = [fp];
if (after) { where += ' AND m.timestamp > ?'; params.push(after); } if (after) { where += ' AND m.timestamp > ?'; params.push(after); }
if (before) { where += ' AND m.timestamp < ?'; params.push(before); } if (before) { where += ' AND m.timestamp < ?'; params.push(before); }
if (source && source !== 'all') { where += " AND COALESCE(s.source, 'claude') = ?"; params.push(source); }
params.push(limit); params.push(limit);
return db.prepare( return db.prepare(
`SELECT tc.*,s.title as s_title,s.project as s_project,m.timestamp as ts FROM tool_calls tc LEFT JOIN sessions s ON s.id=tc.session_id LEFT JOIN messages m ON m.uuid=tc.message_uuid WHERE ${where} ORDER BY m.timestamp LIMIT ?` `SELECT tc.*,s.title as s_title,s.project as s_project,m.timestamp as ts FROM tool_calls tc LEFT JOIN sessions s ON s.id=tc.session_id LEFT JOIN messages m ON m.uuid=tc.message_uuid WHERE ${where} ORDER BY m.timestamp LIMIT ?`
@@ -180,8 +188,8 @@ function createQueryApi(db) {
const failures = (optsOrSid) => { const failures = (optsOrSid) => {
const opts = normalizeOpts(optsOrSid); const opts = normalizeOpts(optsOrSid);
const { limit = 50 } = opts; const { limit = 50 } = opts;
const needsJoin = opts.project || opts.branch; const needsJoin = opts.project || opts.branch || opts.source;
const { where, params: filterParams } = buildWhere(opts, { sessionId: 'tr.session_id', project: 's.project', timestamp: 'rm.timestamp', branch: 's.git_branch' }); const { where, params: filterParams } = buildWhere(opts, { sessionId: 'tr.session_id', project: 's.project', timestamp: 'rm.timestamp', branch: 's.git_branch', source: 's.source' });
const join = needsJoin ? 'LEFT JOIN sessions s ON s.id=tr.session_id' : ''; const join = needsJoin ? 'LEFT JOIN sessions s ON s.id=tr.session_id' : '';
const errorCond = `(tr.is_error = 1 OR tr.content LIKE '${BASH_EXIT_PAT}')`; const errorCond = `(tr.is_error = 1 OR tr.content LIKE '${BASH_EXIT_PAT}')`;
const allParams = [...filterParams, limit]; const allParams = [...filterParams, limit];
@@ -198,7 +206,7 @@ function createQueryApi(db) {
const sessions = (optsOrN) => { const sessions = (optsOrN) => {
const opts = normalizeOpts(optsOrN, 'sessionId'); const opts = normalizeOpts(optsOrN, 'sessionId');
const { limit = 50 } = opts; const { limit = 50 } = opts;
const { where, params } = buildWhere(opts, { sessionId: 's.id', project: 's.project', timestamp: 's.started_at', branch: 's.git_branch' }); const { where, params } = buildWhere(opts, { sessionId: 's.id', project: 's.project', timestamp: 's.started_at', branch: 's.git_branch', source: 's.source' });
params.push(limit); params.push(limit);
return db.prepare(`SELECT * FROM sessions s WHERE ${where} ORDER BY ended_at DESC LIMIT ?`).all(...params); return db.prepare(`SELECT * FROM sessions s WHERE ${where} ORDER BY ended_at DESC LIMIT ?`).all(...params);
}; };
@@ -208,7 +216,7 @@ function createQueryApi(db) {
const summaries = (optsOrSid) => { const summaries = (optsOrSid) => {
const opts = normalizeOpts(optsOrSid); const opts = normalizeOpts(optsOrSid);
const { limit = 100 } = opts; const { limit = 100 } = opts;
const { where, params } = buildWhere(opts, { sessionId: 'su.session_id', project: 's.project', timestamp: 'su.timestamp', branch: 's.git_branch' }); const { where, params } = buildWhere(opts, { sessionId: 'su.session_id', project: 's.project', timestamp: 'su.timestamp', branch: 's.git_branch', source: 's.source' });
params.push(limit); params.push(limit);
return db.prepare(`SELECT su.*, s.title as session_title, s.project FROM summaries su LEFT JOIN sessions s ON s.id=su.session_id WHERE ${where} ORDER BY su.timestamp DESC LIMIT ?`).all(...params); return db.prepare(`SELECT su.*, s.title as session_title, s.project FROM summaries su LEFT JOIN sessions s ON s.id=su.session_id WHERE ${where} ORDER BY su.timestamp DESC LIMIT ?`).all(...params);
}; };
@@ -330,7 +338,7 @@ function createQueryApi(db) {
if (currentProject?.project) { if (currentProject?.project) {
const sessionTotal = db.prepare('SELECT COUNT(*) AS c FROM sessions WHERE project = ?').get(currentProject.project)?.c || 0; const sessionTotal = db.prepare('SELECT COUNT(*) AS c FROM sessions WHERE project = ?').get(currentProject.project)?.c || 0;
const sessionsForProject = db.prepare(` const sessionsForProject = db.prepare(`
SELECT id, title, project, project_path, started_at, ended_at, git_branch, message_count SELECT id, title, project, project_path, started_at, ended_at, git_branch, message_count, COALESCE(source, 'claude') AS source
FROM sessions FROM sessions
WHERE project = ? WHERE project = ?
ORDER BY COALESCE(ended_at, started_at) DESC ORDER BY COALESCE(ended_at, started_at) DESC
@@ -364,6 +372,14 @@ function createQueryApi(db) {
`).get()?.c || 0; `).get()?.c || 0;
const totalSessions = db.prepare('SELECT COUNT(*) AS c FROM sessions').get()?.c || 0; const totalSessions = db.prepare('SELECT COUNT(*) AS c FROM sessions').get()?.c || 0;
const totalMemories = db.prepare('SELECT COUNT(*) AS c FROM memories WHERE deleted_at IS NULL').get()?.c || 0; const totalMemories = db.prepare('SELECT COUNT(*) AS c FROM memories WHERE deleted_at IS NULL').get()?.c || 0;
const sources = db.prepare(`
SELECT COALESCE(source, 'claude') AS source,
COUNT(*) AS session_count,
MAX(COALESCE(ended_at, started_at)) AS last_session_at
FROM sessions
GROUP BY COALESCE(source, 'claude')
ORDER BY last_session_at DESC
`).all();
return { return {
current: { current: {
@@ -376,13 +392,29 @@ function createQueryApi(db) {
projects: totalProjects, projects: totalProjects,
sessions: totalSessions, sessions: totalSessions,
memories: totalMemories, memories: totalMemories,
sources,
}, },
}; };
}; };
const resolveJsonlPath = (messageUuid) => { const resolveJsonlPath = (messageUuid) => {
const msg = db.prepare('SELECT session_id, agent_id FROM messages WHERE uuid=?').get(messageUuid); const msg = db.prepare('SELECT session_id, agent_id, source FROM messages WHERE uuid=?').get(messageUuid);
if (!msg) return null; if (!msg) return null;
if (msg.source === 'codex' || String(messageUuid).startsWith('codex:')) {
const match = /^codex:([^:]+):(\d+)$/.exec(String(messageUuid));
if (!match) return null;
const rawThreadId = match[1];
if (!msg.agent_id) {
const ses = db.prepare('SELECT jsonl_path FROM sessions WHERE id=?').get(msg.session_id);
if (ses?.jsonl_path) return ses.jsonl_path;
}
return db.prepare(`
SELECT jsonl_path FROM index_state
WHERE jsonl_path LIKE ? AND jsonl_path LIKE '%.jsonl'
ORDER BY length(jsonl_path) ASC
LIMIT 1
`).get(`%${rawThreadId}.jsonl`)?.jsonl_path || null;
}
if (msg.agent_id) { if (msg.agent_id) {
const wa = db.prepare('SELECT agent_id, run_id, session_id FROM workflow_agents WHERE agent_id=?').get(msg.agent_id); const wa = db.prepare('SELECT agent_id, run_id, session_id FROM workflow_agents WHERE agent_id=?').get(msg.agent_id);
if (wa) { if (wa) {
@@ -401,8 +433,24 @@ function createQueryApi(db) {
return null; return null;
}; };
const findCodexRawLine = (jsonlPath, uuid) => {
const match = /^codex:[^:]+:(\d+)$/.exec(String(uuid));
if (!match || !jsonlPath || !fs.existsSync(jsonlPath)) return null;
const targetLine = Number(match[1]);
let lineNum = 0;
let found = null;
readLines(jsonlPath, (line) => {
lineNum++;
if (lineNum !== targetLine) return;
found = line;
return false;
});
return found;
};
const findRawLine = (jsonlPath, uuid) => { const findRawLine = (jsonlPath, uuid) => {
if (!jsonlPath || !fs.existsSync(jsonlPath)) return null; if (!jsonlPath || !fs.existsSync(jsonlPath)) return null;
if (String(uuid).startsWith('codex:')) return findCodexRawLine(jsonlPath, uuid);
let found = null; let found = null;
readLines(jsonlPath, (line) => { readLines(jsonlPath, (line) => {
if (!line.includes(uuid)) return; if (!line.includes(uuid)) return;
@@ -429,12 +477,13 @@ function createQueryApi(db) {
const opts = normalizeOpts(optsOrSid); const opts = normalizeOpts(optsOrSid);
const { limit = 50, query } = opts; const { limit = 50, query } = opts;
assertEnglishMemoryText(query, 'memories() query'); assertEnglishMemoryText(query, 'memories() query');
const needsJoin = opts.branch; const needsJoin = opts.branch || opts.source;
const { where: baseWhere, params } = buildWhere(opts, { const { where: baseWhere, params } = buildWhere(opts, {
sessionId: 'mem.session_id', sessionId: 'mem.session_id',
project: 'mem.project', project: 'mem.project',
timestamp: 'mem.created_at', timestamp: 'mem.created_at',
branch: 's.git_branch', branch: 's.git_branch',
source: 's.source',
}); });
let where = baseWhere + ' AND mem.deleted_at IS NULL'; let where = baseWhere + ' AND mem.deleted_at IS NULL';
const join = needsJoin ? 'LEFT JOIN sessions s ON s.id=mem.session_id' : ''; const join = needsJoin ? 'LEFT JOIN sessions s ON s.id=mem.session_id' : '';
+5 -2
View File
@@ -1,14 +1,15 @@
CREATE TABLE IF NOT EXISTS sessions ( CREATE TABLE IF NOT EXISTS sessions (
id TEXT PRIMARY KEY, title TEXT, project TEXT, project_path TEXT, id TEXT PRIMARY KEY, title TEXT, project TEXT, project_path TEXT,
started_at TEXT, ended_at TEXT, git_branch TEXT, version TEXT, started_at TEXT, ended_at TEXT, git_branch TEXT, version TEXT,
message_count INTEGER DEFAULT 0, jsonl_path TEXT); message_count INTEGER DEFAULT 0, jsonl_path TEXT, source TEXT DEFAULT 'claude');
CREATE TABLE IF NOT EXISTS messages ( CREATE TABLE IF NOT EXISTS messages (
uuid TEXT PRIMARY KEY, session_id TEXT, type TEXT, parent_uuid TEXT, uuid TEXT PRIMARY KEY, session_id TEXT, type TEXT, parent_uuid TEXT,
timestamp TEXT, role TEXT, text TEXT, content_type TEXT, timestamp TEXT, role TEXT, text TEXT, content_type TEXT,
is_meta INTEGER DEFAULT 0, model TEXT, is_meta INTEGER DEFAULT 0, model TEXT,
is_sidechain INTEGER DEFAULT 0, agent_id TEXT, is_sidechain INTEGER DEFAULT 0, agent_id TEXT,
input_tokens INTEGER, output_tokens INTEGER, input_tokens INTEGER, output_tokens INTEGER,
cwd TEXT, skill TEXT, turn_duration_ms INTEGER); cwd TEXT, skill TEXT, turn_duration_ms INTEGER,
source TEXT DEFAULT 'claude');
CREATE TABLE IF NOT EXISTS tool_calls ( CREATE TABLE IF NOT EXISTS tool_calls (
id TEXT PRIMARY KEY, message_uuid TEXT, session_id TEXT, id TEXT PRIMARY KEY, message_uuid TEXT, session_id TEXT,
name TEXT, input_json TEXT, file_path TEXT); name TEXT, input_json TEXT, file_path TEXT);
@@ -51,6 +52,8 @@ END;
CREATE INDEX IF NOT EXISTS idx_messages_session ON messages(session_id); CREATE INDEX IF NOT EXISTS idx_messages_session ON messages(session_id);
CREATE INDEX IF NOT EXISTS idx_messages_agent ON messages(agent_id); CREATE INDEX IF NOT EXISTS idx_messages_agent ON messages(agent_id);
CREATE INDEX IF NOT EXISTS idx_messages_ts ON messages(session_id, timestamp); CREATE INDEX IF NOT EXISTS idx_messages_ts ON messages(session_id, timestamp);
CREATE INDEX IF NOT EXISTS idx_sessions_source ON sessions(source);
CREATE INDEX IF NOT EXISTS idx_messages_source ON messages(source);
CREATE INDEX IF NOT EXISTS idx_tc_session_name ON tool_calls(session_id, name); CREATE INDEX IF NOT EXISTS idx_tc_session_name ON tool_calls(session_id, name);
CREATE INDEX IF NOT EXISTS idx_tc_file ON tool_calls(file_path); CREATE INDEX IF NOT EXISTS idx_tc_file ON tool_calls(file_path);
CREATE INDEX IF NOT EXISTS idx_sa_session ON subagents(session_id); CREATE INDEX IF NOT EXISTS idx_sa_session ON subagents(session_id);