feat: add registry-driven Kimi provider

This commit is contained in:
tommy0103
2026-07-20 22:43:23 +08:00
parent 21c3a1b9fc
commit 3ee44de4e5
39 changed files with 2240 additions and 732 deletions
+11 -8
View File
@@ -1,7 +1,7 @@
# Obelisk # Obelisk
Obelisk is explicit memory infrastructure for coding agents: it indexes local Obelisk is explicit memory infrastructure for coding agents: it indexes local
Claude Code and Codex transcripts into a queryable SQLite evidence layer, and a Claude Code, Codex, and Kimi Code sessions into a queryable SQLite evidence layer, and a
CodeAct runtime lets an agent write a small query, run it, and answer from real CodeAct runtime lets an agent write a small query, run it, and answer from real
session history. This glossary pins the terms that are specific to Obelisk; it is session history. This glossary pins the terms that are specific to Obelisk; it is
not a spec. not a spec.
@@ -29,9 +29,12 @@ promoted to an external tool surface.
## Indexing ## Indexing
**Provider adapter**: **Provider adapter**:
A pure per-source module (claude, codex, later opencode, pi, …) that discovers a A pure per-source module (claude, codex, kimi, later pi, …) that owns its
source's transcript files and parses one into a stream of records. It never opens descriptor, watch roots, discovery, parsing, cursor interpretation, and raw
or writes a database; adding a source means adding one adapter. The shared pure record lookup. It discovers `IndexUnit`s rather than assuming one transcript
file per unit; Kimi uses a session directory containing multiple wire logs. It
never opens or writes a database; adding a source means adding one adapter and
registering it. The shared pure
parse/discover helpers live in `packages/core/src/parsing.ts`, which imports only parse/discover helpers live in `packages/core/src/parsing.ts`, which imports only
node:fs/path/os — deliberately node:sqlite-free so the compiled providers can be node:fs/path/os — deliberately node:sqlite-free so the compiled providers can be
consumed by the app (whose Electron runtime has no `node:sqlite`). consumed by the app (whose Electron runtime has no `node:sqlite`).
@@ -62,10 +65,10 @@ active daemon: the command brings the index up to date, then answers.
_Avoid_: lazy indexing, on-read indexing _Avoid_: lazy indexing, on-read indexing
**index_state**: **index_state**:
The bookkeeping table shared by both indexing modes. It records, per transcript The bookkeeping table shared by both indexing modes. It stores the adapter's
path, the last-seen `mtime` and `lines_processed` (enabling resume-from-line numeric cursor pair in the existing `mtime` and `lines_processed` columns (a
incremental indexing), plus heartbeat/last-build markers used for daemon file adapter can use mtime + line offset; Kimi uses aggregate max-mtime + total
arbitration. lines), plus heartbeat/last-build markers used for daemon arbitration.
**Daemon arbitration**: **Daemon arbitration**:
The policy by which the passive pull mode detects a fresh daemon from the The policy by which the passive pull mode detects a fresh daemon from the
+10 -5
View File
@@ -9,7 +9,7 @@
[![version](https://img.shields.io/github/v/tag/tommy0103/obelisk?label=version&style=flat-square)](https://github.com/tommy0103/obelisk/releases) [![version](https://img.shields.io/github/v/tag/tommy0103/obelisk?label=version&style=flat-square)](https://github.com/tommy0103/obelisk/releases)
[![license](https://img.shields.io/badge/license-AGPL--3.0-blue.svg?style=flat-square)](LICENSE) [![license](https://img.shields.io/badge/license-AGPL--3.0-blue.svg?style=flat-square)](LICENSE)
Every past Claude Code and Codex session -- queryable by your agent, browsable by you. Past Claude Code, Codex, and Kimi Code sessions -- queryable by your agent, browsable by you.
</div> </div>
@@ -25,15 +25,20 @@ The agent writes JS queries, runs them locally, and answers in plain language.
**App side** — an Electron desktop app for humans to browse sessions, manage memories, view usage stats, and see weekly recap cards. **App side** — an Electron desktop app for humans to browse sessions, manage memories, view usage stats, and see weekly recap cards.
Both read from the same `~/.obelisk/obelisk.sqlite` database. The indexer reads Claude Code transcripts from `~/.claude/projects` and Codex transcripts from `~/.codex/sessions`. Both read from the same `~/.obelisk/obelisk.sqlite` database. The indexer reads Claude Code transcripts from `~/.claude/projects`, Codex transcripts from `~/.codex/sessions`, and Kimi Code sessions from `~/.kimi-code/sessions` (or `$KIMI_CODE_HOME/sessions`).
## Codex support ## Multi-provider support
Obelisk indexes Claude Code and Codex into the same SQLite schema instead of keeping separate databases. Rows carry a `source` value (`claude` or `codex`), and Codex IDs are prefixed with `codex:` so they cannot collide with Claude session IDs. Obelisk indexes every provider into the same SQLite schema instead of keeping separate databases. Rows carry a `source` value, and non-Claude IDs are provider-prefixed so they cannot collide.
Codex root threads become normal Obelisk sessions. Codex child threads are attached through the same `subagents` table when parent-thread metadata is available. Codex does not emit Claude-style workflow metadata, so workflow tables may be empty for Codex-only history. Codex root threads become normal Obelisk sessions. Codex child threads are attached through the same `subagents` table when parent-thread metadata is available. Codex does not emit Claude-style workflow metadata, so workflow tables may be empty for Codex-only history.
For live app refresh, Obelisk watches `~/.claude/projects` and `~/.codex/sessions`. It does not watch the whole `~/.codex` root. Codex's `session_index.jsonl` is used as lightweight title/update metadata during indexing, not as the message transcript source. Kimi session directories become one Obelisk session each. Main and child-agent
`wire.jsonl` streams are projected into the same messages, tools, summaries and
subagents tables. Undo/clear is handled as a full session replay, so retracted
wire records do not remain in the index.
For live app refresh, Obelisk watches the roots declared by every registered provider, including `~/.claude/projects`, `~/.codex/sessions`, and `~/.kimi-code/sessions`. Codex's `session_index.jsonl` is used as lightweight title/update metadata during indexing, not as the message transcript source.
## Skill: agent-first retrieval ## Skill: agent-first retrieval
+83 -157
View File
@@ -10,6 +10,12 @@ import { createIndexerService } from './indexer-service.ts';
import { createWorkerBuildIndex } from './indexer-worker-client.ts'; import { createWorkerBuildIndex } from './indexer-worker-client.ts';
import { buildRecapExportQuery } from './recap-capture-query.ts'; import { buildRecapExportQuery } from './recap-capture-query.ts';
import { acquireWriterLease, writerLockPathFor } from '../../../packages/core/src/writer-lease.ts'; import { acquireWriterLease, writerLockPathFor } from '../../../packages/core/src/writer-lease.ts';
import { createBuiltinProviderRegistry } from '../../../packages/core/src/providers/builtins.ts';
import {
buildSourceCatalog,
resolveProviderRoots,
setPersistedSetting,
} from './provider-settings.ts';
import type { import type {
SessionPatchCursor, SessionPatchCursor,
SessionPatchSnapshot, SessionPatchSnapshot,
@@ -69,18 +75,18 @@ function acquireAppWriterLease(dbPath: string, waitMs = 0) {
}); });
} }
function getConfiguredClaudeDir() { function getRuntimePaths(persisted = loadPersistedSettings()) {
const persisted = loadPersistedSettings(); const defaultRegistry = createBuiltinProviderRegistry({
return persisted.claudeDir || DEFAULT_CLAUDE_DIR; claude: DEFAULT_CLAUDE_DIR,
} codex: DEFAULT_CODEX_DIR,
});
function getConfiguredCodexDir() { const providerRoots = resolveProviderRoots(defaultRegistry, persisted);
const persisted = loadPersistedSettings(); const providerRegistry = createBuiltinProviderRegistry(providerRoots);
return persisted.codexDir || DEFAULT_CODEX_DIR; const claudeDir = providerRoots['claude'] ?? DEFAULT_CLAUDE_DIR;
} const codexDir = providerRoots['codex'] ?? DEFAULT_CODEX_DIR;
function getPathsForClaudeDir(claudeDir = getConfiguredClaudeDir(), codexDir = getConfiguredCodexDir()) {
return { return {
providerRoots,
providerRegistry,
claudeDir, claudeDir,
codexDir, codexDir,
dbPath: path.join(OBELISK_DIR, 'obelisk.sqlite'), dbPath: path.join(OBELISK_DIR, 'obelisk.sqlite'),
@@ -89,7 +95,7 @@ function getPathsForClaudeDir(claudeDir = getConfiguredClaudeDir(), codexDir = g
} }
function migrateLegacyDbIfNeeded( function migrateLegacyDbIfNeeded(
paths = getPathsForClaudeDir(), paths = getRuntimePaths(),
{ writerLeaseMode = 'acquire' }: { writerLeaseMode?: WriterLeaseMode } = {}, { writerLeaseMode = 'acquire' }: { writerLeaseMode?: WriterLeaseMode } = {},
) { ) {
if (fs.existsSync(paths.dbPath)) return; if (fs.existsSync(paths.dbPath)) return;
@@ -191,7 +197,7 @@ function closeDb() {
} }
function openDb( function openDb(
dbPath = getPathsForClaudeDir().dbPath, dbPath = getRuntimePaths().dbPath,
{ writerLeaseMode = 'acquire' }: { writerLeaseMode?: WriterLeaseMode } = {}, { writerLeaseMode = 'acquire' }: { writerLeaseMode?: WriterLeaseMode } = {},
) { ) {
closeDb(); closeDb();
@@ -212,7 +218,7 @@ function openDb(
function runAppDbWrite(work: () => void): boolean { function runAppDbWrite(work: () => void): boolean {
if (!db) return false; if (!db) return false;
const lease = acquireAppWriterLease(getPathsForClaudeDir().dbPath, 250); const lease = acquireAppWriterLease(getRuntimePaths().dbPath, 250);
if (!lease) { if (!lease) {
throw new Error('Obelisk index writer is busy; memory change was not applied'); throw new Error('Obelisk index writer is busy; memory change was not applied');
} }
@@ -249,16 +255,16 @@ function appendWhere(sql, params, clause) {
} }
function startIndexerService({ buildOnStart = false } = {}) { function startIndexerService({ buildOnStart = false } = {}) {
const paths = getPathsForClaudeDir(); const paths = getRuntimePaths();
migrateLegacyDbIfNeeded(paths); migrateLegacyDbIfNeeded(paths);
const codexSessionsDir = path.join(paths.codexDir, 'sessions');
indexerService = createIndexerService({ indexerService = createIndexerService({
projectsDir: paths.projectsDir, projectsDir: paths.projectsDir,
watchDirs: [paths.projectsDir, codexSessionsDir], watchDirs: paths.providerRegistry.watchRoots(paths.providerRoots),
buildIndex: async ({ reason, changedPaths }) => { buildIndex: async ({ reason, changedPaths }) => {
const result = await indexerWorker.buildIndex({ const result = await indexerWorker.buildIndex({
reason, reason,
changedPaths, changedPaths,
providerRoots: paths.providerRoots,
claudeDir: paths.claudeDir, claudeDir: paths.claudeDir,
codexDir: paths.codexDir, codexDir: paths.codexDir,
projectsDir: paths.projectsDir, projectsDir: paths.projectsDir,
@@ -282,7 +288,7 @@ function startIndexerService({ buildOnStart = false } = {}) {
function startBackgroundResources({ runStartupBuild = false } = {}) { function startBackgroundResources({ runStartupBuild = false } = {}) {
if (!indexerWorker) indexerWorker = createWorkerBuildIndex(); if (!indexerWorker) indexerWorker = createWorkerBuildIndex();
const paths = getPathsForClaudeDir(); const paths = getRuntimePaths();
migrateLegacyDbIfNeeded(paths); migrateLegacyDbIfNeeded(paths);
openDb(paths.dbPath); openDb(paths.dbPath);
if (!indexerService) { if (!indexerService) {
@@ -575,87 +581,25 @@ ipcMain.handle('db:getMemories', () => {
ipcMain.handle('db:getMessageFullText', (_, uuid) => { ipcMain.handle('db:getMessageFullText', (_, uuid) => {
if (!db) return null; if (!db) return null;
const msg = db.prepare('SELECT session_id, agent_id, source FROM messages WHERE uuid=?').get(uuid); const msg = db.prepare('SELECT * FROM messages WHERE uuid=?').get(uuid);
if (!msg) return null; if (!msg) return null;
const session = db.prepare('SELECT * FROM sessions WHERE id=?').get(msg.session_id) ?? null;
if (msg.source === 'codex' || String(uuid).startsWith('codex:')) { const subagent = msg.agent_id
const match = /^codex:([^:]+):(\d+)$/.exec(String(uuid)); ? db.prepare('SELECT * FROM subagents WHERE agent_id=?').get(msg.agent_id) ?? null
if (!match) return null; : null;
const rawThreadId = match[1]; const workflowAgent = msg.agent_id
const targetLine = Number(match[2]); ? db.prepare('SELECT * FROM workflow_agents WHERE agent_id=?').get(msg.agent_id) ?? null
let jsonlPath: string | null = null; : null;
if (!msg.agent_id) { const paths = getRuntimePaths();
jsonlPath = db.prepare('SELECT jsonl_path FROM sessions WHERE id=?').get(msg.session_id)?.jsonl_path || null; const raw = paths.providerRegistry.raw({
} source: msg.source || session?.source || 'claude',
if (!jsonlPath) { messageUuid: String(uuid),
jsonlPath = db.prepare(` session,
SELECT jsonl_path FROM index_state agentId: msg.agent_id || null,
WHERE jsonl_path LIKE ? AND jsonl_path LIKE '%.jsonl' subagent,
ORDER BY length(jsonl_path) ASC workflowAgent,
LIMIT 1 });
`).get(`%${rawThreadId}.jsonl`)?.jsonl_path || null; return raw?.messageText ?? msg.text ?? null;
}
if (!jsonlPath || !fs.existsSync(jsonlPath)) return null;
const lines = fs.readFileSync(jsonlPath, 'utf-8').split('\n').filter(Boolean);
const line = lines[targetLine - 1];
if (!line) return null;
try {
const obj = JSON.parse(line);
const payload = obj.payload || {};
if (obj.type === 'event_msg') {
if (typeof payload.message === 'string') return payload.message;
if (typeof payload.text === 'string') return payload.text;
}
if (obj.type === 'response_item' && payload.type === 'message' && Array.isArray(payload.content)) {
const parts = payload.content.map(b => b.text).filter(Boolean);
return parts.join('\n') || null;
}
} catch {}
return null;
}
// Resolve JSONL path
let jsonlPath: string | null = null;
if (msg.agent_id) {
const wa = db.prepare('SELECT agent_id, run_id, session_id FROM workflow_agents WHERE agent_id=?').get(msg.agent_id);
if (wa) {
const ses = db.prepare('SELECT jsonl_path FROM sessions WHERE id=?').get(wa.session_id);
if (ses) jsonlPath = path.join(path.dirname(ses.jsonl_path), wa.session_id, 'subagents', 'workflows', wa.run_id, wa.agent_id + '.jsonl');
}
if (!jsonlPath) {
const sa = db.prepare('SELECT agent_id, session_id FROM subagents WHERE agent_id=?').get(msg.agent_id);
if (sa) {
const ses = db.prepare('SELECT jsonl_path FROM sessions WHERE id=?').get(sa.session_id);
if (ses) jsonlPath = path.join(path.dirname(ses.jsonl_path), sa.session_id, 'subagents', sa.agent_id + '.jsonl');
}
}
}
if (!jsonlPath) {
const ses = db.prepare('SELECT jsonl_path FROM sessions WHERE id=?').get(msg.session_id);
if (ses) jsonlPath = ses.jsonl_path;
}
if (!jsonlPath || !fs.existsSync(jsonlPath)) return null;
// Scan JSONL for the message UUID and extract full text
const data = fs.readFileSync(jsonlPath, 'utf-8');
const lines = data.split('\n');
for (const line of lines) {
if (!line.includes(uuid)) continue;
try {
const obj = JSON.parse(line);
if (obj.uuid !== uuid) continue;
const content = obj.message?.content;
if (typeof content === 'string') return content;
if (!Array.isArray(content)) return null;
const parts: string[] = [];
for (const b of content) {
if (b.type === 'text' && b.text) parts.push(b.text);
else if (b.type === 'thinking' && b.thinking) parts.push(b.thinking);
}
return parts.join('\n') || null;
} catch { continue; }
}
return null;
}); });
ipcMain.handle('db:readMemoryFile', (_, filePath) => { ipcMain.handle('db:readMemoryFile', (_, filePath) => {
@@ -849,76 +793,59 @@ function savePersistedSettings(settings) {
ipcMain.handle('settings:get', () => { ipcMain.handle('settings:get', () => {
const persisted = loadPersistedSettings(); const persisted = loadPersistedSettings();
const { claudeDir, codexDir, dbPath: dbFile } = getPathsForClaudeDir( const paths = getRuntimePaths(persisted);
persisted.claudeDir || DEFAULT_CLAUDE_DIR, const { providerRoots, providerRegistry, claudeDir, codexDir, dbPath: dbFile } = paths;
persisted.codexDir || DEFAULT_CODEX_DIR,
);
const recapDir = persisted.recapDir || RECAP_DIR; const recapDir = persisted.recapDir || RECAP_DIR;
const claudeExists = fs.existsSync(claudeDir);
const codexExists = fs.existsSync(codexDir);
let claudeSessionCount = 0;
let codexSessionCount = 0;
let memoryCount = 0; let memoryCount = 0;
let claudeLastIndexed = ''; const sourceStats = new Map<string, { sessionCount: number; lastIndexed: string }>();
let codexLastIndexed = '';
if (db) { if (db) {
try { try {
claudeSessionCount = db.prepare("SELECT COUNT(*) as c FROM sessions WHERE COALESCE(source, 'claude') = 'claude'").get()?.c || 0; const rows = db.prepare(`
codexSessionCount = db.prepare("SELECT COUNT(*) as c FROM sessions WHERE source = 'codex'").get()?.c || 0; SELECT COALESCE(source, 'claude') AS source,
COUNT(*) AS session_count,
MAX(started_at) AS last_indexed
FROM sessions
GROUP BY COALESCE(source, 'claude')
`).all();
for (const row of rows) {
sourceStats.set(row.source, {
sessionCount: row.session_count || 0,
lastIndexed: row.last_indexed || '',
});
}
memoryCount = db.prepare('SELECT COUNT(*) as c FROM memories WHERE deleted_at IS NULL').get()?.c || 0; memoryCount = db.prepare('SELECT COUNT(*) as c FROM memories WHERE deleted_at IS NULL').get()?.c || 0;
const claudeLatest = db.prepare("SELECT MAX(started_at) as t FROM sessions WHERE COALESCE(source, 'claude') = 'claude'").get();
claudeLastIndexed = claudeLatest?.t || '';
const codexLatest = db.prepare("SELECT MAX(started_at) as t FROM sessions WHERE source = 'codex'").get();
codexLastIndexed = codexLatest?.t || '';
} catch {} } catch {}
} }
const sources = buildSourceCatalog({
registry: providerRegistry,
roots: providerRoots,
stats: sourceStats,
pathExists: fs.existsSync,
});
const sessionCount = sources.reduce((sum, source) => sum + source.sessionCount, 0);
const lastIndexed = sources.map((source) => source.lastIndexed).filter(Boolean).sort().at(-1) || '';
const connected = sources.some((source) => source.status !== 'error');
return { return {
providerRoots,
claudeDir, claudeDir,
codexDir, codexDir,
dbPath: dbFile, dbPath: dbFile,
recapDir, recapDir,
autoRefresh: persisted.autoRefresh !== false, autoRefresh: persisted.autoRefresh !== false,
sources: [ sources,
{
id: 'claude',
name: 'Claude Code',
vendor: 'Anthropic',
path: claudeDir,
exists: claudeExists,
sessionCount: claudeSessionCount,
lastIndexed: claudeLastIndexed,
status: claudeExists ? 'ok' : 'error',
statusText: claudeExists ? 'Connected' : 'Folder not found',
},
{
id: 'codex',
name: 'Codex',
vendor: 'OpenAI',
path: codexDir,
exists: codexExists,
sessionCount: codexSessionCount,
lastIndexed: codexLastIndexed,
status: codexExists ? (codexSessionCount > 0 ? 'ok' : 'warn') : 'error',
statusText: codexExists ? (codexSessionCount > 0 ? 'Connected' : 'No sessions found') : 'Folder not found',
},
],
memoryCount, memoryCount,
sessionCount: claudeSessionCount + codexSessionCount, sessionCount,
lastIndexed: claudeLastIndexed, lastIndexed,
status: claudeExists ? 'ok' : 'error', status: connected ? 'ok' : 'error',
statusText: claudeExists ? 'Connected' : 'Folder not found', statusText: connected ? 'Connected' : 'No source folders found',
}; };
}); });
ipcMain.handle('settings:set', async (_, key, value) => { ipcMain.handle('settings:set', async (_, key, value) => {
const persisted = loadPersistedSettings(); const persisted = loadPersistedSettings();
if (value === null) { const providerRootChanged = setPersistedSetting(persisted, key, value);
delete persisted[key];
} else {
persisted[key] = value;
}
savePersistedSettings(persisted); savePersistedSettings(persisted);
if (key === 'autoRefresh') { if (key === 'autoRefresh') {
@@ -930,12 +857,13 @@ ipcMain.handle('settings:set', async (_, key, value) => {
} }
} }
if (key === 'claudeDir' || key === 'codexDir') { const knownLegacyRootChanged = createBuiltinProviderRegistry({
claude: DEFAULT_CLAUDE_DIR,
codex: DEFAULT_CODEX_DIR,
}).catalog().some((provider) => key === `${provider.id}Dir`);
if (providerRootChanged || knownLegacyRootChanged) {
await stopIndexerServiceAndWait(); await stopIndexerServiceAndWait();
const paths = getPathsForClaudeDir( const paths = getRuntimePaths(persisted);
persisted.claudeDir || DEFAULT_CLAUDE_DIR,
persisted.codexDir || DEFAULT_CODEX_DIR,
);
migrateLegacyDbIfNeeded(paths); migrateLegacyDbIfNeeded(paths);
openDb(paths.dbPath); openDb(paths.dbPath);
if (persisted.autoRefresh !== false) { if (persisted.autoRefresh !== false) {
@@ -951,7 +879,7 @@ ipcMain.handle('settings:browseFolder', async (event) => {
if (!win) return null; if (!win) return null;
const { filePaths } = await dialog.showOpenDialog(win, { const { filePaths } = await dialog.showOpenDialog(win, {
properties: ['openDirectory'], properties: ['openDirectory'],
title: 'Select Claude Code data folder', title: 'Select session data folder',
}); });
if (filePaths && filePaths[0]) return filePaths[0]; if (filePaths && filePaths[0]) return filePaths[0];
return null; return null;
@@ -964,10 +892,7 @@ ipcMain.handle('settings:revealPath', (_, p) => {
ipcMain.handle('settings:rebuildIndex', async () => { ipcMain.handle('settings:rebuildIndex', async () => {
if (!indexerWorker) return null; if (!indexerWorker) return null;
const persisted = loadPersistedSettings(); const persisted = loadPersistedSettings();
const paths = getPathsForClaudeDir( const paths = getRuntimePaths(persisted);
persisted.claudeDir || DEFAULT_CLAUDE_DIR,
persisted.codexDir || DEFAULT_CODEX_DIR,
);
const tempDbPath = rebuildTempDbPath(paths.dbPath); const tempDbPath = rebuildTempDbPath(paths.dbPath);
const shouldRestartWatcher = persisted.autoRefresh !== false; const shouldRestartWatcher = persisted.autoRefresh !== false;
await stopIndexerServiceAndWait({ waitForIdle: false }); await stopIndexerServiceAndWait({ waitForIdle: false });
@@ -1000,6 +925,7 @@ ipcMain.handle('settings:rebuildIndex', async () => {
const result = await indexerWorker.buildIndex({ const result = await indexerWorker.buildIndex({
reason: 'manual-rebuild', reason: 'manual-rebuild',
force: true, force: true,
providerRoots: paths.providerRoots,
claudeDir: paths.claudeDir, claudeDir: paths.claudeDir,
codexDir: paths.codexDir, codexDir: paths.codexDir,
projectsDir: paths.projectsDir, projectsDir: paths.projectsDir,
+4 -2
View File
@@ -74,11 +74,13 @@ function createIndexerService({
const existingRoots = roots.filter(root => fs.existsSync(root)); const existingRoots = roots.filter(root => fs.existsSync(root));
if (!existingRoots.length) return null; if (!existingRoots.length) return null;
const watchers: any[] = []; const watchers: any[] = [];
for (const root of existingRoots) {
const onFileChange = (filename) => { const onFileChange = (filename) => {
const name = filename ? String(filename) : ''; const name = filename ? String(filename) : '';
if (!name || name.endsWith('.jsonl') || name.endsWith('.json')) onChange(name); if (!name || name.endsWith('.jsonl') || name.endsWith('.json')) {
onChange(name && !path.isAbsolute(name) ? path.join(root, name) : name);
}
}; };
for (const root of existingRoots) {
const watcher = (chokidar || chokidarModule).watch(root, { const watcher = (chokidar || chokidarModule).watch(root, {
cwd: root, cwd: root,
ignoreInitial: true, ignoreInitial: true,
+63 -313
View File
@@ -3,12 +3,13 @@ import os from 'node:os';
import path from 'node:path'; import path from 'node:path';
import { fileURLToPath } from 'node:url'; import { fileURLToPath } from 'node:url';
import Database from 'better-sqlite3'; import Database from 'better-sqlite3';
import { createBuiltinProviderRegistry } from '../../../packages/core/src/providers/builtins.ts';
import type { ProviderRegistry } from '../../../packages/core/src/providers/registry.ts';
import { import {
CLAUDE_INPUT_TOKEN_SEMANTICS_MARKER, createProviderIndexPlan,
parse as claudeParse, indexProviderPlan,
} from '../../../packages/core/src/providers/claude.ts'; writeProviderIndexMarkers,
import { parse as codexParse } from '../../../packages/core/src/providers/codex.ts'; } from '../../../packages/core/src/provider-indexing.ts';
import { persist } from '../../../packages/core/src/persist.ts';
import { runWriteTransaction, configureConnection, betterSqliteTransactionAdapter } from '../../../packages/core/src/tx.ts'; import { runWriteTransaction, configureConnection, betterSqliteTransactionAdapter } from '../../../packages/core/src/tx.ts';
import { acquireWriterLease, writerLockPathFor } from '../../../packages/core/src/writer-lease.ts'; import { acquireWriterLease, writerLockPathFor } from '../../../packages/core/src/writer-lease.ts';
import { runRetryableWriteTransaction, isBeginBusyFailure, hasUnusableTransaction } from '../../../packages/core/src/write-coordinator.ts'; import { runRetryableWriteTransaction, isBeginBusyFailure, hasUnusableTransaction } from '../../../packages/core/src/write-coordinator.ts';
@@ -17,9 +18,6 @@ import {
isDir, isDir,
readLines, readLines,
codexDbId, codexDbId,
codexRawId,
codexParentThreadId,
readCodexGuardianThreadInfo,
} from '../../../packages/core/src/parsing.ts'; } from '../../../packages/core/src/parsing.ts';
const __dirname = path.dirname(fileURLToPath(import.meta.url)); const __dirname = path.dirname(fileURLToPath(import.meta.url));
@@ -31,16 +29,6 @@ const DEFAULT_DB_PATH = path.join(DEFAULT_OBELISK_DIR, 'obelisk.sqlite');
const DEFAULT_PROJECTS_DIR = path.join(DEFAULT_CLAUDE_DIR, 'projects'); const DEFAULT_PROJECTS_DIR = path.join(DEFAULT_CLAUDE_DIR, 'projects');
const DEFAULT_HISTORY_PATH = path.join(DEFAULT_CLAUDE_DIR, 'history.jsonl'); const DEFAULT_HISTORY_PATH = path.join(DEFAULT_CLAUDE_DIR, 'history.jsonl');
interface FileInfo {
path: string;
sessionId?: string;
project?: string;
isSubagent?: boolean;
agentId?: string;
workflowRunId?: string;
source?: string;
}
function resolveSchemaPath() { function resolveSchemaPath() {
const candidates = [ const candidates = [
path.join(__dirname, 'schema.sql'), path.join(__dirname, 'schema.sql'),
@@ -119,54 +107,12 @@ function copyMemoriesFromDb(db, sourceDbPath) {
} }
} }
function discoverJsonlFiles({ projectsDir = DEFAULT_PROJECTS_DIR, changedPaths = undefined }: { projectsDir?: string; changedPaths?: string[] } = {}) {
if (Array.isArray(changedPaths) && changedPaths.length) {
const changedFiles = discoverJsonlFilesForChanges({ projectsDir, changedPaths });
if (changedFiles.length) return changedFiles;
}
return discoverJsonlFilesFull({ projectsDir });
}
function normalizeChangedPath(projectsDir, changedPath) { function normalizeChangedPath(projectsDir, changedPath) {
if (!changedPath) return null; if (!changedPath) return null;
const raw = String(changedPath); const raw = String(changedPath);
return path.isAbsolute(raw) ? path.normalize(raw) : path.normalize(path.join(projectsDir, raw)); return path.isAbsolute(raw) ? path.normalize(raw) : path.normalize(path.join(projectsDir, raw));
} }
function jsonlFileInfoFromPath(projectsDir, changedPath) {
let fp = normalizeChangedPath(projectsDir, changedPath);
if (fp?.toLowerCase().endsWith('.meta.json')) {
fp = fp.slice(0, -'.meta.json'.length) + '.jsonl';
}
if (!fp || !fp.endsWith('.jsonl')) return null;
if (!fs.existsSync(fp)) return null;
const rel = path.relative(projectsDir, fp);
if (!rel || rel.startsWith('..') || path.isAbsolute(rel)) return null;
const parts = rel.split(path.sep);
const project = parts[0];
if (!project) return null;
if (parts.length === 2) {
const filename = parts[1];
return { path: fp, sessionId: filename.slice(0, -6), project, isSubagent: false };
}
if (parts.length === 4 && parts[2] === 'subagents') {
const filename = parts[3];
return { path: fp, sessionId: parts[1], project, isSubagent: true, agentId: filename.slice(0, -6) };
}
if (parts.length === 6 && parts[2] === 'subagents' && parts[3] === 'workflows') {
const filename = parts[5];
return {
path: fp,
sessionId: parts[1],
project,
isSubagent: true,
agentId: filename.slice(0, -6),
workflowRunId: parts[4],
};
}
return null;
}
function sessionIdFromChangedPath(projectsDir, changedPath) { function sessionIdFromChangedPath(projectsDir, changedPath) {
const fp = normalizeChangedPath(projectsDir, changedPath); const fp = normalizeChangedPath(projectsDir, changedPath);
if (!fp) return null; if (!fp) return null;
@@ -180,186 +126,6 @@ function sessionIdFromChangedPath(projectsDir, changedPath) {
return null; return null;
} }
function dedupeFileInfos(files) {
const byPath = new Map();
for (const file of files) byPath.set(file.path, file);
return [...byPath.values()];
}
function discoverJsonlFilesForChanges({ projectsDir = DEFAULT_PROJECTS_DIR, changedPaths = [] }: { projectsDir?: string; changedPaths?: string[] } = {}) {
const files: FileInfo[] = [];
for (const changedPath of changedPaths) {
const info = jsonlFileInfoFromPath(projectsDir, changedPath);
if (info) files.push(info);
}
return dedupeFileInfos(files);
}
function discoverJsonlFilesFull({ projectsDir = DEFAULT_PROJECTS_DIR } = {}) {
const files: FileInfo[] = [];
if (!fs.existsSync(projectsDir)) return files;
let projects;
try { projects = fs.readdirSync(projectsDir); } catch { return files; }
for (const proj of projects) {
const projPath = path.join(projectsDir, proj);
if (!isDir(projPath)) continue;
let entries;
try { entries = fs.readdirSync(projPath); } catch { continue; }
for (const f of entries) {
if (f.endsWith('.jsonl'))
files.push({ path: path.join(projPath, f), sessionId: f.slice(0, -6), project: proj, isSubagent: false });
}
for (const sd of entries) {
const saDir = path.join(projPath, sd, 'subagents');
if (!isDir(saDir)) continue;
let saEntries;
try { saEntries = fs.readdirSync(saDir); } catch { continue; }
for (const sf of saEntries) {
if (sf.endsWith('.jsonl'))
files.push({ path: path.join(saDir, sf), sessionId: sd, project: proj, isSubagent: true, agentId: sf.slice(0, -6) });
}
const wfRoot = path.join(saDir, 'workflows');
if (!isDir(wfRoot)) continue;
let wfDirs;
try { wfDirs = fs.readdirSync(wfRoot); } catch { continue; }
for (const wfDir of wfDirs) {
const wfPath = path.join(wfRoot, wfDir);
if (!isDir(wfPath)) continue;
let wfEntries;
try { wfEntries = fs.readdirSync(wfPath); } catch { continue; }
for (const wf of wfEntries) {
if (wf.endsWith('.jsonl'))
files.push({ path: path.join(wfPath, wf), sessionId: sd, project: proj, isSubagent: true, agentId: wf.slice(0, -6), workflowRunId: wfDir });
}
}
}
}
return files;
}
function discoverCodexJsonlFiles({ codexDir = DEFAULT_CODEX_DIR, changedPaths = undefined }: { codexDir?: string; changedPaths?: string[] } = {}) {
if (Array.isArray(changedPaths) && changedPaths.length) {
const changedFiles = discoverCodexJsonlFilesForChanges({ codexDir, changedPaths });
if (changedFiles.length) return changedFiles;
return [];
}
return discoverCodexJsonlFilesFull({ codexDir });
}
function codexSessionsDir(codexDir = DEFAULT_CODEX_DIR) {
return path.join(codexDir, 'sessions');
}
function normalizeChangedPathForRoot(rootDir, changedPath) {
if (!changedPath) return null;
const raw = String(changedPath);
return path.isAbsolute(raw) ? path.normalize(raw) : path.normalize(path.join(rootDir, raw));
}
function isPathInside(rootDir, candidate) {
if (!rootDir || !candidate) return false;
const rel = path.relative(rootDir, candidate);
return !!rel && !rel.startsWith('..') && !path.isAbsolute(rel);
}
function discoverCodexJsonlFilesForChanges({ codexDir = DEFAULT_CODEX_DIR, changedPaths = [] }: { codexDir?: string; changedPaths?: string[] } = {}) {
const files: FileInfo[] = [];
const sessionsDir = codexSessionsDir(codexDir);
for (const changedPath of changedPaths) {
const rootRelativePath = normalizeChangedPathForRoot(codexDir, changedPath);
if (!rootRelativePath) continue;
if (path.normalize(rootRelativePath) === path.join(codexDir, 'session_index.jsonl')) {
return discoverCodexJsonlFilesFull({ codexDir });
}
const sessionRelativePath = normalizeChangedPathForRoot(sessionsDir, changedPath);
const fp = isPathInside(sessionsDir, rootRelativePath) ? rootRelativePath : sessionRelativePath;
if (!fp || !fp.endsWith(".jsonl") || !isPathInside(sessionsDir, fp)) continue;
if (!fs.existsSync(fp)) continue;
files.push({ path: fp, source: 'codex' });
}
return dedupeFileInfos(files);
}
function discoverCodexJsonlFilesFull({ codexDir = DEFAULT_CODEX_DIR } = {}) {
const root = codexSessionsDir(codexDir);
const files: FileInfo[] = [];
if (!fs.existsSync(root)) return files;
const stack = [root];
while (stack.length) {
const current = stack.pop()!;
let entries;
try { entries = fs.readdirSync(current, { withFileTypes: true }); } catch { continue; }
for (const entry of entries) {
const fp = path.join(current, entry.name);
if (entry.isDirectory()) {
stack.push(fp);
} else if (entry.isFile() && entry.name.endsWith('.jsonl')) {
files.push({ path: fp, source: 'codex' });
}
}
}
return files.sort((a, b) => a.path.localeCompare(b.path));
}
function needsReindex(db, fp) {
const mt = fs.statSync(fp).mtimeMs;
const row = db.prepare('SELECT mtime, lines_processed FROM index_state WHERE jsonl_path = ?').get(fp);
if (!row) return { needed: true, skip: 0, mtime: mt };
return mt > row.mtime ? { needed: true, skip: row.lines_processed, mtime: mt } : { needed: false, skip: 0, mtime: mt };
}
// Index one Claude transcript via the shared provider + persist core.
// Returns { sessionId, path } when reindexed, undefined when skipped.
function indexClaudeFile(db, file, { forceFull = false } = {}) {
const { needed, skip, mtime } = needsReindex(db, file.path);
if (!needed && !forceFull) return undefined;
const unit = {
key: file.path,
sessionId: file.sessionId,
project: file.project,
isSubagent: file.isSubagent,
agentId: file.agentId,
};
const cursor = !forceFull && skip > 0 ? `${mtime}:${skip}` : null;
persist(db, unit, claudeParse(unit, cursor));
return { sessionId: file.sessionId, path: file.path };
}
function codexSessionMeta(filePath) {
let meta: any = null;
readLines(filePath, (line) => {
let obj;
try { obj = JSON.parse(line); } catch { return; }
if (obj?.type === 'session_meta' && obj.payload?.id) {
meta = obj.payload;
return false;
}
});
return meta;
}
// Index one Codex rollout via the shared provider + persist core (full reparse).
// Returns { sessionId, path } when reindexed, undefined when skipped.
function indexCodexFile(db, file) {
const { needed } = needsReindex(db, file.path);
const guardian = readCodexGuardianThreadInfo(file.path);
if (!needed) {
if (guardian) {
persist(db, { key: file.path, sessionId: '' }, (function* () {
yield { kind: "delete-session", sessionId: codexDbId(guardian.threadRawId) as string };
return null;
})());
}
return undefined;
}
const unit = { key: file.path, sessionId: '' };
persist(db, unit, codexParse(unit, null));
if (guardian) return undefined;
const meta = codexSessionMeta(file.path);
const sessionId = meta ? codexDbId(codexParentThreadId(meta) || codexRawId(meta.id)) : undefined;
return { sessionId, path: file.path };
}
function indexCodexSessionIndex(db, { codexDir = DEFAULT_CODEX_DIR } = {}) { function indexCodexSessionIndex(db, { codexDir = DEFAULT_CODEX_DIR } = {}) {
const indexPath = path.join(codexDir, 'session_index.jsonl'); const indexPath = path.join(codexDir, 'session_index.jsonl');
if (!fs.existsSync(indexPath)) return; if (!fs.existsSync(indexPath)) return;
@@ -392,28 +158,6 @@ function refreshSessionProjectPaths(db) {
} }
} }
function indexSubagentMeta(db, fi) {
if (!fi.isSubagent) return false;
const mp = fi.path.replace('.jsonl', '.meta.json');
if (!fs.existsSync(mp)) return false;
let meta;
try {
meta = JSON.parse(fs.readFileSync(mp, 'utf8'));
} catch (error) {
console.warn(`Warning: failed to read subagent meta ${mp}: ${(error as Error).message}`);
return false;
}
const tok = db.prepare('SELECT COALESCE(SUM(input_tokens),0)+COALESCE(SUM(output_tokens),0) as t FROM messages WHERE agent_id=?').get(fi.agentId);
const ts = db.prepare('SELECT MIN(timestamp) as t0, MAX(timestamp) as t1 FROM messages WHERE agent_id=?').get(fi.agentId);
const dur = ts?.t0 && ts?.t1 ? new Date(ts.t1).getTime() - new Date(ts.t0).getTime() : null;
if (fi.workflowRunId) {
db.prepare('INSERT OR REPLACE INTO workflow_agents (agent_id,run_id,session_id,agent_type,description) VALUES(?,?,?,?,?)').run(fi.agentId, fi.workflowRunId, fi.sessionId, meta.agentType||null, meta.description||null);
} else {
db.prepare('INSERT OR REPLACE INTO subagents VALUES(?,?,?,?,?,?,?)').run(fi.agentId, fi.sessionId, meta.toolUseId||null, meta.agentType||null, meta.description||null, dur, tok?.t||0);
}
return true;
}
function indexWorkflows(db, { projectsDir = DEFAULT_PROJECTS_DIR } = {}) { function indexWorkflows(db, { projectsDir = DEFAULT_PROJECTS_DIR } = {}) {
if (!fs.existsSync(projectsDir)) return; if (!fs.existsSync(projectsDir)) return;
let projects; let projects;
@@ -537,6 +281,8 @@ function writeHeartbeat({
} }
interface BuildIndexOptions { interface BuildIndexOptions {
providerRoots?: Record<string, string>;
providerRegistry?: ProviderRegistry;
claudeDir?: string; claudeDir?: string;
codexDir?: string; codexDir?: string;
projectsDir?: string; projectsDir?: string;
@@ -588,6 +334,8 @@ function deferredBuildResult(
} }
function buildIndex({ function buildIndex({
providerRoots = {},
providerRegistry,
claudeDir = DEFAULT_CLAUDE_DIR, claudeDir = DEFAULT_CLAUDE_DIR,
codexDir = path.join(path.dirname(claudeDir), '.codex'), codexDir = path.join(path.dirname(claudeDir), '.codex'),
projectsDir = path.join(claudeDir, 'projects'), projectsDir = path.join(claudeDir, 'projects'),
@@ -620,30 +368,40 @@ function buildIndex({
try { try {
const db = openIndexDb({ dbPath, schemaPath, DatabaseImpl }); const db = openIndexDb({ dbPath, schemaPath, DatabaseImpl });
const txDb = betterSqliteTransactionAdapter(db); const txDb = betterSqliteTransactionAdapter(db);
const claudeInputMarkerMissing = !db.prepare(
'SELECT jsonl_path FROM index_state WHERE jsonl_path = ?',
).get(CLAUDE_INPUT_TOKEN_SEMANTICS_MARKER);
const claudeInputSemanticsOutdated = claudeInputMarkerMissing && Boolean(db.prepare(`
SELECT 1 FROM messages
WHERE COALESCE(source, 'claude') = 'claude'
AND (input_tokens IS NOT NULL OR output_tokens IS NOT NULL)
LIMIT 1
`).get());
let messageFtsTriggersDropped = false; let messageFtsTriggersDropped = false;
try { try {
if (preserveDbPath && path.resolve(preserveDbPath) !== path.resolve(dbPath)) { if (preserveDbPath && path.resolve(preserveDbPath) !== path.resolve(dbPath)) {
copyMemoriesFromDb(db, preserveDbPath); copyMemoriesFromDb(db, preserveDbPath);
} }
const files = [ const defaultHome = os.homedir();
...discoverJsonlFiles({ const compatibilityHome = path.dirname(claudeDir);
projectsDir, const relocatedDefaults = Object.fromEntries(
changedPaths: force || claudeInputSemanticsOutdated ? undefined : changedPaths, createBuiltinProviderRegistry().catalog().map((descriptor) => {
const relativeDefault = path.relative(defaultHome, descriptor.defaultRoot);
const root = compatibilityHome !== defaultHome
&& relativeDefault
&& !relativeDefault.startsWith('..')
&& !path.isAbsolute(relativeDefault)
? path.join(compatibilityHome, relativeDefault)
: descriptor.defaultRoot;
return [descriptor.id, root];
}), }),
...discoverCodexJsonlFiles({ codexDir, changedPaths: force ? undefined : changedPaths }), );
]; const roots = {
const latestSourceMtime = files.reduce((latest, file) => { ...relocatedDefaults,
claude: claudeDir,
codex: codexDir,
...providerRoots,
};
const registry = providerRegistry ?? createBuiltinProviderRegistry(roots);
const providerPlan = createProviderIndexPlan(db, registry, { force, changedPaths });
let latestSourceMtime = providerPlan.items.reduce((latest, { unit }) => {
const providerCursor = (unit.meta as { currentCursor?: unknown } | undefined)?.currentCursor;
if (typeof providerCursor === 'string') {
return Math.max(latest, Number(providerCursor.split(':')[0]) || 0);
}
try { try {
return Math.max(latest, fs.statSync(file.path).mtimeMs); return Math.max(latest, fs.statSync(unit.key).mtimeMs);
} catch { } catch {
return latest; return latest;
} }
@@ -668,7 +426,7 @@ function buildIndex({
} catch (error) { } catch (error) {
if (isBeginBusyFailure(error)) { if (isBeginBusyFailure(error)) {
return deferredBuildResult('database_busy', { return deferredBuildResult('database_busy', {
files: files.length, files: providerPlan.items.length,
latestSourceMtime, latestSourceMtime,
}); });
} }
@@ -697,40 +455,35 @@ function buildIndex({
} }
} }
const skipped: SkippedFile[] = []; const skipped: SkippedFile[] = [];
let claudeInputMigrationFailed = false; const providerResult = indexProviderPlan({
for (const file of files) { db,
try { plan: providerPlan,
// The write is committed before affectedSessionIds is updated, so a runTransaction: (label, work) => runRetryableWriteTransaction(txDb, work, { label }),
// failed/rolled-back file never reports a phantom updated session. onCommitted: ({ unit }, nextCursor) => {
const indexed = runRetryableWriteTransaction(txDb, () => { if (nextCursor) latestSourceMtime = Math.max(latestSourceMtime, Number(nextCursor.split(':')[0]) || 0);
const result = file.source === 'codex' if (unit.sessionId) affectedSessionIds.add(unit.sessionId);
? indexCodexFile(db, file) },
: indexClaudeFile(db, file, { forceFull: claudeInputSemanticsOutdated }); onError: (error, { provider, unit }) => {
const metaIndexed = file.source !== 'codex' && indexSubagentMeta(db, file); if (isBeginBusyFailure(error)) return 'stop';
if (!result?.sessionId && metaIndexed && changedMetaJsonlPaths.has(file.path)) { if (hasUnusableTransaction(error)) throw error;
return { sessionId: file.sessionId, path: file.path }; skipped.push({
} path: unit.key,
return result; error: (error as Error).message,
}, { label: `file:${file.path}` }); diagnostics: (error as { obelisk?: unknown }).obelisk,
if (indexed?.sessionId) affectedSessionIds.add(indexed.sessionId); });
} catch (error) { console.warn(`Warning: failed to index ${provider.name} unit ${unit.key}: ${(error as Error).message}`);
if (isBeginBusyFailure(error)) { return 'skip';
},
});
if (providerResult.stopped) {
return deferredBuildResult('database_busy', { return deferredBuildResult('database_busy', {
files: files.length, files: providerPlan.items.length,
latestSourceMtime, latestSourceMtime,
affectedSessionIds: [...affectedSessionIds], affectedSessionIds: [...affectedSessionIds],
skipped: skipped.length, skipped: skipped.length,
skippedFiles: skipped, skippedFiles: skipped,
}); });
} }
if (hasUnusableTransaction(error)) throw error;
if (claudeInputSemanticsOutdated && file.source !== 'codex') {
claudeInputMigrationFailed = true;
}
skipped.push({ path: file.path, error: (error as Error).message, diagnostics: (error as { obelisk?: unknown }).obelisk });
console.warn(`Warning: failed to index ${file.path}: ${(error as Error).message}`);
}
}
let ftsRebuilt = false; let ftsRebuilt = false;
// Finalize is one transaction; a failure here fails the whole build (the // Finalize is one transaction; a failure here fails the whole build (the
// index would otherwise be left inconsistent). // index would otherwise be left inconsistent).
@@ -745,15 +498,13 @@ function buildIndex({
writeIndexMarker(db, '__last_build__'); writeIndexMarker(db, '__last_build__');
writeIndexMarker(db, '__app_last_successful_build__'); writeIndexMarker(db, '__app_last_successful_build__');
writeIndexMarker(db, '__indexer_owner_app__'); writeIndexMarker(db, '__indexer_owner_app__');
if (!claudeInputSemanticsOutdated || !claudeInputMigrationFailed) { writeProviderIndexMarkers(db, providerPlan, providerResult);
writeIndexMarker(db, CLAUDE_INPUT_TOKEN_SEMANTICS_MARKER);
}
if (latestSourceMtime) writeIndexMarker(db, '__last_source_mtime__', latestSourceMtime); if (latestSourceMtime) writeIndexMarker(db, '__last_source_mtime__', latestSourceMtime);
}, { label: 'finalize' }); }, { label: 'finalize' });
} catch (error) { } catch (error) {
if (isBeginBusyFailure(error)) { if (isBeginBusyFailure(error)) {
return deferredBuildResult('database_busy', { return deferredBuildResult('database_busy', {
files: files.length, files: providerPlan.items.length,
latestSourceMtime, latestSourceMtime,
affectedSessionIds: [...affectedSessionIds], affectedSessionIds: [...affectedSessionIds],
skipped: skipped.length, skipped: skipped.length,
@@ -764,7 +515,7 @@ function buildIndex({
} }
for (const sessionId of finalizeAffectedSessionIds) affectedSessionIds.add(sessionId); for (const sessionId of finalizeAffectedSessionIds) affectedSessionIds.add(sessionId);
return { return {
files: files.length, files: providerPlan.items.length,
latestSourceMtime, latestSourceMtime,
affectedSessionIds: [...affectedSessionIds], affectedSessionIds: [...affectedSessionIds],
ftsRebuilt, ftsRebuilt,
@@ -792,6 +543,5 @@ export {
buildIndex, buildIndex,
writeHeartbeat, writeHeartbeat,
openIndexDb, openIndexDb,
discoverJsonlFiles,
inferProjectPath, inferProjectPath,
}; };
+88
View File
@@ -0,0 +1,88 @@
import type { ProviderRegistry } from '../../../packages/core/src/providers/registry.ts';
type PersistedSettings = Record<string, unknown> & {
providerRoots?: Record<string, unknown>;
};
interface SourceStats {
sessionCount: number;
lastIndexed: string;
}
interface BuildSourceCatalogOptions {
registry: ProviderRegistry;
roots: Readonly<Record<string, string>>;
stats?: ReadonlyMap<string, SourceStats>;
pathExists?: (path: string) => boolean;
}
function configuredPath(value: unknown): string | null {
return typeof value === 'string' && value.trim().length > 0 ? value : null;
}
export function resolveProviderRoots(
registry: ProviderRegistry,
persisted: PersistedSettings = {},
): Record<string, string> {
const configured = persisted.providerRoots ?? {};
return Object.fromEntries(registry.catalog().map((descriptor) => {
const root = configuredPath(configured[descriptor.id])
?? configuredPath(persisted[`${descriptor.id}Dir`])
?? descriptor.defaultRoot;
return [descriptor.id, root];
}));
}
export function setPersistedSetting(
persisted: PersistedSettings,
key: string,
value: unknown,
): boolean {
const providerMatch = /^providerRoots\.(.+)$/.exec(key);
if (providerMatch === null) {
if (value === null) delete persisted[key];
else persisted[key] = value;
return false;
}
const providerId = providerMatch[1]!;
const roots = persisted.providerRoots && typeof persisted.providerRoots === 'object'
? persisted.providerRoots
: {};
if (value === null) delete roots[providerId];
else roots[providerId] = value;
if (Object.keys(roots).length === 0) delete persisted.providerRoots;
else persisted.providerRoots = roots;
return true;
}
export function buildSourceCatalog({
registry,
roots,
stats = new Map(),
pathExists = () => false,
}: BuildSourceCatalogOptions) {
return registry.catalog().map((descriptor) => {
const path = roots[descriptor.id] ?? descriptor.defaultRoot;
const exists = pathExists(path);
const sourceStats = stats.get(descriptor.id) ?? { sessionCount: 0, lastIndexed: '' };
const status = !exists ? 'error' : sourceStats.sessionCount > 0 ? 'ok' : 'warn';
return {
id: descriptor.id,
name: descriptor.name,
vendor: descriptor.vendor,
color: descriptor.color,
path,
settingKey: `providerRoots.${descriptor.id}`,
exists,
sessionCount: sourceStats.sessionCount,
lastIndexed: sourceStats.lastIndexed,
status,
statusText: !exists
? 'Folder not found'
: sourceStats.sessionCount > 0
? 'Connected'
: 'No sessions found',
};
});
}
+7 -5
View File
@@ -17,6 +17,7 @@ import {
import { formatProjectLabel } from './utils.js'; import { formatProjectLabel } from './utils.js';
import { buildSidebarProjects } from './sidebar-projects.mjs'; import { buildSidebarProjects } from './sidebar-projects.mjs';
import { resolveGlobalShortcut } from './keyboard-shortcuts.mjs'; import { resolveGlobalShortcut } from './keyboard-shortcuts.mjs';
import { sourceLabel } from './source-catalog.mjs';
const router = useRouter(); const router = useRouter();
const route = useRoute(); const route = useRoute();
@@ -207,8 +208,9 @@ const showSourcePopover = ref(false);
async function loadSourceDots() { async function loadSourceDots() {
if (!window.obelisk?.getSettings) return; if (!window.obelisk?.getSettings) return;
const s = await window.obelisk.getSettings(); const s = await window.obelisk.getSettings();
sourceDots.value = (s.sources || []).map(src => ({ id: src.id, status: src.status })); sourceDots.value = (s.sources || []).map(src => ({ id: src.id, status: src.status, color: src.color }));
sourceDetails.value = s.sources || []; sourceDetails.value = s.sources || [];
state.sources = s.sources || [];
} }
loadSourceDots(); loadSourceDots();
@@ -223,7 +225,7 @@ const showSourceFilter = ref(false);
const sourceFilterActive = computed(() => state.sourceFilter !== 'all' && state.sourceFilter !== undefined); const sourceFilterActive = computed(() => state.sourceFilter !== 'all' && state.sourceFilter !== undefined);
const sourceFilterLabel = computed(() => { const sourceFilterLabel = computed(() => {
if (!state.sourceFilter || state.sourceFilter === 'all') return 'All sources'; if (!state.sourceFilter || state.sourceFilter === 'all') return 'All sources';
return state.sourceFilter === 'claude' ? 'Claude Code' : 'Codex'; return sourceLabel(state.sourceFilter, state.sources);
}); });
function toggleSourceFilter() { showSourceFilter.value = !showSourceFilter.value; } function toggleSourceFilter() { showSourceFilter.value = !showSourceFilter.value; }
function setSourceFilter(id) { function setSourceFilter(id) {
@@ -271,13 +273,13 @@ provide('recapGenerateOpen', recapGenerateOpen);
</svg> </svg>
<span class="name">Obelisk</span> <span class="name">Obelisk</span>
<button class="source-health" title="Connected sources" @click="showSourcePopover = !showSourcePopover"> <button class="source-health" title="Connected sources" @click="showSourcePopover = !showSourcePopover">
<span v-for="src in sourceDots" :key="src.id" class="h-dot" :class="src.id + '-' + src.status"></span> <span v-for="src in sourceDots" :key="src.id" class="h-dot" :class="src.status" :style="{ '--source-color': src.color }"></span>
</button> </button>
<div class="sources-popover" :class="{ show: showSourcePopover }"> <div class="sources-popover" :class="{ show: showSourcePopover }">
<div class="sp-head">Connected sources</div> <div class="sp-head">Connected sources</div>
<div class="sp-list"> <div class="sp-list">
<button v-for="src in sourceDetails" :key="src.id" class="sp-row" @click="router.push('/settings')"> <button v-for="src in sourceDetails" :key="src.id" class="sp-row" @click="router.push('/settings')">
<span class="sp-dot" :class="src.id"></span> <span class="sp-dot" :style="{ '--source-color': src.color }"></span>
<div class="sp-body"> <div class="sp-body">
<div class="sp-name">{{ src.name }} <span class="sp-count" v-if="src.sessionCount">{{ src.sessionCount }} sessions</span></div> <div class="sp-name">{{ src.name }} <span class="sp-count" v-if="src.sessionCount">{{ src.sessionCount }} sessions</span></div>
<div class="sp-meta" :class="src.status">{{ src.statusText }}</div> <div class="sp-meta" :class="src.status">{{ src.statusText }}</div>
@@ -535,7 +537,7 @@ provide('recapGenerateOpen', recapGenerateOpen);
<div class="fd-check"> <div class="fd-check">
<svg viewBox="0 0 12 12" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.5 6l2.5 2.5 4.5-5"/></svg> <svg viewBox="0 0 12 12" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M2.5 6l2.5 2.5 4.5-5"/></svg>
</div> </div>
<span class="fd-name">{{ src.id === 'claude' ? 'Claude Code' : 'Codex' }}</span> <span class="fd-name">{{ sourceLabel(src.id, sourceDetails) }}</span>
</div> </div>
<div class="fd-divider"></div> <div class="fd-divider"></div>
<div class="fd-row" :class="{ checked: state.sourceFilter === 'all' }" @click.stop="setSourceFilter('all')"> <div class="fd-row" :class="{ checked: state.sourceFilter === 'all' }" @click.stop="setSourceFilter('all')">
+6 -6
View File
@@ -1,13 +1,12 @@
import { sourceLabel } from './source-catalog.mjs';
export function activitySourceKey(session) { export function activitySourceKey(session) {
const source = typeof session?.source === 'string' ? session.source.trim().toLowerCase() : ''; const source = typeof session?.source === 'string' ? session.source.trim().toLowerCase() : '';
return source || 'claude'; return source || 'claude';
} }
export function activitySourceLabel(session) { export function activitySourceLabel(session, sourceCatalog = []) {
const source = activitySourceKey(session); return sourceLabel(activitySourceKey(session), sourceCatalog);
if (source === 'codex') return 'Codex';
if (source === 'claude') return 'Claude Code';
return source.charAt(0).toUpperCase() + source.slice(1);
} }
export function activityGroupSessions(split) { export function activityGroupSessions(split) {
@@ -22,9 +21,10 @@ export function activitySessionMetaParts(session, {
mixedSources = false, mixedSources = false,
projectLabel = '', projectLabel = '',
includeProject = true, includeProject = true,
sourceCatalog = [],
} = {}) { } = {}) {
const parts = []; const parts = [];
if (mixedSources) parts.push({ kind: 'source', text: activitySourceLabel(session) }); if (mixedSources) parts.push({ kind: 'source', text: activitySourceLabel(session, sourceCatalog) });
if (includeProject && projectLabel) parts.push({ kind: 'project', text: projectLabel }); if (includeProject && projectLabel) parts.push({ kind: 'project', text: projectLabel });
parts.push({ parts.push({
kind: 'count', kind: 'count',
@@ -2,6 +2,7 @@
import { computed } from 'vue'; import { computed } from 'vue';
import { formatProjectLabel } from '../utils.js'; import { formatProjectLabel } from '../utils.js';
import { activitySessionMetaParts } from '../activity-ledger.mjs'; import { activitySessionMetaParts } from '../activity-ledger.mjs';
import { state } from '../store.js';
const props = defineProps({ const props = defineProps({
session: { type: Object, required: true }, session: { type: Object, required: true },
@@ -17,6 +18,7 @@ const metaParts = computed(() => activitySessionMetaParts(props.session, {
mixedSources: props.mixedSources, mixedSources: props.mixedSources,
projectLabel: formatProjectLabel(props.session.project) || '', projectLabel: formatProjectLabel(props.session.project) || '',
includeProject: props.includeProject, includeProject: props.includeProject,
sourceCatalog: state.sources,
})); }));
</script> </script>
+28
View File
@@ -0,0 +1,28 @@
const FALLBACK_COLOR = '#8b8b93';
function sourceId(value) {
const normalized = typeof value === 'string' ? value.trim().toLowerCase() : '';
return normalized || 'claude';
}
function descriptorFor(source, catalog = []) {
const id = sourceId(source);
return catalog.find(candidate => candidate?.id === id) || null;
}
function titleCaseId(id) {
return id
.split(/[-_]+/)
.filter(Boolean)
.map(part => part.charAt(0).toUpperCase() + part.slice(1))
.join(' ');
}
export function sourceLabel(source, catalog = []) {
const id = sourceId(source);
return descriptorFor(id, catalog)?.name || titleCaseId(id);
}
export function sourceColor(source, catalog = []) {
return descriptorFor(source, catalog)?.color || FALLBACK_COLOR;
}
+1
View File
@@ -8,6 +8,7 @@ export const state = reactive({
sessions: [], sessions: [],
sessionTitleOverrides: shallowReactive(new Map()), sessionTitleOverrides: shallowReactive(new Map()),
projects: [], projects: [],
sources: [],
stats: {}, stats: {},
view: 'active', // 'active' | 'archived' view: 'active', // 'active' | 'archived'
query: '', query: '',
+3 -2
View File
@@ -23,6 +23,7 @@ import {
fmtRelative, fmtRelative,
formatProjectLabel formatProjectLabel
} from '../utils.js'; } from '../utils.js';
import { sourceColor, sourceLabel } from '../source-catalog.mjs';
defineOptions({ name: 'SessionDetail' }); defineOptions({ name: 'SessionDetail' });
const props = defineProps({ id: String }); const props = defineProps({ id: String });
@@ -490,8 +491,8 @@ function navigateToSubagent(agentId) {
<span class="sep">&middot;</span> <span class="sep">&middot;</span>
<span class="project-path">{{ session.project_path || '' }}</span> <span class="project-path">{{ session.project_path || '' }}</span>
<span class="via"> <span class="via">
<span class="via-dot" :class="session.source || 'claude'"></span> <span class="via-dot" :style="{ '--source-color': sourceColor(session.source, state.sources) }"></span>
via {{ (session.source || 'claude') === 'codex' ? 'Codex' : 'Claude Code' }} via {{ sourceLabel(session.source, state.sources) }}
</span> </span>
</div> </div>
<div class="session-title">{{ session.title || '(untitled)' }}</div> <div class="session-title">{{ session.title || '(untitled)' }}</div>
+3 -6
View File
@@ -29,8 +29,7 @@ async function browseSourcePath(source) {
if (!window.obelisk?.browseFolder) return; if (!window.obelisk?.browseFolder) return;
const result = await window.obelisk.browseFolder(); const result = await window.obelisk.browseFolder();
if (result) { if (result) {
const key = source.id === 'claude' ? 'claudeDir' : 'codexDir'; await saveSetting(source.settingKey || `providerRoots.${source.id}`, result);
await saveSetting(key, result);
await loadSettings(); await loadSettings();
} }
} }
@@ -107,8 +106,8 @@ function fmtRelative(iso) {
:class="{ error: src.status === 'error', warn: src.status === 'warn' }" :class="{ error: src.status === 'error', warn: src.status === 'warn' }"
> >
<div class="source-card-head"> <div class="source-card-head">
<div class="source-card-mark" :class="src.id"> <div class="source-card-mark">
<span class="mark-dot"></span> <span class="mark-dot" :style="{ background: src.color, boxShadow: `0 0 6px ${src.color}80` }"></span>
</div> </div>
<div class="source-card-info"> <div class="source-card-info">
<div class="source-card-name"> <div class="source-card-name">
@@ -261,8 +260,6 @@ function fmtRelative(iso) {
display: grid; place-items: center; flex-shrink: 0; display: grid; place-items: center; flex-shrink: 0;
} }
.source-card-mark .mark-dot { width: 8px; height: 8px; border-radius: 50%; } .source-card-mark .mark-dot { width: 8px; height: 8px; border-radius: 50%; }
.source-card-mark.claude .mark-dot { background: #d97757; box-shadow: 0 0 6px rgba(217,119,87,0.5); }
.source-card-mark.codex .mark-dot { background: #10a37f; box-shadow: 0 0 6px rgba(16,163,127,0.5); }
.source-card-info { flex: 1; min-width: 0; } .source-card-info { flex: 1; min-width: 0; }
.source-card-name { .source-card-name {
font-size: 14px; color: var(--fg); font-weight: 600; letter-spacing: -0.005em; font-size: 14px; color: var(--fg); font-weight: 600; letter-spacing: -0.005em;
+1 -2
View File
@@ -434,8 +434,7 @@
margin-left: 6px; margin-left: 6px;
} }
.session-eyebrow .via .via-dot { width: 4px; height: 4px; border-radius: 50%; } .session-eyebrow .via .via-dot { width: 4px; height: 4px; border-radius: 50%; }
.session-eyebrow .via .via-dot.claude { background: #d97757; box-shadow: 0 0 4px rgba(217,119,87,0.5); } .session-eyebrow .via .via-dot { background: var(--source-color); box-shadow: 0 0 4px var(--source-color); }
.session-eyebrow .via .via-dot.codex { background: #10a37f; box-shadow: 0 0 4px rgba(16,163,127,0.5); }
.session-title { .session-title {
font-size: 22px; font-weight: 600; color: var(--fg); font-size: 22px; font-weight: 600; color: var(--fg);
line-height: 1.3; margin-bottom: 14px; letter-spacing: -0.01em; line-height: 1.3; margin-bottom: 14px; letter-spacing: -0.01em;
+4 -8
View File
@@ -29,12 +29,9 @@
.source-health .h-dot { .source-health .h-dot {
width: 5px; height: 5px; border-radius: 50%; flex-shrink: 0; width: 5px; height: 5px; border-radius: 50%; flex-shrink: 0;
} }
.source-health .h-dot.claude-ok { background: #d97757; box-shadow: 0 0 4px rgba(217,119,87,0.6); } .source-health .h-dot.ok { background: var(--source-color); box-shadow: 0 0 4px var(--source-color); }
.source-health .h-dot.claude-warn { background: rgba(217,119,87,0.4); } .source-health .h-dot.warn { background: color-mix(in srgb, var(--source-color) 40%, transparent); }
.source-health .h-dot.claude-error { background: rgba(217,119,87,0.25); } .source-health .h-dot.error { background: color-mix(in srgb, var(--source-color) 25%, transparent); }
.source-health .h-dot.codex-ok { background: #10a37f; box-shadow: 0 0 4px rgba(16,163,127,0.6); }
.source-health .h-dot.codex-warn { background: rgba(16,163,127,0.4); }
.source-health .h-dot.codex-error { background: rgba(16,163,127,0.25); }
.source-health .h-dot.off { background: var(--muted-3); } .source-health .h-dot.off { background: var(--muted-3); }
/* Sources popover */ /* Sources popover */
@@ -60,8 +57,7 @@
} }
.sp-row:hover { background: rgba(255,255,255,0.03); } .sp-row:hover { background: rgba(255,255,255,0.03); }
.sp-dot { width: 7px; height: 7px; border-radius: 50%; flex-shrink: 0; } .sp-dot { width: 7px; height: 7px; border-radius: 50%; flex-shrink: 0; }
.sp-dot.claude { background: #d97757; box-shadow: 0 0 6px rgba(217,119,87,0.5); } .sp-dot { background: var(--source-color); box-shadow: 0 0 6px var(--source-color); }
.sp-dot.codex { background: #10a37f; box-shadow: 0 0 6px rgba(16,163,127,0.5); }
.sp-dot.off { background: var(--muted-3); } .sp-dot.off { background: var(--muted-3); }
.sp-body { flex: 1; min-width: 0; } .sp-body { flex: 1; min-width: 0; }
.sp-name { font-size: 12.5px; color: var(--fg); font-weight: 500; display: flex; align-items: baseline; gap: 6px; } .sp-name { font-size: 12.5px; color: var(--fg); font-weight: 500; display: flex; align-items: baseline; gap: 6px; }
+21 -5
View File
@@ -20,13 +20,20 @@ binding-agnostic and does not need a per-binding implementation.
**Decision.** Split indexing along two orthogonal axes. **Decision.** Split indexing along two orthogonal axes.
- **Provider axis — a registry of pure adapters.** Each source (claude, codex, - **Provider axis — a registry of pure adapters.** Each source (Claude Code,
later opencode, pi, …) is a provider adapter implementing Codex, Kimi Code, later Pi, …) is a provider adapter implementing one complete
`discover(opts) → files` and `parse(file, fromLine) → Iterable<Record>`. An boundary: serializable descriptor metadata, `watchRoots(root)`,
`discover(context) → IndexUnit[]`, `parse(unit, cursor) → Iterable<Record>`,
and `raw(lookup)`. An `IndexUnit` is deliberately not a file abstraction: Kimi
uses one session directory containing state plus multiple agent wire logs. An
adapter is *pure*: it emits normalized records and never touches a database. adapter is *pure*: it emits normalized records and never touches a database.
Adding a source means adding one adapter and registering it; nothing else Adding a source means adding one adapter and registering it; nothing else
changes. `parse` is a streaming iterator, preserving memory-friendly indexing changes. `parse` exposes an iterator as its common interface and streams when
and the `lines_processed` resume-from-line semantics in `index_state`. the provider semantics permit it. An adapter may buffer one complete
`IndexUnit` when correctness requires whole-unit semantics — for example,
Codex duplicate reconciliation or Kimi `context.undo` / `context.clear`
replay. Each adapter maps its own resume/change semantics onto the existing
`mtime` and `lines_processed` cursor pair in `index_state`.
- **Persist axis — one shared orchestration.** A single provider-agnostic, - **Persist axis — one shared orchestration.** A single provider-agnostic,
binding-agnostic layer consumes records from any adapter and writes them: binding-agnostic layer consumes records from any adapter and writes them:
incremental `index_state` bookkeeping, FTS maintenance, and the canonical incremental `index_state` bookkeeping, FTS maintenance, and the canonical
@@ -48,3 +55,12 @@ strategy injected into the shared orchestration, not a fork of it. The Electron
main process migrates to ESM (ADR-0003) to import the shared core. The real work main process migrates to ESM (ADR-0003) to import the shared core. The real work
is disentangling the currently interleaved parse-and-write inside `indexJsonl` / is disentangling the currently interleaved parse-and-write inside `indexJsonl` /
`indexCodexJsonl` into (pure adapter parse) + (shared persist). `indexCodexJsonl` into (pure adapter parse) + (shared persist).
The SQLite schema and normalized `IndexRecord` union are the stable center of
the design. Provider-only concepts are either projected lossily into that
language or ignored; they do not add provider columns or tables. The registry,
not provider switches, drives both indexers, watcher roots, persisted source
roots, source catalog/UI labels and colors, and raw-record routing. Adding Pi
therefore changes the Pi adapter, its registration, and its conformance tests;
the shared schema, persist layer, indexers, settings, query API, and renderer do
not acquire Pi-specific branches.
+4
View File
@@ -10,8 +10,12 @@
"./indexer": "./dist/indexer.js", "./indexer": "./dist/indexer.js",
"./parsing": "./dist/parsing.js", "./parsing": "./dist/parsing.js",
"./persist": "./dist/persist.js", "./persist": "./dist/persist.js",
"./provider-indexing": "./dist/provider-indexing.js",
"./providers/claude": "./dist/providers/claude.js", "./providers/claude": "./dist/providers/claude.js",
"./providers/codex": "./dist/providers/codex.js", "./providers/codex": "./dist/providers/codex.js",
"./providers/kimi": "./dist/providers/kimi.js",
"./providers/registry": "./dist/providers/registry.js",
"./providers/builtins": "./dist/providers/builtins.js",
"./providers/types": "./dist/providers/types.js", "./providers/types": "./dist/providers/types.js",
"./query": "./dist/query.js", "./query": "./dist/query.js",
"./sqlite-types": "./dist/sqlite-types.js", "./sqlite-types": "./dist/sqlite-types.js",
+25 -108
View File
@@ -2,19 +2,17 @@
import { DB_PATH, openDb, openReadDb, openWriterLeaseDb, rebuildMemoryFts } from './db.ts'; import { DB_PATH, openDb, openReadDb, openWriterLeaseDb, rebuildMemoryFts } from './db.ts';
import { import {
CLAUDE_DIR, CODEX_DIR, PROJECTS_DIR, fs, path, isDir, readLines, CLAUDE_DIR, CODEX_DIR, PROJECTS_DIR, fs, path, isDir, readLines,
inferProjectPath, discoverJsonlFiles, discoverCodexJsonlFiles, codexDbId, readCodexGuardianThreadInfo, inferProjectPath, codexDbId,
} from './parsing.ts'; } from './parsing.ts';
import { persist } from './persist.ts'; import {
createProviderIndexPlan,
indexProviderPlan,
writeProviderIndexMarkers,
} from './provider-indexing.ts';
import { nodeSqliteTransactionAdapter } from './tx.ts'; import { nodeSqliteTransactionAdapter } from './tx.ts';
import { acquireWriterLease, writerLockPathFor } from './writer-lease.ts'; import { acquireWriterLease, writerLockPathFor } from './writer-lease.ts';
import { runRetryableWriteTransaction, isBeginBusyFailure, hasUnusableTransaction } from './write-coordinator.ts'; import { runRetryableWriteTransaction, isBeginBusyFailure, hasUnusableTransaction } from './write-coordinator.ts';
import { import { createBuiltinProviderRegistry } from './providers/builtins.ts';
CLAUDE_INPUT_TOKEN_SEMANTICS_MARKER,
parse as claudeParse,
} from './providers/claude.ts';
import { parse as codexParse } from './providers/codex.ts';
import type { Cursor, IndexRecord } from './providers/types.ts';
import type { ClaudeJsonlFile } from './parsing.ts';
import type { NodeSqliteDb, SqliteRow } from './sqlite-types.ts'; import type { NodeSqliteDb, SqliteRow } from './sqlite-types.ts';
const HISTORY_PATH = path.join(CLAUDE_DIR, 'history.jsonl'); const HISTORY_PATH = path.join(CLAUDE_DIR, 'history.jsonl');
@@ -37,14 +35,6 @@ function errorMessage(error: unknown): string {
} }
function needsReindex(db: NodeSqliteDb, fp: string) {
const mt = fs.statSync(fp).mtimeMs;
const row = db.prepare('SELECT mtime, lines_processed FROM index_state WHERE jsonl_path = ?').get(fp);
if (!row) return { needed: true, skip: 0 };
return mt > row.mtime ? { needed: true, skip: row.lines_processed } : { needed: false, skip: 0 };
}
function indexCodexSessionIndex(db: NodeSqliteDb): void { function indexCodexSessionIndex(db: NodeSqliteDb): void {
const indexPath = path.join(CODEX_DIR, 'session_index.jsonl'); const indexPath = path.join(CODEX_DIR, 'session_index.jsonl');
if (!fs.existsSync(indexPath)) return; if (!fs.existsSync(indexPath)) return;
@@ -78,27 +68,6 @@ function refreshSessionProjectPaths(db: NodeSqliteDb): void {
} }
} }
function indexSubagentMeta(db: NodeSqliteDb, fi: ClaudeJsonlFile): void {
if (!fi.isSubagent) return;
const mp = fi.path.replace('.jsonl', '.meta.json');
if (!fs.existsSync(mp)) return;
let meta: JsonRecord;
try {
meta = JSON.parse(fs.readFileSync(mp, 'utf8'));
} catch (e) {
process.stderr.write(`Warning: failed to read subagent meta ${mp}: ${errorMessage(e)}\n`);
return;
}
const tok = db.prepare('SELECT COALESCE(SUM(input_tokens),0)+COALESCE(SUM(output_tokens),0) as t FROM messages WHERE agent_id=?').get(fi.agentId);
const ts = db.prepare('SELECT MIN(timestamp) as t0, MAX(timestamp) as t1 FROM messages WHERE agent_id=?').get(fi.agentId);
const dur = ts?.t0 && ts?.t1 ? new Date(ts.t1).getTime() - new Date(ts.t0).getTime() : null;
if (fi.workflowRunId) {
db.prepare('INSERT OR REPLACE INTO workflow_agents (agent_id,run_id,session_id,agent_type,description) VALUES(?,?,?,?,?)').run(fi.agentId, fi.workflowRunId, fi.sessionId, meta.agentType||null, meta.description||null);
} else {
db.prepare('INSERT OR REPLACE INTO subagents VALUES(?,?,?,?,?,?,?)').run(fi.agentId, fi.sessionId, meta.toolUseId||null, meta.agentType||null, meta.description||null, dur, tok?.t||0);
}
}
function indexWorkflows(db: NodeSqliteDb): void { function indexWorkflows(db: NodeSqliteDb): void {
if (!fs.existsSync(PROJECTS_DIR)) return; if (!fs.existsSync(PROJECTS_DIR)) return;
let projects; let projects;
@@ -191,13 +160,6 @@ function inspectBuildOwnership({ force = false }: { force?: boolean } = {}) {
} }
} }
// A one-shot record stream that retracts a session, for routing guardian sweeps
// through persist (the single db writer) instead of deleting rows directly.
function* guardianDelete(sessionId: string): Generator<IndexRecord, Cursor> {
yield { kind: 'delete-session', sessionId };
return null;
}
function buildIndex({ force = false }: { force?: boolean } = {}) { function buildIndex({ force = false }: { force?: boolean } = {}) {
const ownership = inspectBuildOwnership({ force }); const ownership = inspectBuildOwnership({ force });
if (ownership.skip) return ownership; if (ownership.skip) return ownership;
@@ -213,17 +175,7 @@ function buildIndex({ force = false }: { force?: boolean } = {}) {
const db = openDb(); const db = openDb();
const txDb = nodeSqliteTransactionAdapter(db); const txDb = nodeSqliteTransactionAdapter(db);
const claudeInputMarkerMissing = !db.prepare(
'SELECT jsonl_path FROM index_state WHERE jsonl_path = ?',
).get(CLAUDE_INPUT_TOKEN_SEMANTICS_MARKER);
const claudeInputSemanticsOutdated = claudeInputMarkerMissing && Boolean(db.prepare(`
SELECT 1 FROM messages
WHERE COALESCE(source, 'claude') = 'claude'
AND (input_tokens IS NOT NULL OR output_tokens IS NOT NULL)
LIMIT 1
`).get());
const skippedFiles: SkippedFile[] = []; const skippedFiles: SkippedFile[] = [];
let claudeInputMigrationFailed = false;
try { try {
try { try {
if (force) { if (force) {
@@ -246,57 +198,25 @@ function buildIndex({ force = false }: { force?: boolean } = {}) {
throw error; throw error;
} }
const files = [ const registry = createBuiltinProviderRegistry();
...discoverJsonlFiles(), const providerPlan = createProviderIndexPlan(db, registry, { force });
...discoverCodexJsonlFiles(), const providerResult = indexProviderPlan({
]; db,
for (const f of files) { plan: providerPlan,
try { runTransaction: (label, work) => runRetryableWriteTransaction(txDb, work, { label }),
runRetryableWriteTransaction(txDb, () => { onError: (error, { provider, unit }) => {
if (f.source === 'codex') { if (isBeginBusyFailure(error)) return 'stop';
// Codex goes through the pure adapter + shared persist (docs/adr/0001), if (hasUnusableTransaction(error)) throw error;
// full-reparse (countMode 'total') when the file changed. An unchanged const detail = error as { message?: unknown; obelisk?: unknown } | null;
// file is not reparsed, but is still swept for stale guardian rows: a const message = errorMessage(error);
// guardian/auto-review thread must never linger in the index, even if it skippedFiles.push({ path: unit.key, error: message, diagnostics: detail?.obelisk });
// was indexed before guardian detection removed it. process.stderr.write(`Warning: failed to index ${provider.name} unit ${unit.key}: ${message}\n`);
const { needed } = needsReindex(db, f.path); return 'skip';
if (needed) { },
persist(db, { key: f.path, sessionId: '' }, codexParse({ key: f.path, sessionId: '' }, null)); });
} else { if (providerResult.stopped) {
const guardian = readCodexGuardianThreadInfo(f.path);
if (guardian) {
const sessionId = codexDbId(guardian.threadRawId);
if (sessionId) persist(db, { key: f.path, sessionId: '' }, guardianDelete(sessionId));
}
}
} else {
// Claude transcripts now go through the pure adapter + shared persist
// (docs/adr/0001). needsReindex keeps the "skip unchanged file" fast path;
// the cursor's line count drives incremental resume inside parse().
const { needed, skip } = needsReindex(db, f.path);
if (needed || claudeInputSemanticsOutdated) {
const unit = { key: f.path, sessionId: f.sessionId, project: f.project, isSubagent: f.isSubagent, agentId: f.agentId };
const cursor = !claudeInputSemanticsOutdated && skip > 0 ? `0:${skip}` : null;
persist(db, unit, claudeParse(unit, cursor));
}
indexSubagentMeta(db, f);
}
}, { label: `file:${f.path}` });
} catch (e) {
if (isBeginBusyFailure(e)) {
return { skip: true, reason: 'database_busy', skipped: skippedFiles.length, skippedFiles }; return { skip: true, reason: 'database_busy', skipped: skippedFiles.length, skippedFiles };
} }
if (hasUnusableTransaction(e)) throw e;
if (claudeInputSemanticsOutdated && f.source !== 'codex') {
claudeInputMigrationFailed = true;
}
// A per-file failure is skippable: log and move on.
const error = e as { message?: unknown; obelisk?: unknown } | null;
const message = errorMessage(e);
skippedFiles.push({ path: f.path, error: message, diagnostics: error?.obelisk });
process.stderr.write(`Warning: failed to index ${f.path}: ${message}\n`);
}
}
// Finalize is one transaction and is NOT swallowed: a finalize failure fails // Finalize is one transaction and is NOT swallowed: a finalize failure fails
// the build (a half-finalized index would be inconsistent). // the build (a half-finalized index would be inconsistent).
try { try {
@@ -308,10 +228,7 @@ function buildIndex({ force = false }: { force?: boolean } = {}) {
db.exec("INSERT INTO messages_fts(messages_fts) VALUES('rebuild')"); db.exec("INSERT INTO messages_fts(messages_fts) VALUES('rebuild')");
rebuildMemoryFts(db); rebuildMemoryFts(db);
db.prepare("INSERT OR REPLACE INTO index_state (jsonl_path, mtime, lines_processed) VALUES ('__last_build__', ?, 0)").run(Date.now()); db.prepare("INSERT OR REPLACE INTO index_state (jsonl_path, mtime, lines_processed) VALUES ('__last_build__', ?, 0)").run(Date.now());
if (!claudeInputSemanticsOutdated || !claudeInputMigrationFailed) { writeProviderIndexMarkers(db, providerPlan, providerResult);
db.prepare('INSERT OR REPLACE INTO index_state (jsonl_path, mtime, lines_processed) VALUES (?, ?, 0)')
.run(CLAUDE_INPUT_TOKEN_SEMANTICS_MARKER, Date.now());
}
}, { label: 'finalize' }); }, { label: 'finalize' });
} catch (error) { } catch (error) {
if (isBeginBusyFailure(error)) { if (isBeginBusyFailure(error)) {
+7 -7
View File
@@ -150,13 +150,13 @@ function inferProjectPath(project: string | null | undefined, observedCwds: unkn
return best?.path || legacyProjectPathFromSlug(project); return best?.path || legacyProjectPathFromSlug(project);
} }
function discoverJsonlFiles(): ClaudeJsonlFile[] { function discoverJsonlFiles(projectsDir = PROJECTS_DIR): ClaudeJsonlFile[] {
const files: ClaudeJsonlFile[] = []; const files: ClaudeJsonlFile[] = [];
if (!fs.existsSync(PROJECTS_DIR)) return files; if (!fs.existsSync(projectsDir)) return files;
let projects; let projects;
try { projects = fs.readdirSync(PROJECTS_DIR); } catch (e) { process.stderr.write(`Warning: cannot read projects dir: ${e instanceof Error ? e.message : String(e)}\n`); return files; } try { projects = fs.readdirSync(projectsDir); } catch (e) { process.stderr.write(`Warning: cannot read projects dir: ${e instanceof Error ? e.message : String(e)}\n`); return files; }
for (const proj of projects) { for (const proj of projects) {
const projPath = path.join(PROJECTS_DIR, proj); const projPath = path.join(projectsDir, proj);
if (!isDir(projPath)) continue; if (!isDir(projPath)) continue;
let entries; let entries;
try { entries = fs.readdirSync(projPath); } catch { continue; } try { entries = fs.readdirSync(projPath); } catch { continue; }
@@ -192,9 +192,9 @@ function discoverJsonlFiles(): ClaudeJsonlFile[] {
return files; return files;
} }
function discoverCodexJsonlFiles(): CodexJsonlFile[] { function discoverCodexJsonlFiles(sessionsDir = CODEX_SESSIONS_DIR): CodexJsonlFile[] {
const files: CodexJsonlFile[] = []; const files: CodexJsonlFile[] = [];
if (!fs.existsSync(CODEX_SESSIONS_DIR)) return files; if (!fs.existsSync(sessionsDir)) return files;
const walk = (dir: string): void => { const walk = (dir: string): void => {
let entries; let entries;
try { entries = fs.readdirSync(dir, { withFileTypes: true }); } catch { return; } try { entries = fs.readdirSync(dir, { withFileTypes: true }); } catch { return; }
@@ -207,7 +207,7 @@ function discoverCodexJsonlFiles(): CodexJsonlFile[] {
} }
} }
}; };
walk(CODEX_SESSIONS_DIR); walk(sessionsDir);
return files; return files;
} }
+106
View File
@@ -0,0 +1,106 @@
import { persist } from './persist.ts';
import type { ProviderRegistry } from './providers/registry.ts';
import type { Cursor, IndexUnit, ProviderAdapter } from './providers/types.ts';
import type { SqliteDb } from './sqlite-types.ts';
export interface ProviderIndexItem {
readonly provider: ProviderAdapter;
readonly unit: IndexUnit;
readonly cursor: Cursor;
}
export interface ProviderIndexPlan {
readonly items: ProviderIndexItem[];
readonly pendingMarkers: ReadonlyMap<string, string>;
}
export interface ProviderIndexResult {
readonly committed: ProviderIndexItem[];
readonly failedProviders: ReadonlySet<string>;
readonly stopped?: { item: ProviderIndexItem; error: unknown };
}
export function storedProviderCursor(db: SqliteDb, key: string): Cursor {
const row = db.prepare('SELECT mtime, lines_processed FROM index_state WHERE jsonl_path = ?').get(key);
return row ? `${String(row.mtime)}:${String(row.lines_processed)}` : null;
}
function sourceAlreadyIndexed(db: SqliteDb, source: string): boolean {
return Boolean(db.prepare('SELECT 1 FROM sessions WHERE source = ? LIMIT 1').get(source));
}
export function createProviderIndexPlan(
db: SqliteDb,
registry: ProviderRegistry,
{ force = false, changedPaths }: { force?: boolean; changedPaths?: string[] } = {},
): ProviderIndexPlan {
const items: ProviderIndexItem[] = [];
const pendingMarkers = new Map<string, string>();
for (const provider of registry.list()) {
const marker = provider.indexVersionMarker;
const markerMissing = marker !== undefined && !db.prepare(
'SELECT jsonl_path FROM index_state WHERE jsonl_path = ?',
).get(marker);
if (markerMissing) pendingMarkers.set(provider.name, marker);
const fullReindex = force || (markerMissing && sourceAlreadyIndexed(db, provider.name));
const units = provider.discover({
lastCursor: fullReindex ? () => null : (key) => storedProviderCursor(db, key),
changedPaths: fullReindex ? undefined : changedPaths,
});
for (const unit of units) {
items.push({
provider,
unit,
cursor: fullReindex ? null : storedProviderCursor(db, unit.key),
});
}
}
return { items, pendingMarkers };
}
export function indexProviderPlan({
db,
plan,
runTransaction,
onCommitted = () => {},
onError,
}: {
db: SqliteDb;
plan: ProviderIndexPlan;
runTransaction: <T>(label: string, work: () => T) => T;
onCommitted?: (item: ProviderIndexItem, cursor: Cursor) => void;
onError: (error: unknown, item: ProviderIndexItem) => 'skip' | 'stop';
}): ProviderIndexResult {
const committed: ProviderIndexItem[] = [];
const failedProviders = new Set<string>();
for (const item of plan.items) {
try {
const cursor = runTransaction(`provider:${item.provider.name}:${item.unit.key}`, () => (
persist(db, item.unit, item.provider.parse(item.unit, item.cursor))
));
committed.push(item);
onCommitted(item, cursor);
} catch (error) {
failedProviders.add(item.provider.name);
if (onError(error, item) === 'stop') {
return { committed, failedProviders, stopped: { item, error } };
}
}
}
return { committed, failedProviders };
}
export function writeProviderIndexMarkers(
db: SqliteDb,
plan: ProviderIndexPlan,
result: ProviderIndexResult,
): void {
const write = db.prepare(
'INSERT OR REPLACE INTO index_state (jsonl_path, mtime, lines_processed) VALUES (?, ?, 0)',
);
for (const [provider, marker] of plan.pendingMarkers) {
if (!result.failedProviders.has(provider) && result.stopped === undefined) {
write.run(marker, Date.now());
}
}
}
+14
View File
@@ -0,0 +1,14 @@
import { createClaudeProvider } from './claude.ts';
import { createCodexProvider } from './codex.ts';
import { createKimiProvider } from './kimi.ts';
import { createProviderRegistry, type ProviderRegistry } from './registry.ts';
export type BuiltinProviderRoots = Readonly<Record<string, string | undefined>>;
export function createBuiltinProviderRegistry(roots: BuiltinProviderRoots = {}): ProviderRegistry {
return createProviderRegistry([
createClaudeProvider({ rootDir: roots['claude'] }),
createCodexProvider({ rootDir: roots['codex'] }),
createKimiProvider({ rootDir: roots['kimi'] }),
]);
}
+139 -7
View File
@@ -7,6 +7,8 @@
// (started_at/ended_at/message_count); persist merges them with any existing row. // (started_at/ended_at/message_count); persist merges them with any existing row.
import { createRequire } from 'node:module'; import { createRequire } from 'node:module';
import { homedir } from 'node:os';
import { dirname, isAbsolute, join, normalize, relative } from 'node:path';
const require = createRequire(import.meta.url); const require = createRequire(import.meta.url);
const fs = require('node:fs'); const fs = require('node:fs');
@@ -15,7 +17,15 @@ import {
filePath, trunc, truncJson, readLines, discoverJsonlFiles, filePath, trunc, truncJson, readLines, discoverJsonlFiles,
} from '../parsing.ts'; } from '../parsing.ts';
import type { Cursor, DiscoverContext, IndexRecord, IndexUnit, Provider } from './types.ts'; import type {
Cursor,
DiscoverContext,
IndexRecord,
IndexUnit,
ProviderAdapter,
RawLookup,
RawRecord,
} from './types.ts';
// Claude cursor encodes the file mtime and the number of lines already indexed: // Claude cursor encodes the file mtime and the number of lines already indexed:
// "<mtimeMs>:<linesProcessed>". mtime lets discovery detect change; lines lets // "<mtimeMs>:<linesProcessed>". mtime lets discovery detect change; lines lets
@@ -46,8 +56,32 @@ function totalInputTokens(usage: Record<string, unknown>): number | null {
return seen ? total : null; return seen ? total : null;
} }
export function discover(_ctx: DiscoverContext): IndexUnit[] { function discoverAt(rootDir: string, ctx: DiscoverContext): IndexUnit[] {
return discoverJsonlFiles().map((f: any) => ({ const projectsDir = join(rootDir, 'projects');
const changedTranscriptPaths = new Set<string>();
const forcedPaths = new Set<string>();
for (const changedPath of ctx.changedPaths ?? []) {
const absolute = isAbsolute(changedPath)
? normalize(changedPath)
: normalize(join(projectsDir, changedPath));
const inside = relative(projectsDir, absolute);
if (!inside || inside.startsWith('..') || isAbsolute(inside)) continue;
if (absolute.toLowerCase().endsWith('.meta.json')) {
const transcript = absolute.slice(0, -'.meta.json'.length) + '.jsonl';
changedTranscriptPaths.add(transcript);
forcedPaths.add(transcript);
} else if (absolute.toLowerCase().endsWith('.jsonl')) {
changedTranscriptPaths.add(absolute);
}
}
return discoverJsonlFiles(projectsDir).filter((file) => {
const normalizedPath = normalize(file.path);
if (ctx.changedPaths !== undefined && !changedTranscriptPaths.has(normalizedPath)) return false;
const cursor = ctx.lastCursor(file.path);
return forcedPaths.has(normalizedPath)
|| cursor === null
|| Number(cursor.split(':')[0]) < fs.statSync(file.path).mtimeMs;
}).map((f: any) => ({
key: f.path, key: f.path,
sessionId: f.sessionId, sessionId: f.sessionId,
project: f.project, project: f.project,
@@ -57,6 +91,10 @@ export function discover(_ctx: DiscoverContext): IndexUnit[] {
})); }));
} }
export function discover(ctx: DiscoverContext): IndexUnit[] {
return discoverAt(join(homedir(), '.claude'), ctx);
}
export function* parse(unit: IndexUnit, cursor: Cursor): Generator<IndexRecord, Cursor> { export function* parse(unit: IndexUnit, cursor: Cursor): Generator<IndexRecord, Cursor> {
const skip = cursorToSkip(cursor); const skip = cursorToSkip(cursor);
const mtime = fs.statSync(unit.key).mtimeMs; const mtime = fs.statSync(unit.key).mtimeMs;
@@ -70,15 +108,28 @@ export function* parse(unit: IndexUnit, cursor: Cursor): Generator<IndexRecord,
title: null as string | null, title: null as string | null,
n: 0, n: 0,
}; };
const subagentStats = {
startedAt: null as string | null,
endedAt: null as string | null,
totalTokens: 0,
};
let lineNum = 0; let lineNum = 0;
readLines(unit.key, (line: string) => { readLines(unit.key, (line: string) => {
lineNum++; lineNum++;
if (lineNum <= skip) return;
let obj: any; let obj: any;
try { obj = JSON.parse(line); } catch { return; } try { obj = JSON.parse(line); } catch { return; }
const sid = unit.sessionId; const sid = unit.sessionId;
const ts = obj.timestamp || null; const ts = obj.timestamp || null;
const msg = obj.message || {};
const usage = msg.usage || {};
if (isSubagent && (obj.type === 'user' || obj.type === 'assistant')) {
if (ts && (!subagentStats.startedAt || ts < subagentStats.startedAt)) subagentStats.startedAt = ts;
if (ts && (!subagentStats.endedAt || ts > subagentStats.endedAt)) subagentStats.endedAt = ts;
subagentStats.totalTokens += (totalInputTokens(usage) ?? 0) + (usage.output_tokens ?? 0);
}
if (lineNum <= skip) return;
if (obj.type === 'ai-title' && obj.aiTitle) { sm.title = obj.aiTitle; return; } if (obj.type === 'ai-title' && obj.aiTitle) { sm.title = obj.aiTitle; return; }
if (obj.type === 'system' && obj.subtype === 'away_summary' && obj.content) { if (obj.type === 'system' && obj.subtype === 'away_summary' && obj.content) {
@@ -97,11 +148,9 @@ export function* parse(unit: IndexUnit, cursor: Cursor): Generator<IndexRecord,
if (obj.version) sm.version = obj.version; if (obj.version) sm.version = obj.version;
sm.n++; sm.n++;
const msg = obj.message || {};
const text = extractText(msg.content); const text = extractText(msg.content);
const contentType = extractContentType(msg.content); const contentType = extractContentType(msg.content);
const isMeta = extractMessageIsMeta(obj, text); const isMeta = extractMessageIsMeta(obj, text);
const usage = msg.usage || {};
const aid = isSubagent ? (unit.agentId ?? null) : (obj.agentId || null); const aid = isSubagent ? (unit.agentId ?? null) : (obj.agentId || null);
if (obj.uuid) { if (obj.uuid) {
@@ -132,6 +181,39 @@ export function* parse(unit: IndexUnit, cursor: Cursor): Generator<IndexRecord,
} }
}); });
if (isSubagent && unit.agentId) {
const metaPath = unit.key.replace(/\.jsonl$/, '.meta.json');
if (fs.existsSync(metaPath)) {
try {
const meta = JSON.parse(fs.readFileSync(metaPath, 'utf8'));
const workflowRunId = (unit.meta as { workflowRunId?: string } | undefined)?.workflowRunId;
if (workflowRunId) {
records.push({
kind: 'workflow_agent',
agent_id: unit.agentId,
run_id: workflowRunId,
session_id: unit.sessionId,
agent_type: meta.agentType || null,
description: meta.description || null,
});
} else {
const started = subagentStats.startedAt ? new Date(subagentStats.startedAt).getTime() : null;
const ended = subagentStats.endedAt ? new Date(subagentStats.endedAt).getTime() : null;
records.push({
kind: 'subagent',
agent_id: unit.agentId,
session_id: unit.sessionId,
parent_tool_use_id: meta.toolUseId || null,
agent_type: meta.agentType || null,
description: meta.description || null,
duration_ms: started !== null && ended !== null ? ended - started : null,
total_tokens: subagentStats.totalTokens,
});
}
} catch { /* malformed optional subagent metadata */ }
}
}
// Subagent transcripts do not own a session row (matches indexJsonl). // Subagent transcripts do not own a session row (matches indexJsonl).
if (!isSubagent) { if (!isSubagent) {
records.push({ records.push({
@@ -146,4 +228,54 @@ export function* parse(unit: IndexUnit, cursor: Cursor): Generator<IndexRecord,
return `${mtime}:${lineNum}`; return `${mtime}:${lineNum}`;
} }
export const claudeProvider: Provider = { name, discover, parse }; function rawClaude(input: RawLookup): RawRecord | null {
const mainPath = typeof input.session?.jsonl_path === 'string' ? input.session.jsonl_path : null;
if (mainPath === null) return null;
let sourcePath = mainPath;
if (input.agentId !== null) {
const runId = input.workflowAgent?.['run_id'];
sourcePath = typeof runId === 'string'
? join(dirname(mainPath), String(input.session?.id ?? ''), 'subagents', 'workflows', runId, `${input.agentId}.jsonl`)
: join(dirname(mainPath), String(input.session?.id ?? ''), 'subagents', `${input.agentId}.jsonl`);
}
if (!fs.existsSync(sourcePath)) return null;
let found: string | null = null;
readLines(sourcePath, (line: string) => {
if (!line.includes(input.messageUuid)) return;
try {
if (JSON.parse(line)?.uuid === input.messageUuid) {
found = line;
return false;
}
} catch { /* malformed source line */ }
});
const raw = found as string | null;
let messageText: string | null = null;
if (raw !== null) {
try {
const content = JSON.parse(raw)?.message?.content;
if (typeof content === 'string') messageText = content;
else if (Array.isArray(content)) {
const parts = content.map((part) => part?.text ?? part?.thinking).filter((part) => typeof part === 'string');
messageText = parts.length > 0 ? parts.join('\n') : null;
}
} catch { /* malformed source line */ }
}
return raw === null
? null
: { text: raw, totalLength: raw.length, offset: 0, limit: raw.length, hasMore: false, messageText };
}
export function createClaudeProvider({ rootDir = join(homedir(), '.claude') }: { rootDir?: string } = {}): ProviderAdapter {
return {
name,
descriptor: { id: name, name: 'Claude Code', vendor: 'Anthropic', defaultRoot: rootDir, color: '#d97757' },
indexVersionMarker: CLAUDE_INPUT_TOKEN_SEMANTICS_MARKER,
watchRoots: (configuredRoot) => [join(configuredRoot, 'projects')],
discover: (ctx) => discoverAt(rootDir, ctx),
parse,
raw: rawClaude,
};
}
export const claudeProvider = createClaudeProvider();
+117 -4
View File
@@ -9,6 +9,8 @@
// The per-line logic mirrors the original indexCodexJsonl. // The per-line logic mirrors the original indexCodexJsonl.
import { createRequire } from 'node:module'; import { createRequire } from 'node:module';
import { homedir } from 'node:os';
import { isAbsolute, join, normalize, relative } from 'node:path';
const require = createRequire(import.meta.url); const require = createRequire(import.meta.url);
const fs = require('node:fs'); const fs = require('node:fs');
@@ -19,14 +21,62 @@ import {
codexIsGuardianThread, codexAgentNickname, codexAgentRole, codexUsage, codexIsGuardianThread, codexAgentNickname, codexAgentRole, codexUsage,
codexEventText, codexMessagePayloadText, codexVisibleMessageKey, codexEventText, codexMessagePayloadText, codexVisibleMessageKey,
codexToolInput, codexToolOutput, codexToolInput, codexToolOutput,
readCodexGuardianThreadInfo,
} from '../parsing.ts'; } from '../parsing.ts';
import type { Cursor, DiscoverContext, IndexRecord, IndexUnit, MessageRecord, Provider } from './types.ts'; import type {
Cursor,
DiscoverContext,
IndexRecord,
IndexUnit,
MessageRecord,
ProviderAdapter,
RawLookup,
RawRecord,
} from './types.ts';
export const name = 'codex'; export const name = 'codex';
export function discover(_ctx: DiscoverContext): IndexUnit[] { function discoverAt(rootDir: string, ctx: DiscoverContext): IndexUnit[] {
return discoverCodexJsonlFiles().map((f: any) => ({ key: f.path, sessionId: '', meta: { source: 'codex' } })); const sessionsDir = join(rootDir, 'sessions');
const changedFiles = new Set<string>();
for (const changedPath of ctx.changedPaths ?? []) {
const absolute = isAbsolute(changedPath)
? normalize(changedPath)
: normalize(join(sessionsDir, changedPath));
const inside = relative(sessionsDir, absolute);
if (!inside || inside.startsWith('..') || isAbsolute(inside)) continue;
if (absolute.toLowerCase().endsWith('.jsonl')) changedFiles.add(absolute);
}
return discoverCodexJsonlFiles(sessionsDir).flatMap((file) => {
if (ctx.changedPaths !== undefined && !changedFiles.has(normalize(file.path))) return [];
const cursor = ctx.lastCursor(file.path);
const guardian = readCodexGuardianThreadInfo(file.path);
if (cursor !== null && Number(cursor.split(':')[0]) >= fs.statSync(file.path).mtimeMs && guardian === null) {
return [];
}
let meta: any = null;
readLines(file.path, (line: string) => {
try {
const record = JSON.parse(line);
if (record?.type === 'session_meta' && record.payload?.id) {
meta = record.payload;
return false;
}
} catch { /* malformed source line */ }
});
const rawId = meta ? codexRawId(meta.id) : null;
const parentId = meta ? codexParentThreadId(meta) : null;
return [{
key: file.path,
sessionId: guardian === null ? codexDbId(parentId || rawId) ?? '' : '',
meta: { source: 'codex', guardian: guardian !== null },
}];
});
}
export function discover(ctx: DiscoverContext): IndexUnit[] {
return discoverAt(join(homedir(), '.codex'), ctx);
} }
export function* parse(unit: IndexUnit, _cursor: Cursor): Generator<IndexRecord, Cursor> { export function* parse(unit: IndexUnit, _cursor: Cursor): Generator<IndexRecord, Cursor> {
@@ -217,4 +267,67 @@ export function* parse(unit: IndexUnit, _cursor: Cursor): Generator<IndexRecord,
return outCursor; return outCursor;
} }
export const codexProvider: Provider = { name, discover, parse }; function findCodexFile(rootDir: string, rawThreadId: string): string | null {
const stack = [join(rootDir, 'sessions')];
while (stack.length > 0) {
const current = stack.pop()!;
if (!fs.existsSync(current)) continue;
for (const entry of fs.readdirSync(current, { withFileTypes: true })) {
const path = join(current, entry.name);
if (entry.isDirectory()) stack.push(path);
else if (entry.isFile() && entry.name.endsWith(`${rawThreadId}.jsonl`)) return path;
}
}
return null;
}
function rawCodex(rootDir: string, input: RawLookup): RawRecord | null {
const match = /^codex:([^:]+):(\d+)$/.exec(input.messageUuid);
if (match === null) return null;
const path = input.agentId === null && typeof input.session?.jsonl_path === 'string'
? input.session.jsonl_path
: findCodexFile(rootDir, match[1]!);
if (path === null || !fs.existsSync(path)) return null;
let lineNumber = 0;
let found: string | null = null;
readLines(path, (line: string) => {
lineNumber++;
if (lineNumber !== Number(match[2])) return;
found = line;
return false;
});
const raw = found as string | null;
let messageText: string | null = null;
if (raw !== null) {
try {
const obj = JSON.parse(raw);
const payload = obj?.payload ?? {};
if (obj?.type === 'event_msg') {
messageText = typeof payload.message === 'string'
? payload.message
: typeof payload.text === 'string'
? payload.text
: null;
} else if (obj?.type === 'response_item' && payload.type === 'message' && Array.isArray(payload.content)) {
const parts = payload.content.map((part: any) => part?.text).filter((part: unknown) => typeof part === 'string');
messageText = parts.length > 0 ? parts.join('\n') : null;
}
} catch { /* malformed source line */ }
}
return raw === null
? null
: { text: raw, totalLength: raw.length, offset: 0, limit: raw.length, hasMore: false, messageText };
}
export function createCodexProvider({ rootDir = join(homedir(), '.codex') }: { rootDir?: string } = {}): ProviderAdapter {
return {
name,
descriptor: { id: name, name: 'Codex', vendor: 'OpenAI', defaultRoot: rootDir, color: '#10a37f' },
watchRoots: (configuredRoot) => [join(configuredRoot, 'sessions'), join(configuredRoot, 'session_index.jsonl')],
discover: (ctx) => discoverAt(rootDir, ctx),
parse,
raw: (input) => rawCodex(rootDir, input),
};
}
export const codexProvider = createCodexProvider();
+656
View File
@@ -0,0 +1,656 @@
import {
existsSync,
readdirSync,
readFileSync,
statSync,
} from 'node:fs';
import { homedir } from 'node:os';
import { basename, dirname, isAbsolute, join, normalize, relative, sep } from 'node:path';
import { filePath, projectSlugFromPath, trunc, truncJson } from '../parsing.ts';
import type {
Cursor,
DiscoverContext,
IndexRecord,
IndexUnit,
MessageRecord,
ProviderAdapter,
RawLookup,
RawRecord,
SubagentRecord,
SummaryRecord,
ToolCallRecord,
ToolResultRecord,
} from './types.ts';
type JsonRecord = Record<string, any>;
interface KimiWireFile {
readonly agentId: string;
readonly main: boolean;
readonly path: string;
}
interface KimiSessionUnitMeta {
readonly kind: 'session';
readonly sessionDir: string;
readonly statePath: string;
readonly wireFiles: readonly KimiWireFile[];
readonly currentCursor: Exclude<Cursor, null>;
}
interface LineRecord {
readonly line: number;
readonly record: JsonRecord;
}
interface ProjectedSession {
readonly messages: MessageRecord[];
readonly toolCalls: ToolCallRecord[];
readonly toolResults: ToolResultRecord[];
readonly summaries: SummaryRecord[];
readonly subagents: SubagentRecord[];
readonly durations: IndexRecord[];
readonly mainMessageCount: number;
readonly mainWirePath: string;
}
const SOURCE = 'kimi';
function defaultKimiRoot(): string {
return process.env['KIMI_CODE_HOME'] ?? join(homedir(), '.kimi-code');
}
function readState(path: string): JsonRecord {
try {
const parsed = JSON.parse(readFileSync(path, 'utf8')) as unknown;
return parsed !== null && typeof parsed === 'object' && !Array.isArray(parsed)
? parsed as JsonRecord
: {};
} catch {
return {};
}
}
function readWire(path: string): LineRecord[] {
const lines = readFileSync(path, 'utf8').split('\n');
const records: LineRecord[] = [];
for (let index = 0; index < lines.length; index++) {
const line = lines[index]!.endsWith('\r') ? lines[index]!.slice(0, -1) : lines[index]!;
if (line.length === 0) continue;
try {
records.push({ line: index + 1, record: JSON.parse(line) as JsonRecord });
} catch (error) {
if (index === lines.length - 1) break;
throw new Error(`wire.jsonl: corrupted line ${index + 1} in ${path}: ${String(error)}`, {
cause: error,
});
}
}
return records;
}
function listWireFiles(sessionDir: string): KimiWireFile[] {
const agentsDir = join(sessionDir, 'agents');
const files: KimiWireFile[] = [];
if (existsSync(agentsDir)) {
for (const entry of readdirSync(agentsDir, { withFileTypes: true })) {
if (!entry.isDirectory()) continue;
const path = join(agentsDir, entry.name, 'wire.jsonl');
if (existsSync(path)) files.push({ agentId: entry.name, main: entry.name === 'main', path });
}
}
if (!files.some((file) => file.main)) {
const legacy = join(sessionDir, 'wire.jsonl');
if (existsSync(legacy)) files.push({ agentId: 'main', main: true, path: legacy });
}
return files.sort((a, b) => Number(b.main) - Number(a.main) || a.agentId.localeCompare(b.agentId));
}
function fileLineCount(path: string): number {
const raw = readFileSync(path, 'utf8');
if (raw.length === 0) return 0;
const newlines = raw.match(/\n/g)?.length ?? 0;
return newlines + (raw.endsWith('\n') ? 0 : 1);
}
function cursorFor(statePath: string, wires: readonly KimiWireFile[]): Exclude<Cursor, null> {
const paths = [statePath, ...wires.map((wire) => wire.path)].filter(existsSync);
let maxMtime = 0;
let totalLines = 0;
for (const path of paths) {
maxMtime = Math.max(maxMtime, statSync(path).mtimeMs);
totalLines += fileLineCount(path);
}
return `${maxMtime}:${totalLines}`;
}
function normalizeTime(value: unknown): string | null {
if (typeof value === 'number' && Number.isFinite(value)) return new Date(value).toISOString();
if (typeof value === 'string' && !Number.isNaN(Date.parse(value))) {
return new Date(value).toISOString();
}
return null;
}
function contentParts(content: unknown): JsonRecord[] {
if (typeof content === 'string') return [{ type: 'text', text: content }];
return Array.isArray(content)
? content.filter((part): part is JsonRecord => part !== null && typeof part === 'object')
: [];
}
function partText(part: JsonRecord): string | null {
if (part.type === 'text' && typeof part.text === 'string') return trunc(part.text);
if (part.type === 'thinking' && typeof part.thinking === 'string') return trunc(part.thinking);
return null;
}
function messageText(content: unknown): string | null {
const parts = contentParts(content);
const text = parts.map(partText).filter((value): value is string => value !== null);
return text.length > 0 ? trunc(text.join('\n')) : null;
}
function messageContentType(content: unknown): string {
const types = new Set(contentParts(content).map((part) => String(part.type ?? 'unknown')));
return types.size === 1 ? [...types][0]! : 'unknown';
}
function namespacedSessionId(nativeId: string): string {
return `kimi:${nativeId}`;
}
function namespacedAgentId(sessionId: string, agentId: string): string {
return `${sessionId}:${agentId}`;
}
function namespacedEventId(sessionId: string, agentId: string, nativeId: unknown, line: number): string {
const suffix = typeof nativeId === 'string' && nativeId.length > 0 ? nativeId : `line-${line}`;
return `${sessionId}:${agentId}:${suffix}`;
}
function namespacedToolId(sessionId: string, agentId: string, nativeId: unknown): string {
return `${sessionId}:${agentId}:${String(nativeId)}`;
}
function numericField(record: JsonRecord, ...fields: string[]): number | null {
const value = fields.map((field) => record[field]).find((candidate) => (
typeof candidate === 'number' && Number.isFinite(candidate)
));
return typeof value === 'number' && Number.isFinite(value) ? value : null;
}
function inputUsage(usage: JsonRecord): number | null {
const normalized = numericField(usage, 'input_tokens', 'inputTokens');
if (normalized !== null) return normalized;
const fields = ['inputOther', 'inputCacheRead', 'inputCacheCreation'];
const values = fields.map((field) => numericField(usage, field));
return values.some((value) => value !== null)
? values.reduce<number>((sum, value) => sum + (value ?? 0), 0)
: null;
}
function outputUsage(usage: JsonRecord): number | null {
return numericField(usage, 'output_tokens', 'outputTokens', 'output');
}
function isRealUserMessage(message: JsonRecord): boolean {
if (message.role !== 'user') return false;
const origin = message.origin as JsonRecord | undefined;
if (origin === undefined || origin.kind === 'user') return true;
return (origin.kind === 'skill_activation' || origin.kind === 'plugin_command')
&& origin.trigger === 'user-slash';
}
function projectSession(meta: KimiSessionUnitMeta, sessionId: string, state: JsonRecord): ProjectedSession {
const cwd = typeof state.cwd === 'string'
? state.cwd
: typeof state.workDir === 'string'
? state.workDir
: null;
const messages: MessageRecord[] = [];
const toolCalls: ToolCallRecord[] = [];
const toolResults: ToolResultRecord[] = [];
const summaries: SummaryRecord[] = [];
const durations: IndexRecord[] = [];
const childParentCalls = new Map<string, string>();
let mainMessageCount = 0;
for (const wire of meta.wireFiles) {
const wireMessageStart = messages.length;
const records = readWire(wire.path);
const agentDbId = wire.main ? null : namespacedAgentId(sessionId, wire.agentId);
let previousUuid: string | null = null;
let model: string | null = null;
const stepStarts = new Map<string, number>();
const stepMessages = new Map<string, MessageRecord[]>();
const callMessageUuids = new Map<string, string>();
const injectionMessageUuids = new Set<string>();
const realUserMessageUuids = new Set<string>();
let undoFloor = wireMessageStart;
const resetOpenState = (): void => {
stepStarts.clear();
stepMessages.clear();
callMessageUuids.clear();
};
const applyUndo = (count: number): void => {
if (count <= 0) return;
const removedMessageUuids = new Set<string>();
let removedUserCount = 0;
for (let index = messages.length - 1; index >= undoFloor; index--) {
const message = messages[index]!;
if (injectionMessageUuids.has(message.uuid)) continue;
messages.splice(index, 1);
removedMessageUuids.add(message.uuid);
injectionMessageUuids.delete(message.uuid);
if (wire.main) mainMessageCount--;
if (realUserMessageUuids.delete(message.uuid)) {
removedUserCount++;
if (removedUserCount >= count) break;
}
}
const removedToolIds = new Set(
toolCalls.filter((call) => removedMessageUuids.has(call.message_uuid)).map((call) => call.id),
);
for (let index = toolCalls.length - 1; index >= 0; index--) {
if (removedMessageUuids.has(toolCalls[index]!.message_uuid)) toolCalls.splice(index, 1);
}
for (let index = toolResults.length - 1; index >= 0; index--) {
const result = toolResults[index]!;
if (removedMessageUuids.has(result.message_uuid) || removedToolIds.has(result.tool_use_id)) {
toolResults.splice(index, 1);
}
}
for (let index = durations.length - 1; index >= 0; index--) {
const duration = durations[index]!;
if (duration.kind === 'message-turn-duration' && removedMessageUuids.has(duration.uuid)) {
durations.splice(index, 1);
}
}
previousUuid = messages.slice(wireMessageStart).at(-1)?.uuid ?? null;
resetOpenState();
};
const pushMessage = (message: MessageRecord, stepUuid?: string): void => {
messages.push(message);
if (wire.main) mainMessageCount++;
previousUuid = message.uuid;
if (stepUuid !== undefined) {
const entries = stepMessages.get(stepUuid) ?? [];
entries.push(message);
stepMessages.set(stepUuid, entries);
}
};
for (const { line, record } of records) {
const timestamp = normalizeTime(record.time);
if (record.type === 'config.update') {
model = typeof record.modelAlias === 'string' ? record.modelAlias : model;
continue;
}
if (record.type === 'context.clear') {
undoFloor = messages.length;
resetOpenState();
continue;
}
if (record.type === 'context.undo') {
applyUndo(typeof record.count === 'number' ? record.count : 0);
continue;
}
if (record.type === 'context.append_message') {
const source = record.message as JsonRecord | undefined;
if (source === undefined || typeof source.role !== 'string') continue;
const uuid = namespacedEventId(sessionId, wire.agentId, source.id, line);
const origin = source.origin as JsonRecord | undefined;
const messageUuid = uuid;
pushMessage({
kind: 'message',
uuid,
session_id: sessionId,
type: source.role,
parent_uuid: previousUuid,
timestamp,
role: source.role,
text: messageText(source.content),
content_type: messageContentType(source.content),
is_meta: origin !== undefined && origin.kind !== 'user' ? 1 : 0,
model,
is_sidechain: wire.main ? 0 : 1,
agent_id: agentDbId,
input_tokens: null,
output_tokens: null,
cwd,
skill: null,
source: SOURCE,
});
if (origin?.kind === 'injection') injectionMessageUuids.add(messageUuid);
if (isRealUserMessage(source)) realUserMessageUuids.add(messageUuid);
if (Array.isArray(source.toolCalls)) {
for (const call of source.toolCalls) {
if (call === null || typeof call !== 'object' || typeof call.id !== 'string') continue;
const fn = call.function as JsonRecord | undefined;
const name = typeof fn?.name === 'string' ? fn.name : 'tool';
let args: unknown = fn?.arguments ?? {};
if (typeof args === 'string') {
try { args = JSON.parse(args); } catch { args = { raw: args }; }
}
const toolId = namespacedToolId(sessionId, wire.agentId, call.id);
toolCalls.push({
kind: 'tool_call',
id: toolId,
message_uuid: messageUuid,
session_id: sessionId,
name,
input_json: truncJson(args) ?? '{}',
file_path: filePath(name, args as JsonRecord | undefined),
});
callMessageUuids.set(call.id, messageUuid);
}
}
if (source.role === 'tool' && typeof source.toolCallId === 'string') {
const toolId = namespacedToolId(sessionId, wire.agentId, source.toolCallId);
toolResults.push({
kind: 'tool_result',
tool_use_id: toolId,
message_uuid: messageUuid,
session_id: sessionId,
content: messageText(source.content) ?? '',
file_path: null,
is_error: source.isError === true ? 1 : 0,
});
}
continue;
}
if (record.type === 'context.apply_compaction') {
const content = typeof record.contextSummary === 'string'
? record.contextSummary
: typeof record.summary === 'string'
? record.summary
: messageText((record.summary as JsonRecord | undefined)?.content);
if (content !== null) {
summaries.push({
kind: 'summary',
id: namespacedEventId(sessionId, wire.agentId, undefined, line),
session_id: sessionId,
timestamp,
source: 'compaction',
content,
});
}
undoFloor = messages.length;
resetOpenState();
continue;
}
if (record.type !== 'context.append_loop_event') continue;
const event = record.event as JsonRecord | undefined;
if (event === undefined || typeof event.type !== 'string') continue;
if (event.type === 'step.begin' && typeof event.uuid === 'string') {
stepStarts.set(event.uuid, typeof record.time === 'number' ? record.time : 0);
continue;
}
if (event.type === 'content.part' && typeof event.stepUuid === 'string') {
const part = event.part as JsonRecord | undefined;
if (part === undefined) continue;
pushMessage({
kind: 'message',
uuid: namespacedEventId(sessionId, wire.agentId, event.uuid, line),
session_id: sessionId,
type: 'assistant',
parent_uuid: previousUuid,
timestamp,
role: 'assistant',
text: partText(part),
content_type: typeof part.type === 'string' ? part.type : 'unknown',
is_meta: 0,
model,
is_sidechain: wire.main ? 0 : 1,
agent_id: agentDbId,
input_tokens: null,
output_tokens: null,
cwd,
skill: null,
source: SOURCE,
}, event.stepUuid);
continue;
}
if (event.type === 'tool.call' && typeof event.stepUuid === 'string' && event.toolCallId !== undefined) {
const uuid = namespacedEventId(sessionId, wire.agentId, event.uuid, line);
const toolId = namespacedToolId(sessionId, wire.agentId, event.toolCallId);
pushMessage({
kind: 'message', uuid, session_id: sessionId, type: 'assistant', parent_uuid: previousUuid,
timestamp, role: 'assistant', text: null, content_type: 'tool_use', is_meta: 0, model,
is_sidechain: wire.main ? 0 : 1, agent_id: agentDbId, input_tokens: null,
output_tokens: null, cwd, skill: null, source: SOURCE,
}, event.stepUuid);
toolCalls.push({
kind: 'tool_call',
id: toolId,
message_uuid: uuid,
session_id: sessionId,
name: String(event.name ?? 'tool'),
input_json: truncJson(event.args ?? {}) ?? '{}',
file_path: filePath(String(event.name ?? 'tool'), event.args as JsonRecord | undefined),
});
callMessageUuids.set(String(event.toolCallId), uuid);
continue;
}
if (event.type === 'tool.result' && event.toolCallId !== undefined) {
const nativeToolId = String(event.toolCallId);
const result = event.result as JsonRecord | undefined;
const output = result?.output;
const content = typeof output === 'string' ? trunc(output) : truncJson(output ?? '') ?? '';
const toolId = namespacedToolId(sessionId, wire.agentId, nativeToolId);
toolResults.push({
kind: 'tool_result',
tool_use_id: toolId,
message_uuid: callMessageUuids.get(nativeToolId) ?? '',
session_id: sessionId,
content,
file_path: null,
is_error: result?.isError === true ? 1 : 0,
});
const childId = typeof content === 'string' ? /^agent_id:\s*(\S+)/m.exec(content)?.[1] : undefined;
if (childId !== undefined) childParentCalls.set(childId, toolId);
continue;
}
if (event.type === 'step.end' && typeof event.uuid === 'string') {
const step = stepMessages.get(event.uuid) ?? [];
const last = step.at(-1);
if (last !== undefined) {
const usage = event.usage as JsonRecord | undefined;
if (usage !== undefined) {
last.input_tokens = inputUsage(usage);
last.output_tokens = outputUsage(usage);
}
const started = stepStarts.get(event.uuid);
if (started !== undefined && typeof record.time === 'number' && record.time >= started) {
durations.push({ kind: 'message-turn-duration', uuid: last.uuid, turn_duration_ms: record.time - started });
}
}
}
}
}
const agents = state.agents as JsonRecord | undefined;
const subagents: SubagentRecord[] = [];
if (agents !== undefined) {
for (const [agentId, candidate] of Object.entries(agents)) {
if (agentId === 'main' || candidate === null || typeof candidate !== 'object') continue;
const agent = candidate as JsonRecord;
const labels = agent.labels as JsonRecord | undefined;
subagents.push({
kind: 'subagent',
agent_id: namespacedAgentId(sessionId, agentId),
session_id: sessionId,
parent_tool_use_id: childParentCalls.get(agentId) ?? null,
agent_type: typeof labels?.profile === 'string'
? labels.profile
: typeof agent.type === 'string'
? agent.type
: null,
description: typeof agent.swarmItem === 'string' ? agent.swarmItem : null,
duration_ms: null,
total_tokens: null,
});
}
}
return {
messages,
toolCalls,
toolResults,
summaries,
subagents,
durations,
mainMessageCount,
mainWirePath: meta.wireFiles.find((wire) => wire.main)?.path ?? join(meta.sessionDir, 'wire.jsonl'),
};
}
function sessionDirectories(rootDir: string): string[] {
const sessionsDir = join(rootDir, 'sessions');
if (!existsSync(sessionsDir)) return [];
const result: string[] = [];
for (const workspace of readdirSync(sessionsDir, { withFileTypes: true })) {
if (!workspace.isDirectory()) continue;
const workspaceDir = join(sessionsDir, workspace.name);
for (const session of readdirSync(workspaceDir, { withFileTypes: true })) {
if (session.isDirectory()) result.push(join(workspaceDir, session.name));
}
}
return result.sort();
}
function changedSessionDirectories(rootDir: string, changedPaths: readonly string[]): Set<string> {
const sessionsDir = join(rootDir, 'sessions');
const result = new Set<string>();
for (const changedPath of changedPaths) {
const absolute = isAbsolute(changedPath)
? normalize(changedPath)
: normalize(join(sessionsDir, changedPath));
const inside = relative(sessionsDir, absolute);
if (!inside || inside.startsWith('..') || isAbsolute(inside)) continue;
const [workspaceId, sessionId] = inside.split(sep);
if (workspaceId && sessionId) result.add(join(sessionsDir, workspaceId, sessionId));
}
return result;
}
function rawFromWire(path: string, messageUuid: string): RawRecord | null {
if (!existsSync(path)) return null;
const fallbackLine = /:line-(\d+)$/.exec(messageUuid)?.[1];
const nativeId = messageUuid.split(':').at(-1);
const lines = readFileSync(path, 'utf8').split(/\r?\n/);
const line = fallbackLine !== undefined
? lines[Number(fallbackLine) - 1]
: lines.find((candidate) => nativeId !== undefined && candidate.includes(nativeId));
if (!line) return null;
let projectedText: string | null = null;
try {
const record = JSON.parse(line) as JsonRecord;
if (record.type === 'context.append_message') {
const content = (record.message as JsonRecord | undefined)?.content;
const parts = contentParts(content).map((part) => {
if (part.type === 'text' && typeof part.text === 'string') return part.text;
if (part.type === 'thinking' && typeof part.thinking === 'string') return part.thinking;
return null;
}).filter((part): part is string => part !== null);
projectedText = parts.length > 0 ? parts.join('\n') : null;
} else if (record.type === 'context.append_loop_event') {
const part = (record.event as JsonRecord | undefined)?.part as JsonRecord | undefined;
if (part?.type === 'text' && typeof part.text === 'string') projectedText = part.text;
if (part?.type === 'thinking' && typeof part.thinking === 'string') projectedText = part.thinking;
}
} catch { /* malformed torn source line */ }
return {
text: line,
totalLength: line.length,
offset: 0,
limit: line.length,
hasMore: false,
messageText: projectedText,
};
}
export function createKimiProvider({ rootDir = defaultKimiRoot() }: { rootDir?: string } = {}): ProviderAdapter {
const name = SOURCE;
return {
name,
descriptor: { id: name, name: 'Kimi Code', vendor: 'Moonshot AI', defaultRoot: rootDir, color: '#6d6afc' },
watchRoots: (configuredRoot) => [join(configuredRoot, 'sessions'), join(configuredRoot, 'session_index.jsonl')],
discover(ctx: DiscoverContext): IndexUnit[] {
const units: IndexUnit[] = [];
const changedSessions = ctx.changedPaths === undefined
? null
: changedSessionDirectories(rootDir, ctx.changedPaths);
for (const sessionDir of sessionDirectories(rootDir)) {
if (changedSessions !== null && !changedSessions.has(sessionDir)) continue;
const statePath = join(sessionDir, 'state.json');
const wireFiles = listWireFiles(sessionDir);
if (wireFiles.length === 0) continue;
const currentCursor = cursorFor(statePath, wireFiles);
if (changedSessions === null && ctx.lastCursor(sessionDir) === currentCursor) continue;
const state = readState(statePath);
const cwd = typeof state.cwd === 'string' ? state.cwd : typeof state.workDir === 'string' ? state.workDir : null;
units.push({
key: sessionDir,
sessionId: namespacedSessionId(basename(sessionDir)),
project: projectSlugFromPath(cwd) ?? undefined,
meta: { kind: 'session', sessionDir, statePath, wireFiles, currentCursor } satisfies KimiSessionUnitMeta,
});
}
return units;
},
*parse(unit: IndexUnit, _cursor: Cursor): Generator<IndexRecord, Cursor> {
const meta = unit.meta as KimiSessionUnitMeta;
const before = cursorFor(meta.statePath, meta.wireFiles);
const state = readState(meta.statePath);
const projected = projectSession(meta, unit.sessionId, state);
const after = cursorFor(meta.statePath, meta.wireFiles);
if (before !== after) throw new Error(`Kimi session changed while indexing: ${meta.sessionDir}`);
yield { kind: 'delete-session', sessionId: unit.sessionId };
yield {
kind: 'session',
id: unit.sessionId,
title: typeof state.title === 'string'
? state.title
: typeof state.lastPrompt === 'string'
? state.lastPrompt
: null,
project: unit.project ?? null,
started_at: normalizeTime(state.createdAt),
ended_at: normalizeTime(state.updatedAt),
git_branch: null,
version: null,
message_count: projected.mainMessageCount,
countMode: 'total',
jsonl_path: projected.mainWirePath,
source: SOURCE,
};
yield* projected.messages;
yield* projected.toolCalls;
yield* projected.toolResults;
yield* projected.summaries;
yield* projected.subagents;
yield* projected.durations;
return after;
},
raw(input: RawLookup): RawRecord | null {
const mainPath = typeof input.session?.jsonl_path === 'string' ? input.session.jsonl_path : null;
if (mainPath === null) return null;
if (input.agentId === null) return rawFromWire(mainPath, input.messageUuid);
const rawAgentId = input.agentId.split(':').at(-1);
if (rawAgentId === undefined) return null;
const sessionDir = basename(dirname(mainPath)) === 'main'
? dirname(dirname(dirname(mainPath)))
: dirname(mainPath);
return rawFromWire(join(sessionDir, 'agents', rawAgentId, 'wire.jsonl'), input.messageUuid);
},
};
}
export const kimiProvider = createKimiProvider();
+41
View File
@@ -0,0 +1,41 @@
import type {
ProviderAdapter,
ProviderDescriptor,
RawLookup,
RawRecord,
} from './types.ts';
export interface ProviderRegistry {
catalog(): ProviderDescriptor[];
get(source: string): ProviderAdapter | undefined;
list(): ProviderAdapter[];
watchRoots(configuredRoots?: Readonly<Record<string, string>>): string[];
raw(input: RawLookup): RawRecord | null;
}
export function createProviderRegistry(providers: readonly ProviderAdapter[]): ProviderRegistry {
const byId = new Map<string, ProviderAdapter>();
for (const provider of providers) {
const id = provider.descriptor.id;
if (provider.name !== id) {
throw new Error(`Provider name "${provider.name}" must match descriptor id "${id}"`);
}
if (byId.has(id)) throw new Error(`Duplicate provider id: ${id}`);
byId.set(id, provider);
}
const list = (): ProviderAdapter[] => [...byId.values()];
return {
catalog: () => list().map((provider) => ({ ...provider.descriptor })),
get: (source) => byId.get(source),
list,
watchRoots: (configuredRoots = {}) => [
...new Set(
list().flatMap((provider) =>
provider.watchRoots(configuredRoots[provider.name] ?? provider.descriptor.defaultRoot),
),
),
],
raw: (input) => byId.get(input.source)?.raw(input) ?? null,
};
}
+38 -1
View File
@@ -222,6 +222,43 @@ export interface Provider {
readonly name: string; readonly name: string;
/** Discover units needing (re)indexing, using stored cursors to detect change. */ /** Discover units needing (re)indexing, using stored cursors to detect change. */
discover(ctx: DiscoverContext): IndexUnit[]; discover(ctx: DiscoverContext): IndexUnit[];
/** Stream records for one unit resuming from `cursor`; return the new cursor. */ /** Yield records for one unit resuming from `cursor`; return the new cursor. */
parse(unit: IndexUnit, cursor: Cursor): Generator<IndexRecord, Cursor>; parse(unit: IndexUnit, cursor: Cursor): Generator<IndexRecord, Cursor>;
} }
/** Serializable source metadata consumed by settings and renderer surfaces. */
export interface ProviderDescriptor {
readonly id: string;
readonly name: string;
readonly vendor: string;
readonly defaultRoot: string;
readonly color: string;
}
export interface RawLookup {
readonly source: string;
readonly messageUuid: string;
readonly session: Record<string, unknown> | null;
readonly agentId: string | null;
readonly subagent?: Record<string, unknown> | null;
readonly workflowAgent?: Record<string, unknown> | null;
}
export interface RawRecord {
readonly text: string;
readonly totalLength?: number;
readonly offset?: number;
readonly limit?: number;
readonly hasMore?: boolean;
/** Provider-projected full message body for renderer expansion. */
readonly messageText?: string | null;
}
/** Complete adapter interface used by every indexing and presentation caller. */
export interface ProviderAdapter extends Provider {
readonly descriptor: ProviderDescriptor;
/** Optional index semantics marker; absence forces one provider-owned replay. */
readonly indexVersionMarker?: string;
watchRoots(configuredRoot: string): string[];
raw(input: RawLookup): RawRecord | null;
}
+30 -70
View File
@@ -1,5 +1,7 @@
// Query and attune sandbox helpers for the Core package. // Query and attune sandbox helpers for the Core package.
import { readLines, fs, path } from './db.ts'; import { fs, path } from './db.ts';
import { createBuiltinProviderRegistry } from './providers/builtins.ts';
import type { ProviderRegistry } from './providers/registry.ts';
import type { SqliteDb, SqliteRow } from './sqlite-types.ts'; import type { SqliteDb, SqliteRow } from './sqlite-types.ts';
type DbRow = SqliteRow; type DbRow = SqliteRow;
@@ -100,7 +102,10 @@ function buildSafeFtsQuery(text: unknown): string {
.join(' '); .join(' ');
} }
function createQueryApi(db: SqliteDb) { function createQueryApi(
db: SqliteDb,
{ providerRegistry = createBuiltinProviderRegistry() }: { providerRegistry?: ProviderRegistry } = {},
) {
const q = (sql: string, ...p: any[]) => { const q = (sql: string, ...p: any[]) => {
assertReadOnlySql(sql); assertReadOnlySql(sql);
return db.prepare(sql).all(...p); return db.prepare(sql).all(...p);
@@ -450,79 +455,34 @@ function createQueryApi(db: SqliteDb) {
}; };
}; };
const resolveJsonlPath = (messageUuid: string): string | null => {
const msg = db.prepare('SELECT session_id, agent_id, source FROM messages WHERE uuid=?').get(messageUuid);
if (!msg) return null;
if (msg.source === 'codex' || String(messageUuid).startsWith('codex:')) {
const match = /^codex:([^:]+):(\d+)$/.exec(String(messageUuid));
if (!match) return null;
const rawThreadId = match[1];
if (!msg.agent_id) {
const ses = db.prepare('SELECT jsonl_path FROM sessions WHERE id=?').get(msg.session_id);
if (ses?.jsonl_path) return ses.jsonl_path;
}
return db.prepare(`
SELECT jsonl_path FROM index_state
WHERE jsonl_path LIKE ? AND jsonl_path LIKE '%.jsonl'
ORDER BY length(jsonl_path) ASC
LIMIT 1
`).get(`%${rawThreadId}.jsonl`)?.jsonl_path || null;
}
if (msg.agent_id) {
const wa = db.prepare('SELECT agent_id, run_id, session_id FROM workflow_agents WHERE agent_id=?').get(msg.agent_id);
if (wa) {
const ses = db.prepare('SELECT jsonl_path FROM sessions WHERE id=?').get(wa.session_id);
if (ses) return path.join(path.dirname(ses.jsonl_path), wa.session_id, 'subagents', 'workflows', wa.run_id, wa.agent_id + '.jsonl');
}
const sa = db.prepare('SELECT agent_id, session_id FROM subagents WHERE agent_id=?').get(msg.agent_id);
if (sa) {
const ses = db.prepare('SELECT jsonl_path FROM sessions WHERE id=?').get(sa.session_id);
if (ses) return path.join(path.dirname(ses.jsonl_path), sa.session_id, 'subagents', sa.agent_id + '.jsonl');
}
} else {
const ses = db.prepare('SELECT jsonl_path FROM sessions WHERE id=?').get(msg.session_id);
if (ses) return ses.jsonl_path;
}
return null;
};
const findCodexRawLine = (jsonlPath: string | null, uuid: string): string | null => {
const match = /^codex:[^:]+:(\d+)$/.exec(String(uuid));
if (!match || !jsonlPath || !fs.existsSync(jsonlPath)) return null;
const targetLine = Number(match[1]);
let lineNum = 0;
let found = null;
readLines(jsonlPath, (line) => {
lineNum++;
if (lineNum !== targetLine) return;
found = line;
return false;
});
return found;
};
const findRawLine = (jsonlPath: string | null, uuid: string): string | null => {
if (!jsonlPath || !fs.existsSync(jsonlPath)) return null;
if (String(uuid).startsWith('codex:')) return findCodexRawLine(jsonlPath, uuid);
let found = null;
readLines(jsonlPath, (line) => {
if (!line.includes(uuid)) return;
try { const obj = JSON.parse(line); if (obj.uuid === uuid) { found = line; return false; } } catch { /* skip malformed JSONL lines */ }
});
return found;
};
const raw = (messageUuid: string, opts: { offset?: number; limit?: number } = {}) => { const raw = (messageUuid: string, opts: { offset?: number; limit?: number } = {}) => {
const { offset = 0, limit = 10000 } = opts; const { offset = 0, limit = 10000 } = opts;
const jsonlPath = resolveJsonlPath(messageUuid); const message = db.prepare('SELECT * FROM messages WHERE uuid=?').get(messageUuid);
const line = findRawLine(jsonlPath, messageUuid); if (!message) return null;
if (!line) return null; const session = db.prepare('SELECT * FROM sessions WHERE id=?').get(message.session_id) ?? null;
const subagent = message.agent_id
? db.prepare('SELECT * FROM subagents WHERE agent_id=?').get(message.agent_id) ?? null
: null;
const workflowAgent = message.agent_id
? db.prepare('SELECT * FROM workflow_agents WHERE agent_id=?').get(message.agent_id) ?? null
: null;
const source = message.source || session?.source || 'claude';
const record = providerRegistry.raw({
source,
messageUuid,
session,
agentId: message.agent_id || null,
subagent,
workflowAgent,
});
if (record === null) return null;
const totalLength = record.totalLength ?? record.text.length;
return { return {
text: line.slice(offset, offset + limit), text: record.text.slice(offset, offset + limit),
totalLength: line.length, totalLength,
offset, offset,
limit, limit,
hasMore: offset + limit < line.length, hasMore: offset + limit < totalLength,
}; };
}; };
+5
View File
@@ -17,6 +17,10 @@ const codexSession = {
project: '-Users-tomiya-Code-quiet-zero', project: '-Users-tomiya-Code-quiet-zero',
message_count: 2052, message_count: 2052,
}; };
const sourceCatalog = [
{ id: 'claude', name: 'Claude Code', color: '#d97757' },
{ id: 'codex', name: 'Codex', color: '#10a37f' },
];
test('single-source activity groups omit provider provenance', () => { test('single-source activity groups omit provider provenance', () => {
const split = { normal: [codexSession], noise: [{ ...codexSession, message_count: 12 }] }; const split = { normal: [codexSession], noise: [{ ...codexSession, message_count: 12 }] };
@@ -42,6 +46,7 @@ test('mixed activity groups expose provider before project and count', () => {
activitySessionMetaParts(claudeSession, { activitySessionMetaParts(claudeSession, {
mixedSources: true, mixedSources: true,
projectLabel: 'quiet-zero', projectLabel: 'quiet-zero',
sourceCatalog,
}), }),
[ [
{ kind: 'source', text: 'Claude Code' }, { kind: 'source', text: 'Claude Code' },
+3 -3
View File
@@ -292,8 +292,8 @@ test('indexer service passes changed JSONL paths to the build worker', async ()
assert.equal(calls.length, 1); assert.equal(calls.length, 1);
assert.equal(calls[0].reason, 'watch'); assert.equal(calls[0].reason, 'watch');
assert.deepEqual(calls[0].changedPaths, [ assert.deepEqual(calls[0].changedPaths, [
'project-a/session-1.jsonl', join(projectsDir, 'project-a/session-1.jsonl'),
'project-a/session-2.json', join(projectsDir, 'project-a/session-2.json'),
]); ]);
}); });
@@ -342,6 +342,6 @@ test('indexer service watches Claude projects and Codex sessions for app-side in
assert.equal(calls.length, 1); assert.equal(calls.length, 1);
assert.deepEqual(calls[0].changedPaths, [ assert.deepEqual(calls[0].changedPaths, [
'2026/06/15/rollout-2026-06-15T00-00-00-codex.jsonl', join(codexSessionsDir, '2026/06/15/rollout-2026-06-15T00-00-00-codex.jsonl'),
]); ]);
}); });
+125
View File
@@ -0,0 +1,125 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { createRequire } from 'node:module';
import { mkdirSync, mkdtempSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { buildIndex } from '../app/src/main/indexer.ts';
const require = createRequire(import.meta.url);
const { DatabaseSync } = require('node:sqlite');
class TestDatabase {
constructor(dbPath) {
this.db = new DatabaseSync(dbPath);
}
pragma(statement) { this.db.exec(`PRAGMA ${statement}`); }
exec(sql) { return this.db.exec(sql); }
prepare(sql) { return this.db.prepare(sql); }
close() { return this.db.close(); }
}
function writeSession(kimiDir) {
const sessionDir = join(kimiDir, 'sessions', 'workspace-1', 'session-index-1');
const mainDir = join(sessionDir, 'agents', 'main');
mkdirSync(mainDir, { recursive: true });
writeFileSync(join(sessionDir, 'state.json'), JSON.stringify({
title: 'Indexed Kimi session',
workDir: '/tmp/indexed-kimi',
createdAt: '2026-07-20T10:00:00.000Z',
updatedAt: '2026-07-20T10:01:00.000Z',
agents: { main: { type: 'main' } },
}));
const wirePath = join(mainDir, 'wire.jsonl');
const records = [
{ type: 'metadata', protocol_version: '1.5', created_at: 1753005600000 },
{ type: 'context.append_message', time: 1753005601000, message: { role: 'user', content: [{ type: 'text', text: 'kimi index needle' }], toolCalls: [], origin: { kind: 'user' } } },
];
writeFileSync(wirePath, records.map((record) => JSON.stringify(record)).join('\n') + '\n');
return { sessionDir, wirePath, records };
}
test('app build indexes Kimi sessions through the provider registry without changing schema', () => {
const home = mkdtempSync(join(tmpdir(), 'obelisk-kimi-index-'));
const claudeDir = join(home, '.claude');
const codexDir = join(home, '.codex');
const kimiDir = join(home, '.kimi-code');
const dbPath = join(home, '.obelisk', 'obelisk.sqlite');
writeSession(kimiDir);
const first = buildIndex({
claudeDir,
codexDir,
providerRoots: { kimi: kimiDir },
dbPath,
DatabaseImpl: TestDatabase,
});
assert.deepEqual(first.affectedSessionIds, ['kimi:session-index-1']);
const db = new TestDatabase(dbPath);
assert.deepEqual(
db.prepare('SELECT id,title,source,message_count FROM sessions').all().map((row) => ({ ...row })),
[{ id: 'kimi:session-index-1', title: 'Indexed Kimi session', source: 'kimi', message_count: 1 }],
);
assert.equal(db.prepare('SELECT text FROM messages').get().text, 'kimi index needle');
assert.equal(db.prepare('SELECT COUNT(*) AS c FROM index_state WHERE jsonl_path = ?').get(
join(kimiDir, 'sessions', 'workspace-1', 'session-index-1'),
).c, 1);
const schema = db.prepare("SELECT sql FROM sqlite_master WHERE type='table' AND name='messages'").get().sql;
assert.doesNotMatch(schema, /kimi/i);
db.close();
const second = buildIndex({
claudeDir,
codexDir,
providerRoots: { kimi: kimiDir },
dbPath,
DatabaseImpl: TestDatabase,
});
assert.deepEqual(second.affectedSessionIds, []);
});
test('Kimi undo and clear replace the indexed session instead of leaving stale rows', () => {
const home = mkdtempSync(join(tmpdir(), 'obelisk-kimi-replay-'));
const claudeDir = join(home, '.claude');
const codexDir = join(home, '.codex');
const kimiDir = join(home, '.kimi-code');
const dbPath = join(home, '.obelisk', 'obelisk.sqlite');
const { wirePath, records } = writeSession(kimiDir);
const assistant = {
type: 'context.append_loop_event',
time: 1753005602000,
event: { type: 'content.part', uuid: 'answer-1', stepUuid: 'step-1', part: { type: 'text', text: 'answer removed by undo' } },
};
writeFileSync(wirePath, [...records, assistant].map(record => JSON.stringify(record)).join('\n') + '\n');
const options = {
claudeDir,
codexDir,
providerRoots: { kimi: kimiDir },
dbPath,
DatabaseImpl: TestDatabase,
};
buildIndex(options);
let db = new TestDatabase(dbPath);
assert.equal(db.prepare('SELECT COUNT(*) AS c FROM messages').get().c, 2);
db.close();
// Kimi undo shrinks the durable wire transcript.
writeFileSync(wirePath, records.map(record => JSON.stringify(record)).join('\n') + '\n');
buildIndex(options);
db = new TestDatabase(dbPath);
assert.equal(db.prepare('SELECT COUNT(*) AS c FROM messages').get().c, 1);
assert.equal(db.prepare("SELECT COUNT(*) AS c FROM messages WHERE text LIKE '%removed by undo%'").get().c, 0);
db.close();
// Clear retains the session container but removes all projected messages.
writeFileSync(wirePath, JSON.stringify(records[0]) + '\n');
buildIndex(options);
db = new TestDatabase(dbPath);
assert.equal(db.prepare('SELECT COUNT(*) AS c FROM messages').get().c, 0);
assert.equal(db.prepare('SELECT message_count FROM sessions WHERE id=?').get('kimi:session-index-1').message_count, 0);
db.close();
});
+6 -5
View File
@@ -197,13 +197,14 @@ test('dev mode does not open DevTools unless explicitly requested', async () =>
assert.equal(devtoolsWindows[0].devToolsOpened, true); assert.equal(devtoolsWindows[0].devToolsOpened, true);
}); });
test('main process watches Codex sessions directory instead of Codex root', async () => { test('main process watches every root declared by the built-in provider registry', async () => {
const originalHome = process.env.HOME; const originalHome = process.env.HOME;
const home = join(tmpdir(), `obelisk-main-watch-dirs-${Date.now()}`); const home = join(tmpdir(), `obelisk-main-watch-dirs-${Date.now()}`);
const claudeDir = join(home, '.claude'); const claudeDir = join(home, '.claude');
const codexDir = join(home, '.codex'); const codexDir = join(home, '.codex');
mkdirSync(join(claudeDir, 'projects'), { recursive: true }); mkdirSync(join(claudeDir, 'projects'), { recursive: true });
mkdirSync(join(codexDir, 'sessions'), { recursive: true }); mkdirSync(join(codexDir, 'sessions'), { recursive: true });
mkdirSync(join(home, '.kimi-code', 'sessions'), { recursive: true });
mkdirSync(join(home, '.obelisk'), { recursive: true }); mkdirSync(join(home, '.obelisk'), { recursive: true });
writeFileSync(join(home, '.obelisk', 'obelisk.sqlite'), ''); writeFileSync(join(home, '.obelisk', 'obelisk.sqlite'), '');
process.env.HOME = home; process.env.HOME = home;
@@ -258,6 +259,9 @@ test('main process watches Codex sessions directory instead of Codex root', asyn
assert.deepEqual(serviceOptions[0].watchDirs, [ assert.deepEqual(serviceOptions[0].watchDirs, [
join(claudeDir, 'projects'), join(claudeDir, 'projects'),
join(codexDir, 'sessions'), join(codexDir, 'sessions'),
join(codexDir, 'session_index.jsonl'),
join(home, '.kimi-code', 'sessions'),
join(home, '.kimi-code', 'session_index.jsonl'),
]); ]);
assert.equal(serviceOptions[0].watchDirs.includes(codexDir), false); assert.equal(serviceOptions[0].watchDirs.includes(codexDir), false);
} finally { } finally {
@@ -414,10 +418,7 @@ test('session IPC hides Codex rows by default and supports explicit source opt-i
await ipcHandlers.get('settings:get')(); await ipcHandlers.get('settings:get')();
assert.ok( assert.ok(
queries.some(q => /COUNT\(\*\) as c FROM sessions WHERE COALESCE\(source, 'claude'\) = 'claude'/.test(q.sql)), queries.some(q => /GROUP BY COALESCE\(source, 'claude'\)/.test(q.sql)),
);
assert.ok(
queries.some(q => /MAX\(started_at\) as t FROM sessions WHERE COALESCE\(source, 'claude'\) = 'claude'/.test(q.sql)),
); );
} finally { } finally {
restore(); restore();
+79
View File
@@ -0,0 +1,79 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { createRequire } from 'node:module';
import { mkdtempSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { buildIndex } from '../app/src/main/indexer.ts';
import { createProviderRegistry } from '../packages/core/src/providers/registry.ts';
const require = createRequire(import.meta.url);
const { DatabaseSync } = require('node:sqlite');
class TestDatabase {
constructor(dbPath) { this.db = new DatabaseSync(dbPath); }
pragma(statement) { this.db.exec(`PRAGMA ${statement}`); }
exec(sql) { return this.db.exec(sql); }
prepare(sql) { return this.db.prepare(sql); }
close() { return this.db.close(); }
}
test('app indexer persists every provider through one registry-driven loop', () => {
const home = mkdtempSync(join(tmpdir(), 'obelisk-provider-indexer-'));
const dbPath = join(home, '.obelisk', 'obelisk.sqlite');
const registry = createProviderRegistry([{
name: 'alpha',
descriptor: { id: 'alpha', name: 'Alpha', vendor: 'Test', defaultRoot: '/alpha', color: '#123456' },
watchRoots: () => [],
discover(ctx) {
return ctx.lastCursor('alpha:unit') === '10:1'
? []
: [{ key: 'alpha:unit', sessionId: 'alpha:session', project: '-tmp-alpha' }];
},
*parse(unit) {
yield {
kind: 'session', id: unit.sessionId, title: 'Alpha session', project: unit.project,
started_at: '2026-07-20T10:00:00.000Z', ended_at: '2026-07-20T10:01:00.000Z',
git_branch: null, version: null, message_count: 1, countMode: 'total',
jsonl_path: unit.key, source: 'alpha',
};
yield {
kind: 'message', uuid: 'alpha:message', session_id: unit.sessionId, type: 'user',
parent_uuid: null, timestamp: '2026-07-20T10:00:00.000Z', role: 'user',
text: 'registry tracer bullet', content_type: 'text', is_meta: 0, model: null,
is_sidechain: 0, agent_id: null, input_tokens: null, output_tokens: null,
cwd: '/tmp/alpha', skill: null, source: 'alpha',
};
return '10:1';
},
raw: () => null,
}]);
const first = buildIndex({
providerRegistry: registry,
providerRoots: { alpha: '/alpha' },
claudeDir: join(home, 'empty-claude'),
codexDir: join(home, 'empty-codex'),
dbPath,
DatabaseImpl: TestDatabase,
});
assert.deepEqual(first.affectedSessionIds, ['alpha:session']);
assert.equal(first.files, 1);
const db = new TestDatabase(dbPath);
assert.deepEqual(
{ ...db.prepare('SELECT id,source,message_count FROM sessions').get() },
{ id: 'alpha:session', source: 'alpha', message_count: 1 },
);
db.close();
const second = buildIndex({
providerRegistry: registry,
providerRoots: { alpha: '/alpha' },
dbPath,
DatabaseImpl: TestDatabase,
});
assert.deepEqual(second.affectedSessionIds, []);
assert.equal(second.files, 0);
});
+227
View File
@@ -0,0 +1,227 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { createHash } from 'node:crypto';
import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { createKimiProvider } from '../packages/core/src/providers/kimi.ts';
function drain(gen) {
const values = [];
let step = gen.next();
while (!step.done) {
values.push(step.value);
step = gen.next();
}
return { values, ret: step.value };
}
function writeKimiFixture() {
const root = mkdtempSync(join(tmpdir(), 'obelisk-kimi-'));
const sessionDir = join(root, 'sessions', 'workspace-1', 'session-native-1');
const mainDir = join(sessionDir, 'agents', 'main');
const childDir = join(sessionDir, 'agents', 'agent-7');
mkdirSync(mainDir, { recursive: true });
mkdirSync(childDir, { recursive: true });
writeFileSync(join(sessionDir, 'state.json'), JSON.stringify({
title: 'Kimi fixture',
createdAt: '2026-07-20T10:00:00.000Z',
updatedAt: '2026-07-20T10:01:00.000Z',
workDir: '/tmp/kimi-project',
agents: {
main: { type: 'main' },
'agent-7': { type: 'sub', parentAgentId: 'main', labels: { profile: 'explore' } },
},
}));
const mainRecords = [
{ type: 'metadata', protocol_version: '1.5', created_at: 1753005600000 },
{ type: 'config.update', time: 1753005600100, modelAlias: 'kimi-k2' },
{ type: 'context.append_message', time: 1753005601000, message: { role: 'user', content: [{ type: 'text', text: 'inspect the project' }], toolCalls: [], origin: { kind: 'user' } } },
{ type: 'context.append_loop_event', time: 1753005602000, event: { type: 'step.begin', uuid: 'step-1', turnId: '0' } },
{ type: 'context.append_loop_event', time: 1753005602100, event: { type: 'content.part', uuid: 'thinking-1', stepUuid: 'step-1', part: { type: 'thinking', thinking: 'I should inspect it' } } },
{ type: 'context.append_loop_event', time: 1753005602200, event: { type: 'tool.call', uuid: 'tool-event-1', stepUuid: 'step-1', toolCallId: 'call-1', name: 'Read', args: { file_path: '/tmp/kimi-project/a.ts' } } },
{ type: 'context.append_loop_event', time: 1753005602300, event: { type: 'tool.result', parentUuid: 'tool-result-1', toolCallId: 'call-1', result: { output: 'agent_id: agent-7\nfile body', isError: false } } },
{ type: 'context.append_loop_event', time: 1753005602500, event: { type: 'content.part', uuid: 'text-1', stepUuid: 'step-1', part: { type: 'text', text: 'done' } } },
{ type: 'context.append_loop_event', time: 1753005603000, event: { type: 'step.end', uuid: 'step-1', usage: { inputOther: 7, inputCacheRead: 3, inputCacheCreation: 2, output: 3 } } },
{ type: 'context.apply_compaction', time: 1753005604000, summary: 'Earlier work summary', compactedCount: 2 },
];
writeFileSync(join(mainDir, 'wire.jsonl'), mainRecords.map((record) => JSON.stringify(record)).join('\n') + '\n');
const childRecords = [
{ type: 'metadata', protocol_version: '1.5', created_at: 1753005600000 },
{ type: 'context.append_message', time: 1753005602400, message: { role: 'user', content: [{ type: 'text', text: 'child prompt' }], toolCalls: [], origin: { kind: 'system_trigger', name: 'subagent' } } },
];
writeFileSync(join(childDir, 'wire.jsonl'), childRecords.map((record) => JSON.stringify(record)).join('\n') + '\n');
return { root, sessionDir };
}
test('kimi provider discovers a changed session directory and returns a stable cursor', () => {
const { root, sessionDir } = writeKimiFixture();
const provider = createKimiProvider({ rootDir: root });
const units = provider.discover({ lastCursor: () => null });
assert.equal(units.length, 1);
assert.equal(units[0].key, sessionDir);
assert.equal(units[0].sessionId, 'kimi:session-native-1');
assert.match(units[0].meta.currentCursor, /^\d+(?:\.\d+)?:\d+$/);
const unchanged = provider.discover({ lastCursor: () => units[0].meta.currentCursor });
assert.deepEqual(unchanged, []);
});
test('kimi provider folds main and subagent wire logs into the existing record language', () => {
const { root } = writeKimiFixture();
const provider = createKimiProvider({ rootDir: root });
const unit = provider.discover({ lastCursor: () => null })[0];
const { values, ret } = drain(provider.parse(unit, null));
const byKind = (kind) => values.filter((record) => record.kind === kind);
const goldenRecords = values.map((record) => record.kind === 'session'
? { ...record, jsonl_path: '<fixture-wire>' }
: record);
assert.equal(
createHash('sha256').update(JSON.stringify(goldenRecords)).digest('hex'),
'ce3c70798bbc50e438605d86eafb28482630ee38dc2baa41a695975c84646822',
'complete yielded record sequence changed',
);
assert.deepEqual(values[0], { kind: 'delete-session', sessionId: 'kimi:session-native-1' });
assert.equal(ret, unit.meta.currentCursor);
const session = byKind('session')[0];
assert.deepEqual(
(({ id, title, project, source, countMode }) => ({ id, title, project, source, countMode }))(session),
{
id: 'kimi:session-native-1',
title: 'Kimi fixture',
project: '-tmp-kimi-project',
source: 'kimi',
countMode: 'total',
},
);
const messages = byKind('message');
assert.deepEqual(messages.map((message) => [message.role, message.content_type, message.text]), [
['user', 'text', 'inspect the project'],
['assistant', 'thinking', 'I should inspect it'],
['assistant', 'tool_use', null],
['assistant', 'text', 'done'],
['user', 'text', 'child prompt'],
]);
assert.equal(messages.at(-1).agent_id, 'kimi:session-native-1:agent-7');
assert.equal(messages.at(-1).is_sidechain, 1);
assert.equal(messages.find((message) => message.text === 'done').input_tokens, 12);
assert.equal(messages.find((message) => message.text === 'done').output_tokens, 3);
assert.deepEqual(byKind('tool_call').map((record) => [record.id, record.name, record.file_path]), [
['kimi:session-native-1:main:call-1', 'Read', '/tmp/kimi-project/a.ts'],
]);
assert.deepEqual(byKind('tool_result').map((record) => [record.tool_use_id, record.is_error]), [
['kimi:session-native-1:main:call-1', 0],
]);
assert.deepEqual(byKind('summary').map((record) => record.content), ['Earlier work summary']);
assert.deepEqual(byKind('subagent').map((record) => [record.agent_id, record.parent_tool_use_id, record.agent_type]), [
['kimi:session-native-1:agent-7', 'kimi:session-native-1:main:call-1', 'explore'],
]);
});
test('kimi provider ignores a torn final wire line until it is completed', () => {
const { root, sessionDir } = writeKimiFixture();
const wirePath = join(sessionDir, 'agents', 'main', 'wire.jsonl');
writeFileSync(wirePath, readFileSync(wirePath, 'utf8') + '{"type":"context.append_message"');
const provider = createKimiProvider({ rootDir: root });
const unit = provider.discover({ lastCursor: () => null })[0];
const { values, ret } = drain(provider.parse(unit, null));
assert.equal(values.filter(record => record.kind === 'message').length, 5);
assert.equal(ret, unit.meta.currentCursor);
});
test('kimi provider replays clear and undo markers with Kimi transcript semantics', () => {
const root = mkdtempSync(join(tmpdir(), 'obelisk-kimi-undo-'));
const sessionDir = join(root, 'sessions', 'workspace-1', 'session-undo-1');
const mainDir = join(sessionDir, 'agents', 'main');
mkdirSync(mainDir, { recursive: true });
writeFileSync(join(sessionDir, 'state.json'), JSON.stringify({
workDir: '/tmp/kimi-undo',
agents: { main: { type: 'main' } },
}));
const records = [
{ type: 'metadata', protocol_version: '1.5', created_at: 1753005600000 },
{ type: 'context.append_message', time: 1, message: { role: 'user', content: 'before clear', toolCalls: [], origin: { kind: 'user' } } },
{ type: 'context.append_loop_event', time: 2, event: { type: 'content.part', uuid: 'before-answer', stepUuid: 's1', part: { type: 'text', text: 'kept answer' } } },
{ type: 'context.clear', time: 3 },
{ type: 'context.append_message', time: 4, message: { role: 'user', content: 'undone prompt', toolCalls: [], origin: { kind: 'user' } } },
{ type: 'context.append_message', time: 5, message: { role: 'user', content: 'persistent injection', toolCalls: [], origin: { kind: 'injection' } } },
{ type: 'context.append_message', time: 6, message: { role: 'user', content: 'ephemeral system trigger', toolCalls: [], origin: { kind: 'system_trigger' } } },
{ type: 'context.append_loop_event', time: 7, event: { type: 'content.part', uuid: 'undone-answer', stepUuid: 's2', part: { type: 'text', text: 'undone answer' } } },
{ type: 'context.undo', time: 8, count: 1 },
];
writeFileSync(join(mainDir, 'wire.jsonl'), records.map(record => JSON.stringify(record)).join('\n') + '\n');
const provider = createKimiProvider({ rootDir: root });
const unit = provider.discover({ lastCursor: () => null })[0];
const { values } = drain(provider.parse(unit, null));
assert.deepEqual(
values.filter(record => record.kind === 'message').map(record => record.text),
['before clear', 'kept answer', 'persistent injection'],
);
assert.equal(values.find(record => record.kind === 'session').message_count, 3);
});
test('kimi provider scopes changed-path discovery to one session and bypasses an unchanged cursor', () => {
const root = mkdtempSync(join(tmpdir(), 'obelisk-kimi-changed-path-'));
const firstDir = join(root, 'sessions', 'workspace-1', 'session-1');
const secondDir = join(root, 'sessions', 'workspace-1', 'session-2');
for (const sessionDir of [firstDir, secondDir]) {
mkdirSync(join(sessionDir, 'agents', 'main'), { recursive: true });
writeFileSync(join(sessionDir, 'state.json'), JSON.stringify({ workDir: '/tmp/project' }));
writeFileSync(join(sessionDir, 'agents', 'main', 'wire.jsonl'), '{"type":"metadata"}\n');
}
const provider = createKimiProvider({ rootDir: root });
const initial = provider.discover({ lastCursor: () => null });
const cursorByKey = new Map(initial.map(unit => [unit.key, unit.meta.currentCursor]));
const units = provider.discover({
lastCursor: key => cursorByKey.get(key) ?? null,
changedPaths: [join(firstDir, 'state.json')],
});
assert.deepEqual(units.map(unit => unit.key), [firstDir]);
});
test('kimi provider maps protocol-1.0 embedded tool calls and results', () => {
const root = mkdtempSync(join(tmpdir(), 'obelisk-kimi-legacy-tools-'));
const sessionDir = join(root, 'sessions', 'workspace-1', 'session-tools-1');
const mainDir = join(sessionDir, 'agents', 'main');
mkdirSync(mainDir, { recursive: true });
writeFileSync(join(sessionDir, 'state.json'), JSON.stringify({ workDir: '/tmp/tools' }));
const records = [
{ type: 'metadata', protocol_version: '1.0', created_at: 1 },
{ type: 'context.append_message', time: 2, message: {
role: 'assistant', content: [],
toolCalls: [{ type: 'function', id: 'legacy-call', function: { name: 'Read', arguments: '{"file_path":"/tmp/tools/a.ts"}' } }],
} },
{ type: 'context.append_message', time: 3, message: {
role: 'tool', content: [{ type: 'text', text: 'legacy result' }], toolCalls: [], toolCallId: 'legacy-call',
} },
];
writeFileSync(join(mainDir, 'wire.jsonl'), records.map(record => JSON.stringify(record)).join('\n') + '\n');
const provider = createKimiProvider({ rootDir: root });
const unit = provider.discover({ lastCursor: () => null })[0];
const { values } = drain(provider.parse(unit, null));
assert.deepEqual(values.filter(record => record.kind === 'tool_call').map(record => ({
id: record.id, name: record.name, file_path: record.file_path,
})), [{
id: 'kimi:session-tools-1:main:legacy-call', name: 'Read', file_path: '/tmp/tools/a.ts',
}]);
assert.deepEqual(values.filter(record => record.kind === 'tool_result').map(record => ({
tool_use_id: record.tool_use_id, content: record.content,
})), [{
tool_use_id: 'kimi:session-tools-1:main:legacy-call', content: 'legacy result',
}]);
});
+46
View File
@@ -0,0 +1,46 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { mkdirSync, mkdtempSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { spawnSync } from 'node:child_process';
import { pathToFileURL } from 'node:url';
test('passive-pull runtime indexes Kimi sessions from the default home', () => {
const home = mkdtempSync(join(tmpdir(), 'obelisk-kimi-runtime-'));
const sessionDir = join(home, '.kimi-code', 'sessions', 'workspace-1', 'session-runtime-1');
const mainDir = join(sessionDir, 'agents', 'main');
mkdirSync(mainDir, { recursive: true });
writeFileSync(join(sessionDir, 'state.json'), JSON.stringify({
title: 'Runtime Kimi session',
workDir: '/tmp/runtime-kimi',
createdAt: '2026-07-20T10:00:00.000Z',
updatedAt: '2026-07-20T10:01:00.000Z',
agents: { main: { type: 'main' } },
}));
writeFileSync(join(mainDir, 'wire.jsonl'), [
JSON.stringify({ type: 'metadata', protocol_version: '1.5', created_at: 1753005600000 }),
JSON.stringify({ type: 'context.append_message', time: 1753005601000, message: { role: 'user', content: [{ type: 'text', text: 'runtime kimi needle' }], toolCalls: [], origin: { kind: 'user' } } }),
'',
].join('\n'));
const coreUrl = pathToFileURL(join(process.cwd(), 'packages/core/src/core.ts')).href;
const script = `
import { executeQuery } from ${JSON.stringify(coreUrl)};
const result = await executeQuery("return sessions({ source: 'kimi', limit: 5 });");
process.stdout.write(JSON.stringify(result));
`;
const run = spawnSync(process.execPath, ['--experimental-strip-types', '--input-type=module', '-e', script], {
cwd: process.cwd(),
env: { ...process.env, HOME: home, USERPROFILE: home },
encoding: 'utf8',
});
assert.equal(run.status, 0, run.stderr);
const sessions = JSON.parse(run.stdout);
assert.deepEqual(sessions.map(({ id, title, source }) => ({ id, title, source })), [{
id: 'kimi:session-runtime-1',
title: 'Runtime Kimi session',
source: 'kimi',
}]);
});
+78
View File
@@ -0,0 +1,78 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { createProviderRegistry } from '../packages/core/src/providers/registry.ts';
import { createBuiltinProviderRegistry } from '../packages/core/src/providers/builtins.ts';
function fakeProvider(id, root) {
return {
name: id,
descriptor: {
id,
name: `${id} display`,
vendor: `${id} vendor`,
defaultRoot: root,
color: '#123456',
},
watchRoots(configuredRoot) {
return [`${configuredRoot}/sessions`, `${configuredRoot}/session-index`];
},
discover() {
return [];
},
*parse() {
yield* [];
return null;
},
raw(input) {
return { text: `${id}:${input.messageUuid}` };
},
};
}
test('provider registry drives source catalog, watch roots, and raw lookup', () => {
const registry = createProviderRegistry([
fakeProvider('alpha', '/default/alpha'),
fakeProvider('beta', '/default/beta'),
]);
assert.deepEqual(registry.catalog(), [
{ id: 'alpha', name: 'alpha display', vendor: 'alpha vendor', defaultRoot: '/default/alpha', color: '#123456' },
{ id: 'beta', name: 'beta display', vendor: 'beta vendor', defaultRoot: '/default/beta', color: '#123456' },
]);
assert.deepEqual(registry.watchRoots({ alpha: '/custom/alpha' }), [
'/custom/alpha/sessions',
'/custom/alpha/session-index',
'/default/beta/sessions',
'/default/beta/session-index',
]);
assert.deepEqual(
registry.raw({ source: 'beta', messageUuid: 'message-1', session: null, agentId: null }),
{ text: 'beta:message-1' },
);
assert.equal(
registry.raw({ source: 'missing', messageUuid: 'message-1', session: null, agentId: null }),
null,
);
});
test('built-in provider registry exposes every source without caller-side branching', () => {
const registry = createBuiltinProviderRegistry({
claude: '/sources/claude',
codex: '/sources/codex',
kimi: '/sources/kimi',
});
assert.deepEqual(registry.catalog().map(({ id, name }) => ({ id, name })), [
{ id: 'claude', name: 'Claude Code' },
{ id: 'codex', name: 'Codex' },
{ id: 'kimi', name: 'Kimi Code' },
]);
assert.deepEqual(registry.watchRoots(), [
'/sources/claude/projects',
'/sources/codex/sessions',
'/sources/codex/session_index.jsonl',
'/sources/kimi/sessions',
'/sources/kimi/session_index.jsonl',
]);
});
+12
View File
@@ -0,0 +1,12 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { createHash } from 'node:crypto';
import { readFileSync } from 'node:fs';
test('provider adapters do not change the frozen SQLite schema', () => {
const schema = readFileSync(new URL('../packages/core/src/schema.sql', import.meta.url));
assert.equal(
createHash('sha256').update(schema).digest('hex'),
'3e0615ed2db0d7338561df4d51c4240395714c191aa69567ffcdb70efec49826',
);
});
+75
View File
@@ -0,0 +1,75 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { createProviderRegistry } from '../packages/core/src/providers/registry.ts';
import {
buildSourceCatalog,
resolveProviderRoots,
setPersistedSetting,
} from '../app/src/main/provider-settings.ts';
function provider(id, defaultRoot, color) {
return {
name: id,
descriptor: { id, name: `${id} name`, vendor: `${id} vendor`, defaultRoot, color },
watchRoots: () => [],
discover: () => [],
*parse() { yield* []; return null; },
raw: () => null,
};
}
test('provider roots and source settings are derived from the registry without source branches', () => {
const registry = createProviderRegistry([
provider('alpha', '/default/alpha', '#112233'),
provider('beta', '/default/beta', '#445566'),
]);
const persisted = {
alphaDir: '/legacy/alpha',
providerRoots: { beta: '/custom/beta' },
};
assert.deepEqual(resolveProviderRoots(registry, persisted), {
alpha: '/legacy/alpha',
beta: '/custom/beta',
});
const rootsChanged = setPersistedSetting(persisted, 'providerRoots.alpha', '/custom/alpha');
assert.equal(rootsChanged, true);
assert.deepEqual(persisted.providerRoots, {
alpha: '/custom/alpha',
beta: '/custom/beta',
});
const sources = buildSourceCatalog({
registry,
roots: resolveProviderRoots(registry, persisted),
stats: new Map([
['alpha', { sessionCount: 2, lastIndexed: '2026-07-20T10:00:00.000Z' }],
['beta', { sessionCount: 0, lastIndexed: '' }],
]),
pathExists: path => path === '/custom/alpha' || path === '/custom/beta',
});
assert.deepEqual(sources, [
{
id: 'alpha', name: 'alpha name', vendor: 'alpha vendor', color: '#112233',
path: '/custom/alpha', settingKey: 'providerRoots.alpha', exists: true,
sessionCount: 2, lastIndexed: '2026-07-20T10:00:00.000Z',
status: 'ok', statusText: 'Connected',
},
{
id: 'beta', name: 'beta name', vendor: 'beta vendor', color: '#445566',
path: '/custom/beta', settingKey: 'providerRoots.beta', exists: true,
sessionCount: 0, lastIndexed: '', status: 'warn', statusText: 'No sessions found',
},
]);
});
test('removing a generic provider root restores its descriptor default', () => {
const registry = createProviderRegistry([provider('gamma', '/default/gamma', '#778899')]);
const persisted = { providerRoots: { gamma: '/custom/gamma' } };
assert.equal(setPersistedSetting(persisted, 'providerRoots.gamma', null), true);
assert.deepEqual(resolveProviderRoots(registry, persisted), { gamma: '/default/gamma' });
});
+48
View File
@@ -0,0 +1,48 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { createRequire } from 'node:module';
import { readFileSync } from 'node:fs';
import { createQueryApi } from '../packages/core/src/query.ts';
import { createProviderRegistry } from '../packages/core/src/providers/registry.ts';
const require = createRequire(import.meta.url);
const { DatabaseSync } = require('node:sqlite');
const SCHEMA = readFileSync(new URL('../packages/core/src/schema.sql', import.meta.url), 'utf8');
test('raw query delegates source semantics to the registered provider', () => {
const calls = [];
const registry = createProviderRegistry([{
name: 'alpha',
descriptor: { id: 'alpha', name: 'Alpha', vendor: 'Test', defaultRoot: '/alpha', color: '#123456' },
watchRoots: () => [],
discover: () => [],
*parse() { yield* []; return null; },
raw(input) {
calls.push(input);
return { text: '0123456789', totalLength: 10 };
},
}]);
const db = new DatabaseSync(':memory:');
db.exec(SCHEMA);
db.prepare('INSERT INTO sessions (id,jsonl_path,source) VALUES (?,?,?)')
.run('alpha:session', '/alpha/session.data', 'alpha');
db.prepare('INSERT INTO messages (uuid,session_id,agent_id,source) VALUES (?,?,?,?)')
.run('alpha:message', 'alpha:session', 'alpha:agent', 'alpha');
db.prepare('INSERT INTO subagents (agent_id,session_id,description) VALUES (?,?,?)')
.run('alpha:agent', 'alpha:session', 'agent metadata');
const result = createQueryApi(db, { providerRegistry: registry }).raw('alpha:message', {
offset: 2,
limit: 4,
});
assert.deepEqual(result, {
text: '2345', totalLength: 10, offset: 2, limit: 4, hasMore: true,
});
assert.equal(calls.length, 1);
assert.equal(calls[0].source, 'alpha');
assert.equal(calls[0].session.id, 'alpha:session');
assert.equal(calls[0].subagent.description, 'agent metadata');
db.close();
});
+16
View File
@@ -0,0 +1,16 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { sourceColor, sourceLabel } from '../app/src/renderer/src/source-catalog.mjs';
const catalog = [
{ id: 'alpha', name: 'Alpha Agent', color: '#112233' },
{ id: 'beta', name: 'Beta Code', color: '#445566' },
];
test('renderer source presentation comes from the runtime provider catalog', () => {
assert.equal(sourceLabel('beta', catalog), 'Beta Code');
assert.equal(sourceColor('alpha', catalog), '#112233');
assert.equal(sourceLabel('future-provider', catalog), 'Future Provider');
assert.equal(sourceColor('future-provider', catalog), '#8b8b93');
});