Restructure app into src/{main,preload,renderer}; electron.vite.config.ts builds
all three (each main module its own input so CJS requires + the indexer worker
resolve; better-sqlite3 externalized; Vue plugin for renderer). main/index.js
paths updated for the out/ layout + ELECTRON_RENDERER_URL. Still JS/CJS — TS+ESM
and core consumption are the next stages. Verified: npm run dev launches clean;
electron-vite build succeeds; root suite 119/119.
Complete the skill-side provider migration: codex now goes through a pure adapter
and the shared persist layer, and the two original monolithic indexers are gone.
New:
- scripts/providers/codex.ts — pure codex adapter. Full-reparse (buffers the whole
file) because the event_msg↔response_item dedup needs whole-file, bidirectional
knowledge; emits SessionRecord with countMode 'total'. Handles guardian threads
(→ delete-session), agent spawns/tool calls (→ tool_call/subagent), token_count
(patched onto the message record) and task_complete (→ message-turn-duration).
Contract:
- SessionRecord.countMode ('total' | 'delta') tells persist whether to replace or
accumulate message_count — claude is line-incremental (delta), codex full-reparse
(total). SubagentRecord non-key fields are optional; persist merges them
column-wise with COALESCE. MessageTurnDurationRecord.turn_duration_ms is nullable.
Orchestration:
- buildIndex's codex branch parses via the adapter and writes via persist. An
unchanged file is skipped but still swept for stale guardian rows (routed through
persist as a delete-session), preserving prior behavior.
Cleanup:
- Remove the now-unused indexJsonl, indexCodexJsonl, deleteCodexThreadRows and
upsertCodexSubagent — their semantics now live in the adapters + persist.
indexer.mjs drops from ~840 to 428 lines. Codex pure helpers stay exported for
codex.ts and the guardian sweep (physical move deferred to the app-side reorg).
- Migrate the upsert drift test off indexJsonl to the claude.parse + persist path,
keeping the rowid-stability and count-replace regression guards.
Tests: tests/codex-parse.test.mjs (record-stream golden: dedup, tools, token patch,
turn-duration, guardian→delete) and tests/codex-index.test.mjs (full buildIndex
path: fresh build + incremental full-reparse, total-count replace, no duplicates).
Verified equivalent on the real ~/.obelisk index: codex messages 82476 and
subagents 522 identical before/after, zero guardian leakage; real incremental
confirmed (touch a codex file → reparsed idempotently, unchanged files skipped).
lint + typecheck clean, 119/119.
buildIndex's claude branch now parses via providers/claude.ts and writes via
persist.ts instead of the inlined indexJsonl. Behavior is equivalent — the full
buildIndex integration suite (runtime.test.mjs) stays green, 116/116 — and a
force rebuild of the real ~/.obelisk index (327 sessions, 119k messages)
reproduced identical session counts with project_path fully populated.
codex, indexSubagentMeta, workflows, history and the project_path pass are
untouched. indexJsonl is now unused by buildIndex (kept for its drift test;
removed once codex is migrated).
Adds tests/incremental-index.test.mjs: verifies resume/accumulate through the
full buildIndex path (append new lines to an indexed session → incremental
build resumes from the cursor, message_count accumulates, no duplicates). The
30s shouldSkipBuild debounce is cleared in-test so the incremental run fires.
Introduce scripts/persist.ts — the single, provider- and binding-agnostic
layer that consumes an adapter's IndexRecord stream and writes rows into an
injected SQLite handle (node:sqlite for skill/CLI, better-sqlite3 for the app).
It is the only layer that touches the database.
Write semantics are the canonical ones reconciled from the earlier drift:
- messages upsert via ON CONFLICT (turn_duration_ms not in the column list, so
it is never clobbered)
- sessions merge with the existing row: started_at MIN, ended_at MAX,
message_count reset-or-accumulate by resume state, fill-if-null for the rest;
project_path is preserved and left to refreshSessionProjectPaths
- message-turn-duration applies as a targeted UPDATE
- delete-session cascades across all tables
- the generator's return cursor is written back to index_state (mtime:lines →
the two existing columns; no schema migration yet)
Purely additive — buildIndex still uses the old indexJsonl path, so existing
behavior is unchanged. Rewiring happens in 5b-2b.
Adds tests/persist.test.mjs: all record kinds written, resume does not
double-count message_count, fresh re-scan resets it, delete-session cascades.
Full suite 115/115, lint + typecheck green.
claude.parse mirrors indexJsonl line-for-line but yields IndexRecords (no db).
Adds MessageTurnDurationRecord op. Purely additive — buildIndex still uses the
old path. 111/111 green.
scripts/indexer.mjs indexJsonl used INSERT OR REPLACE (rowid/FTS churn) and
carried message_count forward on full re-scan; align to app's ON CONFLICT upsert
+ count reset so the two indexers no longer silently diverge. Regression test added.
search() falls back to safe per-token quoting on malformed FTS input instead of
crashing (documented in api-reference.md). Adds raw() to the doc-synced shape
contract. Full suite 107/107.
tommy0103