Implement the full ADR-0006 plan: three-layer separation of transaction
correctness, retry policy, and cross-process writer coordination.
Layer 1 — scripts/tx.ts (transaction correctness):
- runWriteTransaction executes work exactly once; no internal retry.
- BEGIN IMMEDIATE takes the write lock up front (avoids SQLITE_BUSY_SNAPSHOT).
- Guarded rollback: checks inTransaction() via adapter before attempting
ROLLBACK; never masks the primary exception.
- WriteTxDiagnostics attached to errors: phase, code, label,
rollbackSucceeded, rollbackError, transactionActive.
- Binding adapters (betterSqliteTransactionAdapter, nodeSqliteTransactionAdapter)
mapping better-sqlite3's `.inTransaction` and node:sqlite's `.isTransaction`.
- configureConnection centralizes WAL + synchronous + busy_timeout.
Layer 2 — scripts/write-coordinator.ts (retry policy):
- runRetryableWriteTransaction: bounded retry with total time budget.
- Only retries when the transaction confirmed ended (transactionActive=false)
and the error is SQLITE_BUSY during work/commit phase.
- BEGIN-phase BUSY = abort entire build (isBeginBusyFailure); the caller
returns `{ deferred: true, reason: 'writer_busy' }` instead of waiting.
- hasUnusableTransaction detects a still-active transaction after failure;
aborts the build immediately, never retries.
Layer 3 — scripts/writer-lease.ts (cross-process coordination):
- acquireWriterLease: dedicated writer.lock.sqlite with busy_timeout=0 +
BEGIN IMMEDIATE. Non-blocking attempt; bounded wait with retryDelayMs.
- writerLockPathFor derives lock path from the target DB path.
- Lease held for the entire build; released on completion or failure.
- Lock DB uses DELETE journal (not WAL); crash/close auto-releases.
- All consumers obey: skill acquires at build start (returns deferred if
unavailable); app daemon (via worker) acquires for its build cycle.
Build semantics changes:
- affectedSessionIds updated only after successful commit.
- BuildIndexResult gains skipped/skippedFiles for observability.
- Skill finalize failure now fails the build (was silently warned).
- Checkpoint changed to PASSIVE (TRUNCATE reserved for maintenance/exit).
- Skill buildIndex returns { deferred, reason } on lease contention;
indexer-service reschedules the build (deferredRetryMs) without publishing
a heartbeat (so the build-deferred state is visible to cross-process
arbitration).
- Service publishes heartbeat immediately on start() for correct arbitration.
Tests:
- tests/write-transaction.test.mjs: single-shot execution, diagnostics
propagation, auto-rolled-back transaction detected, rollback failure
captured as metadata, BEGIN IMMEDIATE semantics.
- tests/writer-lease.test.mjs: acquire/release, contention returns null,
bounded wait with release during budget.
- tests/app-writer-lease.test.mjs: better-sqlite3 adapter integration.
- tests/app-rollback-guard.test.mjs: rewritten — transient BUSY recovered
by coordinator, persistent BUSY skips file, begin-busy aborts build,
live-transaction aborts build, phantom affectedSessionIds prevented.
- tests/daemon-arbitration.test.mjs: skill defers to fresh app heartbeat,
builds when heartbeat is stale.
- tests/app-indexer-service.test.mjs: new cases for deferred-retry
scheduling and immediate heartbeat on start.
- app/tests/electron-concurrency.mjs + child: dual-child IPC structure for
real better-sqlite3 contention (holder acquires lock → build child starts
→ delayed release → result collected; persistent contention bounded).
ADR-0006 updated to reflect the implemented design.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reported as "cannot rollback - no transaction is active" — a secondary error.
SQLite auto-rolls back certain failures (SQLITE_BUSY / SQLITE_BUSY_SNAPSHOT,
disk full), after which the per-file loop's unguarded ROLLBACK in its catch
threw over the real error and aborted the whole build instead of skipping just
the bad file.
Stopgap only (see docs/adr/0006 for the planned full fix):
- Add safeRollback(db) in both indexers: it swallows only the rollback's own
error, so the true cause surfaces. Per-file failures are logged and the build
continues; the finalize failure still propagates.
- Give the skill's node:sqlite connection an explicit PRAGMA busy_timeout=5000
(it has no default). The app adds none: better-sqlite3 already defaults to
5000ms, so busy_timeout is NOT the root-cause fix and is not treated as one.
Add tests/app-rollback-guard.test.mjs: injects a DB that faithfully reproduces
"a write auto-rolls back the txn, then ROLLBACK errors" and asserts the build
survives (bad file skipped, other file indexed). Revert-checked: without the
guard the test fails with the exact "cannot rollback - no transaction is active".
docs/adr/0006 records the real fix (shared runWriteTransaction, single-writer
coordination, BEGIN IMMEDIATE, whole-transaction retry, PASSIVE checkpointing)
as deferred, two-phase work — and why bumping busy_timeout is not it.
Verified: suite 124/124, typecheck clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move all pure parse/discover helpers (message extraction, project-path, codex
helpers, discovery) out of db.mjs/indexer.mjs into scripts/parsing.mjs, which
imports only node:fs/path/os. Providers now import from parsing.mjs, so the
provider import graph no longer transitively loads node:sqlite — a prerequisite
for the app (Electron/Node without node:sqlite) to consume the compiled core.
Verbatim move, no behavior change; indexer.mjs 840→209 lines. 119/119.
Add `source` column to sessions and messages ('claude' | 'codex').
Discover and parse Codex JSONL files from ~/.codex/sessions/, mapping
Codex thread/item structures to the same schema (messages, tool_calls,
tool_results, subagents). Move DB to ~/.obelisk/ with legacy migration.
Add rebuild-to-temp-then-swap for safe full rebuilds. On the app side:
source filter toggle, collapsible untitled session fold, configurable
codexDir in Settings, and a dev script. Update SKILL.md and query
helpers to expose source fields and accept source filter opt.
Introduce a Settings view for configuring the Claude data directory
(with WSL auto-detection on Windows), sidebar project grouping module,
and an empty-state onboarding screen for SessionList. Refactor
recap-patterns.md into per-card reference files under references/recap/
with separate retrieval and writing guides. Remove the legacy panel.html.
On the data layer: incremental indexing via changedPaths, per-session
live-update IPC (obelisk:session-updated), and session dirty-tracking
in the renderer.
Extract schema DDL into scripts/schema.sql shared between CLI and app.
Add an in-process chokidar-based indexer-service that watches ~/.claude/projects
for JSONL changes, debounces, and triggers background rebuilds via a worker
thread. Rename Usage view to Activity, flesh out MemoryDetail and SubagentDetail
views, and refine App.vue layout/routing. The main process now starts/stops the
indexer lifecycle and notifies renderer windows on index updates.
Introduce an Electron app with session browser, memory list, and usage
views (vanilla JS + Vue scaffolding). On the data layer: add content_type
and is_meta to messages for transcript control-plane filtering, introduce
FTS5-backed memory recall with safe tokenization, support memory archival
via forget() through the renamed --attune runtime, and expose anchors on
memory records.
Add a memories table (survives index rebuilds) for agent-written
conclusions with provenance (session, message range, project). The agent
writes markdown files via Write tool (user-approved), then registers
them via a --remember CodeAct script with remember(). Recall via
memories() in --query scripts, filtered by project/session/time.
Separates query (read-only, assertReadOnlySql) from remember (write)
execution contexts in runtime.mjs.
lightweight workflowTree, build debounce, minor fixes
- Index phase, label, model, state, duration, tokens per workflow agent
- Index duration, total_tokens, status, name per workflow run
- workflowTree returns parsed result + agent summaries instead of
dumping all messages
- 30s debounce on buildIndex to avoid repeated directory scans
- Fix broken BASH_EXIT_PAT (SQLite LIKE has no character classes)
- Fix SKILL.md step numbering, document FTS5 hyphen limitation
Index the is_error boolean from JSONL tool_result blocks into a new
column, replacing the old ERROR_PATS text-matching approach that
produced ~90% false positives. Bash exit code pattern kept as fallback.
Prepares for adding Codex JSONL support — new indexers can be added
without touching the query layer.
- db.mjs: schema, openDb, truncation helpers, shared utilities
- indexer.mjs: file discovery, JSONL parsing, SQLite writes
- query.mjs: all query API functions (search, context, raw, etc.)
- runtime.mjs: thin CLI entry point + VM script executor
tommy0103andClaude Opus 4.8