Pi cannot be read as another linear JSONL stream. Its history is a tree with
a durable leaf, orphan roots, branch summaries, and two compaction forms, so
the active context is something the format states rather than something line
order implies. The adapter keeps those semantics inside itself and projects
the result into the existing canonical tables.
Sessions are keyed by (normalized header cwd, header id) rather than by path,
because Pi's --session-id lookup is project-local: two projects may reuse an
id, while a move or an identical copy is still one session. Discovery covers
both layouts Pi writes and fingerprints each file by mtime, ctime, size and
inode, so a rewrite that preserves mtime is not read as unchanged.
Abandoned branches are preserved rather than dropped. Visibility becomes
three-state -- visible, inactive, hidden -- and helpers return only visible
rows until includeInactive asks for the superseded path, labeling every row
so a caller knows which it holds. Usage counts all three, because an
abandoned call still spent tokens; message_count reports only the visible
transcript.
A committed MIT-licensed oracle transcribed from Pi 0.83.0 pins the context
algorithms, and a fixed-seed differential runs 512 generated sessions against
it on every test run. Schema changes are additive.
Implement the full ADR-0006 plan: three-layer separation of transaction
correctness, retry policy, and cross-process writer coordination.
Layer 1 — scripts/tx.ts (transaction correctness):
- runWriteTransaction executes work exactly once; no internal retry.
- BEGIN IMMEDIATE takes the write lock up front (avoids SQLITE_BUSY_SNAPSHOT).
- Guarded rollback: checks inTransaction() via adapter before attempting
ROLLBACK; never masks the primary exception.
- WriteTxDiagnostics attached to errors: phase, code, label,
rollbackSucceeded, rollbackError, transactionActive.
- Binding adapters (betterSqliteTransactionAdapter, nodeSqliteTransactionAdapter)
mapping better-sqlite3's `.inTransaction` and node:sqlite's `.isTransaction`.
- configureConnection centralizes WAL + synchronous + busy_timeout.
Layer 2 — scripts/write-coordinator.ts (retry policy):
- runRetryableWriteTransaction: bounded retry with total time budget.
- Only retries when the transaction confirmed ended (transactionActive=false)
and the error is SQLITE_BUSY during work/commit phase.
- BEGIN-phase BUSY = abort entire build (isBeginBusyFailure); the caller
returns `{ deferred: true, reason: 'writer_busy' }` instead of waiting.
- hasUnusableTransaction detects a still-active transaction after failure;
aborts the build immediately, never retries.
Layer 3 — scripts/writer-lease.ts (cross-process coordination):
- acquireWriterLease: dedicated writer.lock.sqlite with busy_timeout=0 +
BEGIN IMMEDIATE. Non-blocking attempt; bounded wait with retryDelayMs.
- writerLockPathFor derives lock path from the target DB path.
- Lease held for the entire build; released on completion or failure.
- Lock DB uses DELETE journal (not WAL); crash/close auto-releases.
- All consumers obey: skill acquires at build start (returns deferred if
unavailable); app daemon (via worker) acquires for its build cycle.
Build semantics changes:
- affectedSessionIds updated only after successful commit.
- BuildIndexResult gains skipped/skippedFiles for observability.
- Skill finalize failure now fails the build (was silently warned).
- Checkpoint changed to PASSIVE (TRUNCATE reserved for maintenance/exit).
- Skill buildIndex returns { deferred, reason } on lease contention;
indexer-service reschedules the build (deferredRetryMs) without publishing
a heartbeat (so the build-deferred state is visible to cross-process
arbitration).
- Service publishes heartbeat immediately on start() for correct arbitration.
Tests:
- tests/write-transaction.test.mjs: single-shot execution, diagnostics
propagation, auto-rolled-back transaction detected, rollback failure
captured as metadata, BEGIN IMMEDIATE semantics.
- tests/writer-lease.test.mjs: acquire/release, contention returns null,
bounded wait with release during budget.
- tests/app-writer-lease.test.mjs: better-sqlite3 adapter integration.
- tests/app-rollback-guard.test.mjs: rewritten — transient BUSY recovered
by coordinator, persistent BUSY skips file, begin-busy aborts build,
live-transaction aborts build, phantom affectedSessionIds prevented.
- tests/daemon-arbitration.test.mjs: skill defers to fresh app heartbeat,
builds when heartbeat is stale.
- tests/app-indexer-service.test.mjs: new cases for deferred-retry
scheduling and immediate heartbeat on start.
- app/tests/electron-concurrency.mjs + child: dual-child IPC structure for
real better-sqlite3 contention (holder acquires lock → build child starts
→ delayed release → result collected; persistent contention bounded).
ADR-0006 updated to reflect the implemented design.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Convert the app's main and preload source from .js to .ts (git mv preserves
history), adding types where they carry value: the core-consumption seam
(BuildIndexOptions/BuildIndexResult, FileInfo), the indexer service/worker
factories, and the preload IPC bridge. Module-to-module specifiers use the
real .ts extension (mirroring scripts/, since Node type-stripping does not
rewrite .js->.ts); the worker's runtime path stays indexer-worker.js because
that is the built output.
Toolchain:
- Add app/tsconfig.json: strict but noImplicitAny:false (the app orchestrates
the already-strict core; annotating every SQLite-handle helper is low-value
churn) + allowImportingTsExtensions (safe under noEmit).
- Add @types/better-sqlite3 for the injected binding.
- electron.vite.config.ts inputs -> .ts; refresh the stale CommonJS comment.
- typecheck script runs root + app projects. Root tsconfig excludes the
app-importing tests (app-*.test.mjs, recap-capture-query.test.mjs) so the
lenient app files are not dragged into the strict root program; the app
source is covered by app/tsconfig.json instead. See docs/adr/0005.
Verified: npm run typecheck (root + app) clean; suite 121/121; electron-vite
build emits all 6 main entries + preload with no .ts/node:sqlite residue in
the bundles.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The desktop app now indexes through the shared provider adapters + persist
layer (scripts/providers/{claude,codex}, scripts/persist, scripts/parsing)
instead of maintaining its own parallel indexer. buildIndex shrinks from
~1173 to ~592 lines, eliminating the skill<->app parse duplication that
Phase 5 set out to remove. electron-vite bundles the .ts core from source
with better-sqlite3 injected; the provider->parsing graph stays
node:sqlite-free so nothing drags node:sqlite into the app.
Also fix a misleading log: when a manual rebuild tears down the worker
mid-build, the cancelled background build is a deliberate stop, not a
failure. Guard the service's failure log with the stopped flag so it no
longer prints "Obelisk index build failed: Indexer worker stopped" on
every rebuild.
- CONTEXT.md: provider-adapter + single-persist + node:sqlite-free parsing.
- docs/adr/0005: app builds with electron-vite (TS+ESM), packages with
electron-builder; preload CJS for sandbox; app consumes core from source.
Verified: full suite 121/121; a node:sqlite-adapter dogfood of the rebuild
path over real data (969 files, 285 sessions, FTS rebuilt) runs clean; app
Rebuild confirmed in real Electron/better-sqlite3.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
App source (main/preload/worker/renderer) -> ESM; app is now type: module;
__dirname via import.meta.url; worker spawned with type module. Preload is built
as CJS (electron-vite output format) because the sandboxed renderer does not
support ESM preload; main loads ../preload/index.js. Removed dead imports
(nativeImage, readline) and the obsolete scripts/dev.js.
Tests: 4 app tests require->import; app-main-settings rewritten with node:test
mock.module + dynamic import (replacing CJS Module._load mocking); test script
adds --experimental-test-module-mocks. electron-vite build clean, 119/119, and
npm run dev verified: app launches, preload bridges IPC, data loads.
Restructure app into src/{main,preload,renderer}; electron.vite.config.ts builds
all three (each main module its own input so CJS requires + the indexer worker
resolve; better-sqlite3 externalized; Vue plugin for renderer). main/index.js
paths updated for the out/ layout + ELECTRON_RENDERER_URL. Still JS/CJS — TS+ESM
and core consumption are the next stages. Verified: npm run dev launches clean;
electron-vite build succeeds; root suite 119/119.
SaladDay
tommy0103
