Files
obelisk/scripts/tx.ts
T
tommy0103andClaude Opus 4.8 e3e61cc7ab fix: coordinate sqlite index writers
Implement the full ADR-0006 plan: three-layer separation of transaction
correctness, retry policy, and cross-process writer coordination.

Layer 1 — scripts/tx.ts (transaction correctness):
- runWriteTransaction executes work exactly once; no internal retry.
- BEGIN IMMEDIATE takes the write lock up front (avoids SQLITE_BUSY_SNAPSHOT).
- Guarded rollback: checks inTransaction() via adapter before attempting
  ROLLBACK; never masks the primary exception.
- WriteTxDiagnostics attached to errors: phase, code, label,
  rollbackSucceeded, rollbackError, transactionActive.
- Binding adapters (betterSqliteTransactionAdapter, nodeSqliteTransactionAdapter)
  mapping better-sqlite3's `.inTransaction` and node:sqlite's `.isTransaction`.
- configureConnection centralizes WAL + synchronous + busy_timeout.

Layer 2 — scripts/write-coordinator.ts (retry policy):
- runRetryableWriteTransaction: bounded retry with total time budget.
- Only retries when the transaction confirmed ended (transactionActive=false)
  and the error is SQLITE_BUSY during work/commit phase.
- BEGIN-phase BUSY = abort entire build (isBeginBusyFailure); the caller
  returns `{ deferred: true, reason: 'writer_busy' }` instead of waiting.
- hasUnusableTransaction detects a still-active transaction after failure;
  aborts the build immediately, never retries.

Layer 3 — scripts/writer-lease.ts (cross-process coordination):
- acquireWriterLease: dedicated writer.lock.sqlite with busy_timeout=0 +
  BEGIN IMMEDIATE. Non-blocking attempt; bounded wait with retryDelayMs.
- writerLockPathFor derives lock path from the target DB path.
- Lease held for the entire build; released on completion or failure.
- Lock DB uses DELETE journal (not WAL); crash/close auto-releases.
- All consumers obey: skill acquires at build start (returns deferred if
  unavailable); app daemon (via worker) acquires for its build cycle.

Build semantics changes:
- affectedSessionIds updated only after successful commit.
- BuildIndexResult gains skipped/skippedFiles for observability.
- Skill finalize failure now fails the build (was silently warned).
- Checkpoint changed to PASSIVE (TRUNCATE reserved for maintenance/exit).
- Skill buildIndex returns { deferred, reason } on lease contention;
  indexer-service reschedules the build (deferredRetryMs) without publishing
  a heartbeat (so the build-deferred state is visible to cross-process
  arbitration).
- Service publishes heartbeat immediately on start() for correct arbitration.

Tests:
- tests/write-transaction.test.mjs: single-shot execution, diagnostics
  propagation, auto-rolled-back transaction detected, rollback failure
  captured as metadata, BEGIN IMMEDIATE semantics.
- tests/writer-lease.test.mjs: acquire/release, contention returns null,
  bounded wait with release during budget.
- tests/app-writer-lease.test.mjs: better-sqlite3 adapter integration.
- tests/app-rollback-guard.test.mjs: rewritten — transient BUSY recovered
  by coordinator, persistent BUSY skips file, begin-busy aborts build,
  live-transaction aborts build, phantom affectedSessionIds prevented.
- tests/daemon-arbitration.test.mjs: skill defers to fresh app heartbeat,
  builds when heartbeat is stale.
- tests/app-indexer-service.test.mjs: new cases for deferred-retry
  scheduling and immediate heartbeat on start.
- app/tests/electron-concurrency.mjs + child: dual-child IPC structure for
  real better-sqlite3 contention (holder acquires lock → build child starts
  → delayed release → result collected; persistent contention bounded).

ADR-0006 updated to reflect the implemented design.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 00:42:43 +08:00

139 lines
4.3 KiB
TypeScript

// Binding-agnostic SQLite write plumbing shared by the skill and app indexers
// (docs/adr/0006). The injected db must expose `exec(sql)`; this works for both
// node:sqlite (skill/CLI) and better-sqlite3 (app), same injection model as
// `persist`.
export interface WriteTxDb {
exec(sql: string): unknown;
inTransaction(): boolean;
}
export interface SqliteConnection {
exec(sql: string): unknown;
}
type Phase = 'begin' | 'work' | 'commit' | 'rollback';
export interface WriteTxDiagnostics {
phase: Phase;
code: string | null;
label?: string;
rollbackSucceeded: boolean | null;
rollbackError: string | null;
transactionActive: boolean | null;
attempts: number;
}
export interface WriteTxOptions {
// Diagnostic label for this transaction (e.g. a file path or 'finalize').
label?: string;
}
const BUSY_MESSAGE = /SQLITE_BUSY|database is locked|database is busy/i;
function busyCode(error: unknown): string | null {
const raw = error as { code?: unknown; errcode?: unknown; message?: unknown } | null;
const code = (raw?.code ?? raw?.errcode);
if (typeof code === 'string' && code.startsWith('SQLITE_BUSY')) return code;
if (typeof raw?.message === 'string' && BUSY_MESSAGE.test(raw.message)) return 'SQLITE_BUSY';
return null;
}
function errorCode(error: unknown): string | null {
const raw = error as { code?: unknown } | null;
return typeof raw?.code === 'string' ? raw.code : null;
}
interface BetterSqliteHandle {
exec(sql: string): unknown;
readonly inTransaction: boolean;
}
interface NodeSqliteHandle {
exec(sql: string): unknown;
readonly isTransaction: boolean;
}
export function betterSqliteTransactionAdapter(db: BetterSqliteHandle): WriteTxDb {
return {
exec: sql => db.exec(sql),
inTransaction: () => db.inTransaction,
};
}
export function nodeSqliteTransactionAdapter(db: NodeSqliteHandle): WriteTxDb {
return {
exec: sql => db.exec(sql),
inTransaction: () => db.isTransaction,
};
}
function transactionState(db: WriteTxDb): boolean | null {
try {
return db.inTransaction();
} catch {
return null;
}
}
function attachDiagnostics(error: unknown, diagnostics: WriteTxDiagnostics): void {
if (!error || typeof error !== 'object') return;
try {
(error as { obelisk?: WriteTxDiagnostics }).obelisk = diagnostics;
} catch {
// Frozen/native errors must still be rethrown unchanged.
}
}
// Runs `work` exactly once inside a transaction and returns its value. Retry and
// scheduling policy belongs to the build coordinator, which knows the operation's
// idempotency and total time budget. Cleanup never masks the primary exception.
export function runWriteTransaction<T>(db: WriteTxDb, work: () => T, options: WriteTxOptions = {}): T {
const { label } = options;
let phase: Phase = 'begin';
try {
db.exec('BEGIN IMMEDIATE');
phase = 'work';
const value = work();
phase = 'commit';
db.exec('COMMIT');
return value;
} catch (error) {
let rollbackSucceeded: boolean | null = null;
let rollbackError: string | null = null;
const activeBeforeRollback = transactionState(db);
if (activeBeforeRollback !== false) {
try {
db.exec('ROLLBACK');
rollbackSucceeded = true;
} catch (rollbackFailure) {
rollbackSucceeded = false;
rollbackError = rollbackFailure instanceof Error ? rollbackFailure.message : String(rollbackFailure);
}
}
const busy = busyCode(error);
const diagnostics: WriteTxDiagnostics = {
phase,
code: busy ?? errorCode(error),
label,
rollbackSucceeded,
rollbackError,
transactionActive: transactionState(db),
attempts: 1,
};
attachDiagnostics(error, diagnostics);
throw error;
}
}
// Applies the connection-level pragmas used by every Obelisk writer/reader. Uses
// exec (not better-sqlite3's .pragma) so one implementation covers both bindings.
// busy_timeout is a real behavior change for node:sqlite (no default); it is set
// explicitly for better-sqlite3 too, whose own default already happens to be
// 5000ms. It is NOT the concurrency fix — see docs/adr/0006.
export function configureConnection(db: SqliteConnection, { busyTimeoutMs = 5000 } = {}): void {
db.exec(`PRAGMA busy_timeout=${busyTimeoutMs}`);
db.exec('PRAGMA journal_mode=WAL');
db.exec('PRAGMA synchronous=NORMAL');
}