fix(indexer): guard cleanup rollback so it never masks the real build error

Reported as "cannot rollback - no transaction is active" — a secondary error.
SQLite auto-rolls back certain failures (SQLITE_BUSY / SQLITE_BUSY_SNAPSHOT,
disk full), after which the per-file loop's unguarded ROLLBACK in its catch
threw over the real error and aborted the whole build instead of skipping just
the bad file.

Stopgap only (see docs/adr/0006 for the planned full fix):
- Add safeRollback(db) in both indexers: it swallows only the rollback's own
  error, so the true cause surfaces. Per-file failures are logged and the build
  continues; the finalize failure still propagates.
- Give the skill's node:sqlite connection an explicit PRAGMA busy_timeout=5000
  (it has no default). The app adds none: better-sqlite3 already defaults to
  5000ms, so busy_timeout is NOT the root-cause fix and is not treated as one.

Add tests/app-rollback-guard.test.mjs: injects a DB that faithfully reproduces
"a write auto-rolls back the txn, then ROLLBACK errors" and asserts the build
survives (bad file skipped, other file indexed). Revert-checked: without the
guard the test fails with the exact "cannot rollback - no transaction is active".

docs/adr/0006 records the real fix (shared runWriteTransaction, single-writer
coordination, BEGIN IMMEDIATE, whole-transaction retry, PASSIVE checkpointing)
as deferred, two-phase work — and why bumping busy_timeout is not it.

Verified: suite 124/124, typecheck clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
tommy0103
2026-07-10 16:36:36 +08:00
co-authored by Claude Opus 4.8
parent e44ab7a1da
commit 44029676d4
5 changed files with 193 additions and 4 deletions
@@ -0,0 +1,76 @@
# Write-transaction rollback safety and SQLite concurrency
**Context.** The app surfaced `Obelisk index build failed: cannot rollback - no
transaction is active`. That message is a *secondary* error: SQLite auto-rolls
back certain failures (notably `SQLITE_BUSY` / `SQLITE_BUSY_SNAPSHOT`, also disk
full), after which the per-file build loop's unguarded `db.exec('ROLLBACK')` in
its `catch` threw over the real error and aborted the whole build instead of
skipping just the offending file. The underlying trigger is concurrency: the app
runs a daemon indexer, manual rebuilds, and read queries against one WAL
database, and the skill's passive-pull build can write the same database from a
separate process.
`busy_timeout` is **not** the root-cause fix and must not be treated as one.
better-sqlite3's constructor already defaults `timeout` to 5000ms, so the app hit
`SQLITE_BUSY` *despite* a 5s wait — which points at `SQLITE_BUSY_SNAPSHOT` from
deferred (read-then-write) transactions, a snapshot conflict that `busy_timeout`
does not wait on. Only `BEGIN IMMEDIATE` plus whole-transaction retry addresses
that.
**Decision.** Split the work into a cheap stopgap now and a correctness/
concurrency fix later.
*Stopgap (done).* Both indexers use a guarded rollback (`safeRollback`): a
cleanup rollback swallows only its own error and never masks the primary one.
Per-file failures are logged and the build continues; the finalize failure still
propagates. The skill's connection (`node:sqlite`, which has **no** default busy
timeout) gets an explicit `PRAGMA busy_timeout = 5000`; the app adds no such
pragma because better-sqlite3 already defaults to 5000ms — adding it there was
redundant and removed, precisely so nobody reads it as "the fix".
*Planned full fix (deferred, two phases).*
Phase 1 — transaction semantics:
- A shared, binding-agnostic `runWriteTransaction(db, work)` (same injection
model as `persist`): BEGIN → work → COMMIT, guarded rollback on failure,
original exception always preserved. Both app and skill call it, so their
behaviour is identical.
- Per-file callers catch and continue; finalize does **not** swallow — a finalize
failure fails the build (the skill currently only warns; that changes).
- In-memory state such as `affectedSessionIds` is updated **only after** a
successful COMMIT (today the app adds the id before COMMIT, so a failed commit
can report a wrong affected set).
- Structured diagnostics: `phase` (begin/file-write/commit/finalize/checkpoint),
SQLite `code`, file path, whether rollback succeeded, whether a txn is still
active; surface the skipped-file count in the build result rather than only
`console.warn` (no silent coverage gaps).
Phase 2 — concurrency:
- A stable concurrency test against real Electron `better-sqlite3` (daemon +
rebuild + skill writer), not an injected fake BUSY. Note this cannot run under
standalone `node --test` (better-sqlite3 is Electron-ABI); it needs an
Electron-hosted harness. The fast injected-BUSY unit test is kept as a
lower-level guard for `runWriteTransaction`, alongside it.
- Serialize all index writes through a single writer: an in-process
`BuildCoordinator`, plus the existing cross-process daemon arbitration
(`__app_heartbeat__` markers) so the skill defers to a live daemon. Single
writer = both layers together.
- `BEGIN IMMEDIATE` to take the write lock up front and avoid
`SQLITE_BUSY_SNAPSHOT` on read-then-write.
- Bounded, short-backoff retry of the **whole** transaction (not the single
failed statement) on `SQLITE_BUSY` / `SQLITE_BUSY_SNAPSHOT`. This relies on the
per-file work being idempotent (upsert/replace + delete-session cascade), which
it is — an invariant the retry depends on.
- Stop forcing `wal_checkpoint(TRUNCATE)` after every build; prefer a `PASSIVE`
checkpoint on idle, with `TRUNCATE` reserved for maintenance/exit.
- Centralized connection configuration (explicit 5000ms for `node:sqlite` — a
real behaviour change; explicit for better-sqlite3 too, acknowledging its
default is already 5000ms).
**Consequences.** With only the stopgap in place, contention no longer crashes a
build, but a conflicting file is *skipped* (non-fatal) and its data is briefly
missing until the next build — acceptable as a stopgap because the guard keeps
the index consistent and self-healing. The true fix is tracked as the two-phase
plan above. A future contributor should not "fix" the concurrency by bumping or
re-adding `busy_timeout`; the direction is a shared transaction module, single
writer, `BEGIN IMMEDIATE`, and whole-transaction retry.