fix(indexer): guard cleanup rollback so it never masks the real build error
Reported as "cannot rollback - no transaction is active" — a secondary error. SQLite auto-rolls back certain failures (SQLITE_BUSY / SQLITE_BUSY_SNAPSHOT, disk full), after which the per-file loop's unguarded ROLLBACK in its catch threw over the real error and aborted the whole build instead of skipping just the bad file. Stopgap only (see docs/adr/0006 for the planned full fix): - Add safeRollback(db) in both indexers: it swallows only the rollback's own error, so the true cause surfaces. Per-file failures are logged and the build continues; the finalize failure still propagates. - Give the skill's node:sqlite connection an explicit PRAGMA busy_timeout=5000 (it has no default). The app adds none: better-sqlite3 already defaults to 5000ms, so busy_timeout is NOT the root-cause fix and is not treated as one. Add tests/app-rollback-guard.test.mjs: injects a DB that faithfully reproduces "a write auto-rolls back the txn, then ROLLBACK errors" and asserts the build survives (bad file skipped, other file indexed). Revert-checked: without the guard the test fails with the exact "cannot rollback - no transaction is active". docs/adr/0006 records the real fix (shared runWriteTransaction, single-writer coordination, BEGIN IMMEDIATE, whole-transaction retry, PASSIVE checkpointing) as deferred, two-phase work — and why bumping busy_timeout is not it. Verified: suite 124/124, typecheck clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
e44ab7a1da
commit
44029676d4
@@ -0,0 +1,76 @@
|
||||
# Write-transaction rollback safety and SQLite concurrency
|
||||
|
||||
**Context.** The app surfaced `Obelisk index build failed: cannot rollback - no
|
||||
transaction is active`. That message is a *secondary* error: SQLite auto-rolls
|
||||
back certain failures (notably `SQLITE_BUSY` / `SQLITE_BUSY_SNAPSHOT`, also disk
|
||||
full), after which the per-file build loop's unguarded `db.exec('ROLLBACK')` in
|
||||
its `catch` threw over the real error and aborted the whole build instead of
|
||||
skipping just the offending file. The underlying trigger is concurrency: the app
|
||||
runs a daemon indexer, manual rebuilds, and read queries against one WAL
|
||||
database, and the skill's passive-pull build can write the same database from a
|
||||
separate process.
|
||||
|
||||
`busy_timeout` is **not** the root-cause fix and must not be treated as one.
|
||||
better-sqlite3's constructor already defaults `timeout` to 5000ms, so the app hit
|
||||
`SQLITE_BUSY` *despite* a 5s wait — which points at `SQLITE_BUSY_SNAPSHOT` from
|
||||
deferred (read-then-write) transactions, a snapshot conflict that `busy_timeout`
|
||||
does not wait on. Only `BEGIN IMMEDIATE` plus whole-transaction retry addresses
|
||||
that.
|
||||
|
||||
**Decision.** Split the work into a cheap stopgap now and a correctness/
|
||||
concurrency fix later.
|
||||
|
||||
*Stopgap (done).* Both indexers use a guarded rollback (`safeRollback`): a
|
||||
cleanup rollback swallows only its own error and never masks the primary one.
|
||||
Per-file failures are logged and the build continues; the finalize failure still
|
||||
propagates. The skill's connection (`node:sqlite`, which has **no** default busy
|
||||
timeout) gets an explicit `PRAGMA busy_timeout = 5000`; the app adds no such
|
||||
pragma because better-sqlite3 already defaults to 5000ms — adding it there was
|
||||
redundant and removed, precisely so nobody reads it as "the fix".
|
||||
|
||||
*Planned full fix (deferred, two phases).*
|
||||
|
||||
Phase 1 — transaction semantics:
|
||||
- A shared, binding-agnostic `runWriteTransaction(db, work)` (same injection
|
||||
model as `persist`): BEGIN → work → COMMIT, guarded rollback on failure,
|
||||
original exception always preserved. Both app and skill call it, so their
|
||||
behaviour is identical.
|
||||
- Per-file callers catch and continue; finalize does **not** swallow — a finalize
|
||||
failure fails the build (the skill currently only warns; that changes).
|
||||
- In-memory state such as `affectedSessionIds` is updated **only after** a
|
||||
successful COMMIT (today the app adds the id before COMMIT, so a failed commit
|
||||
can report a wrong affected set).
|
||||
- Structured diagnostics: `phase` (begin/file-write/commit/finalize/checkpoint),
|
||||
SQLite `code`, file path, whether rollback succeeded, whether a txn is still
|
||||
active; surface the skipped-file count in the build result rather than only
|
||||
`console.warn` (no silent coverage gaps).
|
||||
|
||||
Phase 2 — concurrency:
|
||||
- A stable concurrency test against real Electron `better-sqlite3` (daemon +
|
||||
rebuild + skill writer), not an injected fake BUSY. Note this cannot run under
|
||||
standalone `node --test` (better-sqlite3 is Electron-ABI); it needs an
|
||||
Electron-hosted harness. The fast injected-BUSY unit test is kept as a
|
||||
lower-level guard for `runWriteTransaction`, alongside it.
|
||||
- Serialize all index writes through a single writer: an in-process
|
||||
`BuildCoordinator`, plus the existing cross-process daemon arbitration
|
||||
(`__app_heartbeat__` markers) so the skill defers to a live daemon. Single
|
||||
writer = both layers together.
|
||||
- `BEGIN IMMEDIATE` to take the write lock up front and avoid
|
||||
`SQLITE_BUSY_SNAPSHOT` on read-then-write.
|
||||
- Bounded, short-backoff retry of the **whole** transaction (not the single
|
||||
failed statement) on `SQLITE_BUSY` / `SQLITE_BUSY_SNAPSHOT`. This relies on the
|
||||
per-file work being idempotent (upsert/replace + delete-session cascade), which
|
||||
it is — an invariant the retry depends on.
|
||||
- Stop forcing `wal_checkpoint(TRUNCATE)` after every build; prefer a `PASSIVE`
|
||||
checkpoint on idle, with `TRUNCATE` reserved for maintenance/exit.
|
||||
- Centralized connection configuration (explicit 5000ms for `node:sqlite` — a
|
||||
real behaviour change; explicit for better-sqlite3 too, acknowledging its
|
||||
default is already 5000ms).
|
||||
|
||||
**Consequences.** With only the stopgap in place, contention no longer crashes a
|
||||
build, but a conflicting file is *skipped* (non-fatal) and its data is briefly
|
||||
missing until the next build — acceptable as a stopgap because the guard keeps
|
||||
the index consistent and self-healing. The true fix is tracked as the two-phase
|
||||
plan above. A future contributor should not "fix" the concurrency by bumping or
|
||||
re-adding `busy_timeout`; the direction is a shared transaction module, single
|
||||
writer, `BEGIN IMMEDIATE`, and whole-transaction retry.
|
||||
Reference in New Issue
Block a user