feat(providers): migrate codex indexing to adapter + persist, remove legacy indexers (Phase 5c)

Complete the skill-side provider migration: codex now goes through a pure adapter
and the shared persist layer, and the two original monolithic indexers are gone.

New:
- scripts/providers/codex.ts — pure codex adapter. Full-reparse (buffers the whole
  file) because the event_msg↔response_item dedup needs whole-file, bidirectional
  knowledge; emits SessionRecord with countMode 'total'. Handles guardian threads
  (→ delete-session), agent spawns/tool calls (→ tool_call/subagent), token_count
  (patched onto the message record) and task_complete (→ message-turn-duration).

Contract:
- SessionRecord.countMode ('total' | 'delta') tells persist whether to replace or
  accumulate message_count — claude is line-incremental (delta), codex full-reparse
  (total). SubagentRecord non-key fields are optional; persist merges them
  column-wise with COALESCE. MessageTurnDurationRecord.turn_duration_ms is nullable.

Orchestration:
- buildIndex's codex branch parses via the adapter and writes via persist. An
  unchanged file is skipped but still swept for stale guardian rows (routed through
  persist as a delete-session), preserving prior behavior.

Cleanup:
- Remove the now-unused indexJsonl, indexCodexJsonl, deleteCodexThreadRows and
  upsertCodexSubagent — their semantics now live in the adapters + persist.
  indexer.mjs drops from ~840 to 428 lines. Codex pure helpers stay exported for
  codex.ts and the guardian sweep (physical move deferred to the app-side reorg).
- Migrate the upsert drift test off indexJsonl to the claude.parse + persist path,
  keeping the rowid-stability and count-replace regression guards.

Tests: tests/codex-parse.test.mjs (record-stream golden: dedup, tools, token patch,
turn-duration, guardian→delete) and tests/codex-index.test.mjs (full buildIndex
path: fresh build + incremental full-reparse, total-count replace, no duplicates).

Verified equivalent on the real ~/.obelisk index: codex messages 82476 and
subagents 522 identical before/after, zero guardian leakage; real incremental
confirmed (touch a codex file → reparsed idempotently, unchanged files skipped).
lint + typecheck clean, 119/119.
This commit is contained in:
tommy0103
2026-07-08 20:43:54 +08:00
parent 1c346039f1
commit 0598c29aad
8 changed files with 474 additions and 469 deletions
+16 -6
View File
@@ -113,15 +113,18 @@ export interface SummaryRecord {
content: string;
}
// One codex subagent. Like workflow_agent, a row can be contributed by more than
// one point in the parse (the spawn event vs the agent's own thread), so non-key
// fields are optional and persist merges them column-wise with COALESCE.
export interface SubagentRecord {
kind: 'subagent';
agent_id: string;
session_id: string;
parent_tool_use_id: string | null;
agent_type: string | null;
description: string | null;
duration_ms: number | null;
total_tokens: number | null;
parent_tool_use_id?: string | null;
agent_type?: string | null;
description?: string | null;
duration_ms?: number | null;
total_tokens?: number | null;
}
// A workflow run. `agent_count` is intentionally absent: it is a derived
@@ -170,7 +173,7 @@ export interface WorkflowAgentRecord {
export interface MessageTurnDurationRecord {
kind: 'message-turn-duration';
uuid: string;
turn_duration_ms: number;
turn_duration_ms: number | null;
}
// Retraction op (not a table). The adapter emits this when a previously-indexed
@@ -188,6 +191,12 @@ export interface DeleteSessionRecord {
// fill-if-null (COALESCE) so those never clobber a value already present.
// project_path is NOT set here — the orchestration's global pass derives it from
// persisted message cwds (refreshSessionProjectPaths).
//
// countMode tells persist how to treat message_count, because providers differ:
// a line-incremental adapter (claude) yields only new messages ('delta', persist
// accumulates onto the existing row); a full-reparse adapter (codex) yields every
// message each run ('total', persist replaces). A 'delta' parse from an empty
// cursor is equivalent to 'total'.
export interface SessionRecord {
kind: 'session';
id: string;
@@ -198,6 +207,7 @@ export interface SessionRecord {
git_branch: string | null;
version: string | null;
message_count: number;
countMode: 'total' | 'delta';
jsonl_path: string;
source: string;
}