feat(providers): migrate codex indexing to adapter + persist, remove legacy indexers (Phase 5c)
Complete the skill-side provider migration: codex now goes through a pure adapter
and the shared persist layer, and the two original monolithic indexers are gone.
New:
- scripts/providers/codex.ts — pure codex adapter. Full-reparse (buffers the whole
file) because the event_msg↔response_item dedup needs whole-file, bidirectional
knowledge; emits SessionRecord with countMode 'total'. Handles guardian threads
(→ delete-session), agent spawns/tool calls (→ tool_call/subagent), token_count
(patched onto the message record) and task_complete (→ message-turn-duration).
Contract:
- SessionRecord.countMode ('total' | 'delta') tells persist whether to replace or
accumulate message_count — claude is line-incremental (delta), codex full-reparse
(total). SubagentRecord non-key fields are optional; persist merges them
column-wise with COALESCE. MessageTurnDurationRecord.turn_duration_ms is nullable.
Orchestration:
- buildIndex's codex branch parses via the adapter and writes via persist. An
unchanged file is skipped but still swept for stale guardian rows (routed through
persist as a delete-session), preserving prior behavior.
Cleanup:
- Remove the now-unused indexJsonl, indexCodexJsonl, deleteCodexThreadRows and
upsertCodexSubagent — their semantics now live in the adapters + persist.
indexer.mjs drops from ~840 to 428 lines. Codex pure helpers stay exported for
codex.ts and the guardian sweep (physical move deferred to the app-side reorg).
- Migrate the upsert drift test off indexJsonl to the claude.parse + persist path,
keeping the rowid-stability and count-replace regression guards.
Tests: tests/codex-parse.test.mjs (record-stream golden: dedup, tools, token patch,
turn-duration, guardian→delete) and tests/codex-index.test.mjs (full buildIndex
path: fresh build + incremental full-reparse, total-count replace, no duplicates).
Verified equivalent on the real ~/.obelisk index: codex messages 82476 and
subagents 522 identical before/after, zero guardian leakage; real incremental
confirmed (touch a codex file → reparsed idempotently, unchanged files skipped).
lint + typecheck clean, 119/119.
This commit is contained in:
@@ -113,15 +113,18 @@ export interface SummaryRecord {
|
||||
content: string;
|
||||
}
|
||||
|
||||
// One codex subagent. Like workflow_agent, a row can be contributed by more than
|
||||
// one point in the parse (the spawn event vs the agent's own thread), so non-key
|
||||
// fields are optional and persist merges them column-wise with COALESCE.
|
||||
export interface SubagentRecord {
|
||||
kind: 'subagent';
|
||||
agent_id: string;
|
||||
session_id: string;
|
||||
parent_tool_use_id: string | null;
|
||||
agent_type: string | null;
|
||||
description: string | null;
|
||||
duration_ms: number | null;
|
||||
total_tokens: number | null;
|
||||
parent_tool_use_id?: string | null;
|
||||
agent_type?: string | null;
|
||||
description?: string | null;
|
||||
duration_ms?: number | null;
|
||||
total_tokens?: number | null;
|
||||
}
|
||||
|
||||
// A workflow run. `agent_count` is intentionally absent: it is a derived
|
||||
@@ -170,7 +173,7 @@ export interface WorkflowAgentRecord {
|
||||
export interface MessageTurnDurationRecord {
|
||||
kind: 'message-turn-duration';
|
||||
uuid: string;
|
||||
turn_duration_ms: number;
|
||||
turn_duration_ms: number | null;
|
||||
}
|
||||
|
||||
// Retraction op (not a table). The adapter emits this when a previously-indexed
|
||||
@@ -188,6 +191,12 @@ export interface DeleteSessionRecord {
|
||||
// fill-if-null (COALESCE) so those never clobber a value already present.
|
||||
// project_path is NOT set here — the orchestration's global pass derives it from
|
||||
// persisted message cwds (refreshSessionProjectPaths).
|
||||
//
|
||||
// countMode tells persist how to treat message_count, because providers differ:
|
||||
// a line-incremental adapter (claude) yields only new messages ('delta', persist
|
||||
// accumulates onto the existing row); a full-reparse adapter (codex) yields every
|
||||
// message each run ('total', persist replaces). A 'delta' parse from an empty
|
||||
// cursor is equivalent to 'total'.
|
||||
export interface SessionRecord {
|
||||
kind: 'session';
|
||||
id: string;
|
||||
@@ -198,6 +207,7 @@ export interface SessionRecord {
|
||||
git_branch: string | null;
|
||||
version: string | null;
|
||||
message_count: number;
|
||||
countMode: 'total' | 'delta';
|
||||
jsonl_path: string;
|
||||
source: string;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user