feat(core): add first-class Pi session indexing (#23)
Pi cannot be read as another linear JSONL stream. Its history is a tree with a durable leaf, orphan roots, branch summaries, and two compaction forms, so the active context is something the format states rather than something line order implies. The adapter keeps those semantics inside itself and projects the result into the existing canonical tables. Sessions are keyed by (normalized header cwd, header id) rather than by path, because Pi's --session-id lookup is project-local: two projects may reuse an id, while a move or an identical copy is still one session. Discovery covers both layouts Pi writes and fingerprints each file by mtime, ctime, size and inode, so a rewrite that preserves mtime is not read as unchanged. Abandoned branches are preserved rather than dropped. Visibility becomes three-state -- visible, inactive, hidden -- and helpers return only visible rows until includeInactive asks for the superseded path, labeling every row so a caller knows which it holds. Usage counts all three, because an abandoned call still spent tokens; message_count reports only the visible transcript. A committed MIT-licensed oracle transcribed from Pi 0.83.0 pins the context algorithms, and a fixed-seed differential runs 512 generated sessions against it on every test run. Schema changes are additive.
This commit is contained in:
SaladDay
@@ -9,7 +9,7 @@
|
||||
[](https://github.com/tommy0103/obelisk/releases)
|
||||
[](LICENSE)
|
||||
|
||||
Past Claude Code, Codex, and Kimi Code sessions -- queryable by your agent, browsable by you.
|
||||
Past Claude Code, Codex, Kimi Code, and Pi sessions -- queryable by your agent, browsable by you.
|
||||
|
||||
</div>
|
||||
|
||||
@@ -25,7 +25,7 @@ The agent writes JS queries, runs them locally, and answers in plain language.
|
||||
|
||||
**App side** — an Electron desktop app for humans to browse sessions, manage memories, view usage stats, and see weekly recap cards.
|
||||
|
||||
Both read from the same `~/.obelisk/obelisk.sqlite` database. The indexer reads Claude Code transcripts from `~/.claude/projects`, Codex transcripts from `~/.codex/sessions`, and Kimi Code sessions from `~/.kimi-code/sessions` (or `$KIMI_CODE_HOME/sessions`).
|
||||
Both read from the same `~/.obelisk/obelisk.sqlite` database. The indexer reads Claude Code transcripts from `~/.claude/projects`, Codex transcripts from `~/.codex/sessions`, Kimi Code sessions from `~/.kimi-code/sessions` (or `$KIMI_CODE_HOME/sessions`), and Pi sessions from `~/.pi/agent/sessions`.
|
||||
|
||||
## Multi-provider support
|
||||
|
||||
@@ -38,7 +38,20 @@ Kimi session directories become one Obelisk session each. Main and child-agent
|
||||
subagents tables. Undo/clear is handled as a full session replay, so retracted
|
||||
wire records do not remain in the index.
|
||||
|
||||
For live app refresh, Obelisk watches the roots declared by every registered provider, including `~/.claude/projects`, `~/.codex/sessions`, and `~/.kimi-code/sessions`. Codex's `session_index.jsonl` is used as lightweight title/update metadata during indexing, not as the message transcript source.
|
||||
Pi JSONL v1-v3 sessions are projected through the same provider contract. Pi's tree, branch summaries, compactions, durable leaf, retained checkpoint tail, custom messages, bash records, tool calls, token usage, and raw JSONL evidence stay inside the adapter; no Pi-specific database or renderer branch is needed. Active visibility follows Pi's own context rules: a retained tail replaces pre-compaction ancestors even when those physical entries still exist and bounds any later legacy compaction, while a legacy-only chain retains ancestors beginning at `firstKeptEntryId`. Missing parents form orphan branch roots, matching Pi's recovery behavior. Pi entries that the source explicitly superseded are stored as `inactive`: the app and normal agent queries omit them, while supported query helpers can include them with `includeInactive: true`. Display-suppressed or transport-only records remain `hidden` and are never returned by those helpers.
|
||||
|
||||
| Provider | Superseded-history support |
|
||||
| --- | --- |
|
||||
| Pi | Branch, leaf, and compaction state attests inactive history |
|
||||
| Kimi Code | Undo/clear can attest supersession; preservation is a follow-up |
|
||||
| Claude Code | The source does not attest rewind or current-leaf state |
|
||||
| Codex | Sessions have no branching semantics |
|
||||
|
||||
Because Pi's explicit `--session-id` is project-local, Obelisk combines the header ID with a deterministic hash of the normalized header `cwd`; this keeps the identity stable across file moves and v1-v3 migration while allowing two projects to use the same custom ID. Replacement and deletion replay is provenance-aware, so stale session snapshots are retracted atomically; compaction and branch-summary model usage is included in usage totals.
|
||||
|
||||
For live app refresh, Obelisk watches the roots declared by every registered provider, including `~/.claude/projects`, `~/.codex/sessions`, `~/.kimi-code/sessions`, and `~/.pi/agent/sessions`. Codex's `session_index.jsonl` is used as lightweight title/update metadata during indexing, not as the message transcript source.
|
||||
|
||||
Pi chooses its session directory in this order: `--session-dir`, `PI_CODING_AGENT_SESSION_DIR`, `sessionDir` in settings, then the default under `~/.pi/agent/sessions`. Obelisk automatically follows absolute or `~`-prefixed environment/global settings and the project setting for Obelisk's launch cwd; a relative project setting is resolved against that cwd. CLI-only roots, relative environment/global settings, and project settings from another launch cwd cannot be inferred safely, so select the resolved directory in Obelisk **Settings** instead of letting Obelisk guess.
|
||||
|
||||
## Skill: agent-first retrieval
|
||||
|
||||
@@ -155,12 +168,7 @@ npm ci
|
||||
npm run dev
|
||||
```
|
||||
|
||||
`electron-vite` starts the renderer dev server and launches Electron. On first
|
||||
run, Obelisk creates `~/.obelisk/obelisk.sqlite`, indexes the available Claude
|
||||
Code and Codex transcripts, and then watches them for changes. The default
|
||||
sources are `~/.claude/projects` and `~/.codex/sessions`; use **Settings** to
|
||||
point the app at different directories. On Windows, Obelisk also checks common
|
||||
WSL distributions for the Claude Code directory.
|
||||
`electron-vite` starts the renderer dev server and launches Electron. On first run, Obelisk creates `~/.obelisk/obelisk.sqlite`, indexes the available registered-provider transcripts, and then watches them for changes. The default sources include `~/.claude/projects`, `~/.codex/sessions`, `~/.kimi-code/sessions`, and `~/.pi/agent/sessions`; use **Settings** to point the app at different directories. On Windows, Obelisk also checks common WSL distributions for the Claude Code directory.
|
||||
|
||||
### Debug the app
|
||||
|
||||
@@ -184,7 +192,7 @@ run `npm ci` again.
|
||||
|
||||
| Layer | Source | What's captured |
|
||||
|-------|--------|----------------|
|
||||
| **Sessions** | Claude `<project>/<sessionId>.jsonl`; Codex `sessions/YYYY/MM/DD/*.jsonl` | Title, project, timestamps, git branch, source |
|
||||
| **Sessions** | Claude `<project>/<sessionId>.jsonl`; Codex `sessions/YYYY/MM/DD/*.jsonl`; Kimi session directories; Pi recursive `*.jsonl` | Title, project, timestamps, git branch, source |
|
||||
| **Messages** | user + assistant turns | Full text, model, token usage, parent chain |
|
||||
| **Tool calls** | every tool invocation | Tool name, input, file paths |
|
||||
| **Subagents** | Claude `subagents/agent-<id>.jsonl`; Codex child threads | Agent type, description, full conversation |
|
||||
@@ -203,7 +211,8 @@ packages/core/ # @obelisk/core npm workspace (TypeScript + ESM)
|
||||
│ │ ├── types.ts # Provider + TranscriptRecord contract
|
||||
│ │ ├── claude.ts # Claude Code adapter (line-incremental)
|
||||
│ │ ├── codex.ts # Codex adapter (full-reparse)
|
||||
│ │ └── kimi.ts # Kimi Code adapter (session projection)
|
||||
│ │ ├── kimi.ts # Kimi Code adapter (session projection)
|
||||
│ │ └── pi.ts # Pi adapter (tree-aware full-reparse)
|
||||
│ ├── session-detail.ts # Provider-independent transcript projection
|
||||
│ ├── persist.ts # Binding-agnostic record writer (upsert/merge)
|
||||
│ ├── tx.ts # Write transaction + connection config
|
||||
|
||||
Reference in New Issue
Block a user