The inference command isn't an abstraction, it's a devtool — so it's named like one. `bun run cli pi` and `bun run cli opencode` are the same mental shape as the dev server or the browser: pick the tool, drive it. pi (clients/pi.ts, was agent.ts): runPi() gains resumable sessions — -s NAME maps to a stable session UUID + pinned project dir under e2e/.dev/pi-sessions/<name> (pi sessions are project-scoped, so the cwd must persist with them). Verified: seed a codeword in one process, recall it from another. opencode (clients/opencode.ts): the existing MCP-actor home machinery gains a headless one-shot/multi-turn runner. runOpenCode() drives the real binary hermetically (XDG home seeded with the machine credential), -s NAME pins a reusable home under e2e/.dev/opencode-homes/<name>, --continue resumes its last conversation, and the mcp option wires OpenCode's native MCP client at a target. makeOpenCodeHome learns an optional persistent root (existing config left alone on reattach). Verified: pong, codeword seed + --continue recall, both live. Division of labor stands: pi = everyday inference workhorse (~2s, no MCP by design), opencode = the MCP-native client-behavior actor.
Unified · Split
| @@ -76,7 +76,8 @@ bun run cli identity selfhost # fresh identity (headers / cookies / creds) · 75 unmodified lines | ||
| 76 | 76 | bun run cli api selfhost tools.list |
| 77 | 77 | bun run cli mcp selfhost call execute '{"code":"return 1+1;"}' |
| 78 | 78 | bun run cli ledger cloud workos # what hit the emulator |
| 79 | −bun run cli infer "..." # real model inference (see below) | |
| 79 | +bun run cli pi "..." # real model inference via pi (see below) | |
| 80 | +bun run cli opencode "..." # the same through the OpenCode binary | |
| 80 | 81 | bun run cli down selfhost # tear down (also removes tailscale serves) |
| 81 | 82 | ``` |
| 82 | 83 | |
| @@ -97,20 +98,33 @@ hostname (`__VITE_ADDITIONAL_SERVER_ALLOWED_HOSTS`). · 14 unmodified lines | ||
| 97 | 98 | |
| 98 | 99 | ## Real inference for development |
| 99 | 100 | |
| 100 | −`bun run cli infer [-m model] [--tools] [--json] "prompt"` runs a real model | |
| 101 | −through the **pi agent harness** (`@earendil-works/pi-coding-agent`, | |
| 102 | −headless `--mode json`), reached via the machine's OpenCode Zen subscription | |
| 103 | −(key auto-discovered from `~/.local/share/opencode/auth.json`; run | |
| 104 | −`opencode auth login` once if missing). Every run is hermetic: throwaway pi | |
| 105 | −config dir, no session persisted, and `--tools` confines the model's | |
| 106 | −read/bash/edit/write to a fresh scratch dir. Default model is cheap+fast | |
| 107 | −(deepseek-v4-flash); `-m glm-5.1` etc. selects others on the subscription. | |
| 108 | −`--json` prints the distilled result (answer, thinking, tool calls, usage, | |
| 109 | −full event stream) for programmatic use; the same primitive is importable as | |
| 110 | −`runAgent()` from `e2e/src/clients/agent.ts` for eval-style harnesses. | |
| 111 | −OpenCode (`e2e/src/clients/opencode.ts`) remains the MCP-native real-client | |
| 112 | −actor for scenarios that test OAuth/tool-discovery behavior; pi is the | |
| 113 | −inference workhorse. | |
| 101 | +The agent-tool commands are named after the actual tools — they're devtools, | |
| 102 | +same as the dev server or the browser. Both ride the machine's OpenCode | |
| 103 | +subscription (`~/.local/share/opencode/auth.json`; run `opencode auth login` | |
| 104 | +once if missing); both are hermetic (throwaway homes, nothing touches your | |
| 105 | +own pi/OpenCode state); both take `--json` for the distilled result (answer, | |
| 106 | +events, tool calls) for programmatic/grading use. | |
| 107 | + | |
| 108 | +`bun run cli pi [-m model] [-s session] [--tools] [--json] "prompt"` — the | |
| 109 | +**pi agent harness** (`@earendil-works/pi-coding-agent`, headless | |
| 110 | +`--mode json`), reaching the Zen gateway through a generated provider | |
| 111 | +extension. Fast (~2s), `--no-tools` by default; `--tools` confines pi's | |
| 112 | +read/bash/edit/write to a scratch dir. `-s NAME` makes the conversation | |
| 113 | +resumable (state under `e2e/.dev/pi-sessions/<name>`). Programmatic: | |
| 114 | +`runPi()` in `e2e/src/clients/pi.ts`. pi has no MCP by design — for | |
| 115 | +MCP-tool work, either drive the typed API/MCP through this CLI's other | |
| 116 | +commands, or use opencode below. | |
| 117 | + | |
| 118 | +`bun run cli opencode [-m provider/model] [-s session] [--continue] | |
| 119 | +[--json] "prompt"` — the same ask through the **real OpenCode binary** | |
| 120 | +(hermetic XDG home seeded with the credential). `-s NAME` pins a reusable | |
| 121 | +home under `e2e/.dev/opencode-homes/<name>`; `--continue` resumes its last | |
| 122 | +conversation. OpenCode is also the **MCP-native client**: programmatically, | |
| 123 | +`runOpenCode({ mcp: { serverName, url } })` configures a remote MCP server | |
| 124 | +(OAuth consent is the caller's job — see the `mcp-opencode-real` scenario | |
| 125 | +for the full dance). This is the actor to use when the thing under test is | |
| 126 | +client behavior (tool discovery, OAuth hops); pi is the everyday inference | |
| 127 | +workhorse. | |
| 114 | 128 | |
| 115 | 129 | ## Environment gotchas (learned the hard way) |
| 116 | 130 | |
| @@ -496,33 +496,71 @@ const logs = (targetName: string) => { · 495 unmodified lines | ||
| 496 | 496 | console.log(readFileSync(state.logFile, "utf8")); |
| 497 | 497 | }; |
| 498 | 498 | |
| 499 | −// Real inference through the pi agent harness (Zen subscription). Plain | |
| 500 | −// question→answer by default; --tools gives the model pi's coding tools in a | |
| 501 | −// scratch dir; --json dumps the full distilled result for grading. | |
| 502 | −const infer = async (args: ReadonlyArray<string>, flags: ReadonlySet<string>) => { | |
| 503 | − const { runAgent, DEFAULT_MODEL, hasInferenceCredential } = await import("../src/clients/agent"); | |
| 504 | − if (!hasInferenceCredential()) { | |
| 505 | − throw new Error( | |
| 506 | − "infer: no inference credential — run `opencode auth login` once on this machine", | |
| 507 | − ); | |
| 508 | − } | |
| 509 | − let model = DEFAULT_MODEL; | |
| 499 | +// Shared arg parsing for the agent-tool commands: -m model, -s session name, | |
| 500 | +// everything else is the prompt. | |
| 501 | +const parseAgentArgs = (args: ReadonlyArray<string>) => { | |
| 502 | + let model: string | undefined; | |
| 503 | + let session: string | undefined; | |
| 510 | 504 | const prompt: string[] = []; |
| 511 | 505 | for (let index = 0; index < args.length; index++) { |
| 512 | − if (args[index] === "-m") model = args[++index] ?? model; | |
| 506 | + if (args[index] === "-m") model = args[++index]; | |
| 507 | + else if (args[index] === "-s") session = args[++index]; | |
| 513 | 508 | else prompt.push(args[index]!); |
| 514 | 509 | } |
| 515 | − if (prompt.length === 0) throw new Error('usage: infer [-m model] [--tools] [--json] "prompt"'); | |
| 516 | − const result = await runAgent({ | |
| 517 | − prompt: prompt.join(" "), | |
| 510 | + return { model, session, prompt: prompt.join(" ") }; | |
| 511 | +}; | |
| 512 | + | |
| 513 | +// `bun run cli pi` — real inference through the pi agent harness (Zen | |
| 514 | +// subscription). Plain question→answer by default; --tools gives the model | |
| 515 | +// pi's coding tools in a scratch dir; -s NAME makes it multi-turn. | |
| 516 | +const piCmd = async (args: ReadonlyArray<string>, flags: ReadonlySet<string>) => { | |
| 517 | + const { runPi, hasInferenceCredential } = await import("../src/clients/pi"); | |
| 518 | + if (!hasInferenceCredential()) { | |
| 519 | + throw new Error("pi: no inference credential — run `opencode auth login` once on this machine"); | |
| 520 | + } | |
| 521 | + const { model, session, prompt } = parseAgentArgs(args); | |
| 522 | + if (!prompt) throw new Error('usage: pi [-m model] [-s session] [--tools] [--json] "prompt"'); | |
| 523 | + const result = await runPi({ prompt, model, session, tools: flags.has("--tools") }); | |
| 524 | + if (flags.has("--json")) { | |
| 525 | + console.log(JSON.stringify(result, null, 2)); | |
| 526 | + } else { | |
| 527 | + if (result.exitCode !== 0 && !result.answerText) { | |
| 528 | + throw new Error(`pi exited ${result.exitCode}\n${result.stderr.slice(0, 800)}`); | |
| 529 | + } | |
| 530 | + console.log(result.answerText); | |
| 531 | + } | |
| 532 | +}; | |
| 533 | + | |
| 534 | +// `bun run cli opencode` — the same ask through the real OpenCode binary | |
| 535 | +// (hermetic home, subscription credential seeded). --continue resumes the | |
| 536 | +// hermetic home's last session — pass -s NAME to pick which home. | |
| 537 | +const opencodeCmd = async (args: ReadonlyArray<string>, flags: ReadonlySet<string>) => { | |
| 538 | + const { runOpenCode, hasOpenCode, makeOpenCodeHome } = await import("../src/clients/opencode"); | |
| 539 | + if (!hasOpenCode()) throw new Error("opencode: binary not installed"); | |
| 540 | + const { model, session, prompt } = parseAgentArgs(args); | |
| 541 | + if (!prompt) { | |
| 542 | + throw new Error( | |
| 543 | + 'usage: opencode [-m provider/model] [-s session] [--continue] [--json] "prompt"', | |
| 544 | + ); | |
| 545 | + } | |
| 546 | + // A named "session" here = a reusable hermetic home under .dev, so a later | |
| 547 | + // invocation with the same -s NAME --continue finds the same conversation. | |
| 548 | + const home = session | |
| 549 | + ? makeOpenCodeHome("none", "http://127.0.0.1:9/", { | |
| 550 | + root: join(devDir, "opencode-homes", session), | |
| 551 | + }) | |
| 552 | + : undefined; | |
| 553 | + const result = await runOpenCode({ | |
| 554 | + prompt, | |
| 518 | 555 | model, |
| 519 | − tools: flags.has("--tools"), | |
| 556 | + home, | |
| 557 | + continueSession: flags.has("--continue"), | |
| 520 | 558 | }); |
| 521 | 559 | if (flags.has("--json")) { |
| 522 | − console.log(JSON.stringify(result, null, 2)); | |
| 560 | + console.log(JSON.stringify({ ...result, home: undefined }, null, 2)); | |
| 523 | 561 | } else { |
| 524 | 562 | if (result.exitCode !== 0 && !result.answerText) { |
| 525 | − throw new Error(`infer: pi exited ${result.exitCode}\n${result.stderr.slice(0, 800)}`); | |
| 563 | + throw new Error(`opencode exited ${result.exitCode}\n${result.stderr.slice(0, 800)}`); | |
| 526 | 564 | } |
| 527 | 565 | console.log(result.answerText); |
| 528 | 566 | } |
| @@ -540,9 +578,14 @@ const HELP = `e2e dev CLI — the scenario primitives, interactive (see e2e/AGEN · 11 unmodified lines | ||
| 540 | 578 | api <target> <group.endpoint> [json] typed API call as a fresh identity |
| 541 | 579 | mcp <target> tools | call <tool> [json] MCP session call |
| 542 | 580 | ledger <target> [workos|autumn] the emulator's request ledger (cloud) |
| 543 | − infer [-m model] [--tools] [--json] "prompt" real inference via the pi | |
| 544 | − agent harness (Zen subscription; --tools gives | |
| 545 | − the model coding tools in a scratch dir) | |
| 581 | + pi [-m model] [-s session] [--tools] [--json] "prompt" | |
| 582 | + real inference via the pi agent harness (Zen | |
| 583 | + subscription); --tools = coding tools in a | |
| 584 | + scratch dir; -s NAME = resumable conversation | |
| 585 | + opencode [-m provider/model] [-s session] [--continue] [--json] "prompt" | |
| 586 | + the same ask through the real OpenCode binary | |
| 587 | + (hermetic home; native MCP client available | |
| 588 | + programmatically via runOpenCode({mcp})) | |
| 546 | 589 | logs <target> dump the instance's dev-server log |
| 547 | 590 | down <target> tear down (kills servers, removes tailscale serves) |
| 548 | 591 | |
| @@ -569,8 +612,10 @@ const main = async () => { · 20 unmodified lines | ||
| 569 | 612 | return mcpCall(args[0] ?? "", args[1], args.slice(2)); |
| 570 | 613 | case "ledger": |
| 571 | 614 | return ledger(args[0] ?? "", args[1]); |
| 572 | − case "infer": | |
| 573 | − return infer(args, flags); | |
| 615 | + case "pi": | |
| 616 | + return piCmd(args, flags); | |
| 617 | + case "opencode": | |
| 618 | + return opencodeCmd(args, flags); | |
| 574 | 619 | case "logs": |
| 575 | 620 | return logs(args[0] ?? ""); |
| 576 | 621 | case "down": |
| @@ -3,9 +3,16 @@ · 2 unmodified lines | ||
| 3 | 3 | // /mcp, and an `open`(1) shim on PATH so the OAuth browser hop becomes a file |
| 4 | 4 | // we can read instead of a window. What OpenCode does with discovery, scopes, |
| 5 | 5 | // tokens, and refresh is entirely its own code — that is the point. |
| 6 | −import { spawnSync } from "node:child_process"; | |
| 7 | −import { existsSync, mkdirSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs"; | |
| 8 | −import { tmpdir } from "node:os"; | |
| 6 | +import { spawn, spawnSync } from "node:child_process"; | |
| 7 | +import { | |
| 8 | + copyFileSync, | |
| 9 | + existsSync, | |
| 10 | + mkdirSync, | |
| 11 | + mkdtempSync, | |
| 12 | + readFileSync, | |
| 13 | + writeFileSync, | |
| 14 | +} from "node:fs"; | |
| 15 | +import { homedir, tmpdir } from "node:os"; | |
| 9 | 16 | import { join } from "node:path"; |
| 10 | 17 | |
| 11 | 18 | /** Whether the real OpenCode binary is installed — the "opencode" capability. */ |
| @@ -30,42 +37,48 @@ export interface OpenCodeHome { · 18 unmodified lines | ||
| 30 | 37 | * pointing OpenCode's LLM traffic at a local replay brain |
| 31 | 38 | * (clients/replay-brain.ts) and selects it as the model — real agent, |
| 32 | 39 | * scripted conversation. Tool permissions are pre-allowed so the recorded |
| 33 | − * TUI session flows without approval dialogs. */ | |
| 40 | + * TUI session flows without approval dialogs. | |
| 41 | + * | |
| 42 | + * Pass `root` to make the home persistent/reattachable (multi-turn CLI use) | |
| 43 | + * instead of a fresh temp dir; an existing root's config is left alone. */ | |
| 34 | 44 | export const makeOpenCodeHome = ( |
| 35 | 45 | serverName: string, |
| 36 | 46 | mcpUrl: string, |
| 37 | − options?: { readonly chatBrainUrl?: string }, | |
| 47 | + options?: { readonly chatBrainUrl?: string; readonly root?: string }, | |
| 38 | 48 | ): OpenCodeHome => { |
| 39 | − const root = mkdtempSync(join(tmpdir(), "e2e-opencode-")); | |
| 49 | + const root = options?.root ?? mkdtempSync(join(tmpdir(), "e2e-opencode-")); | |
| 40 | 50 | const projectDir = join(root, "project"); |
| 41 | 51 | const dataDir = join(root, "data"); |
| 42 | 52 | const binDir = join(root, "bin"); |
| 43 | 53 | const openedUrlsFile = join(root, "opened-urls.txt"); |
| 54 | + const fresh = !existsSync(projectDir); | |
| 44 | 55 | for (const dir of [projectDir, dataDir, binDir]) mkdirSync(dir, { recursive: true }); |
| 45 | 56 | |
| 46 | − writeFileSync( | |
| 47 | − join(projectDir, "opencode.json"), | |
| 48 | − JSON.stringify({ | |
| 49 | − $schema: "https://opencode.ai/config.json", | |
| 50 | − mcp: { [serverName]: { type: "remote", url: mcpUrl } }, | |
| 51 | − ...(options?.chatBrainUrl | |
| 52 | − ? { | |
| 53 | − autoupdate: false, | |
| 54 | − share: "disabled", | |
| 55 | − model: "replay/replay-model", | |
| 56 | − permission: { "*": "allow" }, | |
| 57 | − provider: { | |
| 58 | − replay: { | |
| 59 | − name: "Replay", | |
| 60 | − npm: "@ai-sdk/openai-compatible", | |
| 61 | − options: { baseURL: options.chatBrainUrl, apiKey: "replay-key" }, | |
| 62 | − models: { "replay-model": { name: "Replay Model" } }, | |
| 57 | + if (fresh) { | |
| 58 | + writeFileSync( | |
| 59 | + join(projectDir, "opencode.json"), | |
| 60 | + JSON.stringify({ | |
| 61 | + $schema: "https://opencode.ai/config.json", | |
| 62 | + mcp: { [serverName]: { type: "remote", url: mcpUrl } }, | |
| 63 | + ...(options?.chatBrainUrl | |
| 64 | + ? { | |
| 65 | + autoupdate: false, | |
| 66 | + share: "disabled", | |
| 67 | + model: "replay/replay-model", | |
| 68 | + permission: { "*": "allow" }, | |
| 69 | + provider: { | |
| 70 | + replay: { | |
| 71 | + name: "Replay", | |
| 72 | + npm: "@ai-sdk/openai-compatible", | |
| 73 | + options: { baseURL: options.chatBrainUrl, apiKey: "replay-key" }, | |
| 74 | + models: { "replay-model": { name: "Replay Model" } }, | |
| 75 | + }, | |
| 63 | 76 | }, |
| 64 | − }, | |
| 65 | − } | |
| 66 | − : {}), | |
| 67 | − }), | |
| 68 | − ); | |
| 77 | + } | |
| 78 | + : {}), | |
| 79 | + }), | |
| 80 | + ); | |
| 81 | + } | |
| 69 | 82 | // OpenCode launches the OAuth URL via `open`; the shim records it instead. |
| 70 | 83 | writeFileSync(join(binDir, "open"), `#!/bin/sh\necho "$@" >> ${openedUrlsFile}\nexit 0\n`, { |
| 71 | 84 | mode: 0o755, |
| @@ -134,3 +147,112 @@ export const completeOAuthConsent = async ( · 62 unmodified lines | ||
| 134 | 147 | } |
| 135 | 148 | throw new Error("opencode never opened an authorization URL"); |
| 136 | 149 | }; |
| 150 | + | |
| 151 | +// --------------------------------------------------------------------------- | |
| 152 | +// One-shot / multi-turn headless runs (`bun run cli opencode`) | |
| 153 | +// --------------------------------------------------------------------------- | |
| 154 | + | |
| 155 | +const hostAuthFile = (): string => join(homedir(), ".local", "share", "opencode", "auth.json"); | |
| 156 | + | |
| 157 | +export const hasOpenCodeCredential = (): boolean => existsSync(hostAuthFile()); | |
| 158 | + | |
| 159 | +export interface OpenCodeRunOptions { | |
| 160 | + readonly prompt: string; | |
| 161 | + /** `provider/model`, e.g. `opencode/deepseek-v4-flash` (the default). */ | |
| 162 | + readonly model?: string; | |
| 163 | + /** Continue the home's most recent session (multi-turn). */ | |
| 164 | + readonly continueSession?: boolean; | |
| 165 | + /** Reuse a prior run's home (REQUIRED for continueSession to see it). */ | |
| 166 | + readonly home?: OpenCodeHome; | |
| 167 | + /** Expose one remote MCP server to the model (OpenCode's native client). | |
| 168 | + * OAuth, if the server needs it, is the caller's job (mcp auth + consent | |
| 169 | + * — see the mcp-opencode-real scenario). */ | |
| 170 | + readonly mcp?: { readonly serverName: string; readonly url: string }; | |
| 171 | + readonly timeoutMs?: number; | |
| 172 | +} | |
| 173 | + | |
| 174 | +export interface OpenCodeRunResult { | |
| 175 | + readonly answerText: string; | |
| 176 | + /** Raw JSONL events (--format json), parsed, for grading/artifacts. */ | |
| 177 | + readonly events: readonly Record<string, unknown>[]; | |
| 178 | + readonly home: OpenCodeHome; | |
| 179 | + readonly exitCode: number | null; | |
| 180 | + readonly durationMs: number; | |
| 181 | + readonly stderr: string; | |
| 182 | +} | |
| 183 | + | |
| 184 | +/** Drive the real OpenCode binary headless and hermetic: its own XDG home | |
| 185 | + * seeded with the machine's subscription credential. Pass `home` + | |
| 186 | + * `continueSession` for multi-turn; pass `mcp` to use OpenCode's native MCP | |
| 187 | + * client against one of our targets. */ | |
| 188 | +export const runOpenCode = async (options: OpenCodeRunOptions): Promise<OpenCodeRunResult> => { | |
| 189 | + if (!hasOpenCodeCredential()) { | |
| 190 | + throw new Error( | |
| 191 | + `opencode: no credential at ${hostAuthFile()} — run \`opencode auth login\` once on this machine`, | |
| 192 | + ); | |
| 193 | + } | |
| 194 | + const PLACEHOLDER_MCP = "http://127.0.0.1:9/"; | |
| 195 | + const home = | |
| 196 | + options.home ?? | |
| 197 | + makeOpenCodeHome(options.mcp?.serverName ?? "none", options.mcp?.url ?? PLACEHOLDER_MCP); | |
| 198 | + const configPath = join(home.projectDir, "opencode.json"); | |
| 199 | + if (!options.mcp && readFileSync(configPath, "utf8").includes(PLACEHOLDER_MCP)) { | |
| 200 | + // makeOpenCodeHome always writes an mcp block; a plain run wants none. | |
| 201 | + writeFileSync( | |
| 202 | + configPath, | |
| 203 | + JSON.stringify({ $schema: "https://opencode.ai/config.json", share: "disabled" }), | |
| 204 | + ); | |
| 205 | + } | |
| 206 | + // Seed the subscription credential into the hermetic home. | |
| 207 | + const authDir = join(home.env.XDG_DATA_HOME ?? "", "opencode"); | |
| 208 | + mkdirSync(authDir, { recursive: true }); | |
| 209 | + copyFileSync(hostAuthFile(), join(authDir, "auth.json")); | |
| 210 | + | |
| 211 | + const args = [ | |
| 212 | + "run", | |
| 213 | + "-m", | |
| 214 | + options.model ?? "opencode/deepseek-v4-flash", | |
| 215 | + "--format", | |
| 216 | + "json", | |
| 217 | + ...(options.continueSession ? ["--continue"] : []), | |
| 218 | + options.prompt, | |
| 219 | + ]; | |
| 220 | + const startedAt = Date.now(); | |
| 221 | + const child = spawn("opencode", args, { | |
| 222 | + cwd: home.projectDir, | |
| 223 | + env: { ...home.env, PWD: home.projectDir }, | |
| 224 | + stdio: ["ignore", "pipe", "pipe"], | |
| 225 | + }); | |
| 226 | + let stdout = ""; | |
| 227 | + let stderr = ""; | |
| 228 | + child.stdout.on("data", (chunk: Buffer) => (stdout += chunk.toString("utf8"))); | |
| 229 | + child.stderr.on("data", (chunk: Buffer) => (stderr += chunk.toString("utf8"))); | |
| 230 | + const exitCode = await new Promise<number | null>((resolve) => { | |
| 231 | + const killer = setTimeout(() => child.kill("SIGKILL"), options.timeoutMs ?? 240_000); | |
| 232 | + child.once("exit", (code) => { | |
| 233 | + clearTimeout(killer); | |
| 234 | + resolve(code); | |
| 235 | + }); | |
| 236 | + }); | |
| 237 | + | |
| 238 | + const events: Record<string, unknown>[] = []; | |
| 239 | + for (const line of stdout.split("\n")) { | |
| 240 | + if (!line.trim()) continue; | |
| 241 | + // oxlint-disable-next-line executor/no-try-catch-or-throw, executor/no-json-parse -- boundary: tolerant parse of opencode's JSONL stream | |
| 242 | + try { | |
| 243 | + events.push(JSON.parse(line) as Record<string, unknown>); | |
| 244 | + } catch { | |
| 245 | + // Non-JSON line — keep going. | |
| 246 | + } | |
| 247 | + } | |
| 248 | + const answerText = events | |
| 249 | + .filter( | |
| 250 | + (event) => | |
| 251 | + event.type === "text" && | |
| 252 | + typeof (event.part as { text?: string } | undefined)?.text === "string", | |
| 253 | + ) | |
| 254 | + .map((event) => (event.part as { text: string }).text) | |
| 255 | + .join("\n"); | |
| 256 | + | |
| 257 | + return { answerText, events, home, exitCode, durationMs: Date.now() - startedAt, stderr }; | |
| 258 | +}; | |
| @@ -7,17 +7,19 @@ · 6 unmodified lines | ||
| 7 | 7 | // credentials and without touching anyone's pi/OpenCode state. |
| 8 | 8 | // |
| 9 | 9 | // Consumers: |
| 10 | −// - `bun run cli infer "..."` — the interactive command (scripts/cli.ts). | |
| 11 | −// - future eval tiers — fan out runAgent() trials and grade the results. | |
| 10 | +// - `bun run cli pi "..."` — the interactive command (scripts/cli.ts). | |
| 11 | +// - future eval tiers — fan out runPi() trials and grade the results. | |
| 12 | 12 | // |
| 13 | 13 | // This is deliberately NOT the OpenCode binary: pi's headless mode is built |
| 14 | 14 | // for this (clean JSONL events, --no-tools, hermetic config dir via env), |
| 15 | 15 | // while OpenCode stays what it already is in this suite — the MCP-native |
| 16 | 16 | // real-client actor (src/clients/opencode.ts). |
| 17 | 17 | import { spawn } from "node:child_process"; |
| 18 | +import { createHash } from "node:crypto"; | |
| 18 | 19 | import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs"; |
| 19 | 20 | import { homedir, tmpdir } from "node:os"; |
| 20 | 21 | import { join } from "node:path"; |
| 22 | +import { fileURLToPath } from "node:url"; | |
| 21 | 23 | |
| 22 | 24 | // --------------------------------------------------------------------------- |
| 23 | 25 | // The subscription credential (OpenCode Zen — an OpenAI-compatible gateway) |
| @@ -57,19 +59,19 @@ export const DEFAULT_MODEL: string = ZEN_MODELS[0]; · 33 unmodified lines | ||
| 57 | 59 | // Result shape — distilled from pi's JSONL event stream |
| 58 | 60 | // --------------------------------------------------------------------------- |
| 59 | 61 | |
| 60 | −export interface AgentToolCall { | |
| 62 | +export interface PiToolCall { | |
| 61 | 63 | readonly name: string; |
| 62 | 64 | readonly args: unknown; |
| 63 | 65 | readonly result?: unknown; |
| 64 | 66 | readonly isError?: boolean; |
| 65 | 67 | } |
| 66 | 68 | |
| 67 | −export interface AgentRunResult { | |
| 69 | +export interface PiRunResult { | |
| 68 | 70 | /** All assistant text parts of the final answer, joined. */ |
| 69 | 71 | readonly answerText: string; |
| 70 | 72 | /** The model's reasoning text, when the model emits it. */ |
| 71 | 73 | readonly thinkingText: string; |
| 72 | − readonly toolCalls: readonly AgentToolCall[]; | |
| 74 | + readonly toolCalls: readonly PiToolCall[]; | |
| 73 | 75 | /** Every JSONL event pi emitted, parsed, in order — for grading/artifacts. */ |
| 74 | 76 | readonly events: readonly Record<string, unknown>[]; |
| 75 | 77 | readonly usage?: { readonly input: number; readonly output: number }; |
| @@ -79,7 +81,7 @@ export interface AgentRunResult { · 3 unmodified lines | ||
| 79 | 81 | readonly stderr: string; |
| 80 | 82 | } |
| 81 | 83 | |
| 82 | −export interface AgentRunOptions { | |
| 84 | +export interface PiRunOptions { | |
| 83 | 85 | readonly prompt: string; |
| 84 | 86 | /** Zen model id (see ZEN_MODELS). Default: cheap + fast. */ |
| 85 | 87 | readonly model?: string; |
| @@ -91,11 +93,25 @@ export interface AgentRunOptions { · 5 unmodified lines | ||
| 91 | 93 | readonly cwd?: string; |
| 92 | 94 | /** Replace pi's coding system prompt. */ |
| 93 | 95 | readonly systemPrompt?: string; |
| 96 | + /** Resume/create a named persistent session (multi-turn). Sessions live in | |
| 97 | + * `sessionDir` (default e2e/.dev/pi-sessions), so a follow-up call with | |
| 98 | + * the same name continues the conversation. Omit for a one-shot. */ | |
| 99 | + readonly session?: string; | |
| 100 | + readonly sessionDir?: string; | |
| 94 | 101 | readonly timeoutMs?: number; |
| 95 | 102 | } |
| 96 | 103 | |
| 97 | 104 | const DEFAULT_TIMEOUT_MS = 120_000; |
| 98 | 105 | |
| 106 | +const defaultSessionDir = (): string => | |
| 107 | + fileURLToPath(new URL("../../.dev/pi-sessions/", import.meta.url)); | |
| 108 | + | |
| 109 | +/** pi wants a UUID session id; derive one stably from the friendly name. */ | |
| 110 | +const sessionUuid = (name: string): string => { | |
| 111 | + const hex = createHash("sha256").update(name).digest("hex"); | |
| 112 | + return `${hex.slice(0, 8)}-${hex.slice(8, 12)}-4${hex.slice(13, 16)}-8${hex.slice(17, 20)}-${hex.slice(20, 32)}`; | |
| 113 | +}; | |
| 114 | + | |
| 99 | 115 | // --------------------------------------------------------------------------- |
| 100 | 116 | // Run |
| 101 | 117 | // --------------------------------------------------------------------------- |
| @@ -121,7 +137,7 @@ const providerExtension = (modelIds: readonly string[]): string => `export defau · 19 unmodified lines | ||
| 121 | 137 | } |
| 122 | 138 | `; |
| 123 | 139 | |
| 124 | −export const runAgent = async (options: AgentRunOptions): Promise<AgentRunResult> => { | |
| 140 | +export const runPi = async (options: PiRunOptions): Promise<PiRunResult> => { | |
| 125 | 141 | const key = zenApiKey(); |
| 126 | 142 | if (!key) { |
| 127 | 143 | throw new Error( |
| @@ -142,7 +158,23 @@ export const runAgent = async (options: AgentRunOptions): Promise<AgentRunResult · 14 unmodified lines | ||
| 142 | 158 | ? ZEN_MODELS |
| 143 | 159 | : [...ZEN_MODELS, model]; |
| 144 | 160 | writeFileSync(extensionPath, providerExtension(modelIds)); |
| 145 | − const cwd = options.cwd ?? join(scratch, "project"); | |
| 161 | + | |
| 162 | + // Named sessions persist (and resume) under sessionDir; one-shots don't. | |
| 163 | + // pi session ids are PROJECT-scoped, so a named session also pins its cwd — | |
| 164 | + // otherwise every call's fresh scratch dir would start a new conversation. | |
| 165 | + const sessionRoot = options.session | |
| 166 | + ? join(options.sessionDir ?? defaultSessionDir(), options.session) | |
| 167 | + : undefined; | |
| 168 | + const sessionArgs = sessionRoot | |
| 169 | + ? [ | |
| 170 | + "--session-id", | |
| 171 | + sessionUuid(options.session!), | |
| 172 | + "--session-dir", | |
| 173 | + join(sessionRoot, "sessions"), | |
| 174 | + ] | |
| 175 | + : ["--no-session"]; | |
| 176 | + const cwd = | |
| 177 | + options.cwd ?? (sessionRoot ? join(sessionRoot, "project") : join(scratch, "project")); | |
| 146 | 178 | mkdirSync(cwd, { recursive: true }); |
| 147 | 179 | |
| 148 | 180 | const args = [ |
| @@ -152,7 +184,7 @@ export const runAgent = async (options: AgentRunOptions): Promise<AgentRunResult · 3 unmodified lines | ||
| 152 | 184 | "zen", |
| 153 | 185 | "--model", |
| 154 | 186 | model, |
| 155 | − "--no-session", | |
| 187 | + ...sessionArgs, | |
| 156 | 188 | "--mode", |
| 157 | 189 | "json", |
| 158 | 190 | ...(options.tools ? [] : ["--no-tools"]), |
| @@ -230,7 +262,7 @@ interface AgentEndEvent { · 71 unmodified lines | ||
| 230 | 262 | |
| 231 | 263 | const distill = ( |
| 232 | 264 | jsonl: string, |
| 233 | −): Pick<AgentRunResult, "answerText" | "thinkingText" | "toolCalls" | "events" | "usage"> => { | |
| 265 | +): Pick<PiRunResult, "answerText" | "thinkingText" | "toolCalls" | "events" | "usage"> => { | |
| 234 | 266 | const events: Record<string, unknown>[] = []; |
| 235 | 267 | for (const line of jsonl.split("\n")) { |
| 236 | 268 | if (!line.trim()) continue; |
| @@ -242,10 +274,10 @@ const distill = ( · 5 unmodified lines | ||
| 242 | 274 | } |
| 243 | 275 | } |
| 244 | 276 | |
| 245 | − const toolCalls = new Map<string, AgentToolCall>(); | |
| 277 | + const toolCalls = new Map<string, PiToolCall>(); | |
| 246 | 278 | let answerText = ""; |
| 247 | 279 | let thinkingText = ""; |
| 248 | − let usage: AgentRunResult["usage"]; | |
| 280 | + let usage: PiRunResult["usage"]; | |
| 249 | 281 | |
| 250 | 282 | for (const event of events as readonly AgentEndEvent[]) { |
| 251 | 283 | if (event.type === "tool_execution_start" && event.toolCallId) { |