| @@ -1,488 +1,429 @@ |
| 1 | | −What is executor: |
| 2 | | − |
| 3 | | −- Approvals |
| 4 | | −- Permission model |
| 5 | | −- Code execution |
| 6 | | − - Support abitrary tool targets (API / GraphQL / MCP / in memory) |
| 7 | | −- Authentication |
| 8 | | − |
| 9 | | −Usage locally: |
| 10 | | − |
| 11 | | −- Approvals are stored in memory |
| 12 | | −- Permissions are stored to file system |
| 13 | | −- Auth creds are stored to file system |
| 14 | | − |
| 15 | | −Usage via AI SDK |
| 16 | | − |
| 17 | | −const executor = new Executor({ |
| 18 | | −runtime: deno() | nodevm() | cloudflare-dynamic-worker(), |
| 19 | | −// all the same RPC? |
| 20 | | −approvals: |
| 21 | | −authentication: |
| 22 | | −permissions: |
| 23 | | −}) |
| 24 | | − |
| 25 | | −Usage via Cloud: |
| 26 | | − |
| 27 | | −- Everything in Convex |
| 28 | | −- Runtime is cloudflare dynamic worker |
| 29 | | −- |
| 30 | | − |
| 31 | 1 | # Executor v2 Plan |
| 32 | 2 | |
| 33 | 3 | Reference implementations and inspiration for this plan: |
| 34 | 4 | `../AnswerOverflow` |
| 35 | 5 | `../quickhub` |
| 36 | 6 | `../create-epoch-app` |
| 37 | | − |
| 38 | 7 | ## 1) Why v2 exists |
| 39 | 8 | |
| 40 | 9 | Executor today is optimized for enterprise/team correctness (workspaces, org flows, cloud control), but has too much friction for per-user adoption. |
| 41 | 10 | |
| 42 | 11 | v2 changes the default: |
| 43 | 12 | |
| 44 | | −- **First-run must be local and immediate** (`npx executor <agent>` works fast). |
| 45 | | −- **No database dependency** in local-lite mode. |
| 46 | | −- **Cloud/self-host are upgrade targets**, not prerequisites. |
| 47 | | −- **One architecture for all modes** via adapters and shared contracts. |
| 13 | +- First run is local and immediate (`npx executor <agent>` works fast). |
| 14 | +- No database dependency in local-lite mode. |
| 15 | +- Cloud/self-host are upgrade targets, not prerequisites. |
| 16 | +- One architecture for local and remote via shared contracts and thin adapters. |
| 48 | 17 | |
| 49 | 18 | --- |
| 50 | 19 | |
| 51 | 20 | ## 2) Product principles |
| 52 | 21 | |
| 53 | | −1. **Local-first** |
| 22 | +1. Local-first |
| 54 | 23 | - User gets value without login, org setup, or web onboarding. |
| 55 | | −2. **One mental model** |
| 56 | | − - Same commands, same execution semantics across local and remote targets. |
| 57 | | −3. **Schema-first** |
| 58 | | − - Effect Schema is the single source of truth for persisted/domain models. |
| 59 | | −4. **Execute-first tooling** |
| 60 | | − - Top-level model tool surface is minimal; code execution is primary. |
| 61 | | −5. **Progressive complexity** |
| 62 | | − - Advanced controls exist but are hidden until needed. |
| 63 | | −6. **Service-first Effect architecture** |
| 64 | | − - Effectful orchestration is exposed as `Context.Tag` services + Layers; plain functions are for pure transforms. |
| 24 | +2. One mental model |
| 25 | + - Same execution semantics across local and remote. |
| 26 | +3. Schema-first |
| 27 | + - Effect Schema is canonical for domain and persistence contracts. |
| 28 | +4. Execute-first |
| 29 | + - Keep top-level model-facing surface minimal (`execute` first). |
| 30 | +5. Service-first architecture |
| 31 | + - Effect orchestration is exposed via services and layers. |
| 32 | +6. Progressive complexity |
| 33 | + - Advanced control and infra stay out of the default happy path. |
| 65 | 34 | |
| 66 | 35 | --- |
| 67 | 36 | |
| 68 | | −## 3) Product modes and targets |
| 69 | | − |
| 70 | | −### Modes |
| 71 | | − |
| 72 | | −- **local-lite**: PM daemon + in-memory hot state + file persistence. No DB. |
| 73 | | −- **remote**: Convex-backed target (applies to both cloud and self-hosted). |
| 74 | | − |
| 75 | | −### Key simplification |
| 37 | +## 3) The architecture (locked) |
| 76 | 38 | |
| 77 | | −- `cloud` and `self-hosted` are not separate architecture branches. |
| 78 | | −- They are both **remote Convex targets**; only URL/auth/deployment ownership differs. |
| 39 | +v2 is split into three planes. This is the key simplification. |
| 79 | 40 | |
| 80 | | −### Runtime targets (adapter-based) |
| 41 | +### Execution plane |
| 81 | 42 | |
| 82 | | −- **`deno-subprocess`**: local sandbox runtime (current local secure target in v2). |
| 83 | | −- **`cloudflare-worker-loader`**: production cloud runtime target. |
| 84 | | −- **`node:vm`**: explicit unsafe/dev fallback only, not default. |
| 43 | +Runs code and resolves `tools.*` calls. |
| 85 | 44 | |
| 86 | | −--- |
| 87 | | − |
| 88 | | −## 4) Control + execution model |
| 45 | +- run lifecycle |
| 46 | +- runtime dispatch |
| 47 | +- tool invocation pipeline |
| 48 | +- approvals/policy enforcement |
| 49 | +- credential resolution for tool calls |
| 89 | 50 | |
| 90 | | −### Control role (conceptual) |
| 51 | +### Control plane |
| 91 | 52 | |
| 92 | | −Control includes source registry, policies, approvals/input workflow, identity/link state, and sync metadata. |
| 53 | +Mutates and queries executor configuration/state. |
| 93 | 54 | |
| 94 | | −- In local-lite: handled by `apps/pm`. |
| 95 | | −- In remote: handled by Convex + web stack. |
| 55 | +- source registration/removal |
| 56 | +- tool artifact refresh/rebuild |
| 57 | +- policy management |
| 58 | +- sync metadata |
| 96 | 59 | |
| 97 | | −### Execution role |
| 60 | +Control is optional capability, not required for execute-only environments. |
| 98 | 61 | |
| 99 | | −Execution is provider-based: the runtime engine runs generated TypeScript, resolves `tools.*` calls, and routes invocation through source/tool providers (OpenAPI, MCP, GraphQL, in-memory, and future formats). |
| 62 | +### Transport plane |
| 100 | 63 | |
| 101 | | −--- |
| 64 | +How clients talk to executor. |
| 102 | 65 | |
| 103 | | −## 5) Monorepo shape (Executor-only) |
| 66 | +- MCP server/gateway |
| 67 | +- OpenAPI/RPC control API |
| 68 | +- AI SDK tool bridge |
| 104 | 69 | |
| 105 | | −```text |
| 106 | | −v2/ |
| 107 | | − apps/ |
| 108 | | − cli/ |
| 109 | | − pm/ |
| 110 | | − web/ |
| 111 | | − convex/ |
| 112 | | − |
| 113 | | − packages/ |
| 114 | | − confect/ # imported from quickhub; Effect<->Convex schema bridge |
| 115 | | − schema/ # canonical Effect schemas |
| 116 | | − persistence-ports/ |
| 117 | | − persistence-local/ |
| 118 | | − persistence-convex/ |
| 119 | | − domain/ |
| 120 | | − approvals/ |
| 121 | | − source-manager/ |
| 122 | | − engine/ |
| 123 | | − mcp-gateway/ |
| 124 | | − oauth/ |
| 125 | | − secrets/ |
| 126 | | − sync/ |
| 127 | | − rpc/ |
| 128 | | − sdk/ |
| 129 | | − ai-sdk-adapter/ |
| 130 | | −``` |
| 70 | +Transport does not own core execution semantics. |
| 131 | 71 | |
| 132 | 72 | --- |
| 133 | 73 | |
| 134 | | −## 6) Dependency order (high-level) |
| 135 | | − |
| 136 | | −```text |
| 137 | | −confect -> schema -> persistence-ports |
| 74 | +## 4) Core contracts |
| 138 | 75 | |
| 139 | | −persistence-ports -> persistence-local |
| 140 | | −persistence-ports -> persistence-convex |
| 76 | +We do not need a mandatory public `new Executor()` product surface. |
| 141 | 77 | |
| 142 | | −schema + persistence-ports + oauth + secrets -> domain |
| 78 | +Primary internal shape is a kernel service (name can vary, e.g. `createExecutorKernel`). |
| 143 | 79 | |
| 144 | | −domain -> approvals/source-manager/engine |
| 145 | | −approvals/source-manager/engine -> mcp-gateway |
| 80 | +### Kernel contract |
| 146 | 81 | |
| 147 | | −domain/features -> rpc -> sdk -> ai-sdk-adapter |
| 82 | +- `execute(request) -> terminal result` (or async run id if configured) |
| 83 | +- `handleRuntimeToolCall(call) -> encoded tool-call result` |
| 84 | +- optional `control` capability |
| 148 | 85 | |
| 149 | | −apps/pm -> mcp-gateway + domain + persistence-local + rpc |
| 150 | | −apps/convex -> mcp-gateway + domain + persistence-convex + rpc |
| 151 | | −apps/cli/web -> sdk |
| 152 | | −``` |
| 86 | +### Adapter contracts |
| 153 | 87 | |
| 154 | | −Rules: |
| 88 | +- RuntimeDispatchAdapter |
| 89 | + - only decides where code runs (`local-inproc`, `deno-subprocess`, `cloudflare-worker-loader`) |
| 90 | +- ToolInvocationService |
| 91 | + - resolves tool path and invokes tool with policy/approval checks |
| 92 | +- CredentialResolver |
| 93 | + - resolves credential headers during tool invocation callback |
| 94 | +- ControlService (optional) |
| 95 | + - source/tool/policy management commands |
| 155 | 96 | |
| 156 | | −- `schema` has no dependency on feature/runtime packages. |
| 157 | | −- `domain` depends on interfaces, not concrete state adapters. |
| 158 | | −- apps are leaves; packages do not import apps. |
| 159 | | −- packages expose effectful orchestration through services (`Context.Tag`), with Layers wiring implementations. |
| 97 | +Important boundary: |
| 98 | +- Credential resolution is not part of runtime dispatch. |
| 99 | +- Credential resolution happens in callback-time tool invocation. |
| 160 | 100 | |
| 161 | 101 | --- |
| 162 | 102 | |
| 163 | | −## 7) Local-lite PM architecture |
| 164 | | − |
| 165 | | −`apps/pm` is the local daemon and source of hot truth. |
| 166 | | − |
| 167 | | −Responsibilities: |
| 103 | +## 5) Modes and targets |
| 168 | 104 | |
| 169 | | −- MCP endpoint surface |
| 170 | | −- source registration and status lifecycle |
| 171 | | −- approvals/input broker |
| 172 | | −- task orchestration and runtime execution coordination |
| 173 | | −- persistence (snapshot + event log) |
| 174 | | − |
| 175 | | −### Persistence model |
| 105 | +### Modes |
| 176 | 106 | |
| 177 | | −- Hot state: in-memory |
| 178 | | −- Durable: file-based |
| 179 | | − - `snapshot.json` |
| 180 | | − - `events.jsonl` (WAL) |
| 181 | | − - `artifacts/` (compiled tool manifests) |
| 182 | | − - secrets in secure store / dedicated protected file |
| 107 | +- local-lite: PM daemon + in-memory hot state + file durability |
| 108 | +- remote: Convex-backed target (cloud and self-host are deployment variants of same target) |
| 183 | 109 | |
| 184 | | −Crash/restart behavior: |
| 110 | +### Runtime targets |
| 185 | 111 | |
| 186 | | −- Rehydrate from snapshot + events. |
| 187 | | −- Rebuild derived indexes in memory. |
| 188 | | −- Mark previously running tasks terminally with restart reason. |
| 112 | +- `deno-subprocess`: local secure default |
| 113 | +- `cloudflare-worker-loader`: remote production default |
| 114 | +- `node:vm`: explicit unsafe/dev fallback only |
| 189 | 115 | |
| 190 | 116 | --- |
| 191 | 117 | |
| 192 | | −## 8) Data model strategy |
| 193 | | − |
| 194 | | −Effect Schema is canonical (`packages/schema`). |
| 118 | +## 6) Local and cloud runtime flows |
| 119 | + |
| 120 | +### Local flow |
| 121 | + |
| 122 | +1. Request starts run (`execute`). |
| 123 | +2. Runtime dispatch executes locally (`inproc` or `deno-subprocess`). |
| 124 | +3. Runtime `tools.*` call hits local callback handler. |
| 125 | +4. Callback handler invokes shared tool invocation pipeline: |
| 126 | + - resolve tool |
| 127 | + - policy check |
| 128 | + - approval gating |
| 129 | + - credential resolution (if needed) |
| 130 | + - provider invoke |
| 131 | +5. Runtime resumes and completes. |
| 132 | +6. Run/events persisted locally. |
| 133 | + |
| 134 | +### Cloud flow (Convex + Cloudflare worker loader) |
| 135 | + |
| 136 | +1. Request starts run in Convex. |
| 137 | +2. Convex dispatches run to Cloudflare loader. |
| 138 | +3. Worker `tools.*` call hits Convex runtime callback action (`handleToolCall`). |
| 139 | +4. Callback action invokes shared tool invocation pipeline: |
| 140 | + - resolve tool |
| 141 | + - policy check |
| 142 | + - approval gating |
| 143 | + - credential resolution from Convex/vault |
| 144 | + - provider invoke |
| 145 | +5. Worker resumes and returns terminal status. |
| 146 | +6. Convex marks run terminal and emits events. |
| 147 | + |
| 148 | +Serverless note: |
| 149 | +- There is no long-lived in-memory `onToolCall` loop in cloud. |
| 150 | +- Callback requests are stateless/durable and reconstruct context per request. |
| 195 | 151 | |
| 196 | | −Current scaffold includes: |
| 197 | | − |
| 198 | | −- domain IDs (`src/ids.ts`) |
| 199 | | −- enums/common primitives |
| 200 | | −- model schemas: |
| 201 | | − - profile, workspace |
| 202 | | − - source, tool artifact |
| 203 | | − - credential ref, oauth token |
| 204 | | − - policy, approval |
| 205 | | − - task run, sync state |
| 206 | | − - event envelope |
| 207 | | −- local snapshot/WAL schemas in `persistence-local` |
| 152 | +--- |
| 208 | 153 | |
| 209 | | −### Convex ID compatibility |
| 154 | +## 7) Blocking and non-blocking execution |
| 210 | 155 | |
| 211 | | −IDs are now intentionally split by layer: |
| 156 | +Both are first-class and share one implementation path. |
| 212 | 157 | |
| 213 | | −- **Domain IDs** live in `packages/schema` as portable branded IDs. |
| 214 | | −- **Convex table IDs** live in `packages/persistence-convex/src/convex-ids.ts` via `Id("table")` from `@executor-v2/confect`. |
| 158 | +- Async canonical path: |
| 159 | + - submit run -> return `runId` |
| 160 | + - query/watch run status -> terminal result |
| 161 | +- Blocking convenience path: |
| 162 | + - submit run |
| 163 | + - wait/watch internally |
| 164 | + - return terminal result |
| 215 | 165 | |
| 216 | | −This keeps canonical schema backend-agnostic while preserving strict table-typed IDs inside the Convex adapter boundary. |
| 166 | +This is mostly transport/API behavior; core runtime pipeline stays the same. |
| 217 | 167 | |
| 218 | 168 | --- |
| 219 | 169 | |
| 220 | | −## 9) Tooling, providers, and MCP surface |
| 170 | +## 8) MCP, RPC, and AI SDK strategy |
| 221 | 171 | |
| 222 | | −### Top-level model-facing tools |
| 172 | +### MCP |
| 223 | 173 | |
| 224 | | −Keep top-level surface minimal: |
| 174 | +MCP remains the main execution transport surface. |
| 225 | 175 | |
| 176 | +Top-level model-facing tools stay minimal: |
| 226 | 177 | - `execute` (primary) |
| 227 | | −- optional diagnostics (`health`, `version`) only if needed |
| 228 | | − |
| 229 | | −### Configuration model |
| 178 | +- optional diagnostics (`health`, `version`) |
| 230 | 179 | |
| 231 | | −Executor config is handled as tool calls from inside `execute` via built-in namespace: |
| 180 | +### Control API |
| 232 | 181 | |
| 233 | | −- `tools.executor.sources.add(...)` |
| 234 | | −- `tools.executor.sources.list(...)` |
| 235 | | −- `tools.executor.sources.remove(...)` |
| 236 | | −- other `tools.executor.*` controls as needed |
| 182 | +Control is exposed through Executor OpenAPI/RPC contracts. |
| 237 | 183 | |
| 238 | | −No mandatory top-level `set_enabled` in v1 (remove/add is enough for initial UX). |
| 184 | +- Host/app control calls should use control API directly. |
| 185 | +- Model control calls can use `tools.executor.*` inside `execute` code. |
| 186 | +- Both host and model control paths must hit the same underlying control handlers. |
| 239 | 187 | |
| 240 | | −### Provider architecture (source-agnostic) |
| 188 | +### AI SDK |
| 241 | 189 | |
| 242 | | −Use a provider registry, not format-specific execution in core runtime. |
| 190 | +AI SDK is a bridge over executor execute capability. |
| 243 | 191 | |
| 244 | | −- Each provider implements: |
| 245 | | − - `discover(source) -> ToolManifest` |
| 246 | | − - `invoke(call) -> ToolResult` |
| 247 | | −- Core runtime stays source-format agnostic; it resolves tool identity and dispatches to provider. |
| 248 | | −- Initial provider kinds: |
| 249 | | − - `openapi` |
| 250 | | − - `mcp` |
| 251 | | − - `graphql` |
| 252 | | − - `in_memory` (host-registered tools; AI-SDK style) |
| 253 | | −- Future source formats are added as new providers, without changing core execution semantics. |
| 192 | +- `toAiSdkTools(executorLike)` maps to regular AI SDK tools. |
| 193 | +- Local and remote both work if they expose `execute(...)` semantics. |
| 194 | +- If client is MCP-native, AI SDK bridge is optional. |
| 254 | 195 | |
| 255 | | −### Canonical tool descriptor |
| 196 | +AI SDK should not become a second control-plane abstraction. |
| 256 | 197 | |
| 257 | | −All providers publish into one canonical descriptor used by runtime and SDK: |
| 198 | +--- |
| 258 | 199 | |
| 259 | | −- stable tool id |
| 260 | | −- display metadata (name, description) |
| 261 | | −- invocation metadata (kind + provider payload) |
| 262 | | −- availability metadata (`local_only` vs `remote_capable`) |
| 200 | +## 9) Tooling and provider model |
| 263 | 201 | |
| 264 | | −Provider-specific details remain in a typed payload at the edge. |
| 202 | +Provider registry is canonical. |
| 265 | 203 | |
| 266 | | −### Execution semantics |
| 204 | +Each provider implements: |
| 205 | +- discover(source) -> ToolManifest |
| 206 | +- invoke(call) -> ToolResult |
| 267 | 207 | |
| 268 | | −- Agent writes TS using `tools.*`. |
| 269 | | −- Runtime proxy resolves tool calls against the canonical registry. |
| 270 | | −- Provider receives invocation and returns canonical result envelope. |
| 271 | | −- Credentials/policies/approvals are applied server-side during invocation. |
| 208 | +Initial provider kinds: |
| 209 | +- `openapi` |
| 210 | +- `in_memory` |
| 211 | +- `mcp` (planned) |
| 212 | +- `graphql` (planned) |
| 272 | 213 | |
| 273 | | −--- |
| 214 | +Canonical descriptor includes: |
| 215 | +- stable tool id/path |
| 216 | +- display metadata |
| 217 | +- invocation metadata |
| 218 | +- capability metadata |
| 274 | 219 | |
| 275 | | −## 10) Dynamic source registration |
| 220 | +Provider-specific details remain in provider payload types. |
| 276 | 221 | |
| 277 | | −Goal: user can ask the agent to use a service; agent can register the source through Executor tooling. |
| 222 | +--- |
| 278 | 223 | |
| 279 | | −- Discovery itself may happen outside Executor (model/web). |
| 280 | | −- Registration happens via Executor control tools (`tools.executor.sources.add`). |
| 281 | | −- `source.kind` selects provider (`openapi`, `mcp`, `graphql`, etc.). |
| 282 | | −- In-memory tools are host-registered (runtime side), not persisted remote source records by default. |
| 224 | +## 10) Persistence and registry model |
| 283 | 225 | |
| 284 | | −Registration flow states (networked providers): |
| 226 | +Persistence is source of truth. Runtime registry is derived. |
| 285 | 227 | |
| 286 | | −- `draft -> probing -> auth_required -> connected | error` |
| 228 | +### Persisted |
| 287 | 229 | |
| 288 | | −Registration flow states (in-memory provider): |
| 230 | +- sources |
| 231 | +- tool artifacts/manifests |
| 232 | +- runs/events |
| 233 | +- approvals/policies |
| 234 | +- credential references (secret values in secure backing store) |
| 289 | 235 | |
| 290 | | −- `registered -> connected | error` |
| 236 | +### Derived |
| 291 | 237 | |
| 292 | | −Auth handling: |
| 238 | +- in-memory runnable tool index/registry |
| 239 | +- discovery indexes |
| 293 | 240 | |
| 294 | | −- MCP: dynamic auth/OAuth detection, client registration where needed |
| 295 | | −- OpenAPI/GraphQL: inspect auth schemes, prompt for API key/OAuth when required |
| 241 | +Mutation rule: |
| 242 | +1. persist authoritative record |
| 243 | +2. refresh/rebuild derived artifact/index |
| 244 | +3. bump registry version |
| 296 | 245 | |
| 297 | | −Secrets are entered through host UX, not model transcript. |
| 246 | +Run rule: |
| 247 | +- each run snapshots tool registry version at start |
| 248 | +- in-flight runs keep snapshot semantics |
| 249 | +- new runs see newest version |
| 298 | 250 | |
| 299 | 251 | --- |
| 300 | 252 | |
| 301 | | −## 11) Approvals and elicitation/input |
| 302 | | − |
| 303 | | −Use one internal broker for human-required inputs: |
| 253 | +## 11) Approval and secret boundaries |
| 304 | 254 | |
| 255 | +Approvals/input are host UX concerns, not model tools. |
| 305 | 256 | - approval |
| 306 | 257 | - confirm |
| 307 | 258 | - form input |
| 308 | 259 | - secret input |
| 309 | | − |
| 310 | | −### Capability-aware delivery |
| 311 | | − |
| 312 | | −- If client supports MCP elicitation: inline prompt. |
| 313 | | −- If not: out-of-band path (tmux pane, local web, notifications). |
| 314 | | − |
| 315 | | −### Terminal UX |
| 316 | | − |
| 317 | | −For fullscreen terminal clients: |
| 318 | | − |
| 319 | | −- tmux split/popup watcher |
| 320 | | −- auto-show when pending approvals appear |
| 321 | | −- auto-hide after queue drains (with debounce) |
| 322 | | − |
| 323 | | −### Important boundary |
| 324 | | − |
| 325 | | −Approval/input actions are **host UX callbacks**, not AI SDK model tools. |
| 260 | +Secrets/credentials: |
| 261 | +- never emitted to model transcript |
| 262 | +- resolved server-side during invocation |
| 263 | +- converted to headers/token material for provider calls |
| 326 | 264 | |
| 327 | 265 | --- |
| 328 | 266 | |
| 329 | | −## 12) AI SDK strategy |
| 330 | | − |
| 331 | | −API shape should be Executor session/adapters, not single-source helper APIs. |
| 332 | | − |
| 333 | | −### Adapter-first target design |
| 334 | | − |
| 335 | | −`Executor.create({ adapter })` with adapters such as: |
| 336 | | − |
| 337 | | −- `inproc` (embedded runtime, no install) |
| 338 | | −- `pm` (local daemon transport) |
| 339 | | −- `remote` (Convex target transport) |
| 340 | | − |
| 341 | | −### Public SDK shape |
| 342 | | − |
| 343 | | −Primary operations: |
| 344 | | − |
| 345 | | −- execute code |
| 346 | | −- register/list/remove sources |
| 347 | | −- register/list/remove in-memory tools (AI-SDK style) |
| 348 | | −- stream run status/results |
| 349 | | − |
| 350 | | −Do not expose approval/input-response as model tools. |
| 351 | | −Use host callbacks (e.g. `onInputRequest`) instead. |
| 267 | +## 12) Security posture |
| 268 | +- PM binds to loopback by default. |
| 269 | +- Runtime callbacks require internal token validation. |
| 270 | +- Secrets do not pass through model args/transcripts. |
| 271 | +- Credential refs in state; secret values in secure stores. |
| 272 | +- Audit events store actions/decisions, not raw secret material. |
| 352 | 273 | |
| 353 | 274 | --- |
| 354 | 275 | |
| 355 | | −## 13) Security posture |
| 276 | +## 13) Monorepo shape (Executor v2) |
| 356 | 277 | |
| 357 | | −- PM binds to loopback by default. |
| 358 | | −- Secrets never pass through model args/transcripts. |
| 359 | | −- Credential references in state; secret values in secure store. |
| 360 | | −- Strict origin/token checks if hosted web talks to localhost target. |
| 361 | | −- Audit events store decisions/actions, not raw secret material. |
| 362 | | − |
| 363 | | −--- |
| 278 | +```text |
| 279 | +v2/ |
| 280 | + apps/ |
| 281 | + cli/ |
| 282 | + pm/ |
| 283 | + web/ |
| 284 | + convex/ |
| 285 | + packages/ |
| 286 | + confect/ |
| 287 | + schema/ |
| 288 | + state-contracts/ |
| 289 | + state-local-file/ |
| 290 | + state-convex/ |
| 291 | + domain/ |
| 292 | + approvals/ |
| 293 | + source-manager/ |
| 294 | + engine/ |
| 295 | + mcp-gateway/ |
| 296 | + oauth/ |
| 297 | + secrets/ |
| 298 | + sync/ |
| 299 | + rpc/ |
| 300 | + sdk/ |
| 301 | + ai-sdk-adapter/ |
| 302 | +``` |
| 364 | 303 | |
| 365 | | −## 14) Upgrade and sync |
| 304 | +Planned follow-up split (after behavior stabilizes): |
| 305 | +- runtime adapters as dedicated packages |
| 306 | +- transport adapters as dedicated packages |
| 366 | 307 | |
| 367 | | −Primary path: |
| 308 | +--- |
| 368 | 309 | |
| 369 | | −- `local-lite -> remote` |
| 310 | +## 14) Dependency direction |
| 370 | 311 | |
| 371 | | −Optional reverse: |
| 312 | +```text |
| 313 | +confect -> schema -> state-contracts |
| 372 | 314 | |
| 373 | | −- `remote -> local-lite` export/backup |
| 315 | +state-contracts -> state-local-file |
| 316 | +state-contracts -> state-convex |
| 374 | 317 | |
| 375 | | −Sync principles: |
| 318 | +schema + state-contracts + oauth + secrets -> domain |
| 319 | +domain -> approvals/source-manager/engine |
| 320 | +approvals/source-manager/engine -> mcp-gateway |
| 321 | +domain/features -> rpc -> sdk -> ai-sdk-adapter |
| 376 | 322 | |
| 377 | | −- deterministic source identity |
| 378 | | −- idempotent imports |
| 379 | | −- explicit credential migration policy |
| 380 | | −- migration cursor/mapping persisted in local state |
| 323 | +apps/pm -> mcp-gateway + domain + state-local-file + rpc |
| 324 | +apps/convex -> mcp-gateway + domain + state-convex + rpc |
| 325 | +apps/cli/web -> sdk (and/or ai-sdk bridge) |
| 326 | +``` |
| 327 | +Rules: |
| 328 | +- `schema` is backend-agnostic. |
| 329 | +- apps are leaves. |
| 330 | +- packages do not import apps. |
| 331 | +- orchestration services are injected via interfaces/tags/layers. |
| 381 | 332 | |
| 382 | 333 | --- |
| 383 | 334 | |
| 384 | 335 | ## 15) Rollout phases |
| 385 | 336 | |
| 386 | | −### Phase 0: Architecture + contracts |
| 337 | +### Phase 0: lock contracts |
| 387 | 338 | |
| 388 | | −- finalize package boundaries |
| 389 | | −- finalize schema model set and invariants |
| 390 | | −- define state/repository interfaces |
| 339 | +- finalize kernel + adapter contracts |
| 340 | +- finalize execution/control/transport boundaries |
| 341 | +- align package names/imports to current v2 structure |
| 391 | 342 | |
| 392 | | −### Phase 1: Local-lite core |
| 343 | +### Phase 1: local-lite execute vertical slice |
| 393 | 344 | |
| 394 | | −- PM daemon skeleton |
| 395 | | −- file persistence (snapshot + WAL) |
| 396 | | −- execute path with runtime proxy integration |
| 397 | | −- provider registry + canonical tool descriptor |
| 398 | | −- basic source add/list/remove through `tools.executor.*` |
| 399 | | −- first provider slices: `openapi` + `in_memory` |
| 345 | +- PM execute path with persisted run lifecycle |
| 346 | +- runtime callback path wired through shared invocation service |
| 347 | +- openapi + in_memory providers |
| 400 | 348 | |
| 401 | | −### Phase 2: Input/approval UX |
| 349 | +### Phase 2: control vertical slice |
| 402 | 350 | |
| 403 | | −- InputBroker + pending state persistence |
| 404 | | −- tmux watcher + local web fallback |
| 405 | | −- capability-aware elicitation fallback paths |
| 351 | +- source add/list/remove via control handlers |
| 352 | +- same handlers callable from host control API and `tools.executor.*` |
| 353 | +- artifact refresh + registry versioning |
| 406 | 354 | |
| 407 | | −### Phase 3: Remote parity |
| 355 | +### Phase 3: approvals and credentials hardening |
| 408 | 356 | |
| 409 | | −- persistence-convex adapter and Convex app wiring |
| 410 | | −- shared RPC contracts for pm/convex |
| 411 | | −- target switching in CLI/web |
| 412 | | −- runtime/provider capability checks for local vs remote |
| 357 | +- persisted pending/resume/deny flow |
| 358 | +- capability-aware UX fallbacks |
| 359 | +- callback-time credential resolver implementations (local + convex) |
| 413 | 360 | |
| 414 | | −### Phase 4: Sync + upgrades |
| 361 | +### Phase 4: remote parity |
| 415 | 362 | |
| 416 | | −- local->remote promotion pipeline |
| 417 | | −- idempotent mapping/cursor strategy |
| 418 | | −- conflict handling and dry-run reporting |
| 363 | +- Convex adapter wiring for kernel and control |
| 364 | +- cloudflare loader integration through runtime dispatch adapter |
| 365 | +- parity/conformance suite across local and remote |
| 419 | 366 | |
| 420 | | −### Phase 5: AI SDK adapters |
| 367 | +### Phase 5: transport polish |
| 421 | 368 | |
| 422 | | −- inproc adapter |
| 423 | | −- pm transport adapter |
| 424 | | −- remote transport adapter |
| 425 | | −- host callback API for inputs |
| 369 | +- MCP surface stabilization |
| 370 | +- OpenAPI/RPC control surface stabilization |
| 371 | +- AI SDK bridge (`toAiSdkTools`) and docs/examples |
| 426 | 372 | |
| 427 | 373 | --- |
| 428 | 374 | |
| 429 | 375 | ## 16) Success criteria |
| 430 | 376 | |
| 431 | | −- First successful `npx executor <agent>` run with no login in <2 minutes. |
| 432 | | −- No required database for local-lite mode. |
| 433 | | −- Same execution semantics across local and remote targets. |
| 434 | | −- Agent can add sources and execute against them in one flow. |
| 435 | | −- Provider-based execution works for at least `openapi` + `in_memory` in local-lite. |
| 436 | | −- Approval/input is usable in fullscreen terminal clients. |
| 377 | +- First successful local run with no login in under 2 minutes. |
| 378 | +- No required DB in local-lite mode. |
| 379 | +- Same execution semantics across local and remote. |
| 380 | +- Callback-time approvals/policy/credentials work in both local and cloud. |
| 381 | +- Control operations are callable both by host API and from inside execute namespace. |
| 382 | +- Blocking and non-blocking execution both available via shared run pipeline. |
| 437 | 383 | |
| 438 | 384 | --- |
| 439 | 385 | |
| 440 | 386 | ## 17) Non-goals (initial) |
| 441 | 387 | |
| 442 | | −- Recreating all enterprise admin UX in local-lite. |
| 443 | | −- Exposing huge top-level tool catalogs to the model by default. |
| 388 | +- Recreating full enterprise admin UX in local-lite. |
| 389 | +- Expanding model-facing top-level tool catalog by default. |
| 444 | 390 | - Maintaining separate cloud vs self-host architecture branches. |
| 391 | +- Coupling runtime adapters with credential/secrets responsibilities. |
| 445 | 392 | |
| 446 | 393 | --- |
| 447 | 394 | |
| 448 | 395 | ## 18) Current scaffold status |
| 449 | 396 | |
| 450 | 397 | Implemented in `v2` so far: |
| 451 | | − |
| 452 | 398 | - monorepo app/package skeleton |
| 453 | 399 | - `confect` imported from quickhub |
| 454 | | −- schema package scaffold with Effect models and event envelope |
| 455 | | −- schema-driven runtime tool models (canonical descriptor + OpenAPI invocation/manifest) |
| 456 | | −- domain ID groundwork in `schema/src/ids.ts` |
| 457 | | −- Convex table ID groundwork in `persistence-convex/src/convex-ids.ts` |
| 458 | | −- shared persistence ports for `SourceStore` and `ToolArtifactStore` |
| 459 | | −- local persistence adapters for source/tool artifacts |
| 460 | | −- local-only snapshot/WAL contract and persistence implementation |
| 461 | | −- OpenAPI extraction + artifact refresh/reuse flow |
| 462 | | −- provider registry contracts and routing in `engine` |
| 400 | +- schema package scaffold with Effect models/event envelope |
| 401 | +- shared state contracts for source/tool artifacts |
| 402 | +- local file adapters for source/tool artifacts + snapshot/WAL state store |
| 403 | +- source manager OpenAPI extraction + artifact refresh/reuse |
| 404 | +- provider registry contracts and routing in engine |
| 463 | 405 | - OpenAPI provider invocation + in-memory provider path through registry |
| 464 | | −- local in-process JS runner with `tools.*` proxy dispatch |
| 406 | +- local in-process JS runner with `tools.*` proxy |
| 465 | 407 | - Deno subprocess runtime path with IPC tool-call proxying and tests |
| 466 | | −- runtime adapter contract + registry scaffold in `engine` with mapped adapters (`local-inproc`, `deno-subprocess`, `cloudflare-worker-loader`) |
| 467 | | −- minimal PM execute lifecycle wired through MCP `executor.execute` -> runtime adapter registry (single run, no persisted run state yet) |
| 468 | | −- vertical test: OpenAPI spec -> manifest -> execute code -> HTTP tool call |
| 469 | | −- service-first wiring (`Context.Tag` + Layer) across source manager and persistence/local runtime orchestration |
| 408 | +- runtime adapter contract + registry scaffold |
| 409 | +- PM execute lifecycle scaffold via MCP execute path |
| 470 | 410 | |
| 471 | | −Not implemented yet: |
| 411 | +Still missing: |
| 472 | 412 | |
| 473 | | −- `tools.executor.sources.add/list/remove` end-to-end PM + MCP gateway wiring |
| 474 | | −- approval adapter/state machine integration (pending/resume/deny persistence) in v2 runtime path |
| 475 | | −- provider/runtime conformance suite beyond current baseline tests (timeouts, cancellation, pending approval) |
| 476 | | −- Cloudflare worker loader adapter execution integration (current adapter is explicit scaffold/not-implemented) |
| 477 | | −- MCP/GraphQL provider implementations |
| 478 | | −- full rpc/sdk adapter wiring and sync engine |
| 413 | +- fully wired control handlers (`tools.executor.sources.add/list/remove`) end-to-end |
| 414 | +- shared tool invocation service wired with persisted approval lifecycle in v2 path |
| 415 | +- cloudflare-worker-loader runtime adapter implementation in v2 runtime path |
| 416 | +- parity conformance tests for runtime/provider/pending-approval/cancel flows |
| 417 | +- full rpc/sdk/ai-sdk adapter wiring and usage examples |
| 479 | 418 | |
| 480 | 419 | --- |
| 481 | 420 | |
| 482 | 421 | ## 19) Immediate next steps |
| 483 | 422 | |
| 484 | | −1. Add first `tools.executor.sources.add/list/remove` control operations end-to-end in PM + MCP gateway. |
| 485 | | −2. Implement approval adapter contract with persisted pending/resume/deny flow for runtime tool calls. |
| 486 | | −3. Add provider/runtime conformance tests (invoke success/failure, timeout, pending approval, cancel). |
| 487 | | −4. Add Cloudflare worker loader adapter integration against the runtime adapter contract. |
| 488 | | −5. Extend PM execute lifecycle from single-run scaffold to persisted run state + streaming status. |
| 423 | +1. Land shared `ToolInvocationService` contract in v2 and wire PM execute callback path through it. |
| 424 | +2. Add `CredentialResolver` interfaces and local/convex implementations; integrate at invocation-time only. |
| 425 | +3. Implement control handlers for sources (`add/list/remove`) and route both host API and `tools.executor.*` through same code path. |
| 426 | +4. Implement cloudflare-worker-loader runtime dispatch adapter in v2 and wire Convex callback endpoints. |
| 427 | +5. Add run API split for async + blocking convenience (`submit` + `wait/watch`) without duplicating execution logic. |
| 428 | +6. Add conformance tests that run same scenarios across local and remote adapters. |
| 429 | + |