History

feat: enhance PM and OpenAPI integration with source credential management

- Added support for managing source credentials in OpenAPI execution, allowing for dynamic header injection based on source configuration. - Introduced new tools for adding, listing, and removing sources within the PM context. - Updated the approval client to handle new PM configuration paths. - Enhanced error handling in local and deno subprocess runners to include detailed output from tool calls. - Updated documentation to reflect new features and changes in source management.

Rhys Sullivan committed 8 months ago

Commit 785e08f1 parent aa5b69e22 files · +1217 −80Browse files

Unified · Split

22 files
  1. experiemental/approval-client.ts +6 −0
  2. experiemental/claude-mcp-pm.json +8 −0
  3. v2/.gitignore +2 −0
  4. v2/PLAN.md +7 −5
  5. v2/apps/convex/README.md +1 −1
  6. v2/apps/convex/convex/mcp.ts +4 −0
  7. v2/apps/pm/README.md +1 −0
  8. v2/apps/pm/package.json +5 −0
  9. v2/apps/pm/src/main.ts +740 −6
  10. v2/bun.lock +5 −0
  11. v2/packages/engine/README.md +1 −0
  12. v2/packages/engine/src/deno-subprocess-runner.ts +30 −5
  13. v2/packages/engine/src/deno-subprocess-worker.mjs +28 −1
  14. v2/packages/engine/src/local-runner.ts +79 −15
  15. v2/packages/engine/src/openapi-execution.test.ts +124 −0
  16. v2/packages/engine/src/openapi-provider.ts +79 −0
  17. v2/packages/engine/src/runtime-adapters.test.ts +53 −0
  18. v2/packages/mcp-gateway/README.md +2 −1
  19. v2/packages/mcp-gateway/src/server.ts +12 −46
  20. v2/packages/schema/README.md +1 −0
  21. v2/packages/schema/src/index.ts +1 −0
  22. v2/packages/schema/src/models/source-config.ts +28 −0
experiemental/approval-client.ts+6−0
@@ -25,6 +25,7 @@ interface ParsedArgs { · 24 unmodified lines
25}25}
26 26
27const DEFAULT_MCP_CONFIG_PATH = path.resolve(import.meta.dir, "claude-mcp-demo.json");27const DEFAULT_MCP_CONFIG_PATH = path.resolve(import.meta.dir, "claude-mcp-demo.json");
28const DEFAULT_PM_MCP_CONFIG_PATH = path.resolve(import.meta.dir, "claude-mcp-pm.json");
28const HOME_DIR = process.env.HOME ?? ".";29const HOME_DIR = process.env.HOME ?? ".";
29const STATE_PATH = path.join(HOME_DIR, ".executor-lite-approvals", "mcp-state.json");30const STATE_PATH = path.join(HOME_DIR, ".executor-lite-approvals", "mcp-state.json");
30 31
@@ -47,6 +48,11 @@ function defaultAgentCommand(session: string): string { · 16 unmodified lines
47 return `claude --strict-mcp-config --mcp-config ${shellQuote(DEFAULT_MCP_CONFIG_PATH)} --allowedTools ${shellQuote(allowedTools)} --permission-mode bypassPermissions`;48 return `claude --strict-mcp-config --mcp-config ${shellQuote(DEFAULT_MCP_CONFIG_PATH)} --allowedTools ${shellQuote(allowedTools)} --permission-mode bypassPermissions`;
48 }49 }
49 50
51 if (session === "claude-pm") {
52 const allowedTools = "mcp__executor-v2-local__executor.execute";
53 return `claude --strict-mcp-config --mcp-config ${shellQuote(DEFAULT_PM_MCP_CONFIG_PATH)} --allowedTools ${shellQuote(allowedTools)} --permission-mode bypassPermissions`;
54 }
55
50 return session;56 return session;
51}57}
52 58
experiemental/claude-mcp-pm.json+8−0
@@ -0,0 +1,8 @@
1{
2 "mcpServers": {
3 "executor-v2-local": {
4 "type": "http",
5 "url": "http://127.0.0.1:8787/mcp"
6 }
7 }
8}
v2/.gitignore+2−0
@@ -1 +1,3 @@
1.reference/1.reference/
2.executor-v2/
3apps/pm/.executor-v2/
v2/PLAN.md+7−5
@@ -410,23 +410,25 @@ Implemented in `v2` so far: · 409 unmodified lines
410- Deno subprocess runtime path with IPC tool-call proxying and tests410- Deno subprocess runtime path with IPC tool-call proxying and tests
411- runtime adapter contract + registry scaffold in `engine` with mapped adapters (`local-inproc`, `deno-subprocess`, `cloudflare-worker-loader`)411- runtime adapter contract + registry scaffold in `engine` with mapped adapters (`local-inproc`, `deno-subprocess`, `cloudflare-worker-loader`)
412- minimal PM execute lifecycle wired through MCP `executor.execute` -> runtime adapter registry (single run, no persisted run state yet)412- minimal PM execute lifecycle wired through MCP `executor.execute` -> runtime adapter registry (single run, no persisted run state yet)
413- PM source control path wired through execute-time tools namespace (`tools.executor.sources.add/list/remove`) with local file persistence and OpenAPI probe/extraction
414- source credential-aware OpenAPI invocation support via source config (`api_key`/`bearer` modes)
413- vertical test: OpenAPI spec -> manifest -> execute code -> HTTP tool call415- vertical test: OpenAPI spec -> manifest -> execute code -> HTTP tool call
414- service-first wiring (`Context.Tag` + Layer) across source manager and persistence/local runtime orchestration416- service-first wiring (`Context.Tag` + Layer) across source manager and persistence/local runtime orchestration
415 417
416Not implemented yet:418Not implemented yet:
417- `tools.executor.sources.add/list/remove` end-to-end PM + MCP gateway wiring
418- approval adapter/state machine integration (pending/resume/deny persistence) in v2 runtime path419- approval adapter/state machine integration (pending/resume/deny persistence) in v2 runtime path
419- provider/runtime conformance suite beyond current baseline tests (timeouts, cancellation, pending approval)420- provider/runtime conformance suite beyond current baseline tests (timeouts, cancellation, pending approval)
420- Cloudflare worker loader adapter execution integration (current adapter is explicit scaffold/not-implemented)421- Cloudflare worker loader adapter execution integration (current adapter is explicit scaffold/not-implemented)
421- MCP/GraphQL provider implementations422- MCP/GraphQL provider implementations
423- secure secret-store-backed credential persistence (current local MVP stores credential value in source config)
422- full rpc/sdk adapter wiring and sync engine424- full rpc/sdk adapter wiring and sync engine
423 425
424---426---
425 427
426## 19) Immediate next steps428## 19) Immediate next steps
427 429
4281. Add first `tools.executor.sources.add/list/remove` control operations end-to-end in PM + MCP gateway.4301. Implement approval adapter contract with persisted pending/resume/deny flow for runtime tool calls.
4292. Implement approval adapter contract with persisted pending/resume/deny flow for runtime tool calls.4312. Add provider/runtime conformance tests (invoke success/failure, timeout, pending approval, cancel).
4303. Add provider/runtime conformance tests (invoke success/failure, timeout, pending approval, cancel).4323. Add Cloudflare worker loader adapter integration against the runtime adapter contract.
4314. Add Cloudflare worker loader adapter integration against the runtime adapter contract.4334. Move source credential persistence to secure store-backed refs (remove raw credential values from source config).
4325. Extend PM execute lifecycle from single-run scaffold to persisted run state + streaming status.4345. Extend PM execute lifecycle from single-run scaffold to persisted run state + streaming status.
v2/apps/convex/README.md+1−1
@@ -4,4 +4,4 @@ Convex remote server app scaffold for Executor v2. · 3 unmodified lines
4 4
5Current scaffold includes:5Current scaffold includes:
6- Convex schema wiring at `convex/schema.ts`, sourced from `@executor-v2/persistence-convex`6- Convex schema wiring at `convex/schema.ts`, sourced from `@executor-v2/persistence-convex`
7- bare-minimum MCP HTTP endpoint with a stub tool at `convex/http.ts` and `convex/mcp.ts`7- bare-minimum MCP HTTP endpoint exposing `executor.execute` (currently returns not-wired error) at `convex/http.ts` and `convex/mcp.ts`
v2/apps/convex/convex/mcp.ts+4−0
@@ -6,4 +6,8 @@ export const mcpHandler = httpAction(async (_ctx, request) => · 5 unmodified lines
6 target: "remote",6 target: "remote",
7 serverName: "executor-v2-convex",7 serverName: "executor-v2-convex",
8 serverVersion: "0.0.0",8 serverVersion: "0.0.0",
9 execute: async () => ({
10 isError: true,
11 error: "Remote execute path is not wired yet",
12 }),
9 }));13 }));
v2/apps/pm/README.md+1−0
@@ -7,3 +7,4 @@ Current scaffold includes: · 6 unmodified lines
7- health endpoint at `GET /healthz`7- health endpoint at `GET /healthz`
8- MCP tool routing via `@executor-v2/mcp-gateway`8- MCP tool routing via `@executor-v2/mcp-gateway`
9- `executor.execute` wired to engine runtime adapters (`local-inproc` default, `deno-subprocess` optional)9- `executor.execute` wired to engine runtime adapters (`local-inproc` default, `deno-subprocess` optional)
10- `tools.executor.sources.add/list/remove` available inside execute runtime via in-memory provider wiring
v2/apps/pm/package.json+5−0
@@ -8,8 +8,13 @@ · 7 unmodified lines
8 "typecheck": "bunx tsc --noEmit -p tsconfig.json"8 "typecheck": "bunx tsc --noEmit -p tsconfig.json"
9 },9 },
10 "dependencies": {10 "dependencies": {
11 "@effect/platform-bun": "^0.86.0",
11 "@executor-v2/engine": "workspace:*",12 "@executor-v2/engine": "workspace:*",
12 "@executor-v2/mcp-gateway": "workspace:*",13 "@executor-v2/mcp-gateway": "workspace:*",
14 "@executor-v2/persistence-local": "workspace:*",
15 "@executor-v2/persistence-ports": "workspace:*",
16 "@executor-v2/schema": "workspace:*",
17 "@executor-v2/source-manager": "workspace:*",
13 "effect": "^3.19.8"18 "effect": "^3.19.8"
14 }19 }
15}20}
v2/apps/pm/src/main.ts+740−6
@@ -1,20 +1,147 @@
1import { resolve } from "node:path";
2
3import * as BunContext from "@effect/platform-bun/BunContext";
1import {4import {
5 ToolProviderError,
2 makeCloudflareWorkerLoaderRuntimeAdapter,6 makeCloudflareWorkerLoaderRuntimeAdapter,
3 makeDenoSubprocessRuntimeAdapter,7 makeDenoSubprocessRuntimeAdapter,
4 makeLocalInProcessRuntimeAdapter,8 makeLocalInProcessRuntimeAdapter,
9 makeOpenApiToolProvider,
5 makeRuntimeAdapterRegistry,10 makeRuntimeAdapterRegistry,
6 makeToolProviderRegistry,11 makeToolProviderRegistry,
12 openApiToolDescriptorsFromManifest,
7 ToolProviderRegistryService,13 ToolProviderRegistryService,
14 type RuntimeRunnableTool,
15 type ToolProvider,
8} from "@executor-v2/engine";16} from "@executor-v2/engine";
9import {17import {
10 handleMcpHttpRequest,18 handleMcpHttpRequest,
11 type ExecuteToolInput,19 type ExecuteToolInput,
12 type ExecuteToolResult,20 type ExecuteToolResult,
13} from "@executor-v2/mcp-gateway";21} from "@executor-v2/mcp-gateway";
22import {
23 makeLocalSourceStore,
24 makeLocalToolArtifactStore,
25} from "@executor-v2/persistence-local";
26import type {
27 SourceStore,
28 ToolArtifactStore,
29} from "@executor-v2/persistence-ports";
30import {
31 CanonicalToolDescriptorSchema,
32 OpenApiSourceConfigSchema,
33 SourceSchema,
34 SourceIdSchema,
35 WorkspaceIdSchema,
36 type CanonicalToolDescriptor,
37 type OpenApiSourceConfig,
38 type Source,
39} from "@executor-v2/schema";
40import {
41 makeSourceManagerService,
42 type SourceManagerService,
43} from "@executor-v2/source-manager";
14import * as Effect from "effect/Effect";44import * as Effect from "effect/Effect";
15import * as Either from "effect/Either";45import * as Either from "effect/Either";
46import * as Option from "effect/Option";
47import * as Schema from "effect/Schema";
16 48
17const port = Number(Bun.env.PORT ?? 8787);49const port = Number(Bun.env.PORT ?? 8787);
50const defaultStateRootDir = resolve(import.meta.dir, "../../../.executor-v2/local");
51const stateRootDir = Bun.env.EXECUTOR_V2_STATE_DIR?.trim()
52 ? resolve(Bun.env.EXECUTOR_V2_STATE_DIR)
53 : defaultStateRootDir;
54
55const decodeSource = Schema.decodeUnknownSync(SourceSchema);
56const decodeSourceId = Schema.decodeUnknownSync(SourceIdSchema);
57const decodeWorkspaceId = Schema.decodeUnknownSync(WorkspaceIdSchema);
58const decodeOpenApiSourceConfig = Schema.decodeUnknownSync(OpenApiSourceConfigSchema);
59const decodeCanonicalToolDescriptor = Schema.decodeUnknownSync(
60 CanonicalToolDescriptorSchema,
61);
62
63const workspaceId = decodeWorkspaceId(
64 Bun.env.EXECUTOR_V2_WORKSPACE_ID?.trim() || "ws_local",
65);
66
67const BUILTIN_PROVIDER_KIND = "in_memory" as const;
68const SOURCE_ADD_TOOL_ID = "executor.sources.add";
69const SOURCE_LIST_TOOL_ID = "executor.sources.list";
70const SOURCE_REMOVE_TOOL_ID = "executor.sources.remove";
71
72const sourceAddDescriptor: CanonicalToolDescriptor = decodeCanonicalToolDescriptor({
73 providerKind: BUILTIN_PROVIDER_KIND,
74 sourceId: null,
75 workspaceId: null,
76 toolId: SOURCE_ADD_TOOL_ID,
77 name: "executor.sources.add",
78 description: "Add a source and probe OpenAPI connectivity",
79 invocationMode: "in_memory",
80 availability: "local_only",
81 providerPayload: {
82 operation: "add",
83 },
84});
85
86const sourceListDescriptor: CanonicalToolDescriptor = decodeCanonicalToolDescriptor({
87 providerKind: BUILTIN_PROVIDER_KIND,
88 sourceId: null,
89 workspaceId: null,
90 toolId: SOURCE_LIST_TOOL_ID,
91 name: "executor.sources.list",
92 description: "List configured sources",
93 invocationMode: "in_memory",
94 availability: "local_only",
95 providerPayload: {
96 operation: "list",
97 },
98});
99
100const sourceRemoveDescriptor: CanonicalToolDescriptor =
101 decodeCanonicalToolDescriptor({
102 providerKind: BUILTIN_PROVIDER_KIND,
103 sourceId: null,
104 workspaceId: null,
105 toolId: SOURCE_REMOVE_TOOL_ID,
106 name: "executor.sources.remove",
107 description: "Remove a source by id or name",
108 invocationMode: "in_memory",
109 availability: "local_only",
110 providerPayload: {
111 operation: "remove",
112 },
113 });
114
115const builtInDescriptors = [
116 sourceAddDescriptor,
117 sourceListDescriptor,
118 sourceRemoveDescriptor,
119] as const;
120
121type LocalPersistenceContext = {
122 sourceStore: SourceStore;
123 artifactStore: ToolArtifactStore;
124 sourceManager: SourceManagerService;
125};
126
127type SourceCredentialMode = "none" | "api_key" | "bearer";
128
129type AddSourceArgs = {
130 kind: "openapi" | "mcp" | "graphql" | "internal";
131 name?: string;
132 endpoint: string;
133 credential?: {
134 mode: SourceCredentialMode;
135 value?: string;
136 headerName?: string;
137 };
138 staticHeaders?: Record<string, string>;
139};
140
141type RemoveSourceArgs = {
142 sourceId?: string;
143 name?: string;
144};
18 145
19const runtimeRegistry = makeRuntimeAdapterRegistry([146const runtimeRegistry = makeRuntimeAdapterRegistry([
20 makeLocalInProcessRuntimeAdapter(),147 makeLocalInProcessRuntimeAdapter(),
@@ -22,11 +149,34 @@ const runtimeRegistry = makeRuntimeAdapterRegistry([ · 1 unmodified line
22 makeCloudflareWorkerLoaderRuntimeAdapter(),149 makeCloudflareWorkerLoaderRuntimeAdapter(),
23]);150]);
24 151
25const toolRegistry = makeToolProviderRegistry([]);152const localPersistencePromise = Effect.runPromise(
153 Effect.gen(function* () {
154 const sourceStore = yield* makeLocalSourceStore({ rootDir: stateRootDir });
155 const artifactStore = yield* makeLocalToolArtifactStore({ rootDir: stateRootDir });
156
157 return {
158 sourceStore,
159 artifactStore,
160 sourceManager: makeSourceManagerService(artifactStore),
161 } satisfies LocalPersistenceContext;
162 }).pipe(Effect.provide(BunContext.layer)),
163);
26 164
27const errorToText = (error: unknown): string => {165const errorToText = (error: unknown): string => {
166 const extractDetails = (value: unknown): string | null => {
167 if (typeof value === "object" && value !== null && "details" in value) {
168 const details = (value as { details?: unknown }).details;
169 if (typeof details === "string" && details.length > 0) {
170 return details;
171 }
172 }
173
174 return null;
175 };
176
28 if (error instanceof Error) {177 if (error instanceof Error) {
29 return error.message;178 const details = extractDetails(error);
179 return details ? `${error.message}\n${details}` : error.message;
30 }180 }
31 181
32 if (182 if (
@@ -35,22 +185,596 @@ const errorToText = (error: unknown): string => { · 2 unmodified lines
35 "message" in error &&185 "message" in error &&
36 typeof error.message === "string"186 typeof error.message === "string"
37 ) {187 ) {
38 return error.message;188 const details = extractDetails(error);
189 return details ? `${error.message}\n${details}` : error.message;
39 }190 }
40 191
41 return String(error);192 return String(error);
42};193};
43 194
195const asRecord = (value: unknown): Record<string, unknown> | null => {
196 if (typeof value !== "object" || value === null || Array.isArray(value)) {
197 return null;
198 }
199
200 return value as Record<string, unknown>;
201};
202
203const toSlug = (value: string): string => {
204 const slug = value
205 .trim()
206 .toLowerCase()
207 .replace(/[^a-z0-9]+/g, "_")
208 .replace(/^_+|_+$/g, "");
209
210 return slug.length > 0 ? slug : "source";
211};
212
213const defaultSourceName = (endpoint: string): string => {
214 try {
215 const parsed = new URL(endpoint);
216 return parsed.hostname;
217 } catch {
218 return endpoint;
219 }
220};
221
222const parseOpenApiSourceConfig = (source: Source): OpenApiSourceConfig | null => {
223 const trimmed = source.configJson.trim();
224 if (trimmed.length === 0) {
225 return null;
226 }
227
228 try {
229 const parsed = JSON.parse(trimmed) as unknown;
230 return decodeOpenApiSourceConfig(parsed);
231 } catch {
232 return null;
233 }
234};
235
236const buildOpenApiSourceConfig = (input: AddSourceArgs): OpenApiSourceConfig => ({
237 type: "openapi",
238 auth: input.credential
239 ? {
240 mode: input.credential.mode,
241 headerName: input.credential.headerName,
242 value: input.credential.value,
243 }
244 : {
245 mode: "none",
246 },
247 staticHeaders: input.staticHeaders,
248});
249
250const configuredHeadersForOpenApi = (
251 config: OpenApiSourceConfig,
252): { headers: Headers; error: string | null } => {
253 const headers = new Headers();
254
255 if (config.staticHeaders) {
256 for (const [key, value] of Object.entries(config.staticHeaders)) {
257 headers.set(key, value);
258 }
259 }
260
261 const auth = config.auth;
262 if (!auth || auth.mode === "none") {
263 return { headers, error: null };
264 }
265
266 const credentialValue = auth.value?.trim();
267 if (!credentialValue) {
268 return {
269 headers,
270 error: "Configured credential mode requires a value",
271 };
272 }
273
274 if (auth.mode === "api_key") {
275 headers.set(auth.headerName?.trim() || "x-api-key", credentialValue);
276 return { headers, error: null };
277 }
278
279 headers.set("authorization", `Bearer ${credentialValue}`);
280 return { headers, error: null };
281};
282
283const sourceSummary = (source: Source, toolCount: number) => {
284 const config = parseOpenApiSourceConfig(source);
285
286 return {
287 id: source.id,
288 workspaceId: source.workspaceId,
289 name: source.name,
290 kind: source.kind,
291 endpoint: source.endpoint,
292 status: source.status,
293 enabled: source.enabled,
294 sourceHash: source.sourceHash,
295 lastError: source.lastError,
296 toolCount,
297 credential: config?.auth
298 ? {
299 mode: config.auth.mode,
300 headerName: config.auth.headerName ?? null,
301 configured: Boolean(config.auth.value && config.auth.value.trim().length > 0),
302 }
303 : null,
304 };
305};
306
307const parseSourceCredentialMode = (value: unknown): SourceCredentialMode => {
308 if (value === "none" || value === "api_key" || value === "bearer") {
309 return value;
310 }
311
312 throw new Error("credential.mode must be one of: none, api_key, bearer");
313};
314
315const parseAddSourceArgs = (args: unknown): AddSourceArgs => {
316 const record = asRecord(args);
317 if (!record) {
318 throw new Error("executor.sources.add expects an object argument");
319 }
320
321 const endpoint = record.endpoint;
322 if (typeof endpoint !== "string" || endpoint.trim().length === 0) {
323 throw new Error("executor.sources.add requires endpoint");
324 }
325
326 const kind = record.kind;
327 const normalizedKind =
328 kind === "openapi" || kind === "mcp" || kind === "graphql" || kind === "internal"
329 ? kind
330 : "openapi";
331
332 const name = typeof record.name === "string" ? record.name : undefined;
333
334 let credential: AddSourceArgs["credential"];
335 const credentialRecord = asRecord(record.credential);
336 if (credentialRecord) {
337 credential = {
338 mode: parseSourceCredentialMode(credentialRecord.mode),
339 value:
340 typeof credentialRecord.value === "string" ? credentialRecord.value : undefined,
341 headerName:
342 typeof credentialRecord.headerName === "string"
343 ? credentialRecord.headerName
344 : undefined,
345 };
346 }
347
348 let staticHeaders: Record<string, string> | undefined;
349 const staticHeadersRecord = asRecord(record.staticHeaders);
350 if (staticHeadersRecord) {
351 staticHeaders = {};
352
353 for (const [key, value] of Object.entries(staticHeadersRecord)) {
354 if (typeof value !== "string") {
355 throw new Error("staticHeaders values must be strings");
356 }
357
358 staticHeaders[key] = value;
359 }
360 }
361
362 return {
363 kind: normalizedKind,
364 name,
365 endpoint,
366 credential,
367 staticHeaders,
368 };
369};
370
371const parseRemoveSourceArgs = (args: unknown): RemoveSourceArgs => {
372 const record = asRecord(args);
373 if (!record) {
374 throw new Error("executor.sources.remove expects an object argument");
375 }
376
377 const sourceId = typeof record.sourceId === "string" ? record.sourceId : undefined;
378 const name = typeof record.name === "string" ? record.name : undefined;
379
380 return {
381 sourceId,
382 name,
383 };
384};
385
386const addSource = async (
387 context: LocalPersistenceContext,
388 args: AddSourceArgs,
389): Promise<{ source: ReturnType<typeof sourceSummary>; toolCount: number }> => {
390 if (args.kind !== "openapi") {
391 throw new Error(`Source kind '${args.kind}' is not supported yet (MVP supports openapi)`);
392 }
393
394 const endpoint = args.endpoint.trim();
395 if (endpoint.length === 0) {
396 throw new Error("endpoint is required");
397 }
398
399 const name = args.name?.trim() || defaultSourceName(endpoint);
400 const config = buildOpenApiSourceConfig(args);
401 const configJson = JSON.stringify(config);
402
403 const existingSources = await Effect.runPromise(
404 context.sourceStore.listByWorkspace(workspaceId),
405 );
406 const existingSource = existingSources.find(
407 (source) => source.kind === "openapi" && source.endpoint === endpoint,
408 );
409
410 const now = Date.now();
411 const sourceId =
412 existingSource?.id ??
413 decodeSourceId(
414 `src_${toSlug(name)}_${crypto.randomUUID().replace(/-/g, "").slice(0, 8)}`,
415 );
416
417 const probingSource = decodeSource({
418 id: sourceId,
419 workspaceId,
420 name,
421 kind: "openapi",
422 endpoint,
423 status: "probing",
424 enabled: true,
425 configJson,
426 sourceHash: existingSource?.sourceHash ?? null,
427 lastError: null,
428 createdAt: existingSource?.createdAt ?? now,
429 updatedAt: now,
430 });
431
432 await Effect.runPromise(context.sourceStore.upsert(probingSource));
433
434 const configuredHeaders = configuredHeadersForOpenApi(config);
435 if (configuredHeaders.error) {
436 const authRequiredSource = decodeSource({
437 ...probingSource,
438 status: "auth_required",
439 sourceHash: null,
440 lastError: configuredHeaders.error,
441 updatedAt: Date.now(),
442 });
443
444 await Effect.runPromise(context.sourceStore.upsert(authRequiredSource));
445
446 return {
447 source: sourceSummary(authRequiredSource, 0),
448 toolCount: 0,
449 };
450 }
451
452 let response: Response;
453 try {
454 response = await fetch(endpoint, {
455 method: "GET",
456 headers: configuredHeaders.headers,
457 });
458 } catch (cause) {
459 const failedSource = decodeSource({
460 ...probingSource,
461 status: "error",
462 sourceHash: null,
463 lastError: `Failed to fetch OpenAPI spec: ${errorToText(cause)}`,
464 updatedAt: Date.now(),
465 });
466
467 await Effect.runPromise(context.sourceStore.upsert(failedSource));
468
469 return {
470 source: sourceSummary(failedSource, 0),
471 toolCount: 0,
472 };
473 }
474
475 if (response.status === 401 || response.status === 403) {
476 const authRequiredSource = decodeSource({
477 ...probingSource,
478 status: "auth_required",
479 sourceHash: null,
480 lastError: `OpenAPI endpoint responded with HTTP ${response.status}`,
481 updatedAt: Date.now(),
482 });
483
484 await Effect.runPromise(context.sourceStore.upsert(authRequiredSource));
485
486 return {
487 source: sourceSummary(authRequiredSource, 0),
488 toolCount: 0,
489 };
490 }
491
492 if (!response.ok) {
493 const failedSource = decodeSource({
494 ...probingSource,
495 status: "error",
496 sourceHash: null,
497 lastError: `OpenAPI endpoint responded with HTTP ${response.status}`,
498 updatedAt: Date.now(),
499 });
500
501 await Effect.runPromise(context.sourceStore.upsert(failedSource));
502
503 return {
504 source: sourceSummary(failedSource, 0),
505 toolCount: 0,
506 };
507 }
508
509 let openApiSpec: unknown;
510 try {
511 openApiSpec = await response.json();
512 } catch (cause) {
513 const failedSource = decodeSource({
514 ...probingSource,
515 status: "error",
516 sourceHash: null,
517 lastError: `Failed to parse OpenAPI response JSON: ${errorToText(cause)}`,
518 updatedAt: Date.now(),
519 });
520
521 await Effect.runPromise(context.sourceStore.upsert(failedSource));
522
523 return {
524 source: sourceSummary(failedSource, 0),
525 toolCount: 0,
526 };
527 }
528
529 const refreshResult = await Effect.runPromise(
530 Effect.either(
531 context.sourceManager.refreshOpenApiArtifact({
532 source: probingSource,
533 openApiSpec,
534 }),
535 ),
536 );
537
538 if (Either.isLeft(refreshResult)) {
539 const failedSource = decodeSource({
540 ...probingSource,
541 status: "error",
542 sourceHash: null,
543 lastError: `OpenAPI extraction failed: ${errorToText(refreshResult.left)}`,
544 updatedAt: Date.now(),
545 });
546
547 await Effect.runPromise(context.sourceStore.upsert(failedSource));
548
549 return {
550 source: sourceSummary(failedSource, 0),
551 toolCount: 0,
552 };
553 }
554
555 const connectedSource = decodeSource({
556 ...probingSource,
557 status: "connected",
558 sourceHash: refreshResult.right.artifact.sourceHash,
559 lastError: null,
560 updatedAt: Date.now(),
561 });
562
563 await Effect.runPromise(context.sourceStore.upsert(connectedSource));
564
565 return {
566 source: sourceSummary(connectedSource, refreshResult.right.artifact.toolCount),
567 toolCount: refreshResult.right.artifact.toolCount,
568 };
569};
570
571const listSources = async (
572 context: LocalPersistenceContext,
573): Promise<{ sources: ReadonlyArray<ReturnType<typeof sourceSummary>> }> => {
574 const result = await Effect.runPromise(
575 Effect.either(
576 Effect.gen(function* () {
577 const sources = yield* context.sourceStore.listByWorkspace(workspaceId);
578
579 return yield* Effect.forEach(sources, (source) =>
580 Effect.gen(function* () {
581 const artifactOption = yield* context.artifactStore.getBySource(
582 source.workspaceId,
583 source.id,
584 );
585 const artifact = Option.getOrUndefined(artifactOption);
586 return sourceSummary(source, artifact?.toolCount ?? 0);
587 }),
588 );
589 }),
590 ),
591 );
592
593 if (Either.isLeft(result)) {
594 throw new Error(errorToText(result.left));
595 }
596
597 return {
598 sources: result.right,
599 };
600};
601
602const removeSource = async (
603 context: LocalPersistenceContext,
604 args: RemoveSourceArgs,
605): Promise<{ removed: boolean }> => {
606 const targetSourceId = args.sourceId?.trim();
607 const targetName = args.name?.trim();
608
609 if (!targetSourceId && !targetName) {
610 throw new Error("Provide either sourceId or name");
611 }
612
613 const sources = await Effect.runPromise(context.sourceStore.listByWorkspace(workspaceId));
614
615 const sourceToRemove = targetSourceId
616 ? sources.find((source) => source.id === targetSourceId)
617 : sources.find((source) => source.name === targetName);
618
619 if (!sourceToRemove) {
620 return {
621 removed: false,
622 };
623 }
624
625 const removed = await Effect.runPromise(
626 context.sourceStore.removeById(workspaceId, sourceToRemove.id),
627 );
628
629 return {
630 removed,
631 };
632};
633
634const builtInToolError = (
635 operation: string,
636 message: string,
637 details: string | null,
638): ToolProviderError =>
639 new ToolProviderError({
640 operation,
641 providerKind: BUILTIN_PROVIDER_KIND,
642 message,
643 details,
644 });
645
646const makeBuiltInToolsProvider = (
647 context: LocalPersistenceContext,
648): ToolProvider => ({
649 kind: BUILTIN_PROVIDER_KIND,
650 invoke: (input) =>
651 Effect.gen(function* () {
652 if (input.tool.toolId === SOURCE_ADD_TOOL_ID) {
653 const output = yield* Effect.tryPromise({
654 try: () => addSource(context, parseAddSourceArgs(input.args)),
655 catch: (cause) =>
656 builtInToolError(
657 "invoke.add_source",
658 "Failed to add source",
659 errorToText(cause),
660 ),
661 });
662
663 return {
664 output,
665 isError: false,
666 } as const;
667 }
668
669 if (input.tool.toolId === SOURCE_LIST_TOOL_ID) {
670 const output = yield* Effect.tryPromise({
671 try: () => listSources(context),
672 catch: (cause) =>
673 builtInToolError(
674 "invoke.list_sources",
675 "Failed to list sources",
676 errorToText(cause),
677 ),
678 });
679
680 return {
681 output,
682 isError: false,
683 } as const;
684 }
685
686 if (input.tool.toolId === SOURCE_REMOVE_TOOL_ID) {
687 const output = yield* Effect.tryPromise({
688 try: () => removeSource(context, parseRemoveSourceArgs(input.args)),
689 catch: (cause) =>
690 builtInToolError(
691 "invoke.remove_source",
692 "Failed to remove source",
693 errorToText(cause),
694 ),
695 });
696
697 return {
698 output,
699 isError: false,
700 } as const;
701 }
702
703 return yield* builtInToolError(
704 "invoke",
705 `Unknown in-memory tool id: ${input.tool.toolId}`,
706 null,
707 );
708 }),
709});
710
711const collectRunnableTools = async (
712 context: LocalPersistenceContext,
713): Promise<ReadonlyArray<RuntimeRunnableTool>> => {
714 const tools: Array<RuntimeRunnableTool> = builtInDescriptors.map((descriptor) => ({
715 descriptor,
716 source: null,
717 }));
718
719 const sources = await Effect.runPromise(
720 context.sourceStore.listByWorkspace(workspaceId),
721 );
722
723 for (const source of sources) {
724 if (source.kind !== "openapi" || !source.enabled || source.status !== "connected") {
725 continue;
726 }
727
728 const artifactOption = await Effect.runPromise(
729 context.artifactStore.getBySource(source.workspaceId, source.id),
730 );
731
732 if (Option.isNone(artifactOption)) {
733 continue;
734 }
735
736 const descriptors = await Effect.runPromise(
737 openApiToolDescriptorsFromManifest(source, artifactOption.value.manifestJson),
738 );
739
740 for (const descriptor of descriptors) {
741 tools.push({
742 descriptor,
743 source,
744 });
745 }
746 }
747
748 return tools;
749};
750
44const executeTool = async (input: ExecuteToolInput): Promise<ExecuteToolResult> => {751const executeTool = async (input: ExecuteToolInput): Promise<ExecuteToolResult> => {
752 const context = await localPersistencePromise;
753
754 let runnableTools: ReadonlyArray<RuntimeRunnableTool>;
755 try {
756 runnableTools = await collectRunnableTools(context);
757 } catch (cause) {
758 return {
759 isError: true,
760 error: errorToText(cause),
761 };
762 }
763
764 const providerRegistry = makeToolProviderRegistry([
765 makeOpenApiToolProvider(),
766 makeBuiltInToolsProvider(context),
767 ]);
768
45 const result = await Effect.runPromise(769 const result = await Effect.runPromise(
46 Effect.either(770 Effect.either(
47 runtimeRegistry771 runtimeRegistry
48 .execute({772 .execute({
49 runtimeKind: input.runtimeKind ?? "local-inproc",773 runtimeKind: input.runtimeKind ?? "local-inproc",
50 code: input.code,774 code: input.code,
51 tools: [],775 tools: runnableTools,
52 })776 })
53 .pipe(Effect.provideService(ToolProviderRegistryService, toolRegistry)),777 .pipe(Effect.provideService(ToolProviderRegistryService, providerRegistry)),
54 ),778 ),
55 );779 );
56 780
@@ -79,7 +803,16 @@ const server = Bun.serve({ · 22 unmodified lines
79 port,803 port,
80 routes: {804 routes: {
81 "/healthz": {805 "/healthz": {
82 GET: () => Response.json({ ok: true, service: "pm" }, { status: 200 }),806 GET: () =>
807 Response.json(
808 {
809 ok: true,
810 service: "pm",
811 workspaceId,
812 stateRootDir,
813 },
814 { status: 200 },
815 ),
83 },816 },
84 "/mcp": {817 "/mcp": {
85 GET: handleMcp,818 GET: handleMcp,
@@ -95,3 +828,4 @@ const server = Bun.serve({ · 9 unmodified lines
95});828});
96 829
97console.log(`executor-v2 PM listening on http://127.0.0.1:${server.port}`);830console.log(`executor-v2 PM listening on http://127.0.0.1:${server.port}`);
831console.log(`executor-v2 PM state dir: ${stateRootDir}`);
v2/bun.lock+5−0
@@ -25,8 +25,13 @@ · 24 unmodified lines
25 "apps/pm": {25 "apps/pm": {
26 "name": "@executor-v2/app-pm",26 "name": "@executor-v2/app-pm",
27 "dependencies": {27 "dependencies": {
28 "@effect/platform-bun": "^0.86.0",
28 "@executor-v2/engine": "workspace:*",29 "@executor-v2/engine": "workspace:*",
29 "@executor-v2/mcp-gateway": "workspace:*",30 "@executor-v2/mcp-gateway": "workspace:*",
31 "@executor-v2/persistence-local": "workspace:*",
32 "@executor-v2/persistence-ports": "workspace:*",
33 "@executor-v2/schema": "workspace:*",
34 "@executor-v2/source-manager": "workspace:*",
30 "effect": "^3.19.8",35 "effect": "^3.19.8",
31 },36 },
32 },37 },
v2/packages/engine/README.md+1−0
@@ -6,6 +6,7 @@ Current scaffold includes: · 5 unmodified lines
6- provider contracts (`ToolProvider`) and canonical tool descriptor model6- provider contracts (`ToolProvider`) and canonical tool descriptor model
7- provider registry service (`ToolProviderRegistryService`) with `discover`/`invoke` routing7- provider registry service (`ToolProviderRegistryService`) with `discover`/`invoke` routing
8- OpenAPI provider invocation and manifest-to-descriptor conversion helpers8- OpenAPI provider invocation and manifest-to-descriptor conversion helpers
9- source-config-aware OpenAPI auth header injection (`api_key` / `bearer`)
9- minimal in-process JavaScript runner with `tools.*` proxy dispatch into provider registry10- minimal in-process JavaScript runner with `tools.*` proxy dispatch into provider registry
10- Deno subprocess runner with line-delimited JSON IPC for proxied `tools.*` calls11- Deno subprocess runner with line-delimited JSON IPC for proxied `tools.*` calls
11- runtime adapter contract + registry (`local-inproc`, `deno-subprocess`, `cloudflare-worker-loader` scaffold)12- runtime adapter contract + registry (`local-inproc`, `deno-subprocess`, `cloudflare-worker-loader` scaffold)
v2/packages/engine/src/deno-subprocess-runner.ts+30−5
@@ -102,12 +102,36 @@ const missingToolBindingError = (toolId: string): DenoSubprocessRunnerError => · 101 unmodified lines
102 details: null,102 details: null,
103 });103 });
104 104
105const toolCallFailedError = (toolId: string): DenoSubprocessRunnerError =>105const formatToolErrorDetails = (output: unknown): string | null => {
106 new DenoSubprocessRunnerError({106 if (output === undefined) {
107 return null;
108 }
109
110 if (typeof output === "string") {
111 return output;
112 }
113
114 try {
115 const serialized = JSON.stringify(output, null, 2);
116 return typeof serialized === "string" ? serialized : String(output);
117 } catch {
118 return String(output);
119 }
120};
121
122const toolCallFailedError = (
123 toolId: string,
124 output: unknown,
125): DenoSubprocessRunnerError => {
126 const details = formatToolErrorDetails(output);
127 const baseMessage = `Tool call returned error: ${toolId}`;
128
129 return new DenoSubprocessRunnerError({
107 operation: "invoke_tool",130 operation: "invoke_tool",
108 message: `Tool call returned error: ${toolId}`,131 message: details ? `${baseMessage}\n${details}` : baseMessage,
109 details: null,132 details,
110 });133 });
134};
111 135
112const parseWorkerMessageError = (136const parseWorkerMessageError = (
113 line: string,137 line: string,
@@ -231,11 +255,12 @@ const handleToolCall = ( · 117 unmodified lines
231 });255 });
232 256
233 if (invokeResult.isError) {257 if (invokeResult.isError) {
258 const error = toolCallFailedError(message.toolId, invokeResult.output);
234 yield* writeMessage(stdin, {259 yield* writeMessage(stdin, {
235 type: "tool_result",260 type: "tool_result",
236 requestId: message.requestId,261 requestId: message.requestId,
237 ok: false,262 ok: false,
238 error: toolCallFailedError(message.toolId).message,263 error: error.message,
239 });264 });
240 return;265 return;
241 }266 }
v2/packages/engine/src/deno-subprocess-worker.mjs+28−1
@@ -30,10 +30,37 @@ const createToolCaller = (toolId) => (args) => · 29 unmodified lines
30 });30 });
31 });31 });
32 32
33const assignToolCaller = (toolsRoot, toolId, callTool) => {
34 toolsRoot[toolId] = callTool;
35
36 const segments = toolId.split(".").filter((segment) => segment.length > 0);
37 if (segments.length <= 1) {
38 return;
39 }
40
41 let cursor = toolsRoot;
42 for (let index = 0; index < segments.length - 1; index += 1) {
43 const segment = segments[index];
44 const current = cursor[segment];
45
46 if (!current || typeof current !== "object" || Array.isArray(current)) {
47 const next = Object.create(null);
48 cursor[segment] = next;
49 cursor = next;
50 continue;
51 }
52
53 cursor = current;
54 }
55
56 const leafSegment = segments[segments.length - 1];
57 cursor[leafSegment] = callTool;
58};
59
33const runUserCode = async (code, toolIds) => {60const runUserCode = async (code, toolIds) => {
34 const tools = Object.create(null);61 const tools = Object.create(null);
35 for (const toolId of toolIds) {62 for (const toolId of toolIds) {
36 tools[toolId] = createToolCaller(toolId);63 assignToolCaller(tools, toolId, createToolCaller(toolId));
37 }64 }
38 65
39 const execute = new Function(66 const execute = new Function(
v2/packages/engine/src/local-runner.ts+79−15
@@ -33,19 +33,53 @@ const duplicateToolIdError = (toolId: string): LocalCodeRunnerError => · 32 unmodified lines
33 details: null,33 details: null,
34 });34 });
35 35
36const toolCallFailedError = (toolId: string): LocalCodeRunnerError =>36const formatToolErrorDetails = (output: unknown): string | null => {
37 new LocalCodeRunnerError({37 if (output === undefined) {
38 return null;
39 }
40
41 if (typeof output === "string") {
42 return output;
43 }
44
45 try {
46 const serialized = JSON.stringify(output, null, 2);
47 return typeof serialized === "string" ? serialized : String(output);
48 } catch {
49 return String(output);
50 }
51};
52
53const toolCallFailedError = (
54 toolId: string,
55 output: unknown,
56): LocalCodeRunnerError => {
57 const details = formatToolErrorDetails(output);
58 const baseMessage = `Tool call returned error: ${toolId}`;
59
60 return new LocalCodeRunnerError({
38 operation: "invoke_tool",61 operation: "invoke_tool",
39 message: `Tool call returned error: ${toolId}`,62 message: details ? `${baseMessage}\n${details}` : baseMessage,
40 details: null,63 details,
41 });64 });
65};
42 66
43const toExecutionError = (cause: unknown): LocalCodeRunnerError =>67const toExecutionError = (cause: unknown): LocalCodeRunnerError => {
44 new LocalCodeRunnerError({68 let details = cause instanceof Error ? cause.stack ?? cause.message : String(cause);
69
70 if (typeof cause === "object" && cause !== null && "details" in cause) {
71 const causeDetails = (cause as { details?: unknown }).details;
72 if (typeof causeDetails === "string" && causeDetails.length > 0) {
73 details = `${details}\n${causeDetails}`;
74 }
75 }
76
77 return new LocalCodeRunnerError({
45 operation: "execute",78 operation: "execute",
46 message: "JavaScript execution failed",79 message: "JavaScript execution failed",
47 details: cause instanceof Error ? cause.stack ?? cause.message : String(cause),80 details,
48 });81 });
82};
49 83
50const buildToolBindings = (84const buildToolBindings = (
51 tools: ReadonlyArray<RunnableTool>,85 tools: ReadonlyArray<RunnableTool>,
@@ -64,17 +98,47 @@ const buildToolBindings = ( · 12 unmodified lines
64 return byToolId;98 return byToolId;
65 });99 });
66 100
101const assignToolBinding = (
102 toolsRoot: Record<string, unknown>,
103 toolId: string,
104 callTool: (args: unknown) => Promise<unknown>,
105): void => {
106 toolsRoot[toolId] = callTool;
107
108 const segments = toolId.split(".").filter((segment) => segment.length > 0);
109 if (segments.length <= 1) {
110 return;
111 }
112
113 let cursor: Record<string, unknown> = toolsRoot;
114 for (let index = 0; index < segments.length - 1; index += 1) {
115 const segment = segments[index];
116 const current = cursor[segment];
117
118 if (typeof current !== "object" || current === null || Array.isArray(current)) {
119 const next = Object.create(null) as Record<string, unknown>;
120 cursor[segment] = next;
121 cursor = next;
122 continue;
123 }
124
125 cursor = current as Record<string, unknown>;
126 }
127
128 const leafSegment = segments[segments.length - 1];
129 cursor[leafSegment] = callTool;
130};
131
67const runJavaScript = (132const runJavaScript = (
68 code: string,133 code: string,
69 toolsObject: Record<string, (args: unknown) => Promise<unknown>>,134 toolsObject: Record<string, unknown>,
70): Effect.Effect<unknown, LocalCodeRunnerError> =>135): Effect.Effect<unknown, LocalCodeRunnerError> =>
71 Effect.tryPromise({136 Effect.tryPromise({
72 try: async () => {137 try: async () => {
73 const execute = new Function(138 const execute = new Function(
74 "tools",139 "tools",
75 `"use strict"; return (async () => {\n${code}\n})();`,140 `"use strict"; return (async () => {\n${code}\n})();`,
76 ) as (tools: Record<string, (args: unknown) => Promise<unknown>>) =>141 ) as (tools: Record<string, unknown>) => Promise<unknown>;
77 Promise<unknown>;
78 142
79 return await execute(toolsObject);143 return await execute(toolsObject);
80 },144 },
@@ -94,11 +158,10 @@ export const executeJavaScriptWithTools = ( · 13 unmodified lines
94 const runPromise = Runtime.runPromise(runtime);158 const runPromise = Runtime.runPromise(runtime);
95 const toolBindings = yield* buildToolBindings(input.tools);159 const toolBindings = yield* buildToolBindings(input.tools);
96 160
97 const toolsObject: Record<string, (args: unknown) => Promise<unknown>> =161 const toolsObject: Record<string, unknown> = Object.create(null);
98 Object.create(null);
99 162
100 for (const [toolId, binding] of toolBindings.entries()) {163 for (const [toolId, binding] of toolBindings.entries()) {
101 toolsObject[toolId] = (args: unknown) =>164 assignToolBinding(toolsObject, toolId, (args: unknown) =>
102 runPromise(165 runPromise(
103 registry166 registry
104 .invoke({167 .invoke({
@@ -109,11 +172,12 @@ export const executeJavaScriptWithTools = ( · 4 unmodified lines
109 .pipe(172 .pipe(
110 Effect.flatMap((result) =>173 Effect.flatMap((result) =>
111 result.isError174 result.isError
112 ? toolCallFailedError(toolId)175 ? toolCallFailedError(toolId, result.output)
113 : Effect.succeed(result.output),176 : Effect.succeed(result.output),
114 ),177 ),
115 ),178 ),
116 );179 ),
180 );
117 }181 }
118 182
119 return yield* runJavaScript(input.code, toolsObject);183 return yield* runJavaScript(input.code, toolsObject);
v2/packages/engine/src/openapi-execution.test.ts+124−0
@@ -241,4 +241,128 @@ return await tools.getUser({ · 240 unmodified lines
241 expect(testServer.requests[0]?.apiKey).toBe("sk_test");241 expect(testServer.requests[0]?.apiKey).toBe("sk_test");
242 }),242 }),
243 );243 );
244
245 it.scoped("uses configured source credentials when required header arg is omitted", () =>
246 Effect.gen(function* () {
247 const testServer = yield* makeTestServer;
248
249 const openApiSpec = {
250 openapi: "3.1.0",
251 paths: {
252 "/users/{userId}": {
253 get: {
254 operationId: "getUser",
255 parameters: [
256 {
257 name: "userId",
258 in: "path",
259 required: true,
260 },
261 {
262 name: "verbose",
263 in: "query",
264 },
265 {
266 name: "x-api-key",
267 in: "header",
268 required: true,
269 },
270 ],
271 },
272 },
273 },
274 };
275
276 const source: Source = decodeSource({
277 id: "src_openapi",
278 workspaceId: "ws_local",
279 name: "local-openapi",
280 kind: "openapi",
281 endpoint: testServer.baseUrl,
282 status: "connected",
283 enabled: true,
284 configJson: JSON.stringify({
285 type: "openapi",
286 auth: {
287 mode: "api_key",
288 headerName: "x-api-key",
289 value: "sk_from_config",
290 },
291 }),
292 sourceHash: null,
293 lastError: null,
294 createdAt: Date.now(),
295 updatedAt: Date.now(),
296 });
297
298 const artifactsByKey = new Map<string, ToolArtifact>();
299 const artifactStore: ToolArtifactStore = {
300 getBySource: (workspaceId: Source["workspaceId"], sourceId: Source["id"]) =>
301 Effect.succeed(
302 Option.fromNullable(artifactsByKey.get(`${workspaceId}:${sourceId}`)),
303 ),
304 upsert: (artifact: ToolArtifact) =>
305 Effect.sync(() => {
306 artifactsByKey.set(
307 `${artifact.workspaceId}:${artifact.sourceId}`,
308 artifact,
309 );
310 }),
311 };
312 const sourceManager = makeSourceManagerService(artifactStore);
313
314 const refreshResult = yield* sourceManager.refreshOpenApiArtifact({
315 source,
316 openApiSpec,
317 });
318
319 const tools = yield* openApiToolDescriptorsFromManifest(
320 source,
321 refreshResult.artifact.manifestJson,
322 );
323
324 const getUserTool = tools.find((tool) => tool.toolId === "getUser");
325 if (!getUserTool) {
326 throw new Error("expected getUser tool");
327 }
328
329 const registry = makeToolProviderRegistry([makeOpenApiToolProvider()]);
330
331 const executionResult = yield* executeJavaScriptWithTools({
332 code: `
333return await tools.getUser({
334 userId: "u123",
335 verbose: "true"
336});
337`,
338 tools: [
339 {
340 descriptor: getUserTool,
341 source,
342 },
343 ],
344 }).pipe(
345 Effect.provideService(ToolProviderRegistryService, registry),
346 );
347
348 const output = executionResult as {
349 status: number;
350 body: {
351 id: string;
352 verbose: boolean;
353 apiKey: string | null;
354 };
355 };
356
357 expect(output.status).toBe(200);
358 expect(output.body.id).toBe("u123");
359 expect(output.body.verbose).toBe(true);
360 expect(output.body.apiKey).toBe("sk_from_config");
361
362 expect(testServer.requests).toHaveLength(1);
363 expect(testServer.requests[0]?.path).toBe("/users/u123");
364 expect(testServer.requests[0]?.query).toBe("?verbose=true");
365 expect(testServer.requests[0]?.apiKey).toBe("sk_from_config");
366 }),
367 );
244});368});
v2/packages/engine/src/openapi-provider.ts+79−0
@@ -1,9 +1,11 @@
1import {1import {
2 OpenApiInvocationPayloadSchema,2 OpenApiInvocationPayloadSchema,
3 OpenApiSourceConfigSchema,
3 OpenApiToolManifestSchema,4 OpenApiToolManifestSchema,
4 type CanonicalToolDescriptor,5 type CanonicalToolDescriptor,
5 type OpenApiCanonicalToolDescriptor,6 type OpenApiCanonicalToolDescriptor,
6 type OpenApiInvocationPayload,7 type OpenApiInvocationPayload,
8 type OpenApiSourceConfig,
7 type OpenApiToolManifest,9 type OpenApiToolManifest,
8 type Source,10 type Source,
9} from "@executor-v2/schema";11} from "@executor-v2/schema";
@@ -27,6 +29,9 @@ const decodeOpenApiManifestJson = Schema.decodeUnknown( · 17 unmodified lines
27const decodeOpenApiInvocationPayload = Schema.decodeUnknown(29const decodeOpenApiInvocationPayload = Schema.decodeUnknown(
28 OpenApiInvocationPayloadSchema,30 OpenApiInvocationPayloadSchema,
29);31);
32const decodeOpenApiSourceConfigSync = Schema.decodeUnknownSync(
33 OpenApiSourceConfigSchema,
34);
30const decodeOpenApiToolArgs = Schema.decodeUnknown(OpenApiToolArgsSchema);35const decodeOpenApiToolArgs = Schema.decodeUnknown(OpenApiToolArgsSchema);
31const encodeUnknownToJson = Schema.encode(Schema.parseJson(Schema.Unknown));36const encodeUnknownToJson = Schema.encode(Schema.parseJson(Schema.Unknown));
32 37
@@ -63,6 +68,72 @@ const argsValueToString = (value: unknown): string => { · 30 unmodified lines
63 return String(value);68 return String(value);
64};69};
65 70
71const parseOpenApiSourceConfig = (source: Source): OpenApiSourceConfig | null => {
72 const trimmed = source.configJson.trim();
73 if (trimmed.length === 0) {
74 return null;
75 }
76
77 try {
78 const parsed = JSON.parse(trimmed) as unknown;
79 return decodeOpenApiSourceConfigSync(parsed);
80 } catch {
81 return null;
82 }
83};
84
85const hasHeaderCaseInsensitive = (headers: Headers, headerName: string): boolean => {
86 const target = headerName.toLowerCase();
87
88 for (const [key] of headers.entries()) {
89 if (key.toLowerCase() === target) {
90 return true;
91 }
92 }
93
94 return false;
95};
96
97const applySourceConfiguredHeaders = (
98 source: Source,
99 headers: Headers,
100): Effect.Effect<void, ToolProviderError> =>
101 Effect.gen(function* () {
102 const config = parseOpenApiSourceConfig(source);
103 if (!config) {
104 return;
105 }
106
107 if (config.staticHeaders) {
108 for (const [key, value] of Object.entries(config.staticHeaders)) {
109 headers.set(key, value);
110 }
111 }
112
113 const auth = config.auth;
114 if (!auth || auth.mode === "none") {
115 return;
116 }
117
118 const credentialValue = auth.value?.trim();
119 if (!credentialValue) {
120 return yield* new ToolProviderError({
121 operation: "invoke.auth",
122 providerKind: "openapi",
123 message: "Configured source credential is missing a value",
124 details: source.id,
125 });
126 }
127
128 if (auth.mode === "api_key") {
129 const headerName = auth.headerName?.trim() || "x-api-key";
130 headers.set(headerName, credentialValue);
131 return;
132 }
133
134 headers.set("authorization", `Bearer ${credentialValue}`);
135 });
136
66const replacePathTemplate = (137const replacePathTemplate = (
67 pathTemplate: string,138 pathTemplate: string,
68 args: OpenApiToolArgs,139 args: OpenApiToolArgs,
@@ -118,6 +189,7 @@ const buildFetchRequest = ( · 49 unmodified lines
118 const url = new URL(resolvedPath, source.endpoint);189 const url = new URL(resolvedPath, source.endpoint);
119 190
120 const headers = new Headers();191 const headers = new Headers();
192 yield* applySourceConfiguredHeaders(source, headers);
121 const cookieParts: Array<string> = [];193 const cookieParts: Array<string> = [];
122 194
123 for (const parameter of payload.parameters) {195 for (const parameter of payload.parameters) {
@@ -128,6 +200,13 @@ const buildFetchRequest = ( · 4 unmodified lines
128 const parameterValue = args[parameter.name];200 const parameterValue = args[parameter.name];
129 if (parameterValue === undefined || parameterValue === null) {201 if (parameterValue === undefined || parameterValue === null) {
130 if (parameter.required) {202 if (parameter.required) {
203 if (
204 parameter.location === "header" &&
205 hasHeaderCaseInsensitive(headers, parameter.name)
206 ) {
207 continue;
208 }
209
131 return yield* new ToolProviderError({210 return yield* new ToolProviderError({
132 operation: "invoke.validate_args",211 operation: "invoke.validate_args",
133 providerKind: "openapi",212 providerKind: "openapi",
v2/packages/engine/src/runtime-adapters.test.ts+53−0
@@ -35,6 +35,18 @@ const sumToolDescriptor: CanonicalToolDescriptor = decodeCanonicalToolDescriptor · 34 unmodified lines
35 providerPayload: null,35 providerPayload: null,
36});36});
37 37
38const failToolDescriptor: CanonicalToolDescriptor = decodeCanonicalToolDescriptor({
39 providerKind: "in_memory",
40 sourceId: null,
41 workspaceId: null,
42 toolId: "fail",
43 name: "fail",
44 description: null,
45 invocationMode: "in_memory",
46 availability: "local_only",
47 providerPayload: null,
48});
49
38describe("runtime adapters", () => {50describe("runtime adapters", () => {
39 it.effect("executes with local-inproc adapter via runtime registry", () =>51 it.effect("executes with local-inproc adapter via runtime registry", () =>
40 Effect.gen(function* () {52 Effect.gen(function* () {
@@ -67,6 +79,47 @@ describe("runtime adapters", () => { · 26 unmodified lines
67 }),79 }),
68 );80 );
69 81
82 it.effect("surfaces tool-provider error output details for local-inproc", () =>
83 Effect.gen(function* () {
84 const failingProvider: ToolProvider = {
85 kind: "in_memory",
86 invoke: () =>
87 Effect.succeed({
88 output: {
89 status: 400,
90 body: {
91 message: "domain is required",
92 },
93 },
94 isError: true,
95 } as const),
96 };
97
98 const toolRegistry = makeToolProviderRegistry([failingProvider]);
99 const runtimeRegistry = makeRuntimeAdapterRegistry([
100 makeLocalInProcessRuntimeAdapter(),
101 ]);
102
103 const result = yield* Effect.either(
104 runtimeRegistry
105 .execute({
106 runtimeKind: "local-inproc",
107 code: "return await tools.fail({});",
108 tools: [{ descriptor: failToolDescriptor, source: null }],
109 })
110 .pipe(Effect.provideService(ToolProviderRegistryService, toolRegistry)),
111 );
112
113 expect(Either.isLeft(result)).toBe(true);
114 if (Either.isLeft(result)) {
115 if ("details" in result.left && typeof result.left.details === "string") {
116 expect(result.left.details).toContain("Tool call returned error: fail");
117 expect(result.left.details).toContain("domain is required");
118 }
119 }
120 }),
121 );
122
70 it.effect("returns typed not-implemented error for cloudflare adapter", () =>123 it.effect("returns typed not-implemented error for cloudflare adapter", () =>
71 Effect.gen(function* () {124 Effect.gen(function* () {
72 const toolRegistry = makeToolProviderRegistry([]);125 const toolRegistry = makeToolProviderRegistry([]);
v2/packages/mcp-gateway/README.md+2−1
@@ -5,4 +5,5 @@ Bare-minimum MCP gateway for Executor v2. · 4 unmodified lines
5Current scaffold includes:5Current scaffold includes:
6- MCP server built with `@modelcontextprotocol/sdk` in `src/server.ts`6- MCP server built with `@modelcontextprotocol/sdk` in `src/server.ts`
7- streamable HTTP request handling (`handleMcpHttpRequest`)7- streamable HTTP request handling (`handleMcpHttpRequest`)
8- tools: `executor.ping` and `executor.execute` (the execute tool is host-wired via callback)8- single top-level tool: `executor.execute`
9- runtime/control operations are intended to flow through code execution via `tools.*` (for example `tools.executor.sources.*`)
v2/packages/mcp-gateway/src/server.ts+12−46
@@ -24,18 +24,13 @@ export type McpGatewayOptions = { · 23 unmodified lines
24 target: GatewayTarget;24 target: GatewayTarget;
25 serverName?: string;25 serverName?: string;
26 serverVersion?: string;26 serverVersion?: string;
27 execute?: (input: ExecuteToolInput) => Promise<ExecuteToolResult>;27 execute: (input: ExecuteToolInput) => Promise<ExecuteToolResult>;
28};28};
29 29
30const DEFAULT_SERVER_NAME = "executor-v2";30const DEFAULT_SERVER_NAME = "executor-v2";
31const DEFAULT_SERVER_VERSION = "0.0.0";31const DEFAULT_SERVER_VERSION = "0.0.0";
32const STUB_TOOL_NAME = "executor.ping";
33const EXECUTE_TOOL_NAME = "executor.execute";32const EXECUTE_TOOL_NAME = "executor.execute";
34 33
35const PingToolInput = z.object({
36 message: z.string().optional(),
37});
38
39const ExecuteToolInputSchema = z.object({34const ExecuteToolInputSchema = z.object({
40 code: z.string(),35 code: z.string(),
41 runtimeKind: z36 runtimeKind: z
@@ -49,60 +44,31 @@ const contentText = (value: unknown): string => { · 7 unmodified lines
49 }44 }
50 45
51 try {46 try {
52 return JSON.stringify(value, null, 2);47 const serialized = JSON.stringify(value, null, 2);
48 if (typeof serialized === "string") {
49 return serialized;
50 }
53 } catch {51 } catch {
54 return String(value);52 // fall through
55 }53 }
54
55 return String(value);
56};56};
57 57
58const createStubMcpServer = (options: McpGatewayOptions): McpServer => {58const createMcpServer = (options: McpGatewayOptions): McpServer => {
59 const mcp = new McpServer({59 const mcp = new McpServer({
60 name: options.serverName ?? DEFAULT_SERVER_NAME,60 name: options.serverName ?? DEFAULT_SERVER_NAME,
61 version: options.serverVersion ?? DEFAULT_SERVER_VERSION,61 version: options.serverVersion ?? DEFAULT_SERVER_VERSION,
62 });62 });
63 63
64 mcp.registerTool(
65 STUB_TOOL_NAME,
66 {
67 description: "Stub MCP tool that replies with pong",
68 inputSchema: PingToolInput,
69 },
70 async (input: { message?: string }) => {
71 const text = input.message
72 ? `pong (${options.target}) - ${input.message}`
73 : `pong (${options.target})`;
74
75 return {
76 content: [
77 {
78 type: "text" as const,
79 text,
80 },
81 ],
82 isError: false,
83 };
84 },
85 );
86
87 mcp.registerTool(64 mcp.registerTool(
88 EXECUTE_TOOL_NAME,65 EXECUTE_TOOL_NAME,
89 {66 {
90 description: "Execute JavaScript against configured runtime adapter",67 description:
68 "Execute JavaScript against configured runtime adapter. Runtime receives tools namespace, including executor source controls under tools.executor.sources.*.",
91 inputSchema: ExecuteToolInputSchema,69 inputSchema: ExecuteToolInputSchema,
92 },70 },
93 async (input: ExecuteToolInput) => {71 async (input: ExecuteToolInput) => {
94 if (!options.execute) {
95 return {
96 content: [
97 {
98 type: "text" as const,
99 text: "executor.execute is not configured for this gateway target",
100 },
101 ],
102 isError: true,
103 };
104 }
105
106 try {72 try {
107 const result = await options.execute(input);73 const result = await options.execute(input);
108 return {74 return {
@@ -141,7 +107,7 @@ export const handleMcpHttpRequest = async ( · 32 unmodified lines
141 sessionIdGenerator: undefined,107 sessionIdGenerator: undefined,
142 enableJsonResponse: true,108 enableJsonResponse: true,
143 });109 });
144 const mcp = createStubMcpServer(options);110 const mcp = createMcpServer(options);
145 111
146 try {112 try {
147 await mcp.connect(transport);113 await mcp.connect(transport);
v2/packages/schema/README.md+1−0
@@ -6,5 +6,6 @@ Current scaffold includes: · 5 unmodified lines
6- branded domain IDs (`src/ids.ts`)6- branded domain IDs (`src/ids.ts`)
7- shared enums and primitives (`src/enums.ts`, `src/common.ts`)7- shared enums and primitives (`src/enums.ts`, `src/common.ts`)
8- core model schemas in `src/models/*`8- core model schemas in `src/models/*`
9- source config schema for provider/runtime auth metadata in `src/models/source-config.ts`
9- core event envelope schema in `src/models/event-envelope.ts`10- core event envelope schema in `src/models/event-envelope.ts`
10- local snapshot/WAL schemas live in `@executor-v2/persistence-local`11- local snapshot/WAL schemas live in `@executor-v2/persistence-local`
v2/packages/schema/src/index.ts+1−0
@@ -5,6 +5,7 @@ export * from "./ids"; · 4 unmodified lines
5export * from "./models/profile";5export * from "./models/profile";
6export * from "./models/workspace";6export * from "./models/workspace";
7export * from "./models/source";7export * from "./models/source";
8export * from "./models/source-config";
8export * from "./models/tool-artifact";9export * from "./models/tool-artifact";
9export * from "./models/credential-ref";10export * from "./models/credential-ref";
10export * from "./models/oauth-token";11export * from "./models/oauth-token";
v2/packages/schema/src/models/source-config.ts+28−0
@@ -0,0 +1,28 @@
1import { Schema } from "effect";
2
3export const OpenApiSourceAuthModeSchema = Schema.Literal(
4 "none",
5 "api_key",
6 "bearer",
7);
8
9export const OpenApiSourceAuthSchema = Schema.Struct({
10 mode: OpenApiSourceAuthModeSchema,
11 headerName: Schema.optional(Schema.String),
12 value: Schema.optional(Schema.String),
13});
14
15export const OpenApiSourceConfigSchema = Schema.Struct({
16 type: Schema.Literal("openapi"),
17 auth: Schema.optional(OpenApiSourceAuthSchema),
18 staticHeaders: Schema.optional(
19 Schema.Record({
20 key: Schema.String,
21 value: Schema.String,
22 }),
23 ),
24});
25
26export type OpenApiSourceAuthMode = typeof OpenApiSourceAuthModeSchema.Type;
27export type OpenApiSourceAuth = typeof OpenApiSourceAuthSchema.Type;
28export type OpenApiSourceConfig = typeof OpenApiSourceConfigSchema.Type;