Plugin Source Configuration Overhaul

Date: 2026-05-18 Status: implemented in progress

Goal

Turn the source credential/configuration work into a full plugin overhaul instead of an incremental OpenAPI cleanup.

Implementation goal for this PR:

PR #844 should become the complete source-configuration overhaul for first-party
source plugins. OpenAPI, GraphQL, and MCP should all move onto plugin-derived
configure APIs, shared core credential bindings, shared HTTP credential helpers
where applicable, JSON-backed plugin source config, and composed shared React
credential UI. The old plugin-specific binding wrappers, endpoints, stores,
credential child tables, and duplicated UI atoms should be removed rather than
kept as compatibility shims.

By the end, OpenAPI, GraphQL, and MCP should share the same underlying source configuration architecture:

• Core owns generic source identity, scoped credential bindings, source configure dispatch, and validation.
• Plugins own their source-specific configure schemas and source config decoding.
• HTTP-ish plugins share HTTP credential config, runtime helpers, and React components through a shared HTTP package.
• Plugin-private storage remains an overridable plugin facility, backed by one shared plugin storage table rather than plugin-specific SQL tables.
• Duplicated OpenAPI/GraphQL/MCP binding endpoints, stores, React atoms, and child tables should be deleted aggressively.

The desired result is that a future improvement to HTTP credential handling such as adding a plaintext header path, changing the secret input UI, or improving OAuth flows benefits OpenAPI, GraphQL, and MCP HTTP rather than being reimplemented per plugin.

Settled Decisions

• This can be one large PR/overhaul, not a series of small incremental PRs.
• executor.sources.configure(...) should be implemented in this overhaul.
• Plugin-native configure APIs should remain:
  • executor.openapi.configure(...)
  • executor.graphql.configure(...)
  • executor.mcp.configure(...)
• Low-level binding APIs should be hidden from normal consumers where possible. They may remain exported for internal/advanced use, but product and SDK happy paths should lead with configure.
• Source config may move to JSON. Use Effect Schema at boundaries.
• Migrations are one-shot migrations. Preserve existing source configuration and credential values.
• Persist source config; derive slot manifests from source config.
• Add a shared HTTP package, likely @executor-js/plugin-http-source.
• MCP configure should be transport-discriminated.
• Shared UI components can omit MCP stdio for this pass.
• Plaintext credential values are not needed unless a current flow already has them.
• OAuth should be designed for the long-term model, not as a short-term header hack.
• UI should be hand-composed from reusable components, not generated entirely from manifests.
• GraphQL can use the same request URL/credential config for introspection for now.
• MCP may require auth at add time if auth is needed to list tools.
• Naming like request, specFetch, introspection, transport, env, and query is acceptable.

Storage Model

Do not kill plugin storage.

The overhaul should separate three storage concerns that are currently blurred:

1. First-class Executor source records.
2. Scoped credential values.
3. Plugin-private state/cache/documents.

First-Class Source Records

Sources are product entities. Core needs to list them, scope them, remove them, refresh them, attach tools/policies to them, and expose them through generic source APIs.

Source records should remain first-class, but plugin-specific source details can move into plugin-owned JSON config decoded by Effect Schema.

For example:

openapi_source
  id
  scope_id
  name
  config_json

graphql_source
  id
  scope_id
  name
  config_json

mcp_source
  id
  scope_id
  name
  config_json

The exact physical table shape can follow the repo's current storage layout, but the duplicated credential child tables should go away where they only represent source config and binding placeholders.

Plugin packages own their config schemas:

export const OpenApiStoredSourceConfig = Schema.Struct({
  request: Schema.optional(HttpRequestSourceConfig),
  specFetch: Schema.optional(HttpRequestSourceConfig),
  // OpenAPI-specific spec/base URL/operation details...
});

export const GraphqlStoredSourceConfig = Schema.Struct({
  request: Schema.optional(HttpRequestSourceConfig),
  // GraphQL-specific endpoint/schema details...
});

export const McpStoredSourceConfig = Schema.Union(
  Schema.Struct({
    transport: Schema.Literal("http"),
    request: Schema.optional(HttpRequestSourceConfig),
    // MCP HTTP details...
  }),
  Schema.Struct({
    transport: Schema.Literal("stdio"),
    command: Schema.String,
    args: Schema.Array(Schema.String),
    env: Schema.Record({ key: Schema.String, value: ProcessEnvSourceConfig }),
  }),
);

Credential Bindings

Core credential_binding remains the only place scoped credential values live.

credential_binding
  plugin_id
  source_id
  source_scope_id
  scope_id
  slot_key
  kind
  secret_id
  secret_scope_id
  connection_id
  text_value
  created_at
  updated_at

Source config declares slots and their wire-format meaning. Credential bindings store scoped values for those slots.

Example OpenAPI source config:

{
  request: {
    headers: {
      Authorization: {
        slotKey: "request.headers.authorization",
        prefix: "Bearer ",
      },
    },
    query: {},
  },
  specFetch: {
    headers: {},
    query: {},
  },
}

Example value:

credential_binding
  plugin_id = "openapi"
  source_id = "stripe"
  source_scope_id = "org_acme"
  scope_id = "user_rhys"
  slot_key = "request.headers.authorization"
  kind = "secret"
  secret_id = "stripe_api_key"

Plugin-Private Storage

Plugin storage should remain as an overridable facility.

The target is one shared table for all plugin-private storage, partitioned by plugin and collection:

plugin_storage
  plugin_id
  collection
  scope_id
  key
  data_json
  created_at
  updated_at

Primary key:

(plugin_id, collection, scope_id, key)

This is for plugin-owned private state/cache/documents:

• OAuth state caches.
• Remote metadata caches.
• Probe results.
• Background job state.
• Plugin-specific documents that are not first-class Executor sources.

The plugin storage API should remain overridable when constructing the executor or installing/creating a plugin. This is separate from source storage and credential binding storage.

Conceptually:

createExecutor({
  pluginStorage: myPluginStorageProvider,
});

or, if plugin-level override is already the local pattern:

openApiPlugin({
  storage: myPluginStorageProvider,
});

The exact wiring should preserve the repo's existing ability for users to override plugin storage. The overhaul should consolidate plugin-private storage tables, not remove plugin storage.

What We Are Deleting

Delete plugin-specific storage that only duplicates source config or credential-binding behavior.

Likely delete/replace:

• OpenAPI source binding wrappers and backing adapters.
• OpenAPI header/query/spec-fetch child tables that only store slot config.
• GraphQL header/query child tables.
• MCP header/query child tables.
• Plugin-specific binding resolvers/listers/validators.
• Plugin-specific HTTP binding endpoints.
• Plugin-specific React binding atoms.

Do not delete:

• First-class source records.
• Core credential_binding.
• Generic plugin-private storage.
• The ability to provide/override plugin storage.

Configure API Model

Core gets a plugin-derived configure dispatch:

await executor.sources.configure(source, {
  type: "openapi",
  scope: user,
  request: {
    headers: {
      Authorization: SecretId.make("stripe_api_key"),
    },
  },
});

Plugin-native APIs remain:

await executor.openapi.configure(source, {
  scope: user,
  request: {
    headers: {
      Authorization: SecretId.make("stripe_api_key"),
    },
  },
});

await executor.graphql.configure(source, {
  scope: user,
  request: {
    headers: {
      Authorization: SecretId.make("github_token"),
    },
  },
});

await executor.mcp.configure(source, {
  scope: user,
  transport: "http",
  request: {
    headers: {
      Authorization: SecretId.make("mcp_token"),
    },
  },
});

MCP stdio is transport-specific and should not be forced through HTTP helpers:

await executor.mcp.configure(source, {
  scope: user,
  transport: "stdio",
  env: {
    GITHUB_TOKEN: SecretId.make("github_token"),
  },
});

Core dispatch:

const configure = (source, input) =>
  Effect.gen(function* () {
    const storedSource = yield* Sources.get(source);
    const implementation = yield* SourceConfigureRegistry.get(input.type);

    if (storedSource.type !== input.type) {
      return yield* new SourceTypeMismatch({
        source,
        expected: storedSource.type,
        received: input.type,
      });
    }

    const parsed = yield* Schema.decodeUnknown(implementation.schema)(input);

    return yield* implementation.configure(source, parsed);
  });

Plugin registration:

openapiPlugin.registerSourceConfigure({
  type: "openapi",
  schema: OpenApiConfigureInput,
  configure: openApiConfigure,
  manifest: deriveOpenApiCredentialManifest,
});

The plugin configure implementation compiles plugin input into core binding operations:

const openApiConfigure = (source, input) =>
  Effect.gen(function* () {
    const bindings = yield* compileOpenApiConfigureBindings(source, input);

    yield* Sources.replaceBindings({
      source,
      scope: input.scope,
      slotPrefixes: ["request.", "specFetch."],
      bindings,
    });
  });

Shared HTTP Package

Create a shared package for HTTP source helpers, likely:

packages/plugins/http-source

Package name:

@executor-js/plugin-http-source

This package is not core. It exists because OpenAPI, GraphQL, and MCP HTTP share HTTP credential concepts while databases, CLIs, and MCP stdio do not.

It should own:

• HTTP credential config types.
• Header/query slot key helpers.
• OAuth config types.
• Binding compiler helpers.
• Runtime resolution helpers.
• Helpers that apply resolved credentials to HTTP requests.
• Slot manifest helpers.
• React credential components, either directly or via a /react subpath export.

Possible layout:

packages/plugins/http-source
  src/
    sdk/
      types.ts
      slots.ts
      configure.ts
      resolve.ts
      oauth.ts
    react/
      HttpCredentialsProvider.tsx
      HttpHeaderCredentials.tsx
      HttpQueryCredentials.tsx
      OAuthCredentials.tsx
    index.ts

Example shared config:

type HttpRequestSourceConfig = {
  headers?: Record<string, HttpCredentialSlotConfig>;
  query?: Record<string, HttpCredentialSlotConfig>;
  oauth?: HttpOAuthSourceConfig;
};

Example configure input:

type HttpRequestConfigureInput = {
  headers?: Record<string, CredentialInput>;
  query?: Record<string, CredentialInput>;
  oauth?: HttpOAuthConfigureInput;
};

OpenAPI can embed the shared shape twice:

type OpenApiConfigureInput = {
  type: "openapi";
  scope: ScopeId;
  request?: HttpRequestConfigureInput;
  specFetch?: HttpRequestConfigureInput;
};

GraphQL can embed it once for now:

type GraphqlConfigureInput = {
  type: "graphql";
  scope: ScopeId;
  request?: HttpRequestConfigureInput;
};

MCP HTTP can embed it behind a transport discriminant:

type McpConfigureInput =
  | {
      type: "mcp";
      transport: "http";
      scope: ScopeId;
      request?: HttpRequestConfigureInput;
    }
  | {