Dynamic Plugin Loading — v1

Date: 2026-05-02 (revised)

Status: shipped. The implementation lives on main. This doc was revised after the fact to match what actually shipped — three things changed from the original design:

1. handlers(self) => Layer became late-binding handlers() => Layer plus a extensionService Service tag on the spec. Cloud's per-request executor pattern needed the Tag to be satisfied at request time, not bake-in time. Local satisfies it eagerly via composePluginHandlers, cloud satisfies it per request via providePluginExtensions.
2. <ExecutorPluginsProvider> was added in @executor-js/sdk/client so frontend pages read useSourcePlugins() / useSecretProviderPlugins() / useClientPlugins() from React context instead of taking plugin lists as props. Routes/shells stop importing per-app aggregator files.
3. SourcePlugin and SecretProviderPlugin types moved to @executor-js/sdk/client and are populated as fields on defineClientPlugin({ sourcePlugin, secretProviderPlugin }) so everything frontend-facing flows through virtual:executor/plugins-client.

Context

Before this work, plugins were workspace deps statically imported in apps/local/src/server/executor.ts inside a createLocalPlugins() factory; the cloud app had its own equivalent in apps/cloud/src/services/executor.ts. They register secret providers, connection providers, and dynamic tools through definePlugin. The DX of authoring one was good — the problem was that adding a plugin meant editing host source. The goal: install plugins from npm and pick them up by editing config alone.

The existing in-process plugin model already gets a lot right and we want to keep it: Effect-native everything, end-to-end type inference from schema → storage → extension → consumer (executor[pluginId]), scope-aware ctx, closure methods, zero-allocation static tools delegating via staticSources(self) => [...]. Whatever changes here must preserve those properties.

What's missing is registration of three new surfaces from a plugin:

• API routes — HTTP endpoints owned by the plugin, contributed as an HttpApiGroup so they compose into the host's existing typed HttpApi
• Frontend — pages/widgets/components contributed to the host UI, with reactive AtomHttpApi-backed clients matching the existing toolsAtom / sourcesAtom pattern in packages/react/src/api/atoms.tsx
• SDK — already present via extension; the new HTTP routes give the frontend a typed reactive client without hand-written fetch glue

Goal and non-goals

v1 goals:

• A plugin is a single npm package. bun add @executor-js/plugin-foo, add it to executor.config.ts, restart, and it works.
• Plugins can register: extension methods (today), API routes (new), frontend pages/widgets (new).
• Frontend half gets a typed reactive client from the plugin's HttpApiGroup via the same AtomHttpApi pattern the core uses today.
• A plugin author can write the whole thing importing only from @executor-js/sdk (and @executor-js/sdk/client on the frontend). Effect imports are optional — for authors who want them, not required for those who don't.
• executor.config.ts is the single source of truth — same file consumed by the schema-gen CLI and the runtime.
• Type inference end-to-end stays intact (executor.foo.method() autocompletes).

Non-goals for v1:

• Cloud / multi-tenant deployment.
• Electron desktop dynamism. Desktop builds bake plugins at build time.
• Sandboxing, capability enforcement, marketplace, signing.
• Hot reload of plugin code (restart is fine).
• Loading plugins by string spec at runtime (we use import-and-call instead; see decision #1).

Reference research summary

Surveyed five plugin systems in .reference/:

• pi-mono — filesystem scanning + jiti, factory function + ExtensionAPI. Two-phase load (registration vs. action) is a clean idea.
• opencode — string specs in JSONC, dynamic import(), hook trigger pattern with (input, output) mutation.
• openclaw — manifest-first control plane (openclaw.plugin.json declares capabilities statically so the host can plan activation without loading code).
• emdash — closest match to our setup. Astro/Vite + React, plugins are npm packages with separate ./ and ./admin exports, registered via import-and-call in astro.config.mjs. definePlugin({ id, hooks, routes, admin }) declarative shape.
• dynamic-software — most ambitious; Cloudflare Worker Loader for cloud isolation, iframe + postMessage RPC for UI, Proxy-based typed API client.

The pattern that fits us cleanest is emdash's: import-and-call in a config file, single npm package with separate server/client exports, host integrates via Vite. We don't need dynamic-software's Proxy RPC client because @effect/platform already gives us HttpApiClient plus AtomHttpApi for the reactive React side — both already used heavily in packages/core/api/src/api.ts and packages/react/src/api/.

Decisions

1. Config is import-and-call, not string specs

executor.config.ts holds real imports of plugin factory functions. Type inference flows naturally; no codegen step.

// apps/local/executor.config.ts (after)
import { defineExecutorConfig } from "@executor-js/sdk";
import { openApiPlugin } from "@executor-js/plugin-openapi";
import { mcpPlugin } from "@executor-js/plugin-mcp";
import { fileSecretsPlugin } from "@executor-js/plugin-file-secrets";

export default defineExecutorConfig({
  dialect: "sqlite",
  plugins: [
    openApiPlugin(),
    mcpPlugin({ dangerouslyAllowStdioMCP: true }),
    fileSecretsPlugin(),
  ] as const,
});

"Dynamic" means npm-installable, not load-by-string-name. Same model as emdash. Codegen-based string specs (TanStack Router style) deferred to later if remote config becomes a need.

2. Plugin = single npm package, two exports

// @executor-js/plugin-foo/package.json
{
  "name": "@executor-js/plugin-foo",
  "type": "module",
  "exports": {
    "./server": "./dist/server.js",
    "./client": "./dist/client.js",
  },
  "executor": {
    "id": "foo",
    "version": "0.1.0",
  },
}

@executor-js/plugin-foo/
├── package.json
├── src/
│   ├── server.ts    # Effect, Node deps, definePlugin
│   ├── client.tsx   # React, defineClientPlugin
│   └── shared.ts    # Schemas, types shared across the boundary
└── dist/

Strict separation: server bundle never imports React, client bundle never imports Effect/Node modules. Shared types live in src/shared.ts and are imported by both halves.

3. Extend PluginSpec with optional routes; add parallel defineClientPlugin

Server side: keep definePlugin. Add optional routes field (the HttpApiGroup) and handlers field (the typed Layer). No frontend concepts on the server side.

Client side: separate primitive defineClientPlugin lives in @executor-js/sdk/client. Can only be imported in the ./client entry, so React types never leak into server bundles.

Layering — extension is the canonical SDK; routes/handlers is optional HTTP transport. The HTTP layer is not a peer of the SDK; it's a transport over it. Plugin authors should treat extension as the implementation and write handlers as thin wrappers that delegate via the self parameter (same pattern as staticSources(self) => [...] already in the codebase). This keeps:

• a single source of truth for the plugin's behavior
• in-process callers paying zero serialization cost
• HTTP callers getting auth/scope/observability middleware
• error contracts identical across the two surfaces

Three plugin shapes fall out of this layering:

• SDK-only. Pure programmatic. No routes, no handlers, no ./client export. Examples: file-secrets, keychain, onepassword, anything that's a utility for other plugins or scripts. CLI/embedded consumers use executor.<id>.method(). Vite plugin notices no ./client and skips the plugin in the frontend bundle entirely.
• Both. Extension and routes/handlers and a ./client. Examples: openapi, mcp, anything that needs a frontend. Routes are thin wrappers over extension methods.
• HTTP-only. Rare — webhook receivers, OAuth callback URLs. Routes without a meaningful in-process equivalent. May or may not have an extension.

4. Routes are an HttpApiGroup; client uses AtomHttpApi

Plugins ship the same primitive the core uses (HttpApiGroup from effect/unstable/httpapi). The host composes via the existing addGroup helper at packages/core/api/src/api.ts:21. OpenAPI annotations and docs flow automatically.

For the frontend, plugins build a per-plugin AtomHttpApi.Service against their own group, wrapped behind a createPluginAtomClient(group, opts) helper. The resulting atoms are consumed via the existing useAtomValue / useAtomSet + AsyncResult.match idiom — same pattern as toolsAtom, sourcesAtom, etc.

5. SDK re-exports the Effect HttpApi/Schema primitives

Plugin authors can write a complete plugin importing only from @executor-js/sdk (server) and @executor-js/sdk/client (frontend). The SDK re-exports Schema, HttpApi, HttpApiGroup, HttpApiEndpoint, HttpApiBuilder, Effect so authors who don't want to dig into Effect don't have to. Authors who do want Effect-native code keep importing from effect directly. This keeps the door open without forcing the dependency.

6. Skip capabilities declaration for v1

["read:secrets", "network:fetch"] style declarations are useful for sandboxing but unenforced metadata is just noise. Add when there's a real isolation story.

7. executor.config.ts is the single source of truth

Today the schema-gen CLI reads executor.config.ts but the runtime hardcodes createLocalPlugins() in apps/local/src/server/executor.ts:96. Consolidate: the runtime imports from executor.config.ts too. One list, two consumers.

8. Canary plugin: build new tiny one first, then migrate openapi

Validate the shape with a minimal @executor-js/plugin-example (one extension method, one route, one widget) before changing real plugins.

9. Cross-plugin pluggable capabilities: per-capability typed fields

Some capabilities (secrets, eventually artifacts, maybe more) are "pluggable" — many plugins can implement them, the host swaps between providers via config, consumer code stays agnostic.

Don't generalize this. The existing secretProviders field on the spec already handles this exact pattern for secrets and works fine:

// what plugins do today
secretProviders: (ctx) => [makeScopedProvider(...)]

Each new pluggable capability gets its own typed field on PluginSpec, same shape. When artifacts lands, that's artifactStore: () => ArtifactStore. If connection providers want to be modeled this way, the existing connectionProviders field already is.

No provides / requires / service machinery, no Effect-Tag-as-generic- primitive abstraction, no "protocols" concept, no naming debate. The artifacts note's "protocol" framing translates directly to "the v2 artifactStore field on PluginSpec."

If we eventually have 5–6 of these and the boilerplate genuinely screams for generalization, we generalize then. Likely won't.

For v1 nothing changes here — secretProviders is what it is, and artifacts aren't shipping yet.

New type sketches

PluginSpec extension

Existing fields unchanged. Three new fields:

// packages/core/sdk/src/plugin.ts (extension)
import type { Context, Layer } from "effect"
import type { HttpApiGroup } from "effect/unstable/httpapi"

export interface PluginSpec<
  TId, TExtension, TStore, TSchema,
  TExtensionService extends Context.Service<any, any> | undefined = undefined,
  THandlersLayer extends Layer.Layer<any, any, any> = Layer.Layer<unknown, never, never>,
> {
  // ... existing: id, schema, storage, extension, staticSources,
  //     invokeTool, secretProviders, etc.

  /** npm package name. The Vite plugin emits
   *  `import "${packageName}/client"` into the frontend bundle, so the
   *  same `executor.config.ts` drives both server and client. Author
   *  writes the same string they publish to npm — no transforms, no
   *  scope conventions, no fallbacks. Required for plugins that ship
   *  a `./client` entry; omit for SDK-only plugins. */
  readonly packageName?: string

  /** HttpApiGroup contributed by this plugin. Composed into the host's
   *  HttpApi via the `composePluginApi` helper at runtime. The host
   *  serves it under whatever base URL the host wires (`/api` for the
   *  local app's vite middleware) and supplies auth + scope
   *  middleware. Endpoints automatically appear in the executor OpenAPI
   *  doc and the typed client. */
  readonly routes?: () => HttpApiGroup.Any

  /** Late-binding handlers Layer for this plugin's group. The layer
   *  leaves the plugin's extension as a Service Tag requirement (see
   *  `extensionService` below). The host satisfies it however its
   *  runtime wants:
   *    - local: at boot via `Layer.succeed(extensionService)(executor[id])`
   *      inside `composePluginHandlers(plugins, executor)`
   *    - cloud: per request via `Effect.provideService(extensionService,
   *      requestExecutor[id])` inside the auth middleware via
   *      `providePluginExtensions(plugins)(executor)` */
  readonly handlers?: () => THandlersLayer

  /** Service tag the plugin's `handlers` layer requires. The established
   *  pattern in each plugin's `*/api/handlers.ts`: `*Handlers` reads
   *  `*ExtensionService`. The host binds the tag to the live extension —
   *  at boot for local, per request for cloud. Pairs with `handlers`;
   *  either both fields are set or neither. */
  readonly extensionService?: TExtensionService
}

Why late-binding (handlers() not handlers(self)). The original design had handlers(self) => Layer so the host called the factory with the boot-time extension and got a fully-resolved Layer. That fits local fine — one executor at boot, one Layer. Cloud builds a fresh executor per HTTP request (Cloudflare Workers + Hyperdrive needs fresh DB connections per request), so the executor that satisfies the extension Service is per-request, not per-boot. The late-binding shape lets the same plugin spec satisfy both: the *Handlers Layer is built once and consumes *ExtensionService from context; local provides it at boot via Layer.succeed, cloud provides it per request via Effect.provideService.

Example plugin shape — group definition in shared.ts so client and server both import it:

// @executor-js/plugin-foo/src/shared.ts
import { HttpApiEndpoint, HttpApiGroup, Schema } from "@executor-js/sdk";

export const Thing = Schema.Struct({
  id: Schema.String,
  name: Schema.String,
});

export const FooApi = HttpApiGroup.make("foo")
  .add(HttpApiEndpoint.get("listThings")`/things`.addSuccess(Schema.Array(Thing)))
  .add(
    HttpApiEndpoint.post("syncThing")`/sync/:id`
      .setPath(Schema.Struct({ id: Schema.String }))
      .addSuccess(Thing)
      .addError(SyncError),
  );

// @executor-js/plugin-foo/src/server.ts
import {
  Context, definePlugin, Effect, HttpApi, HttpApiBuilder,
} from "@executor-js/sdk"
import { FooApi } from "./shared"

// Bundle the group into a small HttpApi *for typing purposes only*. The
// handlers Layer is keyed by the FooApi group's identity, so it composes
// cleanly into the host's FullApi at runtime regardless of what other
// groups are around it.
const FooApiBundle = HttpApi.make("foo").add(FooApi)

interface FooExtension {
  readonly listThings: () => Effect.Effect<readonly Thing[]>
  readonly syncThing: (id: string) => Effect.Effect<Thing>
}

// Service tag for late-binding — the host satisfies this at boot
// (local) or per request (cloud). Same pattern as the established
// `*ExtensionService` in each plugin's `*/api/handlers.ts`.
export class FooExtensionService extends Context.Service<
  FooExtensionService, FooExtension,
>()("FooExtensionService") {}

const FooHandlers = HttpApiBuilder.group(FooApiBundle, "foo", (h) =>
  h
    .handle("listThings", () =>
      Effect.gen(function* () {
        const ext = yield* FooExtensionService
        return yield* ext.listThings()
      }),
    )
    .handle("syncThing", ({ path }) =>
      Effect.gen(function* () {
        const ext = yield* FooExtensionService
        return yield* ext.syncThing(path.id)
      }),
    ),
)

export const fooPlugin = definePlugin((opts?: FooConfig) => ({
  id: "foo" as const,
  packageName: "@executor-js/plugin-foo",
  storage: () => ({ /* ... */ }),

  // Canonical implementation. CLI/tests/embedded callers and the HTTP
  // handlers all hit this same code path.
  extension: (ctx): FooExtension => ({
    listThings: () => /* Effect — canonical impl */,
    syncThing: (id: string) => /* Effect — canonical impl */,
  }),

  routes: () => FooApi,
  handlers: () => FooHandlers,
  extensionService: FooExtensionService,
}))

Why routes and handlers are split: routes is the API description (a group), handlers is the implementation Layer keyed by group identity. The host needs the group at composition time (to build FullApi) and the Layer at serve time (to provide handler implementations). extensionService is the third leg: the bridge that lets the host bind the live extension (at boot or per request) without the plugin author having to write two handler shapes.

defineClientPlugin

Lives in @executor-js/sdk/client. Server bundles cannot import this module.

// packages/core/sdk/src/client.ts
export interface ClientPluginSpec<TId extends string = string> {
  readonly id: TId;

  /** Pages contributed to the host's TanStack router. Mounted under
   *  /plugins/{id}/{path}. Sidebar nav metadata declared on the route. */
  readonly pages?: readonly PageDecl[];

  /** Dashboard / overview widgets the host can render in known slots. */
  readonly widgets?: readonly WidgetDecl[];

  /** Components the host can render in named slots (e.g., source-detail
   *  panels, secret-picker variants). Slot names are part of the host
   *  contract — plugin opts in by registering. */
  readonly slots?: Record<string, SlotComponent>;

  /** Source-plugin contribution — populated by plugins that expose
   *  `kind` rows in the core `source` table (openapi, mcp, graphql,
   *  google-discovery). The host's sources page derives its provider
   *  list from the union of every loaded plugin's `sourcePlugin`
   *  via `useSourcePlugins()`. */
  readonly sourcePlugin?: SourcePlugin;

  /** Secret-provider-plugin contribution — populated by plugins that
   *  also ship a `secretProviders` server-side capability AND want to
   *  expose a settings card on the host's secrets page. Surfaced via
   *  `useSecretProviderPlugins()`. */
  readonly secretProviderPlugin?: SecretProviderPlugin;
}

type PageDecl = {
  path: string; // "/", "/edit/$id"
  component: ComponentType;
  nav?: { label: string; section?: string };
};

type WidgetDecl = {
  id: string;
  component: ComponentType<WidgetProps>;
  size?: "half" | "full";
};

export const defineClientPlugin = <TId extends string>(spec: ClientPluginSpec<TId>) => spec;

createPluginAtomClient — typed reactive client per plugin

Plugins build their own AtomHttpApi.Service against their own group bundled into a small HttpApi. A helper hides the boilerplate so plugin authors write one line per atom. Same shape as the existing ExecutorApiClient in packages/react/src/api/client.tsx:11.

// packages/core/sdk/src/client.ts (helper)
import { HttpApi } from "effect/unstable/httpapi";
import { FetchHttpClient } from "effect/unstable/http";
import * as AtomHttpApi from "effect/unstable/reactivity/AtomHttpApi";

export const createPluginAtomClient = <G extends HttpApiGroup.Any>(
  group: G,
  opts: { pluginId: string },
) => {
  const bundle = HttpApi.make(`plugin-${opts.pluginId}`).add(group);
  return AtomHttpApi.Service<`Plugin_${string}Client`>()(`Plugin_${opts.pluginId}Client`, {
    api: bundle,
    httpClient: FetchHttpClient.layer,
    baseUrl: `/_executor/plugins/${opts.pluginId}`,
  });
};

Plugin author writes: