History

Add isolate lifecycle signals to worker spans (#1694)

Every span we emit starts inside the fetch handler, so nothing could tell a cold isolate paying Start's loadEntries import from an isolate that came up slowly before our code ran. Measured with a temporary entry-lag probe against wrangler tail event timestamps: a cold isolate spends 840-1844ms before our first line, a reused one 0-2ms. Stamp request_seq and start_graph.entered on the worker spans so cold-vs-warm is queryable from Axiom without a probe. Also drops a stray plans file that #1692 swept in via git add -A.

Rhys Sullivan committed 2 months ago

Commit cd9b9781 parent d5beb462 files · +52 −516Browse files

Unified · Split

2 files
  1. apps/cloud/src/server.ts +52 −1
  2. plans/kill-plugin-system.md +0 −515
apps/cloud/src/server.ts+52−1
@@ -103,12 +103,26 @@ export { McpExecutionOwnerDirectoryDO } from "@executor-js/cloudflare/mcp/execut · 102 unmodified lines
103// until the in-flight export resolves.103// until the in-flight export resolves.
104// ---------------------------------------------------------------------------104// ---------------------------------------------------------------------------
105 105
106const fetchHandler = handler.fetch as (106const rawFetchHandler = handler.fetch as (
107 request: Request,107 request: Request,
108 env: Env,108 env: Env,
109 ctx: ExecutionContext,109 ctx: ExecutionContext,
110) => Response | Promise<Response>;110) => Response | Promise<Response>;
111 111
112/**
113 * Every entry into TanStack Start goes through here so `startGraphEntered`
114 * reflects whether this isolate has already paid the lazy `loadEntries`
115 * import — the cost that dominates a cold page request.
116 */
117const fetchHandler = (
118 request: Request,
119 env: Env,
120 ctx: ExecutionContext,
121): Response | Promise<Response> => {
122 markStartGraphEntered();
123 return rawFetchHandler(request, env, ctx);
124};
125
112const tracer = trace.getTracer("executor-cloud-worker");126const tracer = trace.getTracer("executor-cloud-worker");
113 127
114const traceparentValueFor = (spanContext: SpanContext): string =>128const traceparentValueFor = (spanContext: SpanContext): string =>
@@ -178,8 +192,43 @@ const mcpAgentHandler = makeCloudMcpAgentHandler({ · 63 unmodified lines
178 traceRequest: traceCloudMcpRequest,192 traceRequest: traceCloudMcpRequest,
179});193});
180 194
195// ---------------------------------------------------------------------------
196// Isolate lifecycle signals
197// ---------------------------------------------------------------------------
198//
199// The Aug 2026 page-latency hunt kept stalling on one blind spot: every span we
200// emit starts INSIDE the fetch handler, so nothing could distinguish "this
201// isolate is cold and paying Start's lazy `loadEntries` import" from "this
202// isolate came up slowly before our code ran at all". Both look like one slow
203// span. These three attributes make the distinction queryable:
204//
205// executor.isolate.request_seq - 1 means this request is the isolate's
206// first; page requests measured 1.04 per
207// isolate, which is why nearly every one
208// pays the cold cost.
209// executor.start_graph.entered - whether anything had already driven
210// TanStack Start in this isolate. /mcp and
211// the passthrough proxies return before
212// `fetchHandler`, so an isolate can serve
213// plenty of traffic and still be cold here.
214// Measured with a temporary entry-lag probe correlated against `wrangler tail`
215// event timestamps: a cold isolate (`request_seq` 1) spends **840-1844ms**
216// coming up before our first line runs, while a reused one spends **0-2ms**.
217// That startup is invisible to every span we emit, and it sits on top of the
218// ~3.1s `loadEntries` import — together the 3-5s a signed-in page costs.
219//
220// Both counters are cheap: two increments and no I/O.
221let isolateRequestSeq = 0;
222let startGraphEntered = false;
223
224const markStartGraphEntered = (): void => {
225 startGraphEntered = true;
226};
227
181const cloudflareHandler: ExportedHandler<Env> = {228const cloudflareHandler: ExportedHandler<Env> = {
182 fetch: async (request, env, ctx) => {229 fetch: async (request, env, ctx) => {
230 isolateRequestSeq += 1;
231
183 // Public pages must not enter TanStack Start: its first-request dynamic232 // Public pages must not enter TanStack Start: its first-request dynamic
184 // import loads the entire React + Effect server graph and can take seconds233 // import loads the entire React + Effect server graph and can take seconds
185 // on a cold isolate. Classify and service-bind marketing at the Worker234 // on a cold isolate. Classify and service-bind marketing at the Worker
@@ -250,6 +299,8 @@ const cloudflareHandler: ExportedHandler<Env> = { · 64 unmodified lines
250 async (span) => {299 async (span) => {
251 span.setAttribute(ATTR_HTTP_REQUEST_METHOD, request.method);300 span.setAttribute(ATTR_HTTP_REQUEST_METHOD, request.method);
252 span.setAttribute(ATTR_URL_PATH, url.pathname);301 span.setAttribute(ATTR_URL_PATH, url.pathname);
302 span.setAttribute("executor.isolate.request_seq", isolateRequestSeq);
303 span.setAttribute("executor.start_graph.entered", startGraphEntered);
253 // oxlint-disable-next-line executor/no-try-catch-or-throw -- adapter boundary; observe response/error for span status, keep the flush alive past the response304 // oxlint-disable-next-line executor/no-try-catch-or-throw -- adapter boundary; observe response/error for span status, keep the flush alive past the response
254 try {305 try {
255 const response = await fetchHandler(306 const response = await fetchHandler(
plans/kill-plugin-system.md+0−515
@@ -1,515 +0,0 @@
1# Kill the plugin system — in-place migration plan
2
3Decision (Rhys, 2026-07-27): abandon the big-bang rework
4(`UsefulSoftwareCo/executor-rework`, cloned read-only at
5`.reference/executor-rework`); fix this repo in place over a sequence of PRs,
6each keeping format/lint/typecheck/test/e2e green. Priority order: **data
7models first**, then the plugin kill, then the concept-package (apps) model,
8then the minimal SDK / platform split.
9
10RESEQUENCED (Rhys, 2026-07-28): **Phase 3 (apps/concept-package model) now
11lands BEFORE Phase 2 (plugin kill), side by side with the protocol plugins.**
12Rationale: the new model's tables are greenfield and additive — landing it
13first means the legacy apps tables never need an in-place migration; the whole
14model swaps underneath while openapi/mcp/graphql keep running as plugins.
15Structurally sound because (a) the kernel + MCP host are already plugin-free,
16so the new model's runtime never touched the registry, and (b) the apps plugin
17already exercises every extension point, so the new model ships as the apps
18plugin's new internals wearing the same costume. When 2.3 later converts the
19four kinds, apps converts as a stable, proven model instead of mid-rewrite.
20Phase 0 and 2.1 remain freely parallel — they don't touch apps.
21
22Hard line: side-by-side means new-apps-model alongside the _protocol plugins_,
23never two app models with dual read paths inside the apps kind. Existing app
24sources get a one-off migration to published-source form (enumerated
25invariants, explicit approval, no permanent compatibility path). Accepted
26cost: the new model's HTTP surface temporarily wears plugin-route ceremony
27(HttpApiGroup + handler layers) that 2.2/2.6 later deletes — wiring, not
28model. The authoring SDK (`defineApp`/`defineProvider`/`defineTool`) goes
29public early; its boundaries take Standard Schema (StandardSchemaV1 &
30StandardJSONSchemaV1), never Effect Schema, and Phase 4's vocabulary rules
31(opaque owner refs, no "tenant") bind on it from day one.
32
33## Diagnosis (verified against the code)
34
35**The plugin system is a fiction.** `packages/core/sdk/src/plugin.ts` (~850
36lines, ~20 optional hooks) pretends the plugin set is open, but:
37
38- `Integration.kind` _is_ the plugin id string; all hot paths dispatch via
39 `runtimes.get(row.plugin_id)` (`executor.ts`).
40- Core and React hardcode the closed set anyway: `migration-spec.ts` hardcodes
41 plugin ids; `packages/react` has `KIND_TO_PLUGIN_KEY` maps and an
42 `if (pluginId === "toolkits")` route branch.
43- Nobody ever shipped a fourth protocol plugin, and the two provider plugins
44 (google/microsoft) were already collapsed into openapi (#1366).
45- Whole packages exist only to support the open set: `plugin-routes.ts`
46 type-union machinery, `@executor-js/config`'s jsonc+jiti loader,
47 `packages/core/vite-plugin`'s virtual client module. The jsonc plugin path is
48 vestigial — no `executor.jsonc` exists anywhere in the repo, no docs mention
49 it.
50- Toolkits (~880 LoC of pure curation + the single `toolPolicyProvider` slot)
51 and apps (~6.8k custom-tools gateway) are first-party capabilities wearing a
52 plugin costume. Kernel packages and `packages/hosts/mcp` have zero plugin
53 references — the indirection is not load-bearing for the real surfaces.
54
55**The data model is connection-first, not user-first**
56(`packages/core/sdk/src/core-schema.ts`):
57
58- No user/subject table. `subject` is an opaque partition-key string smeared
59 across owned tables; a user "exists" only if they have a connection row.
60- Disconnect is a hard delete (`connectionsRemove`) — a signed-out user is
61 bit-identical to one who never connected. "Which users are signed out" is
62 structurally unanswerable.
63- Every Executor is bound to one `{tenant, subject}` via `owner-policy.ts`;
64 there is no cross-subject read path, so no admin view.
65- Toolkit state lives in generic `plugin_storage` KV, not core tables.
66- Cloud's real identity tables (WorkOS accounts/organizations) have no join to
67 `subject`.
68
69**What to salvage from the rework** (its model docs are the valuable artifact;
70its implementation ported only ~10 of 131 legacy e2e scenarios — that's why it
71failed, and why this plan is incremental):
72
73- Apps-as-published-source: `defineApp`/`defineProvider`/`defineTool`/
74 `defineToolGroup`, immutable content-hashed versions, manifest derived by
75 executing the source, rollback = pointer move.
76- Generators: OpenAPI/MCP specs generate the _same_ app source a human would
77 write, with a re-verifiable `GeneratorClaim` (first-party vs third-party).
78- Minimal SDK boundary: `createExecutor`/`createRemoteExecutor`, no Effect
79 crossing the boundary, MCP exposure returns a standard MCP SDK server.
80- Custody separation: secrets never on the wire; opaque owner refs.
81
82## Constraints discovered (do not relearn these)
83
841. **Published packages make this semver-major.** `scripts/publish-packages.ts`
85 publishes `@executor-js/{config,sdk}` and plugins
86 `{example,file-secrets,graphql,keychain,mcp,onepassword,openapi}` — killing
87 `definePlugin` is a breaking release for seven packages. Changesets +
88 deprecation story required; `publish-packages.ts`, `pkg-pr-new.yml`,
89 `check-changelog-stubs.ts`, and `examples/*` all have hardcoded lists.
902. **`plugin_id = 'google'` is a live persisted value with no runtime.** The
91 provider-service-split migration skips orgs with missing spec blobs
92 (`sqlite.test.ts` asserts survivors). The `googleDiscovery → "google"`
93 entries in React are compensating, not dangling. The closed SourceType
94 switch must map `"google"`/`"googleDiscovery"` → openapi or degraded rows
95 become hard failures.
963. **FumaDB is not versioned in practice.** All hosts pin `version: "1.0.0"`;
97 runtime bring-up is `ensureDrizzleRuntimeSchemaFromTables` (CREATE TABLE IF
98 NOT EXISTS + additive nullable ALTERs). New tables/nullable columns are free
99 on local/self-host/D1. **Cloud is the exception**: hand-edit
100 `apps/cloud/src/db/executor-schema.ts` + drizzle-kit migration +
101 `db:migrate:prod`; `db.schema.test.ts` enforces the mirror, so "forgot
102 cloud" fails tests.
1034. **93 e2e files import `composePluginApi`** purely to type the client. Keep
104 it as a deprecated argument-ignoring shim returning the static
105 `FullExecutorApi`; sweep the call sites mechanically in one final PR.
1065. **Keep the `plugin_id` column name** (four tables + blob namespace format
107 `o:<tenant>/<plugin_id>`). Renaming to `source_kind` is a cross-host
108 migration for zero behavior. Retype on the read side only.
1096. **URL contracts:** `/integrations/add/$pluginKey` is navigated directly by
110 e2e; `/plugins/desktop-settings/` is asserted by the desktop-packaged suite
111 and baked into a tool's returned URL. Keep shapes / add redirects.
1127. **CI gaps:** PRs run cloud (4 shards) + selfhost e2e only; local/desktop
113 suites gate push-to-main. PRs touching desktop-settings or the local loader
114 need a manual local e2e run pre-merge.
1158. **Toolkit selection filters _connections_, not just tools**
116 (`connectionsList` goes through the active policy provider). Preserve when
117 moving toolkits to core, or the toolkit e2e suites go red.
118
119## Phase 0 — declutter (3 PRs)
120
121- **0.1** Delete empty `packages/plugins/{google,microsoft}`; drop the unused
122 `@executor-js/config` dep from openapi/mcp/graphql; move
123 `provider-service-split` to `packages/migrations/` (keep package name — it's
124 imported by three host data-migration registries). In the React maps, keep
125 `google`/`googleDiscovery` → openapi with a comment citing constraint 2.
126- **0.2** Bake google/microsoft catalogs + spec-format adapters in as openapi
127 defaults; strip the re-threading from all four host configs. Gate:
128 `provider-plugins-ui`, `google-health-checks`, `microsoft-*` scenarios.
129- **0.3** Move desktop-settings (438-line Electron-IPC client, not a
130 one-liner) to a first-class desktop-only route in `packages/app`; update the
131 static tool's URL and the desktop-packaged e2e selector; delete the plugin.
132
133## Phase 1 — data model (4 PRs) — FIRST PRIORITY
134
135Design choices (least-invasive, verified feasible):
136
137- The new **subject table is `tenant`-scoped, not owner-scoped** — readable by
138 any executor bound to the tenant with **zero policy changes** (the
139 `integration` table already proves the pattern).
140- Cross-subject connection reads: add `reach: "bound" | "tenant"` to
141 `ExecutorOwnerPolicyContext` — widens `ownerVisibilityCondition` for reads
142 only; `assertOwnerWritable`/`assertOwnerPatch` untouched, so admin reach can
143 never write. Only two construction sites exist (`executor.ts`,
144 `test-config.ts`).
145- Identity stays with the hosts (WorkOS / Better Auth member lists already
146 exist behind `AccountProvider.listMembers`); the subject table is the missing
147 **join** between host identity and executor connections, not a new identity
148 system. Tolerate `subject = null` (pure-org) and `subject = "local"`.
149
150PRs:
151
152- **1.1** Add tenant-scoped `subject` table (id, tenant, external_id,
153 created_at, last_seen_at, status) to `coreTables` + cloud mirror + one
154 drizzle migration. No readers yet.
155- **1.2** Populate: upsert in `makeScopedExecutor`
156 (`packages/core/api/src/server/scoped-executor.ts` — every HTTP request and
157 MCP session on all four hosts passes through it) and at connection-create.
158 Add to `purgeOrganizationData` + its `TENANT_TABLES` test.
159- **1.3** `reach` mode + `admin.listSubjectsWithConnections`-shaped read on a
160 derived handle. Policy test proving admin reach cannot write. Scope includes
161 (customer ask, owner.com 2026-07-27): per-subject reads ("connections for
162 subject X"), not just the aggregate list, and a privileged org-level API key
163 (keys already carry `{accountId, organizationId}` —
164 `apps/cloud/src/auth/api-keys.ts`) that resolves to the tenant-reach
165 read-only handle so customers can inspect connections programmatically.
166- **1.4** Connection lifecycle: nullable `status` column
167 (active/disconnected/revoked/expired) plus `status_changed_at`; disconnect
168 writes status instead of deleting. Wide blast radius — status predicate
169 through `findConnectionRow`, `connectionsList`, credential resolution,
170 `toolsList`, org purge; own e2e sweep (`connections-*`, `health-checks*`,
171 `oauth-*`). PRIORITY RAISED (customer, 2026-07-28): the customer's
172 block-startup-until-reconnect flow needs connected-then-disconnected to be
173 distinguishable from never-connected, which hard-delete makes impossible
174 today; expose status + statusChangedAt through the admin API when this
175 lands.
176
177Toolkits data does NOT move in this phase (it moves with its reader in 2.4 —
178moving it earlier means either rewriting the plugin storage layer twice or a
179forbidden dual read path).
180
181### Framing: product view vs platform view (Rhys, 2026-07-27)
182
183The two-view model for consumer-facing surfaces, longer-term:
184
185- **Product view** — the bound, per-individual executor every request gets
186 today (their connections, their tools). The default; unchanged.
187- **Platform view** — an operator-level view attached alongside it: read-only,
188 cross-user, explicitly reached for with elevated access (an escape hatch,
189 never a mode product view drifts into). The `reach: "bound" | "tenant"`
190 split in 1.3 is this model's first implementation; 1.5's admin API and the
191 privileged key are its first surface. Later platform-view candidates (usage,
192 policy overview, cross-user health) land on the same handle.
193
194Cloud wrinkle: cloud consumers like owner.com don't bring their own owner ids —
195their users authenticate through our WorkOS, so `subject` is _our_ accountId
196and the customer correlates on their side. The opaque-owner-ids-in-the-
197customer's-id-space story fully applies only to embedded/API consumers; for
198cloud, the platform view (+ `external_id` on the subject table) is what makes
199the correlation tractable.
200
201### Fast path: owner.com admin-visibility ask (ships from Phase 1 alone)
202
203The owner.com ask (admin inspects users' connections; API key that can do the
204same; available-vs-connected icon grid; direct connect links) needs only
2051.1 → 1.2 → 1.3 plus a thin delivery layer — no dependency on Phase 0 or the
206plugin kill. Delivery PRs on top of 1.3:
207
208- **1.5** Public admin API surface: `GET .../admin/users`,
209 `GET .../admin/users/:externalId/connections` (integration slug + status +
210 last_health only — never item_ids/oauth fields), gated on the privileged key
211 / admin membership. Public paths say `users`, not `subjects`/`tenants` — see
212 "Vocabulary" below. The available-vs-connected diff is computable client-side
213 from this plus the existing integration catalog.
214- **1.6** Connect deep links: stable
215 `/connect/:integrationSlug`-shaped URL that lands the arriving user in the
216 connect flow for that integration (auth-aware redirect). Uses integration
217 slugs, not plugin routes, so it survives Phase 2.5 unchanged.
218
219 1.4 (lifecycle status) upgrades the grid from binary connected/not to
220 expired/disconnected states but is not a blocker for first ship.
221
222## Phase 2 — kill the registry (6 PRs)
223
224- **2.1** Remove dynamic loading: delete `load-plugins.ts` from
225 `@executor-js/config` (keep the rest — jsonc integration config is a separate
226 feature), drop `loadPluginsFromJsonc` from `apps/local`, delete
227 `packages/core/vite-plugin`, replace `virtual:executor/plugins-client` with a
228 static module in `packages/app` (5 `__root.tsx` importers + 3 ambient
229 `.d.ts`), edit `turbo.json` `dev.dependsOn` + root `prepare`, fix
230 `packages/app/vite.config.ts`'s dangling jsonc path. Breaking changeset.
231- **2.2** Static route list: `composePluginApi` becomes the deprecated shim
232 (constraint 4); hosts stop passing plugin tuples; delete
233 `apps/{cloud,host-selfhost}/src/plugins.ts` (exist only to hold
234 `PluginExtensionServices` typing).
235- **2.3** The switch: closed internal `SourceType` interface implemented by
236 openapi/graphql/mcp/**apps (current model, unchanged)** as plain libraries,
237 replacing `runtimes.get`. All four kinds convert together — converting three
238 and leaving apps on the plugin path means carrying dual dispatch; rewriting
239 the apps model in the same PR is what killed the rework. Map
240 `"google"`/`"googleDiscovery"` → openapi. `plugin_id` column name unchanged.
241- **2.4** Toolkits to core: core tables + one-off migration out of
242 `plugin_storage` (enumerated invariants, no dual path), `toolPolicyProvider`
243 slot → native policy input (one call site, one implementation, one test),
244 ordinary core routes, `activeToolkitSlug` becomes an executor option.
245 Preserve the connection-visibility filter (constraint 8).
246- **2.5** React: four named kind renderers; move plugin React code (~2.9k LoC,
247 e.g. `AddMcpIntegration.tsx`) into `packages/react` — watch the peer-dep
248 edge for cycles; delete `defineClientPlugin`, the plugins context, both
249 `KIND_TO_PLUGIN_KEY` maps, and `/plugins/$pluginId/$` (with redirects for
250 desktop-settings + toolkits). Keep `/integrations/add/$pluginKey` URL shape.
251 Don't forget the second axis: `useSecretProviderPlugins` (secrets page).
252- **2.6** Detach `CredentialProvider` from `PluginSpec` (the interface in
253 `provider.ts` is already narrow — secret-store plugins keep it as their
254 seam); delete `plugin.ts`, `plugin-routes.ts`, `plugin-example`, and the
255 shim; mechanical sweep of the 93 e2e files; changesets for all seven
256 published packages.
257
258## Phase 3 — custom = concept package (3–4 PRs) — NOW BEFORE PHASE 2
259
260(See RESEQUENCED note at top: lands side by side with the protocol plugins,
261inside the apps plugin's existing costume; Phase 2 kills the registry
262afterward, with apps converting in 2.3 as an already-stable model.)
263
264Port the rework's app model in place as the `apps` source kind's new model:
265published source, manifest derived by execution, content-hashed immutable
266versions, rollback = pointer move. New tables are additive greenfield (same
267pattern as 1.1: coreTables + cloud mirror + migration, no readers first).
268Existing app sources migrate one-off to published-source form — no dual model
269within the apps kind. Then the OpenAPI/MCP generators + `GeneratorClaim` as an
270optional on-ramp (spec → generated app source); GraphQL generator later.
271Long-term option (explicitly out of scope): protocol kinds themselves become
272generators and "custom" is the only kind.
273
274### Feasibility findings (two deep read-throughs, 2026-07-28)
275
276**Current plugin** (~7.6k LoC): only ~2.65k encodes the old model (store
277collections, sync state machine, publish/discover/descriptor, routes,
278console client). ~2.6k is model-agnostic and survives verbatim: all three
279sandbox executors, the bundler stack, bindings/bridge, authoring +
280standard-schema, conformance. ~1.9k (hand-rolled git client + source
281fetchers) hangs on the publish-input decision below. Storage is entirely
282generic `plugin_storage` rows + content-addressed blobs — zero apps-specific
283tables or migrations exist, so new tables are pure greenfield. Cloud prod is
284git-source-only; **source text is never persisted today (only compiled
285bundles)**, so the one-off migration re-fetches source from the recorded git
286URLs (fetcher exists). Load-bearing invariants to preserve: tool addresses
287`tools.<app>.org.published.<name>` + the `<fileSlug>__<key>` naming rule,
288app-slug↔integration-slug 1:1 guard, publish CAS
289(`AppPublishConflictError`), tombstone-not-delete, token-in-provider-item
290(never in the source row). Safety net: `custom-tools.test.ts` (534-line
291cloud+selfhost console loop), local directory-picker, packed-CLI, desktop
292e2e, ~2.9k LoC in-package tests. Pre-existing gaps flagged: R2 blobs never
293GC'd and org purge misses them; cloud silently falls back to in-process
294executor without the worker loader; host-cloudflare doesn't register apps.
295
296**Rework model** (portable tiers): T1 model proper ~3.9k LoC (authoring SDK,
297publish-by-execution, projection, generator verification, storage port,
298content hashing); T2 generators ~3.2k (fromOpenAPI 1k — complete; fromMCP
2990.6k — computed group with real streamable-HTTP client; json-schema engine;
300byte-for-byte GeneratorClaim verification; render endpoint); T4 host/registry
301~2.9k explicitly skipped. Authoring is Standard Schema
302(StandardSchemaV1 & StandardJSONSchemaV1) with raw-JSON-Schema literals as a
303typed first-class alternative — matches our rule. Effect beta.93→beta.59
304delta is small (ConstraintDecoder → one-line fix; hand-roll
305StandardJSONSchemaV1 ~10 lines; reuse core sha256Hex). NOT portable as-is:
306rework `storage-db.ts` (write a new adapter over our tables against the
307~90-line Storage port interface) and `engine.ts` (extract only ~700 LoC of
308model-owned logic: computed-surface cache + requirement resolution +
309runTool). Known rework gaps to fix at port time, not inherit: derive-time
310execution is unsandboxed in-process (`code-executor.ts:23` records it) — our
311port MUST run publish-time evaluation in the existing sandbox `collect` path
312from day one; fan-in computed groups unsupported; OAuth discovery
313unsupported.
314
315**Semantic deltas — decided by Rhys before 3.1 specs:**
316
3171. **Handler credential contract (the deep one).** Current: handlers get an
318 opaque callable `AppIntegrationClient` proxy that re-enters
319 `ctx.execute` — app code never sees credentials, can only call other
320 executor tools. Rework (D49): engine resolves credentials once and hands
321 the handler **credential fields as data** (`ResolvedConnection.fields`,
322 e.g. access_token) for direct HTTP — that's what makes generators
323 possible (generated code does raw fetch with auth) and what
324 `defineProvider` exists for. Not either/or in principle (a tool-call
325 bridge can coexist as a second requirement kind later), but the ported
326 model's primary contract is fields-as-data per the rework.
3272. **Publish input.** Rework publish takes `SourceFiles` (push-based).
328 Decision: keep git/local-directory as _fetchers that produce
329 SourceFiles_ (preserves prod sources, the e2e suite, and the migration
330 path; git-client survives), with direct push publish added as the new
331 API. Delete-git-later stays open.
3323. **App identity** moves from caller-supplied slug to the `defineApp`
333 export inside the source (rework shape).
3344. **Versioning**: content hash of source IS the version identity; the
335 caller-supplied `sourceRef` + descriptor v6 die with the old model.
3365. **Ownership**: app/version rows carry `space` and NO owner — spaces
337 curate apps, principals own custody (connections stay on the legacy
338 owner model in this phase; the opaque-OwnerRef swap is Phase 4).
339
340### Phase 3 PR sequence (revised)
341
342- **3.1** Port the model core as internal packages, no readers: authoring
343 SDK + runtime brands, publish pipeline (evaluation routed through the
344 sandbox `collect` seam), Storage port + adapter, greenfield tables
345 (`app`, `app_version`, `computed_surface` — space column, content-hash
346 identity, `current` pointer) in coreTables + cloud mirror + migration.
347 Ported rework tests come along (authoring-boundary, tool-groups,
348 publish).
349- **3.2** Swap the apps plugin's internals onto the new model behind the
350 same costume: resolveTools/projectToolSchema/invokeTool read the new
351 tables; publish endpoint (`POST` source files); setCurrent/rollback +
352 versions listing routes; git/local-dir sync becomes fetch→publish.
353 Tool addresses and slug guards preserved; old collections
354 (`apps_descriptors`/`apps_tools`/`apps_sources`) still written by
355 NOTHING — one-off migration converts existing sources (re-fetch from
356 git), then the collections are dead. e2e: full existing custom-tools
357 suite green + new version-history/rollback scenario.
358- **3.3** Console UI for the new model (sources → apps with versions,
359 publish status, rollback) replacing the ~1.3k LoC old client/panels.
360- **3.4** Generators: fromOpenAPI + fromMCP + GeneratorClaim verification +
361 render endpoint, surfaced as the "add from spec/server" on-ramp.
362 (Vendors google/microsoft conversion ~1.15k LoC ports later with 0.2.)
363
364## Spaces: the rework's tenancy primitive (added Rhys, 2026-07-28)
365
366The rework's SDK has exactly one tenancy concept: `SpaceId`
367(`.reference/executor-rework/docs/TENANCY.md`, `docs/MODEL.md`). Adopt it as
368the end-state tenancy model here. The shape:
369
370- **Flat, no hierarchy, no default.** No parent, no inheritance, no
371 distinguished rows (rework D39/D44b). Sharing is an explicit act, never a
372 tree walk (R2 — re-ruled twice, do not relitigate). Every scoped row
373 carries a `space` column; every space-scoped route carries the space in
374 the URL path, never ambient context.
375- **No org/project/team/user below the deployment edge.** The engine answers
376 "what is in this space"; the deployment answers "who is this and which
377 spaces may they touch" through named seams (`authorizeBrowserResource`,
378 `dashboardSpacesForPrincipal`, MCP space resolver). Membership is
379 authorized LIVE per request at the edge — projection may be cached,
380 authorization never.
381- **Deployment mappings:** cloud = one WorkOS org mirrors lazily to one
382 space (mirror row `workosOrgId → space` is also the billing join — Autumn
383 keys on workosOrgId, never a space); self-host = one instance, one
384 found-or-created "Default" space, Better Auth org repurposed as the
385 instance membership table; local/desktop = one "local" space, one static
386 principal, single-tenancy declared via explicit sentinel, not defaulted.
387- **Spaces curate, principals own.** Custody (secrets, connections'
388 secret-bearing halves) belongs to principals via opaque `OwnerRef` +
389 per-request `Reach` (R3); spaces curate apps and hold rows. Publishing an
390 app into a space IS its presence there — no installation rows, no
391 bindings; cross-space arrival (sharing/registry) is a future share-edge
392 row, not golden-path ceremony (D45/D42).
393- Cross-org OAuth callbacks complete into the _initiating_ org's space; the
394 org header is a routing hint, not an authorization statement (R7).
395
396**DECIDED (Rhys, 2026-07-28): spaces land in the CORE PRODUCT NOW — Phase S,
397before the apps port.** Apps then publishes into real spaces from birth.
398
399**Mapping onto this repo — the no-migration trick:** today's `tenant` column
400is de facto the space — cloud writes the WorkOS org id there, self-host/local
401write one constant. So: keep the persisted `tenant` column name (same rule as
402`plugin_id`), and declare that **its value IS the space id**. Each org's
403first space is grandfathered with `spaceId = <historical tenant value>` (the
404org id); NEW spaces mint fresh ids written into the same column. The org→
405space mirror table records which space ids belong to which org (and carries
406the billing join — Autumn keys on workosOrgId, never a space). Existing rows
407never move; multi-space needs no schema change on scoped tables. "Space" is
408contract vocabulary; "tenant" retreats to cloud-internal (see Vocabulary).
409
410### Phase S — spaces in core (before apps)
411
412- **S1 — space table + boot ensure (no readers).** Greenfield `space` table
413 (id, name, created_at) in coreTables + cloud mirror + migration, plus the
414 cloud `org→space` mirror table. Boot/first-contact ensures the degenerate
415 space per deployment: cloud lazily on first authenticated org contact
416 (grandfathered id = org id), self-host "Default", local "local". Same
417 no-readers pattern as PR 1.1.
418- **S2 — space threading.** `SpaceId` brand in the SDK; the scoped-executor
419 seam resolves request → space id (today: identical to tenant) and stamps
420 it; reads interpret `tenant` columns as space ids. No behavior change —
421 this is the retype + vocabulary PR. `reach: "tenant"` reads as
422 "space-wide reach" from here on.
423- **S3 — multi-space on cloud + product surface.** Create/rename space,
424 space switcher in the console, membership = org membership (all spaces of
425 the org for now), org→N mirror rows, space-scoped catalog/connection
426 views (rows follow automatically — they're keyed by the space id in the
427 tenant column). Billing stays org-keyed via the mirror. e2e: two spaces
428 in one org, isolation of catalogs/connections between them, switcher.
429- **S4 — sharing model on spaces.** Connect-time "share with space" writes
430 the space's shared owner-ref (replaces the org tier semantically; the
431 persisted `owner: "org"|"user"` column keeps its name, read-side
432 reinterpretation only). Edge computes reach as
433 [principal, sharedRef(space) for each member space]. This is the bridge
434 to Phase 4's full owner-set reach.
435- **Later (not scheduled): local multi-space, file-backed.** Each local
436 space is its own store file (portable — copy/backup/sync/check in
437 alongside a repo); the daemon opens one storage handle per space and
438 routes by space id; the engine still sees flat SpaceIds. Custody never
439 enters the space file — secrets stay in the principal-keyed provider
440 (keychain), so a copied file carries connection metadata, never tokens.
441- Phase 4 completes the picture: space-in-path routes,
442 `dashboardSpacesForPrincipal`-style seams, reach as an explicit owner
443 set.
444
445Apps (Phase 3) then lands after S1/S2 — app rows carry a real `space` FK
446from birth; S3/S4 can proceed in parallel with the apps port.
447
448Interaction with shipped Phase 1 work: the subject table and platform view
449are edge/deployment concepts (who exists in this identity world, what may
450an operator read) — they sit above spaces and survive unchanged; `reach:
451"tenant"` becomes "space-wide reach" vocabulary-wise at Phase 4.
452
453## Vocabulary: retire "tenant" from consumer-facing surfaces (Rhys, 2026-07-27)
454
455Direction: the SDK should be agnostic to the consumer's data-model shape, the
456way Composio's is — consumers pass an opaque user/owner id string and hang
457whatever hierarchy they have (user, team, workspace) off it; the SDK never
458models their org chart. The rework already designed this (`OwnerRef` opaque
459brand, `Reach = {owners}`, deployments translate identity at the edge) — adopt
460its vocabulary, not its schedule. "Tenant" remains a real _internal_ concept
461(which Executor-cloud org owns the rows) but must not leak into surfaces
462consumers touch.
463
464Rules, effective immediately:
465
466- Persisted column names (`tenant`, `subject`, `owner`) stay — renaming
467 storage is a cross-host migration for zero behavior (same rule as
468 `plugin_id`).
469- Internal policy code (`reach: "tenant"`, `ExecutorOwnerPolicyContext`) may
470 keep the words.
471- One contract noun: **`owner`** — the opaque ref naming who a connection
472 belongs to (end-user, team, workspace, or the org itself; the rework's
473 `OwnerRef`). "User" cannot be the contract noun because org-shared
474 connections are a load-bearing feature an end-user word can't cover.
475- **"user" is presentation vocabulary only**, for surfaces whose rows genuinely
476 are the customer's end-users — the Phase 1.5 admin API says `users` because
477 the subject table is populated from authenticated principals.
478- Hazard: the persisted `owner` column keeps its old meaning (the
479 `"org" | "user"` tier discriminator), while Phase 4 introduces the opaque
480 ref as a distinctly named type (`OwnerRef`). Same word, two meanings,
481 separated by layer — say so in policy-layer comments.
482- The full swap (opaque owner refs at the SDK boundary, tenant retreating to a
483 cloud-internal detail) lands with the Phase 4 minimal SDK.
484
485## Phase 4 — minimal SDK + platform/product split
486
487New consumer boundary modeled on the rework: `createExecutor` /
488`createRemoteExecutor`, no Effect at the boundary, `tools.list/execute` +
489connections + the new users/admin surface; policies/toolkits become optional
490layers consumers opt into, not baked opinions (Composio-style minimalism).
491
492Authorization end-state (Rhys, 2026-07-27): the engine enforces scope
493mechanically but never decides policy — the host computes, per request, the
494set of owners the request may read (the rework's `Reach = { owners }`), and
495the storage predicate is just `row.owner ∈ reach.owners`. The 1.3
496`reach: "bound" | "tenant"` enum is a stepping stone and is replaced by this
497set shape here: "bound" = a reach of one, "tenant" = all owners (wildcard so
498cloud needn't enumerate), and in-between cases (team views, support sessions,
499subset-restricted org keys) are just different sets. Writes stay hard-pinned
500to a single owner; only read reach generalizes. Trust model becomes a
501per-deployment choice: a trusted backend with a server key gets tenant-wide
502reach and layers its own authorization (Composio-style); an end-user session
503gets a reach of one. The enum-to-set swap is contained: the policy context is
504constructed in exactly two places.
505Start the platform/product package separation _during_ 2.3–2.6 (each PR moves
506one thing to its final home) rather than as a big bang; the SDK boundary
507itself lands last.
508
509## Bookkeeping
510
511- Create `MISTAKES.md` / `DESIRES.md` / `LEARNINGS.md` (required by AGENTS.md,
512 currently missing) with the first entries from this migration.
513- `apps/local/src/db/executor-schema.ts` is already stale vs `coreTables`
514 (missing several columns); its "keep in sync" comment is untrustworthy —
515 runtime schema comes from the ensure path, not this file.

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t