Drop the throwaway deploy-lab worker from the branch

Rhys Sullivan committed 4 months ago

Commit f1be2b91 parent dda3dad5 files · +18 −347Browse files

Unified · Split

5 files

apps/cloud/DEPLOY.md +16 −33
apps/cloud/scripts/deploy-lab/README.md +0 −86
apps/cloud/scripts/deploy-lab/worker.ts +0 −192
apps/cloud/scripts/deploy-lab/wrangler.jsonc +0 −34
apps/cloud/scripts/promote-cloud.ts +2 −2

@@ -7,9 +7,10 @@ streaming sessions: a cutover mid-call drops in-flight MCP tool results. Staging · 6 unmodified lines
7	7	the promotion bounds that blast radius and keeps a mere merge from disrupting
8	8	anything.
9	9
10		−The mechanics below were validated empirically on a throwaway lab worker
11		−(`executor-cloud-deploy-lab`, see `scripts/deploy-lab/`); the "what happens"
12		−claims are measured, not assumed. The findings are summarized at the end.
	10	+The mechanics below were validated empirically on a throwaway staging worker
	11	+that mounted the real session DO and patched transport with stub auth; the
	12	+"what happens" claims are measured, not assumed. The findings are summarized
	13	+at the end.
13	14
14	15	## The flow
15	16
@@ -66,7 +67,7 @@ The script promotes exactly the version you name (`latest` resolves to the · 50 unmodified lines
66	67	highest version number) and holds the remainder of a partial promotion on the
67	68	current-live version. It refuses to run unless the resolved wrangler config
68	69	targets the `executor-cloud` worker (set `PROMOTE_CLOUD_ALLOW_NAME` to override
69		−for a non-production worker such as the deploy lab).
	70	+for a non-production worker).
70	71
71	72	## Session version-affinity (zone ruleset, MUST be configured manually)
72	73
@@ -104,7 +105,7 @@ session to pin yet. Once the client echoes `mcp-session-id`, every subsequent · 31 unmodified lines
104	105	request for that session is version-pinned to the worker version its DO landed
105	106	on.
106	107
107		−Validated on the lab worker by sending `Cloudflare-Workers-Version-Key`
	108	+Validated on the staging worker by sending `Cloudflare-Workers-Version-Key`
108	109	directly (12 requests per key across a 4-key set during a two-version gradual
109	110	deployment): each distinct key deterministically mapped to exactly one version
110	111	across all its requests, while unkeyed requests split across versions per the
@@ -150,7 +151,7 @@ Constraints: · 39 unmodified lines
150	151
151	152	## What the staged flow does and does not fix
152	153
153		−Measured on the lab worker:
	154	+Measured on the staging worker:
154	155
155	156	- Fixed, merges no longer disrupt anything. `wrangler versions upload` does
156	157	not restart DOs; a session opened before an upload runs straight through it.
@@ -180,31 +181,13 @@ Measured on the lab worker: · 23 unmodified lines
180	181	and even a promotion loses only the single in-flight call, not the session or
181	182	its history.
182	183
183		−## The lab worker (`scripts/deploy-lab/`)
	184	+## Re-validating the mechanics
184	185
185		−`scripts/deploy-lab/` is a self-contained, re-runnable staging worker that
186		−mounts the real `McpAgentSessionDOBase` + the patched `agents` transport
187		−with stub auth and a trivial `execute` tool, so the versions / gradual-
188		−deployment / affinity / rollback mechanics can be validated against the real
189		−transport with no production secrets. Deploy it as
190		−`executor-cloud-deploy-lab` (workers_dev only, never routed):
191		−
192		−```bash
193		−# from the repo root (bypasses apps/cloud's dist config redirect)
194		−wrangler deploy -c apps/cloud/scripts/deploy-lab/wrangler.jsonc --var LAB_VERSION:v1
195		−```
196		−
197		−It exposes `/mcp` (real streamable-HTTP MCP) and `/__lab/version` (echoes the
198		−serving worker version, for observing traffic splits and affinity). Delete it
199		−when done (`wrangler delete -c apps/cloud/scripts/deploy-lab/wrangler.jsonc`).
200		−
201		−> workers.dev caveat (lab only). The lab is reached over its
202		−> `*.workers.dev` subdomain, which had to be explicitly enabled once (Workers
203		−> dashboard -> the worker -> Settings -> Domains & Routes -> enable the
204		−> `workers.dev` route) before requests resolved; the config's `workers_dev:
205		−true` alone was not sufficient on first deploy. This does not apply to
206		−> production executor-cloud, which is served through configured routes on
207		−> `executor.sh`, not a workers.dev subdomain.
208		−
209		−See `scripts/deploy-lab/README.md` for the full experiment runbook and the
210		−measured results.
	186	+The measurements above came from a throwaway staging worker (a cut-down worker
	187	+mounting the real `McpAgentSessionDOBase` + the patched `agents` transport with
	188	+stub auth and a trivial `execute` tool, deployed to a `workers.dev` subdomain,
	189	+never routed). To re-validate after a Cloudflare behavior change, rebuild the
	190	+same shape: exercise affinity with `Cloudflare-Workers-Version-Key` across a
	191	+two-version gradual deployment, fire an upload and a promotion mid-call, and
	192	+roll back mid-session. Keep any such worker on `workers.dev` only and delete it
	193	+when done.

apps/cloud/scripts/deploy-lab/README.md+0−86

@@ -1,86 +0,0 @@
1		−# executor-cloud-deploy-lab
2		−
3		−A throwaway staging worker for validating the executor-cloud
4		−upload -> staged-promotion deploy mechanics against real Cloudflare
5		−deployment machinery (versions, gradual deployments, version affinity, rollback)
6		−and the patched `agents` worker<->DO MCP bridge. It mounts the real shared
7		−MCP session Durable Object base (`McpAgentSessionDOBase`) with the real
8		−patched transport (`patches/agents@0.17.3.patch`), and stubs only the heavy
9		−cloud seams (Postgres/Hyperdrive, WorkOS, Autumn, Sentry/OTEL) so it boots with
10		−no production secrets. See `worker.ts` for the stubs.
11		−
12		−Never point this at production. `workers_dev` only, no routes, no custom
13		−domains, no crons. It is safe to `wrangler delete` at any time.
14		−
15		−The narrative conclusions from these experiments live in
16		−[`../../DEPLOY.md`](../../DEPLOY.md); this file is the runbook that produced them.
17		−
18		−## Deploy / redeploy
19		−
20		−Deploy from the repo root so wrangler picks up this config directly (bypassing
21		−`apps/cloud`'s dist redirect). Each version is uploaded with a distinct
22		−`LAB_VERSION` var so every response echoes which worker version served it:
23		−
24		−```bash
25		−# Upload a version WITHOUT routing traffic (returns a version id):
26		−wrangler versions upload -c apps/cloud/scripts/deploy-lab/wrangler.jsonc --var LAB_VERSION:v6
27		−wrangler versions upload -c apps/cloud/scripts/deploy-lab/wrangler.jsonc --var LAB_VERSION:v7
28		−
29		−# Promote / split traffic:
30		−wrangler versions deploy -c apps/cloud/scripts/deploy-lab/wrangler.jsonc <v6-id>@90% <v7-id>@10% --yes
31		−```
32		−
33		−> workers.dev must be enabled once. `workers_dev: true` in the config was
34		−> not enough on first deploy: the `*.workers.dev` route had to be turned on
35		−> manually (Workers dashboard -> the worker -> Settings -> Domains & Routes ->
36		−> enable workers.dev) before requests resolved. Production is served through
37		−> `executor.sh` routes, so this caveat is lab-only.
38		−
39		−## Endpoints
40		−
41		−- `POST/GET/DELETE /mcp`: the real streamable-HTTP MCP surface (session create,
42		− in-flight `tools/call`, POST-stream disconnect + GET replay). The single tool
43		− `execute` sleeps `delayMs` then echoes `{ marker, servedByVersion }`, enough
44		− to hold a ~30-60s call open across a mid-call deploy.
45		−- `GET /__lab/version`: echoes `{ servedByVersion, receivedVersionKey }` and an
46		− `x-lab-version` header, for observing which worker version served a request
47		− and whether an affinity key was received.
48		−
49		−## Experiment runbook and measured results
50		−
51		−All results below were measured against real Cloudflare deployments of this
52		−worker (2026-07-07).
53		−
54		−- Exp3a, no affinity key. 30 fresh connections over a 10/90 two-version
55		− split: 28 hit v6, 2 hit v7. Confirms per-request (weighted) routing, a
56		− stateless worker with no key does not stick to one version.
57		−- Exp3b, with `Cloudflare-Workers-Version-Key`. 12 requests per key across a
58		− 4-key set (alpha/bravo/charlie/delta) during the same split: each key mapped
59		− to exactly one version for all 12 of its requests (alpha/bravo/delta -> v6,
60		− charlie -> v7), and keys distributed across both versions. **Perfect per-key
61		− stickiness**, this is the affinity mechanism DEPLOY.md's zone rule relies on.
62		−- Exp3c, DO pinning. One MCP session's DO answered on v6 for every request
63		− including while the stateless front-worker hops varied version. DOs are
64		− natively version-pinned.
65		−- Exp4, rollback. `versions deploy old@100` took effect within seconds
66		− (10/10 subsequent requests on v6). A session that predated the rollback still
67		− worked (`tools/list` -> 200) and its DO storage was intact.
68		−- Exp5, upload mid-call. `versions upload` fired during a ~30s in-flight
69		− call caused zero disruption, the call completed and its marker was
70		− delivered (`EXP5_UPLOAD_SAFE`). Uploads are free.
71		−- Exp5b, promotion mid-call. `versions deploy new@100` fired during a ~30s
72		− in-flight call killed the in-flight execution (client saw the stream end
73		− with no result, expected until durable-execution work lands). But the
74		− session survived, the persisted event store survived the version swap, and a
75		− reconnect GET replayed the prior events across the version boundary.
76		−
77		−Conclusion. The upload/promote split plus session affinity works as
78		−documented. Promotion is the only disruptive moment, and even then it loses only
79		−the single in-flight call, not the session or its replayable history. The replay
80		−machinery is version-swap-safe.
81		−
82		−## Cleanup
83		−
84		−```bash
85		−wrangler delete -c apps/cloud/scripts/deploy-lab/wrangler.jsonc
86		−```

apps/cloud/scripts/deploy-lab/worker.ts+0−192

@@ -1,192 +0,0 @@
1		−// ---------------------------------------------------------------------------
2		−// executor-cloud-deploy-lab, a CUT-DOWN staging worker used ONLY to validate
3		−// the upload->staged-promotion deploy mechanics (versions, gradual deployments,
4		−// version affinity, rollback) and the patched MCP bridge transport under real
5		−// Cloudflare deployment machinery.
6		−//
7		−// It mounts the REAL shared MCP session Durable Object base
8		−// (`McpAgentSessionDOBase` from @executor-js/cloudflare) with the REAL patched
9		−// `agents` transport (patches/agents@0.17.3.patch, the worker<->DO websocket
10		−// bridge). The heavy cloud seams (Postgres/Hyperdrive, WorkOS, Autumn billing,
11		−// Sentry/OTEL) are replaced with trivial stubs so the worker boots with NO
12		−// production secrets. The MCP transport path, session create, in-flight
13		−// tools/call, POST-stream disconnect + GET replay, runs for real.
14		−//
15		−// NEVER point this at production. workers_dev only, no routes/domains/crons.
16		−// ---------------------------------------------------------------------------
17		−import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
18		−import { Effect } from "effect";
19		−import * as z from "zod/v4";
20		−
21		−import {
22		− McpAgentSessionDOBase,
23		− type BuiltMcpServer,
24		− type McpSessionInit,
25		− type McpSessionProps,
26		− type SessionDbHandle,
27		− type SessionMeta,
28		−} from "@executor-js/cloudflare/mcp/agent-durable-object";
29		−import {
30		− withVerifiedIdentityHeaders,
31		− readElicitationMode,
32		−} from "@executor-js/cloudflare/mcp/do-headers";
33		−import { defaultMcpResource } from "@executor-js/host-mcp";
34		−
35		−// The version label is injected per-version via a `vars` override at
36		−// `wrangler versions upload` time (`--var LAB_VERSION:vN`). Every response
37		−// echoes it so a client can observe WHICH worker version served each request
38		−// (the crux of the affinity experiment) without needing dashboard access.
39		−interface LabEnv extends Cloudflare.Env {
40		− readonly MCP_SESSION: DurableObjectNamespace;
41		− readonly LAB_VERSION?: string;
42		−}
43		−
44		−// A no-op ExecutionEngine stub. The base only touches the engine on the
45		−// pause/resume approval path; a plain tools/call runs the tool handler
46		−// directly, so these are never exercised by the deploy experiments. Shape
47		−// mirrors packages/core/execution/src/engine.ts::ExecutionEngine.
48		−const makeStubEngine = () =>
49		− // oxlint-disable-next-line executor/no-double-cast -- lab-only test double: the engine is never exercised by the deploy experiments (no pause/resume), so a structural stub standing in for the full ExecutionEngine is intentional.
50		− ({
51		− execute: () => Effect.succeed({ result: "lab-execute" }),
52		− executeWithPause: () =>
53		− Effect.succeed({ status: "completed", result: { result: "lab-execute" } }),
54		− resume: () => Effect.succeed({ status: "completed", result: { result: "lab-resume" } }),
55		− getPausedExecution: () => Effect.succeed(null),
56		− pausedExecutionCount: () => Effect.succeed(0),
57		− hasPausedExecutions: () => Effect.succeed(false),
58		− getDescription: Effect.succeed("lab stub engine"),
59		− }) as unknown as BuiltMcpServer["engine"];
60		−
61		−// The lab MCP server exposes a single `execute` tool that sleeps for a
62		−// caller-supplied duration and echoes a marker, enough to hold a ~60s
63		−// in-flight tool call open across a mid-call deploy (experiments 1 & 5).
64		−const buildLabMcpServer = (version: string): McpServer => {
65		− const server = new McpServer({ name: "executor-cloud-deploy-lab", version: "0.0.1" });
66		− server.registerTool(
67		− "execute",
68		− {
69		− description: "Sleep for delayMs then echo marker. Lab-only, no real execution engine.",
70		− inputSchema: {
71		− code: z
72		− .string()
73		− .optional()
74		− .describe("ignored (compat with the real execute tool signature)"),
75		− delayMs: z.number().optional().describe("how long to sleep before responding"),
76		− marker: z.string().optional().describe("string echoed back in the result"),
77		− },
78		− },
79		− async (args: { code?: string; delayMs?: number; marker?: string }) => {
80		− const delayMs = typeof args.delayMs === "number" ? args.delayMs : 0;
81		− const marker = args.marker ?? "LAB_RESULT";
82		− if (delayMs > 0) await new Promise((r) => setTimeout(r, delayMs));
83		− return {
84		− content: [
85		− { type: "text" as const, text: JSON.stringify({ marker, servedByVersion: version }) },
86		− ],
87		− };
88		− },
89		− );
90		− return server;
91		−};
92		−
93		−// The lab DO: the REAL base + REAL patched transport, trivial seams.
94		−export class McpSessionDOSqlite extends McpAgentSessionDOBase<LabEnv> {
95		− protected openSessionDb(): SessionDbHandle {
96		− return { end: () => undefined };
97		− }
98		−
99		− protected resolveSessionMeta(token: McpSessionInit): Effect.Effect<SessionMeta> {
100		− return Effect.succeed({
101		− organizationId: token.organizationId,
102		− organizationName: "Deploy Lab Org",
103		− userId: token.userId,
104		− elicitationMode:
105		− token.elicitationMode === "browser" \|\| token.elicitationMode === "native"
106		− ? token.elicitationMode
107		− : "model",
108		− resource: token.resource,
109		− webOrigin: token.webOrigin,
110		− });
111		− }
112		−
113		− protected buildMcpServer(): Effect.Effect<BuiltMcpServer> {
114		− const version = (this.env as LabEnv).LAB_VERSION ?? "unknown";
115		− return Effect.succeed({
116		− mcpServer: buildLabMcpServer(version),
117		− engine: makeStubEngine(),
118		− });
119		− }
120		−}
121		−
122		−// A stub auth layer: accepts ANY bearer, mints a fixed principal. This is the
123		−// ONLY thing standing in for the real WorkOS auth provider, the session
124		−// engine, DO, transport, and bridge below are the real, patched code.
125		−const LAB_PRINCIPAL = { accountId: "lab-user", organizationId: "lab-org" } as const;
126		−
127		−const serve = McpAgentSessionDOBase.serve("/mcp", {
128		− binding: "MCP_SESSION",
129		− transport: "streamable-http",
130		−}) as { fetch: (req: Request, env: LabEnv, ctx: ExecutionContext) => Promise<Response> };
131		−
132		−export default {
133		− async fetch(request: Request, env: LabEnv, ctx: ExecutionContext): Promise<Response> {
134		− const url = new URL(request.url);
135		− const version = env.LAB_VERSION ?? "unknown";
136		−
137		− // Version-echo endpoint: which worker version is serving THIS request.
138		− // Used to observe stateless-worker version affinity during a gradual
139		− // deployment. Also echoes the version-affinity header if the client set it.
140		− if (url.pathname === "/__lab/version") {
141		− return new Response(
142		− JSON.stringify({
143		− servedByVersion: version,
144		− receivedVersionKey: request.headers.get("Cloudflare-Workers-Version-Key"),
145		− }),
146		− { headers: { "content-type": "application/json", "x-lab-version": version } },
147		− );
148		− }
149		−
150		− if (url.pathname !== "/mcp") {
151		− return new Response("lab worker: only /mcp and /__lab/version", { status: 404 });
152		− }
153		−
154		− if (request.method === "OPTIONS") {
155		− return new Response(null, {
156		− status: 204,
157		− headers: {
158		− "access-control-allow-origin": "*",
159		− "access-control-allow-methods": "GET, POST, DELETE, OPTIONS",
160		− "access-control-allow-headers":
161		− "content-type, authorization, mcp-session-id, accept, mcp-protocol-version, last-event-id",
162		− "access-control-expose-headers": "mcp-session-id",
163		− },
164		− });
165		− }
166		−
167		− // Stub auth: any bearer -> fixed principal. Build props exactly like the
168		− // real agent-handler so the DO's onStart receives a valid McpSessionProps.
169		− const props: McpSessionProps = {
170		− session: {
171		− organizationId: LAB_PRINCIPAL.organizationId,
172		− userId: LAB_PRINCIPAL.accountId,
173		− elicitationMode: readElicitationMode(request),
174		− resource: defaultMcpResource,
175		− webOrigin: url.origin,
176		− },
177		− };
178		− (ctx as ExecutionContext & { props?: McpSessionProps }).props = props;
179		−
180		− const forwarded = withVerifiedIdentityHeaders(request, LAB_PRINCIPAL, defaultMcpResource);
181		− const response = await serve.fetch(forwarded, env, ctx);
182		− // Stamp the serving worker version on every MCP response header too, so the
183		− // client can log per-request affinity even for streamed responses.
184		− const headers = new Headers(response.headers);
185		− headers.set("x-lab-version", version);
186		− return new Response(response.body, {
187		− status: response.status,
188		− statusText: response.statusText,
189		− headers,
190		− });
191		− },
192		−};

apps/cloud/scripts/deploy-lab/wrangler.jsonc+0−34

@@ -1,34 +0,0 @@
1		−{
2		− // STAGING deploy-mechanics lab worker. NOT production. workers_dev only —
3		− // no routes, no custom_domain, no crons, no hyperdrive/r2/service bindings.
4		− // See scripts/deploy-lab/README.md for the experiment runbook.
5		− "$schema": "../../node_modules/wrangler/config-schema.json",
6		− "name": "executor-cloud-deploy-lab",
7		− "compatibility_date": "2025-04-01",
8		− "compatibility_flags": ["nodejs_compat"],
9		− "main": "worker.ts",
10		− "workers_dev": true,
11		− "observability": {
12		− "enabled": true,
13		− },
14		− "durable_objects": {
15		− "bindings": [
16		− {
17		− "name": "MCP_SESSION",
18		− "class_name": "McpSessionDOSqlite",
19		− },
20		− ],
21		− },
22		− "migrations": [
23		− {
24		− "tag": "v1",
25		− "new_sqlite_classes": ["McpSessionDOSqlite"],
26		− },
27		− ],
28		− "placement": {
29		− "region": "aws:us-east-1",
30		− },
31		− "vars": {
32		− "LAB_VERSION": "unset",
33		− },
34		−}

apps/cloud/scripts/promote-cloud.ts+2−2

@@ -39,12 +39,12 @@ import { spawnSync } from "node:child_process"; · 38 unmodified lines
39	39	import { readFileSync } from "node:fs";
40	40
41	41	// The generated dist config the build produced. Overridable via
42		−// PROMOTE_CLOUD_CONFIG for local/lab testing against a non-prod worker.
	42	+// PROMOTE_CLOUD_CONFIG for testing against a non-prod worker.
43	43	const CONFIG = process.env.PROMOTE_CLOUD_CONFIG ?? "dist/server/wrangler.json";
44	44
45	45	// Target-worker guard: refuse to run against anything but executor-cloud unless
46	46	// the operator explicitly names the worker they expect via
47		−// PROMOTE_CLOUD_ALLOW_NAME (for the deploy lab / a non-prod worker). A promotion
	47	+// PROMOTE_CLOUD_ALLOW_NAME (for a non-prod worker). A promotion
48	48	// routes live traffic and restarts DOs, so a misconfigured config that points at
49	49	// the wrong worker must fail loudly, not promote the wrong thing. `--yes` does
50	50	// NOT bypass this: it only skips wrangler's own interactive confirmation.

Drop the throwaway deploy-lab worker from the branch

Keyboard shortcuts

General

Issues and pull requests

Issues, pull requests and Inbox

Repository code