Promote a named cloud version, decoupled from upload/migrate

A promote-only workflow_dispatch previously re-ran migrate and upload-cloud (building a fresh, un-canaried version) and then promoted whatever was newest, so advancing a canary could ship code that landed on main after the canary started. Decouple promotion: the promote-cloud job no longer needs upload, and migrate/upload/deploy-marketing run on push only. The version to promote is now an explicit promote_version input (a version id, or the literal latest as an opt-in), and promote-cloud.ts promotes exactly that version. Also guard the script against running with a wrangler config whose worker name is not executor-cloud.

Rhys Sullivan committed 4 months ago

Commit 5d11ea51 parent 4b260753 files · +182 −58Browse files

Showing the changes to apps/cloud/scripts (1 of 3 files). Show all

Unified · Split

apps/cloud/scripts/promote-cloud.ts+91−28

@@ -1,21 +1,32 @@
1	1	/* oxlint-disable executor/no-error-constructor, executor/no-try-catch-or-throw -- boundary: out-of-band deploy-promotion CLI */
2	2
3	3	/**
4		− * Promote the LATEST uploaded executor-cloud Worker version to a chosen
5		− * percentage of live traffic, using Cloudflare's staged gradual-deployment
6		− * model (`wrangler versions deploy`).
	4	+ * Promote a specific, already-uploaded executor-cloud Worker version to a
	5	+ * chosen percentage of live traffic, using Cloudflare's staged
	6	+ * gradual-deployment model (`wrangler versions deploy`).
7	7	*
8	8	* This is the second half of the upload -> staged-promotion deploy flow
9	9	* (see apps/cloud/DEPLOY.md and .github/workflows/deploy.yml). Merges upload a
10	10	* version but route no traffic to it; this script promotes it deliberately.
11	11	*
12		− * Behaviour:
13		− * --percent 100 promote the latest uploaded version to 100% (full cutover).
14		− * --percent 10 split traffic 10% latest / 90% current-live version, for a
15		− * canary step. Re-run at a higher percentage to advance the
16		− * rollout; the DOs whose sessions land on the newly promoted
17		− * version restart, so keep early steps small and watch
18		− * telemetry between steps.
	12	+ * The version to promote is chosen EXPLICITLY, never inferred silently:
	13	+ * --version <id> promote exactly that version id (from
	14	+ * `wrangler versions list`). This is what advances an
	15	+ * existing canary: re-run at a higher --percent for the SAME
	16	+ * --version so you widen the version you already reviewed,
	17	+ * not whatever landed on main since.
	18	+ * --version latest promote the most recently uploaded version. This is an
	19	+ * explicit opt-in for "promote what I just merged"; it is
	20	+ * NEVER the default, so advancing a rollout can never
	21	+ * accidentally promote unreviewed code.
	22	+ *
	23	+ * Percentage:
	24	+ * --percent 100 promote the chosen version to 100% (full cutover).
	25	+ * --percent 10 split traffic 10% chosen / 90% current-live version, for a
	26	+ * canary step. Re-run at a higher percentage (same version)
	27	+ * to advance; the DOs whose sessions land on the newly
	28	+ * promoted version restart, so keep early steps small and
	29	+ * watch telemetry between steps.
19	30	*
20	31	* IMPORTANT: a promotion that changes a Durable Object's assigned version
21	32	* restarts that DO (validated empirically, DEPLOY.md experiments 3b/5). Stage
@@ -25,11 +36,36 @@ · 3 unmodified lines
25	36	* in the CI env). Config is the generated dist config the build produced.
26	37	*/
27	38	import { spawnSync } from "node:child_process";
	39	+import { readFileSync } from "node:fs";
28	40
29	41	// The generated dist config the build produced. Overridable via
30	42	// PROMOTE_CLOUD_CONFIG for local/lab testing against a non-prod worker.
31	43	const CONFIG = process.env.PROMOTE_CLOUD_CONFIG ?? "dist/server/wrangler.json";
32	44
	45	+// Target-worker guard: refuse to run against anything but executor-cloud unless
	46	+// the operator explicitly names the worker they expect via
	47	+// PROMOTE_CLOUD_ALLOW_NAME (for the deploy lab / a non-prod worker). A promotion
	48	+// routes live traffic and restarts DOs, so a misconfigured config that points at
	49	+// the wrong worker must fail loudly, not promote the wrong thing. `--yes` does
	50	+// NOT bypass this: it only skips wrangler's own interactive confirmation.
	51	+const EXPECTED_WORKER_NAME = process.env.PROMOTE_CLOUD_ALLOW_NAME ?? "executor-cloud";
	52	+{
	53	+ let configName: unknown;
	54	+ try {
	55	+ configName = (JSON.parse(readFileSync(CONFIG, "utf8")) as { name?: unknown }).name;
	56	+ } catch (error) {
	57	+ throw new Error(
	58	+ `could not read wrangler config ${CONFIG} to verify the target worker: ${String(error)}`,
	59	+ );
	60	+ }
	61	+ if (configName !== EXPECTED_WORKER_NAME) {
	62	+ throw new Error(
	63	+ `refusing to promote: wrangler config ${CONFIG} targets worker ${JSON.stringify(configName)}, expected ${JSON.stringify(EXPECTED_WORKER_NAME)}. ` +
	64	+ `Set PROMOTE_CLOUD_ALLOW_NAME to override for a non-production worker.`,
	65	+ );
	66	+ }
	67	+}
	68	+
33	69	const args = process.argv.slice(2);
34	70	const argValue = (name: string): string \| undefined => {
35	71	const i = args.indexOf(name);
@@ -45,6 +81,16 @@ if (!Number.isInteger(percent) \|\| percent < 1 \|\| percent > 100) { · 9 unmodified lines
45	81	throw new Error(`--percent must be an integer 1-100, got: ${percentRaw}`);
46	82	}
47	83
	84	+// The version to promote is required and explicit: an id, or the literal
	85	+// `latest` to opt in to "promote the most recently uploaded version".
	86	+const versionArg = argValue("--version") ?? process.env.PROMOTE_VERSION;
	87	+if (!versionArg) {
	88	+ throw new Error(
	89	+ "--version is required: pass an explicit version id (from `wrangler versions list`), " +
	90	+ "or the literal `latest` to promote the most recently uploaded version.",
	91	+ );
	92	+}
	93	+
48	94	const wrangler = (wranglerArgs: readonly string[]): string => {
49	95	const result = spawnSync("bun", ["run", "wrangler", ...wranglerArgs], {
50	96	encoding: "utf8",
@@ -67,21 +113,38 @@ type DeploymentStatus = { · 16 unmodified lines
67	113	readonly versions: ReadonlyArray<{ readonly version_id: string; readonly percentage: number }>;
68	114	};
69	115
70		−// `versions list --json` order is not guaranteed, so sort explicitly by
71		−// version number (monotonic per upload) descending to get the newest.
72	116	const versions = JSON.parse(
73	117	wrangler(["versions", "list", "-c", CONFIG, "--json"]),
74	118	) as VersionListEntry[];
75	119	if (versions.length === 0) {
76	120	throw new Error("no uploaded versions found");
77	121	}
78		−const sorted = [...versions].sort((a, b) => {
79		− if (a.number !== b.number) return b.number - a.number;
80		− const at = a.metadata?.created_on ?? "";
81		− const bt = b.metadata?.created_on ?? "";
82		− return bt.localeCompare(at);
83		−});
84		−const latest = sorted[0];
	122	+
	123	+// Resolve the target version EXPLICITLY. `latest` is an opt-in that sorts by
	124	+// version number (monotonic per upload) descending, since `versions list`
	125	+// order is not guaranteed. Any other value is treated as a version id and must
	126	+// match an uploaded version exactly, so a typo or a since-deleted id fails
	127	+// loudly rather than promoting the wrong version.
	128	+let target: VersionListEntry;
	129	+if (versionArg === "latest") {
	130	+ const sorted = [...versions].sort((a, b) => {
	131	+ if (a.number !== b.number) return b.number - a.number;
	132	+ const at = a.metadata?.created_on ?? "";
	133	+ const bt = b.metadata?.created_on ?? "";
	134	+ return bt.localeCompare(at);
	135	+ });
	136	+ target = sorted[0];
	137	+} else {
	138	+ const found = versions.find((v) => v.id === versionArg);
	139	+ if (!found) {
	140	+ throw new Error(
	141	+ `version ${versionArg} not found among uploaded versions. ` +
	142	+ `Run \`wrangler versions list -c ${CONFIG}\` to see valid ids, ` +
	143	+ "or pass `latest` to promote the most recent upload.",
	144	+ );
	145	+ }
	146	+ target = found;
	147	+}
85	148
86	149	// The version(s) currently serving live traffic, so a partial promotion holds
87	150	// the remainder on the current-live version rather than an arbitrary one.
@@ -90,24 +153,24 @@ const status = JSON.parse( · 2 unmodified lines
90	153	) as DeploymentStatus;
91	154	// The version to hold the remainder of a partial promotion: the current-live
92	155	// version carrying the most traffic that is NOT the one we are promoting. If
93		−// the only current version IS the latest, there is nothing to split against.
	156	+// the only current version IS the target, there is nothing to split against.
94	157	const currentLive = [...status.versions]
95		− .filter((v) => v.version_id !== latest.id)
	158	+ .filter((v) => v.version_id !== target.id)
96	159	.sort((a, b) => b.percentage - a.percentage)[0]?.version_id;
97	160
98		−if (latest.id === currentLive && percent === 100) {
99		− console.log(`Latest version ${latest.id} is already live at 100%; nothing to promote.`);
	161	+if (target.id === currentLive && percent === 100) {
	162	+ console.log(`Version ${target.id} is already live at 100%; nothing to promote.`);
100	163	process.exit(0);
101	164	}
102	165
103	166	const deployArgs = ["versions", "deploy", "-c", CONFIG, "--yes"];
104		−if (percent === 100 \|\| !currentLive \|\| currentLive === latest.id) {
105		− deployArgs.push(`${latest.id}@100%`);
	167	+if (percent === 100 \|\| !currentLive \|\| currentLive === target.id) {
	168	+ deployArgs.push(`${target.id}@100%`);
106	169	} else {
107		− // Staged split: latest@percent, current-live@remainder.
108		− deployArgs.push(`${latest.id}@${percent}%`, `${currentLive}@${100 - percent}%`);
	170	+ // Staged split: target@percent, current-live@remainder.
	171	+ deployArgs.push(`${target.id}@${percent}%`, `${currentLive}@${100 - percent}%`);
109	172	}
110	173
111		−console.log(`Promoting ${latest.id} (version #${latest.number}) at ${percent}%...`);
	174	+console.log(`Promoting ${target.id} (version #${target.number}) at ${percent}%...`);
112	175	console.log(wrangler(deployArgs));
113	176	console.log("Done. Monitor telemetry, then re-run at a higher percentage to advance the rollout.");