* e2e: VM substrate for cross-OS supervised-daemon targets + build --target Foundation for codifying the supervised-daemon reboot-survival proof as e2e tests (today it's only a by-hand bash harness). Adds a VM substrate that can provision a guest, drive it over SSH, REBOOT the OS for real, and tear it down: - e2e/src/vm/{types,tart,build-binary}.ts — the tart provider (macOS + Linux guests on an Apple-Silicon host) with a reconnecting SSH forward that survives a guest reboot, plus a helper to compile the guest's `executor` binary. - apps/cli/src/build.ts: a first-class `--target <name>` flag so the harness can build a specific platform's binary (replaces a throwaway env hack). Validated standalone: provision macOS guest → ssh → real reboot (boottime changes) → clean discard. Next: the `cli` target wires this into the scenario suite so restart-persistence runs against a real reboot. * e2e: cli target — supervised daemon in a VM, restart() is a real reboot Wires the VM substrate into the scenario suite as a `cli` target (one project per guest OS: cli-macos / cli-linux / cli-windows). The globalsetup builds the guest's executor binary, provisions a VM, `executor service install`s the supervised daemon, and forwards its loopback port over a reconnecting SSH tunnel — so target.baseUrl works unchanged for the api surface. restart() reboots the guest OS for REAL and waits for the daemon to auto-start, so the existing restart-persistence scenario now proves the boot-time auto-start path (launchd RunAtLoad / systemd linger / Task Scheduler AtStartup), not a process restart. Because globalsetup (main process) and restart() (test worker) are different processes, restart() re-derives the guest address from env and reboots statelessly, mirroring how selfhost-docker's restart re-derives container ops. Verified: `vitest run --project cli-macos` boots a macOS guest, installs the daemon, addSpec's an integration, reboots the guest, and asserts the data survived — green, with a run-record in the matrix. cli-linux reuses the tart provider; cli-windows awaits the ec2 provider. * e2e: recorded film — integration survives a real reboot (terminal cast) A watchable companion to restart-persistence: drives the same register → reboot → survived flow through the chat theater so the on-screen tool spinner runs for the ACTUAL guest reboot. Produces a terminal.cast the viewer plays in the matrix — press-play evidence instead of trusting a green check. Runs against the cli-* VM targets; verified on cli-macos (357-event cast). * e2e: desktop attach film — app connects to the supervised daemon (not a sidecar) Films the desktop ATTACHING to an already-running OS-supervised daemon instead of spawning its own sidecar: starts a real cli-daemon (the desktop sidecar server in EXECUTOR_SUPERVISED mode → kind "cli-daemon") against a throwaway HOME, launches Electron pointed at the same HOME, and proves the attach — the server manifest still names OUR daemon's pid (a spawned sidecar would be a new pid + kind "desktop-sidecar"). session.mp4 is the artifact. The daemon half is verified (EXECUTOR_READY + kind cli-daemon + 401). Like the existing desktop scenario it drives Electron via Playwright, so it needs a GUI display session to run (the desktop project is already display-gated, not in the default test chain) — run on a real desktop / CI-with-display to capture the video. * e2e/cli: bearer-gated daemon target, reboot down-gate, EC2 provider for Windows - cli target authenticates with the daemon's auth.json bearer (was Basic from service.key); globalsetup reads the token from the guest and publishes it. - restart() gates on the daemon going DOWN before polling it back up, so an orderly shutdown / reconnecting tunnel can't false-pass a reboot that never happened (waitForHttpDown). - EC2 VmProvider (e2e/src/vm/ec2.ts) + cli-windows globalsetup unblock Windows: launch, key-based SSH/PowerShell drive, real boot-time-gated reboot, terminate. * e2e/desktop + cli tests: bearer auth token, not a password The supervised-attach film passed EXECUTOR_AUTH_PASSWORD, a dead env the sidecar no longer reads (it reads EXECUTOR_AUTH_TOKEN and otherwise mints/loads the bearer from auth.json). Pin a real bearer instead. Fix two stale launchd/wrapper comments that still pointed the secret at service.key.
Unified · Split
| @@ -501,18 +501,24 @@ const buildPreviewTarballs = async (binaries: Record<string, string>) => { · 500 unmodified lines | ||
| 501 | 501 | } |
| 502 | 502 | }; |
| 503 | 503 | |
| 504 | +// Resolve a comma-separated list of target package names (e.g. | |
| 505 | +// "executor-windows-x64") to Targets. Shared by `--target` and the | |
| 506 | +// EXECUTOR_PREVIEW_TARGETS env used by the preview-wrapper CI job. | |
| 504 | 507 | const resolveTargetsFromEnv = (env: string | undefined): Target[] => { |
| 505 | − if (!env) throw new Error("EXECUTOR_PREVIEW_TARGETS must be set (comma-separated package names)"); | |
| 508 | + if (!env) throw new Error("No build targets given (comma-separated package names)"); | |
| 506 | 509 | const names = env |
| 507 | 510 | .split(",") |
| 508 | 511 | .map((s) => s.trim()) |
| 509 | 512 | .filter(Boolean); |
| 510 | 513 | const resolved = names.map((name) => { |
| 511 | 514 | const match = ALL_TARGETS.find((t) => targetPackageName(t) === name); |
| 512 | − if (!match) throw new Error(`Unknown preview target: ${name}`); | |
| 515 | + if (!match) { | |
| 516 | + const valid = ALL_TARGETS.map(targetPackageName).join(", "); | |
| 517 | + throw new Error(`Unknown build target: ${name}. Expected one of: ${valid}`); | |
| 518 | + } | |
| 513 | 519 | return match; |
| 514 | 520 | }); |
| 515 | − if (resolved.length === 0) throw new Error("EXECUTOR_PREVIEW_TARGETS resolved to an empty list"); | |
| 521 | + if (resolved.length === 0) throw new Error("Build target list resolved to empty"); | |
| 516 | 522 | return resolved; |
| 517 | 523 | }; |
| 518 | 524 | |
| @@ -919,6 +925,11 @@ const { values, positionals } = parseArgs({ · 400 unmodified lines | ||
| 919 | 925 | args: process.argv.slice(2), |
| 920 | 926 | options: { |
| 921 | 927 | single: { type: "boolean", default: false }, |
| 928 | + // Build a specific target (or comma-separated set) by package name, e.g. | |
| 929 | + // `--target executor-windows-x64`. Used by the e2e VM harness to compile | |
| 930 | + // the guest's binary; without it `binary` builds the current platform | |
| 931 | + // (`--single`) or all targets. | |
| 932 | + target: { type: "string" }, | |
| 922 | 933 | mode: { type: "string", default: "production" }, |
| 923 | 934 | }, |
| 924 | 935 | allowPositionals: true, |
| @@ -932,7 +943,11 @@ if (mode !== "production" && mode !== "development") { · 7 unmodified lines | ||
| 932 | 943 | } |
| 933 | 944 | |
| 934 | 945 | if (command === "binary") { |
| 935 | − const targets = values.single ? ALL_TARGETS.filter(isCurrentPlatform) : ALL_TARGETS; | |
| 946 | + const targets = values.target | |
| 947 | + ? resolveTargetsFromEnv(values.target) | |
| 948 | + : values.single | |
| 949 | + ? ALL_TARGETS.filter(isCurrentPlatform) | |
| 950 | + : ALL_TARGETS; | |
| 936 | 951 | const binaries = await buildBinaries(targets, mode); |
| 937 | 952 | await buildWrapperPackage(binaries); |
| 938 | 953 | } else if (command === "preview") { |
| @@ -50,7 +50,7 @@ describe("service unit generation", () => { · 49 unmodified lines | ||
| 50 | 50 | |
| 51 | 51 | it("never leaks the auth password into the unit", () => { |
| 52 | 52 | const plist = generateLaunchdPlist(launchdInput); |
| 53 | − // The secret lives in the 0600 service.key, never in the plist env. | |
| 53 | + // No secret in the unit — the daemon reads the bearer from auth.json at boot. | |
| 54 | 54 | expect(plist).not.toContain("EXECUTOR_AUTH_PASSWORD"); |
| 55 | 55 | }); |
| 56 | 56 | |
| @@ -95,7 +95,7 @@ describe("service unit generation", () => { · 38 unmodified lines | ||
| 95 | 95 | '"C:\\Program Files\\Executor\\executor.exe" daemon run --foreground --port 4789', |
| 96 | 96 | ); |
| 97 | 97 | expect(wrapper).toContain('1>> "C:\\Users\\x\\.executor\\logs\\daemon.log"'); |
| 98 | − // The secret is never baked into the wrapper — the daemon reads service.key. | |
| 98 | + // No secret in the wrapper — the daemon reads the bearer from auth.json at boot. | |
| 99 | 99 | expect(wrapper).not.toContain("EXECUTOR_AUTH_PASSWORD"); |
| 100 | 100 | }); |
| 101 | 101 | |
| @@ -0,0 +1,128 @@ | ||
| 1 | +// The supervised daemon's durability as a WATCHABLE terminal recording: register | |
| 2 | +// a REAL integration, REBOOT the daemon's machine for real (the on-screen | |
| 3 | +// spinner runs for the actual reboot), then show the integration still there. | |
| 4 | +// Same assertions as restart-persistence — but filmed, so you can press play | |
| 5 | +// instead of trusting a green check. Runs against the cli-* VM targets. | |
| 6 | +import { randomBytes } from "node:crypto"; | |
| 7 | +import { join } from "node:path"; | |
| 8 | + | |
| 9 | +import { expect } from "@effect/vitest"; | |
| 10 | +import { Effect } from "effect"; | |
| 11 | + | |
| 12 | +import { composePluginApi } from "@executor-js/api/server"; | |
| 13 | +import { openApiHttpPlugin } from "@executor-js/plugin-openapi/api"; | |
| 14 | + | |
| 15 | +import { withChatTheater } from "../src/clients/chat-theater"; | |
| 16 | +import { scenario } from "../src/scenario"; | |
| 17 | +import { Api, Cli, Restart, RunDir, Target } from "../src/services"; | |
| 18 | + | |
| 19 | +const api = composePluginApi([openApiHttpPlugin()] as const); | |
| 20 | + | |
| 21 | +/** Inline OpenAPI 3 spec with a single GET /ping (its server is never called). */ | |
| 22 | +const pingSpec = JSON.stringify({ | |
| 23 | + openapi: "3.0.3", | |
| 24 | + info: { title: "Reboot Lifecycle API", version: "1.0.0" }, | |
| 25 | + servers: [{ url: "http://127.0.0.1:59998" }], | |
| 26 | + paths: { | |
| 27 | + "/ping": { | |
| 28 | + get: { | |
| 29 | + operationId: "getPing", | |
| 30 | + summary: "Liveness ping", | |
| 31 | + responses: { "200": { description: "pong" } }, | |
| 32 | + }, | |
| 33 | + }, | |
| 34 | + }, | |
| 35 | +}); | |
| 36 | + | |
| 37 | +scenario( | |
| 38 | + "Supervised daemon · an integration survives a real machine reboot (recorded)", | |
| 39 | + {}, | |
| 40 | + Effect.gen(function* () { | |
| 41 | + const target = yield* Target; | |
| 42 | + const restart = yield* Restart; | |
| 43 | + const { client } = yield* Api; | |
| 44 | + const cli = yield* Cli; | |
| 45 | + const runDir = yield* RunDir; | |
| 46 | + | |
| 47 | + const slug = `reboot-film-${randomBytes(4).toString("hex")}`; | |
| 48 | + | |
| 49 | + yield* withChatTheater( | |
| 50 | + cli, | |
| 51 | + { title: "executor — supervised daemon", record: join(runDir, "terminal.cast") }, | |
| 52 | + (chat) => | |
| 53 | + Effect.gen(function* () { | |
| 54 | + yield* chat.user("If this machine reboots, do my connected integrations survive?"); | |
| 55 | + yield* chat.assistant( | |
| 56 | + "Let's prove it — register a real integration, reboot the daemon's machine for real, then check it's still there.", | |
| 57 | + ); | |
| 58 | + | |
| 59 | + const before = yield* client(api, yield* target.newIdentity()); | |
| 60 | + | |
| 61 | + const added = yield* chat.tool( | |
| 62 | + { | |
| 63 | + name: "executor call executor.openapi.addSpec", | |
| 64 | + input: `slug: ${slug}\nspec: inline OpenAPI (GET /ping)`, | |
| 65 | + result: (a) => `registered — ${a.toolCount} tool(s)`, | |
| 66 | + }, | |
| 67 | + before.openapi.addSpec({ | |
| 68 | + payload: { | |
| 69 | + spec: { kind: "blob", value: pingSpec }, | |
| 70 | + slug, | |
| 71 | + authenticationTemplate: [], | |
| 72 | + }, | |
| 73 | + }), | |
| 74 | + ); | |
| 75 | + expect(added.toolCount, "the spec registered with tools").toBeGreaterThan(0); | |
| 76 | + | |
| 77 | + const listed = yield* chat.tool( | |
| 78 | + { | |
| 79 | + name: "executor tools sources", | |
| 80 | + result: (rows) => | |
| 81 | + rows.map((r) => String(r.slug)).includes(slug) ? `${slug} is listed` : "NOT listed", | |
| 82 | + }, | |
| 83 | + before.integrations.list(), | |
| 84 | + ); | |
| 85 | + expect( | |
| 86 | + listed.map((i) => String(i.slug)), | |
| 87 | + "listed before the reboot", | |
| 88 | + ).toContain(slug); | |
| 89 | + | |
| 90 | + // The spinner here runs for the ENTIRE real reboot — the supervised | |
| 91 | + // service must auto-start at boot for this to ever return. | |
| 92 | + yield* chat.tool( | |
| 93 | + { | |
| 94 | + name: "reboot the daemon's machine", | |
| 95 | + input: "guest OS reboot — the OS service manager must auto-start the daemon at boot", | |
| 96 | + result: () => "back online; daemon auto-started", | |
| 97 | + }, | |
| 98 | + restart(), | |
| 99 | + ); | |
| 100 | + | |
| 101 | + const after = yield* client(api, yield* target.newIdentity()); | |
| 102 | + yield* Effect.ensuring( | |
| 103 | + Effect.gen(function* () { | |
| 104 | + const survived = yield* chat.tool( | |
| 105 | + { | |
| 106 | + name: "executor tools sources", | |
| 107 | + result: (rows) => | |
| 108 | + rows.map((r) => String(r.slug)).includes(slug) | |
| 109 | + ? `${slug} SURVIVED the reboot` | |
| 110 | + : "VANISHED", | |
| 111 | + }, | |
| 112 | + after.integrations.list(), | |
| 113 | + ); | |
| 114 | + expect( | |
| 115 | + survived.map((i) => String(i.slug)), | |
| 116 | + "survived the reboot", | |
| 117 | + ).toContain(slug); | |
| 118 | + yield* chat.assistant( | |
| 119 | + "It survived — the OS restarted the daemon at boot and its data was intact.", | |
| 120 | + ); | |
| 121 | + }), | |
| 122 | + // Shared guest, but ephemeral; still, never leave the spec behind. | |
| 123 | + after.openapi.removeSpec({ params: { slug } }).pipe(Effect.ignore), | |
| 124 | + ); | |
| 125 | + }), | |
| 126 | + ); | |
| 127 | + }), | |
| 128 | +); | |
| @@ -0,0 +1,182 @@ | ||
| 1 | +// Desktop-only, on camera: the app ATTACHES to an already-running OS-supervised | |
| 2 | +// daemon instead of spawning its own sidecar. We start a real supervised gateway | |
| 3 | +// (the desktop sidecar server in EXECUTOR_SUPERVISED mode → it self-publishes a | |
| 4 | +// manifest of kind "cli-daemon"), launch the Electron app pointed at the same | |
| 5 | +// HOME, and prove it attached: the manifest still names OUR daemon's pid (a | |
| 6 | +// spawned sidecar would be a fresh pid + kind "desktop-sidecar"). The recording | |
| 7 | +// (session.mp4 + screenshots) is the artifact; the waits are the assertions. No | |
| 8 | +// launchd — only a throwaway home and one short-lived daemon process. | |
| 9 | +import { type ChildProcess, execFile, spawn } from "node:child_process"; | |
| 10 | +import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs"; | |
| 11 | +import { createRequire } from "node:module"; | |
| 12 | +import net from "node:net"; | |
| 13 | +import { tmpdir } from "node:os"; | |
| 14 | +import { join } from "node:path"; | |
| 15 | +import { fileURLToPath } from "node:url"; | |
| 16 | +import { promisify } from "node:util"; | |
| 17 | + | |
| 18 | +import { expect } from "@effect/vitest"; | |
| 19 | +import { Effect } from "effect"; | |
| 20 | +import { _electron } from "playwright"; | |
| 21 | + | |
| 22 | +import { scenario } from "../src/scenario"; | |
| 23 | +import { RunDir } from "../src/services"; | |
| 24 | +import { waitForHttp } from "../setup/boot"; | |
| 25 | + | |
| 26 | +const appDir = fileURLToPath(new URL("../../apps/desktop/", import.meta.url)); | |
| 27 | +const repoRoot = fileURLToPath(new URL("../../", import.meta.url)); | |
| 28 | +const sidecarServer = join(appDir, "src/sidecar/server.ts"); | |
| 29 | +const clientDir = join(repoRoot, "apps/local/dist"); | |
| 30 | +const electronBinary = createRequire(join(appDir, "package.json"))("electron") as string; | |
| 31 | + | |
| 32 | +const freePort = (): Promise<number> => | |
| 33 | + new Promise((resolve, reject) => { | |
| 34 | + const srv = net.createServer(); | |
| 35 | + srv.on("error", reject); | |
| 36 | + srv.listen(0, "127.0.0.1", () => { | |
| 37 | + const port = (srv.address() as net.AddressInfo).port; | |
| 38 | + srv.close(() => resolve(port)); | |
| 39 | + }); | |
| 40 | + }); | |
| 41 | + | |
| 42 | +interface Manifest { | |
| 43 | + readonly kind: string; | |
| 44 | + readonly pid: number; | |
| 45 | +} | |
| 46 | + | |
| 47 | +interface DaemonStart { | |
| 48 | + readonly child: ChildProcess; | |
| 49 | + readonly ready: boolean; | |
| 50 | + readonly stderr: string; | |
| 51 | +} | |
| 52 | + | |
| 53 | +/** Spawn the supervised gateway; resolves once it announces EXECUTOR_READY (or | |
| 54 | + * times out / exits early, with `ready: false`). The caller asserts readiness, | |
| 55 | + * so the executor only ever resolves. */ | |
| 56 | +const startSupervisedDaemon = (env: NodeJS.ProcessEnv): Promise<DaemonStart> => | |
| 57 | + new Promise((resolve) => { | |
| 58 | + const child = spawn("bun", ["run", sidecarServer], { | |
| 59 | + cwd: repoRoot, | |
| 60 | + env, | |
| 61 | + stdio: ["ignore", "pipe", "pipe"], | |
| 62 | + }); | |
| 63 | + let stderr = ""; | |
| 64 | + const settle = (ready: boolean) => resolve({ child, ready, stderr }); | |
| 65 | + const timer = setTimeout(() => settle(false), 60_000); | |
| 66 | + child.stdout.on("data", (chunk: Buffer) => { | |
| 67 | + if (chunk.toString().includes("EXECUTOR_READY:")) { | |
| 68 | + clearTimeout(timer); | |
| 69 | + settle(true); | |
| 70 | + } | |
| 71 | + }); | |
| 72 | + child.stderr.on("data", (chunk: Buffer) => { | |
| 73 | + stderr += chunk.toString(); | |
| 74 | + }); | |
| 75 | + child.on("exit", () => { | |
| 76 | + clearTimeout(timer); | |
| 77 | + settle(false); | |
| 78 | + }); | |
| 79 | + }); | |
| 80 | + | |
| 81 | +scenario( | |
| 82 | + "Desktop · attaches to the OS-supervised daemon instead of spawning a sidecar", | |
| 83 | + { timeout: 240_000 }, | |
| 84 | + Effect.gen(function* () { | |
| 85 | + const runDir = yield* RunDir; | |
| 86 | + yield* Effect.promise(() => run(runDir)); | |
| 87 | + }), | |
| 88 | +); | |
| 89 | + | |
| 90 | +const run = async (runDir: string) => { | |
| 91 | + const home = mkdtempSync(join(tmpdir(), "executor-attach-e2e-")); | |
| 92 | + const dataDir = join(home, ".executor"); | |
| 93 | + const manifestPath = join(dataDir, "server-control", "server.json"); | |
| 94 | + const videoTmp = join(runDir, ".video-tmp"); | |
| 95 | + const port = await freePort(); | |
| 96 | + | |
| 97 | + let daemon: ChildProcess | undefined; | |
| 98 | + let app: Awaited<ReturnType<typeof _electron.launch>> | undefined; | |
| 99 | + let stepIndex = 0; | |
| 100 | + | |
| 101 | + try { | |
| 102 | + const started = await startSupervisedDaemon({ | |
| 103 | + ...process.env, | |
| 104 | + HOME: home, | |
| 105 | + EXECUTOR_SUPERVISED: "1", | |
| 106 | + EXECUTOR_DATA_DIR: dataDir, | |
| 107 | + EXECUTOR_PORT: String(port), | |
| 108 | + EXECUTOR_HOST: "127.0.0.1", | |
| 109 | + EXECUTOR_AUTH_TOKEN: "supervised-attach-film", | |
| 110 | + EXECUTOR_CLIENT_DIR: clientDir, | |
| 111 | + }); | |
| 112 | + daemon = started.child; | |
| 113 | + expect(started.ready, `supervised daemon became ready; stderr:\n${started.stderr}`).toBe(true); | |
| 114 | + await waitForHttp(`http://127.0.0.1:${port}/`, { timeoutMs: 30_000 }); | |
| 115 | + | |
| 116 | + const daemonManifest = JSON.parse(readFileSync(manifestPath, "utf8")) as Manifest; | |
| 117 | + expect(daemonManifest.kind, "the running daemon advertises itself as cli-daemon").toBe( | |
| 118 | + "cli-daemon", | |
| 119 | + ); | |
| 120 | + const daemonPid = daemonManifest.pid; | |
| 121 | + | |
| 122 | + app = await _electron.launch({ | |
| 123 | + executablePath: electronBinary, | |
| 124 | + args: [appDir], | |
| 125 | + cwd: appDir, | |
| 126 | + env: { ...process.env, HOME: home }, | |
| 127 | + recordVideo: { dir: videoTmp, size: { width: 1280, height: 800 } }, | |
| 128 | + timeout: 120_000, | |
| 129 | + }); | |
| 130 | + | |
| 131 | + const page = await app.firstWindow({ timeout: 120_000 }); | |
| 132 | + const step = async (label: string, body: () => Promise<void>) => { | |
| 133 | + await body(); | |
| 134 | + stepIndex += 1; | |
| 135 | + const slug = label.toLowerCase().replace(/[^a-z0-9]+/g, "-"); | |
| 136 | + await page.screenshot({ | |
| 137 | + path: join(runDir, `${String(stepIndex).padStart(2, "0")}-${slug}.png`), | |
| 138 | + }); | |
| 139 | + }; | |
| 140 | + | |
| 141 | + // The window only loads the console once the app has a connection — and it | |
| 142 | + // attaches to the supervised daemon before it would ever spawn a sidecar. | |
| 143 | + await step("desktop boots into the console", async () => { | |
| 144 | + await page.getByText("Settings").first().waitFor({ timeout: 120_000 }); | |
| 145 | + }); | |
| 146 | + | |
| 147 | + // The proof it ATTACHED rather than spawned: the manifest is untouched — | |
| 148 | + // same pid, still cli-daemon. A managed sidecar would have rewritten it to | |
| 149 | + // kind "desktop-sidecar" with a fresh child pid. | |
| 150 | + await step("server manifest still names the supervised daemon", async () => { | |
| 151 | + const after = JSON.parse(readFileSync(manifestPath, "utf8")) as Manifest; | |
| 152 | + expect(after.kind, "still the supervised daemon (not a desktop sidecar)").toBe("cli-daemon"); | |
| 153 | + expect(after.pid, "the desktop attached to our daemon, not a new sidecar").toBe(daemonPid); | |
| 154 | + }); | |
| 155 | + } finally { | |
| 156 | + const page = app?.windows()[0]; | |
| 157 | + const video = page?.video(); | |
| 158 | + await app?.close().catch(() => {}); | |
| 159 | + const recordedPath = await video?.path().catch(() => undefined); | |
| 160 | + if (recordedPath && existsSync(recordedPath)) { | |
| 161 | + await promisify(execFile)("ffmpeg", [ | |
| 162 | + "-y", | |
| 163 | + "-i", | |
| 164 | + recordedPath, | |
| 165 | + "-c:v", | |
| 166 | + "libx264", | |
| 167 | + "-preset", | |
| 168 | + "veryfast", | |
| 169 | + "-crf", | |
| 170 | + "26", | |
| 171 | + "-pix_fmt", | |
| 172 | + "yuv420p", | |
| 173 | + "-movflags", | |
| 174 | + "+faststart", | |
| 175 | + join(runDir, "session.mp4"), | |
| 176 | + ]).catch(() => {}); | |
| 177 | + } | |
| 178 | + daemon?.kill("SIGTERM"); | |
| 179 | + rmSync(videoTmp, { recursive: true, force: true }); | |
| 180 | + rmSync(home, { recursive: true, force: true }); | |
| 181 | + } | |
| 182 | +}; | |
| @@ -100,3 +100,27 @@ export const waitForHttp = async ( · 99 unmodified lines | ||
| 100 | 100 | } |
| 101 | 101 | throw new Error(`timed out waiting for ${url}: ${String(lastError)}`); |
| 102 | 102 | }; |
| 103 | + | |
| 104 | +/** | |
| 105 | + * Reboot DOWN-GATE: wait until `url` stops answering (fetch rejects — connection | |
| 106 | + * refused/reset). An orderly OS shutdown keeps the daemon serving for several | |
| 107 | + * seconds, and a reconnecting tunnel re-establishes the forward, so polling for | |
| 108 | + * "up" right after a reboot command can false-pass a reboot that never happened. | |
| 109 | + * Gating on the server actually going DOWN first makes restart-persistence prove | |
| 110 | + * a real reboot. Throws if it never goes down within the deadline. | |
| 111 | + */ | |
| 112 | +export const waitForHttpDown = async ( | |
| 113 | + url: string, | |
| 114 | + options: { readonly timeoutMs?: number } = {}, | |
| 115 | +): Promise<void> => { | |
| 116 | + const deadline = Date.now() + (options.timeoutMs ?? 120_000); | |
| 117 | + while (Date.now() < deadline) { | |
| 118 | + try { | |
| 119 | + await fetch(url, { redirect: "manual", signal: AbortSignal.timeout(2000) }); | |
| 120 | + } catch { | |
| 121 | + return; // the server (or its tunnel) is gone — the reboot took | |
| 122 | + } | |
| 123 | + await new Promise((resolve) => setTimeout(resolve, 500)); | |
| 124 | + } | |
| 125 | + throw new Error(`${url} never became unreachable — the reboot may not have taken`); | |
| 126 | +}; | |
| @@ -0,0 +1,3 @@ | ||
| 1 | +import { setupCliTarget } from "./cli.globalsetup"; | |
| 2 | + | |
| 3 | +export default (): Promise<(() => Promise<void>) | void> => setupCliTarget("linux"); | |
| @@ -0,0 +1,3 @@ | ||
| 1 | +import { setupCliTarget } from "./cli.globalsetup"; | |
| 2 | + | |
| 3 | +export default (): Promise<(() => Promise<void>) | void> => setupCliTarget("macos"); | |
| @@ -0,0 +1,3 @@ | ||
| 1 | +import { setupCliTarget } from "./cli.globalsetup"; | |
| 2 | + | |
| 3 | +export default (): Promise<(() => Promise<void>) | void> => setupCliTarget("windows"); | |
| @@ -0,0 +1,97 @@ | ||
| 1 | +// Boot the `cli` target: build the guest's `executor` binary, provision a VM, | |
| 2 | +// install it as a supervised OS service, and forward its loopback HTTP port to | |
| 3 | +// the host over a reconnecting SSH tunnel. Publishes connection + reboot info | |
| 4 | +// via env (inherited by the test workers spawned afterward). Per-OS entrypoints | |
| 5 | +// (cli-macos.globalsetup.ts, …) call setupCliTarget with their OS. | |
| 6 | +// | |
| 7 | +// macOS + Linux run on local tart guests (Apple-Silicon host); Windows runs on | |
| 8 | +// an ephemeral EC2 instance. The supervised daemon is bearer-gated: it mints/ | |
| 9 | +// loads its token into auth.json, which we read from the guest and publish so | |
| 10 | +// the api surface authenticates with `Authorization: Bearer`. | |
| 11 | +import { buildGuestBinary } from "../src/vm/build-binary"; | |
| 12 | +import { ec2Vm } from "../src/vm/ec2"; | |
| 13 | +import { tartVm } from "../src/vm/tart"; | |
| 14 | +import type { VmArch, VmHandle, VmOs } from "../src/vm/types"; | |
| 15 | +import { waitForHttp } from "./boot"; | |
| 16 | + | |
| 17 | +const PORT = 4789; | |
| 18 | + | |
| 19 | +// Per-OS guest specifics: working dir, binary name, and the shell idioms for | |
| 20 | +// prep/cleanup — Unix `sh` on tart (macOS/Linux), PowerShell on EC2 Windows. | |
| 21 | +const guestPlan = (os: VmOs) => { | |
| 22 | + if (os === "windows") { | |
| 23 | + const dir = "C:/ed"; | |
| 24 | + const exe = `${dir}/executor.exe`; | |
| 25 | + return { | |
| 26 | + dir, | |
| 27 | + prep: `Remove-Item -Recurse -Force '${dir}' -ErrorAction SilentlyContinue; New-Item -ItemType Directory -Force -Path '${dir}' | Out-Null`, | |
| 28 | + postPush: "Write-Output ok", // no chmod/quarantine on Windows | |
| 29 | + install: `& '${exe}' service install --port ${PORT}`, | |
| 30 | + readToken: 'Get-Content "$env:USERPROFILE\\.executor\\server-control\\auth.json" -Raw', | |
| 31 | + uninstall: `& '${exe}' service uninstall`, | |
| 32 | + }; | |
| 33 | + } | |
| 34 | + const dir = "~/ed"; | |
| 35 | + const exe = `${dir}/executor`; | |
| 36 | + return { | |
| 37 | + dir, | |
| 38 | + prep: `rm -rf ${dir} && mkdir -p ${dir}`, | |
| 39 | + // macOS quarantines scp'd executables; clear it so the binary can run. | |
| 40 | + postPush: | |
| 41 | + os === "macos" | |
| 42 | + ? `chmod +x ${exe}; xattr -dr com.apple.quarantine ${dir} 2>/dev/null || true` | |
| 43 | + : `chmod +x ${exe}`, | |
| 44 | + install: `${exe} service install --port ${PORT}`, | |
| 45 | + readToken: "cat ~/.executor/server-control/auth.json", | |
| 46 | + uninstall: `${exe} service uninstall`, | |
| 47 | + }; | |
| 48 | +}; | |
| 49 | + | |
| 50 | +export async function setupCliTarget(os: VmOs): Promise<(() => Promise<void>) | void> { | |
| 51 | + process.env.E2E_VM_OS = os; // so the worker-side target resolves the same OS | |
| 52 | + | |
| 53 | + const arch: VmArch = os === "windows" ? "x64" : "arm64"; | |
| 54 | + const binDir = await buildGuestBinary(os, arch); | |
| 55 | + const vm: VmHandle = | |
| 56 | + os === "windows" ? await ec2Vm(os, arch).provision() : await tartVm(os, arch).provision(); | |
| 57 | + const plan = guestPlan(os); | |
| 58 | + | |
| 59 | + let tunnelClose: (() => void) | undefined; | |
| 60 | + try { | |
| 61 | + await vm.ssh(plan.prep); | |
| 62 | + await vm.push(`${binDir}/.`, os === "windows" ? plan.dir : `${plan.dir}/`); | |
| 63 | + await vm.ssh(plan.postPush); | |
| 64 | + | |
| 65 | + const install = await vm.ssh(plan.install); | |
| 66 | + if (install.code !== 0) { | |
| 67 | + throw new Error(`service install failed: ${install.stderr.trim() || install.stdout.trim()}`); | |
| 68 | + } | |
| 69 | + | |
| 70 | + // The supervised daemon mints/loads its bearer into auth.json on first boot. | |
| 71 | + const tokenRaw = (await vm.ssh(plan.readToken)).stdout.trim(); | |
| 72 | + const token = (JSON.parse(tokenRaw) as { token: string }).token; | |
| 73 | + | |
| 74 | + const tunnel = await vm.tunnel(PORT); | |
| 75 | + tunnelClose = tunnel.close; | |
| 76 | + const baseUrl = `http://127.0.0.1:${tunnel.localPort}`; | |
| 77 | + await waitForHttp(`${baseUrl}/`, { timeoutMs: 60_000 }); | |
| 78 | + | |
| 79 | + process.env.E2E_CLI_BASE_URL = baseUrl; | |
| 80 | + process.env.E2E_CLI_AUTH_TOKEN = token; | |
| 81 | + process.env.E2E_CLI_VM_HOST = vm.host; | |
| 82 | + if (vm.sshKeyPath) process.env.E2E_CLI_SSH_KEY = vm.sshKeyPath; | |
| 83 | + process.env.E2E_CLI_TUNNEL_PORT = String(tunnel.localPort); | |
| 84 | + process.env.E2E_CLI_BIN_DIR = plan.dir; | |
| 85 | + } catch (error) { | |
| 86 | + tunnelClose?.(); | |
| 87 | + await vm.ssh(plan.uninstall).catch(() => undefined); | |
| 88 | + await vm.discard(); | |
| 89 | + throw error; | |
| 90 | + } | |
| 91 | + | |
| 92 | + return async () => { | |
| 93 | + tunnelClose?.(); | |
| 94 | + await vm.ssh(plan.uninstall).catch(() => undefined); | |
| 95 | + await vm.discard(); | |
| 96 | + }; | |
| 97 | +} | |
| @@ -0,0 +1,35 @@ | ||
| 1 | +// Build the compiled `executor` for a guest os/arch. `service install` refuses | |
| 2 | +// to run from a dev (.ts) entrypoint, so the VM targets need a real binary — | |
| 3 | +// produced via the `--target` flag on apps/cli/src/build.ts. | |
| 4 | + | |
| 5 | +import { execFile } from "node:child_process"; | |
| 6 | +import { existsSync } from "node:fs"; | |
| 7 | +import path from "node:path"; | |
| 8 | +import { promisify } from "node:util"; | |
| 9 | + | |
| 10 | +import type { VmArch, VmOs } from "./types"; | |
| 11 | + | |
| 12 | +const execFileP = promisify(execFile); | |
| 13 | + | |
| 14 | +const PLATFORM_TAG: Record<VmOs, string> = { macos: "darwin", linux: "linux", windows: "windows" }; | |
| 15 | + | |
| 16 | +// e2e/src/vm/build-binary.ts → repo root. | |
| 17 | +const REPO_ROOT = path.resolve(import.meta.dirname, "../../.."); | |
| 18 | +const CLI_DIR = path.join(REPO_ROOT, "apps", "cli"); | |
| 19 | + | |
| 20 | +/** | |
| 21 | + * Build the `executor` binary for a guest and return its `bin` directory | |
| 22 | + * (executor[.exe] + the native libsql/keyring modules). The build cleans | |
| 23 | + * `dist/` each run, so callers should push the result before building another. | |
| 24 | + */ | |
| 25 | +export const buildGuestBinary = async (os: VmOs, arch: VmArch): Promise<string> => { | |
| 26 | + const target = `executor-${PLATFORM_TAG[os]}-${arch}`; | |
| 27 | + await execFileP("bun", ["run", "src/build.ts", "binary", "--target", target], { | |
| 28 | + cwd: CLI_DIR, | |
| 29 | + maxBuffer: 256 * 1024 * 1024, | |
| 30 | + }); | |
| 31 | + const binDir = path.join(CLI_DIR, "dist", target, "bin"); | |
| 32 | + const exe = path.join(binDir, os === "windows" ? "executor.exe" : "executor"); | |
| 33 | + if (!existsSync(exe)) throw new Error(`buildGuestBinary: ${exe} not produced`); | |
| 34 | + return binDir; | |
| 35 | +}; | |
| @@ -0,0 +1,422 @@ | ||
| 1 | +// ec2 provider: ephemeral guests on AWS EC2 for the cross-OS supervised-daemon | |
| 2 | +// e2e where tart can't run (Windows; optionally Linux). Mirrors tart.ts — launch | |
| 3 | +// a fresh instance, drive over SSH (key-based; PowerShell on Windows), REBOOT for | |
| 4 | +// real, tear down. | |
| 5 | +// | |
| 6 | +// Credentials are NEVER embedded here: the `aws` CLI uses the ambient sign-in | |
| 7 | +// (`aws configure` / env). Every instance is tagged `executor-e2e` and always | |
| 8 | +// terminated on discard; the security group is scoped to this host's egress IP. | |
| 9 | +// | |
| 10 | +// Reboot is gated on a real boot-time change (Windows `LastBootUpTime`), not mere | |
| 11 | +// SSH reachability — an orderly shutdown keeps the daemon serving for several | |
| 12 | +// seconds, so "SSH answered" alone can false-pass a reboot that never happened. | |
| 13 | + | |
| 14 | +import { execFile } from "node:child_process"; | |
| 15 | +import { chmodSync, mkdtempSync, writeFileSync } from "node:fs"; | |
| 16 | +import net from "node:net"; | |
| 17 | +import { tmpdir } from "node:os"; | |
| 18 | +import { join } from "node:path"; | |
| 19 | +import { promisify } from "node:util"; | |
| 20 | + | |
| 21 | +import { | |
| 22 | + type SshResult, | |
| 23 | + sleep, | |
| 24 | + type VmArch, | |
| 25 | + type VmHandle, | |
| 26 | + type VmOs, | |
| 27 | + type VmProvider, | |
| 28 | +} from "./types"; | |
| 29 | + | |
| 30 | +const execFileP = promisify(execFile); | |
| 31 | + | |
| 32 | +const REGION = process.env.E2E_EC2_REGION ?? "us-west-2"; | |
| 33 | +const INSTANCE_TYPE = process.env.E2E_EC2_INSTANCE_TYPE ?? "t3.medium"; | |
| 34 | +const TAG = "executor-e2e"; | |
| 35 | + | |
| 36 | +const SSH_OPTS = [ | |
| 37 | + "-o", | |
| 38 | + "StrictHostKeyChecking=no", | |
| 39 | + "-o", | |
| 40 | + "UserKnownHostsFile=/dev/null", | |
| 41 | + "-o", | |
| 42 | + "ConnectTimeout=10", | |
| 43 | + "-o", | |
| 44 | + "ServerAliveInterval=10", | |
| 45 | + "-o", | |
| 46 | + "LogLevel=ERROR", | |
| 47 | +]; | |
| 48 | + | |
| 49 | +const guestUser = (os: VmOs): string => | |
| 50 | + os === "windows" ? "Administrator" : (process.env.E2E_EC2_LINUX_USER ?? "ubuntu"); | |
| 51 | + | |
| 52 | +/** | |
| 53 | + * Reboot an EC2 guest by address, statelessly (no live handle) — the mirror of | |
| 54 | + * tart's sshRebootGuest, for the worker-side `restart()`. The connection drops | |
| 55 | + * mid-call, so errors are swallowed; the caller's down-gate + up-poll confirm | |
| 56 | + * the real reboot. | |
| 57 | + */ | |
| 58 | +export const ec2RebootGuest = async ( | |
| 59 | + host: string, | |
| 60 | + keyPath: string, | |
| 61 | + os: VmOs = "windows", | |
| 62 | +): Promise<void> => { | |
| 63 | + const cmd = os === "windows" ? "Restart-Computer -Force" : "sudo reboot"; | |
| 64 | + await execFileP("ssh", ["-i", keyPath, ...SSH_OPTS, `${guestUser(os)}@${host}`, cmd]).catch( | |
| 65 | + () => undefined, | |
| 66 | + ); | |
| 67 | +}; | |
| 68 | + | |
| 69 | +const aws = async (args: ReadonlyArray<string>): Promise<string> => { | |
| 70 | + const { stdout } = await execFileP("aws", ["--region", REGION, "--output", "text", ...args], { | |
| 71 | + maxBuffer: 64 * 1024 * 1024, | |
| 72 | + }); | |
| 73 | + return stdout.trim(); | |
| 74 | +}; | |
| 75 | + | |
| 76 | +/** This host's public egress IP, for the inbound-SSH security-group rule. */ | |
| 77 | +const egressIp = async (): Promise<string> => { | |
| 78 | + const { stdout } = await execFileP("curl", [ | |
| 79 | + "-s", | |
| 80 | + "--max-time", | |
| 81 | + "10", | |
| 82 | + "https://checkip.amazonaws.com", | |
| 83 | + ]); | |
| 84 | + return stdout.trim(); | |
| 85 | +}; | |
| 86 | + | |
| 87 | +/** Latest AWS-published base AMI for the guest OS (resolve dynamically — ids rotate). */ | |
| 88 | +const latestAmi = async (os: VmOs): Promise<string> => { | |
| 89 | + if (os === "windows") { | |
| 90 | + const viaSsm = await aws([ | |
| 91 | + "ssm", | |
| 92 | + "get-parameters", | |
| 93 | + "--names", | |
| 94 | + "/aws/service/ami-windows-latest/Windows_Server-2022-English-Full-Base", | |
| 95 | + "--query", | |
| 96 | + "Parameters[0].Value", | |
| 97 | + ]).catch(() => ""); | |
| 98 | + if (viaSsm && viaSsm !== "None") return viaSsm; | |
| 99 | + return aws([ | |
| 100 | + "ec2", | |
| 101 | + "describe-images", | |
| 102 | + "--owners", | |
| 103 | + "amazon", | |
| 104 | + "--filters", | |
| 105 | + "Name=name,Values=Windows_Server-2022-English-Full-Base-*", | |
| 106 | + "Name=state,Values=available", | |
| 107 | + "--query", | |
| 108 | + "reverse(sort_by(Images,&CreationDate))[0].ImageId", | |
| 109 | + ]); | |
| 110 | + } | |
| 111 | + return aws([ | |
| 112 | + "ssm", | |
| 113 | + "get-parameters", | |
| 114 | + "--names", | |
| 115 | + "/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id", | |
| 116 | + "--query", | |
| 117 | + "Parameters[0].Value", | |
| 118 | + ]); | |
| 119 | +}; | |
| 120 | + | |
| 121 | +const defaultSubnet = async (): Promise<string> => { | |
| 122 | + const vpc = await aws([ | |
| 123 | + "ec2", | |
| 124 | + "describe-vpcs", | |
| 125 | + "--filters", | |
| 126 | + "Name=isDefault,Values=true", | |
| 127 | + "--query", | |
| 128 | + "Vpcs[0].VpcId", | |
| 129 | + ]); | |
| 130 | + const subnet = await aws([ | |
| 131 | + "ec2", | |
| 132 | + "describe-subnets", | |
| 133 | + "--filters", | |
| 134 | + `Name=vpc-id,Values=${vpc}`, | |
| 135 | + "Name=default-for-az,Values=true", | |
| 136 | + "--query", | |
| 137 | + "Subnets[0].SubnetId", | |
| 138 | + ]); | |
| 139 | + return subnet && subnet !== "None" | |
| 140 | + ? subnet | |
| 141 | + : aws([ | |
| 142 | + "ec2", | |
| 143 | + "describe-subnets", | |
| 144 | + "--filters", | |
| 145 | + `Name=vpc-id,Values=${vpc}`, | |
| 146 | + "--query", | |
| 147 | + "Subnets[0].SubnetId", | |
| 148 | + ]); | |
| 149 | +}; | |
| 150 | + | |
| 151 | +/** Create (idempotently) a security group allowing inbound SSH from this host. */ | |
| 152 | +const ensureSecurityGroup = async (myIp: string): Promise<string> => { | |
| 153 | + const name = `${TAG}-sg`; | |
| 154 | + let sg = await aws([ | |
| 155 | + "ec2", | |
| 156 | + "describe-security-groups", | |
| 157 | + "--filters", | |
| 158 | + `Name=group-name,Values=${name}`, | |
| 159 | + "--query", | |
| 160 | + "SecurityGroups[0].GroupId", | |
| 161 | + ]).catch(() => ""); | |
| 162 | + if (!sg || sg === "None") { | |
| 163 | + sg = await aws([ | |
| 164 | + "ec2", | |
| 165 | + "create-security-group", | |
| 166 | + "--group-name", | |
| 167 | + name, | |
| 168 | + "--description", | |
| 169 | + "executor e2e ephemeral guests (SSH from CI host)", | |
| 170 | + "--query", | |
| 171 | + "GroupId", | |
| 172 | + ]); | |
| 173 | + } | |
| 174 | + // Authorize this host's IP for SSH; ignore "already exists". | |
| 175 | + await aws([ | |
| 176 | + "ec2", | |
| 177 | + "authorize-security-group-ingress", | |
| 178 | + "--group-id", | |
| 179 | + sg, | |
| 180 | + "--protocol", | |
| 181 | + "tcp", | |
| 182 | + "--port", | |
| 183 | + "22", | |
| 184 | + "--cidr", | |
| 185 | + `${myIp}/32`, | |
| 186 | + ]).catch(() => undefined); | |
| 187 | + return sg; | |
| 188 | +}; | |
| 189 | + | |
| 190 | +/** PowerShell user-data: enable OpenSSH, default the shell to PowerShell, and | |
| 191 | + * authorize our public key for the Administrator account. */ | |
| 192 | +const windowsUserData = (publicKey: string): string => | |
| 193 | + [ | |
| 194 | + "<powershell>", | |
| 195 | + "Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0", | |
| 196 | + "Set-Service -Name sshd -StartupType Automatic", | |
| 197 | + "Start-Service sshd", | |
| 198 | + "New-ItemProperty -Path 'HKLM:\\SOFTWARE\\OpenSSH' -Name DefaultShell -Value 'C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe' -PropertyType String -Force", | |
| 199 | + "New-NetFirewallRule -Name sshd -DisplayName 'OpenSSH Server (sshd)' -Enabled True -Direction Inbound -Protocol TCP -Action Allow -LocalPort 22 -ErrorAction SilentlyContinue", | |
| 200 | + "$akf = 'C:\\ProgramData\\ssh\\administrators_authorized_keys'", | |
| 201 | + `Set-Content -Path $akf -Value '${publicKey}'`, | |
| 202 | + "icacls $akf /inheritance:r /grant 'Administrators:F' /grant 'SYSTEM:F'", | |
| 203 | + "</powershell>", | |
| 204 | + ].join("\n"); | |
| 205 | + | |
| 206 | +const linuxUserData = (publicKey: string): string => | |
| 207 | + ["#cloud-config", "ssh_authorized_keys:", ` - ${publicKey}`].join("\n"); | |
| 208 | + | |
| 209 | +const freePort = (): Promise<number> => | |
| 210 | + new Promise((resolve, reject) => { | |
| 211 | + const srv = net.createServer(); | |
| 212 | + srv.on("error", reject); | |
| 213 | + srv.listen(0, "127.0.0.1", () => { | |
| 214 | + const port = (srv.address() as net.AddressInfo).port; | |
| 215 | + srv.close(() => resolve(port)); | |
| 216 | + }); | |
| 217 | + }); | |
| 218 | + | |
| 219 | +const waitLocalPort = async (port: number, attempts = 40): Promise<void> => { | |
| 220 | + for (let i = 0; i < attempts; i++) { | |
| 221 | + const ok = await new Promise<boolean>((resolve) => { | |
| 222 | + const sock = net.connect({ host: "127.0.0.1", port }, () => { | |
| 223 | + sock.destroy(); | |
| 224 | + resolve(true); | |
| 225 | + }); | |
| 226 | + sock.on("error", () => resolve(false)); | |
| 227 | + sock.setTimeout(1000, () => { | |
| 228 | + sock.destroy(); | |
| 229 | + resolve(false); | |
| 230 | + }); | |
| 231 | + }); | |
| 232 | + if (ok) return; | |
| 233 | + await sleep(500); | |
| 234 | + } | |
| 235 | + throw new Error(`tunnel local port ${port} never came up`); | |
| 236 | +}; | |
| 237 | + | |
| 238 | +export const ec2Vm = (os: VmOs, arch: VmArch = "x64"): VmProvider => ({ | |
| 239 | + os, | |
| 240 | + provision: async () => { | |
| 241 | + const user = guestUser(os); | |
| 242 | + // A throwaway SSH keypair, authorized via user-data (no EC2 key pair needed — | |
| 243 | + // we drive over OpenSSH key auth, not the Windows password). | |
| 244 | + const keyDir = mkdtempSync(join(tmpdir(), "executor-ec2-")); | |
| 245 | + const keyPath = join(keyDir, "id"); | |
| 246 | + await execFileP("ssh-keygen", ["-t", "ed25519", "-N", "", "-q", "-f", keyPath]); | |
| 247 | + chmodSync(keyPath, 0o600); | |
| 248 | + const publicKey = (await execFileP("ssh-keygen", ["-y", "-f", keyPath])).stdout.trim(); | |
| 249 | + | |
| 250 | + const [myIp, ami, subnet] = await Promise.all([egressIp(), latestAmi(os), defaultSubnet()]); | |
| 251 | + const sg = await ensureSecurityGroup(myIp); | |
| 252 | + const userData = os === "windows" ? windowsUserData(publicKey) : linuxUserData(publicKey); | |
| 253 | + const userDataFile = join(keyDir, "user-data.txt"); | |
| 254 | + writeFileSync(userDataFile, userData); | |
| 255 | + | |
| 256 | + const instanceId = await aws([ | |
| 257 | + "ec2", | |
| 258 | + "run-instances", | |
| 259 | + "--image-id", | |
| 260 | + ami, | |
| 261 | + "--instance-type", | |
| 262 | + INSTANCE_TYPE, | |
| 263 | + "--count", | |
| 264 | + "1", | |
| 265 | + "--security-group-ids", | |
| 266 | + sg, | |
| 267 | + "--subnet-id", | |
| 268 | + subnet, | |
| 269 | + "--associate-public-ip-address", | |
| 270 | + "--instance-initiated-shutdown-behavior", | |
| 271 | + "terminate", | |
| 272 | + "--user-data", | |
| 273 | + `file://${userDataFile}`, | |
| 274 | + "--tag-specifications", | |
| 275 | + `ResourceType=instance,Tags=[{Key=Name,Value=${TAG}-${os}},{Key=purpose,Value=e2e}]`, | |
| 276 | + "--query", | |
| 277 | + "Instances[0].InstanceId", | |
| 278 | + ]); | |
| 279 | + | |
| 280 | + let ip = ""; | |
| 281 | + const tunnelClosers: Array<() => void> = []; | |
| 282 | + | |
| 283 | + const ssh = async (command: string): Promise<SshResult> => { | |
| 284 | + try { | |
| 285 | + const { stdout, stderr } = await execFileP( | |
| 286 | + "ssh", | |
| 287 | + ["-i", keyPath, ...SSH_OPTS, `${user}@${ip}`, command], | |
| 288 | + { maxBuffer: 64 * 1024 * 1024 }, | |
| 289 | + ); | |
| 290 | + return { stdout, stderr, code: 0 }; | |
| 291 | + } catch (err) { | |
| 292 | + const e = err as { stdout?: string; stderr?: string; code?: number }; | |
| 293 | + return { | |
| 294 | + stdout: e.stdout ?? "", | |
| 295 | + stderr: e.stderr ?? "", | |
| 296 | + code: typeof e.code === "number" ? e.code : 1, | |
| 297 | + }; | |
| 298 | + } | |
| 299 | + }; | |
| 300 | + | |
| 301 | + const waitSshUp = async (attempts: number): Promise<boolean> => { | |
| 302 | + for (let i = 0; i < attempts; i++) { | |
| 303 | + if ((await ssh(os === "windows" ? "echo ok" : "true")).code === 0) return true; | |
| 304 | + await sleep(5000); | |
| 305 | + } | |
| 306 | + return false; | |
| 307 | + }; | |
| 308 | + | |
| 309 | + const waitSshDown = async (attempts = 40): Promise<void> => { | |
| 310 | + for (let i = 0; i < attempts; i++) { | |
| 311 | + if ((await ssh("echo up").catch(() => ({ code: 1 }) as SshResult)).code !== 0) return; | |
| 312 | + await sleep(3000); | |
| 313 | + } | |
| 314 | + // never observed down — caller's boot-time check is the backstop. | |
| 315 | + }; | |
| 316 | + | |
| 317 | + const bootTime = async (): Promise<string> => | |
| 318 | + os === "windows" | |
| 319 | + ? ( | |
| 320 | + await ssh("(Get-CimInstance Win32_OperatingSystem).LastBootUpTime.ToString('o')") | |
| 321 | + ).stdout.trim() | |
| 322 | + : (await ssh("cat /proc/sys/kernel/random/boot_id")).stdout.trim(); | |
| 323 | + | |
| 324 | + const handle: VmHandle = { | |
| 325 | + os, | |
| 326 | + arch, | |
| 327 | + sshKeyPath: keyPath, | |
| 328 | + get host() { | |
| 329 | + return ip; | |
| 330 | + }, | |
| 331 | + ssh, | |
| 332 | + push: async (localPath, remotePath) => { | |
| 333 | + await execFileP("scp", [ | |
| 334 | + "-i", | |
| 335 | + keyPath, | |
| 336 | + "-r", | |
| 337 | + ...SSH_OPTS, | |
| 338 | + localPath, | |
| 339 | + `${user}@${ip}:${remotePath}`, | |
| 340 | + ]); | |
| 341 | + }, | |
| 342 | + reboot: async () => { | |
| 343 | + const before = await bootTime(); | |
| 344 | + await ssh(os === "windows" ? "Restart-Computer -Force" : "sudo reboot").catch( | |
| 345 | + () => undefined, | |
| 346 | + ); | |
| 347 | + await waitSshDown(); | |
| 348 | + if (!(await waitSshUp(60))) throw new Error(`ec2 ${os}: SSH did not return after reboot`); | |
| 349 | + const after = await bootTime(); | |
| 350 | + if (before && after && before === after) { | |
| 351 | + throw new Error( | |
| 352 | + `ec2 ${os}: boot time unchanged after reboot — the guest never actually rebooted`, | |
| 353 | + ); | |
| 354 | + } | |
| 355 | + }, | |
| 356 | + tunnel: async (guestPort) => { | |
| 357 | + const localPort = await freePort(); | |
| 358 | + let closed = false; | |
| 359 | + let child: ReturnType<typeof import("node:child_process").spawn> | undefined; | |
| 360 | + const { spawn } = await import("node:child_process"); | |
| 361 | + const spawnOnce = (): void => { | |
| 362 | + child = spawn( | |
| 363 | + "ssh", | |
| 364 | + [ | |
| 365 | + "-i", | |
| 366 | + keyPath, | |
| 367 | + ...SSH_OPTS, | |
| 368 | + "-N", | |
| 369 | + "-L", | |
| 370 | + `${localPort}:127.0.0.1:${guestPort}`, | |
| 371 | + `${user}@${ip}`, | |
| 372 | + ], | |
| 373 | + { stdio: "ignore" }, | |
| 374 | + ); | |
| 375 | + child.on("exit", () => { | |
| 376 | + if (!closed) setTimeout(spawnOnce, 2000); | |
| 377 | + }); | |
| 378 | + }; | |
| 379 | + spawnOnce(); | |
| 380 | + const close = (): void => { | |
| 381 | + closed = true; | |
| 382 | + child?.kill(); | |
| 383 | + }; | |
| 384 | + tunnelClosers.push(close); | |
| 385 | + await waitLocalPort(localPort); | |
| 386 | + return { localPort, close }; | |
| 387 | + }, | |
| 388 | + discard: async () => { | |
| 389 | + for (const close of tunnelClosers) close(); | |
| 390 | + await aws(["ec2", "terminate-instances", "--instance-ids", instanceId]).catch( | |
| 391 | + () => undefined, | |
| 392 | + ); | |
| 393 | + }, | |
| 394 | + }; | |
| 395 | + | |
| 396 | + // Wait for a public IP, then for OpenSSH (Windows boot + FoD install ≈ 2-4 min). | |
| 397 | + for (let i = 0; i < 60; i++) { | |
| 398 | + const got = await aws([ | |
| 399 | + "ec2", | |
| 400 | + "describe-instances", | |
| 401 | + "--instance-ids", | |
| 402 | + instanceId, | |
| 403 | + "--query", | |
| 404 | + "Reservations[0].Instances[0].PublicIpAddress", | |
| 405 | + ]).catch(() => ""); | |
| 406 | + if (got && got !== "None") { | |
| 407 | + ip = got; | |
| 408 | + break; | |
| 409 | + } | |
| 410 | + await sleep(5000); | |
| 411 | + } | |
| 412 | + if (!ip) { | |
| 413 | + await handle.discard(); | |
| 414 | + throw new Error(`ec2 ${os}: no public IP within 300s`); | |
| 415 | + } | |
| 416 | + if (!(await waitSshUp(60))) { | |
| 417 | + await handle.discard(); | |
| 418 | + throw new Error(`ec2 ${os}: SSH never came up`); | |
| 419 | + } | |
| 420 | + return handle; | |
| 421 | + }, | |
| 422 | +}); | |
| @@ -0,0 +1,224 @@ | ||
| 1 | +// tart provider: macOS + Linux guests on an Apple-Silicon host (the Mini). | |
| 2 | +// Mirrors the by-hand reboot harness — clone a base image, boot headless, drive | |
| 3 | +// over sshpass, reboot the guest OS for real, tear down the clone. | |
| 4 | + | |
| 5 | +import { execFile, spawn, type ChildProcess } from "node:child_process"; | |
| 6 | +import net from "node:net"; | |
| 7 | +import { promisify } from "node:util"; | |
| 8 | + | |
| 9 | +import { | |
| 10 | + type SshResult, | |
| 11 | + sleep, | |
| 12 | + type Tunnel, | |
| 13 | + type VmArch, | |
| 14 | + type VmHandle, | |
| 15 | + type VmProvider, | |
| 16 | +} from "./types"; | |
| 17 | + | |
| 18 | +const execFileP = promisify(execFile); | |
| 19 | + | |
| 20 | +const TART = process.env.E2E_TART_BIN ?? "/opt/homebrew/bin/tart"; | |
| 21 | +const SSHPASS = process.env.E2E_SSHPASS_BIN ?? "/opt/homebrew/bin/sshpass"; | |
| 22 | +const SSH_OPTS = [ | |
| 23 | + "-o", | |
| 24 | + "StrictHostKeyChecking=no", | |
| 25 | + "-o", | |
| 26 | + "UserKnownHostsFile=/dev/null", | |
| 27 | + "-o", | |
| 28 | + "ConnectTimeout=8", | |
| 29 | + "-o", | |
| 30 | + "ServerAliveInterval=5", | |
| 31 | + "-o", | |
| 32 | + "LogLevel=ERROR", | |
| 33 | +]; | |
| 34 | +const GUEST_USER = "admin"; | |
| 35 | +const GUEST_PASS = "admin"; | |
| 36 | + | |
| 37 | +/** | |
| 38 | + * Reboot a tart guest by address, with no live handle. `restart()` runs in a | |
| 39 | + * vitest worker (separate process from the globalsetup that owns the VM), so it | |
| 40 | + * re-derives the guest address from env and triggers the reboot statelessly — | |
| 41 | + * the reconnecting tunnel and a health poll confirm recovery. | |
| 42 | + */ | |
| 43 | +export const sshRebootGuest = async (ip: string): Promise<void> => { | |
| 44 | + await execFileP(SSHPASS, [ | |
| 45 | + "-p", | |
| 46 | + GUEST_PASS, | |
| 47 | + "ssh", | |
| 48 | + ...SSH_OPTS, | |
| 49 | + `${GUEST_USER}@${ip}`, | |
| 50 | + "sudo reboot", | |
| 51 | + ]).catch(() => undefined); // the connection drops mid-call | |
| 52 | +}; | |
| 53 | + | |
| 54 | +const baseImage = (os: "macos" | "linux"): string => | |
| 55 | + os === "macos" | |
| 56 | + ? (process.env.E2E_TART_MACOS_BASE ?? "executor-macos-base") | |
| 57 | + : (process.env.E2E_TART_LINUX_BASE ?? "executor-linux-base"); | |
| 58 | + | |
| 59 | +/** Ask the OS for a free localhost port (for SSH tunnels). */ | |
| 60 | +const freePort = (): Promise<number> => | |
| 61 | + new Promise((resolve, reject) => { | |
| 62 | + const srv = net.createServer(); | |
| 63 | + srv.on("error", reject); | |
| 64 | + srv.listen(0, "127.0.0.1", () => { | |
| 65 | + const port = (srv.address() as net.AddressInfo).port; | |
| 66 | + srv.close(() => resolve(port)); | |
| 67 | + }); | |
| 68 | + }); | |
| 69 | + | |
| 70 | +/** Resolve once a TCP connect to localhost:port succeeds (SSH bound the forward). */ | |
| 71 | +const waitLocalPort = async (port: number, attempts = 40): Promise<void> => { | |
| 72 | + for (let i = 0; i < attempts; i++) { | |
| 73 | + const ok = await new Promise<boolean>((resolve) => { | |
| 74 | + const sock = net.connect({ host: "127.0.0.1", port }, () => { | |
| 75 | + sock.destroy(); | |
| 76 | + resolve(true); | |
| 77 | + }); | |
| 78 | + sock.on("error", () => resolve(false)); | |
| 79 | + sock.setTimeout(1000, () => { | |
| 80 | + sock.destroy(); | |
| 81 | + resolve(false); | |
| 82 | + }); | |
| 83 | + }); | |
| 84 | + if (ok) return; | |
| 85 | + await sleep(500); | |
| 86 | + } | |
| 87 | + throw new Error(`tunnel local port ${port} never came up`); | |
| 88 | +}; | |
| 89 | + | |
| 90 | +export const tartVm = (os: "macos" | "linux", arch: VmArch = "arm64"): VmProvider => ({ | |
| 91 | + os, | |
| 92 | + provision: async () => { | |
| 93 | + const name = `executor-e2e-${os}-${process.pid}-${Math.floor(performance.now())}`; | |
| 94 | + await execFileP(TART, ["clone", baseImage(os), name]); | |
| 95 | + const runProc = spawn(TART, ["run", name, "--no-graphics"], { stdio: "ignore" }); | |
| 96 | + | |
| 97 | + const tunnelClosers: Array<() => void> = []; | |
| 98 | + let ip = ""; | |
| 99 | + | |
| 100 | + const fetchIp = async (): Promise<boolean> => { | |
| 101 | + for (let i = 0; i < 90; i++) { | |
| 102 | + try { | |
| 103 | + const { stdout } = await execFileP(TART, ["ip", name]); | |
| 104 | + if (stdout.trim()) { | |
| 105 | + ip = stdout.trim(); | |
| 106 | + return true; | |
| 107 | + } | |
| 108 | + } catch { | |
| 109 | + /* not booted yet */ | |
| 110 | + } | |
| 111 | + await sleep(2000); | |
| 112 | + } | |
| 113 | + return false; | |
| 114 | + }; | |
| 115 | + | |
| 116 | + // Linux systemctl --user calls need XDG_RUNTIME_DIR; harmless elsewhere. | |
| 117 | + const wrap = (command: string): string => | |
| 118 | + os === "linux" ? `export XDG_RUNTIME_DIR=/run/user/$(id -u); ${command}` : command; | |
| 119 | + | |
| 120 | + const ssh = async (command: string): Promise<SshResult> => { | |
| 121 | + try { | |
| 122 | + const { stdout, stderr } = await execFileP( | |
| 123 | + SSHPASS, | |
| 124 | + ["-p", GUEST_PASS, "ssh", ...SSH_OPTS, `${GUEST_USER}@${ip}`, wrap(command)], | |
| 125 | + { maxBuffer: 32 * 1024 * 1024 }, | |
| 126 | + ); | |
| 127 | + return { stdout, stderr, code: 0 }; | |
| 128 | + } catch (err) { | |
| 129 | + const e = err as { stdout?: string; stderr?: string; code?: number }; | |
| 130 | + return { | |
| 131 | + stdout: e.stdout ?? "", | |
| 132 | + stderr: e.stderr ?? "", | |
| 133 | + code: typeof e.code === "number" ? e.code : 1, | |
| 134 | + }; | |
| 135 | + } | |
| 136 | + }; | |
| 137 | + | |
| 138 | + const waitSsh = async (attempts: number): Promise<boolean> => { | |
| 139 | + for (let i = 0; i < attempts; i++) { | |
| 140 | + if ((await ssh("true")).code === 0) return true; | |
| 141 | + await sleep(2000); | |
| 142 | + } | |
| 143 | + return false; | |
| 144 | + }; | |
| 145 | + | |
| 146 | + const handle: VmHandle = { | |
| 147 | + os, | |
| 148 | + arch, | |
| 149 | + get host() { | |
| 150 | + return ip; | |
| 151 | + }, | |
| 152 | + ssh, | |
| 153 | + push: async (localPath, remotePath) => { | |
| 154 | + await execFileP(SSHPASS, [ | |
| 155 | + "-p", | |
| 156 | + GUEST_PASS, | |
| 157 | + "scp", | |
| 158 | + "-r", | |
| 159 | + ...SSH_OPTS, | |
| 160 | + localPath, | |
| 161 | + `${GUEST_USER}@${ip}:${remotePath}`, | |
| 162 | + ]); | |
| 163 | + }, | |
| 164 | + reboot: async () => { | |
| 165 | + await ssh("sudo reboot").catch(() => undefined); // connection drops mid-call | |
| 166 | + await sleep(5000); | |
| 167 | + if (!(await fetchIp())) throw new Error(`tart ${os}: no IP after reboot`); | |
| 168 | + if (!(await waitSsh(120))) throw new Error(`tart ${os}: SSH did not return after reboot`); | |
| 169 | + }, | |
| 170 | + tunnel: async (guestPort) => { | |
| 171 | + const localPort = await freePort(); | |
| 172 | + // Reconnecting forward: when the guest reboots the ssh exits, so respawn | |
| 173 | + // it until closed. `restart()` health-polls through this local port, so | |
| 174 | + // it only goes green once the daemon AND the forward are back. | |
| 175 | + let closed = false; | |
| 176 | + let child: ChildProcess | undefined; | |
| 177 | + const spawnOnce = (): void => { | |
| 178 | + child = spawn( | |
| 179 | + SSHPASS, | |
| 180 | + [ | |
| 181 | + "-p", | |
| 182 | + GUEST_PASS, | |
| 183 | + "ssh", | |
| 184 | + ...SSH_OPTS, | |
| 185 | + "-N", | |
| 186 | + "-L", | |
| 187 | + `${localPort}:127.0.0.1:${guestPort}`, | |
| 188 | + `${GUEST_USER}@${ip}`, | |
| 189 | + ], | |
| 190 | + { stdio: "ignore" }, | |
| 191 | + ); | |
| 192 | + child.on("exit", () => { | |
| 193 | + if (!closed) setTimeout(spawnOnce, 2000); | |
| 194 | + }); | |
| 195 | + }; | |
| 196 | + spawnOnce(); | |
| 197 | + const close = (): void => { | |
| 198 | + closed = true; | |
| 199 | + child?.kill(); | |
| 200 | + }; | |
| 201 | + tunnelClosers.push(close); | |
| 202 | + await waitLocalPort(localPort); | |
| 203 | + const tunnel: Tunnel = { localPort, close }; | |
| 204 | + return tunnel; | |
| 205 | + }, | |
| 206 | + discard: async () => { | |
| 207 | + for (const close of tunnelClosers) close(); | |
| 208 | + runProc.kill(); | |
| 209 | + await sleep(1500); | |
| 210 | + await execFileP(TART, ["delete", name]).catch(() => undefined); | |
| 211 | + }, | |
| 212 | + }; | |
| 213 | + | |
| 214 | + if (!(await fetchIp())) { | |
| 215 | + await handle.discard(); | |
| 216 | + throw new Error(`tart ${os}: no IP within 180s`); | |
| 217 | + } | |
| 218 | + if (!(await waitSsh(90))) { | |
| 219 | + await handle.discard(); | |
| 220 | + throw new Error(`tart ${os}: SSH never came up`); | |
| 221 | + } | |
| 222 | + return handle; | |
| 223 | + }, | |
| 224 | +}); | |
| @@ -0,0 +1,51 @@ | ||
| 1 | +// VM substrate for the cross-OS supervised-daemon e2e targets. | |
| 2 | +// | |
| 3 | +// A VmHandle is a booted guest we can drive over SSH, REBOOT for real, and tear | |
| 4 | +// down. Providers: tart (macOS + Linux, local on an Apple-Silicon host) and ec2 | |
| 5 | +// (Windows, ephemeral). This is the codified form of the by-hand reboot harness. | |
| 6 | + | |
| 7 | +export type VmOs = "macos" | "linux" | "windows"; | |
| 8 | +export type VmArch = "arm64" | "x64"; | |
| 9 | + | |
| 10 | +export interface SshResult { | |
| 11 | + readonly stdout: string; | |
| 12 | + readonly stderr: string; | |
| 13 | + readonly code: number; | |
| 14 | +} | |
| 15 | + | |
| 16 | +/** An open SSH local-forward (`localhost:localPort` → guest:guestPort). */ | |
| 17 | +export interface Tunnel { | |
| 18 | + readonly localPort: number; | |
| 19 | + close(): void; | |
| 20 | +} | |
| 21 | + | |
| 22 | +export interface VmHandle { | |
| 23 | + readonly os: VmOs; | |
| 24 | + readonly arch: VmArch; | |
| 25 | + /** Current reachable address of the guest (re-resolved across reboots). */ | |
| 26 | + readonly host: string; | |
| 27 | + /** | |
| 28 | + * Path to the SSH private key for key-based providers (EC2). Undefined for | |
| 29 | + * password-based providers (tart/sshpass). Published by globalsetup so the | |
| 30 | + * stateless worker-side `restart()` can reboot the guest. | |
| 31 | + */ | |
| 32 | + readonly sshKeyPath?: string; | |
| 33 | + /** Run a command in the guest over SSH (shell on Unix, PowerShell on Windows). */ | |
| 34 | + ssh(command: string): Promise<SshResult>; | |
| 35 | + /** Copy a local file or directory into the guest (recursive for directories). */ | |
| 36 | + push(localPath: string, remotePath: string): Promise<void>; | |
| 37 | + /** Reboot the guest OS; resolves only once SSH is reachable again. */ | |
| 38 | + reboot(): Promise<void>; | |
| 39 | + /** Forward `localhost:<localPort>` → `guest:<guestPort>` over SSH. */ | |
| 40 | + tunnel(guestPort: number): Promise<Tunnel>; | |
| 41 | + /** Discard the VM (delete the tart clone / terminate the EC2 instance). */ | |
| 42 | + discard(): Promise<void>; | |
| 43 | +} | |
| 44 | + | |
| 45 | +export interface VmProvider { | |
| 46 | + readonly os: VmOs; | |
| 47 | + /** Boot a fresh guest and wait until SSH answers. */ | |
| 48 | + provision(): Promise<VmHandle>; | |
| 49 | +} | |
| 50 | + | |
| 51 | +export const sleep = (ms: number): Promise<void> => new Promise((r) => setTimeout(r, ms)); | |
| @@ -0,0 +1,59 @@ | ||
| 1 | +// The supervised local CLI daemon as a target. `executor service install` boots | |
| 2 | +// an OS-managed daemon (launchd / systemd / Task Scheduler) inside a guest VM; | |
| 3 | +// `restart()` reboots that guest for REAL, so restart-persistence proves the | |
| 4 | +// boot-time auto-start path (RunAtLoad / linger / AtStartup), not just a process | |
| 5 | +// restart. The guest daemon binds loopback, so globalsetup forwards it to a | |
| 6 | +// local port over a reconnecting SSH tunnel — making target.baseUrl work | |
| 7 | +// unchanged for the api surface. Boot + tunnel live in setup/cli.globalsetup.ts; | |
| 8 | +// this target only reads what that published via env. | |
| 9 | +import { Effect } from "effect"; | |
| 10 | + | |
| 11 | +import { waitForHttp, waitForHttpDown } from "../setup/boot"; | |
| 12 | +import { ec2RebootGuest } from "../src/vm/ec2"; | |
| 13 | +import { sshRebootGuest } from "../src/vm/tart"; | |
| 14 | +import type { VmOs } from "../src/vm/types"; | |
| 15 | +import type { Capability, Identity, Target } from "../src/target"; | |
| 16 | + | |
| 17 | +const env = (key: string): string => { | |
| 18 | + const value = process.env[key]; | |
| 19 | + if (!value) throw new Error(`cli target: ${key} not set — did cli.globalsetup run?`); | |
| 20 | + return value; | |
| 21 | +}; | |
| 22 | + | |
| 23 | +export const cliTarget = (): Target => { | |
| 24 | + const baseUrl = env("E2E_CLI_BASE_URL"); | |
| 25 | + const os = (process.env.E2E_VM_OS ?? "macos") as VmOs; | |
| 26 | + | |
| 27 | + return { | |
| 28 | + name: process.env.E2E_TARGET ?? "cli", | |
| 29 | + baseUrl, | |
| 30 | + mcpUrl: `${baseUrl}/mcp`, | |
| 31 | + capabilities: new Set<Capability>(["api"]), | |
| 32 | + // The supervised daemon is bearer-gated (auth.json); globalsetup reads the | |
| 33 | + // token from the guest and publishes it. A wrong/absent token still gets a | |
| 34 | + // clean 401, which the api surface treats as "up". | |
| 35 | + newIdentity: () => | |
| 36 | + Effect.sync( | |
| 37 | + (): Identity => ({ | |
| 38 | + label: "cli-daemon", | |
| 39 | + headers: { Authorization: `Bearer ${env("E2E_CLI_AUTH_TOKEN")}` }, | |
| 40 | + }), | |
| 41 | + ), | |
| 42 | + // A genuine machine reboot, not a service kick: reboot the guest OS, GATE on | |
| 43 | + // the daemon actually going down (an orderly shutdown serves for several | |
| 44 | + // seconds + the reconnecting tunnel re-forwards, so "reachable" right after | |
| 45 | + // the reboot command would false-pass), then wait for the supervised daemon | |
| 46 | + // to auto-start and serve again — proving the boot-time auto-start path. | |
| 47 | + restart: () => | |
| 48 | + Effect.promise(async () => { | |
| 49 | + const host = env("E2E_CLI_VM_HOST"); | |
| 50 | + if (os === "windows") { | |
| 51 | + await ec2RebootGuest(host, env("E2E_CLI_SSH_KEY"), os); | |
| 52 | + } else { | |
| 53 | + await sshRebootGuest(host); | |
| 54 | + } | |
| 55 | + await waitForHttpDown(`${baseUrl}/`, { timeoutMs: 120_000 }); | |
| 56 | + await waitForHttp(`${baseUrl}/`, { timeoutMs: 240_000 }); | |
| 57 | + }), | |
| 58 | + }; | |
| 59 | +}; | |
| @@ -2,6 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | // once per worker. Adding a target = one factory entry here + a project in |
| 3 | 3 | // vitest.config.ts + a globalsetup that boots (or attaches to) the instance. |
| 4 | 4 | import type { Target } from "../src/target"; |
| 5 | +import { cliTarget } from "./cli"; | |
| 5 | 6 | import { cloudTarget } from "./cloud"; |
| 6 | 7 | import { cloudflareTarget } from "./cloudflare"; |
| 7 | 8 | import { desktopTarget } from "./desktop"; |
| @@ -16,6 +17,11 @@ const factories: Record<string, () => Target> = { · 8 unmodified lines | ||
| 16 | 17 | cloudflare: cloudflareTarget, |
| 17 | 18 | desktop: desktopTarget, |
| 18 | 19 | local: localTarget, |
| 20 | + // The supervised CLI daemon inside a VM, one project per guest OS — restart() | |
| 21 | + // is a real reboot. See setup/cli.globalsetup.ts. | |
| 22 | + "cli-macos": cliTarget, | |
| 23 | + "cli-linux": cliTarget, | |
| 24 | + "cli-windows": cliTarget, | |
| 19 | 25 | }; |
| 20 | 26 | |
| 21 | 27 | let current: Target | undefined; |
| @@ -68,6 +68,21 @@ export default defineConfig({ · 67 unmodified lines | ||
| 68 | 68 | fileParallelism: true, |
| 69 | 69 | testTimeout: 180_000, |
| 70 | 70 | }), |
| 71 | + // The supervised CLI daemon inside a guest VM, one project per OS. The | |
| 72 | + // globalsetup provisions a VM, `executor service install`s the daemon, and | |
| 73 | + // tunnels it; restart() reboots the guest for REAL, so restart-persistence | |
| 74 | + // proves the boot-time auto-start path. Needs tart (macOS/Linux) or an EC2 | |
| 75 | + // credential (Windows); not part of the default `npm run test` chain — run | |
| 76 | + // with `vitest run --project cli-macos` (etc.) on the Mini. | |
| 77 | + ...(["macos", "linux", "windows"] as const).map((os) => | |
| 78 | + project(`cli-${os}`, { | |
| 79 | + include: ["scenarios/restart-persistence.test.ts", "cli/**/*.test.ts"], | |
| 80 | + env: { E2E_TARGET: `cli-${os}`, E2E_VM_OS: os }, | |
| 81 | + fileParallelism: false, | |
| 82 | + testTimeout: 300_000, | |
| 83 | + hookTimeout: 900_000, | |
| 84 | + }), | |
| 85 | + ), | |
| 71 | 86 | ], |
| 72 | 87 | }, |
| 73 | 88 | }); |