History

Classify ECONNREFUSED as a retryable storage connection fault

Rhys Sullivan committed 2 months ago

Commit 49a351f1 parent bf5263f3 files · +11 −3Browse files

Unified · Split

3 files
  1. .changeset/storage-error-shaping.md +1 −1
  2. packages/core/sdk/src/fuma-runtime.test.ts +5 −0
  3. packages/core/sdk/src/fuma-runtime.ts +5 −2
.changeset/storage-error-shaping.md+1−1
@@ -5,4 +5,4 @@ · 4 unmodified lines
55
66 Build `StorageError.message` from the call-site label plus the driver's error code instead of the driver's raw text. The driver text is drizzle's `Failed query: <sql>\nparams: <bound values>`, so error reporting grouped one storage defect by statement shape and printed bound parameters into issue titles. The full driver error stays on `cause`.
77
8−Add `StorageConnectionError`, a `StorageFailure` variant for postgres.js connection faults (`CONNECTION_ENDED`, `CONNECTION_CLOSED`, `CONNECTION_DESTROYED`, `CONNECT_TIMEOUT`, `ECONNRESET`) and workerd's cross-request I/O rejection. It carries the fault `code` and a `retryable` flag so a lost socket can be told apart from a pool-lifetime bug.
8+Add `StorageConnectionError`, a `StorageFailure` variant for postgres.js connection faults (`CONNECTION_ENDED`, `CONNECTION_CLOSED`, `CONNECTION_DESTROYED`, `CONNECT_TIMEOUT`, `ECONNREFUSED`, `ECONNRESET`) and workerd's cross-request I/O rejection. It carries the fault `code` and a `retryable` flag so a lost socket can be told apart from a pool-lifetime bug.
packages/core/sdk/src/fuma-runtime.test.ts+5−0
@@ -143,6 +143,11 @@ describe("fumaFailureFromCause — classification", () => { · 142 unmodified lines
143143 ["CONNECTION_CLOSED"],
144144 ["CONNECTION_DESTROYED"],
145145 ["CONNECT_TIMEOUT"],
146+ // A backend that is simply down: postgres.js surfaces the socket errno
147+ // verbatim, so `connect ECONNREFUSED …` is the ordinary "database
148+ // unreachable" fault and must classify like the rest.
149+ ["ECONNREFUSED"],
150+ ["ECONNRESET"],
146151 ])("classifies postgres.js %s as a storage connection fault", (code) => {
147152 const failure = fumaFailureFromCause(
148153 "plugin_storage.findFirst",
packages/core/sdk/src/fuma-runtime.ts+5−2
@@ -19,8 +19,10 @@ export class UniqueViolationError extends Data.TaggedError("UniqueViolationError · 18 unmodified lines
1919 * bug that must stay loud.
2020 *
2121 * `retryable` says which. It is a property of the fault, not a policy:
22− * - `true` — the socket died underneath a live pool (`CONNECTION_CLOSED`,
23− * `CONNECT_TIMEOUT`, `ECONNRESET`). Reconnecting can succeed.
22+ * - `true` — the socket died underneath a live pool, or was never
23+ * established because the backend was unreachable (`CONNECTION_CLOSED`,
24+ * `CONNECT_TIMEOUT`, `ECONNREFUSED`, `ECONNRESET`). Reconnecting can
25+ * succeed.
2426 * - `false` — the pool was already torn down or the socket belongs to a
2527 * different request context (`CONNECTION_ENDED`, `CONNECTION_DESTROYED`,
2628 * the workerd cross-request I/O rejection). Retrying is futile by
@@ -94,6 +96,7 @@ const isUniqueViolation = (cause: unknown): boolean => { · 67 unmodified lines
9496 const RETRYABLE_CONNECTION_CODES: ReadonlySet<string> = new Set([
9597 "CONNECTION_CLOSED",
9698 "CONNECT_TIMEOUT",
99+ "ECONNREFUSED",
97100 "ECONNRESET",
98101 ]);
99102 const FATAL_CONNECTION_CODES: ReadonlySet<string> = new Set([