Retry transient network errors + fix CloudFetch prefetch-rejection leak

[msrathore-db]

Summary

Two complementary fixes for the Thrift + CloudFetch ECONNRESET / "socket hang up" failures users hit while draining large (~100MB) results (issue #260). Combined into one branch per request.

1. Retry transient network errors in HttpRetryPolicy (folds in #398)

invokeWithRetry previously only inspected response.status, so when fetch() itself rejected with a network error the rejection propagated on attempt #1 with no retry. Now it catches operation rejections and classifies them via isRetryableNetworkError — FetchError system type, OS errnos (ECONNRESET/ECONNREFUSED/ETIMEDOUT/EHOSTUNREACH/ENETUNREACH/EPIPE/ENOTFOUND/EAI_AGAIN) and message patterns (socket hang up/premature close/aborted) — retrying within the existing attempt/timeout budget. Thrift (response.buffer()) and CloudFetch (response.arrayBuffer()) body reads were moved inside the retried block so mid-stream "Premature close" is retried too.

Idempotency unchanged: ExecuteStatement and other write-y Thrift methods keep NullRetryPolicy — no double-execution risk.

2. Fix CloudFetch prefetch-rejection leak

CloudFetchResultHandler.fetchNext fans out up to cloudFetchConcurrentDownloads downloads but only awaits the head. If the head rejects and the consumer stops iterating, the remaining in-flight promises had no handler and surfaced as unhandledRejection. Each download is now reflected into a SettledDownload value the instant it's created, so a prefetched download can never become an unhandled rejection. The error is re-thrown only when that task is actually consumed, so observable behavior is unchanged.

These are orthogonal: #1 makes a transient reset recover; #2 makes the terminal-failure path clean (including once the retry budget is exhausted).

Test plan

• HttpRetryPolicy unit suite: 14/14 passing, incl. real ECONNRESET and "Premature close" cases.
• New regression test: prefetched downloadTasks settle to { ok: false } (fulfilled, not rejected) when downloads fail.
• Standalone checks against the built dist:
  • downloads all fail + consumer bails → 0 leaked unhandledRejections, fetchNext throws cleanly.
  • real node-fetch GET against a local server that RSTs its first 2 connections → recovers transparently on attempt Update CHANGELOG #3 (HTTP 200).

Live before/after (real warehouse)

Verified end-to-end against a real staging warehouse on AWS us-west-2, draining the original failing query (catalog_sales, 664,893 rows / ~100MB) over the Thrift backend with CloudFetch on, 3 attempts each. Same machine, same warehouse, same query — only the build differs.

Build	thrift + cloudFetch	thrift + NO cloudFetch (control)
base main (no fixes)	❌ FAIL 0/3 — FetchError: socket hang up on the S3 result-file GETs, surfacing as leaked unhandledRejections	✅ OK 3/3
this PR (both fixes)	✅ PASS 3/3 — all 664,893 rows drained, 0 stray async errors	✅ OK 3/3

The non-CloudFetch control passes on both builds, isolating the failure to the CloudFetch download path. The fix turns a reproducible 0/3 failure into a clean 3/3 pass.

Note: the live run used useLZ4Compression: false (the test box lacks the lz4 native module). The failure and fix are at the socket/fetch layer — upstream of LZ4 decompression — so the LZ4-on production path is fixed by the same code; it just wasn't exercised directly in this run.

This pull request and its description were written by Isaac.

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).