Try the next resolved address if HTTP connect fails

[alexey-milovidov]

On macOS hosts where IPv6 is advertised by DNS but unreachable on the network, aiGenerate and other outbound HTTP requests fail outright when the resolver picks the AAAA address:

SELECT aiGenerate('ai_credentials', 'Write a one-line SQL joke')

Code: 1000. DB::Exception: Net Exception: No route to host: [2607:6bc0::10]:443. (POCO_EXCEPTION)

$ host api.anthropic.com
api.anthropic.com has address 160.79.104.10
api.anthropic.com has IPv6 address 2607:6bc0::10

HostResolver already records the failure via setFail and the next request will avoid the broken address, but the current request — the one the user sees — propagates the network error even though a working IPv4 address is known.

`EndpointConnectionPool::prepareNewConnection` now retries on Poco::Net::NetException with a fresh address from the resolver pool, bounded by the number of distinct addresses actually tried (cap of 4). Non-network errors (SSL, config) still propagate immediately, so they are not masked.

This is a generic improvement that applies to all HTTP outbound clients (S3, URL function, AI, Iceberg/Delta REST catalogs, etc.), not just the AI integration.

Changelog category (leave one):

• Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Fix outbound HTTP requests (e.g. `aiGenerate`, `url` table function, S3) failing with `No route to host` on hosts that advertise both IPv4 and IPv6 when only one address family is routable. The HTTP connection pool now falls back to the next resolved address when the first one fails with a network error, instead of propagating the error on the very first request.

Documentation entry for user-facing changes

• Documentation is written (mandatory for new features)

---

Note

Medium Risk
Changes core HTTP connection establishment to retry across resolved IPs and tweaks resolver bookkeeping; regressions could affect outbound connectivity/error reporting, though attempts are bounded and covered by new tests.

Overview
Improves outbound HTTP robustness by retrying connection establishment against the next DNS-resolved address when the first connect attempt fails with network/timeout errors (bounded attempts, duplicate-address suppression), instead of failing the request immediately.

Adds resolver support for non-attempted addresses via HostResolver::Entry::setUnused() (prevents spurious success/failure accounting) and a test-only HostResolversPool::injectResolverForTest() to deterministically exercise multi-address retry behavior. Updates tests to cover direct retry success and to ensure proxy connect failures don’t pessimize target-host resolver state.

^{Reviewed by Cursor Bugbot for commit 03232bc. Bugbot is set up for automated code reviews on this repo. Configure here.}

Version info

• Merged into: 26.7.1.50

[clickhouse-gh]

Workflow [PR], commit [04c29e8]

Summary: ❌

job_name	test_name	status	info	comment
Stress test (amd_msan)		FAIL
	Cannot start clickhouse-server	FAIL	cidb	IGNORED
	Logical error: 'Unexpected exception in refresh scheduling' (STID: 2508-3e7b)	FAIL	cidb, issue	ISSUE EXISTS
	Check failed	FAIL	cidb	IGNORED

---

AI Review

Summary

This PR makes direct outbound HTTP connection creation retry a different resolved address after address-level connect failures, while avoiding resolver attribution for proxy and post-connect failures. I found no remaining correctness, safety, or test-evidence issues after checking the current code, prior review threads, and the current CI report.

Final Verdict

Status: ✅ Approve

[alexey-milovidov]

Pushed b0684bd2c5f..30d24e82e5a (merged master, was 6 days / 2566 commits behind; the only auto-merged overlap in HTTPConnectionPool.cpp was master's unrelated ResourceGuardSessionDataHooks change).

Two changes:

1. Fixed the failing ConnectionPoolTest.ProxyConnectSkipsTargetResolution unit test (red on every sanitizer build). Root cause: the test asserted the proxy address 127.0.0.1:1 appears in the connect-error text, but Poco::Net::SocketImpl::connect with a timeout uses a non-blocking socket and reports a refused loopback peer via SO_ERROR after poll, which calls the address-less error(int) overload → ConnectionRefusedException with message "Connection refused" and no address. The test now matches on the connection-error kind ("Connection refused", accepting the address too for the synchronous-failure path); the real invariant — the unresolvable target host name must be absent from the error — is unchanged.

2. Addressed the AI Request-changes / Bugbot finding on connect_time. The retry loop now accumulates each attempt's connect duration into a cumulative total_connect_time and writes it back on every exit (success or failure) via a scope guard, so a slow/blackholed address followed by a fast success no longer understates connect latency in S3/Azure histograms. Failed attempts are counted (a per-attempt guard fires during unwinding before the catch handlers); connect_time == nullptr callers are unaffected. RetriesNextAddressOnConnectFailure extended to assert connect_time is populated on the retry path.

The remaining performance-comparison failures (array_reduce, hash_table_sizes_stats, insert_select_squashing_variant) are unrelated noise — this PR only touches HTTP connection establishment.

[clickhouse-gh]

LLVM Coverage Report

Metric	Baseline	Current	Δ
Lines	85.30%	85.40%	+0.10%
Functions	92.50%	92.60%	+0.10%
Branches	77.60%	77.50%	-0.10%

Changed lines: Changed C/C++ lines covered by tests: 310/327 (94.80%) | Lost baseline coverage (was covered on master, now uncovered in this PR): 1 line(s) · Uncovered code

Full report · Diff report