Cut compile memory of HashJoin instantiation units by murphy-4o · Pull Request #110212 · ClickHouse/ClickHouse

murphy-4o · 2026-07-13T05:23:45Z

Motivation

Each of the 33 HashJoin instantiation translation units compiled the probe loop joinRightColumns 1488 times: 30 map types × 2 need_filter × 2 single/multi map × 2 selector types × 2 check_null_map × 3 join_mask_kind. These copies are 85% of the object code, and with sanitizers each unit peaks at ~5.5 GB of compiler memory — on a cold compiler cache (any toolchain update, e.g. #109864) ninja schedules all of them at once and the arm_fuzzers builder is OOM-killed.

Approach

Collapse the last two dimensions — check_null_map (×2) and join_mask_kind (×3), introduced in #82308 — into a single fast_path dimension (×2), chosen once per block:

fast_path = true when all join clauses have non-nullable keys and no condition in the ON section (the common case): the loop compiles without any per-row checks, as before.
All other combinations share one generic variant. Its two checks are folded into a single skip byte per row, prepared once per block by JoinOnKeyColumns::buildRowSkipData: a NULL-key row and a mask-filtered row have the same effect (the row matches nothing), so one test suffices. Without an ON-section condition the null map is used directly (no copy); with one, a single vectorizable pass merges both sources. The multi-clause (OR disjuncts) overload gets the same treatment with per-clause skip arrays.

Result

Compile improvement

	master	this PR
loop copies per heaviest unit	1488	496
compiler peak RSS (ASan)	5.54 GiB	2.7 GiB
compile time (same unit)	2:29	1:10
`clickhouse` binary	+70 MB	baseline

Join Performance

	master	this PR
fast path (plain keys, no ON condition)	1.00	1.00
nullable keys (0% / 10% NULLs)	1.00	0.93 / 0.94
ON-section mask	1.00	1.00
`join_many_default_rows` / `join_squashing_blocks` / `join_main_loop` / `join_runtime_filter`	1.00	0.84–1.00

Changelog category (leave one):

Not for changelog (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

...

Version info

Merged into: 26.7.1.1058 (included in 26.7 and later)

… dispatch dimensions Each of the 33 HashJoin instantiation translation units compiled the probe loop joinRightColumns 1488 times: 30 map types x 2 need_filter x 2 single/multi map x 2 selector types x 2 check_null_map x 3 join_mask_kind. The copies accounted for 85% of the object code (10.9 of 12.9 MB), and with sanitizers each unit peaked at ~5.5 GB of compiler memory. On a cold compiler cache (e.g. any toolchain update) ninja starts all of these units at once and the arm_fuzzers builder is OOM-killed: https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=109864&sha=b2a30564a2cd30eaaaf8014aa68e5b49ef751b9a&name_0=PR&name_1=Build%20(arm_fuzzers) Seen in ClickHouse#109864. The last two dimensions buy almost nothing at runtime: - check_null_map only elides a null-pointer test on join_keys.null_map; the runtime form is the same expression with the test kept - one perfectly predicted branch per row next to a hash table probe. - join_mask_kind only elides reading JoinMask::kind; isRowFiltered is already kind-aware, so the runtime form is two predicted compares. The AllFalse specialization compiled 2.7 MB of near-dead loops per unit for a degenerate case (ON condition constant false). Making both decisions runtime and deleting both joinRightColumnsSwitchJoinMaskKind dispatchers, measured on LeftHashJoinAll.cpp in the arm_fuzzers configuration (ASan + fuzzer): - joinRightColumns instantiations: 1488 -> 248 - compiler peak RSS: 5.54 GiB -> 1.77 GiB - compile wall time: 2:29 -> 0:49 - object file: 57.7 -> 19.0 MB Co-Authored-By: Claude <noreply@anthropic.com>

clickhouse-gh · 2026-07-13T05:24:29Z

Workflow [PR], commit [ff28dc7]

Summary: ✅

Performance Comparison: Performance dashboard

AI Review

Summary

This PR reduces HashJoin probe-loop template explosion by collapsing the null_map / join_mask specializations into one runtime skip-byte path plus a compile-time fast path for non-nullable keys without ON-section masks. I traced the current buildRowSkipData contract through continuous selectors, scattered parallel_hash selectors, and continuation chunks, then checked the final head against every existing review thread. I did not find a remaining correctness, compatibility, or unsupported-path regression in the current code.

Final Verdict

Status: ✅ Approve

rschu1ze · 2026-07-13T08:50:32Z

@m-selmi Would you like to have a look at this PR (review)?

…n mask The first CI perf run showed that fully-runtime null-map/join-mask checks regress probe-dominated joins against small (cache-resident) tables by ~20% (join_many_default_rows, join_squashing_blocks - both historically stable on master), while joins against large tables are unaffected: with ~10-cycle loop iterations the extra per-row checks are no longer hidden under the hash-probe latency. Reintroduce a single compile-time axis: fast_path, true when all join clauses have non-nullable keys and no ON-section condition (the common case). The fast variant compiles without the per-row checks, matching the codegen of the previous Kind::AllTrue/no-null-map specialization; all other cases share one generic variant with runtime checks. Compared to master this still collapses 2 (null) x 3 (mask) = 6 variants into 2: joinRightColumns instantiations per TU go 1488 -> 496 and compiler peak RSS 5.54 GiB -> 2.55 GiB (LeftHashJoinAll.cpp, ASan + fuzzer), which still resolves the arm_fuzzers cold-cache OOM. Co-Authored-By: Claude <noreply@anthropic.com>

alexey-milovidov · 2026-07-13T20:19:02Z

Your PR was poisoned by the test 02346_text_index_bug108519_qcc_skip_index. It is fixed in master. Please update the branch.

Probe-dominated joins against cache-resident hash tables are the most sensitive shape for per-row changes in the probe loop, and the Nullable-key variant of that shape had no coverage: the existing Nullable join tests (right_full_hash_join, parallel_hash_join_various) use large build sides where the per-row overhead drowns in probe cost. Requested in review. Co-Authored-By: Claude <noreply@anthropic.com>

m-selmi

The change looks reasonable to me. I just had a suggestion to try if it shows any improvement and it would be great to improve the perf test coverage.

…nt instead of five Review follow-up collapsing the runtime-checks story to its final shape: - Fold the per-row null-map and join-mask checks of the generic probe-loop variant into a single skip byte, prepared once per block by JoinOnKeyColumns::buildRowSkipData: a NULL key and a mask-filtered row have the same effect (the row matches nothing), so the loop needs one test. Without an ON-section condition the null map is used directly (no copy); with one, a single vectorizable pass merges both sources. The same treatment applies to the multi-clause (OR disjuncts) overload via per-clause skip arrays. - The mask bytes are only boolean-like (castToBoolColumn passes ColumnUInt8 through unchanged, so values > 1 are legal): invert with logical negation, not XOR. Pinned by a stateless test whose condition column holds 0/1/2 and whose reference comes from an unpatched build. - Hoist the skip pointer into a local so it stays in a register across the calls in the loop body (suggested in review). - Extend the perf test with the shapes served by the generic variant: nullable keys with and without NULLs, an ON-section mask, nullable+mask combined, OR disjuncts with nullable keys, and a parallel_hash variant. Measured against master on the worst case (5e8-row probe against a 1e3-row cache-resident build side, paired medians): nullable keys 0.93x (faster than master's specialized variant, which re-read the null-map member per row), ON-mask 1.00x, fast path and all previously regressed CI families unchanged. Compiler peak RSS for the heaviest instantiation unit stays at 2.7 GiB (1488 -> 496 loop copies vs master). Co-Authored-By: Claude <noreply@anthropic.com>

The scattered path merges the skip bytes over the source block once per shard; measured 0.94-0.96x vs master (the vectorized merge replaces master's per-row scalar mask reads), guarded here. Requested in review. Co-Authored-By: Claude <noreply@anthropic.com>

buildRowSkipData filled the merged null-and-mask bytes for the whole source block. Scattered shards and continuation chunks (a probe stopped early by max_joined_block_size_rows and resumed) visit only a subset, so the preparation repeated full-block work per shard / per chunk. Fill only the selector's positions instead: the range overload covers the probed subrange, the index overload the shard's rows; the rest of the buffer stays uninitialized and is never read. Measured on an adversarial continuation case (OR disjuncts, nullable keys plus an ON-section mask, max_joined_block_size_rows=1024 - 64x smaller than the default): full-block preparation cost +3.8% vs master, bounded preparation +2.5%; at the default cap the chunk count shrinks 64x and the difference is below noise. A continuation still re-prepares its remaining tail - fully removing that would mean carrying the buffer through the continuation, which the adversarial-only cost does not justify. The scattered case is exactly bounded now (masked parallel_hash shapes at 0.94-0.96x of master). Covered by a chunked-resume query in the stateless test (reference from an unpatched build). Co-Authored-By: Claude <noreply@anthropic.com>

The flaky check runs a new test repeatedly on debug builds with randomized settings (including max_threads=1); the previous data volume (~5M joined rows through 1024-row chunks) exceeded the 180s budget there. 42k joined rows through 256-row chunks keep ~170 continuation chunks - the path under test - at under a second per run on a release build. Verified through clickhouse-test this time, not just clickhouse local. Co-Authored-By: Claude <noreply@anthropic.com>

OR disjuncts with nullable keys and an ON-section mask, forced through 1024-row chunks: the shape where each resumed chunk re-prepares the skip bytes of its remaining tail. Measured +2.5% vs master at this 64x-smaller- than-default cap and below noise at the default; the query pins that it stays that way. Requested in review. Co-Authored-By: Claude <noreply@anthropic.com>

clickhouse-gh · 2026-07-16T15:36:06Z

LLVM Coverage Report

Metric	Baseline	Current	Δ
Lines	85.90%	85.90%	+0.00%
Functions	92.70%	91.80%	-0.90%
Branches	78.10%	78.00%	-0.10%

Changed lines: Changed C/C++ lines covered: 78/89 (87.64%) · Uncovered code

Full report · Diff report

alexey-milovidov · 2026-07-21T15:13:57Z

Build profile diff (arm_release)

Comparing 4ed98f9dc with master 08a17c7db (binary sizes, per-object sizes, per-symbol sizes, compile and link time).

⚠️ Significant changes: binary sizes, object file sizes, link time.

Binary sizes

⚠️

Binary	Master	PR	Δ
`programs/clickhouse`	4.27 GiB	3.89 GiB	-383.11 MiB (-8.77%)
`programs/clickhouse-keeper`	3.01 GiB	2.63 GiB	-382.10 MiB (-12.41%)
`programs/clickhouse-stripped`	742.76 MiB	687.92 MiB	-54.84 MiB (-7.38%)
`programs/self-extracting/clickhouse`	1.10 GiB	1.05 GiB	-51.37 MiB (-4.58%)

Object file sizes

⚠️

43 object files changed (-522.10 MiB total), 0 added, 0 removed.

Object file	Master	PR	Δ
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/FullHashJoinAll.cpp.o`	41.27 MiB	21.16 MiB	-20.11 MiB (-48.72%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/RightHashJoinAny.cpp.o`	40.42 MiB	20.99 MiB	-19.43 MiB (-48.08%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/RightHashJoinAll.cpp.o`	40.40 MiB	20.98 MiB	-19.42 MiB (-48.08%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/LeftHashJoinAll.cpp.o`	39.41 MiB	20.22 MiB	-19.19 MiB (-48.68%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/RightHashJoinSemi.cpp.o`	39.71 MiB	20.66 MiB	-19.05 MiB (-47.97%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/InnerHashJoinAll.cpp.o`	38.49 MiB	20.01 MiB	-18.48 MiB (-48.02%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/FullHashJoinAsof.cpp.o`	35.79 MiB	17.62 MiB	-18.18 MiB (-50.78%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/LeftHashJoinAsof.cpp.o`	35.79 MiB	17.62 MiB	-18.18 MiB (-50.78%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/FullHashJoinRightAny.cpp.o`	38.12 MiB	19.96 MiB	-18.16 MiB (-47.64%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/RightHashJoinRightAny.cpp.o`	37.11 MiB	19.75 MiB	-17.35 MiB (-46.77%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/LeftHashJoinRightAnyMapsAll.cpp.o`	36.25 MiB	19.05 MiB	-17.20 MiB (-47.45%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/InnerHashJoinAsof.cpp.o`	34.41 MiB	17.22 MiB	-17.19 MiB (-49.95%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/RightHashJoinAsof.cpp.o`	34.41 MiB	17.22 MiB	-17.19 MiB (-49.95%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/RightHashJoinAnti.cpp.o`	36.29 MiB	19.10 MiB	-17.18 MiB (-47.35%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/LeftHashJoinRightAny.cpp.o`	33.56 MiB	16.39 MiB	-17.17 MiB (-51.17%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/LeftHashJoinAnti.cpp.o`	32.98 MiB	15.92 MiB	-17.07 MiB (-51.74%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/LeftHashJoinAntiMapsAll.cpp.o`	35.51 MiB	18.48 MiB	-17.04 MiB (-47.97%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/InnerHashJoinAnyMapsAll.cpp.o`	36.57 MiB	19.54 MiB	-17.04 MiB (-46.58%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/InnerHashJoinAny.cpp.o`	33.83 MiB	16.89 MiB	-16.94 MiB (-50.08%)
`src/CMakeFiles/dbms.dir/Interpreters/HashJoin/FullHashJoinAnti.cpp.o`	32.69 MiB	15.81 MiB	-16.88 MiB (-51.65%)

Link time

⚠️

Binary	Master	PR	Δ
`programs/clickhouse`	2670 s	2039 s	-630.4 s (-24%)
`programs/clickhouse-keeper`	1960 s	1297 s	-663.6 s (-34%)

Compile time of recompiled translation units

149 translation units recompiled, 2298 s compile time in total, 149 of them have a recent master baseline.

Median compile-time ratio to the baselines is ×1.29 (machine-speed difference or a change affecting every TU); per-TU deltas below are relative to that ratio.

Demonstration of the new Build profile diff check from #111164, run manually on the data of this PR's merge commit (4ed98f9dc535) vs the preceding master build (08a17c7dbef2).

clickhouse-gh Bot added the pr-build Pull request with build/testing/packaging improvement label Jul 13, 2026

murphy-4o changed the title ~~Cut compile memory of HashJoin instantiation units 3x by removing two dispatch dimensions~~ Cut compile memory of HashJoin instantiation units Jul 13, 2026

clickhouse-gh Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread src/Interpreters/HashJoin/HashJoinMethodsImpl.h Outdated

rschu1ze assigned m-selmi Jul 13, 2026

murphy-4o changed the title ~~Cut compile memory of HashJoin instantiation units~~ Cut compile memory of HashJoin instantiation units 2x by collapsing probe-loop dispatch into a fast path Jul 13, 2026

murphy-4o changed the title ~~Cut compile memory of HashJoin instantiation units 2x by collapsing probe-loop dispatch into a fast path~~ Cut compile memory of HashJoin instantiation units Jul 13, 2026

Merge branch 'master' into hashjoin-reduce-instantiations

73a3a70

murphy-4o marked this pull request as ready for review July 14, 2026 03:03

clickhouse-gh Bot added pr-not-for-changelog This PR should not be mentioned in the changelog and removed pr-build Pull request with build/testing/packaging improvement labels Jul 14, 2026

clickhouse-gh Bot reviewed Jul 14, 2026

View reviewed changes

Comment thread src/Interpreters/HashJoin/HashJoinMethodsImpl.h

m-selmi approved these changes Jul 15, 2026

View reviewed changes

Comment thread src/Interpreters/HashJoin/HashJoinMethodsImpl.h Outdated

Comment thread tests/performance/join_nullable_keys_small_table.xml

murphy-4o requested review from nickitat and removed request for nickitat July 16, 2026 02:20

murphy-4o and others added 2 commits July 16, 2026 10:30

Merge branch 'master' into hashjoin-reduce-instantiations

4930527

clickhouse-gh Bot reviewed Jul 16, 2026

View reviewed changes

Comment thread src/Interpreters/HashJoin/AddedColumns.cpp Outdated

clickhouse-gh Bot reviewed Jul 16, 2026

View reviewed changes

Comment thread src/Interpreters/HashJoin/AddedColumns.cpp Outdated

murphy-4o and others added 2 commits July 16, 2026 14:58

m-selmi reviewed Jul 16, 2026

View reviewed changes

Comment thread src/Interpreters/HashJoin/AddedColumns.cpp

murphy-4o added this pull request to the merge queue Jul 17, 2026

Merged via the queue into ClickHouse:master with commit 4ed98f9 Jul 17, 2026
178 checks passed

murphy-4o deleted the hashjoin-reduce-instantiations branch July 17, 2026 01:53

robot-clickhouse-ci-1 added the pr-synced-to-cloud The PR is synced to the cloud repo label Jul 17, 2026

This was referenced Jul 21, 2026

Execute mixed-type-pair comparison and arithmetic kernels via conversion to a common type #110131

Merged

Add "Build profile diff" check: compare PR build profile against master #111164

Merged

Navigation Menu

Uh oh!

Cut compile memory of HashJoin instantiation units - #110212

Cut compile memory of HashJoin instantiation units#110212
murphy-4o merged 10 commits into
ClickHouse:masterfrom
murphy-4o:hashjoin-reduce-instantiations

murphy-4o commented Jul 13, 2026 •

edited by robot-clickhouse-ci-2

Loading

Uh oh!

clickhouse-gh Bot commented Jul 13, 2026 •

edited

Loading

Uh oh!

Uh oh!

rschu1ze commented Jul 13, 2026

Uh oh!

alexey-milovidov commented Jul 13, 2026

Uh oh!

Uh oh!

m-selmi left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

clickhouse-gh Bot commented Jul 16, 2026

Uh oh!

Uh oh!

alexey-milovidov commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

murphy-4o commented Jul 13, 2026 • edited by robot-clickhouse-ci-2 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Approach

Result

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Version info

Uh oh!

clickhouse-gh Bot commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

AI Review

Summary

Final Verdict

Uh oh!

Uh oh!

rschu1ze commented Jul 13, 2026

Uh oh!

alexey-milovidov commented Jul 13, 2026

Uh oh!

Uh oh!

m-selmi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

clickhouse-gh Bot commented Jul 16, 2026

LLVM Coverage Report

Uh oh!

Uh oh!

alexey-milovidov commented Jul 21, 2026

Build profile diff (arm_release)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

murphy-4o commented Jul 13, 2026 •

edited by robot-clickhouse-ci-2

Loading

clickhouse-gh Bot commented Jul 13, 2026 •

edited

Loading