AST fuzzer: do not inject expressions into data-type argument lists - #109713
Conversation
|
Workflow [PR], commit [58f118e] Summary: ✅
AI ReviewSummaryThis PR fixes the Findings
Final VerdictNot ready to clear as-is. I did not find any new unthreaded issues worth posting as fresh inline comments, but the existing explicit- LLVM Coverage Report
Changed lines: Changed C/C++ lines covered: 271/383 (70.76%) · Uncovered code |
The AST fuzzer recursed into an ASTDataType's argument list with the generic expression fuzzer, which wraps children in expression functions (multiply(), if(), multiIf(), ...). That produced an ASTDataType whose argument is an ASTFunction, a node ParserDataType can never produce. ASTDataType::formatImpl then emitted text (e.g. Nullable(1 = 2)) that parses back into a different AST, tripping the format-parse-format consistency check in executeQuery (LOGICAL_ERROR "Inconsistent AST formatting", which aborts on DEBUG and sanitizer builds), reached through the server-side AST fuzzer (executeASTFuzzerQueries -> BlockIO::onFinish -> TCPHandler::runImpl). Make the ASTDataType branch own the node unconditionally and only fuzz it via the DataType layer (fuzzDataType), which already produces structurally valid, round-trippable mutations and recurses into nested types. The argument list is never handed to the generic expression fuzzer. This fixes the root cause at the fuzzer, leaving the parser/formatter unchanged (so Dynamic/JSON parameter arguments like Dynamic(max_types = 5), which use the '=' operator form, keep round-tripping as before). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1e60a39 to
47cfb55
Compare
|
Replaced the previous approach (formatter change in Pre-PR validation gate (click to expand)
Session id: cron:clickhouse-worker-slot-1:20260708-081100 |
|
cc @PedroTadim @alexey-milovidov — could you review? This is a root-cause fix in the AST fuzzer for #109706: |
…tic ASTDataType round-trip unit test fuzzDataType had structured mutation arms for most parametric types but not DataTypeObject (JSON/Object) or DataTypeQBit, so after making ASTDataType opaque to the generic walk their parameters and typed-path types stopped being fuzzed. Add a DataTypeObject arm that recursively fuzzes typed-path element types and the max_dynamic_paths/max_dynamic_types parameters (keeping SKIP paths/regexps intact so the result stays round-trippable), and a QBit arm that mutates element type and dimension within the type's own bounds. Add a deterministic gtest around the malformed ASTDataType round-trip so the fix evidence no longer depends on the global server-side fuzzer RNG: an ASTDataType whose argument is an ASTFunction is not round-trippable through ParserDataType, while structurally valid types (including JSON and QBit) are. Related: ClickHouse#109706 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Head branch was pushed to by a user without write access
|
Addressed both automated review comments in 9a7cfad. CR#1 (Object/JSON coverage regression in CR#2 (probabilistic test). Correct — the server-side fuzzer is seeded from Verified locally: the new gtest passes; a debug server running 100 fuzzer iterations over the JSON+QBit CREATE stays alive with no LOGICAL_ERROR / Inconsistent-AST. Pre-PR validation gate (click to expand)
Session id: cron:clickhouse-worker-slot-5:20260708-095600 |
…t-format-datatype-arg-109706
…it strides Master added a stride parameter to DataTypeQBit (element_type, dimension, stride), so the two-argument form in fuzzDataType no longer compiles. Build strided/non-strided QBits with valid (dimension, stride) pairs and include Int8 in the element-type set, matching the getRandomType arm. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fuzzDataType made every ASTDataType opaque to the generic walk, but it had no structured arm for SimpleAggregateFunction (a custom-named type). After that change its aggregate name, parameters, and nested argument types stopped being fuzzed. Add a SimpleAggregateFunction arm that recursively fuzzes the argument types and re-validates the aggregate via the factory, keeping the name and parameters, so the result stays a round-trippable type with no generic-expression injection. The arm runs before the structural Array/Tuple/Nullable/... arms so the custom name is not stripped. Extend the deterministic gtest and the 04344 smoke test with SimpleAggregateFunction forms. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Addressed in da06be2. Added a structured Gate f/g audit of the remaining parametric families: plain Test: extended the deterministic gtest Verified locally (debug, Build ID B299296CE28D5B8B1330553B382A2C22974AD388): clickhouse + unit_tests_dbms built clean, the gtest passes, and 500 Pre-PR validation gate (click to expand)
Session id: cron:clickhouse-worker-slot-4:20260708-122900 |
| /// QBit only accepts Int8/BFloat16/Float32/Float64 element types and a (dimension, stride) pair | ||
| /// where dimension % stride == 0 and, when actually strided, stride % 8 == 0. We mutate within | ||
| /// those bounds rather than handing the argument list to the generic fuzzer. | ||
| static const DataTypePtr qbit_element_types[] |
There was a problem hiding this comment.
I think this code could be shared with random type generation one?
There was a problem hiding this comment.
Done in 53a8aa7. The genuinely duplicated part was the QBit construction (element-type set + dimension/stride constraints), which was near-identical in both functions. Extracted it into a makeRandomQBit() helper called by both fuzzDataType and getRandomType, so the stride rules live in one place (-45/+31 net).
I kept the other arms separate on purpose, since they do different things:
- Object/JSON:
getRandomTypebuilds a bareDataTypeObject(JSON)with no params; the fuzz arm mutates the typed-path element types + max_dynamic_paths/types of an existing node while preserving its SKIP paths. There is no shared construction to factor. - (Simple)AggregateFunction:
getRandomTypesynthesizes from scratch (with acountfallback on rejection); the fuzz arms mutate an existing aggregate, preserving its name/parameters and only re-validating fuzzed argument types. Folding these into one helper would obscure the generate-vs-mutate distinction rather than reduce real duplication.
Build + gtest + the fuzz smoke (JSON/QBit/SimpleAggregateFunction CREATE, 500 runs, server stays alive) all pass.
…mType Extract the duplicated QBit-building logic (element type + dimension/stride constraints) into a single makeRandomQBit helper used by both fuzzDataType and getRandomType, so the stride constraints live in one place. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
| /// Fuzz the typed-path element types (recursively) and the numeric parameters, keeping the | ||
| /// SKIP paths / SKIP REGEXP lists intact so the result stays a structurally valid, round-trippable | ||
| /// DataTypeObject (its getName() re-parses via ParserDataType to the same type). | ||
| std::unordered_map<String, DataTypePtr> typed_paths = type_object->getTypedPaths(); |
There was a problem hiding this comment.
Done in f6966fc. Extracted makeRandomObject() (builds a JSON Object with randomized max_dynamic_paths/max_dynamic_types within the parser limits); both getRandomType (empty paths) and the fuzzDataType Object arm (fuzzed typed paths + preserved SKIP lists) call it now.
| if (type->hasCustomName()) | ||
| { | ||
| if (const auto * type_simple_aggr | ||
| = typeid_cast<const DataTypeCustomSimpleAggregateFunction *>(type->getCustomName()); |
There was a problem hiding this comment.
Done in f6966fc. Extracted makeAggregateFunctionType(name, args, params, simple) which runs the factory get + build (and, for the simple form, checkSupportedFunctions) and returns nullptr if the aggregate rejects the args. Shared by three sites now: getRandomType AggregateFunction case, and the fuzzDataType AggregateFunction and SimpleAggregateFunction arms.
Requested in review: keep the data-type machinery separate from the rest of the fuzzer so it can be reviewed on its own. Pure code motion. fuzzDataType, getRandomType and their nine helpers move from QueryFuzzer.cpp to the new QueryFuzzerDataTypes.cpp, together with the two function-local tables and the timezone list they use. QueryFuzzer.cpp keeps the guard in fuzz() that this PR is actually about, so the fix and the type machinery are no longer read as one block. src/Common is globbed by add_headers_and_sources, so the new translation unit needs no CMake change. The only edit that is not a move: swapAggrs is pre-existing code in QueryFuzzer.cpp used by both units now, so it is reachable through a swapAggregateNames accessor, in the same style as the geoAliasNames accessor this PR already added. Its definition and the fuzz() use of it are untouched. Three includes this PR had added to QueryFuzzer.cpp moved with their code and are dropped from it. Verified behaviour preserving rather than assumed: cityHash64(groupArray(query)) over seeded fuzzQuery output for three fixtures covering the argument bearing types is identical before and after the move. Perturbing makeRandomDateTime inside the moved region changes one of the three hashes, so the comparison is discriminating and not vacuous.
CI finish ledger - f46aa21Every failure below has an owner: a fixing PR (ours or external), or a full-effort fix task
All 174 check-runs on this commit are completed; this is the only failing check. Not caused by this PR. This PR touches Onset is sharp: a 60-day CIDB sweep keyed on the Session id: cron:our-pr-ci-monitor:20260729-223000 |
|
Fixing PR for the plain_rewritable That is a revert of #111883 by @ alexey-milovidov, merged 2026-07-30T09:07:30Z, which restores the The race is verified gone on trunk, not just merged: keyed on the This head predates the revert, so the red here clears once the branch picks up master. One correction to my earlier ledger note on the onset: I flagged two possible pre-#111883 hits |
…09706 The only conflict was src/Common/QueryFuzzer.cpp. This branch moved the data-type fuzzing (fuzzDataType, getRandomType and their helpers) into the new src/Common/QueryFuzzerDataTypes.cpp, so master's change to getRandomType's Map case presented as a delete/modify conflict rather than overlapping text. Master's new guard (03fdf14, "Query fuzzer: do not compose a Map with an invalid key type") is carried over verbatim into its new home in QueryFuzzerDataTypes.cpp, so the Nullable/LowCardinality(Nullable) Map key rejection is preserved. The pre-existing fuzzDataType Map guard is a distinct site and was already present in the merge base. The other seven master hunks land in QueryFuzzer.cpp unchanged: the DateLUT include, the ASTStreamSettings cursor field rename with its buildCursorTree calls, and fake_time_for_view switching to a formatted timestamp.
The branch had to enumerate every ASTDataType subclass by hand because typeid_cast matches the exact type only, so a subclass missing from the list would fall through to the generic expression fuzzer and reintroduce the invalid argument lists of ClickHouse#109706. dynamic_cast covers the whole hierarchy, so the condition is one cast instead of three and a future subclass cannot silently escape it. Behaviour is unchanged: ASTDataType, ASTTupleDataType and ASTEnumDataType are the only subclasses today.
CI finish ledger — d0c97dfCI is fully finished on this head (174 check-runs, 0 incomplete, Nothing to attribute, so there is no owner column to fill. Session id: cron:our-pr-ci-monitor:20260801-180000 |
LLVM Coverage Report
Changed lines: Changed C/C++ lines covered: 340/355 (95.77%) · Uncovered code |
CI finish ledger for 347be61Every failure below has an owner: a fixing PR, or a full-effort fix task whose fixing-PR link will be posted here when it opens.
174 of 174 check-runs completed, Not caused by this diff, which stops the AST fuzzer injecting expressions into data-type arguments. Breadth over 30 days on the message is 48 rows across 14 checks with master rows on 8 of them, so it is reachable on trunk independently of this branch. Session id: cron:our-pr-ci-monitor:20260802-010000 |
# Conflicts: # src/Common/QueryFuzzer.cpp
Making ASTDataType opaque to the generic expression fuzzer removed the only mutation path that used to reach the SKIP clauses of a JSON type: the skipped path is a compound identifier and the skipped regexp a string literal, both of which the generic walk mutated on its way down (the ASTLiteral arm rewrites literal->value via fuzzField). The DataTypeObject arm that took ownership re-fuzzed the typed-path types and the numeric parameters but carried both SKIP collections through verbatim, so JSON(SKIP a, SKIP REGEXP 're') stopped being mutated at all. Add a structured mutation for each collection. Replacements are drawn from a fixed table so they stay inside the grammar the type requires: identifier-shaped values for a skipped path, RE2-compilable ones for a skipped regexp (the DataTypeObject constructor throws CANNOT_COMPILE_REGEXP otherwise). This keeps the result round-trippable, which is the property this PR exists to preserve -- mutating into something ParserDataType cannot re-parse is what produced the inconsistent-AST-formatting abort in the first place. The paths_to_skip set is iterated in unspecified order, so it is sorted before an element is picked to keep a run reproducible for a given seed. A mutated skipped path can collide with a typed path prefix, which the constructor rejects; that throw is already absorbed by the enclosing try/catch, which falls back to the unmutated type. Measured over 12 seeds of the fuzzQuery table function on JSON(a.b UInt32, SKIP zzz, SKIP REGEXP '^qqq.*$'): 80 JSON rows, both clauses mutated (8 distinct regexps, 6 distinct paths) while the originals still occur, confirming the mutation fires without displacing the input shape.
CI finish ledger - 0ae0f0fEvery failure below has an owner: a fixing PR (mine or external), or a full-effort fix task
175/175 check-runs completed; On the first row: the stack sits in Session id: cron:our-pr-ci-monitor:20260803-163000 |
The fixture carried a SKIP path but no SKIP REGEXP, so fuzzObjectPathRegexpsToSkip was never reached for this input and the new mutation had no coverage here. Note this makes the clause reachable; it does not make the test a discriminating oracle for it. An unsafe replacement cannot be caught by this test by construction: DataTypeObject validates the regexp in its constructor, and the enclosing catch in the DataTypeObject arm falls back to the unmutated type, so a bad candidate degrades coverage rather than emitting an unparseable type. Verified by injecting an RE2-invalid candidate and confirming it never reaches the output (0 of 83 generated JSON types) while the test stays green.
CI finish ledger - 4aa045aCI is fully finished on this head: 175 check-runs, 0 incomplete,
Not caused by this PR. The abort is The corpus query confirms the match: my failing run carries #113114 is open and is not an ancestor of this head ( Session id: cron:our-pr-ci-monitor:20260804-003000 |
Requested by the reviewer: the test is not needed. The fuzzer fix in src/Common/QueryFuzzer.cpp and src/Common/QueryFuzzerDataTypes.cpp is unchanged; only the stateless test and its reference file are dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
makeRandomObject is used both to synthesize a fresh JSON type and to rebuild a mutated one. In the mutate path the 25% randomization of max_dynamic_paths / max_dynamic_types fell back to the compile-time defaults rather than to the source type's own values, so a source such as JSON(max_dynamic_paths=0, max_dynamic_types=1, p UInt32) lost that regime on most visits even when only a typed path or a SKIP clause was being mutated. Since ASTDataType is opaque to the generic expression fuzzer, this helper is the only remaining mutation path for those parameters, so the regime was simply unreachable. Thread the source limits through as optional parameters: an unfired randomization now keeps them, a fired one still explores the whole range, and the generate path passes nothing and keeps using the defaults. This mirrors how makeAggregateFunctionType already threads the parsed serialization version. Measured over 12 seeds against JSON(max_dynamic_paths=0, max_dynamic_types=1, p UInt32), counting emitted JSON types (doGetName only prints a limit clause when it differs from the default, so a reset limit is visible as the clause disappearing): before: 86 types, max_dynamic_paths reset 59, max_dynamic_types reset 58 after: 50 types, max_dynamic_paths reset 0, max_dynamic_types reset 1 The single remaining reset is the randomization legitimately drawing the default value. All 23 distinct emitted JSON types reparse to themselves.
CI finish ledger - f2e9025Every failure below has an owner: a fixing PR (mine or external), or a full-effort fix task
Not caused by this PR. The diff only changes how the AST fuzzer mutates Session id: cron:our-pr-ci-monitor:20260804-173000 |
CI finish ledger - 58f118eEvery failure below has an owner: a fixing PR (mine or external), or a full-effort fix task whose
All 174 check-runs completed and Session id: cron:our-pr-ci-monitor:20260805-230000 |
Master commit 7cc0440 (AST-fuzzer follow-up of ClickHouse#109713) added its own `getVersionIfExplicit` to `DataTypeAggregateFunction`; this branch already provides an identical definition. Keep a single one.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
...
Description
Closes: #109706
Root cause (fuzzer, not parser/formatter). The AST fuzzer recursed into an
ASTDataType's argument list with the generic expression fuzzer (QueryFuzzer::fuzzExpressionList), which wraps children in expression functions (multiply(),if(),multiIf(), ...). That produced anASTDataTypewhose argument is anASTFunction(e.g.Nullable(1 = 2)), a nodeParserDataTypecan never produce.ASTDataType::formatImplthen emitted text thatParserDataTypeparses back into a different AST, tripping the format-parse-format consistency check inexecuteQueryImpl(LOGICAL_ERROR"Inconsistent AST formatting", which is an exception that aborts on DEBUG/sanitizer builds). It surfaced through the server-side AST fuzzer:executeASTFuzzerQueries->BlockIO::onFinish->TCPHandler::runImpl.Crash-log AST (STID 1941-26fa):
Fix. Make the
ASTDataTypebranch inQueryFuzzer::fuzzown the node unconditionally and only fuzz it via the DataType layer (fuzzDataType), which already produces structurally valid, round-trippable mutations and recurses into nested types. The argument list is never handed to the generic expression fuzzer. The parser and formatter are left unchanged, so legitimate data-type argument syntax that uses the=operator (e.g.Dynamic(max_types = 5),JSON(max_dynamic_paths = 8),Enum8('a' = 1)) keeps round-tripping exactly as before.Owner. This is a fuzzer defect (
src/Common/QueryFuzzer.cpp), notcomp-query-execution. Thecomp-query-executionclassifier routed the issue to @ davenger because the abort happens in the execution finish callback, but the invalid AST is produced by the fuzzer.Testing.
CREATEcovering the argument-bearing types (aborted before the fix, clean after it).allow_operators = falseinASTDataType::formatImpl; that broke ~26Dynamic/JSONtests (theirparam = valuearguments stopped round-tripping) and introduced a new abort forNullable(1 = 2). Replaced by this fuzzer-level fix. Re-ran the affectedDynamic/JSONstateless tests locally: all pass.CI report for the reported failure: https://s3.amazonaws.com/clickhouse-test-reports/PRs/104424/75c2481d0eb01e855ab2a22561858ee1a6eb6c68/ast_fuzzer_amd_debug_targeted/fatal.log