[fix](fe) Fix struct field slot type in NestedColumnPruning for OFFSET-only access - #62446
Merged
Merged
Conversation
…T-only access Issue Number: close #xxx Problem Summary: When using struct_element() to access a string field inside a struct (e.g., length(struct_element(struct_col, 'f3'))), the NestedColumnPruning rule incorrectly returned Optional.empty() for the slot type, causing the slot type to become nullable and lose its original type information. The fix changes the return value from Optional.empty() to Optional.of(type) when the column is accessed in OFFSET-only mode (e.g., length()), ensuring the slot type remains the original type (e.g., varchar). Release note: None Test: Added test case in string_length_column_pruning.groovy
Contributor
Author
|
run buildall |
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
FE UT Coverage ReportIncrement line coverage |
englefly
approved these changes
Apr 13, 2026
github-actions
Bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
Gabriel39
approved these changes
Apr 14, 2026
yiguolei
pushed a commit
that referenced
this pull request
Sep 21, 2026
… path series (#68214) Cherry-pick of #62205, #62315, #62631, #62304, #63229, #63736, #64486 to branch-4.2 (the meta path series, applied in merge order). Adaptations for branch-4.2: * `AccessPathInfo` already lives in `org.apache.doris.analysis` on this branch; the new `ACCESS_STRING_OFFSET` constant was ported there (aliasing the existing `ACCESS_OFFSET`). * master-only types renamed to this branch's thrift types (`ColumnAccessPath* -> TColumnAccessPath`, `ColumnAccessPathType -> TAccessPathType`); `NestedColumnPruning` keeps branch-4.2's `comparePathSegments`/variant branch alongside upstream's new logic. * `LazyMaterializeTopN` merges upstream's restructure while preserving branch-4.2's Lance lazy-materialization gate; `LogicalJoin.getRightConditionSlot()` (a master-only helper) had to be added for the ported `PushDownTopNThroughJoin`. * `StringEmptyToLengthRuleTest` uses `ElementAt` (branch-4.2 folded `StructElement` into `ElementAt`); the regression suite expectation for the same reason. REQUIRED EXTRA COMMIT: the FE now emits offset-only / null-only nested access paths, which branch-4.2's BE could not read — `null_column_pruning.groovy` aborted the BE with `std::vector<...ColumnIterator>::operator[]: Assertion '__n < this->size()' failed`. This PR therefore also cherry-picks the BE-side prerequisite #61888 "[Exec](be) Support offset prue column and null column in BE" (adapted to branch-4.2's `IColumn::mutate`/Defer COW shape and `ColumnNullable::get_null_map_column_ptr()` signature). Please review that extra commit carefully. Testing on branch-4.2 (FE + BE rebuilt from this branch): * FE unit tests: PruneNestedColumnTest (57), PullUpProjectExprUnderTopNTest (28), OperativeColumnDeriveTest (4), StringEmptyToLengthRuleTest (11), MaterializeProbeVisitorTest (8) — 108 tests, 0 failures. * Regression: nereids_rules_p0/column_pruning (7 suites incl. the new string_length/null/nested_container_offset/topn_expr_pullup/topn_lazy_nested suites) 0 failures; shape_check clickbench/tpcds_sf100/tpcds_sf1000/tpcds_sf10t_orc/tpcds_sf1000_constraints suites touched by the series regenerated with -forceGenOut and green; nereids_rules_p0 push_filter_through / null_un_safe_equals / lazy_materialize_topn green. * Known deviations: #62446 (struct field slot type for OFFSET-only access) is not part of the requested series and was not included; the BE-side `MapAccessAllWithOffsetDoesNotPropagateOffsetToKey` unit test from #61888 was kept byte-identical to upstream although it may not hold (test binary not built with MAKE_TEST=ON). --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: HappenLee <happenlee@selectdb.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem Summary: When using struct_element() to access a string field inside a struct (e.g., length(struct_element(struct_col, 'f3'))), the NestedColumnPruning rule incorrectly returned Optional.empty() for the slot type, causing the slot type to become nullable and lose its original type information.
The fix changes the return value from Optional.empty() to Optional.of(type) when the column is accessed in OFFSET-only mode (e.g., length()), ensuring the slot type remains the original type (e.g., varchar).
Release note: None
Test: Added test case in string_length_column_pruning.groovy