branch-4.1: [fix](multi-catalog) Preserve external partition metadata - #66012
Open
suxiaogang223 wants to merge 2 commits into
Open
branch-4.1: [fix](multi-catalog) Preserve external partition metadata#66012suxiaogang223 wants to merge 2 commits into
suxiaogang223 wants to merge 2 commits into
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Member
Author
|
run buildall |
Contributor
FE UT Coverage ReportIncrement line coverage |
Contributor
FE Regression Coverage ReportIncrement line coverage |
suxiaogang223
marked this pull request as ready for review
July 27, 2026 02:01
suxiaogang223
force-pushed
the
codex/fix-4.1-per-range-partition-values
branch
from
July 30, 2026 06:27
8d7da1a to
19633ca
Compare
Backport the remaining external partition fixes to branch-4.1 while adapting them to its FileScannerV2 infrastructure. Preserve explicit NULL metadata for Hive-style scans, Hudi, and load paths, and make Iceberg identity partition metadata stable and safe across partition spec evolution.
suxiaogang223
force-pushed
the
codex/fix-4.1-per-range-partition-values
branch
from
July 30, 2026 06:32
19633ca to
eb977fc
Compare
Member
Author
|
run buildall |
Contributor
FE Regression Coverage ReportIncrement line coverage |
Member
Author
|
run buildall |
Contributor
FE UT Coverage ReportIncrement line coverage |
Contributor
FE Regression Coverage ReportIncrement line coverage |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Related PR: #62821, #65581, #66010
Problem Summary: branch-4.1 already backported the Paimon and BE partition metadata changes through #65581, but Hive-style scans, Hudi, Iceberg, and file load paths still had incomplete FE metadata propagation. Hive default partitions were encoded as
\N, which lost the distinction between NULL and a literal\N. Iceberg attached partition metadata only when runtime partition pruning was enabled, used unstable per-map ordering, and did not safely handle files written with different partition specs.This change:
\Nas data while representing Hive default partitions as NULL;Release note
Fix partition-column materialization, NULL handling, and Iceberg partition evolution compatibility for external table scans and file loads on branch-4.1.
Check List (For Author)
./run-fe-ut.sh --run org.apache.doris.common.util.BrokerUtilTest,org.apache.doris.datasource.hive.source.HiveScanNodeTest,org.apache.doris.datasource.iceberg.IcebergUtilsTest,org.apache.doris.datasource.iceberg.source.IcebergScanNodeTest,org.apache.doris.datasource.paimon.source.PaimonScanNodeTest