branch-4.1: [opt](schema) Optimize tables schema scan status #64933 - #65631
Open
0AyanamiRei wants to merge 4 commits into
Open
branch-4.1: [opt](schema) Optimize tables schema scan status #64933#656310AyanamiRei wants to merge 4 commits into
0AyanamiRei wants to merge 4 commits into
Conversation
### What problem does this PR solve? Issue Number: None Related PR: apache#64933 Problem Summary: Backport the information_schema.tables column-pruning and single-pass table-status statistics optimization to branch-4.1. Resolve the test import conflict by retaining branch-4.1's JMockit ExecuteEnv setup while adding the Guava Sets dependency used by the new status-pruning tests. All production and test hunks from the source PR are preserved. ### Release note None ### Check List (For Author) - Test: No need to test (this one-time conflict-resolution workflow excludes builds and test execution; C++ format checks and git diff --check passed) - Behavior changed: No - Does this need documentation: No (cherry picked from commit 058dfd3)
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
Author
|
run buildall |
Contributor
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
FE UT Coverage ReportIncrement line coverage |
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
Author
|
run buildall |
Contributor
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
FE Regression Coverage ReportIncrement line coverage |
yiguolei
approved these changes
Jul 30, 2026
github-actions
Bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
Contributor
Author
|
run buildall |
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
|
/review |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Review complete. I did not find a blocking correctness issue in the current head, so my opinion is approval/no changes requested.
Critical checkpoint conclusions:
- Goal and scope: The backport is focused on
information_schema.tablescolumn pruning and single-pass table-status statistics. Required scan slots are propagated through the BE/Thrift/FE path, while OLAP status values are collected in one tablet traversal. - Correctness: Projection and predicate slots are retained in the propagated set; optional status fields map to the fields the BE reads. The prior
CHECK_TIMEaliasing concern is fixed by settinglast_check_timewhenCHECK_TIMEis requested and is covered by a direct test. - Statistics semantics: Row counts, NORMAL-replica filtering for average size, all-replica local/remote segment and inverted-index totals, zero/unknown handling, and table read locking preserve the prior behavior for local and cloud tablets. Other table kinds retain their existing getters through the default aggregate wrapper.
- Compatibility and lifecycle:
required_columnsis an optional new Thrift field. Unset callers keep the full legacy response, and old/new FE-BE combinations safely ignore the unknown optional field or fall back to full status. Required columns are finalized during operator prepare before scanner-local reads; no new shared mutable lifecycle or lock-order issue was found. - Parallel and special paths: Full-status callers, empty required sets, mixed-case column names, views/external tables, local/cloud tablets, abnormal replicas, predicate-only slots, and
CHECK_TIMEwere traced. No distinct issue survived. - Tests and validation: The changed FE tests cover pruning, empty/mixed-case sets,
CHECK_TIME, combined status collection, and replica filtering; the regression suite covers planner-path tuple pruning. The review prompt forbids local builds/tests, so none were run here. Live checks at this head pass compile, FE UT, BE UT, P0, cloud/vault, external regression, and coverage. The lone non-concurrent failure is an unrelated routine-load cleanup failure caused by a still-completing committed transaction; it does not exercise a changed schema-status path. - User focus: No additional review focus was provided; the entire PR was reviewed.
- Convergence: Two normal full-coverage reviewers and one separate risk-focused reviewer all returned
NO_NEW_VALUABLE_FINDINGSin round 1 for the same frozen head, empty existing-thread set, and empty inline comment set. The main-agent final sweep found no unresolved candidate.
Review-environment note: the repository code-review skill referenced by the bundle was absent from both the checkout and HEAD, and no changed-path ancestor AGENTS.md files apply. I followed the complete bundle workflow and preserved this limitation explicitly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Related PR: #64933
Problem Summary: Backport the
information_schema.tablescolumn-pruning and single-pass table-status statistics optimization from #64933 tobranch-4.1.Source merge commit:
058dfd3794aca46617766173ced5ac5574efff94.Conflict resolution
FrontendServiceImplTest.branch-4.1's JMockit@Mocked ExecuteEnvdependency and added the GuavaSetsimport required by the new mixed-case required-column test.The resulting backport retains the source behavior: lightweight projections skip tablet-derived status work, while full status queries compute row/data/index statistics in one tablet traversal.
Release note
None
Check List (For Author)
build-support/clang-format.shbuild-support/check-format.shgit diff HEAD^ HEAD --check