[fix](cloud) checkpoint save cloud tablet stats to image - #60705
Merged
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates FE checkpoint/image generation for cloud mode so that the saved image includes cloud-specific runtime metadata (table/partition versions and tablet/replica stats), reducing reliance on rebuilding those values after restart.
Changes:
- Add a cloud-mode post-processing step during checkpoint generation to copy table/partition versions and replica stats from the serving env into the checkpoint env before saving the image.
- Persist additional cloud metadata by adding Gson
@SerializedNameannotations (e.g., table cached version, replica rowset/segment counts). - Make
OlapTable.setCachedTableVersion()callable from checkpoint code.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
fe/fe-core/src/main/java/org/apache/doris/master/Checkpoint.java |
Adds postProcessCloudMetadata() to copy versions and tablet stats into the checkpoint catalog before saveImage(). |
fe/fe-core/src/main/java/org/apache/doris/cloud/catalog/CloudReplica.java |
Persists segmentCount and rowsetCount into image via @SerializedName. |
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java |
Persists cached table version via @SerializedName and exposes setter for checkpoint to populate it. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 30543 ms |
TPC-DS: Total hot run time: 189007 ms |
ClickBench: Total hot run time: 28.85 s |
Contributor
FE Regression Coverage ReportIncrement line coverage |
mymeiyi
force-pushed
the
checkpoint-save-cloud-meta
branch
2 times, most recently
from
February 13, 2026 09:36
7c0af96 to
f7bf894
Compare
Contributor
Author
|
run buildall |
Contributor
FE Regression Coverage ReportIncrement line coverage |
mymeiyi
force-pushed
the
checkpoint-save-cloud-meta
branch
from
February 24, 2026 04:08
f7bf894 to
66cffc1
Compare
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 28693 ms |
TPC-DS: Total hot run time: 183259 ms |
Contributor
FE UT Coverage ReportIncrement line coverage |
Contributor
FE Regression Coverage ReportIncrement line coverage |
gavinchou
previously approved these changes
Feb 26, 2026
github-actions
Bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
mymeiyi
force-pushed
the
checkpoint-save-cloud-meta
branch
from
February 27, 2026 02:56
66cffc1 to
c91c38a
Compare
github-actions
Bot
removed
the
approved
mymeiyi
force-pushed
the
checkpoint-save-cloud-meta
branch
from
March 19, 2026 03:52
d3a9bc8 to
007ddd6
Compare
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 27044 ms |
TPC-DS: Total hot run time: 169627 ms |
Contributor
FE UT Coverage ReportIncrement line coverage |
gavinchou
approved these changes
Mar 19, 2026
github-actions
Bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
Author
|
run p0 |
gavinchou
approved these changes
Mar 20, 2026
gavinchou
self-requested a review
March 20, 2026 18:47
mymeiyi
added a commit
to mymeiyi/doris
that referenced
this pull request
Mar 30, 2026
mymeiyi
added a commit
to mymeiyi/doris
that referenced
this pull request
Mar 30, 2026
yiguolei
pushed a commit
that referenced
this pull request
Mar 31, 2026
HappenLee
pushed a commit
to HappenLee/incubator-doris
that referenced
this pull request
Mar 31, 2026
mymeiyi
added a commit
to mymeiyi/doris
that referenced
this pull request
Apr 15, 2026
mymeiyi
added a commit
to mymeiyi/doris
that referenced
this pull request
May 9, 2026
mymeiyi
added a commit
to mymeiyi/doris
that referenced
this pull request
Jul 29, 2026
### What problem does this PR solve? Issue Number: None Related PR: apache#60705, apache#66154 Problem Summary: Periodically regenerating a cloud checkpoint without a newer finalized journal reused the same image sequence. That exposed checkpoint version mismatches, same-version image publication, and follower synchronization ambiguity. Stop triggering checkpoints solely because an image is stale while retaining cloud metadata persistence during normal checkpoints. Do not persist tablet-stat refresh scheduling state so an FE can use cached stats immediately after restart and then rebuild the refresh schedule. ### Release note Cloud FE no longer regenerates a checkpoint solely because the current image is stale. Tablet-stat refresh scheduling state is rebuilt after FE restart. ### Check List (For Author) - Test: No need to test (commit-history squash only; code diff is unchanged) - Behavior changed: Yes (remove periodic stale-image checkpoint generation and persisted tablet-stat scheduling state) - Does this need documentation: No
mymeiyi
added a commit
to mymeiyi/doris
that referenced
this pull request
Jul 31, 2026
### What problem does this PR solve? Issue Number: None Related PR: apache#60705, apache#66154 Problem Summary: Periodically regenerating a cloud checkpoint without a newer finalized journal reused the same image sequence. That exposed checkpoint version mismatches, same-version image publication, and follower synchronization ambiguity. Stop triggering checkpoints solely because an image is stale while retaining cloud metadata persistence during normal checkpoints. Do not persist tablet-stat refresh scheduling state so an FE can use cached stats immediately after restart and then rebuild the refresh schedule. Also align the OlapTable test mock lambda with FE Checkstyle indentation rules. ### Release note Cloud FE no longer regenerates a checkpoint solely because the current image is stale. Tablet-stat refresh scheduling state is rebuilt after FE restart. ### Check List (For Author) - Test: No need to test (commit-history squash; functional code diff is unchanged) - Checkstyle: `mvn -pl fe-core checkstyle:check` - Behavior changed: Yes (remove periodic stale-image checkpoint generation and persisted tablet-stat scheduling state) - Does this need documentation: No
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)