Skip to content

[fix](fe) remove table from waitToCommitTxnCountMap in CloudGlobalTransactionMgr when drop table - #59757

Merged
dataroaring merged 1 commit into
apache:masterfrom
mymeiyi:fix-txn-manager
Jan 13, 2026
Merged

[fix](fe) remove table from waitToCommitTxnCountMap in CloudGlobalTransactionMgr when drop table#59757
dataroaring merged 1 commit into
apache:masterfrom
mymeiyi:fix-txn-manager

Conversation

@mymeiyi

@mymeiyi mymeiyi commented Jan 12, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

  1. CloudGlobalTransactionMgr does not remove waitToCommitTxnCountMap if table is dropped.
  2. simplify lastTxnIdMap and txnLastSignatureMap

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

Copilot AI review requested due to automatic review settings January 12, 2026 06:54
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request fixes a resource leak in CloudGlobalTransactionMgr where table metadata was not properly cleaned up when tables are dropped. Additionally, it simplifies the internal data structure by removing unnecessary nesting of maps that used database IDs as keys.

Changes:

  • Renamed clearTableLastTxnId to afterDropTable and added cleanup for waitToCommitTxnCountMap
  • Simplified lastTxnIdMap from Map<Long, Map<Long, Long>> (dbId -> tableId -> txnId) to Map<Long, Long> (tableId -> txnId)
  • Simplified txnLastSignatureMap from Map<Long, Map<Long, Long>> (dbId -> txnId -> signature) to Map<Long, Long> (txnId -> signature)
  • Changed several methods from public to private as they are only used internally

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
InternalCatalog.java Updated to call the renamed afterDropTable method instead of clearTableLastTxnId when dropping tables in cloud mode
CloudGlobalTransactionMgr.java Refactored map data structures to remove database ID nesting, added cleanup of waitToCommitTxnCountMap in afterDropTable, and changed visibility of internal methods to private

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mymeiyi

mymeiyi commented Jan 12, 2026

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

Copy link
Copy Markdown
TPC-H: Total hot run time: 32227 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 56fd5d84438a5aa9691b3855f82c678e4c055f44, data reload: false

------ Round 1 ----------------------------------
q1	17630	4202	4063	4063
q2	2041	371	252	252
q3	10131	1300	733	733
q4	10217	897	323	323
q5	7499	2099	1868	1868
q6	191	168	141	141
q7	929	788	666	666
q8	9284	1427	1153	1153
q9	5009	4668	4685	4668
q10	6818	1843	1407	1407
q11	527	298	285	285
q12	706	725	566	566
q13	17858	3853	3098	3098
q14	301	290	287	287
q15	602	509	495	495
q16	691	693	626	626
q17	687	825	511	511
q18	6649	6459	7180	6459
q19	1170	1007	626	626
q20	419	390	272	272
q21	3292	2717	2671	2671
q22	1189	1121	1057	1057
Total cold run time: 103840 ms
Total hot run time: 32227 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4500	4258	4271	4258
q2	319	432	309	309
q3	2262	2892	2352	2352
q4	1356	1941	1460	1460
q5	4486	4332	4331	4331
q6	214	168	133	133
q7	2018	1923	1806	1806
q8	2530	2726	2431	2431
q9	7124	7046	7150	7046
q10	2484	2681	2359	2359
q11	570	467	447	447
q12	768	860	633	633
q13	3605	4126	3293	3293
q14	317	328	282	282
q15	552	510	502	502
q16	661	724	655	655
q17	1147	1279	1325	1279
q18	7462	7219	7087	7087
q19	820	772	797	772
q20	1889	1945	1814	1814
q21	4502	4251	4165	4165
q22	1035	1025	969	969
Total cold run time: 50621 ms
Total hot run time: 48383 ms

@doris-robot

Copy link
Copy Markdown
TPC-DS: Total hot run time: 173303 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 56fd5d84438a5aa9691b3855f82c678e4c055f44, data reload: false

query5	4864	590	431	431
query6	357	234	210	210
query7	4219	458	256	256
query8	329	243	227	227
query9	8775	2647	2654	2647
query10	484	375	320	320
query11	15411	15168	15052	15052
query12	190	115	115	115
query13	1258	502	365	365
query14	6458	3006	2733	2733
query14_1	2692	2622	2646	2622
query15	198	196	169	169
query16	979	459	479	459
query17	1045	665	558	558
query18	2655	424	326	326
query19	218	216	192	192
query20	124	118	117	117
query21	216	150	119	119
query22	3839	4041	4008	4008
query23	16012	15644	15327	15327
query23_1	15318	15778	15418	15418
query24	7405	1552	1202	1202
query24_1	1231	1210	1208	1208
query25	563	480	429	429
query26	1246	271	167	167
query27	2744	450	295	295
query28	4561	2151	2139	2139
query29	829	575	464	464
query30	308	242	211	211
query31	770	637	569	569
query32	83	71	81	71
query33	560	359	300	300
query34	915	884	524	524
query35	728	761	690	690
query36	868	904	832	832
query37	125	100	78	78
query38	2708	2703	2642	2642
query39	771	792	771	771
query39_1	718	721	710	710
query40	222	142	123	123
query41	76	69	70	69
query42	111	108	108	108
query43	485	468	439	439
query44	1300	726	728	726
query45	190	185	178	178
query46	856	961	595	595
query47	1343	1443	1362	1362
query48	311	343	245	245
query49	629	423	369	369
query50	647	283	210	210
query51	3780	3805	3745	3745
query52	107	111	97	97
query53	310	332	282	282
query54	297	275	260	260
query55	84	78	73	73
query56	312	304	320	304
query57	995	1044	889	889
query58	281	267	259	259
query59	2048	2103	2175	2103
query60	332	360	286	286
query61	167	167	160	160
query62	411	353	298	298
query63	310	269	274	269
query64	4915	1305	998	998
query65	3814	3796	3704	3704
query66	1384	411	299	299
query67	15210	14991	14995	14991
query68	6440	965	714	714
query69	500	340	316	316
query70	1067	937	978	937
query71	368	300	273	273
query72	6070	3251	3372	3251
query73	732	653	303	303
query74	8915	8736	8622	8622
query75	2798	2794	2427	2427
query76	3487	1061	644	644
query77	508	371	283	283
query78	9684	9867	9219	9219
query79	1229	894	583	583
query80	622	570	498	498
query81	518	265	232	232
query82	209	142	112	112
query83	265	258	240	240
query84	256	118	102	102
query85	895	527	457	457
query86	394	326	328	326
query87	2879	2877	2767	2767
query88	3131	2223	2203	2203
query89	382	356	324	324
query90	2164	149	142	142
query91	177	161	143	143
query92	84	68	66	66
query93	1369	939	531	531
query94	577	333	314	314
query95	576	392	303	303
query96	589	469	204	204
query97	2327	2401	2315	2315
query98	210	199	203	199
query99	572	569	509	509
Total cold run time: 252447 ms
Total hot run time: 173303 ms

@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@dataroaring dataroaring left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Jan 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@dataroaring
dataroaring merged commit a452cc9 into apache:master Jan 13, 2026
36 of 38 checks passed
zzzxl1993 pushed a commit to zzzxl1993/doris that referenced this pull request Jan 13, 2026
…nsactionMgr when drop table (apache#59757)

### What problem does this PR solve?

1. `CloudGlobalTransactionMgr` does not remove `waitToCommitTxnCountMap`
if table is dropped.
2. simplify `lastTxnIdMap` and `txnLastSignatureMap`
mymeiyi added a commit to mymeiyi/doris that referenced this pull request Mar 18, 2026
…nsactionMgr when drop table (apache#59757)

### What problem does this PR solve?

1. `CloudGlobalTransactionMgr` does not remove `waitToCommitTxnCountMap`
if table is dropped.
2. simplify `lastTxnIdMap` and `txnLastSignatureMap`
yiguolei pushed a commit that referenced this pull request Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/4.1.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants