Skip to content

[fix](fe) Guard query instance metric before metric repo init - #62762

Open
seawinde wants to merge 1 commit into
apache:masterfrom
seawinde:fix/cir-20036-metric-init-guard
Open

[fix](fe) Guard query instance metric before metric repo init#62762
seawinde wants to merge 1 commit into
apache:masterfrom
seawinde:fix/cir-20036-metric-init-guard

Conversation

@seawinde

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: N/A

Related PR: N/A

Problem Summary:
CIR-20036 reports that FE may throw a NullPointerException while
registering query instances during startup or role switch.

Root cause: In QeProcessorImpl.registerInstances()
(fe/fe-core/src/main/java/org/apache/doris/qe/QeProcessorImpl.java:164),
query instance accounting always updates
MetricRepo.USER_COUNTER_QUERY_INSTANCE_BEGIN, but MetricRepo.init() may
not have completed yet, so the metric mapping is still unavailable.

This PR keeps the existing query-instance accounting logic unchanged and
only skips the metric increment until the metric repo is initialized.

Change Summary:

File Change Description
fe/fe-core/src/main/java/org/apache/doris/qe/QeProcessorImpl.java Guard USER_COUNTER_QUERY_INSTANCE_BEGIN update with MetricRepo.isInit so FE does not dereference an uninitialized metric during startup or role switch
fe/fe-core/src/test/java/org/apache/doris/qe/QeProcessorImplTest.java Add unit coverage for both pre-init and post-init registerInstances() paths

Release note

Fixed an issue where FE could hit a NullPointerException when query
instance metrics were updated before MetricRepo initialization completed.

Check List (For Author)

  • Test

    • Regression test

    • Unit Test

    • Manual test (add detailed scripts or steps below)

    • No need to test or manual test. Explain why:

      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

      Added QeProcessorImplTest, but ./run-fe-ut.sh --run org.apache.doris.qe.QeProcessorImplTest
      is currently blocked on upstream master by an unrelated FE
      compilation failure in fe-core generate-patterns
      (PatternDescribableProcessor -> ExpressionTypeMappingGenerator
      NoClassDefFoundError).

  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@seawinde

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 50.00% (1/2) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 0.17% (1/592) 🎉
Increment coverage report
Complete coverage report

1 similar comment
@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 0.17% (1/592) 🎉
Increment coverage report
Complete coverage report

morrySnow pushed a commit that referenced this pull request May 21, 2026
…#62766)

### What problem does this PR solve?

Related PR: #62599

Problem Summary:
The `partition_curd_union_rewrite` test intermittently fails in
`cloud_p0` CI because `mv_rewrite_success` calls after data
modifications (insert/delete) do not pass
`is_partition_statistics_ready`. Without this check, the function
defaults to `true` and asserts that CBO chose the MV. In cloud mode,
partition statistics propagation can be slower, so CBO may not select
the MV, causing the test to fail.

The first two `mv_rewrite_success` calls (before any DML) already pass
`is_partition_statistics_ready` correctly. This fix applies the same
pattern to the remaining six calls that follow insert/delete operations.

**Observed failures:**
- PR #62599 — cloud_p0 ❌
- PR #62762 — cloud_p0 ❌
@morrySnow
morrySnow force-pushed the fix/cir-20036-metric-init-guard branch from 69c0184 to 01fd96e Compare May 22, 2026 09:17
@morrySnow

Copy link
Copy Markdown
Contributor

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 31353 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 01fd96e690e0c28f75bf70626150fd6e72ba66cb, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17754	3923	3870	3870
q2	q3	10782	1384	821	821
q4	4688	470	355	355
q5	7563	2325	2127	2127
q6	229	182	137	137
q7	958	777	649	649
q8	9356	1785	1533	1533
q9	5122	4905	4940	4905
q10	6399	2064	1759	1759
q11	434	272	245	245
q12	633	425	297	297
q13	18098	3309	2785	2785
q14	268	255	235	235
q15	q16	822	771	716	716
q17	925	996	931	931
q18	7037	5721	5529	5529
q19	1353	1278	1108	1108
q20	532	436	371	371
q21	6183	2829	2668	2668
q22	454	377	312	312
Total cold run time: 99590 ms
Total hot run time: 31353 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4728	4807	4492	4492
q2	q3	4918	5253	4649	4649
q4	2127	2196	1415	1415
q5	4662	4805	4650	4650
q6	239	177	129	129
q7	1860	1704	1551	1551
q8	2325	2076	2123	2076
q9	7617	7570	7226	7226
q10	4472	4375	3939	3939
q11	519	391	348	348
q12	707	722	510	510
q13	3068	3389	2897	2897
q14	266	283	247	247
q15	q16	676	709	622	622
q17	1256	1224	1229	1224
q18	7159	6726	6700	6700
q19	1118	1105	1118	1105
q20	2204	2181	1950	1950
q21	5273	4631	4467	4467
q22	520	472	398	398
Total cold run time: 55714 ms
Total hot run time: 50595 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 169302 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 01fd96e690e0c28f75bf70626150fd6e72ba66cb, data reload: false

query5	4342	648	535	535
query6	348	216	196	196
query7	4223	563	301	301
query8	328	240	228	228
query9	8827	3982	4008	3982
query10	455	350	296	296
query11	5798	2392	2218	2218
query12	172	125	124	124
query13	1269	571	393	393
query14	5919	5324	5034	5034
query14_1	4318	4335	4314	4314
query15	209	201	181	181
query16	975	464	407	407
query17	946	699	601	601
query18	2443	483	344	344
query19	221	189	163	163
query20	133	131	128	128
query21	214	135	118	118
query22	13664	13721	13402	13402
query23	17199	16401	16106	16106
query23_1	16172	16098	16215	16098
query24	7553	1751	1323	1323
query24_1	1304	1324	1306	1306
query25	542	466	405	405
query26	1315	346	168	168
query27	2678	582	336	336
query28	4424	1946	1940	1940
query29	964	619	489	489
query30	304	238	201	201
query31	1109	1065	942	942
query32	86	73	73	73
query33	519	343	287	287
query34	1179	1132	638	638
query35	749	775	676	676
query36	1320	1303	1170	1170
query37	149	107	93	93
query38	3235	3124	3078	3078
query39	948	917	906	906
query39_1	879	892	879	879
query40	229	153	127	127
query41	71	69	68	68
query42	117	114	117	114
query43	334	361	291	291
query44	
query45	209	202	205	202
query46	1094	1196	716	716
query47	2282	2363	2147	2147
query48	404	413	287	287
query49	651	502	426	426
query50	944	338	257	257
query51	4373	4270	4159	4159
query52	107	107	96	96
query53	262	274	205	205
query54	346	282	270	270
query55	97	93	95	93
query56	312	327	307	307
query57	1398	1391	1321	1321
query58	319	278	280	278
query59	1549	1594	1417	1417
query60	331	346	327	327
query61	184	181	186	181
query62	665	625	553	553
query63	244	198	209	198
query64	2471	867	714	714
query65	
query66	1760	493	373	373
query67	30112	30066	29845	29845
query68	
query69	481	350	316	316
query70	1031	1003	1004	1003
query71	315	283	277	277
query72	3113	2683	2409	2409
query73	846	709	413	413
query74	5060	4890	4738	4738
query75	2661	2597	2252	2252
query76	2281	1150	776	776
query77	422	406	338	338
query78	12196	12206	11678	11678
query79	1457	1068	727	727
query80	927	555	455	455
query81	491	285	244	244
query82	1379	164	122	122
query83	364	279	242	242
query84	308	144	112	112
query85	908	539	462	462
query86	444	335	296	296
query87	3413	3362	3224	3224
query88	3518	2667	2655	2655
query89	446	385	333	333
query90	1782	174	182	174
query91	181	164	136	136
query92	81	86	80	80
query93	1450	1479	806	806
query94	608	379	317	317
query95	680	474	354	354
query96	1091	770	336	336
query97	2670	2700	2577	2577
query98	231	228	230	228
query99	1144	1106	993	993
Total cold run time: 253622 ms
Total hot run time: 169302 ms

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 50.00% (1/2) 🎉
Increment coverage report
Complete coverage report

zhaorongsheng pushed a commit to zhaorongsheng/doris that referenced this pull request Jun 4, 2026
…apache#62766)

### What problem does this PR solve?

Related PR: apache#62599

Problem Summary:
The `partition_curd_union_rewrite` test intermittently fails in
`cloud_p0` CI because `mv_rewrite_success` calls after data
modifications (insert/delete) do not pass
`is_partition_statistics_ready`. Without this check, the function
defaults to `true` and asserts that CBO chose the MV. In cloud mode,
partition statistics propagation can be slower, so CBO may not select
the MV, causing the test to fail.

The first two `mv_rewrite_success` calls (before any DML) already pass
`is_partition_statistics_ready` correctly. This fix applies the same
pattern to the remaining six calls that follow insert/delete operations.

**Observed failures:**
- PR apache#62599 — cloud_p0 ❌
- PR apache#62762 — cloud_p0 ❌
### What problem does this PR solve?

  Issue Number: N/A

  Related PR: N/A

  Problem Summary: Avoid NPE when query instance metrics are updated before MetricRepo initialization completes during FE startup or role switch.

  ### Release note

  None

  ### Check List (For Author)

  - Test: Attempted `./run-fe-ut.sh --run org.apache.doris.qe.QeProcessorImplTest` and `./build.sh --fe`, but FE clean build is currently blocked by an existing upstream/master fe-core pattern-generator
  compilation issue unrelated to this change
  - Behavior changed: Yes (avoid NPE before metric initialization)
  - Does this need documentation: No
@seawinde
seawinde force-pushed the fix/cir-20036-metric-init-guard branch from 01fd96e to 1d461e7 Compare August 19, 2026 03:22
@seawinde

Copy link
Copy Markdown
Member Author

run buildall

@seawinde

Copy link
Copy Markdown
Member Author

/review

@github-actions

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: All Codex review accounts are usage-limited; earliest retry is 2026-08-20T03:35:00Z.
Workflow run: https://github.com/apache/doris/actions/runs/32212237334

Please trigger /review again after that time.

@seawinde

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 17279 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1d461e73a95ce12b8d3add15613b49a54cc0a660, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17642	3046	3014	3014
q2	1894	238	155	155
q3	10453	899	523	523
q4	4675	258	200	200
q5	7668	588	385	385
q6	140	115	95	95
q7	540	504	386	386
q8	9233	927	928	927
q9	3471	2407	2367	2367
q10	6529	843	717	717
q11	451	254	238	238
q12	687	393	330	330
q13	17863	1536	1187	1187
q14	165	158	135	135
q15	q16	439	400	368	368
q17	774	755	712	712
q18	3160	2276	2250	2250
q19	1101	899	764	764
q20	614	550	462	462
q21	4859	1835	1954	1835
q22	332	268	229	229
Total cold run time: 92690 ms
Total hot run time: 17279 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3359	3309	3304	3304
q2	202	209	151	151
q3	2257	2325	2214	2214
q4	1191	1167	901	901
q5	2175	2124	2128	2124
q6	177	125	88	88
q7	1037	942	827	827
q8	1603	1436	1387	1387
q9	3127	3082	3064	3064
q10	1853	1815	1658	1658
q11	352	268	243	243
q12	456	436	335	335
q13	1476	1533	1180	1180
q14	171	176	152	152
q15	q16	391	392	353	353
q17	1037	1026	1013	1013
q18	4947	4444	4779	4444
q19	894	875	900	875
q20	948	943	798	798
q21	3490	3279	3287	3279
q22	404	349	316	316
Total cold run time: 31547 ms
Total hot run time: 28706 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 82781 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1d461e73a95ce12b8d3add15613b49a54cc0a660, data reload: false

query5	4255	415	341	341
query6	409	164	162	162
query7	4866	468	267	267
query8	301	122	112	112
query9	8687	2858	2860	2858
query10	387	288	221	221
query11	5379	1034	921	921
query12	124	72	73	72
query13	1203	456	334	334
query14	6170	2186	2059	2059
query14_1	1970	1942	1954	1942
query15	177	124	112	112
query16	943	366	352	352
query17	800	453	360	360
query18	2331	329	241	241
query19	170	143	118	118
query20	76	69	77	69
query21	218	116	101	101
query22	5416	5399	5446	5399
query23	6812	6308	5868	5868
query23_1	6241	6142	6038	6038
query24	7300	1092	780	780
query24_1	783	787	803	787
query25	443	309	258	258
query26	1254	270	169	169
query27	2720	443	293	293
query28	4623	1489	1489	1489
query29	957	443	364	364
query30	280	184	153	153
query31	842	422	356	356
query32	106	50	49	49
query33	474	234	192	192
query34	997	804	493	493
query35	410	403	355	355
query36	568	547	514	514
query37	135	88	71	71
query38	1018	833	805	805
query39	497	485	450	450
query39_1	458	476	499	476
query40	224	128	115	115
query41	59	55	57	55
query42	83	85	83	83
query43	245	241	208	208
query44	1062	547	545	545
query45	118	104	100	100
query46	779	817	537	537
query47	799	740	736	736
query48	313	294	270	270
query49	532	241	181	181
query50	865	328	257	257
query51	7894	7910	7955	7910
query52	75	72	64	64
query53	202	235	158	158
query54	233	190	187	187
query55	76	61	55	55
query56	221	273	246	246
query57	703	672	672	672
query58	227	192	197	192
query59	1216	1195	1072	1072
query60	266	205	186	186
query61	114	108	129	108
query62	379	212	172	172
query63	184	151	154	151
query64	2703	737	602	602
query65	1660	1562	1572	1562
query66	1958	303	267	267
query67	9792	9506	9461	9461
query68	2761	1215	802	802
query69	334	229	200	200
query70	678	602	621	602
query71	308	280	232	232
query72	2267	1682	1534	1534
query73	622	605	345	345
query74	1572	1214	1137	1137
query75	1223	1153	998	998
query76	2293	758	546	546
query77	252	257	209	209
query78	3836	3713	3179	3179
query79	2562	864	597	597
query80	1589	407	335	335
query81	480	195	174	174
query82	613	118	96	96
query83	325	245	228	228
query84	310	125	108	108
query85	842	449	383	383
query86	395	167	173	167
query87	998	964	883	883
query88	2795	2120	2099	2099
query89	304	226	198	198
query90	1978	137	135	135
query91	162	140	125	125
query92	47	47	42	42
query93	1641	1194	821	821
query94	660	246	210	210
query95	621	360	375	360
query96	784	576	252	252
query97	1066	1025	1033	1025
query98	159	131	138	131
query99	417	338	311	311
Total cold run time: 178066 ms
Total hot run time: 82781 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.67 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1d461e73a95ce12b8d3add15613b49a54cc0a660, data reload: false

query1	0.01	0.01	0.01
query2	0.08	0.04	0.04
query3	0.26	0.11	0.11
query4	1.60	0.10	0.10
query5	0.17	0.15	0.15
query6	1.28	0.72	0.69
query7	0.04	0.00	0.01
query8	0.04	0.02	0.02
query9	0.28	0.22	0.21
query10	0.36	0.35	0.35
query11	0.17	0.12	0.12
query12	0.15	0.12	0.11
query13	0.30	0.31	0.31
query14	0.44	0.44	0.45
query15	0.36	0.34	0.35
query16	0.21	0.21	0.21
query17	0.67	0.70	0.66
query18	0.18	0.16	0.17
query19	1.17	1.18	1.13
query20	0.01	0.01	0.01
query21	15.45	0.16	0.11
query22	5.10	0.05	0.04
query23	16.21	0.26	0.11
query24	2.98	0.34	0.27
query25	0.10	0.05	0.04
query26	0.79	0.16	0.12
query27	0.03	0.03	0.03
query28	3.66	0.57	0.26
query29	12.44	3.20	2.57
query30	0.26	0.11	0.12
query31	2.78	0.37	0.17
query32	3.52	0.32	0.23
query33	1.34	1.56	1.45
query34	15.39	2.16	1.77
query35	1.73	1.77	1.72
query36	0.44	0.28	0.29
query37	0.06	0.04	0.04
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.12	0.07	0.07
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 90.39 s
Total hot run time: 14.67 s

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 50.00% (1/2) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants