[fix](repeat) Preserve grouping ID after repeat decomposition by feiniaofeiafei · Pull Request #65968 · apache/doris

feiniaofeiafei · 2026-07-23T12:12:00Z

What problem does this PR solve?

Related PR: #59116

DecomposeRepeatWithPreAggregation dropped the internal GROUPING_ID column and recalculated IDs after removing the largest grouping set. This breaks upper plans that reference GROUPING_ID, and changes the original grouping-ID semantics.

This PR preserves original grouping-ID values for the residual Repeat, appends the removed grouping set’s original ID to the direct branch, and keeps the internal slot through the rewritten Project and Union.

It also creates the new residual grouping-ID slot as non-nullable, matching the BE Repeat operator’s ColumnInt64 requirement and fixing the ColumnNullable cast failure.

Release note

Fix internal GROUPING_ID handling after Repeat decomposition.

Check List

Test: Added FE unit tests; RepeatTest passed.
Behavior changed: Yes.
Does this need documentation: No.

fix groupingid exprid wrong [fix](fe) Keep decomposed grouping ID non-null Issue Number: None Related PR: None Problem Summary: DecomposeRepeatWithPreAggregation creates a new grouping-id slot for the residual Repeat. The default SlotReference constructor makes the slot nullable, but the Repeat backend writes grouping IDs directly to a non-nullable Int64 column. This caused a ColumnNullable-to-ColumnInt64 assertion failure when the rewrite ran. Create the new slot as non-nullable so its descriptor matches the Repeat operator contract. None - Test: Unit Test (run-fe-ut.sh --run DecomposeRepeatWithPreAggregationTest; build started, but the test environment previously could not allocate an FE HTTP port) - Behavior changed: Yes (prevents decomposed Repeat from producing a nullable internal grouping-id slot) - Does this need documentation: No fix

hello-stephen · 2026-07-23T12:12:05Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

feiniaofeiafei · 2026-07-23T12:13:29Z

run buildall

hello-stephen · 2026-07-23T12:31:43Z

TPC-H: Total hot run time: 29404 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 97785a59d28659b10d54b659160876b7de7dc9c6, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17612	4135	4146	4135
q2	2020	309	201	201
q3	10304	1423	801	801
q4	4688	487	338	338
q5	7506	846	565	565
q6	185	176	144	144
q7	760	815	620	620
q8	9321	1484	1648	1484
q9	5645	4344	4336	4336
q10	6750	1722	1483	1483
q11	525	356	336	336
q12	735	578	448	448
q13	18148	3397	2699	2699
q14	264	258	239	239
q15	q16	790	769	706	706
q17	1020	1002	1048	1002
q18	7009	5828	5539	5539
q19	1367	1218	1042	1042
q20	801	671	579	579
q21	5880	2526	2401	2401
q22	441	364	306	306
Total cold run time: 101771 ms
Total hot run time: 29404 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4438	4297	4398	4297
q2	289	309	208	208
q3	4592	4929	4396	4396
q4	2053	2120	1358	1358
q5	4382	4252	4252	4252
q6	232	178	129	129
q7	1738	1642	1449	1449
q8	2276	1991	1950	1950
q9	7251	7142	7167	7142
q10	4618	4598	4123	4123
q11	541	421	379	379
q12	748	747	529	529
q13	2972	3399	2768	2768
q14	280	296	258	258
q15	q16	681	705	629	629
q17	1285	1274	1277	1274
q18	7416	6923	6807	6807
q19	1063	1074	1160	1074
q20	2222	2205	1947	1947
q21	5208	4669	4343	4343
q22	536	480	405	405
Total cold run time: 54821 ms
Total hot run time: 49717 ms

hello-stephen · 2026-07-23T12:42:53Z

TPC-DS: Total hot run time: 176857 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 97785a59d28659b10d54b659160876b7de7dc9c6, data reload: false

query5	4309	625	497	497
query6	463	221	196	196
query7	4898	581	333	333
query8	339	193	172	172
query9	8778	4126	4070	4070
query10	450	346	302	302
query11	5898	2343	2137	2137
query12	148	101	97	97
query13	1252	629	414	414
query14	6238	5193	4852	4852
query14_1	4249	4279	4241	4241
query15	214	197	177	177
query16	997	473	433	433
query17	908	697	553	553
query18	2428	465	338	338
query19	199	180	144	144
query20	111	113	111	111
query21	233	163	135	135
query22	13507	13617	13354	13354
query23	17242	16480	16047	16047
query23_1	16175	16126	16198	16126
query24	7457	1766	1348	1348
query24_1	1306	1302	1315	1302
query25	572	455	390	390
query26	1341	362	215	215
query27	2588	555	371	371
query28	4531	1995	1994	1994
query29	1101	629	509	509
query30	341	263	232	232
query31	1119	1093	975	975
query32	109	65	61	61
query33	539	323	263	263
query34	1213	1116	654	654
query35	776	782	692	692
query36	1206	1219	1047	1047
query37	162	110	90	90
query38	1878	1707	1677	1677
query39	885	851	869	851
query39_1	825	840	842	840
query40	253	175	154	154
query41	77	70	73	70
query42	95	97	103	97
query43	340	344	296	296
query44	1433	780	774	774
query45	200	191	174	174
query46	1092	1170	704	704
query47	2136	2122	1956	1956
query48	395	399	295	295
query49	588	431	316	316
query50	1149	444	343	343
query51	10882	10560	10641	10560
query52	86	89	84	84
query53	272	288	201	201
query54	288	260	232	232
query55	76	72	68	68
query56	298	308	307	307
query57	1300	1293	1201	1201
query58	321	272	279	272
query59	1631	1681	1474	1474
query60	319	291	257	257
query61	214	148	148	148
query62	544	497	425	425
query63	253	203	200	200
query64	2831	1055	864	864
query65	4725	4657	4649	4649
query66	1837	510	377	377
query67	29206	28465	28286	28286
query68	3143	1587	931	931
query69	408	315	280	280
query70	1031	981	993	981
query71	380	330	301	301
query72	3005	2693	2400	2400
query73	801	820	423	423
query74	5057	4944	4705	4705
query75	2523	2482	2145	2145
query76	2330	1166	769	769
query77	356	385	287	287
query78	11968	11778	11330	11330
query79	1393	1169	768	768
query80	1297	584	474	474
query81	556	329	291	291
query82	627	161	117	117
query83	420	320	296	296
query84	331	158	134	134
query85	996	622	558	558
query86	419	289	265	265
query87	1827	1828	1780	1780
query88	3729	2814	2774	2774
query89	445	384	335	335
query90	1912	204	211	204
query91	203	188	169	169
query92	64	59	56	56
query93	1624	1623	1037	1037
query94	713	381	296	296
query95	826	539	462	462
query96	1126	824	347	347
query97	2600	2600	2491	2491
query98	213	207	202	202
query99	1075	1111	960	960
Total cold run time: 263537 ms
Total hot run time: 176857 ms

hello-stephen · 2026-07-23T12:47:53Z

ClickBench: Total hot run time: 24.89 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 97785a59d28659b10d54b659160876b7de7dc9c6, data reload: false

query1	0.01	0.01	0.00
query2	0.09	0.05	0.05
query3	0.25	0.14	0.13
query4	1.61	0.14	0.14
query5	0.24	0.22	0.21
query6	1.28	1.05	1.07
query7	0.03	0.00	0.01
query8	0.06	0.04	0.04
query9	0.37	0.31	0.32
query10	0.56	0.61	0.54
query11	0.18	0.13	0.12
query12	0.17	0.14	0.13
query13	0.46	0.46	0.47
query14	1.02	1.02	1.00
query15	0.60	0.58	0.58
query16	0.31	0.33	0.32
query17	1.07	1.10	1.11
query18	0.22	0.21	0.20
query19	2.05	1.97	1.98
query20	0.02	0.01	0.01
query21	15.45	0.20	0.13
query22	4.92	0.05	0.05
query23	16.14	0.30	0.13
query24	2.99	0.40	0.31
query25	0.11	0.05	0.04
query26	0.73	0.20	0.14
query27	0.04	0.04	0.04
query28	3.53	0.93	0.54
query29	12.48	4.17	3.30
query30	0.28	0.16	0.15
query31	2.77	0.62	0.31
query32	3.22	0.58	0.48
query33	3.15	3.14	3.17
query34	15.71	4.19	3.53
query35	3.49	3.49	3.51
query36	0.56	0.43	0.40
query37	0.09	0.06	0.07
query38	0.05	0.04	0.03
query39	0.04	0.03	0.03
query40	0.18	0.16	0.15
query41	0.08	0.04	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.04
Total cold run time: 96.69 s
Total hot run time: 24.89 s

feiniaofeiafei requested review from 924060929, englefly, morrySnow and starocean999 as code owners July 23, 2026 12:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[fix](repeat) Preserve grouping ID after repeat decomposition#65968

[fix](repeat) Preserve grouping ID after repeat decomposition#65968
feiniaofeiafei wants to merge 1 commit into
apache:masterfrom
feiniaofeiafei:decompose_repeat_output_grouping_id

feiniaofeiafei commented Jul 23, 2026 •

edited

Loading

Uh oh!

hello-stephen commented Jul 23, 2026

Uh oh!

feiniaofeiafei commented Jul 23, 2026

Uh oh!

hello-stephen commented Jul 23, 2026

Uh oh!

hello-stephen commented Jul 23, 2026

Uh oh!

hello-stephen commented Jul 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

feiniaofeiafei commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List

Uh oh!

hello-stephen commented Jul 23, 2026

Uh oh!

feiniaofeiafei commented Jul 23, 2026

Uh oh!

hello-stephen commented Jul 23, 2026

Uh oh!

hello-stephen commented Jul 23, 2026

Uh oh!

hello-stephen commented Jul 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feiniaofeiafei commented Jul 23, 2026 •

edited

Loading