Skip to content

[enhance](inverted index) support for multi indexes - #54406

Merged
eldenmoon merged 8 commits into
apache:masterfrom
csun5285:add_multi_index_master
Aug 11, 2025
Merged

[enhance](inverted index) support for multi indexes#54406
eldenmoon merged 8 commits into
apache:masterfrom
csun5285:add_multi_index_master

Conversation

@csun5285

@csun5285 csun5285 commented Aug 6, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

1.Supports two inverted indexes (tokenized & non-tokenized) on one STRING column.
2.Supports for creating multiple inverted indexes on VARIANT subcolumns.

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

hello-stephen commented Aug 6, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@csun5285

csun5285 commented Aug 6, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

1 similar comment
@csun5285

csun5285 commented Aug 6, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@morrySnow

morrySnow commented Aug 6, 2025

Copy link
Copy Markdown
Contributor

add description

@doris-robot

doris-robot commented Aug 6, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 33867 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0568d7a960468514a7949f785f46c6914752b67c, data reload: false

------ Round 1 ----------------------------------
q1	17588	5261	5174	5174
q2	1940	278	185	185
q3	10296	1328	718	718
q4	10223	1037	515	515
q5	7525	2387	2329	2329
q6	175	167	131	131
q7	908	771	608	608
q8	9324	1349	1086	1086
q9	6803	5104	5158	5104
q10	6886	2364	2000	2000
q11	460	278	270	270
q12	351	375	231	231
q13	17782	3455	2975	2975
q14	260	234	216	216
q15	528	464	456	456
q16	421	430	378	378
q17	585	833	358	358
q18	7442	7189	7063	7063
q19	1235	977	554	554
q20	339	333	225	225
q21	3505	3150	2301	2301
q22	1076	1044	990	990
Total cold run time: 105652 ms
Total hot run time: 33867 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5443	5307	5272	5272
q2	237	308	224	224
q3	2100	2577	2180	2180
q4	1416	1744	1324	1324
q5	4154	4463	4532	4463
q6	218	172	149	149
q7	1914	1954	1753	1753
q8	2544	2518	2597	2518
q9	7247	7125	7347	7125
q10	3171	3349	2903	2903
q11	559	502	487	487
q12	729	740	658	658
q13	3492	3790	3376	3376
q14	321	327	280	280
q15	502	452	454	452
q16	507	714	460	460
q17	1225	1458	1415	1415
q18	14111	7819	7511	7511
q19	14792	960	859	859
q20	1998	1975	1764	1764
q21	14895	4286	4284	4284
q22	1082	1046	1012	1012
Total cold run time: 82657 ms
Total hot run time: 50469 ms

@doris-robot

doris-robot commented Aug 6, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 160762 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0568d7a960468514a7949f785f46c6914752b67c, data reload: false

reason	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:56	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
============================================
query1	999	376	416	376
query2	6540	1836	1739	1739
query3	6753	224	227	224
query4	26413	23986	22789	22789
query5	4315	640	521	521
query6	316	232	236	232
query7	4629	536	301	301
query8	275	233	224	224
query9	8605	2997	2978	2978
query10	492	327	283	283
query11	15384	14926	14703	14703
query12	188	137	134	134
query13	1676	551	439	439
query14	8627	5908	5837	5837
query15	214	192	167	167
query16	7816	659	487	487
query17	1632	806	650	650
query18	2098	453	328	328
query19	309	212	188	188
query20	149	150	138	138
query21	219	126	113	113
query22	3923	3924	3823	3823
query23	34308	34229	34365	34229
query24	8113	2423	2449	2423
query25	542	513	441	441
query26	889	289	164	164
query27	2633	504	367	367
query28	4221	2366	2344	2344
query29	718	596	490	490
query30	300	228	197	197
query31	917	819	717	717
query32	89	77	80	77
query33	570	438	362	362
query34	819	838	512	512
query35	791	876	762	762
query36	1075	1038	921	921
query37	130	122	95	95
query38	3919	4008	3958	3958
query39	1435	1372	1360	1360
query40	231	145	136	136
query41	62	57	55	55
query42	142	123	125	123
query43	528	515	474	474
query44	1418	890	889	889
query45	206	191	184	184
query46	964	1060	679	679
query47	1809	1846	1777	1777
query48	407	435	316	316
query49	711	520	411	411
query50	682	688	415	415
query51	4192	4251	4179	4179
query52	125	137	125	125
query53	257	293	215	215
query54	647	678	555	555
query55	89	94	89	89
query56	365	358	361	358
query57	1226	1206	1135	1135
query58	334	333	340	333
query59	2671	2653	2569	2569
query60	421	414	393	393
query61	126	124	124	124
query62	835	742	685	685
query63	257	216	211	211
query64	3553	1110	784	784
query65	4220	4137	4124	4124
query66	934	452	337	337
query67	query68	17971	944	856	856
query69	1240	328	296	296
query70	1485	1118	1112	1112
query71	657	314	322	314
query72	9047	2364	2211	2211
query73	4234	699	359	359
query74	9076	8987	8845	8845
query75	7024	3161	2634	2634
query76	8247	1222	790	790
query77	1160	407	382	382
query78	9549	query79	16483	621	586	586
query80	2326	598	504	504
query81	542	257	229	229
query82	563	150	120	120
query83	365	292	275	275
query84	313	104	83	83
query85	1496	432	350	350
query86	364	327	306	306
query87	4245	4242	4142	4142
query88	5521	2242	2234	2234
query89	522	410	307	307
query90	2491	236	225	225
query91	149	141	115	115
query92	89	75	68	68
query93	6935	998	663	663
query94	1188	402	274	274
query95	436	324	315	315
query96	506	586	279	279
query97	2731	2732	2600	2600
query98	264	230	230	230
query99	1474	1354	1290	1290
Total cold run time: 299597 ms
Total hot run time: 160762 ms

@doris-robot

doris-robot commented Aug 6, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 33.2 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0568d7a960468514a7949f785f46c6914752b67c, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.24	0.07	0.07
query4	1.61	0.11	0.11
query5	0.44	0.41	0.42
query6	1.16	0.65	0.64
query7	0.03	0.02	0.03
query8	0.06	0.04	0.04
query9	0.56	0.50	0.46
query10	0.52	0.54	0.52
query11	0.16	0.10	0.11
query12	0.14	0.11	0.11
query13	0.66	0.65	0.64
query14	0.90	1.07	1.12
query15	0.94	0.90	0.86
query16	0.40	0.40	0.39
query17	1.06	1.06	1.09
query18	0.22	0.21	0.20
query19	1.89	1.88	1.88
query20	0.01	0.01	0.01
query21	15.42	0.87	0.55
query22	0.81	1.25	0.83
query23	14.73	1.20	0.62
query24	6.50	0.88	0.73
query25	0.56	0.13	0.14
query26	0.61	0.16	0.13
query27	0.06	0.06	0.06
query28	9.81	0.84	0.44
query29	12.58	3.79	3.27
query30	3.03	3.06	2.92
query31	2.81	0.57	0.40
query32	3.23	0.57	0.50
query33	3.02	3.19	3.19
query34	15.81	5.42	4.96
query35	4.91	4.94	4.92
query36	0.70	0.51	0.51
query37	0.10	0.08	0.07
query38	0.06	0.05	0.05
query39	0.04	0.03	0.03
query40	0.18	0.14	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.03	0.04
Total cold run time: 106.26 s
Total hot run time: 33.2 s

@csun5285
csun5285 force-pushed the add_multi_index_master branch from dcf0dba to 82ea400 Compare August 7, 2025 06:15
@csun5285

csun5285 commented Aug 7, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Aug 7, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 33910 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 82ea40006b97d6845be89af2fb2dc7272d7a6afa, data reload: false

------ Round 1 ----------------------------------
q1	17594	5327	5304	5304
q2	1928	283	187	187
q3	10330	1297	687	687
q4	10220	986	523	523
q5	7539	2253	2393	2253
q6	183	169	135	135
q7	911	750	642	642
q8	9306	1362	1153	1153
q9	6962	5464	5116	5116
q10	6914	2371	1964	1964
q11	477	280	264	264
q12	351	362	220	220
q13	17762	3466	2980	2980
q14	239	237	226	226
q15	519	467	467	467
q16	440	419	386	386
q17	583	847	360	360
q18	7252	7029	6956	6956
q19	1329	1001	565	565
q20	330	320	209	209
q21	3595	3094	2308	2308
q22	1057	1035	1005	1005
Total cold run time: 105821 ms
Total hot run time: 33910 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5353	5506	5306	5306
q2	243	316	218	218
q3	2097	2571	2222	2222
q4	1351	1689	1315	1315
q5	4210	4450	4397	4397
q6	229	185	139	139
q7	2047	1933	1710	1710
q8	2704	2610	2450	2450
q9	7377	7196	7430	7196
q10	3200	3362	2921	2921
q11	533	520	507	507
q12	767	988	605	605
q13	3387	3554	3310	3310
q14	324	307	309	307
q15	488	491	447	447
q16	455	490	464	464
q17	1163	1449	1403	1403
q18	7999	7684	8027	7684
q19	8990	1018	1079	1018
q20	1991	2025	1945	1945
q21	14942	4310	4209	4209
q22	1068	1041	997	997
Total cold run time: 70918 ms
Total hot run time: 50770 ms

@doris-robot

doris-robot commented Aug 7, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 171171 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 82ea40006b97d6845be89af2fb2dc7272d7a6afa, data reload: false

============================================
query1	1006	408	439	408
query2	6536	1922	1685	1685
query3	6740	230	231	230
query4	27615	23843	23036	23036
query5	4362	641	539	539
query6	340	245	234	234
query7	4643	526	304	304
query8	304	238	235	235
query9	8566	3012	2993	2993
query10	502	358	309	309
query11	16047	14955	14948	14948
query12	182	137	135	135
query13	1687	565	428	428
query14	8784	5957	5940	5940
query15	251	189	163	163
query16	7854	660	480	480
query17	1610	786	610	610
query18	2078	430	315	315
query19	287	207	192	192
query20	154	145	138	138
query21	218	125	108	108
query22	3992	3857	3811	3811
query23	34286	34371	34125	34125
query24	7830	2405	2451	2405
query25	564	523	446	446
query26	726	298	162	162
query27	2465	494	351	351
query28	4144	2352	2342	2342
query29	679	636	493	493
query30	311	226	205	205
query31	909	819	708	708
query32	91	81	77	77
query33	554	422	397	397
query34	823	856	531	531
query35	809	819	774	774
query36	1098	1038	933	933
query37	129	111	94	94
query38	3964	3960	3925	3925
query39	1433	1365	1368	1365
query40	235	142	134	134
query41	62	61	56	56
query42	148	123	130	123
query43	547	515	500	500
query44	1485	895	889	889
query45	201	193	176	176
query46	954	1103	678	678
query47	1786	1823	1737	1737
query48	423	452	323	323
query49	670	500	439	439
query50	666	711	429	429
query51	4174	4169	4141	4141
query52	129	134	124	124
query53	269	304	224	224
query54	660	662	563	563
query55	94	89	89	89
query56	354	362	367	362
query57	1198	1225	1130	1130
query58	335	341	330	330
query59	2646	2672	2584	2584
query60	400	405	403	403
query61	122	124	123	123
query62	816	707	670	670
query63	261	221	214	214
query64	3096	1123	808	808
query65	4214	4171	4132	4132
query66	899	457	337	337
query67	query68	19215	1053	858	858
query69	1015	273	295	273
query70	1422	1077	1105	1077
query71	707	330	319	319
query72	9228	2303	2364	2303
query73	3724	647	361	361
query74	9076	9178	8755	8755
query75	7772	3105	2679	2679
query76	8901	1219	849	849
query77	1156	417	346	346
query78	9590	11157	9837	9837
query79	14494	660	601	601
query80	2044	594	504	504
query81	580	260	227	227
query82	392	149	119	119
query83	433	286	269	269
query84	300	94	79	79
query85	944	387	332	332
query86	377	306	300	300
query87	4268	4172	4110	4110
query88	5181	2265	2266	2265
query89	479	365	322	322
query90	2618	248	245	245
query91	149	139	112	112
query92	88	73	66	66
query93	5845	1032	667	667
query94	1029	394	276	276
query95	407	335	327	327
query96	513	588	286	286
query97	2739	2730	2609	2609
query98	254	237	227	227
query99	1489	1432	1279	1279
Total cold run time: 307846 ms
Total hot run time: 171171 ms

@doris-robot

doris-robot commented Aug 7, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 33.59 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 82ea40006b97d6845be89af2fb2dc7272d7a6afa, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.25	0.08	0.07
query4	1.61	0.10	0.10
query5	0.42	0.43	0.45
query6	1.17	0.66	0.68
query7	0.03	0.02	0.02
query8	0.04	0.04	0.03
query9	0.55	0.50	0.48
query10	0.53	0.52	0.52
query11	0.16	0.10	0.10
query12	0.15	0.11	0.11
query13	0.67	0.67	0.66
query14	0.97	1.09	1.05
query15	0.95	0.90	0.88
query16	0.38	0.39	0.39
query17	1.06	1.09	1.03
query18	0.22	0.20	0.21
query19	2.05	1.77	1.83
query20	0.02	0.01	0.02
query21	15.39	0.88	0.56
query22	0.80	1.16	0.73
query23	14.86	1.22	0.62
query24	6.55	1.47	1.25
query25	0.48	0.27	0.09
query26	0.45	0.16	0.13
query27	0.08	0.05	0.05
query28	10.31	0.87	0.44
query29	12.60	3.79	3.31
query30	3.02	2.95	2.96
query31	2.82	0.57	0.40
query32	3.24	0.58	0.49
query33	3.08	3.21	3.21
query34	15.90	5.34	4.95
query35	4.92	4.97	5.00
query36	0.68	0.51	0.49
query37	0.10	0.07	0.07
query38	0.06	0.05	0.05
query39	0.04	0.03	0.03
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 107.06 s
Total hot run time: 33.59 s

eldenmoon
eldenmoon reviewed Aug 7, 2025
View reviewed changes
Comment thread fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
eldenmoon
eldenmoon reviewed Aug 7, 2025
View reviewed changes
Comment thread fe/fe-core/src/main/java/org/apache/doris/catalog/Column.java
@hello-stephen

hello-stephen commented Aug 7, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 60.82% (298/490) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 58.33% (16390/28101)
Line Coverage 47.21% (148211/313939)
Region Coverage 36.19% (110948/306564)
Branch Coverage 39.09% (49304/126127)

@csun5285
csun5285 force-pushed the add_multi_index_master branch from 82ea400 to b19d958 Compare August 8, 2025 04:36
@csun5285

csun5285 commented Aug 8, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

hello-stephen commented Aug 8, 2025

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 12.34% (29/235) 🎉
Increment coverage report
Complete coverage report

@doris-robot

doris-robot commented Aug 8, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 33749 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b19d958e53831190f3dcb4217b8a9e15e2ec90aa, data reload: false

------ Round 1 ----------------------------------
q1	17583	5285	5234	5234
q2	1908	289	190	190
q3	10323	1349	700	700
q4	10234	986	522	522
q5	7541	2339	2338	2338
q6	179	169	132	132
q7	894	761	641	641
q8	9317	1323	1080	1080
q9	6910	5072	5071	5071
q10	6884	2355	1993	1993
q11	460	283	271	271
q12	354	371	221	221
q13	17778	3472	2956	2956
q14	228	242	213	213
q15	544	474	458	458
q16	433	417	368	368
q17	586	829	349	349
q18	7556	7045	6918	6918
q19	1348	985	553	553
q20	334	311	214	214
q21	3359	3175	2346	2346
q22	1032	1024	981	981
Total cold run time: 105785 ms
Total hot run time: 33749 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5317	5315	5239	5239
q2	235	316	219	219
q3	2073	2616	2198	2198
q4	1289	1706	1326	1326
q5	4131	4369	4535	4369
q6	227	181	135	135
q7	1945	1963	1771	1771
q8	2572	2633	2511	2511
q9	7484	7206	7197	7197
q10	3204	3341	2883	2883
q11	544	502	487	487
q12	699	773	671	671
q13	3470	3727	3240	3240
q14	375	323	305	305
q15	512	471	482	471
q16	493	505	459	459
q17	1156	1626	1437	1437
q18	11035	7194	7084	7084
q19	980	869	894	869
q20	1859	1926	1759	1759
q21	9134	4338	4221	4221
q22	1033	1045	991	991
Total cold run time: 59767 ms
Total hot run time: 49842 ms

@doris-robot

doris-robot commented Aug 8, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 161031 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b19d958e53831190f3dcb4217b8a9e15e2ec90aa, data reload: false

reason	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:56	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
============================================
query1	993	387	439	387
query2	6521	1739	1820	1739
query3	6740	223	225	223
query4	26847	23748	23476	23476
query5	4340	625	530	530
query6	336	242	253	242
query7	4639	521	299	299
query8	278	237	227	227
query9	8566	2960	2938	2938
query10	472	344	275	275
query11	15506	14883	15300	14883
query12	196	135	137	135
query13	1677	579	443	443
query14	9017	5874	5886	5874
query15	280	187	177	177
query16	7964	678	492	492
query17	1644	816	647	647
query18	2097	452	333	333
query19	304	212	203	203
query20	154	141	137	137
query21	223	128	114	114
query22	4038	3881	3931	3881
query23	34721	34432	34281	34281
query24	8293	2360	2427	2360
query25	616	514	439	439
query26	924	289	164	164
query27	2650	500	348	348
query28	4253	2321	2309	2309
query29	691	616	496	496
query30	295	230	197	197
query31	914	832	679	679
query32	89	80	81	80
query33	556	481	378	378
query34	823	835	513	513
query35	834	820	772	772
query36	1016	1028	924	924
query37	140	107	98	98
query38	4011	3976	3907	3907
query39	1415	1391	1375	1375
query40	236	143	131	131
query41	62	61	57	57
query42	141	125	128	125
query43	515	518	483	483
query44	1393	865	857	857
query45	201	191	177	177
query46	954	1059	678	678
query47	1830	1806	1734	1734
query48	392	421	328	328
query49	699	532	426	426
query50	653	684	414	414
query51	4169	4191	4092	4092
query52	127	130	113	113
query53	265	285	215	215
query54	662	646	558	558
query55	88	82	86	82
query56	356	360	352	352
query57	1211	1204	1132	1132
query58	376	340	343	340
query59	2669	2671	2513	2513
query60	396	412	387	387
query61	125	123	121	121
query62	799	722	646	646
query63	254	210	216	210
query64	3596	1164	787	787
query65	4225	4124	4185	4124
query66	1078	478	330	330
query67	query68	16416	614	592	592
query69	1004	322	288	288
query70	1415	1094	1143	1094
query71	721	339	337	337
query72	9175	2285	4769	2285
query73	3112	781	358	358
query74	9004	8972	8579	8579
query75	7435	3101	2666	2666
query76	8595	1216	778	778
query77	1154	419	350	350
query78	query79	17184	651	613	613
query80	3194	571	481	481
query81	588	268	225	225
query82	484	159	120	120
query83	471	345	281	281
query84	290	97	87	87
query85	1717	399	337	337
query86	367	306	317	306
query87	4227	4184	4180	4180
query88	4612	2210	2219	2210
query89	536	382	319	319
query90	2631	234	228	228
query91	147	138	110	110
query92	92	74	75	74
query93	6443	1008	690	690
query94	1160	394	285	285
query95	417	335	316	316
query96	507	580	287	287
query97	2705	2746	2614	2614
query98	248	231	229	229
query99	1545	1337	1325	1325
Total cold run time: 300141 ms
Total hot run time: 161031 ms

@doris-robot

doris-robot commented Aug 8, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 33.61 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b19d958e53831190f3dcb4217b8a9e15e2ec90aa, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.05	0.04
query3	0.25	0.08	0.07
query4	1.62	0.11	0.10
query5	0.43	0.44	0.41
query6	1.17	0.68	0.67
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.55	0.48	0.47
query10	0.53	0.54	0.53
query11	0.16	0.10	0.10
query12	0.15	0.11	0.11
query13	0.67	0.65	0.68
query14	0.92	1.26	1.10
query15	0.91	0.88	0.88
query16	0.39	0.39	0.40
query17	1.07	1.08	1.10
query18	0.22	0.21	0.21
query19	2.01	1.90	1.99
query20	0.02	0.02	0.02
query21	15.36	0.81	0.55
query22	0.79	1.26	0.63
query23	14.90	1.25	0.63
query24	6.63	1.05	1.15
query25	0.53	0.12	0.19
query26	0.66	0.15	0.12
query27	0.06	0.04	0.05
query28	10.14	0.85	0.44
query29	12.63	3.84	3.34
query30	3.07	2.98	2.96
query31	2.81	0.57	0.41
query32	3.23	0.58	0.50
query33	3.04	3.18	3.26
query34	16.02	5.37	4.94
query35	4.91	5.00	4.99
query36	0.69	0.51	0.50
query37	0.10	0.08	0.07
query38	0.06	0.05	0.04
query39	0.04	0.03	0.03
query40	0.18	0.14	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 107.3 s
Total hot run time: 33.61 s

@csun5285
csun5285 force-pushed the add_multi_index_master branch from b19d958 to bc0a8aa Compare August 8, 2025 08:12
@csun5285

csun5285 commented Aug 8, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Aug 8, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 33373 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bc0a8aaa357b87cb1053cfaf49947b912f1cc3c3, data reload: false

------ Round 1 ----------------------------------
q1	17626	5253	5134	5134
q2	1962	301	186	186
q3	10275	1330	692	692
q4	10204	970	520	520
q5	7479	2268	2325	2268
q6	176	167	131	131
q7	918	753	619	619
q8	9304	1215	1018	1018
q9	6845	5109	5040	5040
q10	6886	2362	1991	1991
q11	463	283	261	261
q12	353	364	232	232
q13	17794	3459	2963	2963
q14	233	225	211	211
q15	519	462	462	462
q16	424	412	370	370
q17	550	814	358	358
q18	7356	7089	6887	6887
q19	1685	982	543	543
q20	319	301	213	213
q21	3398	2447	2279	2279
q22	1023	1072	995	995
Total cold run time: 105792 ms
Total hot run time: 33373 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5327	5209	5431	5209
q2	233	318	216	216
q3	2065	2571	2242	2242
q4	1296	1701	1347	1347
q5	4148	4116	4422	4116
q6	220	174	136	136
q7	1948	1902	1828	1828
q8	2536	2395	2443	2395
q9	7423	7436	7699	7436
q10	3139	3311	2986	2986
q11	561	536	495	495
q12	698	917	648	648
q13	3419	3696	3257	3257
q14	290	303	289	289
q15	503	459	463	459
q16	444	484	431	431
q17	1156	1531	1392	1392
q18	10904	7605	7497	7497
q19	9176	1078	1082	1078
q20	1907	1902	1771	1771
q21	14873	4273	4351	4273
q22	1084	1039	1031	1031
Total cold run time: 73350 ms
Total hot run time: 50532 ms

@doris-robot

doris-robot commented Aug 8, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 185982 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bc0a8aaa357b87cb1053cfaf49947b912f1cc3c3, data reload: false

============================================
query1	999	411	392	392
query2	6535	1750	1654	1654
query3	6775	225	224	224
query4	28802	26258	25297	25297
query5	4376	619	478	478
query6	249	214	202	202
query7	4620	517	291	291
query8	248	217	198	198
query9	8524	2979	2922	2922
query10	449	369	350	350
query11	17018	16521	16369	16369
query12	183	137	129	129
query13	1603	491	361	361
query14	11265	9134	9066	9066
query15	390	218	189	189
query16	7792	527	336	336
query17	1778	1463	1355	1355
query18	2056	386	286	286
query19	204	185	159	159
query20	156	148	163	148
query21	215	124	104	104
query22	4682	4741	4606	4606
query23	35087	34349	34169	34169
query24	7464	1435	1470	1435
query25	1189	1174	1142	1142
query26	708	287	156	156
query27	2395	500	360	360
query28	2982	2344	2341	2341
query29	2222	3185	3047	3047
query30	466	330	518	330
query31	880	716	737	716
query32	89	72	74	72
query33	522	370	340	340
query34	826	908	509	509
query35	682	737	625	625
query36	1098	1096	984	984
query37	131	116	99	99
query38	4277	4278	4240	4240
query39	1663	1611	1576	1576
query40	305	271	251	251
query41	57	55	52	52
query42	143	119	120	119
query43	516	492	464	464
query44	1396	876	873	873
query45	194	178	183	178
query46	918	1049	653	653
query47	2057	2305	2247	2247
query48	632	375	308	308
query49	731	556	442	442
query50	720	743	503	503
query51	4094	4247	4100	4100
query52	124	128	118	118
query53	258	269	207	207
query54	691	724	596	596
query55	97	96	95	95
query56	335	360	322	322
query57	1376	1519	1446	1446
query58	454	307	313	307
query59	2760	2647	2702	2647
query60	386	402	384	384
query61	90	85	84	84
query62	851	772	683	683
query63	260	233	224	224
query64	3049	1257	812	812
query65	4351	4210	4213	4210
query66	1122	431	376	376
query67	query68	20353	884	582	582
query69	823	302	299	299
query70	1407	1110	1075	1075
query71	708	338	330	330
query72	9294	6630	6278	6278
query73	4468	618	363	363
query74	8776	8776	8753	8753
query75	6967	3166	2840	2840
query76	8442	1193	789	789
query77	1174	380	335	335
query78	9158	8945	8475	8475
query79	12423	662	587	587
query80	1149	1127	996	996
query81	520	253	229	229
query82	848	151	134	134
query83	321	283	286	283
query84	286	88	74	74
query85	609	236	189	189
query86	384	333	308	308
query87	4417	4457	4329	4329
query88	2933	2259	2255	2255
query89	463	368	326	326
query90	2581	237	244	237
query91	130	121	96	96
query92	96	72	68	68
query93	3145	1056	740	740
query94	931	298	238	238
query95	401	320	319	319
query96	495	652	284	284
query97	2652	2659	2581	2581
query98	259	244	232	232
query99	1564	1405	1400	1400
Total cold run time: 307744 ms
Total hot run time: 185982 ms

@csun5285

csun5285 commented Aug 8, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Aug 8, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 33836 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1686e95d4d80743206fccc38fe13b0229f6713f2, data reload: false

------ Round 1 ----------------------------------
q1	17583	5311	5302	5302
q2	1938	316	185	185
q3	10264	1475	740	740
q4	10225	991	526	526
q5	7543	2372	2347	2347
q6	186	165	129	129
q7	902	768	632	632
q8	9296	1292	1016	1016
q9	6782	5087	5096	5087
q10	6927	2360	1962	1962
q11	463	276	255	255
q12	363	366	227	227
q13	17790	3458	2962	2962
q14	229	243	212	212
q15	519	449	461	449
q16	424	421	374	374
q17	565	816	343	343
q18	7514	7150	7076	7076
q19	1796	983	531	531
q20	327	318	206	206
q21	3418	2426	2251	2251
q22	1095	1046	1024	1024
Total cold run time: 106149 ms
Total hot run time: 33836 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5410	5353	5438	5353
q2	241	312	223	223
q3	2056	2571	2251	2251
q4	1311	1728	1304	1304
q5	4162	4369	4482	4369
q6	250	174	131	131
q7	1978	1922	1794	1794
q8	2534	2457	2483	2457
q9	7370	7384	7173	7173
q10	3249	3329	2895	2895
q11	553	520	472	472
q12	935	752	626	626
q13	3313	3563	3129	3129
q14	351	307	276	276
q15	503	461	453	453
q16	485	751	1333	751
q17	1246	1571	1412	1412
q18	8013	7839	7637	7637
q19	12337	957	895	895
q20	4304	2015	1851	1851
q21	15326	4467	4460	4460
q22	1113	1022	977	977
Total cold run time: 77040 ms
Total hot run time: 50889 ms

@doris-robot

doris-robot commented Aug 8, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 161280 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1686e95d4d80743206fccc38fe13b0229f6713f2, data reload: false

reason	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:56	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
============================================
query1	991	415	421	415
query2	6513	1760	1690	1690
query3	6742	234	222	222
query4	26956	23268	22889	22889
query5	4325	647	525	525
query6	342	247	229	229
query7	4634	564	301	301
query8	288	254	240	240
query9	8613	3008	2966	2966
query10	468	364	303	303
query11	15802	15112	14772	14772
query12	182	148	144	144
query13	1669	555	416	416
query14	8737	5886	5867	5867
query15	233	201	170	170
query16	7134	705	481	481
query17	1006	814	653	653
query18	2031	470	329	329
query19	234	242	189	189
query20	153	143	142	142
query21	224	139	114	114
query22	4091	3979	3834	3834
query23	34676	34319	34250	34250
query24	8168	2394	2478	2394
query25	580	531	457	457
query26	1237	307	160	160
query27	2689	526	356	356
query28	4325	2347	2323	2323
query29	813	645	488	488
query30	303	236	204	204
query31	888	845	720	720
query32	86	76	80	76
query33	580	448	389	389
query34	863	909	525	525
query35	841	856	771	771
query36	1072	1066	936	936
query37	145	118	95	95
query38	3943	4012	3978	3978
query39	1450	1469	1525	1469
query40	245	146	137	137
query41	66	59	57	57
query42	147	125	123	123
query43	528	527	475	475
query44	1489	894	883	883
query45	209	193	185	185
query46	1042	1133	706	706
query47	1770	1835	1747	1747
query48	411	437	318	318
query49	748	533	441	441
query50	719	742	425	425
query51	4276	4207	4135	4135
query52	127	130	122	122
query53	266	309	213	213
query54	669	636	554	554
query55	93	92	87	87
query56	347	359	341	341
query57	1213	1234	1139	1139
query58	339	329	336	329
query59	2584	2579	2571	2571
query60	416	401	416	401
query61	134	121	121	121
query62	840	792	650	650
query63	259	214	214	214
query64	4404	1095	767	767
query65	4261	4055	4118	4055
query66	1221	470	340	340
query67	query68	18866	1027	1213	1027
query69	994	269	282	269
query70	1433	1143	1037	1037
query71	660	335	318	318
query72	9116	2368	4832	2368
query73	4544	822	356	356
query74	9101	8960	8776	8776
query75	7081	3248	2641	2641
query76	8338	1351	834	834
query77	1162	435	356	356
query78	query79	17875	637	582	582
query80	2038	613	496	496
query81	570	269	228	228
query82	449	171	198	171
query83	352	311	273	273
query84	307	104	88	88
query85	1581	416	369	369
query86	349	320	305	305
query87	4231	4217	4085	4085
query88	5524	2273	2291	2273
query89	512	379	323	323
query90	2628	247	253	247
query91	147	153	112	112
query92	90	74	71	71
query93	6878	1023	662	662
query94	1028	418	289	289
query95	420	343	326	326
query96	528	645	288	288
query97	2731	2802	2593	2593
query98	256	239	225	225
query99	1527	1466	1274	1274
Total cold run time: 304166 ms
Total hot run time: 161280 ms

@doris-robot

doris-robot commented Aug 8, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 33.09 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1686e95d4d80743206fccc38fe13b0229f6713f2, data reload: false

query1	0.05	0.04	0.04
query2	0.09	0.05	0.04
query3	0.24	0.07	0.08
query4	1.62	0.10	0.10
query5	0.42	0.43	0.44
query6	1.16	0.67	0.68
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.56	0.48	0.46
query10	0.53	0.53	0.54
query11	0.17	0.10	0.11
query12	0.15	0.11	0.11
query13	0.66	0.66	0.65
query14	0.94	1.25	0.93
query15	0.97	0.87	0.89
query16	0.39	0.40	0.39
query17	1.06	1.09	1.04
query18	0.22	0.20	0.20
query19	1.93	1.84	1.82
query20	0.01	0.01	0.02
query21	15.41	0.85	0.57
query22	0.79	1.10	0.82
query23	14.84	1.21	0.61
query24	6.70	2.11	0.73
query25	0.47	0.10	0.08
query26	0.59	0.17	0.13
query27	0.06	0.05	0.06
query28	9.73	0.84	0.45
query29	12.62	3.86	3.32
query30	3.10	3.06	3.05
query31	2.83	0.59	0.39
query32	3.23	0.57	0.49
query33	3.02	3.24	3.11
query34	16.08	5.33	4.97
query35	4.87	5.01	4.96
query36	0.69	0.51	0.49
query37	0.10	0.08	0.07
query38	0.07	0.05	0.04
query39	0.03	0.02	0.03
query40	0.16	0.14	0.14
query41	0.08	0.03	0.02
query42	0.04	0.04	0.03
query43	0.04	0.04	0.03
Total cold run time: 106.8 s
Total hot run time: 33.09 s

@hello-stephen

hello-stephen commented Aug 8, 2025

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 12.34% (29/235) 🎉
Increment coverage report
Complete coverage report

@doris-robot

doris-robot commented Aug 8, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 59.21% (315/532) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 58.89% (16587/28168)
Line Coverage 47.78% (150353/314647)
Region Coverage 36.67% (112640/307132)
Branch Coverage 39.63% (50055/126300)

@hello-stephen

hello-stephen commented Aug 8, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 78.01% (415/532) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.41% (22503/27642)
Line Coverage 74.06% (233017/314639)
Region Coverage 61.25% (193175/315374)
Branch Coverage 65.47% (83695/127846)

eldenmoon
eldenmoon approved these changes Aug 11, 2025
View reviewed changes

@eldenmoon eldenmoon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Aug 11, 2025
@github-actions

github-actions Bot commented Aug 11, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actions Bot commented Aug 11, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

zzzxl1993
zzzxl1993 approved these changes Aug 11, 2025
View reviewed changes

@zzzxl1993 zzzxl1993 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eldenmoon

eldenmoon commented Aug 11, 2025

Copy link
Copy Markdown
Member

skip check_coverage

@eldenmoon
eldenmoon merged commit 5b37589 into apache:master Aug 11, 2025
25 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zzzxl1993 zzzxl1993 zzzxl1993 approved these changes
@eldenmoon eldenmoon eldenmoon approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@csun5285 @hello-stephen @morrySnow @doris-robot @eldenmoon @zzzxl1993