Skip to content

[fix](arrow-flight) Let GetFlightInfo pass the session layer's Flight status through again - #68221

Merged
morningman merged 1 commit into
apache:masterfrom
morningman:fix-flight-getinfo-passthrough
Sep 20, 2026
Merged

morningman merged 1 commit into
apache:masterfrom
morningman:fix-flight-getinfo-passthrough

Conversation

@morningman

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: #67883, #67820, #67966, #68101

Problem Summary:

DorisFlightSqlProducer.getFlightInfoStatement used to rethrow a FlightRuntimeException as is (#67883), so the status the session layer chose reaches the client: UNAVAILABLE from the session's command lock and UNAUTHENTICATED from a closed session (#67900, #67966), and whatever a refused session is answered with. #67820 rewrote that catch block into a catch-all that lets only its two incremental-window errors through (by doris-error-code metadata) and wraps every other FlightRuntimeException as INTERNAL: get flight info statement failed, <message> -- the wrapping #67883 had removed. The producer's other entry points (setSessionOptions, streamMetadata) still let the status through and say they do it "as in getFlightInfoStatement".

This PR restores the passthrough ahead of the catch-all. The window errors are FlightRuntimeExceptions built by queryFailure, so #67820's metadata special case is subsumed and removed; a non-Flight failure is still wrapped as INTERNAL with the same message.

Found through #68101: its test_connection_quota asserts the RESOURCE_EXHAUSTED the connection pool answers a refused Flight session with, and since the pipelines compile a PR merged into master, every run after #67820 landed got the wrapped INTERNAL instead.

Release note

None

Check List (For Author)

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

… status through again

apache#67883 made DorisFlightSqlProducer.getFlightInfoStatement rethrow a FlightRuntimeException
as is, so that the status the session layer chose reaches the client: UNAVAILABLE from the
session's command lock and UNAUTHENTICATED from a closed session (apache#67900, apache#67966), and
whatever a refused session is answered with. apache#67820 rewrote that catch block into a
catch-all that lets only its two incremental-window errors (by doris-error-code metadata)
through and wraps every other FlightRuntimeException as
"INTERNAL: get flight info statement failed, <message>" -- the wrapping apache#67883 had removed.
The other entry points of the producer (setSessionOptions, streamMetadata) still let the
status through and say they do "as in getFlightInfoStatement".

Restore the FlightRuntimeException passthrough ahead of the catch-all. The window errors
are FlightRuntimeExceptions built by queryFailure, so the metadata special case is
subsumed and goes. The unit tests apache#67820 added for the wrapping now assert the
passthrough; the one for a non-Flight failure keeps asserting the INTERNAL wrapper.

Found through apache#68101, whose test_connection_quota asserts the RESOURCE_EXHAUSTED the
connection pool answers a refused Flight session with: the pipelines compile a PR merged
into master, and every run after apache#67820 landed received the wrapped INTERNAL instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 27148 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 06952c59513fcfa6dffe575801cd7020294fd778, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17707	3873	3785	3785
q2	2241	335	281	281
q3	10044	1388	747	747
q4	4685	468	351	351
q5	7475	812	553	553
q6	176	168	135	135
q7	717	786	593	593
q8	9312	1527	1560	1527
q9	5405	4123	4102	4102
q10	6734	1580	1333	1333
q11	434	277	251	251
q12	631	409	294	294
q13	18059	2627	1996	1996
q14	260	255	235	235
q15	q16	724	710	659	659
q17	1709	1183	980	980
q18	6477	5597	5505	5505
q19	1223	1198	1000	1000
q20	489	391	264	264
q21	5542	2573	2255	2255
q22	420	341	302	302
Total cold run time: 100464 ms
Total hot run time: 27148 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4120	4053	4045	4045
q2	760	613	588	588
q3	4506	4858	4382	4382
q4	2215	2288	1408	1408
q5	4168	4119	4046	4046
q6	224	172	126	126
q7	1698	1613	1424	1424
q8	2127	1868	2349	1868
q9	7423	7518	7300	7300
q10	4267	4274	3873	3873
q11	551	422	399	399
q12	756	730	540	540
q13	2443	2816	2181	2181
q14	298	310	285	285
q15	q16	687	730	656	656
q17	7859	7268	7183	7183
q18	11877	11135	11805	11135
q19	1191	1078	1071	1071
q20	2291	2259	1955	1955
q21	5768	4836	4804	4804
q22	510	465	423	423
Total cold run time: 65739 ms
Total hot run time: 59692 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 153893 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 06952c59513fcfa6dffe575801cd7020294fd778, data reload: false

query5	4347	622	461	461
query6	458	192	168	168
query7	4826	570	303	303
query8	325	172	168	168
query9	8787	3946	3950	3946
query10	439	314	254	254
query11	5815	2178	2013	2013
query12	169	111	96	96
query13	1260	598	382	382
query14	6553	4583	4271	4271
query14_1	4024	4004	4037	4004
query15	209	193	190	190
query16	971	492	460	460
query17	930	683	537	537
query18	2429	468	352	352
query19	204	178	138	138
query20	105	103	105	103
query21	221	148	119	119
query22	13014	12999	12832	12832
query23	15612	14423	14084	14084
query23_1	14161	14043	14100	14043
query24	7575	1734	1206	1206
query24_1	1250	1245	1238	1238
query25	554	425	366	366
query26	1260	342	177	177
query27	2654	565	359	359
query28	4571	1961	1928	1928
query29	1066	616	469	469
query30	317	236	199	199
query31	904	758	627	627
query32	146	96	95	95
query33	517	347	258	258
query34	1187	1136	629	629
query35	744	753	654	654
query36	807	794	698	698
query37	151	104	96	96
query38	1847	1769	1676	1676
query39	705	690	643	643
query39_1	649	670	656	656
query40	224	128	118	118
query41	74	69	69	69
query42	95	96	92	92
query43	347	349	296	296
query44	1420	695	698	695
query45	189	186	173	173
query46	1054	1173	729	729
query47	1525	1496	1380	1380
query48	375	411	303	303
query49	608	418	347	347
query50	970	336	238	238
query51	10196	10507	10421	10421
query52	87	87	75	75
query53	244	268	181	181
query54	252	212	186	186
query55	83	74	69	69
query56	235	230	221	221
query57	1409	1413	1358	1358
query58	245	221	233	221
query59	1971	2071	1796	1796
query60	267	250	224	224
query61	150	146	144	144
query62	402	320	261	261
query63	220	179	172	172
query64	2846	1006	856	856
query65	4005	3952	3933	3933
query66	1829	420	307	307
query67	19900	19822	20003	19822
query68	3130	1474	897	897
query69	407	300	269	269
query70	960	867	863	863
query71	305	239	226	226
query72	2841	2495	1946	1946
query73	805	772	427	427
query74	4673	4503	4289	4289
query75	2388	2334	1947	1947
query76	2293	1168	715	715
query77	378	414	320	320
query78	9190	9023	8535	8535
query79	1436	1124	746	746
query80	1194	465	382	382
query81	505	284	237	237
query82	618	163	125	125
query83	351	267	248	248
query84	305	142	111	111
query85	873	470	377	377
query86	370	237	229	229
query87	2010	1984	1826	1826
query88	3674	2692	2706	2692
query89	363	294	244	244
query90	1897	186	179	179
query91	167	156	125	125
query92	106	92	92	92
query93	1457	1418	886	886
query94	650	339	288	288
query95	655	366	421	366
query96	1058	867	334	334
query97	2478	2410	2308	2308
query98	197	186	182	182
query99	728	721	614	614
Total cold run time: 241742 ms
Total hot run time: 153893 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 23.91 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 06952c59513fcfa6dffe575801cd7020294fd778, data reload: false

query1	0.00	0.00	0.00
query2	0.10	0.05	0.05
query3	0.26	0.14	0.14
query4	1.61	0.15	0.14
query5	0.25	0.23	0.22
query6	1.17	0.93	0.96
query7	0.04	0.01	0.00
query8	0.06	0.04	0.04
query9	0.39	0.33	0.35
query10	0.55	0.56	0.58
query11	0.20	0.14	0.15
query12	0.19	0.14	0.15
query13	0.48	0.47	0.47
query14	0.96	0.95	0.96
query15	0.60	0.58	0.57
query16	0.32	0.32	0.35
query17	1.11	1.10	1.07
query18	0.21	0.20	0.20
query19	1.99	1.86	1.89
query20	0.02	0.01	0.01
query21	15.45	0.22	0.14
query22	4.88	0.06	0.05
query23	16.12	0.31	0.13
query24	2.93	0.41	0.33
query25	0.10	0.04	0.03
query26	0.76	0.21	0.16
query27	0.04	0.04	0.04
query28	3.49	0.74	0.35
query29	12.46	4.06	3.19
query30	0.28	0.16	0.15
query31	2.77	0.56	0.31
query32	3.23	0.58	0.49
query33	3.19	3.16	3.25
query34	15.63	3.98	3.27
query35	3.25	3.21	3.22
query36	0.56	0.45	0.42
query37	0.08	0.06	0.07
query38	0.05	0.04	0.03
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 96.16 s
Total hot run time: 23.91 s

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@morningman

Copy link
Copy Markdown
Contributor Author

/review

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review opinion: Approve. No blocking or non-blocking correctness finding survived the complete static review.

Critical checkpoint conclusions:

  • Goal and proof: the patch restores passthrough of an existing FlightRuntimeException, so the session/command layer status, description, metadata, and cause reach the client unchanged. The tests assert object identity for ordinary Flight statuses and metadata-bearing business errors, while retaining a negative control that non-Flight failures are wrapped as INTERNAL.
  • Scope and clarity: the production change is limited to catch classification plus the directly corresponding unit expectations; the obsolete ErrorCode dependency and metadata allowlist are removed.
  • Concurrency and lifecycle: no locking or state lifecycle is changed. The command-lock CANCELLED/UNAVAILABLE and torn-down-session UNAUTHENTICATED paths retain their intended meaning; query-body failures still run deferred-query cancellation/finalization before queryFailure constructs the outgoing status, and callCommand still restores thread-local state and unlocks in finally.
  • Compatibility and parallel paths: this restores the behavior that other producer entry points and prepared-statement paths already follow. Incremental-window UNAVAILABLE metadata remains intact, ordinary query failures remain INTERNAL, and no FE-BE protocol, serialization, storage, configuration, persistence, transaction, or data-write compatibility surface changes.
  • Tests and observability: the boundary-focused assertions compose with existing real command-lock, teardown, and window-error tests. The pending connection-pool PR owns the real RESOURCE_EXHAUSTED refusal and its integration coverage. Expected client statuses are no longer hidden by a redundant wrapper; no material performance or observability regression was found.
  • User focus: no additional focus points were supplied.

This was a static-only review as required by the runner instructions; I did not run builds or tests. Current CI reports FE UT and CheckStyle passing, but those are CI evidence rather than independent execution here.

Convergence: the main scan plus both normal reviewers and the separate risk-focused reviewer covered both changed files and their upstream/downstream paths. All three reviewers returned NO_NEW_VALUABLE_FINDINGS in Round 1, all initial risks were resolved with code evidence, and no inline comments were proposed.

Reviewed exact head: 06952c59513fcfa6dffe575801cd7020294fd778 (base 979e053f0802fbc48f83dafe7f713c02968415a8).

@morningman
morningman merged commit 1f41d42 into apache:master Sep 20, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants