Skip to content

[fix](auth) add auth check for manager node and query qerror REST APIs#65042

Open
CalvinKirs wants to merge 2 commits into
apache:masterfrom
CalvinKirs:http-node-admin
Open

[fix](auth) add auth check for manager node and query qerror REST APIs#65042
CalvinKirs wants to merge 2 commits into
apache:masterfrom
CalvinKirs:http-node-admin

Conversation

@CalvinKirs

Copy link
Copy Markdown
Member

Proposed changes

Several manager REST APIs under /rest/v2/manager were missing authentication and/or authorization. This PR closes those gaps.

1. Node management endpoints — missing auth + authz

POST /rest/v2/manager/node/{action}/fe, /{action}/be, /{action}/broker (operateFrontends / operateBackend / operateBroker) could add or drop FE / BE / Broker nodes without any authentication or authorization. Any caller able to reach the FE HTTP port could change cluster topology.

Added, consistent with the sibling set_config/fe and set_config/be endpoints:

ActionAuthorizationInfo authInfo = executeCheckPassword(request, response);
checkAdminAuth(authInfo.userIdentity);

2. GET /rest/v2/manager/query/qerror/{id} (getStats) — fully unauthenticated

This endpoint had neither authentication nor authorization: its method signature didn't even take HttpServletRequest/HttpServletResponse, so it could not call executeCheckPassword, and the global AuthInterceptor only covers /rest/v1/**. As a result it was reachable anonymously even with enable_all_http_auth=true, leaking per-query stats-error information.

Aligned it with the /profile and /trace_id endpoints — authenticate, then restrict non-admin users to their own queries:

executeCheckPassword(request, response);
try {
    checkAuthByUserAndQueryId(id);
} catch (AuthenticationException e) {
    return ResponseEntityBuilder.badRequest(e.getMessage());
}

Test

Added regression-test/suites/auth_p0/test_http_node_action_auth.groovy (p0,auth,nonConcurrent):

  • a non-admin user calling ADD /fe and ADD /be is rejected;
  • after grant 'admin', the request passes the auth check;
  • an unauthenticated call to /qerror/{id} is rejected.

FE compiles cleanly (build.sh --fe).

… query qerror REST APIs

The node management endpoints (POST /rest/v2/manager/node/{action}/{fe,be,broker})
allowed adding or dropping cluster nodes without any authentication or
authorization. Add executeCheckPassword + checkAdminAuth so they require an
authenticated ADMIN user, consistent with set_config/fe and set_config/be.

GET /rest/v2/manager/query/qerror/{id} (getStats) had neither authentication
nor authorization: its signature took no request/response and the global
AuthInterceptor only covers /rest/v1/**, so it was reachable anonymously even
with enable_all_http_auth=true. Add executeCheckPassword and
checkAuthByUserAndQueryId, matching the /profile and /trace_id endpoints, so a
non-admin can only read their own query stats.

Add a p0 regression test covering both gaps.
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@CalvinKirs

Copy link
Copy Markdown
Member Author

run buildall

@CalvinKirs

Copy link
Copy Markdown
Member Author

Centralize auth into one interceptor — having every v2 endpoint roll its own is a footgun.

The admin-positive assertions used ADD with 127.0.0.1 addresses, which on a
real (distributed) cluster would not match an existing node and would actually
register a phantom FE observer / BE into the editlog with no cleanup, polluting
cluster state and risking later tests.

Switch the positive path to DROP on RFC 5737 TEST-NET addresses (192.0.2.x),
which can never match a real node: it reaches the operation, returns a harmless
'does not exist' error, proves the ADMIN check passed, and mutates nothing. The
negative (non-admin) cases keep ADD since the auth check rejects them before the
node operation runs.
@CalvinKirs

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 18.18% (2/11) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29592 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4a884f960a42d7422f53311e9f5bd42e9d3a7624, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17844	4185	4062	4062
q2	2096	326	195	195
q3	10344	1426	828	828
q4	4684	470	340	340
q5	7534	877	581	581
q6	185	177	137	137
q7	792	844	632	632
q8	9430	1538	1581	1538
q9	5822	4434	4447	4434
q10	6757	1832	1520	1520
q11	513	352	318	318
q12	720	556	428	428
q13	18077	3357	2806	2806
q14	271	272	240	240
q15	q16	795	773	713	713
q17	1028	1028	1016	1016
q18	7301	5716	5453	5453
q19	1325	1224	1116	1116
q20	808	652	566	566
q21	6025	2772	2356	2356
q22	451	366	313	313
Total cold run time: 102802 ms
Total hot run time: 29592 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4441	4366	4337	4337
q2	305	312	225	225
q3	4545	4970	4403	4403
q4	2114	2192	1382	1382
q5	4465	4332	4329	4329
q6	239	181	129	129
q7	2079	1994	1646	1646
q8	2569	2218	2220	2218
q9	8061	8364	7858	7858
q10	4825	4762	4324	4324
q11	597	428	383	383
q12	753	762	543	543
q13	3299	3568	2943	2943
q14	304	317	277	277
q15	q16	729	747	639	639
q17	1373	1352	1459	1352
q18	7882	7445	7197	7197
q19	1202	1121	1121	1121
q20	2226	2220	1921	1921
q21	5316	4709	4482	4482
q22	513	462	424	424
Total cold run time: 57837 ms
Total hot run time: 52133 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29946 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 223b563cc13d17e24eafacc34e7378f5adb6d42d, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17606	4102	4084	4084
q2	2047	325	192	192
q3	10308	1513	826	826
q4	4688	470	339	339
q5	7531	857	586	586
q6	186	178	136	136
q7	785	852	644	644
q8	9341	1613	1627	1613
q9	5670	4457	4454	4454
q10	6752	1848	1531	1531
q11	510	348	319	319
q12	705	560	444	444
q13	18101	3456	2813	2813
q14	274	263	252	252
q15	q16	802	789	727	727
q17	1050	937	1035	937
q18	7154	5695	5722	5695
q19	1305	1264	1025	1025
q20	762	656	535	535
q21	6020	2731	2487	2487
q22	435	370	307	307
Total cold run time: 102032 ms
Total hot run time: 29946 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4351	4282	4270	4270
q2	288	325	217	217
q3	4633	4935	4396	4396
q4	2094	2189	1394	1394
q5	4435	4305	4347	4305
q6	236	176	128	128
q7	1812	2158	1713	1713
q8	2641	2192	2177	2177
q9	8088	8063	7811	7811
q10	4805	4759	4337	4337
q11	580	411	397	397
q12	759	790	529	529
q13	3351	3676	2944	2944
q14	292	291	270	270
q15	q16	723	755	652	652
q17	1354	1322	1310	1310
q18	7891	7313	7403	7313
q19	1167	1093	1118	1093
q20	2228	2223	1958	1958
q21	5316	4624	4492	4492
q22	511	444	402	402
Total cold run time: 57555 ms
Total hot run time: 52108 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 175487 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4a884f960a42d7422f53311e9f5bd42e9d3a7624, data reload: false

query5	4307	639	478	478
query6	462	224	206	206
query7	4858	608	352	352
query8	343	188	171	171
query9	8773	4171	4132	4132
query10	476	367	288	288
query11	5934	2504	2133	2133
query12	168	101	103	101
query13	1262	622	442	442
query14	6284	5327	5014	5014
query14_1	4309	4313	4309	4309
query15	220	204	180	180
query16	1050	486	468	468
query17	1279	769	597	597
query18	2774	481	365	365
query19	235	197	162	162
query20	115	113	110	110
query21	259	165	136	136
query22	13638	13572	13457	13457
query23	17480	16720	16203	16203
query23_1	16313	16390	16318	16318
query24	7515	1803	1325	1325
query24_1	1319	1335	1328	1328
query25	575	473	403	403
query26	1341	352	221	221
query27	2499	597	411	411
query28	4366	2082	2070	2070
query29	1087	633	509	509
query30	350	261	227	227
query31	1160	1087	1008	1008
query32	111	63	62	62
query33	541	326	258	258
query34	1169	1171	658	658
query35	785	785	677	677
query36	1441	1395	1243	1243
query37	161	114	96	96
query38	1888	1728	1719	1719
query39	941	922	877	877
query39_1	870	908	866	866
query40	253	162	141	141
query41	63	63	63	63
query42	93	92	92	92
query43	321	326	295	295
query44	1463	816	781	781
query45	213	208	181	181
query46	1067	1209	743	743
query47	2380	2386	2270	2270
query48	397	393	303	303
query49	576	417	314	314
query50	1069	420	337	337
query51	4531	4371	4347	4347
query52	84	88	74	74
query53	264	286	195	195
query54	284	231	213	213
query55	75	70	68	68
query56	316	288	308	288
query57	1472	1401	1365	1365
query58	299	263	250	250
query59	1543	1626	1485	1485
query60	312	272	258	258
query61	156	145	163	145
query62	735	660	592	592
query63	247	206	210	206
query64	2494	798	617	617
query65	4871	4773	4787	4773
query66	1772	512	427	427
query67	29878	29766	29590	29590
query68	3242	1630	985	985
query69	428	307	267	267
query70	1099	973	1030	973
query71	363	323	332	323
query72	3113	2625	2345	2345
query73	920	757	448	448
query74	5143	4960	4753	4753
query75	2649	2596	2228	2228
query76	2383	1191	805	805
query77	354	407	283	283
query78	12545	12539	11802	11802
query79	1524	1184	781	781
query80	1305	540	460	460
query81	529	304	286	286
query82	771	162	125	125
query83	381	330	294	294
query84	278	162	133	133
query85	1051	614	522	522
query86	480	300	287	287
query87	1841	1835	1760	1760
query88	3771	2845	2816	2816
query89	454	414	359	359
query90	1946	207	202	202
query91	207	193	169	169
query92	69	62	62	62
query93	1777	1549	1094	1094
query94	854	349	327	327
query95	798	493	577	493
query96	1077	895	394	394
query97	2702	2712	2563	2563
query98	219	209	200	200
query99	1186	1164	1049	1049
Total cold run time: 261718 ms
Total hot run time: 175487 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 174181 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 223b563cc13d17e24eafacc34e7378f5adb6d42d, data reload: false

query5	4328	659	498	498
query6	454	213	214	213
query7	4879	619	346	346
query8	333	186	170	170
query9	8731	4003	4050	4003
query10	441	362	301	301
query11	5705	2357	2179	2179
query12	159	101	104	101
query13	1269	621	425	425
query14	6270	5336	5003	5003
query14_1	4316	4298	4283	4283
query15	220	204	183	183
query16	1002	483	446	446
query17	1153	747	615	615
query18	2479	502	351	351
query19	210	197	154	154
query20	116	109	108	108
query21	234	161	140	140
query22	13699	13596	13398	13398
query23	17409	16620	16200	16200
query23_1	16371	16272	16251	16251
query24	7684	1770	1260	1260
query24_1	1348	1319	1295	1295
query25	579	468	407	407
query26	1331	362	221	221
query27	2569	609	396	396
query28	4392	2073	2057	2057
query29	1088	650	501	501
query30	336	258	228	228
query31	1113	1099	995	995
query32	114	64	63	63
query33	538	338	266	266
query34	1166	1137	674	674
query35	824	796	678	678
query36	1384	1400	1262	1262
query37	151	109	94	94
query38	1878	1716	1672	1672
query39	945	941	905	905
query39_1	873	888	887	887
query40	250	165	140	140
query41	66	63	63	63
query42	94	92	91	91
query43	317	326	283	283
query44	1438	779	780	779
query45	214	196	174	174
query46	1081	1223	749	749
query47	2407	2326	2206	2206
query48	372	413	279	279
query49	574	431	313	313
query50	1051	430	341	341
query51	4429	4386	4301	4301
query52	86	87	78	78
query53	264	286	208	208
query54	279	235	227	227
query55	74	74	68	68
query56	303	297	299	297
query57	1437	1401	1319	1319
query58	281	250	261	250
query59	1598	1700	1411	1411
query60	309	273	259	259
query61	151	154	149	149
query62	699	658	594	594
query63	247	209	204	204
query64	2526	769	624	624
query65	4895	4793	4748	4748
query66	1831	510	392	392
query67	29804	29749	29649	29649
query68	3227	1588	944	944
query69	409	302	268	268
query70	1053	959	977	959
query71	375	333	303	303
query72	2897	2658	2372	2372
query73	837	788	450	450
query74	5108	4955	4758	4758
query75	2621	2601	2236	2236
query76	2308	1199	788	788
query77	360	378	291	291
query78	12450	12616	11993	11993
query79	1460	1210	725	725
query80	654	547	454	454
query81	471	330	281	281
query82	559	160	122	122
query83	401	317	289	289
query84	274	158	129	129
query85	939	599	527	527
query86	376	287	278	278
query87	1853	1836	1804	1804
query88	3715	2799	2807	2799
query89	449	403	354	354
query90	2001	207	196	196
query91	213	188	159	159
query92	62	62	54	54
query93	1614	1523	936	936
query94	543	348	325	325
query95	813	497	477	477
query96	1048	815	362	362
query97	2714	2759	2566	2566
query98	218	210	204	204
query99	1197	1151	1026	1026
Total cold run time: 258812 ms
Total hot run time: 174181 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 25.5 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4a884f960a42d7422f53311e9f5bd42e9d3a7624, data reload: false

query1	0.01	0.00	0.00
query2	0.10	0.05	0.05
query3	0.27	0.13	0.14
query4	1.61	0.15	0.14
query5	0.24	0.22	0.22
query6	1.31	1.09	1.12
query7	0.03	0.01	0.01
query8	0.06	0.04	0.04
query9	0.40	0.33	0.32
query10	0.54	0.58	0.55
query11	0.20	0.15	0.14
query12	0.20	0.15	0.15
query13	0.48	0.47	0.49
query14	1.02	1.04	1.03
query15	0.62	0.60	0.59
query16	0.33	0.33	0.33
query17	1.11	1.10	1.12
query18	0.23	0.21	0.20
query19	2.01	1.94	1.94
query20	0.02	0.01	0.01
query21	15.47	0.22	0.14
query22	4.89	0.06	0.05
query23	16.07	0.31	0.13
query24	3.03	0.44	0.37
query25	0.14	0.07	0.05
query26	1.02	0.21	0.16
query27	0.03	0.05	0.04
query28	3.56	0.98	0.57
query29	12.48	4.35	3.50
query30	0.27	0.15	0.16
query31	2.78	0.60	0.30
query32	3.23	0.60	0.50
query33	3.16	3.25	3.18
query34	15.58	4.22	3.51
query35	3.56	3.54	3.56
query36	0.59	0.43	0.44
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.16	0.15
query41	0.09	0.03	0.04
query42	0.04	0.03	0.04
query43	0.04	0.04	0.03
Total cold run time: 97.18 s
Total hot run time: 25.5 s

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 25.44 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 223b563cc13d17e24eafacc34e7378f5adb6d42d, data reload: false

query1	0.01	0.01	0.01
query2	0.10	0.08	0.05
query3	0.25	0.14	0.14
query4	1.60	0.14	0.14
query5	0.24	0.21	0.22
query6	1.22	1.07	1.08
query7	0.04	0.01	0.01
query8	0.07	0.04	0.04
query9	0.38	0.30	0.30
query10	0.54	0.60	0.56
query11	0.19	0.14	0.14
query12	0.18	0.14	0.15
query13	0.50	0.49	0.48
query14	1.01	1.02	1.01
query15	0.64	0.60	0.59
query16	0.31	0.34	0.31
query17	1.07	1.06	1.10
query18	0.22	0.21	0.21
query19	2.00	1.89	1.96
query20	0.02	0.01	0.02
query21	15.45	0.22	0.15
query22	4.87	0.05	0.05
query23	16.15	0.31	0.12
query24	2.91	0.43	0.33
query25	0.11	0.06	0.04
query26	0.73	0.20	0.16
query27	0.04	0.04	0.03
query28	3.52	0.94	0.55
query29	12.49	4.49	3.63
query30	0.28	0.16	0.15
query31	2.77	0.59	0.32
query32	3.24	0.61	0.49
query33	3.26	3.18	3.21
query34	15.77	4.29	3.50
query35	3.52	3.56	3.58
query36	0.56	0.45	0.44
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.15
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 96.77 s
Total hot run time: 25.44 s

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 12.12% (4/33) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants