Secure cluster control of UAVs under DoS attacks based on APF and DDPG algorithm

doi:10.11772/j.issn.1001-9081.2024040464

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (4): 1241-1248.DOI: 10.11772/j.issn.1001-9081.2024040464

• Cyber security • Previous Articles Next Articles

Secure cluster control of UAVs under DoS attacks based on APF and DDPG algorithm

Bingquan LIN¹, Lei LIU¹(), Huafeng LI², Chen LIU¹

^1.School of Mathematics，Hohai University，Nanjing Jiangsu 211100，China
^2.Department of Computer Science and Technology，Tangshan Normal University，Tangshan Hebei 063000，China

Received:2024-04-16 Revised:2024-08-04 Accepted:2024-08-08 Online:2025-04-08 Published:2025-04-10
Contact: Lei LIU
About author:LIN Bingquan， born in 2000， M. S. candidate. His research interests include reinforcement learning.
LI Huafeng， born in 1979， M. S. His research interests include cybersecurity.
LIU Chen， born in 1993， Ph. D. candidate. His research interests include cyber attack， optimal control.
Supported by:
This work is partially supported by Natural Science Foundation of Hebei Province(A2023209002);Anhui Provincial Key Laboratory Fund Project(KLAHEI18018);Open Fund of Key Laboratory of Ministry of Education(Scip20240111)

DoS攻击下基于APF和DDPG算法的无人机安全集群控制

林柄权¹, 刘磊¹(), 李华峰², 刘晨¹

^1.河海大学数学学院，南京 211100
^2.唐山师范学院计算机科学技术系，河北唐山 063000

通讯作者: 刘磊
作者简介:林柄权（2000—），男，湖南岳阳人，硕士研究生，主要研究方向：强化学习；
李华峰（1979—），男，吉林白山人，硕士，主要研究方向：网络安全；
刘晨（1993—），男，安徽淮南人，博士研究生，主要研究方向：网络攻击、最优控制。
基金资助:
河北省自然科学基金资助项目(A2023209002);安徽省重点实验室基金资助项目(KLAHEI18018);教育部重点实验室开放基金资助项目(Scip20240111)

Abstract

Abstract:

Addressing the issues of communication obstruction and unpredictable motion trajectories of Unmanned Aerial Vehicles （UAVs） under Denial of Service （DoS） attacks， research was conducted on the secure cluster control strategy for multi-UAV during DoS attacks within a framework that integrates Artificial Potential Field （APF） and Deep Deterministic Policy Gradient （DDPG） algorithm. Firstly， Hping3 was utilized to detect DoS attacks on all UAVs， thereby determining the network environment of the UAV cluster in real time. Secondly， when no attack was detected， the traditional APF was employed for cluster flight. After detecting attacks， the targeted UAVs were marked as dynamic obstacles while other UAV switched to control strategies generated by DDPG algorithm. Finally， with the proposed framework， the cooperation and advantage complementary of APF and DDPG were realized， and the effectiveness of the DDPG algorithm was validated through simulation in Gazebo. Simulation results indicate that Hping3 can detect the UAVs under attack in real time， and other normal UAVs can avoid obstacles stably after switching to DDPG algorithm， so as to ensure cluster security； the success rate of the switching obstacle avoidance strategy during DoS attacks is 72.50%， significantly higher than that of the traditional APF （31.25%）， and the switching strategy converges gradually， demonstrating a pretty stability； the trained DDPG obstacle avoidance strategy exhibits a degree of generalization， capable of completing tasks stably with 1 to 2 unknown obstacles appeared in the environment.

Key words: Unmanned Aerial Vehicle (UAV) cluster, Artificial Potential Field (APF), Deep Deterministic Policy Gradient (DDPG), switching strategy, cybersecurity

摘要：

针对拒绝服务（DoS）攻击下无人机（UAV）通信阻塞、运动轨迹不可预测的问题，在人工势场法（APF）和深度确定性策略梯度（DDPG）融合框架下研究DoS攻击期间的多UAV安全集群控制策略。首先，使用Hping3对所有UAV进行DoS攻击检测，以实时确定UAV集群的网络环境；其次，当未检测到攻击时，采用传统的APF进行集群飞行；再次，在检测到攻击后，将被攻击的UAV标记为动态障碍物，而其他UAV切换为DDPG算法生成的控制策略；最后，所提框架实现APF和DDPG的协同配合及优势互补，并通过在Gazebo中进行仿真实验验证DDPG算法的有效性。仿真实验结果表明，Hping3能实时检测出被攻击的UAV，且其他正常UAV切换为DDPG算法后能稳定避开障碍物，从而保障集群安全；在DoS攻击期间，采用切换避障策略的成功率为72.50%，远高于传统APF的31.25%，且切换策略逐渐收敛，表现出较好的稳定性；训练后的DDPG避障策略具有一定泛化性，当环境中出现1~2个未知障碍物时仍能稳定完成任务。

关键词: 无人机集群, 人工势场法, 深度确定性策略梯度, 切换策略, 网络安全

CLC Number:

TP301.6

Bingquan LIN, Lei LIU, Huafeng LI, Chen LIU. Secure cluster control of UAVs under DoS attacks based on APF and DDPG algorithm[J]. Journal of Computer Applications, 2025, 45(4): 1241-1248.

林柄权, 刘磊, 李华峰, 刘晨. DoS攻击下基于APF和DDPG算法的无人机安全集群控制[J]. 《计算机应用》唯一官方网站, 2025, 45(4): 1241-1248.

Figures/Tables 17

Fig. 1 Quadcopter and airframe coordinate system

Tab. 1 Quadcopter parameters

参数	描述	设置值	单位
$m$	质量	0.958	$k g$
$I x$	$x$ 轴上的转动惯量	0.025 17	$k g · m 2$
$I y$	$y$ 轴上的转动惯量	0.025 17	$k g · m 2$
$I z$	$z$ 轴上的转动惯量	0.041 35	$k g · m 2$
$L u$	臂长	0.2	$m$
$g$	重力加速度	9.81	$m / s 2$

Tab. 1 Quadcopter parameters

参数	描述	设置值	单位
$m$	质量	0.958	$k g$
$I x$	$x$ 轴上的转动惯量	0.025 17	$k g · m 2$
$I y$	$y$ 轴上的转动惯量	0.025 17	$k g · m 2$
$I z$	$z$ 轴上的转动惯量	0.041 35	$k g · m 2$
$L u$	臂长	0.2	$m$
$g$	重力加速度	9.81	$m / s 2$

Fig. 2 Process of detecting DoS attack

Fig. 3 Neural network relationship diagram

Tab. 2 Parameters setting for DoS attacks

参数	设置值
Nmap扫描格式	nmap-Pn
DoS攻击强度	hping3-c 100-d 100-S--flood
DoS攻击检测	ping

Tab. 3 DDPG algorithm training parameters

参数	设置值
折扣因子	0.9
Actor网络学习率	0.000 5
Critic网络学习率	0.001
采样池中的样本数	64
经验回放池中的样本数	15 000
软更新率	0.005
探索噪声	0.5
回合中最大运行步长	150

Fig. 4 Two dimensional schematic diagram of UAV obstacle avoidance

Fig. 5 Gazebo simulation environment

Fig. 6 Normal round-trip time

Fig. 7 Round-trip time during DoS attack

Tab. 4 Changes in two indicators during normal communication and DoS attack periods

时段	网络延迟/ms	丢包率/%
正常通信	0.323	0.000 0
DoS攻击期间	224.033	90.724 6

Fig. 8 Reward for single UAV obstacle avoidance task

Tab. 5 UAV communication topology

无人机	无人机1	无人机2	无人机3	无人机4
无人机1		0	1	1
无人机2	0		0	0
无人机3	1	0		1
无人机4	1	0	1

Fig. 9 Reward for multi-UAV obstacle avoidance task

Fig. 10 UAV generalization test

Fig. 11 Convergence and divergence distribution curves of traditional strategy and switching strategy

Fig. 12 Comparison of success rates of traditional strategy and switching strategy

References 17

1	LYU M， ZHAO Y， HUANG C， et al. Unmanned aerial vehicles for search and rescue： a survey［J］. Remote Sensing， 2023， 15（13）： No.3266.
2	HU J， NIU H， CARRASCO J， et al. Fault-tolerant cooperative navigation of networked UAV swarms for forest fire monitoring［J］. Aerospace Science and Technology， 2022， 123： No.107494.
3	MESA A N， STRAGER M P， GRUSHECKY S T， et al. Using unmanned aerial vehicles to evaluate revegetation success on natural gas pipelines［J］. Environmental Management， 2023， 72（3）： 671-681.
4	WANG Y， SHAN M， WANG D. Motion capability analysis for multiple fixed-wing UAV formations with speed and heading rate constraints［J］. IEEE Transactions on Control of Network Systems， 2020， 7（2）： 977-989.
5	LI W， CHEN Y， XIANG L. Cooperative path planning of UAV formation based on improved artificial potential field［C］// Proceedings of the IEEE 2nd International Conference on Electronic Technology， Communication and Information. Piscataway： IEEE， 2022： 636-641.
6	YU C， CHEN X. Leader-follower formation for UAVs with FOVs constraint［C］// Proceedings of the 22nd IEEE International Conference on Industrial Technology. Piscataway： IEEE， 2021： 1119-1124.
7	鲜斌，宋宁. 基于模型预测控制与改进人工势场法的多无人机路径规划［J］. 控制与决策， 2024， 39（7）： 2133-2141.
	XIAN B， SONG N. A multiple UAVs path planning method based on model predictive control and improved artificial potential field［J］. Control and Decision， 2024， 39（7）： 2133-2141.
8	WANG N， DAI J， YING J. UAV formation obstacle avoidance control algorithm based on improved artificial potential field and consensus［J］. International Journal of Aeronautical and Space Sciences， 2021， 22（6）： 1413-1427.
9	YAN C， WANG C， XIANG X， et al. Deep reinforcement learning of collision-free flocking policies for multiple fixed-wing UAVs using local situation maps［J］. IEEE Transactions on Industrial Informatics， 2022， 18（2）： 1260-1270.
10	李湛，薛喜地，杨学博，等. 跨传感器异步迁移学习的室内单目无人机避障［J］. 宇航学报， 2020， 41（6）：811-819.
	LI Z， XUE X D， YANG X B， et al. Collision avoidance for indoor monocular UAV using cross-sensor asynchronous transfer learning［J］. Journal of Astronautics， 2020， 41（6）： 811-819.
11	PENG J， LV B， ZHANG L， et al. An improved DDPG algorithm for UAV navigation in large-scale complex environments［C］// Proceedings of the 2023 IEEE Aerospace Conference. Piscataway： IEEE， 2023： 1-11.
12	杨秀霞，高恒杰，刘伟，等. 基于LSTM深度强化学习的UAV反应式避障方法［J］. 飞行力学， 2022， 40（4）： 47-55， 62.
	YANG X X， GAO H J， LIU W， et al. UAV deep reinforcement learning reactive obstacle avoidance method based on LSTM［J］. Flight Dynamics， 2022， 40（4）： 47-55， 62.
13	黄号，马文卉，李家诚，等. 未知环境下无人机编队智能避障控制方法［J］. 清华大学学报（自然科学版）， 2024， 64（2）：358-369.
	HUANG H， MA W H， LI J C， et al. Intelligent obstacle avoidance control method for unmanned aerial vehicle formations in unknown environments［J］. Journal of Tsinghua University （Science and Technology）， 2024， 64（2）： 358-369.
14	ALREFAEI F， ALZAHRANI A， SONG H， et al. A survey on the jamming and spoofing attacks on the unmanned aerial vehicle networks［C］// Proceedings of the 2022 IEEE International IOT， Electronics and Mechatronics Conference. Piscataway： IEEE， 2022： 1-7.
15	LY B， LY R. Cybersecurity in Unmanned Aerial Vehicles （UAVs）［J］. Journal of Cyber Security Technology， 2021， 5（2）： 120-137.
16	ZHANG D， SHEN Y P， ZHOU S Q， et al. Distributed secure platoon control of connected vehicles subject to DoS attack： theory and application［J］. IEEE Transactions on Systems， Man， and Cybernetics： Systems， 2021， 51（11）： 7269-7278.
17	WANG X， PARK J H， YANG H. An improved protocol to consensus of delayed MASs with UNMS and aperiodic DoS cyber-attacks［J］. IEEE Transactions on Network Science and Engineering， 2021， 8（3）： 2506-2516.

[1]	Haijun GENG, Yun DONG, Zhiguo HU, Haotian CHI, Jing YANG, Xia YIN. Encrypted traffic classification method based on Attention-1DCNN-CE [J]. Journal of Computer Applications, 2025, 45(3): 872-882.
[2]	Rui TANG, Chuanlin PANG, Ruizhi ZHANG, Chuan LIU, Shibo YUE. DDPG-based resource allocation in D2D communication-empowered cellular network [J]. Journal of Computer Applications, 2024, 44(5): 1562-1569.
[3]	Yu WANG, Zhihui GUAN, Yuanpeng LI. Distributed UAV cluster pursuit decision-making based on trajectory prediction and MADDPG [J]. Journal of Computer Applications, 2024, 44(11): 3623-3628.
[4]	Zhengkai DING, Qiming FU, Jianping CHEN, You LU, Hongjie WU, Nengwei FANG, Bin XING. Ultra-short-term photovoltaic power prediction by deep reinforcement learning based on attention mechanism [J]. Journal of Computer Applications, 2023, 43(5): 1647-1654.
[5]	Qing TAN, Hui LI, Haolin WU, Zhuang WANG, Shuchao DENG. Intrinsic curiosity method based on reward prediction error [J]. Journal of Computer Applications, 2022, 42(6): 1822-1828.
[6]	Shaobin DENG, Jun ZHU, Xiaofeng ZHOU, Shuai LI, Shurui LIU. Industrial process control method based on local policy interaction exploration-based deep deterministic policy gradient [J]. Journal of Computer Applications, 2022, 42(5): 1642-1648.
[7]	WANG Jianping, WANG Gang, MAO Xiaobin, MA Enqi. Motion control method of two-link manipulator based on deep reinforcement learning [J]. Journal of Computer Applications, 2021, 41(6): 1799-1804.
[8]	LI Wei, JIN Shijun. Optimal path convergence method based on artificial potential field method and informed sampling [J]. Journal of Computer Applications, 2021, 41(10): 2912-2918.
[9]	DING Jiaru, DU Changping, ZHAO Yao, YIN Dengyu. Path planning algorithm for unmanned aerial vehicles based on improved artificial potential field [J]. Journal of Computer Applications, 2016, 36(1): 287-290.

Secure cluster control of UAVs under DoS attacks based on APF and DDPG algorithm

DoS攻击下基于APF和DDPG算法的无人机安全集群控制

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 17

References 17

Related Articles 9

Recommended Articles

Metrics