Short-range UAV air combat maneuver decision-making via finite tolerance pigeon-inspired optimization

doi:10.11772/j.issn.1001-9081.2023121837

Abstract

Abstract:

Due to the rapid change of the situation during the confrontation， the autonomous maneuver decision-making for short-range Unmanned Aerial Vehicle （UAV） air combat is difficult and complex， which is a difficult point in air combat. To address this issue， a short-range UAV air combat maneuver decision-making method based on Finite Tolerance Pigeon-Inspired Optimization （FTPIO） algorithm was proposed. Two parts were designed in the proposed method： opponent action prediction based on the maneuver library and optimization solution of maneuver control and execution time based on FTPIO algorithm. To improve the global exploration ability of the basic Pigeon-Inspired Optimization （PIO） algorithm， the finite tolerance strategy was introduced. When the pigeon individual failed to find a better solution in several iterations， its attributes would been reset to avoid falling into the local optimal trap. The optimization variables used in the proposed method were the increments of the control variables of UAV motion model， which broke the limitations of the maneuver library. Simulation and adversarial testing results with the MiniMax method， basic PIO algorithm， and Particle Swarm Optimization （PSO） algorithm show that the proposed maneuver decision-making method can effectively defeat opponents during confrontation and generate more flexible deceptive maneuver behaviors.

Key words: Pigeon-Inspired Optimization (PIO) algorithm, short-range air combat, maneuver decision-making, Unmanned Aerial Vehicle (UAV), finite tolerance strategy

摘要：

由于对抗双方态势的快速变化，无人机近距空战机动自主决策困难且复杂，是空中对抗的一个难点。对此，提出一种基于有限忍耐度鸽群优化（FTPIO）算法的无人机近距空战机动决策方法。该方法主要包括基于机动动作库的对手行动预测和基于FTPIO算法的机动控制量和执行时间优化求解两个部分。为提升基本鸽群优化（PIO）算法的全局探索能力，引入有限忍耐度策略，在鸽子个体几次迭代中没有找到更优解时对其属性进行一次重置，避免陷入局部最优陷阱。该方法采用的优化变量是无人机运动模型控制变量的增量，打破了机动库的限制。通过和极小极大方法、基本PIO算法和粒子群优化（PSO）算法的仿真对抗测试结果表明，所提出的机动决策方法能够在近距空战中有效击败对手，产生更为灵活的欺骗性机动行为。

关键词: 鸽群优化算法, 近距空战, 机动决策, 无人机, 有限忍耐度策略

CLC Number:

TP301.6

Zhiqiang ZHENG, Haibin DUAN. Short-range UAV air combat maneuver decision-making via finite tolerance pigeon-inspired optimization[J]. Journal of Computer Applications, 2024, 44(5): 1401-1407.

郑志强, 段海滨. 基于有限忍耐度鸽群优化的无人机近距空战机动决策[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1401-1407.

Figures/Tables 10

Tab. 1 Maneuver library

编号	名称	控制量 $n x$ ， $n z$ ， $γ$
1	匀速直飞	$0,1, 0$
2	加速直飞	$K 1, 1,0$
3	减速直飞	$K 2, 1,0$
4	匀速爬升	$0, K 3, 0$
5	加速爬升	$K 1, K 3, 0$
6	减速爬升	$K 2, K 3, 0$
7	匀速降高	$0, - K 3, 0$
8	加速降高	$K 1, - K 3, 0$
9	减速降高	$K 2, - K 3, 0$
10	匀速左转	$0, K 3, a r c c o s (K 3 - 1)$
11	加速左转	$K 1, K 3, a r c c o s (K 3 - 1)$
12	减速左转	$K 2, K 3, a r c c o s (K 3 - 1)$
13	匀速右转	$0, K 3, - a r c c o s (K 3 - 1)$
14	加速右转	$K 1, K 3, - a r c c o s (K 3 - 1)$
15	减速右转	$K 2, K 3, - a r c c o s (K 3 - 1)$

Tab. 1 Maneuver library

编号	名称	控制量 $n x$ ， $n z$ ， $γ$
1	匀速直飞	$0,1, 0$
2	加速直飞	$K 1, 1,0$
3	减速直飞	$K 2, 1,0$
4	匀速爬升	$0, K 3, 0$
5	加速爬升	$K 1, K 3, 0$
6	减速爬升	$K 2, K 3, 0$
7	匀速降高	$0, - K 3, 0$
8	加速降高	$K 1, - K 3, 0$
9	减速降高	$K 2, - K 3, 0$
10	匀速左转	$0, K 3, a r c c o s (K 3 - 1)$
11	加速左转	$K 1, K 3, a r c c o s (K 3 - 1)$
12	减速左转	$K 2, K 3, a r c c o s (K 3 - 1)$
13	匀速右转	$0, K 3, - a r c c o s (K 3 - 1)$
14	加速右转	$K 1, K 3, - a r c c o s (K 3 - 1)$
15	减速右转	$K 2, K 3, - a r c c o s (K 3 - 1)$

Fig. 1 Situational relationship of red and blue

Fig. 2 Flow chart of short-range air combat

Fig. 3 Flow chart of FTPIO algorithm

Tab. 2 Parameters for algorithms

算法	参数符号	描述	参数值
PIO和 FTPIO	$N c 1, m a x$	地图与指南针算子最大迭代次数	40
	$N c 2, m a x$	地标算子最大迭代次数	8
	$N p$	种群大小	20
	$R$	地图与指南针因子	0.2
PSO	$c 1, c 2$	学习因子	2.0
	$N c, m a x$	最大迭代次数	48
	$N p$	种群大小	20
	$w$	惯性因子	0.8

Tab. 2 Parameters for algorithms

算法	参数符号	描述	参数值
PIO和 FTPIO	$N c 1, m a x$	地图与指南针算子最大迭代次数	40
	$N c 2, m a x$	地标算子最大迭代次数	8
	$N p$	种群大小	20
	$R$	地图与指南针因子	0.2
PSO	$c 1, c 2$	学习因子	2.0
	$N c, m a x$	最大迭代次数	48
	$N p$	种群大小	20
	$w$	惯性因子	0.8

Fig. 4 Iteration curves of test functions

Tab. 3 Initial states of UAVs in tests

情形	阵营	状态值
一般	红方	s =［100 m，2 600 m，1 000 m，60 ms，0，20°］^T
一般	蓝方	s =［4 200 m，3 000 m，1 900 m，60 ms，0，190°］^T
平衡	红方	s =［700 m，2 500m，1 200 m，60 ms，0，0］^T
平衡	蓝方	s =［4 700 m，2 500m，1 200 m，60 ms，0，180°］^T
优势	红方	s =［700 m，2 600 m，1 100 m，60 ms，0，25°］^T
优势	蓝方	s =［1 200 m，3 000 m，1 500 m，60 ms，0，0］^T
劣势	红方	s =［1 200 m，2 600 m，1 100 m，60 ms，0，25°］^T
劣势	蓝方	s =［700 m，3 000 m，1 500m，60 ms，0，0］^T

Fig. 5 3D trajectories of short-range air combat simulations in four cases

Fig. 6 FTPIO （red） vs. Basic PIO （blue）

Fig. 7 FTPIO （red） vs. PSO （blue）

References 27

1	HUO M， DUAN H， LUO D， et al. Parameter estimation for a VTOL UAV using mutant pigeon inspired optimization algorithm with dynamic OBL strategy［C］// Proceedings of the 2019 IEEE 15th International Conference on Control and Automation. Piscataway： IEEE， 2019： 669-674. 10.1109/icca.2019.8899524
2	LUO L B， WANG X Y， MA J F， et al. GrpAvoid： multigroup collision-avoidance control and optimization for UAV swarm［J］. IEEE Transactions on Cybernetics， 2023， 53（3）： 1776-1789. 10.1109/tcyb.2021.3132044
3	YONG K N， CHEN M， WU Q X. Noncertainty-equivalent observer-based noncooperative target tracking control for unmanned aerial vehicles［J］. SCIENCE CHINA Information Sciences， 2022， 65（5）： 152202. 10.1007/s11432-020-3205-4
4	YANG Q M， ZHANG J D， SHI G Q， et al. Maneuver decision of UAV in short-range air combat based on deep reinforcement learning［J］. IEEE Access， 2020， 8： 363-378. 10.1109/access.2019.2961426
5	KIM C-S， JI C-H， KIM B S. Development of a control law to improve the handling qualities for short-range air-to-air combat maneuvers ［J］. Advances in Mechanical Engineering， 2020， 12（7）： 1-15. 10.1177/1687814020936790
6	董一群，艾剑良.自主空战技术中的机动决策：进展与展望［J］.航空学报， 2020， 41（S2）： 724264. 10.7527/S1000-6893.2020.24264
	DONG Y Q， AI J L. Decision making in autonomous air combat： review and prospects［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（S2）： 724264. 10.7527/S1000-6893.2020.24264
7	ISAACS R. Differential Games： A Mathematical Theory with Applications to Warfare and Pursuit， Control and Optimization ［M］. Mineola： Dover Publications， 1999： 305-330.
8	BASPINAR B， KOYUNCU E. Differential flatness-based optimal air combat maneuver strategy generation［C/OL］// Proceedings of the AIAA Scitech 2019 Forum. San Diego： American Institute of Aeronautics and Astronautics， 2019 ［2023-12-05］. . 10.2514/6.2019-1985
9	ERNEST N， CARROLL D， SCHUMACHER C， et al. Genetic fuzzy based artificial intelligence for unmanned combat aerial vehicle control in simulated air combat missions［J］. Journal of Defense Management， 2016， 6： 1000144. 10.4172/2167-0374.1000144
10	WU A， YANG R， LIANG X， et al. Visual range maneuver decision of unmanned combat aerial vehicle based on fuzzy reasoning［J］. International Journal of Fuzzy Systems， 2022， 24（1）： 519-536. 10.1007/s40815-021-01158-y
11	王昱，任田君，范子琳.基于引导Minimax-DDQN的无人机空战机动决策［J］.计算机应用，2023，43（8）：2636-2643. 10.11772/j.issn.1001-9081.2022071069
	WANG Y， REN T J， FAN Z L. Air combat maneuver decision-making of unmanned aerial vehicle based on guided Minimax-DDQN［J］. Journal of Computer Applications， 2023， 43（8）： 2636-2643. 10.11772/j.issn.1001-9081.2022071069
12	CHEN Y， ZHANG J， YANG Q， et al. Design and verification of UAV maneuver decision simulation system based on deep q‑learning network［C］// Proceedings of the 2020 16th International Conference on Control， Automation， Robotics and Vision. Piscataway： IEEE， 2020： 817-823. 10.1109/icarcv50220.2020.9305467
13	RUAN W， DUAN H， DENG Y. Autonomous maneuver decisions via transfer learning pigeon-inspired optimization for UAVs in dogfight engagements［J］. IEEE/CAA Journal of Automatica Sinica， 2022， 9（9）： 1639-1657. 10.1109/jas.2022.105803
14	RAMÍREZ LÓPEZ N， ŻBIKOWSKI R. Effectiveness of autonomous decision making for unmanned combat aerial vehicles in dogfight engagements［J］. Journal of Guidance， Control， and Dynamics， 2018， 41（4）： 1021-1024. 10.2514/1.g002937
15	高岳林，杨钦文，王晓峰，等.新型群体智能优化算法综述［J］.郑州大学学报（工学版）， 2022， 43（3）： 21-30.
	GAO Y L， YANG Q W， WANG X F， et al. Overview of new swarm intelligent optimization algorithms［J］. Journal of Zhengzhou University （Engineering Science）， 2022， 43（3）： 21-30.
16	KENNEDY J， EBERHART R. Particle swarm optimization［C］// Proceedings of ICNN’95 — International Conference on Neural Networks. Piscataway： IEEE， 1995： 1942-1948. 10.1109/ICNN.1995.488968
17	王峰，黄子路，韩孟臣，等.基于KnCMPSO算法的异构无人机协同多任务分配［J］.自动化学报，2023，49（2）：399-414.
	WANG F， HUANG Z L， HAN M C， et al. A knee point based coevolution multi-objective particle swarm optimization algorithm for heterogeneous UAV cooperative multi-task allocation［J］. Acta Automatica Sinica， 2023， 49（2）： 399-414.
18	梁军，洪泽泓，余松森.基于改进粒子群优化算法和遗传变异的图像分割模型［J］.计算机应用，2023，43（6）：1743-1749.
	LIANG J， HONG Z H， YU S S. Image segmentation model based on improved particle swarm optimization algorithm and genetic mutation［J］. Journal of Computer Application， 2023， 43（6）： 1743-1749.
19	DUAN H， QIAO P. Pigeon-inspired optimization： a new swarm intelligence optimizer for air robot path planning［J］. International Journal of Intelligent Computing and Cybernetics， 2014， 7（1）： 24-37. 10.1108/ijicc-02-2014-0005
20	LIAO T， CHEN B， ZHOU Q， et al. Improved pigeon-inspired optimization in an integrated obstacle avoidance method for Mars UAV formation［J］. Guidance， Navigation and Control， 2023， 3（1）： 2350006. 10.1142/s2737480723500061
21	DUAN H， LEI Y， XIA J， et al. Autonomous maneuver decision for unmanned aerial vehicle via improved pigeon-inspired optimization［J］. IEEE Transactions on Aerospace and Electronic Systems， 2023， 59（3）： 3156-3170. 10.1109/taes.2022.3221691
22	LI Y， HAN W， WANG Y. Deep reinforcement learning with application to air confrontation intelligent decision-making of manned/unmanned aerial vehicle cooperative system［J］. IEEE Access， 2020， 8： 67887-67898. 10.1109/access.2020.2985576
23	REN Z， ZHANG D， TANG S， et al. Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game［J］. Defence Technology， 2022， 27（3）： 308-317. 10.1016/j.dt.2022.10.008
24	杨爱武，李战武，李宝，等.基于动态变权重的空战态势评估［J］.兵工学报，2021，42（7）：1553-1563. 10.3969/j.issn.1000-1093.2021.07.023
	YANG A W， LI Z W， LI B， et al. Air combat situation assessment based on dynamic variable weight［J］. Acta Armamentarii， 2021， 42（7）： 1553-1563. 10.3969/j.issn.1000-1093.2021.07.023
25	FENG Q， HAI X， SUN B， et al. Resilience optimization for multi-UAV formation reconfiguration via enhanced pigeon-inspired optimization［J］. Chinese Journal of Aeronautics， 2022， 35（1）： 110-123. 10.1016/j.cja.2020.10.029
26	HU D， YANG R， ZHANG Y， et al. Aerial combat maneuvering policy learning based on confrontation demonstrations and dynamic quality replay［J］. Engineering Applications of Artificial Intelligence， 2022， 111： 104767. 10.1016/j.engappai.2022.104767
27	YAO X， LIU Y， LIN G. Evolutionary programming made faster［J］. IEEE Transactions on Evolutionary Computation， 1999， 3（2）： 82-102. 10.1109/4235.771163

[1]	Rui TANG, Shibo YUE, Ruizhi ZHANG, Chuan LIU, Chuanlin PANG. Energy efficiency optimization mechanism for UAV-assisted and non-orthogonal multiple access-enabled data collection system [J]. Journal of Computer Applications, 2024, 44(4): 1209-1218.
[2]	Meiyu CAI, Runzhe ZHU, Fei WU, Kaiyu ZHANG, Jiale LI. Cross-view matching model based on attention mechanism and multi-granularity feature fusion [J]. Journal of Computer Applications, 2024, 44(3): 901-908.
[3]	Huzhen GAO, Changping DU, Yao ZHENG. Gimbal system control algorithm of unmanned aerial vehicle based on extended state observer [J]. Journal of Computer Applications, 2024, 44(2): 604-610.
[4]	Yu WANG, Tianjun REN, Zilin FAN. Air combat maneuver decision-making of unmanned aerial vehicle based on guided Minimax-DDQN [J]. Journal of Computer Applications, 2023, 43(8): 2636-2643.
[5]	Zhongyuan ZHANG, Wei DAI, Guangyu LI, Xiaoqing CHEN, Qibo DENG. Cooperative obstacle avoidance algorithm based on improved artificial potential field and consensus protocol [J]. Journal of Computer Applications, 2023, 43(8): 2644-2650.
[6]	Chaoshuai QI, Wensi HE, Yi JIAO, Yinghong MA, Wei CAI, Suping REN. Survey on anomaly detection algorithms for unmanned aerial vehicle flight data [J]. Journal of Computer Applications, 2023, 43(6): 1833-1841.
[7]	Xiaolin LI, Yusang JIANG. Task offloading algorithm for UAV-assisted mobile edge computing [J]. Journal of Computer Applications, 2023, 43(6): 1893-1899.
[8]	Yanan SUN, Jiehong WU, Junling SHI, Lijun GAO. Multi-UAV collaborative task assignment method based on improved self-organizing map [J]. Journal of Computer Applications, 2023, 43(5): 1551-1556.
[9]	Xianlan WANG, Jinkun ZHOU, Nan MU, Chen WANG. Cross-view geo-localization method based on multi-task joint learning [J]. Journal of Computer Applications, 2023, 43(5): 1625-1635.
[10]	Longbao WANG, Yinqi LUAN, Liang XU, Xin ZENG, Shuai ZHANG, Shufang XU. Route planning method of UAV swarm based on dynamic cluster particle swarm optimization [J]. Journal of Computer Applications, 2023, 43(12): 3816-3823.
[11]	Lin ZHAO, Ke LYU, Jing GUO, Chen HONG, Xiancai XIANG, Jian XUE, Yong WANG. UAV cluster cooperative combat decision-making method based on deep reinforcement learning [J]. Journal of Computer Applications, 2023, 43(11): 3641-3646.
[12]	Jun MA, Zhen YAO, Cuifeng XU, Shouhong CHEN. Multi-UAV real-time tracking algorithm based on improved PP-YOLO and Deep-SORT [J]. Journal of Computer Applications, 2022, 42(9): 2885-2892.
[13]	Jinghu LI, Qianguo XING, Xiangyang ZHENG, Lin LI, Lili WANG. Noctiluca scintillans red tide extraction method from UAV images based on deep learning [J]. Journal of Computer Applications, 2022, 42(9): 2969-2974.
[14]	Bao CHEN, Zupeng ZHOU, Huan WEI, Yanzhao LYU, Zhicheng SUI. Control method of quadrotor UAV with manipulator based on expert PID [J]. Journal of Computer Applications, 2022, 42(8): 2637-2642.
[15]	Yuanliang XUE, Guodong JIN, Lining TAN, Jiankun XU. Pixel classification-based multiscale UAV aerial object rotational tracking algorithm [J]. Journal of Computer Applications, 2022, 42(7): 2239-2247.