Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (11): 3623-3628.DOI: 10.11772/j.issn.1001-9081.2023101538

• Frontier and comprehensive applications • Previous Articles     Next Articles

Distributed UAV cluster pursuit decision-making based on trajectory prediction and MADDPG

Yu WANG(), Zhihui GUAN, Yuanpeng LI   

  1. School of Automation,Shenyang Aerospace University,Shenyang Liaoning 110136,China
  • Received:2023-11-10 Revised:2024-02-05 Accepted:2024-02-06 Online:2024-03-25 Published:2024-11-10
  • Contact: Yu WANG
  • About author:GUAN Zhihui, born in 1998, M. S. candidate. Her research interests include reinforcement learning, pursuit decision-making.
    LI Yuanpeng, born in 2001, M. S. candidate. His research interests include intelligent decision-making.
  • Supported by:
    National Natural Science Foundation of China(61906125);Scientific Research Funding Project of Educational Department of Liaoning Province(LJKZ0222)

基于轨迹预测和分布式MADDPG的无人机集群追击决策

王昱(), 关智慧, 李远鹏   

  1. 沈阳航空航天大学 自动化学院,沈阳 110136
  • 通讯作者: 王昱
  • 作者简介:关智慧(1998—),女,山东济宁人,硕士研究生,主要研究方向:强化学习、追击决策
    李远鹏(2001—),男,辽宁朝阳人,硕士研究生,主要研究方向:智能决策。
  • 基金资助:
    国家自然科学基金资助项目(61906125);辽宁省教育厅科学研究经费资助项目(LJKZ0222)

Abstract:

A Trajectory Prediction based Distributed Multi-Agent Deep Deterministic Policy Gradient (TP-DMADDPG) algorithm was proposed to address the problems of insufficient flexibility and poor generalization ability of Unmanned Aerial Vehicle (UAV) cluster pursuit decision-making algorithms in complex mission environments. Firstly, to enhance the realism of the pursuit mission, an intelligent escape strategy was designed for the target. Secondly, considering the conditions such as missing information of target due to communication interruption and other reasons, a Long Short-Term Memory (LSTM) network was used to predict the position information of target in real time, and the state space of the decision-making model was constructed on the basis of the prediction information. Finally, TP-DMADDPG was designed based on the distributed framework and Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm, which enhanced the flexibility and generalization ability of pursuit decision-making in the process of complex air combat. Simulation results show that compared with Deep Deterministic Policy Gradient (DDPG), Twin Delayed Deep Deterministic policy gradient (TD3) and MADDPG algorithms, the TP-DMADDPG algorithm increases the success rate of collaborative decision-making by more than 15 percentage points, and can solve the problem of pursuing intelligent escaping target with incomplete information.

Key words: cluster pursuit, trajectory prediction, distributed decision-making, multi-agent, reinforcement learning, Deep Deterministic Policy Gradient (DDPG) algorithm

摘要:

针对复杂任务环境下无人机(UAV)集群追击决策算法灵活性不足、泛化能力差等问题,提出一种基于轨迹预测的分布式多智能体深度确定性策略梯度(TP-DMADDPG)算法。首先,为增强追击任务的真实性,为目标机设计智能化逃逸策略;其次,考虑到因通信中断等原因导致的目标机信息缺失等情况,采用长短时记忆(LSTM)网络实时预测目标机的位置信息,并基于预测信息构建决策模型的状态空间;最后,依据分布式框架和多智能体深度确定性策略梯度(MADDPG)算法设计TP-DMADDPG算法,增强复杂空战进程中集群追击决策的灵活性和泛化能力。仿真实验结果表明,相较于深度确定性策略梯度(DDPG)、双延迟深度确定性策略梯度(TD3)和MADDPG算法,TP?DMADDPG算法将协同决策的成功率提升了至少15个百分点,能够解决不完备信息下追击智能化逃逸目标机的问题。

关键词: 集群追击, 轨迹预测, 分布式决策, 多智能体, 强化学习, 深度确定性策略梯度算法

CLC Number: