Abstract: A distributed Multi-Agent Deep Deterministic Policy Gradient (MADDPG) decision-making algorithm based on trajectory prediction was proposed to address the problems of insufficient flexibility and poor generalization ability of Unmanned Aerial Vehicle (UAV) cluster pursuit decision-making algorithms in complex mission environments. Firstly, to enhance the realism of the pursuit mission, an intelligent escape strategy was designed for the target. Secondly, considering the missing information of target due to communication interruption and other reasons, a Long Short-Term Memory (LSTM) network was used to predict the position information of target in real time, and the state space of the decision-making model was constructed based on the prediction information. Finally, a trajectory prediction based distributed Multi-Agent Deep Deterministic Policy Gradient(TP- DMADDPG),algorithm was designed based on the distributed framework and MADDPG algorithm, which enhanced the flexibility and generalization ability of pursuit decision-making in the process of complex air combat. Simulation experimental results show that compared with Deep Deterministic Policy Gradient (DDPG), Twin Delayed Deep Deterministic policy gradient (TD3) and MADDPG algorithms, the proposed algorithm increases the success rate of cooperative decision-making by more than 15 percentage points, and can solve the problem of pursuing intelligent escaping target under incomplete information.

Key words: cluster pursuit, trajectory prediction, distributed decision making, multi-agent, reinforcement learning

摘要： 针对复杂任务环境下无人机（UAV）集群追击决策算法灵活性不足、泛化能力差等问题，提出一种基于轨迹预测的分布式多智能体强化学习决策算法。首先，为增强追击任务的真实性，为目标机设计智能化逃逸策略；其次，考虑到因通信中断等原因导致的目标机信息缺失等情况，采用长短时记忆(LSTM)网络实时预测目标机的位置信息，并基于预测信息构建决策模型的状态空间；最后，依据分布式框架和多智能体深度确定性策略梯度（MADDPG）算法设计基于轨迹预测的分布式MADDPG（TP-DMADDPG）算法，增强复杂空战进程中集群追击决策的灵活性和泛化能力。仿真实验结果表明，相较于深度确定性策略梯度（DDPG）、双延迟深度确定性策略梯度（TD3）和MADDPG算法，所提算法将协同决策的成功率提升了至少15个百分点，能够解决不完备信息下追击智能化逃逸目标的问题。

关键词: 集群追击, 轨迹预测, 分布式决策, 多智能体, 强化学习