关键词: 深度强化学习, 移动机器人, 三维路径规划, 近端策略优化, 深度图

Abstract: Aiming at the problems of high complexity and uncertainty in 3D unknown environment, a mobile robot 3D path planning method based on deep reinforcement learning was proposed, under a limited observation space optimization strategy. First, the depth map information was used as the agent's input in the limited observation space, which could simulate complex 3D space environments with limited and unknown movement conditions. Second, a two-stage action selection policy in discrete action space was designed, including directional actions and translation actions, which could reduce the searching steps and time. Finally, based on the Proximal Policy Optimization algorithm, the Gated Recurrent Unit is added to combine the historical state information, to enhance the policy stability in unknown environments. So that the accuracy and smoothness of the planned path could be improved. The experimental results show that, compared with Advantage Actor-Critic (A2C), the average search time is reduced by 49.07% and the average planned path length is reduced by 1.03%. Meanwhile, it can complete the multi-objective path planning task under linear sequential logic constraints.

Key words: deep reinforcement learning, mobile robot, three-dimensional path planning, Proximal Policy Optimization(PPO), depth map