关键词: 深度强化学习（DRL）, 近端策略优化算法（PPO）, 不规则物体检测, 软体机械臂, 机械臂抓取

Abstract: In order to cope with the performance limitations of traditional deep reinforcement learning algorithms in processing complex scenes, especially the performance limitations in irregular object grabbing and soft manipulator applications, poor algorithm stability and learning rate issues, a soft robotic arm control based on the Cropped Proximal Policy Optimization (CPPO) algorithm was proposed. By introducing a clipping function, this algorithm optimized the performance of the proximal policy optimization (PPO) algorithm and improved its stability and learning efficiency in high-dimensional state space. First, the state space and action space of the soft robotic arm were defined, and a soft robotic arm model imitating the tentacles of the octopus was designed. Secondly, Matlab's SoRoSim toolbox was used for modeling, and an environmental reward function that combines continuous and sparse functions was defined. Finally, a simulation platform based on Matlab was constructed. The irregular object images were preprocessed through Python scripts and filters, and Redis cache was used to efficiently transmit the processed contour data to the simulation platform. Comparative experiments with the TRPO (Trust Region Policy Optimization) algorithm and the SAC (Soft Actor-Critic) algorithm shew that the CPPO algorithm achieved a success rate of 86.3% in the task of grabbing irregular objects by the soft manipulator, which was higher than the TRPO algorithm. increased by about 3.6%, with better performance. It is shown that the CPPO algorithm can be applied to the control of soft manipulators, and can provide an important reference for the application of soft manipulators in complex grasping tasks in unstructured environments.

Key words: Deep Reinforcement Learning (DRL), Proximal Policy Optimization (PPO), Irregular object detection, Soft robotic arm, Robotic arm grasping