Abstract: Abstract: Aiming at the problem that the background of dynamic gesture recognition is strict and the classification accuracy is not high, a dynamic gesture recognition method based on RFC multi-model fusion is proposed, which combines depth information and color information. Firstly, the accurate and real-time human hand detection was obtained by introducing the target detection framework, and then the manual segmentation method was designed for the color map and the dark image respectively. Secondly, the color map and the depth map were successively input into the 2-dimensional Convolutional Neural Networks(CNN) and the 2-layer Long Short-Term Memory (LSTM) network to extract the spatial and temporal features, and then train the feature classifier; finally, the output of the color map classifier and the depth map classifier were taken as samples, and the RFC model was trained by the multi-model fusion method. In the experimental verification phase, 5 volunteers were invited to repeat 23 different gestures, and 10876 color maps and 609 dynamic gesture sequences were obtained. The experimental results show that the algorithm can accurately detect and segment color maps and depth maps. After multi-model fusion, the classification accuracy rate is 99.7%, which has good practicability. And on the SKIG public dataset, the algorithm has an accuracy rate of 98.5%.