关键词: 立体匹配, 稀疏卷积, 深度学习, 语义分割, 注意力机制

Abstract:

To improve the computational efficiency of stereo matching on foreground disparity estimation tasks， aiming at the disadvantage that the general networks use the complete binocular image as input and the input information redundancy is large due to the small proportion of the foreground space in the scene， a real-time target stereo matching algorithm based on sparse convolution was proposed. In order to realize and improve the sparse foreground disparity estimation of the algorithm， firstly， the sparse foreground mask and scene semantic features were obtained by the segmentation algorithm at the same time. Secondly， the sparse convolution was used to extract the spatial features of the foreground sparse region， and scene semantic features were fused with them. Then， the fused features were input into the decoding module for disparity regression. Finally， the foreground truth graph was used as the loss to generate the disparity graph. The test results on ApolloScape dataset show that the accuracy and real-time performance of the proposed algorithm are better than those of the state-of-the-art algorithms PSMNet （Pyramid Stereo Matching Network） and GANet （Guided Aggregation Network）， and the single run time of the algorithm is as low as 60.5 ms. In addition， the proposed algorithm has certain robustness to the foreground occlusion， and can be used for the real-time depth estimation of targets.

Key words: stereo matching, sparse convolution, deep learning, semantic segmentation, attention mechanism