Abstract:

Convolutional Neural Networks （CNNs） have made good progress in supervised stereo matching tasks， but most CNN algorithms are difficult to perform well in cross-domain situations. Aiming at the stereo matching problem of cross-domain data， a Cross-domain Adaptation Stereo Matching Network （CASM-Net） algorithm was proposed to achieve domain adaptive stereo matching tasks using transfer learning based on CNN. In the algorithm， a transferable feature extraction module was used to extract rich wide-domain features for stereo matching tasks. At the same time， an adaptive cost optimization module was designed to obtain the optimal cost distribution by making use of the similarity information on different receptive fields to optimize the cost. In addition， a disparity score prediction module was proposed to quantify the stereo matching ability of different regions， and the disparity results were further optimized by adjusting the disparity search range of the image. Experimental results show that on KITTI2012 and KITTI2015 datasets， compared with PSMNet （Pyramid Stereo Matching Network） algorithm， CASM-Net algorithm reduces 6.1%， 3.3% and 19.3% in 2-PE-Noc， 2-PE-All and 3-PE-fg， respectively； on Middlebury dataset， without re-training， CASM-Net algorithm achieves the optimal or suboptimal 2-PE results on all samples in the comparison with other algorithms. It can be seen that CASM-Net algorithm can improve cross-domain stereo matching.

Key words: supervised stereo matching, Convolutional Neural Network (CNN), transfer learning, cross-domain, disparity score

摘要：

虽然卷积神经网络（CNN）在有监督立体匹配任务中取得了较好的进展，但多数CNN算法的跨域表现较差。针对跨数据域的立体匹配问题，提出一种基于CNN的使用迁移学习实现域自适应立体匹配任务的跨域自适应立体匹配（CASM-Net）算法。所提算法使用一个可供迁移的特征提取模块提取丰富的广域特征用于跨域立体匹配任务；并且，设计一个自适应代价优化模块，从而通过自适应地利用不同感受野的相似度信息优化代价，进而得到最优的代价分布；此外，提出一个视差分数预测模块，以量化不同区域的立体匹配能力，并通过调整图像的视差搜索范围进一步优化视差结果。实验结果表明：在KITTI2012和KITTI2015数据集上，CASM-Net算法的2-PE-Noc、2-PE-All和3-PE-fg相较于PSMNet（Pyramid Stereo Matching Network）算法分别降低了6.1%、3.3%和19.3%；在Middlebury数据集上，在未经重新训练的情况下，在和其他算法的对比中，CASM-Net算法在所有样本上取得了最优或次优的2-PE结果。可见，CASM-Net算法具有改善跨域立体匹配的作用。

关键词: 有监督立体匹配, 卷积神经网络, 迁移学习, 跨域, 视差分数