Abstract:

To ensure that extracted features contain rich information， current deep learning-based image registration algorithms usually employ deep convolutional neural networks， which have high computational complexity and low discrimination of similar feature points. To address the above issues， a Self-supervised Image Registration Algorithm based on Multi-Feature Fusion （SIRA-MFF） was proposed. First， shallow convolutional neural networks were used to extract image features and reduce the computational complexity. Moreover， the problem of single feature information in shallow networks was remedied by adding feature point direction descriptors to the feature extraction layer. Second， an embedding and interaction layer was added after the feature extraction layer to enlarge the receptive field of feature points， by which local and global information of feature points was fused to improve the discrimination of similar feature points. Finally， the feature matching layer was optimized to obtain the best matching scheme. A cross-entropy based loss function was also designed for model training. The SIRA-MFF achieved the Average Matching Accuracy （AMA） of 95.18% and 93.26% on the two test sets generated from the ILSVRC2012 dataset， which was better than comparison algorithms. In the IMC-PT-SparseGM-50 test set， the SIRA-MFF achieved the AMA of 89.69%， which was also better than comparison algorithms； and compared to ResMtch algorithm， SIRA-MFF decreased the operation time of a single image by 49.45%. Experimental results show that SIRA-MFF has higher accurate and stronger robust.

Key words: image registration, self-supervised learning, feature fusion, feature descriptor, feature embedding

摘要：

为保证提取特征的信息量丰富，当前基于深度学习的图像配准算法通常采用深层卷积神经网络，模型的计算复杂度高，而且还存在相似特征点区分度低的问题。针对上述问题，提出一种基于多特征融合的自监督图像配准算法（SIRA-MFF）。首先，使用浅层卷积神经网络提取图像特征，降低计算复杂度，并且通过在特征提取层添加特征点方向描述符，弥补浅层网络特征信息量单一的问题；其次，在特征提取层后添加用于扩大特征点感受野的嵌入与交互层，融合特征点局部和全局信息以提升相似特征点区分度；最终，最佳匹配方案由改进的特征匹配层计算得到，并同步设计了一种基于交叉熵的损失函数用于模型训练。在ILSVRC2012数据集生成的2个测试集中，SIRA-MFF的平均匹配准确率（AMA）分别为95.18%和93.26%，优于对比算法；在IMC-PT-SparseGM-50测试集中，SIRA-MFF的AMA为89.69%，也优于对比算法，且与ResMtch算法相比，单张图像运算时间降低了49.45%。实验结果表明，SIRA-MFF具有较高精度和较强的鲁棒性。

关键词: 图像配准, 自监督学习, 特征融合, 特征描述符, 特征嵌入