关键词: 图像配准, 自监督学习, 特征融合, 特征描述符, 特征嵌入

Abstract:

To ensure that extracted features contain rich information， current deep learning-based image registration algorithms usually employ deep convolutional neural networks， which have high computational complexity and low discrimination of similar feature points. To address the above issues， a Self-supervised Image Registration Algorithm based on Multi-Feature Fusion （SIRA-MFF） was proposed. First， shallow convolutional neural networks were used to extract image features and reduce the computational complexity. Moreover， the problem of single feature information in shallow networks was remedied by adding feature point direction descriptors to the feature extraction layer. Second， an embedding and interaction layer was added after the feature extraction layer to enlarge the receptive field of feature points， by which local and global information of feature points was fused to improve the discrimination of similar feature points. Finally， the feature matching layer was optimized to obtain the best matching scheme. A cross-entropy based loss function was also designed for model training. The SIRA-MFF achieved the Average Matching Accuracy （AMA） of 95.18% and 93.26% on the two test sets generated from the ILSVRC2012 dataset， which was better than comparison algorithms. In the IMC-PT-SparseGM-50 test set， the SIRA-MFF achieved the AMA of 89.69%， which was also better than comparison algorithms； and compared to ResMtch algorithm， SIRA-MFF decreased the operation time of a single image by 49.45%. Experimental results show that SIRA-MFF has higher accurate and stronger robust.

Key words: image registration, self-supervised learning, feature fusion, feature descriptor, feature embedding