关键词: RGBT跟踪, 孪生神经网络, 多模态融合策略, 信息交互, 无锚跟踪头

Abstract: The core of visible light and thermal infrared tracking (RGBT tracking) lies in the effective utilization of information from different modalities. To address the problem of suboptimal target determination caused by single-branch decision-level fusion, a siamese mixed information fusion algorithm (SiamMIF) for RGBT tracking was proposed. Firstly, the siamese backbone network SBN was used for multi-modal feature extraction. Secondly, the impact of low-quality images on the dual-branch parallel decision-making was analyzed from the perspective of signal-to-noise ratio, and an SNR-driven information interaction module (IIM) was designed for information complementation. Thirdly, a dual-stream anchor-free tracking head employed for the classification and regression of the compensated features. Finally, an adaptive lightweight decision-making module (ALDM) was used to fuse the tracking results and swiftly determine the target's position. Proposed tracker was evaluated on four RGBT benchmarks including GTOT, RGBT234, VOT-RGBT2019 and LasHeR, the success rate and precision of the method on the LasHeR dataset were determined to be 0.396 and 0.518 and exhibited a 9.4% improvement in success rate and a 3.6% enhancement in precision compared to the APFNet, the results show that SiamMIF achieves competitive performance on remaining dataset and runs at 40fps.

Key words: RGBT tracking, siamese neural network, multimodal fusion strategy, information interaction, anchor-free head