Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Siamese mixed information fusion algorithm for RGBT tracking
Ying HUANG, Jiayu YANG, Jiahao JIN, Bangrui WAN
Journal of Computer Applications    2024, 44 (9): 2878-2885.   DOI: 10.11772/j.issn.1001-9081.2023081223
Abstract179)   HTML4)    PDF (3144KB)(807)       Save

The core of visible light and thermal infrared tracking (RGBT (RGB-Thermal) tracking for shot) lies in the effective utilization of information from different modalities. To address the problem of low-quality results produced by single branch in decision-level fusion affecting algorithm’s object decision-making, a Siamese mixed information fusion algorithm — SiamMIF was proposed for RGBT tracking. Firstly, Siamese Backbone Network (SBN) was used for multi-modal feature extraction. Secondly, the affect of low-quality images on the dual-branch parallel decision-making was analyzed from the perspective of signal-to-noise ratio, and an Signal-to-Noise Ratio (SNR)-driven Information Interaction Module (IIM) was designed for information complementation of information with low signal-to-noise ratio. Thirdly, a Dual-stream Anchor-free Head (DAH) was employed for the classification and regression of the compensated features. Finally, an Adaptive Lightweight Decision Module (ALDM) was used to fuse the tracking results and determine the object’s position quickly. Experimental results on four RGBT benchmark datasets including GTOT, RGBT234, VOT-RGBT2019 and LasHeR show that the success rate and precision of the proposed method on LasHeR dataset are 0.396 and 0.518 respectively, and compared to the APFNet (Attribute-based Progressive Fusion Network), there are a 9.4% improvement in success rate and a 3.6% enhancement in precision. At the same time, SiamMIF achieves good results on other three datasets, and the frame rate on GPU can reach 40 frame/s.

Table and Figures | Reference | Related Articles | Metrics