Infrared dim small target tracking method based on Siamese network and Transformer

doi:10.11772/j.issn.1001-9081.2023020167

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (2): 563-571.DOI: 10.11772/j.issn.1001-9081.2023020167

Special Issue: 多媒体计算与计算机仿真

• Multimedia computing and computer simulation • Previous Articles Next Articles

Infrared dim small target tracking method based on Siamese network and Transformer

Chenhui CUI¹, Suzhen LIN¹(), Dawei LI², Xiaofei LU³, Jie WU¹

^1.College of Data Science and Technology，North University of China，Taiyuan Shanxi 030051，China
^2.School of Electrical and Control Engineering，North University of China，Taiyuan Shanxi 030051，China
^3.Jiuquan Satellite Launch Center，Jiuquan Gansu 735000，China

Received:2023-02-23 Revised:2023-04-24 Accepted:2023-05-05 Online:2024-02-22 Published:2024-02-10
Contact: Suzhen LIN
About author:CUI Chenhui， born in 1998， M. S. candidate. His research interests include target tracking， image processing.
LI Dawei， born in 1980， Ph. D.， associate professor. His research interests include pattern recognition， machine learning， image processing.
LU Xiaofei， born in 1981， Ph. D. His research interests include measurement data processing.
WU Jie， born in 1998， M. S. candidate. His research interests include infrared dim small target detection.
Supported by:
Graduate Education Innovation Project of Shanxi Province(2022Y631);Shanxi Province Patent Transformation Special Plan Project(202302001);Education Reform Project of Shanxi Province(J20220593);18th Graduate Technology Project of North University of China(20221845)

基于孪生网络和Transformer的红外弱小目标跟踪方法

崔晨辉¹, 蔺素珍¹(), 李大威², 禄晓飞³, 武杰¹

^1.中北大学计算机科学与技术学院，太原 030051
^2.中北大学电气与控制工程学院，太原 030051
^3.酒泉卫星发射中心，甘肃酒泉 735000

通讯作者: 蔺素珍
作者简介:崔晨辉（1998—），男，山西晋城人，硕士研究生，CCF会员，主要研究方向：目标跟踪、图像处理
李大威（1980—），男，河北衡水人，副教授，博士，主要研究方向：模式识别、机器学习、图像处理
禄晓飞（1981—），男，河南许昌人，博士，主要研究方向：测量数据处理
武杰（1998—），男，山西吕梁人，硕士研究生，CCF会员，主要研究方向：红外弱小目标检测。
基金资助:
山西省研究生教育创新项目(2022Y631);山西省专利转化专项计划项目(202302001);山西省级教改项目(J20220593);中北大学第十八届研究生科技立项项目(20221845)

Abstract

Abstract:

A method based on Siamese network and Transformer was proposed to address the low accuracy problem of infrared dim small target tracking. First， a multi-feature extraction cascading moduling was constructed to separately extract the deep features of the infrared dim small target template frame and the search frame， and concatenate them with their corresponding HOG features at the dimension level. Second， a multi-head attention mechanism Transformer was introduced to perform cross-correlation operations between the template feature map and the search feature map， generating a response map. Finally， the target’s center position in the image and the regression bounding box were obtained through the response map upsampling network and bounding box prediction network to complete the tracking of the infrared dim small targets. Test results on a dataset of 13 655 infrared images show that compared with KeepTrack tracking method， the success rate is improved by 5.9 percentage points and the precision is improved by 1.8 percentage points； compared with TransT （Transformer Tracking） method， the success rate is improved by 14.2 percentage points and the precision is improved by 14.6 percentage points. The proposed method is proved to be more accurate in tracking infrared dim small targets in complex backgrounds.

Key words: target tracking, infrared dim small target, Siamese network, Transformer, multi-feature extraction

摘要：

针对红外弱小目标跟踪准确性较低这一问题，提出一种基于孪生网络和Transformer的红外弱小目标跟踪方法。首先，构建多特征提取级联模块分别提取红外弱小目标模板帧和搜索帧的深度特征，并将二者分别与其对应的HOG特征进行维度层面的串联；其次，引入多头注意力机制Transformer进行模板特征图和搜索特征图的互相关操作，生成响应图；最后，通过响应图上采样网络和边界框预测网络，获得目标在图像的中心位置和回归边界框，完成对红外弱小目标的跟踪。在包含13 655张红外图像数据集上的测试结果表明：与KeepTrack跟踪方法相比，成功率提高5.9个百分点，精确率提高1.8个百分点；与TransT（Transformer Tracking）方法相比，成功率提高14.2个百分点，精确率提高14.6个百分点，证明所提方法对复杂背景下的红外弱小目标跟踪准确性更高。

关键词: 目标跟踪, 红外弱小目标, 孪生网络, Transformer, 多特征提取

CLC Number:

TP391.41

Chenhui CUI, Suzhen LIN, Dawei LI, Xiaofei LU, Jie WU. Infrared dim small target tracking method based on Siamese network and Transformer[J]. Journal of Computer Applications, 2024, 44(2): 563-571.

崔晨辉, 蔺素珍, 李大威, 禄晓飞, 武杰. 基于孪生网络和Transformer的红外弱小目标跟踪方法[J]. 《计算机应用》唯一官方网站, 2024, 44(2): 563-571.

Figures/Tables 11

Fig. 1 Structure of SiamCAR network

Fig. 2 Overall structure of proposed algorithm

Fig. 3 Network structure of multi-feature extraction cascading module

Fig. 4 Network structure of feature cross-correlation module

Fig. 5 Network structure of upsampling module of response map

Fig. 6 Template frame images of different frames in same scene

Fig. 7 Visualization of results of different tracking methods on modified DIRST test dataset

Tab. 1 Quantitative evaluation results of different methods on DIRST dataset

方法	成功率	精确率	方法	成功率	精确率
KeepTrack	84.3	87.9	AGPC	82.5	82.3
TransT	76.0	75.1	DNANet	78.6	77.9
SiamCAR	63.7	67.0	本文方法	90.2	89.7

Fig. 8 Visualization of results of different tracking methods on LaTOT test dataset

Tab. 2 Quantitative evaluation results of different methods on LaTOT test set

方法	成功率	精确率	方法	成功率	精确率
KeepTrack	74.8	73.0	SiamCAR	73.5	68.0
TransT	72.7	71.8	本文方法	75.5	74.7

Tab. 3 Ablation experimental results

编号	方法	成功率	精确率
①	Base	55.4	57.7
②	Base+H	60.2	62.3
	Base+T	70.5	70.3
	Base+U	68.3	67.0
	Base+R	66.1	65.8
③	Base+H+T	72.9	72.6
	Base+H+R	69.7	68.5
	Base+H+U	68.8	68.6
	Base+T+R	78.3	78.4
	Base+T+U	73.2	72.9
	Base+R+U	71.6	71.5
④	Base+H+T+U	80.1	78.9
	Base+H+T+R	86.2	85.6
	Base+H+U+R	79.3	78.0
	Base+T+U+R	84.0	83.9
⑤	Base+H+T+U+R	90.2	89.7

References 22

1	刘征，杨德振，李江勇，等. 红外单帧弱小目标检测算法研究综述［J］. 激光与红外， 2022， 52（2）： 154-162. 10.3969/j.issn.1001-5078.2022.02.002
	LIU Z， YANG D Z， LI J Y， et al. A review of infrared single-frame dim small target detection algorithms［J］. Laser and Infrared， 2022， 52（2）： 154-162. 10.3969/j.issn.1001-5078.2022.02.002
2	JAVED S， DANELLJAN M， KHAN F S， et al. Visual object tracking with discriminative filters and Siamese networks： a survey and outlook［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2023， 45（5）：6552-6574.
3	QIAN K， ZHOU H， RONG S， et al. Infrared dim-small target tracking via singular value decomposition and improved kernelized correlation filter［J］. Infrared Physics and Technology， 2017， 82： 18-27. 10.1016/j.infrared.2017.02.002
4	QIAN K， RONG S H， CHENG K H. Anti-interference small target tracking from infrared dual waveband imagery［J］. Infrared Physics and Technology， 2021， 118： No.103882. 10.1016/j.infrared.2021.103882
5	YAN P， YAO S， ZHU Q， et al. Real-time detection and tracking of infrared small targets based on grid fast density peaks searching and improved KCF［J］. Infrared Physics and Technology， 2022， 123： No.104181. 10.1016/j.infrared.2022.104181
6	戴煜彤，陈志国，傅毅. 尺度估计和多特征融合的目标跟踪算法研究［J］. 信号处理， 2021， 37（1）： 49-58. 10.16798/j.issn.1003-0530.2021.01.006
	DAI Y T， CHEN Z G， FU Y. Research on target tracking algorithm based on scale estimation and multi-feature fusion［J］. Journal of Signal Processing， 2021， 37（1）： 49-58. 10.16798/j.issn.1003-0530.2021.01.006
7	LI B， YAN J， WU W， et al. High performance visual tracking with Siamese region proposal network［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 8971-8980. 10.1109/cvpr.2018.00935
8	LI Y C， YANG S. Infrared small object tracking based on Att-Siam network［J］. IEEE Access， 2022， 10： 133766-133777. 10.1109/access.2022.3171037
9	BERTINETTO L， VALMADRE J， HENRIQUES J F， et al. Fully-convolutional Siamese networks for object tracking［C］// Proceedings of the 2016 European Conference on Computer Vision， LNCS 9914. Cham： Springer， 2016： 850-865.
10	张文波，刘刚，张良，等. 改进全卷积孪生网络红外空中小目标跟踪算法［J］. 航空兵器， 2022， 29（3）： 33-41. 10.12132/ISSN.1673-5048.2021.0147
	ZHANG W B， LIU G， ZHANG L， et al. Improved infrared aerial small target tracking algorithm based on fully-convolutional Siamese networks［J］. Aero Weaponry， 2022， 29（3）： 33-41. 10.12132/ISSN.1673-5048.2021.0147
11	CHEN J， HUANG B， LI J， et al. Learning spatio-temporal attention based Siamese network for tracking UAVs in the wild［J］. Remote Sensing， 2022， 14（8）： 1797. 10.3390/rs14081797
12	CHEN X， YAN B， ZHU J， et al. Transformer tracking［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 8122-8131. 10.1109/cvpr46437.2021.00803
13	梁启花，胡现韬，钟必能，等. 基于孪生网络的目标跟踪算法研究进展［J］. 广西师范大学学报（自然科学版）， 2022， 40（5）：90-103.
	LIANG Q H， HU X T， ZHONG B N， et al. Research progress of target tracking algorithm based on Siamese network［J］. Journal of Guangxi Normal University （Natural Science Edition）， 2022， 40（5）： 90-103.
14	GUO D， WANG J， CUI Y， et al. SiamCAR： Siamese fully convolutional classification and regression for visual tracking［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 6268-6276. 10.1109/cvpr42600.2020.00630
15	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
16	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017：6000-6010.
17	RONNEBERGER O， FISCHER P， BROX T. U-net： convolutional networks for biomedical image segmentation［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS 9351. Cham： Springer， 2015： 234-241.
18	ZHU Y， LI C， LIU Y， et al. Tiny object tracking： a large-scale dataset and a baseline［J］. IEEE Transactions on Neural Networks and Learning Systems， 2023（Early Access）： 1-15. 10.1109/tnnls.2023.3239529
19	回丙伟，宋志勇，范红旗，等. 地/空背景下红外图像弱小飞机目标检测跟踪数据集［J］. 中国科学数据（中英文网络版）， 2020， 5（3）：291-302. 10.11922/csdata.2019.0074.zh
	HUI B W， SONG Z Y， FAN H Q， et al. A dataset for infrared detection and tracking of dim-small aircraft targets underground/air background［J］. China Scientific Data， 2020， 5（3）： 291-302. 10.11922/csdata.2019.0074.zh
20	ZHANG T， CAO S， PU T， et al. AGPCNet： attention-guided pyramid context networks for infrared small target detection［EB/OL］. （2021-11-05）［2023-01-05］.. 10.1109/taes.2023.3238703
21	LI B， XIAO C， WANG L， et al. Dense nested attention network for infrared small target detection［J］. IEEE Transactions on Image Processing， 2023， 32： 1745-1758. 10.1109/tip.2022.3199107
22	MAYER C， DANELLJAN M， PAUDEL D P， et al. Learning target candidate association to keep track of what not to track［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2021： 13424-13434. 10.1109/iccv48922.2021.01319

[1]	Jiepo FANG, Chongben TAO. Hybrid internet of vehicles intrusion detection system for zero-day attacks [J]. Journal of Computer Applications, 2024, 44(9): 2763-2769.
[2]	Jinjin LI, Guoming SANG, Yijia ZHANG. Multi-domain fake news detection model enhanced by APK-CNN and Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2674-2682.
[3]	Liehong REN, Lyuwen HUANG, Xu TIAN, Fei DUAN. Multivariate long-term series forecasting method with DFT-based frequency-sensitive dual-branch Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2739-2746.
[4]	Jieru JIA, Jianchao YANG, Shuorui ZHANG, Tao YAN, Bin CHEN. Unsupervised person re-identification based on self-distilled vision Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2893-2902.
[5]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.
[6]	Xin YANG, Xueni CHEN, Chunjiang WU, Shijie ZHOU. Short-term traffic flow prediction of urban highway based on variant residual model and Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2947-2951.
[7]	Yuwei DING, Hongbo SHI, Jie LI, Min LIANG. Image denoising network based on local and global feature decoupling [J]. Journal of Computer Applications, 2024, 44(8): 2571-2579.
[8]	Kaili DENG, Weibo WEI, Zhenkuan PAN. Industrial defect detection method with improved masked autoencoder [J]. Journal of Computer Applications, 2024, 44(8): 2595-2603.
[9]	Fan YANG, Yao ZOU, Mingzhi ZHU, Zhenwei MA, Dawei CHENG, Changjun JIANG. Credit card fraud detection model based on graph attention Transformation neural network [J]. Journal of Computer Applications, 2024, 44(8): 2634-2642.
[10]	Wentao JIANG, Wanxuan LI, Shengchong ZHANG. Correlation filtering based target tracking with nonlinear temporal consistency [J]. Journal of Computer Applications, 2024, 44(8): 2558-2570.
[11]	Dahai LI, Zhonghua WANG, Zhendong WANG. Dual-branch low-light image enhancement network combining spatial and frequency domain information [J]. Journal of Computer Applications, 2024, 44(7): 2175-2182.
[12]	Wu XIONG, Congjun CAO, Xuefang SONG, Yunlong SHAO, Xusheng WANG. Handwriting identification method based on multi-scale mixed domain attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2225-2232.
[13]	Junfeng SHEN, Xingchen ZHOU, Can TANG. Dual-channel sentiment analysis model based on improved prompt learning method [J]. Journal of Computer Applications, 2024, 44(6): 1796-1806.
[14]	Shibin LI, Jun GONG, Shengjun TANG. Semi-supervised heterophilic graph representation learning model based on Graph Transformer [J]. Journal of Computer Applications, 2024, 44(6): 1816-1823.
[15]	Mengyuan HUANG, Kan CHANG, Mingyang LING, Xinjie WEI, Tuanfa QIN. Progressive enhancement algorithm for low-light images based on layer guidance [J]. Journal of Computer Applications, 2024, 44(6): 1911-1919.

Infrared dim small target tracking method based on Siamese network and Transformer

基于孪生网络和Transformer的红外弱小目标跟踪方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 22

Related Articles 15

Recommended Articles

Metrics