Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (9): 2878-2885.DOI: 10.11772/j.issn.1001-9081.2023081223
• Multimedia computing and computer simulation • Previous Articles Next Articles
Ying HUANG(), Jiayu YANG, Jiahao JIN, Bangrui WAN
Received:
2023-09-08
Revised:
2023-10-30
Accepted:
2023-11-10
Online:
2024-09-14
Published:
2024-09-10
Contact:
Ying HUANG
About author:
YANG Jiayu, born in 1999, M. S. candidate. His research interests include multi-modal object tracking.通讯作者:
黄颖
作者简介:
杨佳宇(1999—),男,山西长治人,硕士研究生,主要研究方向:多模态目标跟踪CLC Number:
Ying HUANG, Jiayu YANG, Jiahao JIN, Bangrui WAN. Siamese mixed information fusion algorithm for RGBT tracking[J]. Journal of Computer Applications, 2024, 44(9): 2878-2885.
黄颖, 杨佳宇, 金家昊, 万邦睿. 用于RGBT跟踪的孪生混合信息融合算法[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2878-2885.
Add to citation manager EndNote|Ris|BibTeX
URL: https://www.joca.cn/EN/10.11772/j.issn.1001-9081.2023081223
算法 | EAO↑ | Acc↑ | Rob↓ | 帧率/(frame·s-1)↑ |
---|---|---|---|---|
FANet | 0.247 | 0.472 | 0.492 | 19.0 |
TFNet | 0.288 | 0.462 | 0.406 | — |
MANet++ | 0.364 | 0.582 | 0.292 | 3.1 |
ADRNet | 0.396 | 0.622 | 0.243 | 25.0 |
TTAT | 0.416 | 0.643 | 0.265 | 34.0 |
SiamCDA | 0.424 | 0.682 | 0.279 | 37.0 |
JMMAC | 0.498 | 0.650 | 0.193 | 4.0 |
本文算法 | 26.0 |
Tab. 1 Evaluation results of different algorithms
算法 | EAO↑ | Acc↑ | Rob↓ | 帧率/(frame·s-1)↑ |
---|---|---|---|---|
FANet | 0.247 | 0.472 | 0.492 | 19.0 |
TFNet | 0.288 | 0.462 | 0.406 | — |
MANet++ | 0.364 | 0.582 | 0.292 | 3.1 |
ADRNet | 0.396 | 0.622 | 0.243 | 25.0 |
TTAT | 0.416 | 0.643 | 0.265 | 34.0 |
SiamCDA | 0.424 | 0.682 | 0.279 | 37.0 |
JMMAC | 0.498 | 0.650 | 0.193 | 4.0 |
本文算法 | 26.0 |
ALDM | Fine-tune | IIM | SR | PR | EAO |
---|---|---|---|---|---|
√ | — | — | 0.692 | 0.847 | 0.367 |
√ | √ | — | 0.726 | 0.886 | 0.412 |
√ | √ | √ | 0.740 | 0.905 | 0.460 |
Tab. 2 Ablation experiment results on VOT-RGBT2019 dataset
ALDM | Fine-tune | IIM | SR | PR | EAO |
---|---|---|---|---|---|
√ | — | — | 0.692 | 0.847 | 0.367 |
√ | √ | — | 0.726 | 0.886 | 0.412 |
√ | √ | √ | 0.740 | 0.905 | 0.460 |
1 | BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional Siamese networks for object tracking [C]// Proceedings of the 2016 European Conference on Computer Vision Workshops, LNCS 9914. Cham: Springer, 2016: 850-865. |
2 | LI B, YAN J, WU W, et al. High performance visual tracking with Siamese region proposal network [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 8971-8980. |
3 | XU Y, WANG Z, LI Z, et al. SiamFC++: towards robust and accurate visual tracking with target estimation guidelines [C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020: 12549-12556. |
4 | YUN X, JING Z, XIAO G, et al. A compressive tracking based on time-space Kalman fusion model [J]. SCIENCE CHINA Information Sciences, 2016, 59(1): No.012106. |
5 | XIAO G, YUN X, WU J. A new tracking approach for visible and infrared sequences based on tracking-before-fusion [J]. International Journal of Dynamics and Control, 2016, 4(1): 40-51. |
6 | XIAO G, YUN X, WU J. A multi-cue mean-shift target tracking approach based on fuzzified region dynamic image fusion [J]. SCIENCE CHINA Information Sciences, 2012, 55(3): 577-589. |
7 | LI L, LI C, TU Z, et al. A fusion approach to grayscale-thermal tracking with cross-modal sparse representation [C]// Proceedings of the 2018 Conference on Image and Graphics Technologies and Applications, CCIS 875. Singapore: Springer, 2018: 494-505. |
8 | LAN X, YE M, ZHANG S, et al. Modality-correlation-aware sparse representation for RGB-infrared object tracking [J]. Pattern Recognition Letters, 2020, 130: 12-20. |
9 | LI C L, ZHU C, ZHENG S, et al. Two-stage modality-graphs regularized manifold ranking for RGB-T tracking [J]. Signal Processing: Image Communication, 2018, 68: 207-217. |
10 | SHEN L, WANG X, LIU L, et al. RGBT tracking based on cooperative low-rank graph model [J]. Neurocomputing, 2022, 492: 370-381. |
11 | NAM H, HAN B. Learning multi-domain convolutional neural networks for visual tracking [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 4293-4302. |
12 | LU A, LI C, YAN Y, et al. RGBT tracking via multi-adapter network with hierarchical divergence loss [J]. IEEE Transactions on Image Processing, 2021, 30: 5613-5625. |
13 | WANG C, XU C, CUI Z, et al. Cross-modal pattern-propagation for RGB-T tracking [C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 7062-7071. |
14 | XIAO Y, YANG M, LI C, et al. Attribute-based progressive fusion network for RGBT tracking [C]// Proceedings of the 36th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2022: 2831-2838. |
15 | LI B, WU W, WANG Q, et al. SiamRPN++: evolution of Siamese visual tracking with very deep networks [C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 4277-4286. |
16 | TANG F, LING Q. Ranking-based Siamese visual tracking [C]// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 8731-8740. |
17 | ZHANG X, YE P, PENG S, et al. SiamFT: an RGB-infrared fusion tracking method via fully convolutional Siamese networks[J]. IEEE Access, 2019, 7: 122122-122133. |
18 | ZHANG T, LIU X, ZHANG Q, et al. SiamCDA: complementarity-and distractor-aware RGB-T tracking based on Siamese network [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(3): 1403-1417. |
19 | LI C, XUE W, JIA Y, et al. LasHeR: a large-scale high-diversity benchmark for RGBT tracking [J]. IEEE Transactions on Image Processing, 2022, 31: 392-404. |
20 | ZHANG P, ZHAO J, WANG D, et al. Visible-thermal UAV tracking: a large-scale benchmark and new baseline [C]// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 8876-8885. |
21 | 李成龙,鹿安东,刘磊,等. 多模态视觉跟踪方法综述[J]. 中国图象图形学报, 2023, 28(1): 37-56. |
LI C L, LU A D, LIU L, et al. Multi-modal visual tracking: a survey [J]. Journal of Image and Graphics, 2023, 28(1): 37-56. | |
22 | ZHANG X, YE P, LEUNG H, et al. Object fusion tracking based on visible and infrared images: a comprehensive review [J]. Information Fusion, 2020, 63: 166-187. |
23 | GUO C, YANG D, LI C, et al. Dual Siamese network for RGBT tracking via fusing predicted position maps [J]. The Visual Computer, 2022, 38(7): 2555-2567. |
24 | ZHANG P, ZHAO J, BO C, et al. Jointly modeling motion and appearance cues for robust RGB-T tracking [J]. IEEE Transactions on Image Processing, 2021, 30: 3335-3347. |
25 | TANG Z, XU T, WU X J. Temporal aggregation for adaptive RGBT tracking [EB/OL]. [2023-08-15]. . |
26 | GUO C, XIAO L. High speed and robust RGB-thermal tracking via dual attentive stream Siamese network [C]// Proceedings of the 2022 IEEE International Geoscience and Remote Sensing Symposium. Piscataway: IEEE, 2022: 803-806. |
27 | 杨岚岚,王文琪,王福田. 基于高秩特征和位置注意力的RGBT目标跟踪[J]. 计算机科学, 2022, 49(12): 236-243. |
YANG L L, WANG W Q, WANG F T. RGBT object tracking based on high rank feature and position attention [J]. Computer Science, 2022, 49(12): 236-243. | |
28 | 王福田,张淑云,李成龙,等. 动态模态交互和特征自适应融合的RGBT跟踪[J]. 中国图象图形学报, 2022, 27(10): 3010-3021. |
WANG F T, ZHANG S Y, LI C L, et al. RGBT tracking based on dynamic modal interaction and adaptive feature fusion[J]. Journal of Image and Graphics, 2022, 27(10): 3010-3021. | |
29 | CHEN Z, ZHONG B, LI G, et al. Siamese box adaptive network for visual tracking [C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 6667-6676. |
30 | KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks [J]. Communications of the ACM, 2017, 60(6): 84-90. |
31 | HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778. |
32 | SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [EB/OL]. (2015-04-10) [2023-06-15]. . |
33 | CUI S, MA A, WAN Y, et al. Cross-modality image matching network with modality-invariant feature representation for airborne-ground thermal infrared and visible datasets [J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: No.5606414. |
34 | HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132-7141. |
35 | CHENG S, ZHONG B, LI G, et al. Learning to filter: Siamese relation network for robust tracking [C]// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 4419-4429. |
36 | YU J, JIANG Y, WANG Z, et al. UnitBox: an advanced object detection network [C]// Proceedings of the 24th ACM International Conference on Multimedia. New York: ACM, 2016: 516-520. |
37 | LI C, CHENG H, HU S, et al. Learning collaborative sparse representation for grayscale-thermal tracking [J]. IEEE Transactions on Image Processing, 2016, 25(12): 5743-5756. |
38 | ZHANG P, WANG D, LU H, et al. Learning adaptive attribute-driven representation for real-time RGB-T tracking [J]. International Journal of Computer Vision, 2021, 129(9): 2714-2729. |
39 | ZHU Y, LI C, TANG J, et al. RGBT tracking by trident fusion network [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(2): 579-592. |
40 | TU Z, LIN C, ZHAO W, et al. M5L: multi-modal multi-margin metric learning for RGBT tracking [J]. IEEE Transactions on Image Processing, 2022, 31: 85-98. |
41 | LI C, LIANG X, LU Y, et al. RGB-T object tracking: benchmark and baseline [J]. Pattern Recognition, 2019, 96: No.106977. |
42 | KRISTAN M, MATAS J, LEONARDIS A, et al. The seventh visual object tracking VOT2019 challenge results [C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshops. Piscataway: IEEE, 2019: 2206-2241. |
43 | ZHU Y, LI C, TANG J, et al. Quality-aware feature aggregation network for robust RGBT tracking [J]. IEEE Transactions on Intelligent Vehicles, 2021, 6(1): 121-130. |
44 | LU A, QIAN C, LI C, et al. Duality-gated mutual condition network for RGBT tracking [J]. IEEE Transactions on Neural Networks and Learning Systems, 2022(Early Access): 1-14. |
45 | ZHANG H, ZHANG L, ZHUO L, et al. Object tracking in RGB-T videos using modal-aware attention network and competitive learning [J]. Sensors, 2020, 20(2): No.393. |
46 | ZHANG L, DANELLJAN M, GONZALEZ-GARCIA A, et al. Multi-modal fusion for end-to-end RGB-T tracking [C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshops. Piscataway: IEEE, 2019:2252-2261. |
47 | LI C L, LU A, ZHENG A H, et al. Multi-adapter RGBT tracking [C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshops. Piscataway: IEEE, 2019: 2262-2270. |
48 | ZHU Y, LI C, LUO B, et al. Dense feature aggregation and pruning for RGBT tracking [C]// Proceedings of the 27th ACM International Conference on Multimedia. New York: ACM, 2019: 465-472. |
49 | LI C, LIU L, LU A, et al. Challenge-aware RGBT tracking [C]// Proceedings of the 2020 European Conference on Computer Vision, LNCS 12367. Cham: Springer, 2020: 222-237. |
50 | GAO Y, LI C, ZHU Y, et al. Deep adaptive fusion network for high performance RGBT tracking [C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshops. Piscataway: IEEE, 2019:91-99. |
[1] | Fangxin NIE, Yujia WANG, Xin JIA. Teaching and learning information interactive particle swarm optimization algorithm [J]. Journal of Computer Applications, 2022, 42(3): 874-882. |
[2] | Taiming FU, Yan CHEN, Taoshen LI. Hard-negative sample mining for metric learning based on linear assignment [J]. Journal of Computer Applications, 2020, 40(2): 352-357. |
[3] | JIANG Yifan, YE Qing. Time series similarity measure based on Siamese neural network [J]. Journal of Computer Applications, 2019, 39(4): 1041-1045. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||