Object tracking algorithm based on jointly-optimized strong-coupled Siamese region proposal network

doi:10.11772/j.issn.1001-9081.2020030297

Abstract

Abstract: For the mismatch between the maximum classification score and the best border in Region Proposal Network (RPN) during the object tracking task, a object tracking algorithm based on Strong-Coupled Siamese Region Proposal Network (SCSiamRPN) was proposed. Firstly, the Bounded IoU method was adopted to optimize the calculation of Intersection over Union (IoU) value of positive samples, and the calculation process was simplified by decomposing formulas, fixing variables, substituting differences, and constraining approximation values. Then, the loss function structures were optimized. A coupling factor with the IoU value as bound was added to the classification loss function to combine the classification task and the border regression task, so as to increase the loss values of high IoU samples. A weighted coefficient with the IoU value as main variable was added to the border regression loss function to increase the contributions of the target center samples, so as to improve the border localization precision. Simulation results showed that the proposed algorithm had the tracking precision and success rate reached 0.86 and 0.64 respectively on OTB100 dataset, both of which were improved by 3% in the comparison with those of high performance visual tracking with Siamese Region Proposal Network (SiamRPN). It is found that the proposed algorithm solves the mismatch between the maximum classification score, enhances the coupling between the classification task and the border regression task, greatly improves the tracking precision without slowing down the tracking speed.

Key words: Region Proposal Network (RPN), joint optimization, object tracking, convolutional neural network, Intersection over Union (IoU)

摘要： 针对区域推荐网络（RPN）在目标跟踪任务中出现的最大分类分数与最佳边框不匹配的问题，提出一种基于联合优化的强耦合孪生区域推荐跟踪算法（SCSiamRPN）。首先，采用Bounded IoU方法来优化正样本交并比（IoU）值的计算，通过分解公式、固定变量、替换差值和约束近似的操作来简化计算过程。然后，优化损失函数结构，通过在分类损失函数中添加以IoU值为纽带的耦合因子来联合分类任务和边框回归任务，以提升高IoU样本的损失值；通过在边框回归损失函数中添加以IoU为主变量的加权系数来提高目标中心样本的贡献，以提升边框定位精度。仿真结果显示：SCSiamRPN算法在OTB100数据集上的精度和成功率为0.86和0.64；同基于孪生区域推荐候选网络的高性能单目标跟踪（SiamRPN）算法相比，均有3%的提升。实验结果表明：SCSiamRPN算法解决了最大分类分数与最佳边框不匹配的问题，增强了分类和边框回归任务的耦合性，且在不损失跟踪速度的前提下实现了跟踪精度的较大幅度提升。

CLC Number:

TP391.4

SHI Guoqiang, ZHAO Xia. Object tracking algorithm based on jointly-optimized strong-coupled Siamese region proposal network[J]. Journal of Computer Applications, 2020, 40(10): 2822-2830.

石国强, 赵霞. 基于联合优化的强耦合孪生区域推荐网络的目标跟踪算法[J]. 计算机应用, 2020, 40(10): 2822-2830.

References

[1] 黄凯奇, 陈晓棠, 康运锋, 等. 智能视频监控技术综述[J]. 计算机学报,2015,38(6):1093-1118.(HUANG K Q,CHEN X T, KANG Y F,et al. Intelligent visual surveillance:a review[J]. Chines Journal of Computers,2015,38(6):1093-1118.)
[2] BOLME D S,BEVERIDGE J,DRAPER B A,et al. Visual object tracking using adaptive correlation filters[C]//Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2010:2544-2550.
[3] HENRIQUES J F,CASEIRO R,MARTINS P,et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]//Proceedings of the 2012 European Conference on Computer Vision, LNCS 7575. Berlin:Springer,2012:702-715.
[4] DANELLJAN M,KHAN F S,FELSBERG M,et al. Adaptive color attributes for real-time visual tracking[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2014:1090-1097.
[5] HENRIQUES J F,CASEIRO R,MARTINS P,et al. High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(3):583-596.
[6] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Red Hook, NY:Curran Associates Inc.,2012:1097-1105.
[7] CHEN L C,PAPANDREOU G,SCHROFF F,et al. Rethinking atrous convolution for semantic image segmentation[EB/OL].[2020-01-16]. https://arxiv.org/pdf/1706.05587.pdf.
[8] HE T,ZHANG Z,ZHANG H,et al. Bag of tricks for image classification with convolutional neural networks[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2019:558-567.
[9] TAO R,GAVVES E,SMEULDERS A W M. Siamese instance search for tracking[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2016:1420-1429.
[10] BERTINETTO L,VALMADRE J,HENRIQUES J F,et al. Fullyconvolutional Siamese networks for object tracking[C]//Proceedings of the 2016 European Conference on Computer Vision,LNCS 9914. Cham:Springer,2016:850-865.
[11] LI B,YAN J,WU W,et al. High performance visual tracking with Siamese region proposal network[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:8971-8980.
[12] LIU W,ANGUELOV D,ERHAN D,et al. SSD:single shot multibox detector[C]//Proceedings of the 2016 European Conference on Computer Vision,LNCS 9905. Cham:Springer, 2016:21-37.
[13] TYCHSEN-SMITH L, PETERSON L. Improving object localization with fitness NMS and bounded IoU loss[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:6877-6885.
[14] WU Y,LIM J,YANG M H. Online object tracking:a benchmark[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2013:2411-2418.
[15] WU Y,LIM J,YANG M H. Object tracking benchmark[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015,37(9):1834-1848.
[16] KRISTAN M, MATS J, LEONRDIS A, et al. A novel performance evaluation methodology for single-target trackers[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016,38(11):2137-2155.
[17] ERGUN H,SERT M. Fusing deep convolutional networks for large scale visual concept classification[C]//Proceedings of the IEEE 2nd International Conference on Multimedia Big Data. Piscataway:IEEE,2016:210-213.
[18] LI B, LIU Y, WANG X. Gradient harmonized single-stage detector[C]//Proceedings of the 2019 AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press,2019:8577-8584.
[19] PASZKE A,GROSS S,MASSA F,et al. PyTorch:an imperative style,high-performance deep learning library[C]//Proceedings of the 32rd Conference on Neural Information Processing Systems. Cambridge:MIT Press,2019:8024-8035.
[20] RUSSAKOVSKY O,DENG J,SU H,et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision,2015,115(3):211-252.
[21] REAL E, SHLENS J, MAZZOCCHI S, et al. YouTubeBoundingBoxes:a large high-precision human-annotated data set for object detection in video[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:7464-7473.
[22] KRISTAN M,LEONRDIS A,MATAS J,et al. The sixth visual object tracking VOT2018 challenge results[C]//Proceedings of the 2018 European Conference on Computer Vision, LNCS 11129. Cham:Springer,2019:3-53.
[23] LIANG P,BLASCH E,LING H. Encoding color information for visual tracking:algorithms and benchmark[J]. IEEE Transactions on Image Processing,2015,24(12):5630-5644.
[24] DANELLJAN M,BHAT G,KHAN F S,et al. ECO:efficient convolution operators for tracking[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:6631-6639.
[25] BERTINETTO L,VALMADRE J,GOLODETZ S,et al. Staple:complementary learners for real-time tracking[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1401-1409.
[26] DANELLJAN M, HÄGER G, KHAN F S, et al. Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE,2015:4310-4318.