Specified object tracking of unmanned aerial vehicle based on Siamese region proposal network

doi:10.11772/j.issn.1001-9081.2020060762

Abstract

Abstract: Object tracking based on Siamese network has made some progresses, that is it overcomes the limitation of the spatial invariance of Siamese network in the deep network. However, there are still factors such as appearance changes, scale changes, and occlusions that affect tracking performance. Focusing on the problems of large changes in object scale, object motion blur and small scale of object in the specified object tracking of Unmanned Aerial Vehicles (UAV), a new tracking algorithm was proposed based on the Siamese region proposal attention mechanism network, namely Attention-SiamRPN+. Firstly, an improved deep residual network ResNet-50 was employed as a feature extractor to extract feature maps. Secondly, the channel attention mechanism module was used to filter the semantic information of different channel feature maps extracted by the residual network, and the corresponding weights to different channel features were reassigned. Thirdly, a hierarchical fusion of two Region Proposal Networks (RPN) was applied. The RPN module was consisted of channel-by-channel deep cross-correlation of feature maps, classification of positive and negative samples and bounding box regression. Finally, the box of the object position was selected. In the test on the VOT2018 platform, the proposed algorithm had the accuracy of 59.4% and the Expected Average Overlap (EAO) of 39.5%. In the experiment with one-pass evaluation mode on the OTB2015 platform, the algorithm had the success rate and precision of 68.7% and 89.4% respectively. Experimental results show that the evaluation results of the proposed algorithm are better than the results of three excellent correlation filtering tracking and Siamese network tracking algorithms in recent years, and the proposed algorithm has good robustness and real-time processing speed when applying to the tracking of specified objects of UAV.

Key words: Siamese network, deep residual network, attention mechanism, Unmanned Aerial Vehicle (UAV), object tracking

摘要： 基于孪生网络的目标跟踪目前取得了阶段性进展，即克服了孪生网络的空间不变性在深度网络中的限制，然而其仍存在外观变化、尺度变化、遮挡等因素影响跟踪性能。针对无人机（UAV）指定目标跟踪中的目标尺度变化大、目标运动模糊及目标尺度小等问题，提出了基于孪生区域候选注意力机制网络的跟踪算法Attention-SiamRPN+。首先，采用改进的深度残差网络ResNet-50作为特征提取器来提取特征；接着，使用通道注意力机制模块筛选残差网络提取出的不同通道特征图的语义属性，并重新为不同通道特征分配相应权值；然后，两个区域候选网络（RPN）进行分层融合，而RPN模块包括特征图的逐通道深度互相关、正负样本分类和边界框回归；最后框选出目标位置。在VOT2018平台上进行测试，所提算法的准确率和预期平均重叠率（EAO）分别为59.4%和39.5%；在OTB2015平台上采用一次通过评估模式进行实验，该算法的成功率和精度分别为68.7%和89.4%。实验结果表明所提算法的评估结果优于近年优秀的三种相关滤波跟踪算法和孪生网络跟踪算法，且该算法应用于UAV指定目标的跟踪上时具有良好的鲁棒性和实时处理速度。

关键词: 孪生网络, 深度残差网络, 注意力机制, 无人机, 目标跟踪

CLC Number:

ZHONG Sha, HUANG Yuqing. Specified object tracking of unmanned aerial vehicle based on Siamese region proposal network[J]. Journal of Computer Applications, 2021, 41(2): 523-529.

钟莎, 黄玉清. 基于孪生区域候选网络的无人机指定目标跟踪[J]. 计算机应用, 2021, 41(2): 523-529.

References

[1] 孟琭, 杨旭. 目标跟踪算法综述[J]. 自动化学化学报,2019,45(7):1244-1260.(MENG L,YANG X. A survey of object tracking algorithms[J]. Acta Automatica Sinica,2019,45(7):1244-1260.)
[2] TAO R,GAVVES E,SMEULDERS A W M. Siamese instance search for tracking[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2016:1420-1429.
[3] BERTINETTO L,VALMADRE J,HENRIQUES J F,et al. Fullyconvolutional Siamese networks for object tracking[C]//Proceedings of the 2016 European Conference on Computer Vision, LNCS 9914. Cham:Springer,2016:850-865.
[4] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Red Hook, NY:Curran Associates Inc.,2012:1097-1105.
[5] LI B,YAN J,WU W,et al. High performance visual tracking with Siamese region proposal network[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:8971-8980.
[6] REN S,HE K,GIRSHICK R,et al. Faster R-CNN:towards realtime object detection with region proposal networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge:MIT Press,2015:91-99.
[7] LI B,WU W,WANG Q,et al. SiamRPN++:evolution of Siamese visual tracking with very deep networks[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2019:4277-4286.
[8] HE K,ZHANG X,REN S,et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2016:770-778.
[9] LONG J,SHELHAMER E,DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:3431-3440.
[10] 马欣. 基于孪生神经网络的目标跟踪研究与应用[D]. 徐州:中国矿业大学,2019:28-39.(MA X. Research and application of object tracking based on Siamese network[D]. Xuzhou:China University of Mining and Technology,2019:28-39.)
[11] WANG Q,TENG Z,XING J,et al. Learning attentions:residual attentional Siamese network for high performance online visual tracking[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2018:4854-4863.
[12] FAN H,LING H. Siamese cascaded region proposal networks for real-time visual tracking[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:7944-7953.
[13] DENG J,DONG W,SOCHER R,et al. ImageNet:a large-scale hierarchical image database[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2009:248-255.
[14] ZHU Z, WANG Q, LI B, et al. Distractor-aware Siamese networks for visual object tracking[C]//Proceedings of the 2018 European Conference on Computer Vision,LNCS 11213. Cham:Springer,2018:103-119.
[15] KRISTAN M,LEONARDIS A,MATAS J,et al. The sixth visual object tracking VOT2018 challenge results[C]//Proceedings of the 2018 European Conference on Computer Vision,LNCS 11129. Cham:Springer,2018:3-53.
[16] WU Y,LIM J,YANG M H. Online object tracking:a benchmark[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2013:2411-2418.