Pedestrian visual positioning algorithm for underground roadway based on deep learning

doi:10.11772/j.issn.1001-9081.2018071501

Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (3): 688-694.DOI: 10.11772/j.issn.1001-9081.2018071501

Previous Articles Next Articles

Pedestrian visual positioning algorithm for underground roadway based on deep learning

HAN Jianghong^1,2, YUAN Jiaxuan¹, WEI Xing^1,2, LU Yang^1,2

1. School of Computer and Information, Hefei University of Technology, Hefei Anhui 230601, China;
2. Engineering Research Center of Safety Critical Industry Measure and Control Technology, Ministry of Education, Hefei Anhui 230601, China

Received:2018-07-19 Revised:2018-09-01 Online:2019-03-10 Published:2019-03-11
Contact: 袁稼轩
Supported by:
This work is partially supported by the National Key Research Development Program of China (2016YFC0801800, 2016YFC0801405).

基于深度学习的井下巷道行人视觉定位算法

韩江洪^1,2, 袁稼轩¹, 卫星^1,2, 陆阳^1,2

1. 合肥工业大学计算机与信息学院, 合肥 230601;
2. 安全关键工业测控技术教育部工程研究中心, 合肥 230601

作者简介:韩江洪(1954-),男,安徽庐江人,教授,博士生导师,硕士,主要研究方向:计算机网络与通信、计算机控制、汽车电子;袁稼轩(1994-),女,新疆阿拉尔人,硕士研究生,主要研究方向:DSP及嵌入式系统;卫星(1980-),男,安徽合肥人,副教授,博士,CCF会员,主要研究方向:物联网工程、离散事件动态系统;陆阳(1967-),男,安徽合肥人,教授,博士生导师,博士,CCF高级会员,主要研究方向:分布式控制技术、无线通信网络、工业物联网。
基金资助:
国家重点研发计划专项（2016YFC0801800；2016YFC0801405）。

Abstract

Abstract: The self-driving mine locomotive needs to detect and locate pedestrians in front of it in the underground roadway in real-time. Non-visual methods such as laser radar are costly, while traditional visual methods based on feature extraction cannot solve the problem of poor illumination and uneven light in the laneway. To solve the problem, a pedestrian visual positioning algorithm for underground roadway based on deep learning was proposed. Firstly, the overall structure of the system based on deep learning network was given. Secondly, a multi-layer Convolutional Neural Network (CNN) for object detection was built to calculate the two-dimensional coordinates and the size of bounding box of pedestrians in visual field of the self-driving locomotive. Thirdly, the third-dimensional distance between the pedestrian in the image and the locomotive was calculated by polynomial fitting. Finally, the model was trained, verified and tested through real sample sets. Experimental results show that the accuracy of the proposed algorithm reaches 94%, the speed achieves 25 frames per second, and the distance detection error is less than 4%, thus efficient and real-time laneway pedestrian visual positioning is realized.

Key words: deep learning, Convolutional Neural Network (CNN), laneway pedestrian detection, visual positioning, image processing

摘要： 自主驾驶矿井机车需要实时检测和定位行驶前方的巷道行人，激光雷达等非视觉类方法成本高昂，而传统基于特征提取视觉类方法无法解决井下光照差且光线不均匀的问题。提出一种基于深度学习的井下巷道行人视觉定位算法。首先给出基于深度学习网络的系统整体结构；其次，搭建目标检测多层卷积神经网络（CNN），生成自主驾驶机车前方视野范围内行人的二维坐标及边界框的尺寸；再次，通过多项式拟合计算出图像中行人到机车之间的第三维距离；最后通过真实样本集实施模型训练、验证与测试。实验结果表明，所提算法的检测准确率达94%，速度达每秒25帧，测距误差小于4%，实现了实时高效的巷道行人视觉定位。

关键词: 深度学习, 卷积神经网络, 巷道行人检测, 视觉定位, 图像处理

CLC Number:

TP391.4

HAN Jianghong, YUAN Jiaxuan, WEI Xing, LU Yang. Pedestrian visual positioning algorithm for underground roadway based on deep learning[J]. Journal of Computer Applications, 2019, 39(3): 688-694.

韩江洪, 袁稼轩, 卫星, 陆阳. 基于深度学习的井下巷道行人视觉定位算法[J]. 计算机应用, 2019, 39(3): 688-694.

References

[1] 乔维高,徐学进.无人驾驶汽车的发展现状及方向[J].上海汽车,2007(7):40-43.(QIAO W G, XU X J. The development situation and direction of the driverless vehicle[J]. Shanghai Auto, 2007(7):40-43.)
[2] 李晓明,郎文辉,马忠磊,等.基于图像处理的井下机车行人检测技术[J].煤矿机械,2017,38(4):167-170.(LI X M, LANG W H, MA Z L, et al. Pedestrian detection technology for mine locomotive based on image processing[J]. Coal Mine Machinery, 2017, 38(4):167-170.)
[3] LIU T, FU H Y, WEN Q, et al. Extended faster R-CNN for long distance human detection:finding pedestrians in UAV images[C]//Proceedings of the 2018 IEEE International Conference on Consumer Electronics. Piscataway, NJ:IEEE, 2018:1-2.
[4] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2014:580-587.
[5] HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9):1904-1916.
[6] GIRSHICK R. Fast R-CNN[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2015:1440-1448.
[7] REDMON J, FARHADI A. YOLO9000:better, faster, stronger[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2017:6517-6525.
[8] 郑嘉祺.基于DCNN的井下行人检测系统的研究与设计[D].西安:西安科技大学,2017:84-87.(ZHENG J Q. Research and design on pedestrian detection system under the mine based on DCNN[D]. Xi'an:Xi'an University of Science and Technology, 2017:84-87.)
[9] REDMON J, DIWALA S, GIRSHICK R, et al. You only look once:unified, real-time object detection[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2016:779-788.
[10] 王琳,卫晨,李伟山,等.结合金字塔池化模块的YOLOv2的井下行人检测[EB/OL].[2018-05-21]. https://www.doc88.com/p-0714870779937.html.(WANG L, WEI C, LI W S, et al. Pedestrian detection based on YOLOv2 with pyramid pooling module in underground coal mine[EB/OL].[2018-05-21]. https://www.doc88.com/p-0714870779937.html.)
[11] 李伟山,卫晨,王琳.改进的Faster R-CNN煤矿井下行人检测算法[EB/OL].[2018-07-15]. http://kns.cnki.net/kcms/detail/11.2127.TP.20180522.0944.002.html.(LI W S, WEI C, WANG L. An improved faster R-CNN approach for pedestrian detection in underground coal mine[EB/OL].[2018-07-15]. http://kns.cnki.net/kcms/detail/11.2127.TP.20180522.0944.002.html.)
[12] REN S, HE K, GIRSHICK R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(6):1137-1149.
[13] 沈彤,刘文波,王京.基于双目立体视觉的目标测距系统[J].电子测量技术,2015,38(4):52-54.(SHEN T, LIU W B, WANG J. Distance measurement system based on binocular stereo vision[J]. Electronic Measurement Technology, 2015, 38(4):52-54.)
[14] 郭磊,徐友春,李克强,等.基于单目视觉的实时测距方法研究[J]. 中国图象图形学报,2006,11(1):74-81.(GUO L, XU Y C, LI K Q, et al. Study on real-time distance detection based on monocular vision technique[J]. Journal of Image and Graphics, 2006, 11(1):74-81.)
[15] BAO D, WANG P. Vehicle distance detection based on monocular vision[C]//Proceedings of the 2016 International Conference on Progress in Informatics and Computing. Piscataway, NJ:IEEE, 2016:187-191.
[16] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Washington, DC:IEEE Computer Society, 2017:2999-3007.

Pedestrian visual positioning algorithm for underground roadway based on deep learning

基于深度学习的井下巷道行人视觉定位算法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	XIE Defeng, JI Jianmin. Syntax-enhanced semantic parsing with syntax-aware representation [J]. Journal of Computer Applications, 2021, 41(9): 2489-2495.
[2]	DAI Yurou, YANG Qing, ZHANG Fengli, ZHOU Fan. Trajectory prediction model of social network users based on self-supervised learning [J]. Journal of Computer Applications, 2021, 41(9): 2545-2551.
[3]	CHEN Chengrui, SUN Ning, HE Shibiao, LIAO Yong. Deep learning-based joint channel estimation and equalization algorithm for C-V2X communications [J]. Journal of Computer Applications, 2021, 41(9): 2687-2693.
[4]	WANG Hebing, ZHANG Chunmei. Facial landmark detection based on ResNeXt with asymmetric convolution and squeeze excitation [J]. Journal of Computer Applications, 2021, 41(9): 2741-2747.
[5]	ZHENG Zhiqiang, HU Xin, WENG Zhi, WANG Yuhe, CHENG Xi. Cattle eye image feature extraction method based on improved DenseNet [J]. Journal of Computer Applications, 2021, 41(9): 2780-2784.
[6]	SONG Zhongshan, LIANG Jiarui, ZHENG Lu, LIU Zhenyu, TIE Jun. Remote sensing scene classification based on bidirectional gated scale feature fusion [J]. Journal of Computer Applications, 2021, 41(9): 2726-2735.
[7]	LI Kangkang, ZHANG Jing. Multi-layer encoding and decoding model for image captioning based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(9): 2504-2509.
[8]	ZHANG Yongbin, CHANG Wenxin, SUN Lianshan, ZHANG Hang. Detection method of domains generated by dictionary-based domain generation algorithm [J]. Journal of Computer Applications, 2021, 41(9): 2609-2614.
[9]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[10]	XU Jianglang, LI Linyan, WAN Xinjun, HU Fuyuan. Indoor scene recognition method combined with object detection [J]. Journal of Computer Applications, 2021, 41(9): 2720-2725.
[11]	HE Zhenghai, XIAN Yantuan, WANG Meng, YU Zhengtao. Case reading comprehension method combining syntactic guidance and character attention mechanism [J]. Journal of Computer Applications, 2021, 41(8): 2427-2431.
[12]	CAO Yuhong, XU Hai, LIU Sun'ao, WANG Zixiao, LI Hongliang. Review of deep learning-based medical image segmentation [J]. Journal of Computer Applications, 2021, 41(8): 2273-2287.
[13]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[14]	HUANG Chengcheng, DONG Xiaoxiao, LI Zhao. Deep pipeline 5×5 convolution method based on two-dimensional Winograd algorithm [J]. Journal of Computer Applications, 2021, 41(8): 2258-2264.
[15]	ZENG Xiangyin, ZHENG Bochuan, LIU Dan. Detection of left and right railway tracks based on deep convolutional neural network and clustering [J]. Journal of Computer Applications, 2021, 41(8): 2324-2329.