Monocular vision obstacle avoidance method for quadcopter based on deep learning

doi:10.11772/j.issn.1001-9081.2018091952

Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (4): 1001-1005.DOI: 10.11772/j.issn.1001-9081.2018091952

Previous Articles Next Articles

Monocular vision obstacle avoidance method for quadcopter based on deep learning

ZHANG Wuyang^1,2, ZHANG Wei¹, SONG Fang¹, LONG Lin²

1. Laboratory of Intelligent Control and Robotics, Shanghai University of Engineering Science, Shanghai 201620, China;
2. College of Mechanical and Automobile Engineering, Shanghai University of Engineering Science, Shanghai 201620, China

Received:2018-09-20 Revised:2018-11-15 Online:2019-04-10 Published:2019-04-10
Supported by:
This work is partially supported by the National Natural Science Foundation of China (51505273).

基于深度学习的四旋翼无人机单目视觉避障方法

张午阳^1,2, 章伟¹, 宋芳¹, 龙林²

1. 上海工程技术大学机器人智能控制实验室, 上海 201620;
2. 上海工程技术大学机械与汽车工程学院, 上海 201620

通讯作者: 宋芳
作者简介:张午阳(1994-),男,安徽淮北人,硕士研究生,主要研究方向:无人机避障、计算机视觉;章伟(1977-),男,安徽桐城人,副教授,博士,主要研究方向:非线性控制与观测器、鲁棒控制、无人机控制;宋芳(1980-),女,黑龙江大庆人,副教授,博士,主要研究方向:机器人、运动控制;龙林(1976-),男,安徽桐城人,硕士,主要研究方向:智能化控制、物联网。
基金资助:
国家自然科学基金资助项目（51505273）。

Abstract

Abstract: A monocular vision obstacle avoidance method for quadrotor based on deep learning was proposed to help quadrotors to avoid obstacles. Firstly, the position of object in the image was obtained by object detection, and by calculating the height of the object box in the image, the distance between quadcopter and obstacle was estimated. Then, whether performing obstacle avoidance was determined by synergetic computer. Finally, experiments were conducted on a flight test platform based on Pixhawk flight control board. The results show that the proposed method can be applied to quadcoptor obstacle avoidance with low speed. Compared with traditional active sensor methods, the proposed method greatly reduces the occupied volume with only one monocular camera as sensor. This method is robust and can identify people with different postures as obstacles.

Key words: deep learning, object detection, monocular vision, quadcopter obstacle avoidance

摘要： 针对无人机避障问题，提出一种基于深度学习的四旋翼无人机单目视觉避障方法。首先通过目标检测框选出目标在图像中的位置，并通过计算目标选框上下边距的长度，以此来估量出障碍物到无人机之间的距离；然后通过协同计算机判断是否执行避障动作；最后使用基于Pixhawk搭建的飞行实验平台进行实验。实验结果表明，该方法可用于无人机低速飞行条件下避障。该方法所用到的传感器只有一块单目摄像头，而且相对于传统的主动式传感器避障方法，所占用无人机的体积大幅减小。该方法鲁棒性较好，能够准确识别不同姿态的人，实现对人避障。

关键词: 深度学习, 目标检测, 单目视觉, 无人机避障

CLC Number:

TP242

ZHANG Wuyang, ZHANG Wei, SONG Fang, LONG Lin. Monocular vision obstacle avoidance method for quadcopter based on deep learning[J]. Journal of Computer Applications, 2019, 39(4): 1001-1005.

张午阳, 章伟, 宋芳, 龙林. 基于深度学习的四旋翼无人机单目视觉避障方法[J]. 计算机应用, 2019, 39(4): 1001-1005.

References

[1] 杨秀霞, 刘小伟, 张毅. 基于时间约束的无人机避障研究[J]. 飞行力学, 2015, 33(2):125-129. (YANG X X, LIU X W, ZHANG Y. Research on obstacle avoidance of UAV based on time constraint[J]. Flight Mechanics, 2015, 33(2):125-129.)
[2] 高迪. 无人机避障雷达目标探测与跟踪算法研究[D]. 哈尔滨:哈尔滨工业大学, 2017:10-17. (GAO D. Research on target detection and tracking algorithm for UAV obstacle avoidance radar[D]. Harbin:Harbin Institute of Technology, 2017:10-17.)
[3] 朱平, 甄子洋, 覃海群,等. 基于立体视觉和光流的无人机避障算法研究[J]. 电光与控制, 2017, 24(12):31-35. (ZHU P, ZHEN Z Y, QIN H Q, el al. Research on UAV obstacle avoidance algorithm based on stereo vision and optical flow[J]. Electronics Optics & Control, 2017, 24(12):31-35.)
[4] YANG Y, WANG T T, CEHN L, et al. Stereo vision based obstacle avoidance strategy for quadcopter UAV[C]//Proceedings of the 2018 Chinese Control and Decision Conference. Piscataway, NJ:IEEE, 2018:490-494.
[5] HU Y Y, WANG Y X. Stereo vision-based fast obstacles avoidance without obstacles discrimination for indoor UAVs[C]//Proceedings of the 20112nd International Conference on Artificial Intelligence, Management Science and Electronic Commerce. Piscataway, NJ:IEEE, 2011:4332-4337.
[6] HU J, NIU Y, WANG Z. Obstacle avoidance methods for rotor UAVs using RealSense camera[C]//Proceedings of the 2017 Chinese Automation Congress. Piscataway, NJ:IEEE, 2017:7151-7155.
[7] XU Z, WER R, ZHANG Q, et al. Obstacle avoidance algorithm for UAVs in unknown environment based on distributional perception and decision making[C]//Proceedings of the 2016 IEEE Chinese Guidance, Navigation and Control Conference. Piscataway, NJ:IEEE, 2016:1072-1075.
[8] GAGEIK N, BENZ P, MONTENEGRO S. Obstacle detection and collision avoidance for a UAV with complementary low-cost sensors[J]. IEEE Access, 2015, 3:599-609.
[9] RⅡSGAARD S, BLAS M R. SLAM for dummies:a tutorial approach to simultaneous localization and mapping[EB/OL].[2018-05-10]. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.208.6289&rep=rep1&type=pdf.
[10] 陈炜楠, 刘冠峰, 李俊良, 等. 室内环境的元胞自动机SLAM算法[J]. 机器人, 2016, 38(2):169-177. (CHEN W N, LIU G F, LI J L, et al. Cellular automaton SLAM algorithm for indoor environment[J]. Robot, 2016, 38(2):169-177.)
[11] 徐伟杰, 李平, 韩波. 基于2点RANSAC的无人直升机单目视觉SLAM[J]. 机器人, 2012, 34(1):65-71. (XU W J, LI P, HAN B. Unmanned helicopter monocular vision SLAM based on 2 points RANSAC[J]. Robot, 2012, 34(1):65-71.)
[12] BAI G, XIANG X, ZHU H, et al. Research on obstacles avoidance technology for UAV based on improved PTAM algorithm[C]//Proceedings of the 2015 IEEE International Conference on Progress in Informatics and Computing. Piscataway, NJ:IEEE, 2015:543-550.
[13] CHEN Z, LUO X, DAI B. Design of obstacle avoidance system for micro-UAV based on binocular vision[C]//Proceedings of the 2017 International Conference on Industrial Informatics - Computing Technology, Intelligent Technology, Industrial Information Integration. Piscataway, NJ:IEEE, 2017:67-70.
[14] HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786):504-507.
[15] ABBAS M A. Improving deep learning performance using random forest HTM cortical learning algorithm[C]//Proceedings of the 2018 First International Workshop on Deep and Representation Learning. Piscataway, NJ:IEEE, 2018:13-18.
[16] WANG Y. Cognitive foundations of knowledge science and deep knowledge learning by cognitive robots[C]//Proceedings of the 2017 IEEE 16th International Conference on Cognitive Informatics & Cognitive Computing. Piscataway, NJ:IEEE, 2017:5.
[17] GOODFELLOW I, BENGIO Y, COURVILLE A, et al. Deep learning[M]. Cambridge, MA:MIT Press, 2016:11-12.
[18] 杜学丹, 蔡莹皓, 鲁涛, 等. 一种基于深度学习的机械臂抓取方法[J]. 机器人, 2017, 39(6):820-828, 837. (DU X D, CAI Y H, LU T, et al. A mechanical arm grabbing method based on deep learning[J]. Robot, 2017, 39(6):820-828, 837.)
[19] ZOU J, SONG R. Microarray camera image segmentation with faster-RCNN[C]//Proceedings of the 2018 IEEE International Conference on Applied System Invention. Piscataway, NJ:IEEE, 2018:86-89.
[20] LECUN Y, KAVUKCUOGLU K, FARABET C. Convolutional networks and applications in vision[C]//Proceedings of the 2010 IEEE International Symposium on Circuits and Systems. Piscataway, NJ:IEEE, 2010:253-256.
[21] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2014:580-587.
[22] GIRSHICK R. Fast R-CNN[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2015:1440-1448.
[23] REN S, HE K, GIRSHICK R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
[24] ZHENG W, XIAO J, XIN T. Integrated navigation system with monocular vision and LIDAR for indoor UAVs[C]//Proceedings of the 2017 12th IEEE Conference on Industrial Electronics and Applications. Piscataway, NJ:IEEE, 2017:924-929.
[25] GUANGLEI M, HAIBING P. The application of ultrasonic sensor in the obstacle avoidance of quad-rotor UAV[C]//Proceedings of the 2016 IEEE Chinese Guidance, Navigation and Control Conference. Piscataway, NJ:IEEE, 2016:976-981.
[26] DIXIT K R, KRISHNA P P, ANTONY R. Design and development of H frame quadcopter for control system with obstacle detection using ultrasound sensors[C]//Proceedings of the 2017 International Conference on Circuits, Controls, and Communications. Piscataway, NJ:IEEE, 2017:100-104.

Monocular vision obstacle avoidance method for quadcopter based on deep learning

基于深度学习的四旋翼无人机单目视觉避障方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	ZHENG Zhiqiang, HU Xin, WENG Zhi, WANG Yuhe, CHENG Xi. Cattle eye image feature extraction method based on improved DenseNet [J]. Journal of Computer Applications, 2021, 41(9): 2780-2784.
[2]	CHEN Chengrui, SUN Ning, HE Shibiao, LIAO Yong. Deep learning-based joint channel estimation and equalization algorithm for C-V2X communications [J]. Journal of Computer Applications, 2021, 41(9): 2687-2693.
[3]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[4]	XU Jianglang, LI Linyan, WAN Xinjun, HU Fuyuan. Indoor scene recognition method combined with object detection [J]. Journal of Computer Applications, 2021, 41(9): 2720-2725.
[5]	XIE Defeng, JI Jianmin. Syntax-enhanced semantic parsing with syntax-aware representation [J]. Journal of Computer Applications, 2021, 41(9): 2489-2495.
[6]	DAI Yurou, YANG Qing, ZHANG Fengli, ZHOU Fan. Trajectory prediction model of social network users based on self-supervised learning [J]. Journal of Computer Applications, 2021, 41(9): 2545-2551.
[7]	MA Jialiang, CHEN Bin, SUN Xiaofei. General object detection framework based on improved Faster R-CNN [J]. Journal of Computer Applications, 2021, 41(9): 2712-2719.
[8]	HE Zhenghai, XIAN Yantuan, WANG Meng, YU Zhengtao. Case reading comprehension method combining syntactic guidance and character attention mechanism [J]. Journal of Computer Applications, 2021, 41(8): 2427-2431.
[9]	FAN Wei, LI Chenxuan, XING Yan, HUANG Rui, PENG Hongjian. Binary classification to multiple classification progressive detection network for aero-engine damage images [J]. Journal of Computer Applications, 2021, 41(8): 2352-2357.
[10]	CHEN Jing, MAO Yingchi, CHEN Hao, WANG Longbao, WANG Zicheng. Dam defect object detection method based on improved single shot multibox detector [J]. Journal of Computer Applications, 2021, 41(8): 2366-2372.
[11]	CAO Yuhong, XU Hai, LIU Sun'ao, WANG Zixiao, LI Hongliang. Review of deep learning-based medical image segmentation [J]. Journal of Computer Applications, 2021, 41(8): 2273-2287.
[12]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[13]	XIAO Zhenyuan, WANG Yihan, LUO Jianqiao, XIONG Ying, LI Bailin. RefineDet based on subsection weighted loss function [J]. Journal of Computer Applications, 2021, 41(7): 1928-1932.
[14]	LI Yafang, LIANG Ye, FENG Weiwei, ZU Baokai, KANG Yujian. Deep network embedding method based on community optimization [J]. Journal of Computer Applications, 2021, 41(7): 1956-1963.
[15]	GAO Qinquan, HUANG Bingcheng, LIU Wenzhe, TONG Tong. Bamboo strip surface defect detection method based on improved CenterNet [J]. Journal of Computer Applications, 2021, 41(7): 1933-1938.