Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (7): 1919-1925.DOI: 10.11772/j.issn.1001-9081.2019101798

• Artificial intelligence • Previous Articles     Next Articles

Motion planning for autonomous driving with directional navigation based on deep spatio-temporal Q-network

HU Xuemin, CHENG Yu, CHEN Guowen, ZHANG Ruohan, TONG Xiuchi   

  1. School of Computer Science and Information Engineering, Hubei University, Wuhan Hubei 430062, China
  • Received:2019-10-24 Revised:2019-12-22 Online:2020-07-10 Published:2020-05-09
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61806076), the Natural Science Foundation of Hubei Province (2018CFB158), the Undergraduate Innovation and Enterpreneurship Training Plan of Hubei Province (S201910512026).

基于深度时空Q网络的定向导航自动驾驶运动规划

胡学敏, 成煜, 陈国文, 张若晗, 童秀迟   

  1. 湖北大学 计算机与信息工程学院, 武汉 430062
  • 通讯作者: 胡学敏
  • 作者简介:胡学敏(1985-),男,湖南岳阳人,副教授,博士,主要研究方向:机器学习、运动规划;成煜(1998-),男,湖北孝感人,主要研究方向:自动驾驶;陈国文(1999-),男,湖北宜昌人,主要研究方向:深度学习;张若晗(1997-),女,湖北襄阳人,硕士研究生,主要研究方向:机器学习;童秀迟(1996-),女,湖北随州人,硕士研究生,主要研究方向:机器学习。
  • 基金资助:
    国家自然科学基金资助项目(61806076);湖北省自然科学基金资助项目(2018CFB158);湖北省大学生创新创业训练计划项目(S201910512026)。

Abstract: To solve the problems of requiring a large number of samples, not associating with time information, and not using global navigation information in motion planning for autonomous driving based on machine learning, a motion planning method for autonomous driving with directional navigation based on deep spatio-temporal Q-network was proposed. Firstly, in order to extract the spatial features in images and the temporal information between continuous frames for autonomous driving, a new deep spatio-temporal Q-network was proposed based on the original deep Q-network and combined with the long short-term memory network. Then, to make full use of the global navigation information of autonomous driving, directional navigation was realized by adding the guide signal into the images for extracting environment information. Finally, based on the proposed deep spatio-temporal Q-network, a learning strategy oriented to autonomous driving motion planning model was designed to achieve the end-to-end motion planning, where the data of steering wheel angle, accelerator and brake were predicted from the input sequential images. The experimental results of training and testing results in the driving simulator named Carla show that in the four test roads, the average deviation of this algorithm is less than 0.7 m, and the stability performance of this algorithm is better than that of four comparison algorithms. It is proved that the proposed method has better learning performance, stability performance and real-time performance to realize the motion planning for autonomous driving with global navigation route.

Key words: autonomous driving, motion planning, reinforcement learning, deep spatio-temporal Q-network, directional navigation

摘要: 针对目前基于机器学习的自动驾驶运动规划需要大量样本、没有关联时间信息,以及没有利用全局导航信息等问题,提出一种基于深度时空Q网络的定向导航自动驾驶运动规划算法。首先,为提取自动驾驶的空间图像特征与前后帧的时间信息,基于原始深度Q网络,结合长短期记忆网络,提出一种新的深度时空Q网络;然后,为充分利用自动驾驶的全局导航信息,在提取环境信息的图像中加入指向信号来实现定向导航的目的;最后,基于提出的深度时空Q网络,设计面向自动驾驶运动规划模型的学习策略,实现端到端的运动规划,从输入的序列图像中预测车辆方向盘转角和油门刹车数据。在Carla驾驶模拟器中进行训练和测试的实验结果表明,在四条测试道路中该算法平均偏差均小于0.7 m,且稳定性能优于四种对比算法。该算法具有较好的学习性、稳定性和实时性,能够实现在全局导航路线下的自动驾驶运动规划。

关键词: 自动驾驶, 运动规划, 强化学习, 深度时空Q网络, 定向导航

CLC Number: