Abstract: Aiming at the problems of low accuracy of driving command prediction, bulky model structure and a large amount of information redundancy in existing end-to-end autonomous driving methods, a new end-to-end autonomous driving model based on deep visual attention neural network was proposed. In order to effectively extract features of autonomous driving scenes, a deep visual attention neural network, which is composed of the convolutional neural network, the visual attention layer and the long short-term memory network, was proposed by introducing a visual attention mechanism into the end-to-end autonomous driving model. The proposed model was able to effectively extract spatial and temporal features of driving scene images, focus on important information and reduce information redundancy for realizing the end-to-end autonomous driving that predicts driving commands from sequential images input by front-facing camera. The data from a simulated driving environment were used for training and testing. The root mean square errors of the proposed model for prediction of the steering angle in four scenes including country road, highway, tunnel and mountain road are 0.009 14, 0.009 48, 0.002 89 and 0.010 78 respectively, which are all lower than the results of the method proposed by NVIDIA and the method based on the deep cascaded neural network. Moreover, the proposed model has fewer network layers compared with the networks without the visual attention mechanism.

Key words: autonomous driving, end-to-end, visual attention, convolutional neural network, long short-term memory network

摘要： 针对现有端到端自动驾驶方法中存在的驾驶指令预测不准确、模型结构体量大和信息冗余多等问题，提出一种新的基于深度视觉注意神经网络的端到端自动驾驶模型。为了更有效地提取自动驾驶场景的特征，在端到端自动驾驶模型中引入视觉注意力机制，将卷积神经网络、视觉注意层和长短期记忆网络进行融合，提出一种深度视觉注意神经网络。该网络模型能够有效提取驾驶场景图像的空间特征和时间特征，并关注重要信息且减少信息冗余，实现用前向摄像机输入的序列图像来预测驾驶指令的端到端自动驾驶。利用模拟驾驶环境的数据进行训练和测试，该模型在乡村路、高速路、隧道和山路四个场景中对方向盘转向角预测的均方根误差分别为0.009 14、0.009 48、0.002 89和0.010 78，均低于对比用的英伟达公司提出的方法和基于深度级联神经网络的方法；并且与未使用视觉注意力机制的网络相比，该模型具有更少的网络层数。

关键词: 自动驾驶, 端到端, 视觉注意力, 卷积神经网络, 长短期记忆网络