计算机应用 ›› 2021, Vol. 41 ›› Issue (7): 2137-2144.DOI: 10.11772/j.issn.1001-9081.2020081334

所属专题: 前沿与综合应用

• 前沿与综合应用 • 上一篇    下一篇

基于注意力的毫米波-激光雷达融合目标检测

李朝1,2, 兰海1, 魏宪1   

  1. 1. 中国科学院海西研究院 泉州装备制造研究所, 福建 泉州 362216;
    2. 中北大学 电气与控制工程学院, 太原 036005
  • 收稿日期:2020-09-01 修回日期:2020-11-28 出版日期:2021-07-10 发布日期:2020-12-17
  • 通讯作者: 兰海
  • 作者简介:李朝(1994-),男,江西萍乡人,硕士研究生,主要研究方向:三维目标检测、传感器融合;兰海(1988-),男,福建莆田人,助理研究员,硕士,主要研究方向:机器学习与模式识别及其在医疗影像中的应用;魏宪(1986-),男,河南泌阳人,研究员,博士,CCF会员,主要研究方向:机器学习与模式识别及其在无人系统中的应用。
  • 基金资助:
    国家自然科学基金青年科学基金资助项目(61806186);福建省智能物流产业技术研究院建设项目(2018H2001);机器人技术与系统国家重点实验室(HIT)资助项目(SKLRS-2019-KF-15);泉州市科技计划项目(2019C112,2019STS08)。

Attention-based object detection with millimeter wave radar-lidar fusion

LI Chao1,2, LAN Hai1, WEI Xian1   

  1. 1. Quanzhou Institute of Equipment Manufacturing, Haixi Institute, Chinese Academy of Sciences, Quanzhou Fujian 362216, China;
    2. School of Electrical and Control Engineering, North University of China, Taiyuan Shanxi 036005, China
  • Received:2020-09-01 Revised:2020-11-28 Online:2021-07-10 Published:2020-12-17
  • Supported by:
    This work is partially supported by the Youth Program of the National Science Foundation of China (61806186), the Construction Program of Fujian Intelligent Logistics Industry Technology Research Institute (2018H2001), the Programe of State Key Laboratory of Robotics and System (Harbin Institute of Technology) (SKLRS-2019-KF-15), the Quanzhou Science and Technology Program (2019C112, 2019STS08).

摘要: 针对自动驾驶中使用激光雷达进行目标检测时漏检被遮挡目标、远距离目标和复杂天气场景下目标的问题,提出一种基于注意力机制的毫米波-激光雷达特征融合的目标检测方法。首先,将毫米波和激光雷达各自的扫描帧数据分别聚合到它们的标注帧上,并将毫米波和激光雷达的数据点进行空间对齐;其次,对两者进行聚合和空间对齐后的数据分别进行PointPillar点云柱快速编码,转换成伪图像;最后,通过中间卷积层提取两者的传感器特征,并利用注意力机制对两者的特征图进行融合,融合后的特征图通过单阶段检测器得到检测结果。实验结果显示,该融合算法在nuScenes数据集中的平均精度均值(mAP)高于PointPillar基础网络,而且注意力融合的检测方法的性能表现优于利用拼接融合、相乘融合、相加融合的检测方法。可视化结果显示所提方法是有效的,能提高网络对被遮挡目标、远处目标和雨雾天气下目标检测的鲁棒性。

关键词: 自动驾驶, 目标检测, 传感器融合, 注意力机制, 激光雷达, 毫米波雷达

Abstract: To address problems of missing occluded objects, distant objects and objects in extreme weather scenarios when using lidar for object detection in autonomous driving, an attention-based object detection method with millimeter wave radar-lidar feature fusion was proposed. Firstly, the scan frame data of millimeter wave radar and lidar were aggregated into their respective labeled frames, and the points of millimeter wave radar and lidar were spatially aligned, then PointPillar was employed to encode both the millimeter wave radar and lidar data into pseudo images. Finally, the features of both millimeter wave radar and lidar sensors were extracted by the middle convolution layer, and the features maps of them were fused by attention mechanism, and the fused feature map was passed through a single-stage detector to obtain detection results. Experimental results on nuScenes dataset show that compared to the basic PointPillar network, the mean Average Precision (mAP) of the proposed attention fusion algorithm is higher, which performs better than concatenation fusion, multiply fusion and add fusion methods. The visualization results show that the proposed method is effective and can improve the robustness of the network for detecting occluded objects, distant objects and objects surrounded by rain and fog.

Key words: autonomous driving, object detection, sensor fusion, attention mechanism, lidar, millimeter wave radar

中图分类号: