Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (11): 3242-3250.DOI: 10.11772/j.issn.1001-9081.2021020327

• Artificial intelligence • Previous Articles     Next Articles

Object detection method based on radar and camera fusion

Jie GAO1, Yuan ZHU2(), Ke LU2   

  1. 1.Chinesisch-Deutsches Hochschulkolleg,Tongji University,Shanghai 200092,China
    2.School of Automotive Studies,Tongji University,Shanghai 201804,China
  • Received:2021-03-05 Revised:2021-04-15 Accepted:2021-04-20 Online:2021-04-29 Published:2021-11-10
  • Contact: Yuan ZHU
  • About author:GAO Jie,born in 1996,M. S. candidate. Her research interests include multi-sensor fusion,multi-object tracking,object detection
    ZHU Yuan, born in 1976, Ph. D., associate professor. His research interests include electrical drive system of new energy vehicles, embedded software for automotive electronics,multi-sensor fusion for intelligent driving
    LU Ke,born in 1983,Ph. D.,engineer. His research interests include automotive embedded system for automotive electronics, perception algorithms for autonomous vehicles,functional safety.

基于雷达和相机融合的目标检测方法

高洁1, 朱元2(), 陆科2   

  1. 1.同济大学 中德学院,上海 200092
    2.同济大学 汽车学院,上海 201804
  • 通讯作者: 朱元
  • 作者简介:高洁(1996—),女,贵州六盘水人,硕士研究生,主要研究方向:多传感器融合、多目标跟踪、目标检测
    朱元(1976—),男,江苏 泰州人,副教授,博士,主要研究方向:新能源汽车电气驱动系统、汽车电子嵌入式软件、智能驾驶多传感器融合
    陆科(1983—),男,江苏常州 人,工程师,博士,主要研究方向:汽车电子嵌入式系统、自动驾驶汽车感知算法、功能安全。

Abstract:

In the automatic driving perception system, multi-sensor fusion is usually used to improve the reliability of the perception results. Aiming at the task of object detection in fusion perception system, a object detection method based on radar and camera fusion, namely Priori and Radar Region Proposal Network (PRRPN), was proposed,with the aim of using radar measurement and the object detection result of the previous frame to improve the generation of region proposals in the image detection network and improve the object detection performance. Firstly, the objects detected in the previous frame with the radar points in the current frame were associated to pre-classify the radar points. Then, the pre-classified radar points were projected into the image, and the corresponding prior region proposals and radar region proposals were obtained according to the distance of the radar and Radar Cross Section (RCS) information. Finally, the regression and classification of the object bounding boxes were performed according to the region proposals. In addition, PRRPN and Region Proposal Network (RPN) were fused to carry out object detection. The newly released nuScenes dataset was adopted to test and evaluate the three detection methods. Experimental results show that, compared with RPN, the proposed PRRPN can not only detect objects faster, but also increase the average detection accuracy of small objects by 2.09 percentage points. And compared with the methods by using PRRPN and RPN alone, the method by fusing the proposed PRRPN and RPN has the average detection accuracy increased by 2.54 percentage points and 0.34 percentage points respectively.

Key words: object detection, neural network, sensor fusion, radar, camera

摘要:

在自动驾驶感知系统中,为了提高感知结果的可靠度,通常采用多传感器融合的方法。针对融合感知系统中的目标检测任务,提出了基于雷达和相机融合的目标检测方法——PRRPN,旨在使用雷达测量和前一帧目标检测结果来改进图像检测网络中的候选区域生成,并提高目标检测性能。首先,将前一帧检测到的目标与当前帧中的雷达点进行关联,以实现雷达预分类。然后,将预分类后的雷达点投影到图像中,并根据雷达的距离和雷达散射截面积(RCS)信息获得相应的先验候选区域和雷达候选区域。最后,根据候选区域进行目标边界框的回归和分类。此外,还将PRRPN与区域生成网络(RPN)融合到一起来进行目标检测。使用新发布的nuScenes数据集来对三种检测方法进行测试评估。实验结果表明,与RPN相比,PRRPN不仅可以更快速地实现目标检测,而且还使得小目标的平均检测精度提升了2.09个百分点;而将所提PRRPN与RPN进行融合的方法,与单独使用PRRPN和RPN相比,平均检测精度分别提升了2.54个百分点和0.34个百分点。

关键词: 目标检测, 神经网络, 传感器融合, 雷达, 相机

CLC Number: