《计算机应用》唯一官方网站 ›› 2022, Vol. 42 ›› Issue (3): 968-973.DOI: 10.11772/j.issn.1001-9081.2021030414

• 多媒体计算与计算机仿真 • 上一篇    

基于特征融合的三维人脸点云质量判断

高工1, 杨红雨1,2, 刘洪1,2()   

  1. 1.视觉合成图形图像技术国防重点学科实验室(四川大学),成都 610065
    2.四川大学 计算机学院,成都 610065
  • 收稿日期:2021-03-19 修回日期:2021-06-22 接受日期:2021-06-23 发布日期:2022-04-09 出版日期:2022-03-10
  • 通讯作者: 刘洪
  • 作者简介:高工(1995—),男,河南周口人,硕士研究生,主要研究方向:计算机视觉、点云识别
    杨红雨(1967—),女,四川成都人,教授,博士,主要研究方向:图像处理;
  • 基金资助:
    四川省重大科技专项(2019ZDZX0039)

Quality judgment of 3D face point cloud based on feature fusion

Gong GAO1, Hongyu YANG1,2, Hong LIU1,2()   

  1. 1.National Key Laboratory of Fundamental Science on Synthetic Vision (Sichuan University),Chengdu Sichuan 610065,China
    2.College of Computer Science,Sichuan University,Chengdu Sichuan 610065,China
  • Received:2021-03-19 Revised:2021-06-22 Accepted:2021-06-23 Online:2022-04-09 Published:2022-03-10
  • Contact: Hong LIU
  • About author:GAO Gong, born in 1995, M. S. candidate. His research interests include computer vision, point cloud recognition.
    YANG Hongyu, born in 1967, Ph. D., professor. Her research interests include image processing.
  • Supported by:
    Major Science and Technology Project of Sichuan Province(2019ZDZX0039)

摘要:

针对使用双目结构光扫描仪获取的三维人脸点云,提出了一种特征融合网络(FFN)来完成人脸点云质量判断任务。首先,对三维点云预处理切割出人脸面部区域,使用点云和对应的二维平面投影得到的图像作为输入;其次,分别训练用于点云学习的动态图卷积神经网络(DGCNN)和ShuffleNet两个模块;然后,提取出两个网络模块的中间层特征进行特征融合,对整个网络进行微调;最后,使用三层全连接层,实现三维人脸点云的5分类(优秀、普通、条纹、毛刺、变形)。所提FFN的分类正确率为83.7%;分类正确率比ShuffleNet提升了5.8%,比DGCNN提升了2.2%。实验结果表明,加权融合二维图像特征和点云特征可以达到不同特征之间的优势互补效果。

关键词: 人脸点云, 点云特征, 二维图像, 加权融合, 质量判断

Abstract:

A Feature Fusion Network (FFN) was proposed to judge the quality of 3D face point cloud acquired by binocular structured light scanner. Firstly, the 3D point cloud was preprocessed to cut out the face area, and the image obtained from the point cloud and the corresponding 2D plane projection was used as the input. Secondly, Dynamic Graph Convolutional Neural Network (DGCNN) and ShuffleNet were trained for point cloud learning. Then, the middle layer features of the two network modules were extracted and fused to fine-tune the whole network. Finally, three full connected layers were used to realize the five-class classification of 3D face point cloud (excellent, ordinary, stripe, burr, deformation). The proposed FFN achieved the classification accuracy of 83.7%, which was 5.8% higher than that of ShufflNet and 2.2% higher than that of DGCNN. The experimental results show that the weighted fusion of two-dimensional image features and point cloud features can achieve the complementary effect between different features.

Key words: face point cloud, point cloud feature, two-dimensional image, weighted fusion, quality judgment

中图分类号: