Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (8): 2351-2357.DOI: 10.11772/j.issn.1001-9081.2020010070

• Virtual reality and multimedia computing • Previous Articles     Next Articles

Application of deep learning to 3D model reconstruction of single image

ZHANG Hao1,2, ZHANG Qiang2, SHAO Siyu2, DING Haibin3   

  1. 1. Graduate School, Air Force Engineering University, Xi'an Shannxi 710038, China;
    2. Air and Missile Defense College, Air Force Engineering University, Xi'an Shannxi 710038, China;
    3. Training Base, Army Engineering University of PLA, XuzhouJiangsu 221004, China
  • Received:2020-01-22 Revised:2020-03-31 Online:2020-08-10 Published:2020-03-31
  • Supported by:
    This work is partially supported by the Scientific Research Innovation Plan for Graduate Students of Academic Degree in Colleges and Universities in Jiangsu Province (KYCX18_0072).

深度学习在单图像三维模型重建的应用

张豪1,2, 张强2, 邵思羽2, 丁海斌3   

  1. 1. 空军工程大学 研究生院, 西安 710038;
    2. 空军工程大学 防空反导学院, 西安 710038;
    3. 陆军工程大学 训练基地, 江苏徐州 221004
  • 通讯作者: 张豪(1995-),男,福建福州人,硕士研究生,主要研究方向:深度学习、模式识别;421821467@qq.com
  • 作者简介:张强(1973-),男,陕西汉中人,副教授,博士,主要研究方向:深度学习、电力系统及其自动化;邵思羽(1991-),女,山东邹城人,讲师,博士,主要研究方向:基于深度学习、迁移学习的机电设备健康状态检测与故障诊断;丁海斌(1985-),男,山东荣成人,讲师,硕士,主要研究方向:防空反导模拟训练。
  • 基金资助:
    江苏省普通高校学术学位研究生科研创新计划项目(KYCX18_0072)。

Abstract: To solve the problem that the reconstructed 3D model of a single image has high uncertainty, a network model based on depth image estimation, spherical projection mapping and 3D generative adversarial network was proposed. Firstly, the depth image of the input image was obtained by the depth estimator, which was helpful for the further analysis of the image. Secondly, the obtained depth image was converted into a 3D model by spherical projection mapping. Finally, 3D generative adversarial network was utilized to judge the authenticity of the reconstructed 3D model, so as to obtain 3D model closer to reality. In the comparison experiments with LVP algorithm which learning view priors for 3D reconstruction, the proposed model has the Intersection-over-Union (IoU) increased by 20.1% and the Charmfer Distance (CD) decreased by 13.2%. Theoretical analysis and simulation results show that the proposed model has good generalization ability in the 3D model reconstruction of a single image.

Key words: depth image, depth estimation, 3D reconstruction, Generative Adversarial Network (GAN), spherical projection

摘要: 针对基于单图像重建的三维模型具有高度不确定性问题,提出了一种基于深度图像估计、球面投影映射、三维对抗生成网络相结合的网络模型算法。首先,通过深度估计器得到输入图像的深度图像,这有利于对图像进一步的分析;其次,将得到的深度图像通过球面投影映射转换为三维模型;最后,利用三维对抗生成网络对重建的三维模型的真实性进行判断,建立更逼真的三维模型。理论分析和仿真实验表明,与学习先验知识生成三维模型的算法LVP相比,所提模型在真实三维模型与重建三维模型的交并比(IoU)上提高了20.1%,倒角距离(CD)缩小了13.2%。实验结果表明,所提模型在单视图三维模型重建中具有良好的泛化能力。

关键词: 深度图像, 深度估计, 三维重建, 对抗生成网络, 球面投影

CLC Number: