Image completion algorithm based on depth information

doi:10.11772/j.issn.1001-9081.2015.10.2955

Journal of Computer Applications ›› 2015, Vol. 35 ›› Issue (10): 2955-2958.DOI: 10.11772/j.issn.1001-9081.2015.10.2955

Previous Articles Next Articles

Image completion algorithm based on depth information

HE Ye, LI Guangyao, XIAO Mang, XIE Li, PENG Lei, TANG Ke

College of Electronics and Information Engineering, Tongji University, Shanghai 201804, China

Received:2015-05-13 Revised:2015-07-06 Online:2015-10-10 Published:2015-10-14

基于深度信息的图像修复算法

何埜, 李光耀, 肖莽, 谢力, 彭磊, 唐可

同济大学电子与信息工程学院, 上海 201804

通讯作者: 李光耀(1965-),男,安徽安庆人,教授,博士,主要研究方向:图形图像学、虚拟现实,saiheye2009@163.com
作者简介:何埜(1992-),男,安徽安庆人,硕士研究生,主要研究方向:数字图像处理、机器学习;肖莽(1982-),男,江西吉安人,博士研究生,主要研究方向:数字图像处理、机器学习;谢力(1992-),男,浙江台州人,博士研究生,主要研究方向:数字图像处理、机器学习;彭磊(1977-),男,山东泰安人,博士研究生,主要研究方向:数字图像处理;唐可(1991-),男,重庆人,硕士研究生,主要研究方向:数字图像处理。
基金资助:
国家自然科学基金资助项目(60771065,51378365)。

Abstract

Abstract: Aiming at the problem of object structure discontinuity and incompleteness occurred in image completion, an image completion algorithm based on depth information was proposed. Firstly, the plane parameter Markov random field model was established to speculate depth information of the pixels in the image where the scene situate, then the coplanar region in the image determined, and the target matching blocks were located. Secondly, according to the principle of perspective projection, the transformation matrix was derived, which guided the geometric transformation of the matching blocks. Finally, the target cost function which includes the depth term was designed. Experimental results show the proposed algorithm has superiority in both subjective details and Peak Signal-to-Noise Ratio (PNSR) statistics.

Key words: image completion, depth information, perspective projection, global optimization

摘要： 针对图像修复结果中存在物体结构上的不连续性和不完整性的问题,提出一种基于深度信息的图像修复算法。首先,通过建立平面参数马尔可夫模型来推测图像中像素点所在场景空间中的深度信息,从而划分出图像中的共面区域,定位匹配块的目标位置;其次,根据透视投影原理推导出相应的变换矩阵,指导对匹配块的几何变换;最后,设计含义深度项的目标代价函数,通过全局优化选择最优修复效果。实验的主观效果对比和峰值信噪比(PSNR)数据都表明该方法更为优越。

关键词: 图像修复, 深度信息, 透视投影, 全局优化

CLC Number:

TP391.41

HE Ye, LI Guangyao, XIAO Mang, XIE Li, PENG Lei, TANG Ke. Image completion algorithm based on depth information[J]. Journal of Computer Applications, 2015, 35(10): 2955-2958.

何埜, 李光耀, 肖莽, 谢力, 彭磊, 唐可. 基于深度信息的图像修复算法[J]. 计算机应用, 2015, 35(10): 2955-2958.

References

[1] CRIMINISI A, PEREZ P, TOYAMA K. Region filling and object removal by exemplar-based image inpainting[J]. IEEE Transactions on Image Processing, 2004, 13(9): 1200-1212.
[2] LUO K, LI D, FENG Y, et al. Depth-aided inpainting for disocclusion restoration of multi-view images using depth-image-based rendering[J]. Journal of Zhejiang University SCIENCE A, 2009, 10(12): 1738-1749.
[3] MANSFIELD A, PRASAD M, ROTHER C, et al. Transforming image completion[EB/OL]. [2014-10-10]. http://www.msr-waypoint.net/en-us/um/people/pkohli/papers/mprskg_bmvc2011.pdf.
[4] DARABI S, SHECHTMAN E, BARNES C, et al. Image melding: combining inconsistent images using patch-based synthesis[EB/OL]. [2014-10-10]. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.364.5147&rep=rep1&type=pdf.
[5] HUANG J B, KANG S B, AHUJA N, et al. Image completion using planar structure guidance[J]. ACM Transactions on Graphics, 2014, 33(4): 129.
[6] SAXENA A, SUN M, NG A Y. Learning 3-D scene structure from a single still image[C]//ICCV 2007: Proceedings of the 11th IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2007:1-8.
[7] GEIGER A, ZIEGLER J, STILLER C. StereoScan: dense 3D reconstruction in real-time[C]//Proceedings of the 2011 IEEE Intelligent Vehicles Symposium. Piscataway: IEEE Press,2011: 963-968.
[8] HILL F, KELLEY S. Computer graphics using OpenGL, 3/E[M]. Upper Saddle River: Prentice Hall, 2007.
[9] LEI J, LI S, ZHU C, et al. Depth coding based on depth-texture motion and structure similarities[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2015, 25(2): 275-286.
[10] LIPSKI C, KLOSE F, MAGNOR M. Correspondence and depth-image based rendering a hybrid approach for free-viewpoint video[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(6): 942-951.
[11] COMANICIU D, MEER P. Mean shift: a robust approach toward feature space analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(5): 603-619.
[12] HE K, SUN J. Statistics of patch offsets for image completion[C]//Proceedings of ECCV 2012. Berlin: Springer-Verlag, 2012: 16-29.

[1]	XIE Benming, HAN Mingming, ZHANG Pan, ZHANG Wei. Optimization algorithm of dynamic time warping for speech recognition of aircraft towing vehicle [J]. Journal of Computer Applications, 2018, 38(6): 1771-1776.
[2]	YUAN Quan, ZHANG Jianfeng, WU Lizhi. RGB-D saliency detection based on improved local background enclosure feature [J]. Journal of Computer Applications, 2018, 38(5): 1432-1435.
[3]	WANG Xiaojing, PENG Hu, DENG Changshou, HUANG Haiyan, ZHANG Yan, TAN Xujie. Firefly algorithm based on uniform local search and variable step size [J]. Journal of Computer Applications, 2018, 38(3): 715-721.
[4]	LI Wei, LYU Naiguang, DONG Mingli, LOU Xiaoping. Robot hand-eye calibration by convex relaxation global optimization [J]. Journal of Computer Applications, 2017, 37(5): 1451-1455.
[5]	LIN Haibo, WANG Shengbin, ZHANG Yi. Gesture segmentation and positioning based on improved depth information [J]. Journal of Computer Applications, 2017, 37(1): 251-254.
[6]	XU Dayu, LIU Renping. Improved self-organized criticality optimized gray wolf optimizer metaheuristic algorithm [J]. Journal of Computer Applications, 2016, 36(6): 1588-1593.
[7]	YANG Binhua, ZHAO Gaopeng, LIU Lujiang, BO Yuming. Global point cloud registration algorithm based on translation domain estimating [J]. Journal of Computer Applications, 2016, 36(6): 1664-1667.
[8]	QIU Hui, WANG Kun, YANG Haopu. Network alerts depth information fusion method based on time confrontation [J]. Journal of Computer Applications, 2016, 36(2): 499-504.
[9]	ZHANG Heng, LIU Dayong, LIU Yanli, NIE Chenxi. Improved binary robust invariant scalable keypoints algorithm fusing depth information [J]. Journal of Computer Applications, 2015, 35(8): 2285-2290.
[10]	HUANG Hao, TANG Hao, ZHOU Lei, CHENG Wenjuan. Robust optimal control of single conveyor-serviced production station with uncertain service rate [J]. Journal of Computer Applications, 2015, 35(7): 2067-2072.
[11]	LIU Jie, WANG Ling, WANG Shan, FENG Wei, LI Wen. Minimum MPR set selection algorithm based on OLSR protocol [J]. Journal of Computer Applications, 2015, 35(2): 305-308.
[12]	XU Xiao WANG Run PENG Guojie YANG Qi WANG Yiwen LI Hui. Implementation of calibration for machine vision electronic whiteboard [J]. Journal of Computer Applications, 2014, 34(1): 139-141.
[13]	WANG Lei CHEN Jindong PAN Feng. Applications of gravitational search algorithm in parameters estimation of penicillin fermentation process model [J]. Journal of Computer Applications, 2013, 33(11): 3296-3299.
[14]	LI Guozhu. Real-coded quantum evolutionary algorithm based on cloud model [J]. Journal of Computer Applications, 2013, 33(09): 2550-2552.
[15]	PENG Shuo OUYANG Aijia YUE Guangxue HE Minghua ZHOU Xu. Improved glowworm swarm optimization algorithm for high-dimensional functions [J]. Journal of Computer Applications, 2013, 33(08): 2253-2256.

Image completion algorithm based on depth information

基于深度信息的图像修复算法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics