Image inpainting algorithm based on priori constraints and statistics

doi:10.11772/j.issn.1001-9081.2017071898

Journal of Computer Applications ›› 2018, Vol. 38 ›› Issue (2): 533-538.DOI: 10.11772/j.issn.1001-9081.2017071898

Previous Articles Next Articles

Image inpainting algorithm based on priori constraints and statistics

CAO Daming¹, ZHAI Donghai^1,2, MENG Hongyue¹, LI Mengxue¹, FENG Yan²

1. School of Information Science and Technology, Southwest Jiaotong University, Chengdu Sichuan 610031, China;
2. School of Engineering, Tibet University, Lhasa Tibet 850000, China

Received:2017-08-04 Revised:2017-09-21 Online:2018-02-10 Published:2018-02-10
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61461048, 61661047), the Young Teachers Innovative Support Program of Tibet (QCZ2016-02).

基于先验约束和统计的图像修复算法

曹大命¹, 翟东海^1,2, 孟红月¹, 李梦雪¹, 冯炎²

1. 西南交通大学信息科学与技术学院, 成都 610031;
2. 西藏大学工学院, 拉萨 850000

通讯作者: 曹大命
作者简介:曹大命(1990-),男,河北沧州人,硕士研究生,主要研究方向:数字图像处理;翟东海(1974-),男,山西芮城人,副教授,博士研究生,主要研究方向:数字图像处理、海量数据挖掘;孟红月(1993-),女,河南永城人,硕士研究生,主要研究方向:数字图像处理;李梦雪(1992-),女,重庆潼南人,硕士研究生,主要研究方向:数字图像处理;冯炎(1979-),男,山西吕梁人,副教授,硕士,主要研究方向:信号与信息处理、数字图像处理。
基金资助:
国家自然科学基金资助项目（61461048，61661047）；西藏自治区高校青年教师创新支持计划项目（QCZ2016-02）。

Abstract

Abstract: When inpainting the image of large damaged region with complex geometric structure and rich texture, the PatchMatch-based image inpainting algorithm has disadvantages like texture extension and some incorrect sample patches being selected as candidate patches. To solve these problems, a new image inpainting algorithm was proposed for improving accuracy and efficiency. In terms of exact matching of sample patches, an image was preprocessed to obtain priori information of the image, which was used to initialize the constraint of the offset map, while PathMatch algorithm used global random initialization. In the process of pixel patch matching, to improve the matching accuracy of the sample, mean method and angle method were introduced to compute the similarity of different categories of pixel patches. In terms of efficiency, according to the statistical characteristics of similar patches of an image, histogram statistical method was introduced to reduce the labels for inpainting. The proposed algorithm was verified by some instances. The simulation results show that compared with the original PatchMatch algorithm, the Peak Signal-to-Noise Ratio (PSNR) of the proposed algorithm was improved by 0.5dB to 1dB, and the running time was reduced by 5s to 10s, which indicates that the proposed algorithm can effectively improve the accuracy and efficiency of image inpainting.

Key words: image inpainting, statistical characteristic, constraint initialization, angle method, mean method

摘要： 利用基于块匹配（PatchMatch）图像修复算法对破损区域较大且周围既含有几何结构信息又含有丰富纹理信息的图片进行修复时，容易出现纹理延伸现象以及样本块误匹配问题。针对此类问题，在样本块的精确匹配和算法的时效性两个方面进行改进，提出新的图像修复算法。在样本块精确匹配方面，改进算法对图像进行预处理以获得图像的先验信息，并利用先验信息约束算法偏移映射图的初始化，从而转变PatchMatch算法中对图像偏移映射图的全局随机初始化为在先验信息指导下的约束初始化；在像素块匹配过程中，利用均值法和夹角法来判断不同类别像素块的相似度，从而提高样本块的匹配精度。在算法的时效性方面，根据图像相似块的统计性特性，引入直方图统计的方法来减少最终用于修复的样本标签，提高改进算法的时效性。最后，将改进算法用于实例验证，相比原算法，改进算法的运行时间减少了5~10 s，峰值信噪比（PSNR）提高了0.5~1 dB。实例验证结果表明改进算法不但可以有效地提高图像修复的精度，而且提高了图像修复的效率。

关键词: 图像修复, 统计特性, 约束初始化, 夹角法, 均值法

CLC Number:

TN911.73

CAO Daming, ZHAI Donghai, MENG Hongyue, LI Mengxue, FENG Yan. Image inpainting algorithm based on priori constraints and statistics[J]. Journal of Computer Applications, 2018, 38(2): 533-538.

曹大命, 翟东海, 孟红月, 李梦雪, 冯炎. 基于先验约束和统计的图像修复算法[J]. 计算机应用, 2018, 38(2): 533-538.

References

[1] GUILLEMOT C, LE MEUR O. Image inpainting:overview and recent advances[J]. IEEE Signal Processing Magazine, 2013, 31(1):127-144.
[2] 王猛,翟东海,聂洪玉,等.邻域窗口权重变分的图像修复[J].中国图象图形学报,2015,20(8):1000-1007. (WANG M, ZHAI D H, NIE H Y, et al. Image inpainting with weight variation of neighborhood window[J]. Journal of Image and Graphics, 2015, 20(8):1000-1007.)
[3] BERTALMIO M, SAPIRO G, CASELLES V, et al. Image inpainting[C]//SIGGRAPH'00:Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques. New York:ACM, 2000:417-424.
[4] CHAN T F, SHEN J. Mathematical models for local nontexture inpaintings[J]. SIAM Journal on Applied Mathematics, 2002, 62(3):1019-1043.
[5] ELAD M, STARCK J-L, QUERRE P, et al. Simultaneous cartoon and texture image inpainting using Morphological Component Analysis (MCA)[J]. Applied and Computational Harmonic Analysis, 2005, 19(3):340-358.
[6] YU G S, SAPIRO G, MALLAT S. Solving inverse problems with piecewise linear estimators:from Gaussian mixture models to structured sparsity[J]. IEEE Transactions on Image Processing, 2012, 21(5):2481-2499.
[7] HE K, SUN J. Image completion approaches using the statistics of similar patches[J]. IEEE Transactions on Pattern Analysis And Machine Intellignce, 2014, 36(12):2423-2435.
[8] EFROSA A, LEUNG T K. Texture synthesis by non-parametric sampling[C]//ICCV'99:Proceedings of the Seventh IEEE International Conference on Computer Vision. Washington, DC:IEEE Computer Society, 1999, 2:1033-1038.
[9] CRIMINISI A, PEREZ P, TOYAMA K, et al. Region filling and object removal by exemplar-based image inpainting[J]. IEEE Transactions on Image Processing, 2004, 13(9):1200-1212.
[10] WEXLER Y, SHECHTMAN E, IRANI M. Space-time video completion[C]//CVPR'04:Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2004:120-127.
[11] KOMODAKIS N. Image completion using global optimization[C]//CVPR'06:Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2006:442-452.
[12] PRITCH Y, KAV-VENAKI E, PELEG S. Shift-map image editing[C]//ICCV'09:Proceedings of the IEEE 12th International Conference on Computer Vision. Piscataway, NJ:IEEE, 2009:151-158.
[13] BARNES C. PatchMatch:a fast randomized matching algorithm with application to image and video[D]. Princeton, NJ:Princeton University, Department of Computer Science, 2011:103-110.
[14] LI X, QIONG Y, YANG X, et al. Structure extraction from texture via relative total variation[J]. ACM Transactions on Graphics, 2012, 31(6):Article No. 139.
[15] ROTHER C, KOLMOGOROV V, ANDREW BLAKE. "GrabCut":interactive foreground extraction using iterated graph cuts[J]. ACM Transactions on Graphics (TOG)-Proceedings of ACM SIGGRAPH 2004, 2004, 23(3):309-314.
[16] KORMAN S, AVIDAN S. Coherency sensitive hashing[C]//ICCV'11:Proceedings of the IEEE 2011 International Conference on Computer Vision. Washington, DC:IEEE Computer Society, 2011:1607-1614.
[17] ZHANG J, ZHAO D, XIONG R, et al. Image restoration using joint statistical modeling in space-transform domain[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(6):915-928.
[18] 王佳君,俞强,张晶晶,等.基于先验置信传播的图像修复改进算法[J].计算机应用,2016,36(4):1115-1119. (WANG J J, YU Q, ZHANG J J, et al. Improvement algorithm of image inpainting based on priority-belief propagation[J]. Journal of Computer Applications, 2016, 36(4):1115-1119.)
[19] SIMAKOV D, CASPI Y, SHECHTMAN E, et al. Summarizing visual data using bidirectional similarity[C]//CVPR'08:Proceedings of the 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2008:1-8.

Image inpainting algorithm based on priori constraints and statistics

基于先验约束和统计的图像修复算法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	FENG Lang, ZHANG Ling, ZHANG Xiaolong. Image inpainting based on dilated convolution [J]. Journal of Computer Applications, 2020, 40(3): 825-831.
[2]	YUAN Linjun, JIANG Min, LUO Dunlang, JIANG Jiajun, GUO Jia. Portrait inpainting based on generative adversarial networks [J]. Journal of Computer Applications, 2020, 40(3): 842-846.
[3]	WANG Xiao, WEI Jiawang, YUAN Yubo. Face image inpainting method based on circular fields of feature parts [J]. Journal of Computer Applications, 2020, 40(3): 847-853.
[4]	YANG Wenxia, WANG Meng, ZHANG Liang. Semantic face image inpainting based on U-Net with dense blocks [J]. Journal of Computer Applications, 2020, 40(12): 3651-3657.
[5]	KONG Jing, GUO Yuanbo, LIU Chunhui, WANG Yifeng. Gait feature identification method based on motion sensor in smartphone [J]. Journal of Computer Applications, 2019, 39(6): 1747-1752.
[6]	YANG Wenxia, ZHANG Liang. Image inpainting model based on structure-texture decomposition and local total variation minimization [J]. Journal of Computer Applications, 2018, 38(8): 2386-2392.
[7]	YANG Wenxia, ZHANG Liang. Logarithmic function based non-local total variation image inpainting model [J]. Journal of Computer Applications, 2018, 38(6): 1784-1789.
[8]	MENG Hongyue, ZHAI Donghai, LI Mengxue, CAO Daming. Image inpainting algorithm based on pruning samples referring to four-neighborhood [J]. Journal of Computer Applications, 2018, 38(4): 1111-1116.
[9]	LIN Jinyong, DENG Dexiang, YAN Jia, LIN Xiaoying. Self-adaptive group based sparse representation for image inpainting [J]. Journal of Computer Applications, 2017, 37(4): 1169-1173.
[10]	ZOU Weigang, ZHOU Zhihui, WANG Yang. Image inpainting algorithm based on non-subsampled contourlet transform [J]. Journal of Computer Applications, 2017, 37(2): 553-558.
[11]	LI Mengxue, ZHAI Donghai, MENG Hongyue, CAO Daming. Image inpainting algorithm for partitioning feature subregions [J]. Journal of Computer Applications, 2017, 37(12): 3541-3546.
[12]	WANG Jiajun, YU Qiang, ZHANG Jingjing. Improvement algorithm of image inpainting based on priority-belief propagation [J]. Journal of Computer Applications, 2016, 36(4): 1115-1119.
[13]	ZHAI Donghai ZUO Wenjie DUAN Weixia YU Jiang LI Tongliang. Image inpainting algorithm based on double-cross curvature-driven diffusion model [J]. Journal of Computer Applications, 2013, 33(12): 3536-3539.
[14]	HAN Zhike WANG Gui. Partial differential equation model of fingerprint image inpaiting based on orientation field [J]. Journal of Computer Applications, 2013, 33(10): 2886-2890.
[15]	ZHAI Donghai XIAO Jie YU Jiang LI Tongliang. Image inpainting algorithm based on adaptive template [J]. Journal of Computer Applications, 2013, 33(10): 2891-2894.