Face image inpainting method based on circular fields of feature parts

doi:10.11772/j.issn.1001-9081.2019071212

Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (3): 847-853.DOI: 10.11772/j.issn.1001-9081.2019071212

• Virtual reality and multimedia computing • Previous Articles Next Articles

Face image inpainting method based on circular fields of feature parts

WANG Xiao¹, WEI Jiawang¹, YUAN Yubo^1,2

1. School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, China;
2. Shanghai Engineering Research Center of Big Data and Internet Audience, Shanghai 200072, China

Received:2019-07-11 Revised:2019-08-21 Online:2019-09-11 Published:2020-03-10
Supported by:
This work is partially supported by the Shanghai Engineering Technology Center Project (18DZ2252300).

基于特征部位圆形域的人脸图像修复方法

王肖¹, 魏嘉旺¹, 袁玉波^1,2

1. 华东理工大学信息科学与工程学院, 上海 200237;
2. 上海大数据与互联网受众工程技术中心, 上海 200072

通讯作者: 王肖
作者简介:王肖(1995-),女,甘肃天水人,硕士研究生,主要研究方向:数据挖掘、机器学习;魏嘉旺(1995-),男,辽宁锦州人,硕士研究生,主要研究方向:数据挖掘、计算机视觉;袁玉波(1976-),男,云南宣威人,副教授,博士,主要研究方向:机器学习、数据科学、数据质量评估、数据挖掘。
基金资助:
上海市工程技术中心项目（18DZ2252300）

Abstract

Abstract: To solve the problem of unreasonable structure and low efficiency of the example block-based image inpainting method, a method for face image inpainting based on circular fields of feature parts was proposed. Firstly, according to the distribution of feature points obtained by feature points localization, the face image was segmented into four circular fields to determine feature search domains. Then, in priority model, the attenuation trend of confidence term was changed in form of exponential function, and with the combination of structural gradient term, the priority was constrained by using local gradient information to improve structural connectivity of inpainting result. In the stage of matching patch search, according to relative position between target patch and each circular domain of feature part, the search domain of matching patch was determined to improve search efficiency. Finally, under the standard of structural similarity, face image inpainting with structural connectivity was completed by choosing the best matching patch. Compared with four state-of-the-art inpainting methods, the proposed method has the Peak Signal-to-Noise Ratio (PSNR) of inpainted image increased by 1.219 to 2.663 dB on average, and the time consumption reduced by 34.7% to 69.6% on average. The experimental results show that the proposed method is effective in maintaining structural connectivity and visual rationality of face image, and has excellent performance in accuracy and time consumption of inpainting.

Key words: face image, image inpainting, feature parts, structural gradient term, structural similarity

摘要： 针对基于样本块的纹理合成方法存在的修复结构不合理和效率较低的问题，提出基于特征部位圆形域的人脸图像修复方法。首先进行人脸特征点定位，依据特征点分布将人脸图像分割出4个特征部位圆形域，明确特征搜索域范围。然后在优先级模型中以指数函数的形式改变置信度项的衰减趋势，并结合结构梯度项使用局部梯度信息约束优先级，提高修复结果的结构连通性。在匹配块搜索阶段，根据目标块与各个特征部位圆形域的相对位置，确定匹配块的搜索域，提升搜索效率。最终在结构相似性的标准下选择结构最佳匹配块，完成结构连通的人脸图像修复。与4个先进的方法相比较，所提方法修复图像的峰值信噪比（PSNR）平均提升了1.219~2.663 dB，时间消耗平均减小了34.7%~69.6%。实验结果表明，该方法对保持人脸图像的结构连通性和视觉合理性有显著效果，在修复的精度和时间上都表现优异。

关键词: 人脸图像, 图像修复, 特征部位, 结构梯度项, 结构相似性

CLC Number:

TP391.413

WANG Xiao, WEI Jiawang, YUAN Yubo. Face image inpainting method based on circular fields of feature parts[J]. Journal of Computer Applications, 2020, 40(3): 847-853.

王肖, 魏嘉旺, 袁玉波. 基于特征部位圆形域的人脸图像修复方法[J]. 计算机应用, 2020, 40(3): 847-853.

References

[1] 张东, 唐向宏, 张少鹏, 等. 小波变换与纹理合成相结合的图像修复[J]. 中国图象图形学报,2015,20(7):882-894.(ZHANG D,TANG X H,ZHANG S P,et al. Image inpainting based on combination of wavelet transform and texture synthesis[J]. Journal of Image and Graphics,2015,20(7):882-894.)
[2] GUO Q,GAO S,ZHANG X,et al. Patch-based image inpainting via two-stage low rank approximation[J]. IEEE Transactions on Visualization and Computer Graphics,2018,24(6):2023-2036.
[3] HOELTGEN L,MAINBERGER M,HOFFMANN S,et al. Optimising spatial and tonal data for PDE-based inpainting[J]. Mathematics,2017(18):35-83.
[4] KUMAR V,MUKHERJEE J,MANDAL S K D. Image inpainting through metric labeling via guided patch mixing[J]. IEEE Transactions on Image Processing,2016,25(11):5212-5226.
[5] HASEGAWA M,KAKO T,HIROBAYASHI S,et al. Image inpainting on the basis of spectral structure from 2-D nonharmonic analysis[J]. IEEE Transactions on Image Processing,2013,22(8):3008-3017.
[6] ZHANG L,WU X,BUADES A,et al. Color demosaicking by local directional interpolation and nonlocal adaptive thresholding[J]. Journal of Electronic Imaging,2011,20(2):No. 023016.
[7] CHAN T F,SHEN J. Variational image inpainting[J]. Communications on Pure and Applied Mathematics,2005,58(5):579-619.
[8] 孟红月, 翟东海, 李梦雪, 等. 参照四邻域裁剪样本的图像修复算法[J]. 计算机应用,2018,38(4):1111-1116.(MENG H Y, ZHAI D H,LI M X,et al. Image inpainting algorithm based on pruning samples referring to four-neighborhood[J]. Journal of Computer Applications,2018,38(4):1111-1116.)
[9] WANG J,LU K,PAN D,et al. Robust object removal with an exemplar-based image inpainting approach[J]. Neurocomputing, 2014,123:150-155.
[10] CRIMINISI A,PATRICK P,TOYANMA K. Object removal by exemplar-based inpainting[C]//Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2003:721-728.
[11] XU Z,SUN J. Image inpainting by patch propagation using patch sparsity[J]. IEEE Transactions on Image Processing,2010,19(5):1153-1165.
[12] KUI L,TAN J,SU B. Exemplar-based image inpainting using structure tensor[C]//Proceedings of the 2013 International Conference on Advanced Computer Science and Electronics Information. Paris:Atlantis Press,2013:18-24.
[13] DENG L,HUANG T,ZHAO X,et al. Exemplar-based image inpainting using a modified priority definition[J]. PLoS One,2015, 10(10):No. e0141199.
[14] 刘华明, 毕学慧, 叶中付, 等. 样本块搜索和优先权填充的弧形推进图像修复[J]. 中国图象图形学报,2018,21(8):993-1003.(LIU H M,BI X H,YE Z F,et al. Arc promoting image inpainting using exemplar searching and priority filling[J]. Journal of Image and Graphics,2018,21(8):993-1003.)
[15] ZHUANG Y,WANG Y,SHIH T K,et al. Patch-guided facial image inpainting by shape propagation[J]. Journal of Zhejiang University-SCIENCE A,2009,10(2):232-238.
[16] SULAM J,ELAD M. Large inpainting of face images with Trainlets[J]. IEEE Signal Processing Letters,2016,23(12):1839-1843.
[17] YU Y,CAO J,TANG Y,et al. Fast wavelet thresholding algorithms for face image inpainting[C]//Proceedings of the 2013 International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber,Physical and Social Computing. Piscataway:IEEE,2013:1445-1449.
[18] 王立, 张勇. 弱纹理人脸图像局部破损点修复方法[J]. 计算机仿真,2018,35(11):417-420. (WANG L,ZHANG Y. Weak texture face image local damage point repair method[J]. Computer Simulation,2018,35(11):417-420.)
[19] 魏嘉旺, 王肖, 袁玉波. 人脸特征点定位的自适应窗回归方法[J]. 计算机应用,2019,39(5):1459-1465. (WEI J W,WANG X,YUAN Y B. Adaptive window regression method for face feature point positioning[J]. Journal of Computer Applications, 2019,39(5):1459-1465.)
[20] CAO X,WEI Y,WEN F,et al. Face alignment by explicit shape regression[J]. International Journal of Computer Vision,2014, 107(2):177-190.

Face image inpainting method based on circular fields of feature parts

基于特征部位圆形域的人脸图像修复方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	Kaili DENG, Weibo WEI, Zhenkuan PAN. Industrial defect detection method with improved masked autoencoder [J]. Journal of Computer Applications, 2024, 44(8): 2595-2603.
[2]	Qiaoling HUANG, Bochuan ZHENG, Zicheng DING, Zedong WU. Improved image inpainting network incorporating supervised attention module and cross-stage feature fusion [J]. Journal of Computer Applications, 2024, 44(2): 572-579.
[3]	Gang CHEN, Yongwei LIAO, Zhenguo YANG, Wenying LIU. Image inpainting algorithm of multi-scale generative adversarial network based on multi-feature fusion [J]. Journal of Computer Applications, 2023, 43(2): 536-544.
[4]	Wanli SHEN, Yujin ZHANG, Wan HU. U-shaped feature pyramid network for image inpainting forensics [J]. Journal of Computer Applications, 2023, 43(2): 545-551.
[5]	Yang ZHAO, Bo LI. Single image dehazing based on conditional generative adversarial network with enhanced generator [J]. Journal of Computer Applications, 2021, 41(12): 3686-3691.
[6]	ZHU Lin, YU Haitao, LEI Xinyu, LIU Jing, WANG Ruofan. Brain network feature identification algorithm for Alzheimer's patients based on MRI image [J]. Journal of Computer Applications, 2020, 40(8): 2455-2459.
[7]	YUAN Linjun, JIANG Min, LUO Dunlang, JIANG Jiajun, GUO Jia. Portrait inpainting based on generative adversarial networks [J]. Journal of Computer Applications, 2020, 40(3): 842-846.
[8]	FENG Lang, ZHANG Ling, ZHANG Xiaolong. Image inpainting based on dilated convolution [J]. Journal of Computer Applications, 2020, 40(3): 825-831.
[9]	YANG Wenxia, WANG Meng, ZHANG Liang. Semantic face image inpainting based on U-Net with dense blocks [J]. Journal of Computer Applications, 2020, 40(12): 3651-3657.
[10]	CAO Yiqin, LIU Longbiao. Rail surface defect detection method based on background differential with defect proportion limitation [J]. Journal of Computer Applications, 2020, 40(10): 3066-3074.
[11]	ZHANG Bin, LUO Xiaoqing, ZHANG Zhancheng. Image fusion quality evaluation algorithm based on TV-L¹ structure and texture decomposition [J]. Journal of Computer Applications, 2019, 39(9): 2701-2706.
[12]	LI Qiuzhen, LUAN Chaoyang, WANG Shuangxi. Quality evaluation of face image based on convolutional neural network [J]. Journal of Computer Applications, 2019, 39(3): 695-699.
[13]	LI Wei, LI Weixiang, ZHANG Fan, JIE Wei. Rapid mismatching elimination algorithm based on motion smoothing constraints [J]. Journal of Computer Applications, 2018, 38(9): 2678-2682.
[14]	YANG Wenxia, ZHANG Liang. Image inpainting model based on structure-texture decomposition and local total variation minimization [J]. Journal of Computer Applications, 2018, 38(8): 2386-2392.
[15]	YANG Wenxia, ZHANG Liang. Logarithmic function based non-local total variation image inpainting model [J]. Journal of Computer Applications, 2018, 38(6): 1784-1789.