基于特征部位圆形域的人脸图像修复方法

doi:10.11772/j.issn.1001-9081.2019071212

计算机应用 ›› 2020, Vol. 40 ›› Issue (3): 847-853.DOI: 10.11772/j.issn.1001-9081.2019071212

• 虚拟现实与多媒体计算 • 上一篇下一篇

基于特征部位圆形域的人脸图像修复方法

王肖¹, 魏嘉旺¹, 袁玉波^1,2

1. 华东理工大学信息科学与工程学院, 上海 200237;
2. 上海大数据与互联网受众工程技术中心, 上海 200072

收稿日期:2019-07-11 修回日期:2019-08-21 出版日期:2020-03-10 发布日期:2019-09-11
通讯作者: 王肖
作者简介:王肖(1995-),女,甘肃天水人,硕士研究生,主要研究方向:数据挖掘、机器学习;魏嘉旺(1995-),男,辽宁锦州人,硕士研究生,主要研究方向:数据挖掘、计算机视觉;袁玉波(1976-),男,云南宣威人,副教授,博士,主要研究方向:机器学习、数据科学、数据质量评估、数据挖掘。
基金资助:
上海市工程技术中心项目（18DZ2252300）

Face image inpainting method based on circular fields of feature parts

WANG Xiao¹, WEI Jiawang¹, YUAN Yubo^1,2

1. School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, China;
2. Shanghai Engineering Research Center of Big Data and Internet Audience, Shanghai 200072, China

Received:2019-07-11 Revised:2019-08-21 Online:2020-03-10 Published:2019-09-11
Supported by:
This work is partially supported by the Shanghai Engineering Technology Center Project (18DZ2252300).

摘要/Abstract

摘要： 针对基于样本块的纹理合成方法存在的修复结构不合理和效率较低的问题，提出基于特征部位圆形域的人脸图像修复方法。首先进行人脸特征点定位，依据特征点分布将人脸图像分割出4个特征部位圆形域，明确特征搜索域范围。然后在优先级模型中以指数函数的形式改变置信度项的衰减趋势，并结合结构梯度项使用局部梯度信息约束优先级，提高修复结果的结构连通性。在匹配块搜索阶段，根据目标块与各个特征部位圆形域的相对位置，确定匹配块的搜索域，提升搜索效率。最终在结构相似性的标准下选择结构最佳匹配块，完成结构连通的人脸图像修复。与4个先进的方法相比较，所提方法修复图像的峰值信噪比（PSNR）平均提升了1.219~2.663 dB，时间消耗平均减小了34.7%~69.6%。实验结果表明，该方法对保持人脸图像的结构连通性和视觉合理性有显著效果，在修复的精度和时间上都表现优异。

关键词: 人脸图像, 图像修复, 特征部位, 结构梯度项, 结构相似性

Abstract: To solve the problem of unreasonable structure and low efficiency of the example block-based image inpainting method, a method for face image inpainting based on circular fields of feature parts was proposed. Firstly, according to the distribution of feature points obtained by feature points localization, the face image was segmented into four circular fields to determine feature search domains. Then, in priority model, the attenuation trend of confidence term was changed in form of exponential function, and with the combination of structural gradient term, the priority was constrained by using local gradient information to improve structural connectivity of inpainting result. In the stage of matching patch search, according to relative position between target patch and each circular domain of feature part, the search domain of matching patch was determined to improve search efficiency. Finally, under the standard of structural similarity, face image inpainting with structural connectivity was completed by choosing the best matching patch. Compared with four state-of-the-art inpainting methods, the proposed method has the Peak Signal-to-Noise Ratio (PSNR) of inpainted image increased by 1.219 to 2.663 dB on average, and the time consumption reduced by 34.7% to 69.6% on average. The experimental results show that the proposed method is effective in maintaining structural connectivity and visual rationality of face image, and has excellent performance in accuracy and time consumption of inpainting.

Key words: face image, image inpainting, feature parts, structural gradient term, structural similarity

中图分类号:

TP391.413

王肖, 魏嘉旺, 袁玉波. 基于特征部位圆形域的人脸图像修复方法[J]. 计算机应用, 2020, 40(3): 847-853.

WANG Xiao, WEI Jiawang, YUAN Yubo. Face image inpainting method based on circular fields of feature parts[J]. Journal of Computer Applications, 2020, 40(3): 847-853.

参考文献

[1] 张东, 唐向宏, 张少鹏, 等. 小波变换与纹理合成相结合的图像修复[J]. 中国图象图形学报,2015,20(7):882-894.(ZHANG D,TANG X H,ZHANG S P,et al. Image inpainting based on combination of wavelet transform and texture synthesis[J]. Journal of Image and Graphics,2015,20(7):882-894.)
[2] GUO Q,GAO S,ZHANG X,et al. Patch-based image inpainting via two-stage low rank approximation[J]. IEEE Transactions on Visualization and Computer Graphics,2018,24(6):2023-2036.
[3] HOELTGEN L,MAINBERGER M,HOFFMANN S,et al. Optimising spatial and tonal data for PDE-based inpainting[J]. Mathematics,2017(18):35-83.
[4] KUMAR V,MUKHERJEE J,MANDAL S K D. Image inpainting through metric labeling via guided patch mixing[J]. IEEE Transactions on Image Processing,2016,25(11):5212-5226.
[5] HASEGAWA M,KAKO T,HIROBAYASHI S,et al. Image inpainting on the basis of spectral structure from 2-D nonharmonic analysis[J]. IEEE Transactions on Image Processing,2013,22(8):3008-3017.
[6] ZHANG L,WU X,BUADES A,et al. Color demosaicking by local directional interpolation and nonlocal adaptive thresholding[J]. Journal of Electronic Imaging,2011,20(2):No. 023016.
[7] CHAN T F,SHEN J. Variational image inpainting[J]. Communications on Pure and Applied Mathematics,2005,58(5):579-619.
[8] 孟红月, 翟东海, 李梦雪, 等. 参照四邻域裁剪样本的图像修复算法[J]. 计算机应用,2018,38(4):1111-1116.(MENG H Y, ZHAI D H,LI M X,et al. Image inpainting algorithm based on pruning samples referring to four-neighborhood[J]. Journal of Computer Applications,2018,38(4):1111-1116.)
[9] WANG J,LU K,PAN D,et al. Robust object removal with an exemplar-based image inpainting approach[J]. Neurocomputing, 2014,123:150-155.
[10] CRIMINISI A,PATRICK P,TOYANMA K. Object removal by exemplar-based inpainting[C]//Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2003:721-728.
[11] XU Z,SUN J. Image inpainting by patch propagation using patch sparsity[J]. IEEE Transactions on Image Processing,2010,19(5):1153-1165.
[12] KUI L,TAN J,SU B. Exemplar-based image inpainting using structure tensor[C]//Proceedings of the 2013 International Conference on Advanced Computer Science and Electronics Information. Paris:Atlantis Press,2013:18-24.
[13] DENG L,HUANG T,ZHAO X,et al. Exemplar-based image inpainting using a modified priority definition[J]. PLoS One,2015, 10(10):No. e0141199.
[14] 刘华明, 毕学慧, 叶中付, 等. 样本块搜索和优先权填充的弧形推进图像修复[J]. 中国图象图形学报,2018,21(8):993-1003.(LIU H M,BI X H,YE Z F,et al. Arc promoting image inpainting using exemplar searching and priority filling[J]. Journal of Image and Graphics,2018,21(8):993-1003.)
[15] ZHUANG Y,WANG Y,SHIH T K,et al. Patch-guided facial image inpainting by shape propagation[J]. Journal of Zhejiang University-SCIENCE A,2009,10(2):232-238.
[16] SULAM J,ELAD M. Large inpainting of face images with Trainlets[J]. IEEE Signal Processing Letters,2016,23(12):1839-1843.
[17] YU Y,CAO J,TANG Y,et al. Fast wavelet thresholding algorithms for face image inpainting[C]//Proceedings of the 2013 International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber,Physical and Social Computing. Piscataway:IEEE,2013:1445-1449.
[18] 王立, 张勇. 弱纹理人脸图像局部破损点修复方法[J]. 计算机仿真,2018,35(11):417-420. (WANG L,ZHANG Y. Weak texture face image local damage point repair method[J]. Computer Simulation,2018,35(11):417-420.)
[19] 魏嘉旺, 王肖, 袁玉波. 人脸特征点定位的自适应窗回归方法[J]. 计算机应用,2019,39(5):1459-1465. (WEI J W,WANG X,YUAN Y B. Adaptive window regression method for face feature point positioning[J]. Journal of Computer Applications, 2019,39(5):1459-1465.)
[20] CAO X,WEI Y,WEN F,et al. Face alignment by explicit shape regression[J]. International Journal of Computer Vision,2014, 107(2):177-190.

基于特征部位圆形域的人脸图像修复方法

Face image inpainting method based on circular fields of feature parts

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	朱琳, 于海涛, 雷新宇, 刘静, 王若凡. 基于MRI图像的阿尔茨海默症患者脑网络特征识别算法[J]. 计算机应用, 2020, 40(8): 2455-2459.
[2]	冯浪, 张玲, 张晓龙. 基于扩张卷积的图像修复[J]. 计算机应用, 2020, 40(3): 825-831.
[3]	袁琳君, 蒋旻, 罗敦浪, 江佳俊, 郭嘉. 基于生成对抗网络的人像修复[J]. 计算机应用, 2020, 40(3): 842-846.
[4]	杨文霞, 王萌, 张亮. 基于密集连接块U-Net的语义人脸图像修复[J]. 计算机应用, 2020, 40(12): 3651-3657.
[5]	李秋珍, 栾朝阳, 汪双喜. 基于卷积神经网络的人脸图像质量评价[J]. 计算机应用, 2019, 39(3): 695-699.
[6]	杨文霞, 张亮. 基于图像结构纹理分解及局部总变分最小化的图像修复模型[J]. 计算机应用, 2018, 38(8): 2386-2392.
[7]	杨文霞, 张亮. 基于对数函数的非局部总变分图像修复模型[J]. 计算机应用, 2018, 38(6): 1784-1789.
[8]	孟红月, 翟东海, 李梦雪, 曹大命. 参照四邻域裁剪样本的图像修复算法[J]. 计算机应用, 2018, 38(4): 1111-1116.
[9]	曹大命, 翟东海, 孟红月, 李梦雪, 冯炎. 基于先验约束和统计的图像修复算法[J]. 计算机应用, 2018, 38(2): 533-538.
[10]	张欢欢, 洪敏, 袁玉波. 基于极端学习机的人脸特征深度稀疏自编码方法[J]. 计算机应用, 2018, 38(11): 3193-3198.
[11]	杨明, 黄颖. 结合代数多重网格的钻石编码隐写算法[J]. 计算机应用, 2017, 37(6): 1609-1615.
[12]	于金明, 孟军, 吴秋峰. 基于改进相似性度量的项目协同过滤推荐算法[J]. 计算机应用, 2017, 37(5): 1387-1391.
[13]	林金勇, 邓德祥, 颜佳, 林晓英. 基于自适应相似组稀疏表示的图像修复算法[J]. 计算机应用, 2017, 37(4): 1169-1173.
[14]	邹玮刚, 周志辉, 王洋. 基于非降采样轮廓波变换的图像修复算法[J]. 计算机应用, 2017, 37(2): 553-558.
[15]	李梦雪, 翟东海, 孟红月, 曹大命. 划分特征子区域的图像修复算法[J]. 计算机应用, 2017, 37(12): 3541-3546.