No-reference image quality assessment method for facial beautification image

doi:10.11772/j.issn.1001-9081.2019091552

Abstract

Abstract: In view of the fact that facial beautification has been widely studied,but the lack of effective beautification image quality evaluation methods limits the further development of beautification technology,a no-reference evaluation method for facial beautification image quality was proposed. In this method,the facial cognition and perception were combined with the facial beautification technologies to unearth the quality representation of beautified images. Firstly,a facial beautification image database was constructed,the facial image was decomposed to three areas:skin,eyes and mouth. Then,facial aesthetic features were extracted from five aspects:skin color,smoothness,illumination,grayscale difference and sharpness. Finally,Support Vector Regression(SVR)was used to train the facial beautification quality model and predict the quality of the beautified image. The experimental results show that the proposed method achieves 0. 920 5 and 0. 900 9 respectively in the Pearson linear correlation coefficient and Spearman RankOrder Correlation Coefficient(SROCC) on the proposed database,which are higher than those of image quality evaluation methods BIQI(Blind Image Quality Indices),and NIQE(Natural Image Quality Evaluation).

Key words: computer vision, image quality assessment, no reference, feature extraction, machine learning

摘要： 针对目前面部美化已被广泛研究，然而缺乏有效美化图像质量评价方法限制美化技术进一步发展的问题，提出一种面部美化图像质量的无参考评价方法。该方法结合面部美感的认知与感知和面部美化技术以挖掘美化图像的质量表示，首先构建面部美化图像数据库，将面部图像分解为皮肤、眼睛和嘴巴三种区域，然后从肤色、光滑度、光照、灰度差和清晰度等五个方面提取面部美学特征，最后用支持向量回归（SVR）训练面部美化质量模型并预测美化图像的质量。实验结果表明，所提方法在构建的数据库上Pearson线性相关系数和Spearman等级相关系数分别达到0.920 5和0.900 9，优于BIQI（Blind Image Quality Indices）、NIQE（Natural Image Quality Evaluation）图像质量评价方法。

关键词: 计算机视觉, 图像质量评价, 无参考, 特征提取, 机器学习

CLC Number:

TP181

ZHANG Junsheng, XU Jingjing, YU Wei. No-reference image quality assessment method for facial beautification image[J]. Journal of Computer Applications, 2020, 40(4): 1184-1190.

张俊升, 徐晶晶, 余伟. 面部美化图像质量无参考评价方法[J]. 计算机应用, 2020, 40(4): 1184-1190.

References

[1] LEYVAND T, COHEN-OR D, DROR G, et al. Digital face beautification[C]//Proceedings of the 2006 ACM SIGGRAPH Sketches. New York:ACM,2006:169.
[2] CHEN C W,HUANG D Y,FUH C S. Automatic skin color beautification[C]//Proceedings of the 2009 International Conference on Arts and Technology,LNICST 30. Berlin:Springer, 2009:157-164.
[3] SAKURAI M, MAKINO H, GOTO T, et al. Digital face beautification utilizing TV filter and super-resolution technology[C]//Proceedings of the IEEE 3rd Global Conference on Consumer Electronics. Piscataway:IEEE,2015:313-314.
[4] CHUNG K Y. Effect of facial makeup style recommendation on visual sensibility[J]. Multimedia Tools and Applications,2014,71(2):843-853.
[5] CHEN F,ZHANG D. A benchmark for geometric facial beauty study[C]//Proceedings of the 2010 International Conference on Medical Biometrics,LNCS 6165. Berlin:Springer,2010:21-32.
[6] ZHANG D,ZHAO Q,CHEN F. Quantitative analysis of human facial beauty using geometric features[J]. Pattern Recognition, 2011,44(4):940-950.
[7] 毛慧芸, 金连文, 杜明辉. 基于几何特征及C4.5的人脸美丽分类方法[J]. 模式识别与人工智能,2010,23(6):809-814. (MAO H Y,JIN L W,DU M H. Facial beauty classification on geometric features and C4.5[J]. Pattern Recognition and Artificial Intelligence,2010,23(6):809-814.)
[8] KAGIAN A,DROR G,LEYVAND T,et al. A machine learning predictor of facial attractiveness revealing human-like psychophysical biases[J]. Vision Research, 2008, 48(2):235-243.
[9] GUO D,SIM T. Digital face makeup by example[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2009:73-79.
[10] JOSHI N,MATUSIK W,ADELSON E H,et al. Personal photo enhancement using example images[J]. ACM Transactions on Graphics,2010,29(2):No. 12.
[11] KANG L,YE P,LI Y,et al. Convolutional neural networks for no-reference image quality assessment[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2014:1733-1740.
[12] KIM J,NGUYEN A D,LEE S. Deep CNN-based blind image quality predictor[J]. IEEE Transactions on Neural Networks and Learning Systems,2019,30(1):11-24.
[13] LIANG L,LIN L,JIN L,et al. SCUT-FBP5500:a diverse benchmark dataset for multi-paradigm facial beauty prediction[C]//Proceedings of the 24th International Conference on Pattern Recognition. Piscataway:IEEE,2018:1598-1603.
[14] LI L,ZHOU Y,LIN W,et al. No-reference quality assessment of deblocked images[J]. Neurocomputing,2016,177:572-584.
[15] COOTES T F,TAYLARS C J,COOPER D H,et al. Active shape models-their training and application[J]. Computer Vision and Image Understanding,1995,61(1):38-59.
[16] COOTES T F, EDWARDS G J, TAYLARS C J. Active appearance models[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2001,23(6):681-685.
[17] AARABI P,HUGHES D,MOHAJER K,et al. The automatic measurement of facial beauty[C]//Proceedings of the 2001 IEEE International Conference on Systems, Man and Cybernetics. Piscataway:IEEE,2001:2644-2647.
[18] KAGIAN A, DROR G, LEYVAND T, et al. A humanlike predictor of facial attractiveness[C]//Proceedings of the 19th International Conference on Neural Information Processing Systems. Cambridge:MIT Press,2006:649-656.
[19] SCHERBAUM K,RITSCHEL T,HULLIN M,et al. Computer-suggested facial makeup[J]. Computer Graphics Forum,2011,30(2):485-492.
[20] STEPHEN I D,SCOTT I M L,COETZEE V,et al. Cross-cultural effects of color,but not morphological masculinity,on perceived attractiveness of men's faces[J]. Evolution and Human Behavior, 2012,33(4):260-267.
[21] COETZEE V,FAERBER S J,GREEFF J M,et al. African perceptions of female attractiveness[J]. PLoS ONE,2012,7(10):(No. e48116.)
[22] STEPHEN I D,SMITH M J L,STIRRAT M R,et al. Facial skin coloration affects perceived health of human faces[J]. International Journal of Primatology,2009,30(6):845-857.
[23] GUO G,WEN L,YAN S. Face authentication with makeup changes[J]. IEEE Transactions on Circuits and Systems for Video Technology,2014,24(5):814-825.
[24] FARBAMN Z, FATTAL R, LISCHINSKI D, et al. Edge-preserving decompositions for multi-scale tone and detail manipulation[J]. ACM Transactions on Graphics,2008,27(3):1-10.
[25] RUSSELL R. Sex,beauty,and the relative luminance of facial features[J]. Perception,2003,32(9):1093-1107,
[26] TOMASI C,MANDUCHI R. Bilateral filtering for gray and color images[C]//Proceedings of the 6th International Conference on Computer Vision. Piscataway:IEEE,1998:839-846.
[27] HE K, SUN J, TANG X. Guided image filtering[C]//Proceedings of the 2010 European Conference on Computer Vision,LNCS 6311. Berlin:Springe,2010:1-14
[28] FINK B, GRAMMER K, THORNHILL R. Human (Homo sapiens)facial attractiveness in relation to skin texture and color[J]. Journal of Comparative Psychology,2001,115(1):92-99.
[29] FINK B, GRAMMER K, MATTS P J. Visible skin color distribution plays a role in the perception of age,attractiveness, and health in female faces[J]. Evolution and Human Behavior, 2006,27(6):433-442.
[30] ZHANG L,GU Z,LI H. SDSP:a novel saliency detection method by combining simple priors[C]//Proceedings of the 2013 IEEE International Conference on Image Processing. Piscataway:IEEE, 2013:171-175.
[31] KULLBACK S,LEIBLER R A. On information and sufficiency[J]. The Annals of Mathematical Statistics,1951,22(1):79-86.
[32] NARVEKAR N D,KARAM L J. A no-reference image blur metric based on the Cumulative Probability of Blur Detection (CPBD)[J]. IEEE Transactions on Image Processing,2011,20(9):2678-2683.
[33] CHANG C C,LIN C J. LIBSVM:a library for support vector machines[J]. ACM Transactions on Intelligent Systems and Technology,2011,2(3):No. 27.
[34] MOORTHY A K, BOVIK A C. A two-step framework for constructing blind image quality indices[J]. IEEE Signal Processing Letters,2010,17(5):513-516.
[35] MITTAL A,MOORTHY A K,BOVIK A C. No-reference image quality assessment in the spatial domain[J]. IEEE Transactions on Image Processing,2012,21(12):4695-4708.
[36] WANG Y, LI N, LI Z. An imaging-inspired no-reference underwater color image quality assessment metric[J]. Computers and Electrical Engineering,2018,70:904-913.
[37] MOORTHY A K,BOVIK A C. Blind image quality assessment:from natural scene statistics to perceptual quality[J]. IEEE Transactions on Image Processing,2011,20(12):3350-3364.
[38] CHOI L K,YOU J,BOVIK A C. Referenceless prediction of perceptual fog density and perceptual image defogging[J]. IEEE Transactions on Image Processing,2015,24(11):3888-3901.
[39] MITTAL A,SOUNDARARAJAN R,BOVIK A C. Making a "completely blind" image quality analyzer[J]. IEEE Signal Processing Letters,2013,20(3):209-212.
[40] LIU L,LIU B,HUANG H,et al. No-reference image quality assessment based on spatial and spectral entropies[J]. Signal Processing:Image Communication,2014,29(8):856-863.