基于语义感知的图像美学质量评估方法

doi:10.11772/j.issn.1001-9081.2018041221

计算机应用 ›› 2018, Vol. 38 ›› Issue (11): 3216-3220.DOI: 10.11772/j.issn.1001-9081.2018041221

• 第七届中国数据挖掘会议(CCDM 2018) • 上一篇下一篇

基于语义感知的图像美学质量评估方法

杨文雅¹, 宋广乐¹, 崔超然¹, 尹义龙^1,2

1. 山东财经大学计算机科学与技术学院, 济南 250014;
2. 山东大学计算机科学与技术学院, 济南 250014

收稿日期:2018-04-30 修回日期:2018-06-19 出版日期:2018-11-10 发布日期:2018-11-10
通讯作者: 崔超然
作者简介:杨文雅(1994-),女,山东菏泽人,硕士研究生,主要研究方向:计算机视觉、机器学习;宋广乐(1992-),男,山东德州人,硕士研究生,主要研究方向:计算机视觉、机器学习;崔超然(1987-),男,山东济南人,教授,博士,CCF会员,主要研究方向:信息检索、计算机视觉、机器学习;尹义龙(1972-),男,山东菏泽人,教授,博士生导师,博士,CCF会员,主要研究方向:机器学习、数据挖掘、生物特征识别。
基金资助:
国家自然科学基金资助项目（61573219，61701281）；山东省自然科学基金资助项目（ZR2017QF009）；山东省高等学校优势学科人才团队培育计划。

Image aesthetic quality assessment method based on semantic perception

YANG Wenya¹, SONG Guangle¹, CUI Chaoran¹, YIN Yilong^1,2

1. School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan Shandong 250014, China;
2. School of Computer Science and Technology, Shandong University, Jinan Shandong 250014, China

Received:2018-04-30 Revised:2018-06-19 Online:2018-11-10 Published:2018-11-10
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61573219, 61701281), the Project of Shandong Provincial Natural Science Foundation (ZR2017QF009), the Fostering Project of Dominant Discipline and Talent Team of Shandong Province Higher Education Institutions.

摘要/Abstract

摘要： 当前图像美学质量评估的研究主要基于图像的视觉内容来给出评价结果，忽视了美感是人的认知活动的事实，在评价时没有考虑用户对图像语义信息的理解。为了解决这一问题，提出了一种基于语义感知的图像美学质量评估方法，将图像的物体类别信息以及场景类别信息也用于图像美学质量评估。运用迁移学习的思想，构建了一种可以融合图像多种特征的混合网络。对于每一幅输入图像，该网络可以分别提取出其物体类别特征、场景类别特征以及美学特征，并将这三种特征进行高质量的融合，以达到更好的图像美学质量评估效果。该方法在AVA数据集上的分类准确率达到89.5%，相对于传统方法平均提高了19.9%，在CUHKPQ数据集上的泛化性能也有了很大提升。实验结果表明，所提方法在图像美学质量评估问题上，能够取得更好的分类性能。

关键词: 图像美学质量评估, 语义感知, 迁移学习, 混合网络

Abstract: Current researches on the assessment of image aesthetic quality are based on visual content of images to give assessment results, ignoring the fact that aesthetics is a person's cognitive activity and not considering the user's understanding towards image semantic information during the evaluating process. In order to solve this problem, an approach to image aesthetic quality assessment based on semantic perception was proposed to apply both the object category information and scene category information of images to the aesthetic quality assessment. Using the transfer learning concept, a hybrid network integrating multiple features of the images was constructed. For each input image, the object category features, scene category features, and aesthetic features were extracted respectively by network, and the three features were combined to achieve better image aesthetic quality evaluation. The classification accuracy of the method on the AVA data set reached 89.5%, which was 19.9% higher than that of the traditional method, and the generalization performance on the CUKHPQ data set was greatly improved. The experimental results show that the proposed approach can achieve better classification performance on the aesthetic quality evaluation of images.

Key words: image aesthetic quality assessment, semantic perception, transfer learning, hybrid network

中图分类号:

TP181

杨文雅, 宋广乐, 崔超然, 尹义龙. 基于语义感知的图像美学质量评估方法[J]. 计算机应用, 2018, 38(11): 3216-3220.

YANG Wenya, SONG Guangle, CUI Chaoran, YIN Yilong. Image aesthetic quality assessment method based on semantic perception[J]. Journal of Computer Applications, 2018, 38(11): 3216-3220.

参考文献

[1] CUI C, FANG H, DENG X, et al. Distribution-oriented aesthetics assessment for image search[C]//Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM, 2017:1013-1016.
[2] TANG X, LUO W, WANG X. Content-based photo quality assessment[J]. IEEE Transactions on Multimedia, 2013, 15(8):1930-1943.
[3] MARCHESOTTI L, PERRONNIN F, LARLUS D, et al. Assessing the aesthetic quality of photographs using generic image descriptors[C]//Proceedings of the 2011 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2011:1784-1791.
[4] KAO Y, HE R, HUANG K. Deep aesthetic quality assessment with semantic information[J]. IEEE Transactions on Image Processing, 2017, 26(3):1482-1495.
[5] LU X, LIN Z, JIN H, et al. Rating image aesthetics using deep learning[J]. IEEE Transactions on Multimedia, 2015, 17(11):2021-2034.
[6] MAI L, JIN H, LIU F. Composition-preserving deep photo aesthetics assessment[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2016:497-506.
[7] DATTA R, JOSHI D, LI J, et al. Studying aesthetics in photographic images using a computational approach[C]//Proceedings of the 9th European Conference on Computer Vision. Berlin:Springer, 2006:288-301.
[8] KE Y, TANG X, JING F. The design of high-level features for photo quality assessment[C]//Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2006:419-426.
[9] DHAR S, ORDONEZ V, BERG T L. High level describable attributes for predicting aesthetics and interestingness[C]//Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2011:1657-1664.
[10] AYDIN T O, SMOLIC A, GROSS M. Automated aesthetic analysis of photographic images[J]. IEEE Transactions on Visualization and Computer graphics, 2015, 21(1):31-42.
[11] MARCHESOTTI L, MURRAY N, PERRONNIN F. Discovering beautiful attributes for aesthetic image analysis[J]. International Journal of Computer Vision, 2015, 113(3):246-266.
[12] 顾婷婷, 郭延文, 殷昆燕. 结合浅景深与构图的图像质量评价[J]. 中国图象图形学报, 2013, 18(5):574-582.(GU T T, GUO Y W, YIN K Y. Image quality assessment combining low DoF and composition[J]. Journal of Image and Graphics, 2013, 18(5):574-582.)
[13] LUO Y, TANG X O. Photo and video quality evaluation:Focusing on the subject[C]//Proceedings of the 10th European Conference on Computer Vision. Berlin:Springer, 2008:386-399.
[14] LUO W, WANG X, TANG X. Content-based photo quality assessment[C]//Proceedings of the 2011 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2011:2206-2213.
[15] 王伟凝, 蚁静缄, 徐向民, 等. 可计算的图像美学分类与评估[J]. 计算机辅助设计与图形学学报, 2014, 26(7):1075-1083.(WANG W N,YI J J,XU X M, et al. Computational aesthetics of image classification and evaluation[J]. Journal of Computer-Aided Design and Computer Graphics, 2014, 26(7):1075-1083.)
[16] MARCHESOTTI L, MURRAY N, PERRONNIN F. Discovering beautiful attributes for aesthetic image analysis[J]. International Journal of Computer Vision, 2015, 113(3):246-266.
[17] ZHANG L, GAO Y, ZHANG C, et al. Perception-guided multimodal feature fusion for photo aesthetics assessment[C]//Proceedings of the 22nd ACM International Conference on Multimedia. New York:ACM, 2014:237-246.
[18] LU X, LIN Z, JIN H, et al. Rapid:Rating pictorial aesthetics using deep learning[C]//Proceedings of the 22nd ACM International Conference on Multimedia. New York:ACM, 2014:457-466.
[19] DONG Z, SHEN X, LI H, et al. Photo quality assessment with DCNN that understands image well[C]//Proceedings of the 21st International Conference on Multimedia Modeling. Berlin:Springer, 2015:524-535.
[20] WANG W, ZHAO M, WANG L, et al. A multi-scene deep learning model for image aesthetic evaluation[J]. Signal Processing:Image Communication, 2016, 47(C):511-518.
[21] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2016:770-778.
[22] MURRAY N, MARCHESOTTI L, PERRONNIN F. AVA:a large-scale database for aesthetic visual analysis[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2012:2408-2415.
[23] GLOROT X, BENGIO Y. Understanding the difficulty of training deep feedforward neural networks[J]. Journal of Machine Learning Research, 2010, 9:249-256.
[24] LU X, LIN Z, SHEN X, et al. Deep multi-patch aggregation network for image style, aesthetics, and quality estimation[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2015:990-998.
[25] LI X, URICCHIO T, BALLAN L, et al. Socializing the semantic gap:a comparative survey on image tag assignment, refinement, and retrieval[J]. ACM Computing Surveys, 2016, 49(1):Article No. 14.
[26] LO K Y, LIU K H, CHEN C S. Assessment of photo aesthetics with efficiency[C]//Proceedings of the 201221st International Conference on Pattern Recognition. Piscataway, NJ:IEEE, 2012:2186-2189.

基于语义感知的图像美学质量评估方法

Image aesthetic quality assessment method based on semantic perception

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	陈争涛, 黄灿, 杨波, 赵立, 廖勇. 基于迁移学习的并行卷积神经网络牦牛脸识别算法[J]. 计算机应用, 2021, 41(5): 1332-1336.
[2]	王锦凯, 贾旭. 基于迁移孪生非负矩阵分解的静脉识别算法[J]. 计算机应用, 2021, 41(3): 898-903.
[3]	曹建芳, 闫敏敏, 贾一鸣, 田晓东. 融合迁移学习的Inception-v3模型在古壁画朝代识别中的应用[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3219-3227.
[4]	赵津, 宋文爱, 邰隽, 杨吉江, 王青, 李晓丹, 雷毅, 邱悦. 儿童阻塞性睡眠呼吸暂停计算机人脸辅助诊断综述[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3394-3401.
[5]	刘晓龙, 王士同. 渐进式分离的开放集模糊域自适应算法[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3127-3131.
[6]	黄学雨, 徐浩特, 陶剑文. 具有特征选择的多源自适应分类框架[J]. 计算机应用, 2020, 40(9): 2499-2506.
[7]	张晨曦, 唐曙, 唐珂. 迁移学习下的火箭发动机参数异常检测策略[J]. 计算机应用, 2020, 40(9): 2774-2780.
[8]	白茹意, 郭小英, 贾春花. 基于特征融合的小样本抽象画图像情感预测[J]. 计算机应用, 2020, 40(8): 2207-2213.
[9]	马金林, 魏萌, 马自萍. 基于深度迁移学习的肺结节分割方法[J]. 计算机应用, 2020, 40(7): 2117-2125.
[10]	肖禾, 刘志勤, 王庆凤, 黄俊, 周莹, 刘启榆, 徐卫云. 基于迁移学习的多视角乳腺肿块和钙化簇分类方法[J]. 计算机应用, 2020, 40(5): 1460-1464.
[11]	陈杰, 张挺, 杜奕. 基于自适应深度迁移学习的多孔介质重构[J]. 计算机应用, 2020, 40(4): 1231-1236.
[12]	谌贵辉, 易欣, 李忠兵, 钱济人, 陈伍. 基于改进YOLOv2和迁移学习的管道巡检航拍图像第三方施工目标检测[J]. 计算机应用, 2020, 40(4): 1062-1068.
[13]	郑宗生, 胡晨雨, 姜晓轶. 基于改进的最大均值差异算法的深度迁移适配网络[J]. 计算机应用, 2020, 40(11): 3107-3112.
[14]	毛文涛, 杨超, 刘亚敏, 田思雨. 面向轴承早期故障检测的多尺度残差注意力深度领域适配模型[J]. 计算机应用, 2020, 40(10): 2890-2898.
[15]	武宽, 秦品乐, 柴锐, 曾建朝. 基于不同超声成像的甲状腺结节良恶性判别[J]. 计算机应用, 2020, 40(1): 77-82.