《计算机应用》唯一官方网站 ›› 2022, Vol. 42 ›› Issue (6): 1957-1964.DOI: 10.11772/j.issn.1001-9081.2021040597

• 多媒体计算与计算机仿真 • 上一篇    

基于显著性深层特征的无参考图像质量评价算法

李佳1, 郑元林1,2(), 廖开阳1,2, 楼豪杰1, 李世宇1, 陈泽豪1   

  1. 1.西安理工大学 印刷包装与数字媒体学院,西安 710048
    2.陕西省印刷包装工程重点实验室,西安 710048
  • 收稿日期:2021-04-16 修回日期:2021-07-02 接受日期:2021-07-15 发布日期:2022-06-22 出版日期:2022-06-10
  • 通讯作者: 郑元林
  • 作者简介:李佳(1997—),女,四川广安人,硕士研究生,主要研究方向:深度学习、图像处理
    廖开阳(1976—),男,湖北荆州人,讲师,博士,主要研究方向:机器视觉、人工智能
    楼豪杰(1996—),男,浙江义乌人,硕士研究生,主要研究方向:计算机视觉、目标检测
    李世宇(1999—),男,陕西西安人,主要研究方向:图像处理
    陈泽豪(2002—),男,陕西渭南人,主要研究方向:图像处理。
  • 基金资助:
    国家自然科学基金资助项目(61771386);陕西省自然科学基金资助项目(2021JM-340)

No-reference image quality assessment algorithm based on saliency deep features

Jia LI1, Yuanlin ZHENG1,2(), Kaiyang LIAO1,2, Haojie LOU1, Shiyu LI1, Zehao CHEN1   

  1. 1.Faculty of Printing,Packaging Engineering and Digital Media Technology,Xi’an University of Technology,Xi’an Shaanxi 710048,China
    2.Printing and Packaging Engineering Technology Research Centre of Shaanxi Province,Xi’an Shaanxi 710048,China
  • Received:2021-04-16 Revised:2021-07-02 Accepted:2021-07-15 Online:2022-06-22 Published:2022-06-10
  • Contact: Yuanlin ZHENG
  • About author:LI Jia,born in 1997,M. S. candidate. Her research interests include deep learning,image processing
    LIAO Kaiyang,born in 1976,Ph. D.,lecturer. His research interests include machine vision,artificial intelligence.
    LOU Haojie,born in 1996,M. S. candidate. His research interests include computer vision,object detection.
    LI Shiyu,born in 1999. His research interests include image processing
    CHEN Zehao,born in 2002,. His research interests include image processing.
  • Supported by:
    National Natural Science Foundation of China(61771386);Natural Science Foundation of Shaanxi Province(2021JM-340)

摘要:

针对通用型无参考图像质量评价(NR-IQA)算法,提出一种基于伪参考图像显著性深层特征的评价算法。首先,在失真图像的基础上,利用微调的ConSinGAN模型生成相应的伪参考图像作为失真图像的补偿信息,弥补NR-IQA算法缺少真实参考信息的不足;然后,提取伪参考图像的显著性信息,将伪参考显著性图像与失真图像输入到VGG16网络中提取深层特征;最后,融合二者的深层特征并将其映射到由全连接层组成的回归网络中,从而产生与人类视觉一致的质量预测。为了验证算法的有效性,在四个大型公开的图像数据集TID2013、TID2008、CSIQ与LIVE上进行实验,结果显示所提算法在TID2013数据集上的斯皮尔曼秩相关系数(SROCC)比H-IQA算法提升了5个百分点,比RankIQA算法提升了14个百分点,针对单一失真类型也具有稳定的性能。实验结果表明,所提算法总体表现优于现有主流全参考图像质量评价(FR-IQA)和NR-IQA算法,与人类主观感知表现一致。

关键词: 无参考图像质量评价, 生成对抗网络, 显著性, 深度学习, 超分辨率

Abstract:

Aiming at the universal No-Reference Image Quality Assessment (NR-IQA) algorithms, a new NR-IQA algorithm based on the saliency deep features of the pseudo reference image was proposed. Firstly, based on the distorted image, the corresponding pseudo reference image of the distorted image generated by ConSinGAN model was used as compensation information of the distorted image, thereby making up for the weakness of NR-IQA methods: lacking real reference information. Secondly, the saliency information of the pseudo reference image was extracted, and the pseudo saliency map and the distorted image were input into VGG16 netwok to extract deep features. Finally, the obtained deep features were merged and mapped into the regression network composed of fully connected layers to obtain a quality prediction consistent with human vision.Experiments were conducted on four large public image datasets TID2013, TID2008, CSIQ and LIVE to prove the effectiveness of the proposed algorithm. The results show that the Spearman Rank-Order Correlation Coefficient (SROCC) of the proposed algorithm on the TID2013 dataset is 5 percentage points higher than that of H-IQA (Hallucinated-IQA) algorithm and 14 percentage points higher than that of RankIQA (learning from Rankings for no-reference IQA) algorithm. The proposed algorithm also has stable performance for the single distortion types. Experimental results indicate that the proposed algorithm is superior to the existing mainstream Full-Reference Image Quality Assessment (FR-IQA) and NR-IQA algorithms, and is consistent with human subjective perception performance.

Key words: No-Reference Image Quality Assessment (NR-IQA), Generative Adversarial Network (GAN), saliency, deep learning, super-resolution

中图分类号: