三维重定向图像主观和客观质量评价方法

doi:10.11772/j.issn.1001-9081.2018102054

计算机应用 ›› 2019, Vol. 39 ›› Issue (5): 1434-1439.DOI: 10.11772/j.issn.1001-9081.2018102054

• 虚拟现实与多媒体计算 • 上一篇下一篇

三维重定向图像主观和客观质量评价方法

富振奇, 邵枫

宁波大学信息科学与工程学院, 浙江宁波 315211

收稿日期:2018-10-11 修回日期:2018-12-02 发布日期:2019-05-14 出版日期:2019-05-10
通讯作者: 邵枫
作者简介:富振奇(1993-),男,浙江嘉兴人,硕士研究生,主要研究方向:图像处理、图像质量评价;邵枫(1980-),男,浙江杭州人,教授,博士生导师,博士,CCF会员,主要研究方向:图像处理、视频编码与质量评价。
基金资助:
国家自然科学基金资助项目（61622109）。

Subjective and objective quality assessment for stereoscopic 3D retargeted images

FU Zhenqi, SHAO Feng

Faculty of Information Science and Engineering, Ningbo University, Ningbo Zhejiang 315211, China

Received:2018-10-11 Revised:2018-12-02 Online:2019-05-14 Published:2019-05-10
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61622109).

摘要/Abstract

摘要： 三维（S3D）图像重定向技术的作用是调整S3D图像的宽高比。为准确和客观地衡量三维重定向图像的视觉质量，建立了一个S3D重定向图像质量评价数据库。首先，使用八种具有代表性的三维重定向算法对45幅原始图像按两种重定向尺度进行分辨率调整，共生成720幅三维重定向图像；然后，每幅重定向图像通过主观测试，得到相应的主观打分值；最后，对主观分数进行处理，得到平均主观意见分（MOS）值。在此基础上，提出一种三维重定向图像客观质量评价方法，即通过提取S3D重定向图像的深度感特征、视觉舒适度特征和左右视点的图像质量特征，使用支持向量回归预测得到S3D重定向图像的视觉质量。在提出的数据库上进行测试可以得知，所提方法的Pearson线性相关系数高于0.82，Spearman等级系数高于0.81，表明其能有效预测S3D重定向图像的视觉质量。

关键词: 质量评价, 图像数据库, 三维图像重定向, 深度感, 舒适度

Abstract: Stereoscopic 3D (S3D) image retargeting aims to adjust aspect ratio of S3D images. To objectively and accurately assess the quality of different retargeted S3D images, a retargeted S3D image quality assessment database was constructed. Firstly, 45 original images were retargeted by eight representative retargeting algorithms with two retargeting scales to generate 720 retargeted S3D images. Then, the subjective quality evaluation score of each retargeted image was obtained via subjective testing. Finally, the subjective scores were converted to MOS (Mean Opinion Score) values. Based on all above, an objective quality assessment method was proposed for retargeted S3D images. In this method, three types of features including depth perception, visual comfort and image quality of left and right views were extracted to calculate the retargeted S3D image quality with the use of support vector regression prediction. Experimental results on the proposed database show that the proposed method has the Pearson linear correlation coefficient and the Spearman rank-order correlation coefficient higher than 0.82 and 0.81 respectively, demonstrating its superiority in retargeted S3D image visual quality assessment.

Key words: quality assessment, image database, stereoscopic 3D image retargeting, depth perception, visual comfort

中图分类号:

TN919.81

富振奇, 邵枫. 三维重定向图像主观和客观质量评价方法[J]. 计算机应用, 2019, 39(5): 1434-1439.

FU Zhenqi, SHAO Feng. Subjective and objective quality assessment for stereoscopic 3D retargeted images[J]. Journal of Computer Applications, 2019, 39(5): 1434-1439.

参考文献

[1] LUO S J, SUN Y T, SHEN I C, et al. Geometrically consistent stereoscopic image editing using patch-based synthesis[J]. IEEE Transactions on Visualization and Computer Graphics, 2015, 21(1): 56-67.
[2] SHAMIR A, SHAMIR A, AVIDAN S. Improved seam carving for video retargeting[J]. ACM Transactions on Graphics, 2008, 27(3): Article No. 16.
[3] WOLF L, GUTTMANN M, COHEN-OR D. Non-homogeneous content-driven video-retargeting[C]// Proceedings of the 2007 International Conference on Computer Vision. Piscataway, NJ: IEEE, 2007: 1-6.
[4] LEE K, MOORTHY A K, LEE S, et al. S3D visual activity assessment based on natural scene statistics[J]. IEEE Transactions on Image Processing, 2014, 23(1): 450-465.
[5] KOOI F L, TOET A. Visual comfort of binocular and S3D displays[J]. Displays, 2004, 25(2/3): 99-108.
[6] BASHA DEKEL T, MOSES Y, AVIDAN S. Stereo seam carving a geometrically consistent approach[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(10): 2513-2525.
[7] SHAO F, LIN W C, LIN W S, et al. Stereoscopic visual attention guided seam carving for stereoscopic image retargeting[J]. Journal of Display Technology, 2015, 12(1): 22-30.
[8] LEI J J, WU M, ZHANG C Q, et al. Depth-preserving stereo image retargeting based on pixel fusion[J]. IEEE Transactions on Multimedia, 2017, 79(7): 1442-1453.
[9] CHANG C H, LIANG C K, CHUANG Y Y. Content-aware display adaptation and editing for stereoscopic images[J]. IEEE Transactions on Multimedia, 2011, 13(4): 589-601.
[10] LIN S S, LIN C H, CHANG S H, et al. Object-coherence warping for stereoscopic image retargeting[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(5): 759-768.
[11] LI B, DUAN L Y, LIN C W, et al. Depth-preserving warping for stereo image retargeting[J]. IEEE Transactions on Image Processing, 2015, 24(9): 2811-2826.
[12] SHAO F, LIN W C, LIN W S, et al. QoE-guided warping for stereoscopic image retargeting[J]. IEEE Transactions on Image Processing, 2017, 26(10): 790-4805.
[13] LIN Y H, WU J L. Quality assessment of stereoscopic S3D image compression by binocular integration behaviors[J]. IEEE Transactions on Image Processing, 2014, 23(4): 1527-1542.
[14] SHAO F, LI K M, LIN W S, et al. Full-reference quality assessment of stereoscopic images by learning binocular receptive field properties[J]. IEEE Transactions on Image Processing, 2015, 24(10): 2971-2983.
[15] WANG J H, REHMAN A, ZENG K, et al. Quality prediction of asymmetrically distorted stereoscopic S3D images[J]. IEEE Transactions on Image Processing, 2015, 24(11): 3400-3414.
[16] FANG Y M, ZENG K, WANG Z, et al. Objective quality assessment for image retargeting based on structural similarity[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2014, 4(1): 95-105.
[17] HSU C C, LIN C W, FANG Y M, et al. Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss[J]. IEEE Journal of Selected Topics in Signal Processing, 2014, 8(3): 377-389.
[18] ZHANG Y B, FANG Y M, LIN W S, et al. Backward registration based aspect ratio similarity for image retargeting quality assessment[J]. IEEE Transactions on Image Processing, 2016, 25(9): 4286-4297.
[19] LIANG Y, LIU Y J, GUTIERREZ D. Objective quality prediction of image retargeting algorithms[J]. IEEE Transactions on Visualization and Computer Graphics, 2017, 23(2): 1099-1110.
[20] JIANG Q P, SHAO F, LIN W S, et al. Learning sparse representation for objective image retargeting quality assessment[J]. IEEE Transactions on Cybernetics, 2018, 48(4): 1276-1289.
[21] CHEN Z B, LIN J X, NING L, et al. Full reference quality assessment for image retargeting based on natural scene statistics modeling and bi-directional saliency similarity[J]. IEEE Transactions on Image Processing, 2017, 26(11): 5138-5148.
[22] ZHANG Y C, NGAN K N, MA L, et al. Objective quality assessment of image retargeting by incorporating fidelity measures and inconsistency detection[J]. IEEE Transactions on Image Processing, 2017, 26(12): 5980-5993.
[23] GUO Y C, HAO Y T, YU M. Image retargeting quality assessment based on content deformation measurement[J]. Signal Processing: Image Communication, 2018, 67:171-181.
[24] WANG W G, SHEN J B, YU Y Z, et al. Stereoscopic thumbnail creation via efficient stereo saliency detection[J]. IEEE Transactions on Visualization and Computer Graphics, 2017, 23(8): 2014-2027.
[25] WANG Y S, TAI C L, SORKINE O, et al. Optimized scale-and-stretch for image resizing[J]. ACM Transactions on Graphics, 2008, 27(5): 118.
[26] SESHADRINATHAN K, SOUNDARARAJAN R, BOVIK A C, et al. Study of subjective and objective quality assessment of video[J]. IEEE Transactions on Image Process. 2010, 19(6): 1427-1441.
[27] SUN D, ROTH S, BLACK M J. Secrets of optical flow estimation and their principles[C]// Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2010:2432-2439.
[28] JUNG Y J, SOHN H, LEE S I, et al. Predicting visual discomfort of stereoscopic images using human attention model[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2013, 23(12): 2077-2082.
[29] HAREL J, KOCH C, PERONA P. Graph-based visual saliency[C]// Proceedings of the 19th International Conference on Neural Information Processing Systems. Cambridge, MA: MIT Press, 2006: 545-552.
[30] LIU C, YUEN J, TORRALBA A. SIFT flow: dense correspondence across scenes and its applications[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5): 978-994.
[31] SIMAKOV D, CASPI Y, SHECHTMAN E, et al. Summarizing visual data using bidirectional similarity[C]// Proceedings of the 2008 Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2008: 1-8.
[32] PELE O, WERMAN M. Fast and robust earth mover's distances[C]// Proceedings of the 2009 International Conference on Computer Vision. Piscataway, NJ: IEEE, 2009: 460-467.
[33] SHEIKH H R, SABIR M F, BOVIK A C. A statistical evaluation of recent full reference image quality assessment algorithms[J]. IEEE Transactions on Image Processing, 2006, 15(11): 3440-3451.

三维重定向图像主观和客观质量评价方法

Subjective and objective quality assessment for stereoscopic 3D retargeted images

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	李佳, 郑元林, 廖开阳, 楼豪杰, 李世宇, 陈泽豪. 基于显著性深层特征的无参考图像质量评价算法[J]. 《计算机应用》唯一官方网站, 2022, 42(6): 1957-1964.
[2]	冉家敏, 倪志伟, 彭鹏, 朱旭辉. 考虑空间众包工作者服务质量的任务分配策略及其萤火虫群优化算法求解[J]. 计算机应用, 2021, 41(3): 794-802.
[3]	蒋阳升, 王胜男, 涂家祺, 李莎, 王红军. 面向高铁站的热舒适度和能耗综合预测[J]. 计算机应用, 2021, 41(1): 249-257.
[4]	陈彦如, 张涂静娃, 杜千, 冉茂亮, 王红军. 基于深度森林的高铁站室内热舒适度等级预测[J]. 计算机应用, 2021, 41(1): 258-264.
[5]	王思宇, 高炜欣, 李璐. 环口焊X射线焊缝图像质量评定模型[J]. 计算机应用, 2020, 40(9): 2748-2753.
[6]	张俊升, 徐晶晶, 余伟. 面部美化图像质量无参考评价方法[J]. 计算机应用, 2020, 40(4): 1184-1190.
[7]	曹玉东, 蔡希彪. 基于增强型对抗学习的无参考图像质量评价算法[J]. 计算机应用, 2020, 40(11): 3166-3171.
[8]	张斌, 罗晓清, 张战成. 基于TV-L¹结构纹理分解的图像融合质量评价算法[J]. 计算机应用, 2019, 39(9): 2701-2706.
[9]	程宇, 邓德祥, 颜佳, 范赐恩. 基于卷积神经网络的弱光照图像增强算法[J]. 计算机应用, 2019, 39(4): 1162-1169.
[10]	李秋珍, 栾朝阳, 汪双喜. 基于卷积神经网络的人脸图像质量评价[J]. 计算机应用, 2019, 39(3): 695-699.
[11]	管非凡, 郁梅, 宋洋, 邵华, 蒋刚毅. 高动态范围图像客观质量评价方法[J]. 计算机应用, 2017, 37(3): 695-698.
[12]	介丹, 胡强强, 徐成武, 高保禄, 李海芳. 基于模糊综合评价的接边质量评价[J]. 计算机应用, 2016, 36(9): 2580-2583.
[13]	李娜, 周蓬勃, 耿国华, 贾晖. 色彩复原图像的质量评价方法[J]. 计算机应用, 2016, 36(6): 1673-1676.
[14]	马允, 王晓东, 章联军. 基于空域自然场景统计的无参考立体图像质量评价模型[J]. 计算机应用, 2016, 36(3): 783-788.
[15]	贾旭, 王锦凯, 崔建江, 孙福明, 薛定宇. 基于稀疏编码的手背静脉识别算法[J]. 计算机应用, 2015, 35(4): 1129-1132.