对象级特征引导的显著性视觉注意方法

doi:10.11772/j.issn.1001-9081.2016.11.3217

计算机应用 ›› 2016, Vol. 36 ›› Issue (11): 3217-3221.DOI: 10.11772/j.issn.1001-9081.2016.11.3217

对象级特征引导的显著性视觉注意方法

杨凡^1,2, 蔡超^1,2

1. 华中科技大学自动化学院, 武汉 430074;
2. 多谱信息处理技术国家重点实验室(华中科技大学), 武汉 430074

收稿日期:2016-03-18 修回日期:2016-06-26 出版日期:2016-11-10 发布日期:2016-11-12
通讯作者: 蔡超
作者简介:杨凡(1990-),男,山东淄博人,硕士研究生,主要研究方向:视觉注意、显著性目标检测、深度学习;蔡超(1971-),男,山东东明人,副教授,博士,主要研究方向:计算机视觉、目标识别、医学图像处理、任务规划。
基金资助:
华为创新基金资助项目（YJCB2010022IN）。

Significant visual attention method guided by object-level features

YANG Fan^1,2, CAI Chao^1,2

1. School of Automation, Huazhong University of Science and Technology, Wuhan Hubei 430074, China;
2. National Key Laboratory of Science and Technology on Multi-spectral Information Processing, (Huazhong University of Science and Technology), Wuhan Hubei 430074, China

Received:2016-03-18 Revised:2016-06-26 Online:2016-11-10 Published:2016-11-12
Supported by:
This work is partially supported by the Huawei Innovation Fund (YJCB2010022IN).

摘要/Abstract

摘要： 针对已有视觉注意模型在整合对象特征方面的不足，提出一种新的结合高层对象特征和低层像素特征的视觉注意方法。首先，利用已训练的卷积神经网（CNN）对多类目标的强大理解能力，获取待处理图像中对象的高层次特征图；然后结合实际的眼动跟踪数据，训练多个对象特征图的加权系数，给出对象级突出图；紧接着提取像素级突出图，并和对象级突出图融合获得显著图；最后，在OSIE和MIT数据集上验证了该方法，并与国际上流行的视觉注意方法进行对比，结果显示该算法在OSIE数据集上获得的AUC值相对更高。实验结果表明，所提方法能够更加充分地利用图像中对象信息，提高显著性预测的准确率。

关键词: 视觉注意, 自顶向下, 显著性, 对象信息, 卷积神经网

Abstract: Concerning the defects of fusing object information by existing visual attention models, a new visual attention method combining high-level object features and low-level pixel features was proposed. Firstly, high-level feature maps were obtained by using Convolutional Neural Network (CNN) which has strong understanding of multi-class targets. Then all object feature maps were combined by training the weights with eye fixation data. Then the saliency map was obtained by fusing pixel-level conspicuity map and object-level conspicuity map. Finally, the proposed method was compared with many popular visual attention methods on OSIE and MIT datasets. Compared with the contrast methods, the Area Under Curve (AUC) result of the proposed method is increased. Experimental results show that the proposed method can make full use of the object information in the image, and increases the saliency prediction accuracy.

Key words: visual attention, top-down, saliency, object information, Convolutional Neural Network (CNN)

中图分类号:

TP391.41

杨凡, 蔡超. 对象级特征引导的显著性视觉注意方法[J]. 计算机应用, 2016, 36(11): 3217-3221.

YANG Fan, CAI Chao. Significant visual attention method guided by object-level features[J]. Journal of Computer Applications, 2016, 36(11): 3217-3221.

参考文献

[1] BORJI A, ITTI L. State-of-the-Art in visual attention modeling[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1):185-207.
[2] ITTI L, KOCH C, NIEBUR E. A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(11):1254-1259.
[3] HAREL J, KOCH C, PERONA P. Graph-based visual saliency[C]//NIPS 2006:Proceedings of the 2006 Advances in Neural Information Processing Systems. Cambridge:MIT Press, 2006:545-552.
[4] KRIZHEVSKY A, SUTSKEVER I, HINTON G. ImageNet classification with deep convolutional neural networks[EB/OL].[2015-10-10]. https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf.
[5] BORJI A, ITTI L. Exploiting local and global patch rarities for saliency detection[C]//CVPR 2012:Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2012:478-485.
[6] VIG E, DORR M, COX D. Large-scale optimization of hierarchical features for saliency prediction in natural images[C]//CVPR 2014:Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2014:2798-2805.
[7] BRUCE N, TSOTSOS J. Saliency based on information maximization[EB/OL].[2015-10-10]. https://papers.nips.cc/paper/2830-saliency-based-on-information-maximization.pdf.
[8] LI J, LEVINE M, AN X, et al. Visual saliency based on scale-space analysis in the frequency domain[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(4):996-1010.
[9] BORJI A, SIHITE D, ITTI L, et al. Probabilistic learning of task-specific visual attention[C]//CVPR 2012:Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2012:470-477.
[10] ZHANG L, TONG M, MARKS, T, et al. SUN:a Bayesian framework for saliency using natural statistics[J]. Journal of Vision, 2008, 8(7):Artile No. 32.
[11] SIAGIAN C, ITTI L. Rapid biologically-inspired scene classification using features shared with visual attention[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(2):300-312.
[12] JUDD T, EHINGER K, DURAND F, et al. Learning to predict where humans look[C]//ICCV 2009:Proceedings of the 2009 International Conference on Computer Vision. Piscataway, NJ:IEEE, 2009:2106-2113.
[13] ZHAO Q, KOCH C. Learning a saliency map using fixated locations in natural scenes[J]. Journal of Vision, 2011, 11(3):Artile No. 9.
[14] BORJI A. Boosting bottom-up and top-down visual features for saliency estimation[C]//CVPR 2012:Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2012:438-445.
[15] XU J, JIANG M, WANG S, et al. Predicting human gaze beyond pixels[J]. Journal of Vision, 2014, 14(1):Artile No. 28.
[16] YOSINSKI J, CLUNE J, NGUYEN A, et al. Understanding neural networks through deep visualization[EB/OL].[2015-10-10]. http://arxiv.org/abs/1506.06579.
[17] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//CVPR 2012:Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2014:580-587.
[18] UIJLINGS J, van de SANDE K E A, GEVERS T, et al. Selective search for object recognition[J]. International Journal of Computer Vision, 2013, 104(2):154-171.
[19] REN S, HE K, GIRSHICK R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[EB/OL].[2015-10-10]. http://arxiv.org/abs/1506.01497.
[20] JIANG H, WANG J, YUAN Z, et al. Salient object detection:a discriminative regional feature integration approach[C]//CVPR 2012:Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2013:2083-2090.
[21] 暴林超, 蔡超, 肖洁, 等.一种用于复杂目标感知的视觉注意模型[J]. 计算机工程, 2011, 37(13):17-19.(BAO L C, CAI C, XIAO J, et al. Visual attention model for complex target perception[J]. Computer Engineering, 2011, 37(13):17-19.)
[22] 肖洁, 蔡超, 丁明跃. 一种图斑特征引导的感知分组视觉注意模型[J]. 航空学报, 2010, 31(11):2266-2274.(XIAO J, CAI C, DING M Y. A novel visual attention model based on blob-guided perceptual grouping[J]. Acta Aeronautica et Astronautica Sinica, 2010, 31(11):2266-2274.)

对象级特征引导的显著性视觉注意方法

Significant visual attention method guided by object-level features

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	王贺兵, 张春梅. 基于非对称卷积-压缩激发-次代残差网络的人脸关键点检测[J]. 计算机应用, 2021, 41(9): 2741-2747.
[2]	宋中山, 梁家锐, 郑禄, 刘振宇, 帖军. 基于双向门控尺度特征融合的遥感场景分类[J]. 计算机应用, 2021, 41(9): 2726-2735.
[3]	李康康, 张静. 基于注意力机制的多层次编码和解码的图像描述模型[J]. 计算机应用, 2021, 41(9): 2504-2509.
[4]	张永斌, 常文欣, 孙连山, 张航. 基于字典的域名生成算法生成域名的检测方法[J]. 计算机应用, 2021, 41(9): 2609-2614.
[5]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[6]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[7]	牟长宁, 王海鹏, 周丕宇, 侯鑫行. 基于图卷积神经网络的串联质谱从头测序[J]. 计算机应用, 2021, 41(9): 2773-2779.
[8]	孙潇, 徐金东. 基于级联生成对抗网络的遥感图像去雾方法[J]. 计算机应用, 2021, 41(8): 2440-2444.
[9]	曾祥银, 郑伯川, 刘丹. 基于深度卷积神经网络和聚类的左右轨道线检测[J]. 计算机应用, 2021, 41(8): 2324-2329.
[10]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287.
[11]	秦斌斌, 彭良康, 卢向明, 钱江波. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 41(8): 2330-2337.
[12]	黄程程, 董霄霄, 李钊. 基于二维Winograd算法的深流水线5×5卷积方法[J]. 计算机应用, 2021, 41(8): 2258-2264.
[13]	谭道强, 曾诚, 乔金霞, 张俊. 基于混合注意力模型的阴影检测方法[J]. 计算机应用, 2021, 41(7): 2076-2081.
[14]	武光利, 李雷霆, 郭振洲, 王成祥. 基于改进的双向长短期记忆网络的视频摘要生成模型[J]. 计算机应用, 2021, 41(7): 1908-1914.
[15]	吴则举, 焦翠娟, 陈亮. 基于改进Faster R-CNN的轮胎缺陷检测方法[J]. 计算机应用, 2021, 41(7): 1939-1946.