计算机应用 ›› 2019, Vol. 39 ›› Issue (8): 2181-2185.DOI: 10.11772/j.issn.1001-9081.2018122452

• 人工智能 •    下一篇

图像整体与局部区域嵌入的视觉情感分析

蔡国永, 贺歆灏, 储阳阳   

  1. 广西可信软件重点实验室(桂林电子科技大学), 广西 桂林 541004
  • 收稿日期:2018-12-21 修回日期:2019-03-23 发布日期:2019-04-18 出版日期:2019-08-10
  • 通讯作者: 贺歆灏
  • 作者简介:蔡国永(1971-),男,广西桂林人,教授,博士,CCF会员,主要研究方向:社交媒体、数据挖掘;贺歆灏(1994-),男,湖南醴陵人,硕士研究生,主要研究方向:深度学习、情感分析;储阳阳(1993-),男,安徽安庆人,硕士研究生,主要研究方向:深度学习、情感分析。
  • 基金资助:
    国家自然科学基金资助项目(61763007);广西自然科学基金重点项目(2017JJD160017)。

Visual sentiment analysis by combining global and local regions of image

CAI Guoyong, HE Xinhao, CHU Yangyang   

  1. Guangxi Key Laboratory of Trusted Software(Guilin University of Electronic Technology), Guilin Guangxi 541004, China
  • Received:2018-12-21 Revised:2019-03-23 Online:2019-04-18 Published:2019-08-10
  • Supported by:
    This work is partially supported by National Natural Science Foundation of China (61763007), the Natural Science Foundation of Guangxi Autonomous Region (2017JJD160017).

摘要: 目前多数图像视觉情感分析方法主要从图像整体构建视觉情感特征表示,然而图像中包含对象的局部区域往往更能突显情感色彩。针对视觉图像情感分析中忽略局部区域情感表示的问题,提出一种嵌入图像整体特征与局部对象特征的视觉情感分析方法。该方法结合整体图像和局部区域以挖掘图像中的情感表示,首先利用对象探测模型定位图像中包含对象的局部区域,然后通过深度神经网络抽取局部区域的情感特征,最后用图像整体抽取的深层特征和局部区域特征来共同训练图像情感分类器并预测图像的情感极性。实验结果表明,所提方法在真实数据集TwitterⅠ和TwitterⅡ上的情感分类准确率分别达到了75.81%和78.90%,高于仅从图像整体特征和仅从局部区域特征分析情感的方法。

关键词: 社交媒体, 情感分析, 图像局部对象检测, 深度学习, 神经网络

Abstract: Most existing visual sentiment analysis methods mainly construct visual sentiment feature representation based on the whole image. However, the local regions with objects in the image are able to highlight the sentiment better. Concerning the problem of ignorance of local regions sentiment representation in visual sentiment analysis, a visual sentiment analysis method by combining global and local regions of image was proposed. Image sentiment representation was mined by combining a whole image with local regions of the image. Firstly, an object detection model was used to locate the local regions with objects in the image. Secondly, the sentiment features of the local regions with objects were extracted by deep neural network. Finally, the deep features extracted from the whole image and the local region features were utilized to jointly train the image sentiment classifier and predict the sentiment polarity of the image. Experimental results show that the classification accuracy of the proposed method reaches 75.81% and 78.90% respectively on the real datasets TwitterⅠand TwitterⅡ, which is higher than the accuracy of sentiment analysis methods based on features extracted from the whole image or features extracted from the local regions of image.

Key words: social media, sentiment analysis, image local object detection, deep learning, neural network

中图分类号: