Abstract: Most existing visual sentiment analysis methods mainly construct visual sentiment feature representation based on the whole image. However, the local regions with objects in the image are able to highlight the sentiment better. Concerning the problem of ignorance of local regions sentiment representation in visual sentiment analysis, a visual sentiment analysis method by combining global and local regions of image was proposed. Image sentiment representation was mined by combining a whole image with local regions of the image. Firstly, an object detection model was used to locate the local regions with objects in the image. Secondly, the sentiment features of the local regions with objects were extracted by deep neural network. Finally, the deep features extracted from the whole image and the local region features were utilized to jointly train the image sentiment classifier and predict the sentiment polarity of the image. Experimental results show that the classification accuracy of the proposed method reaches 75.81% and 78.90% respectively on the real datasets TwitterⅠand TwitterⅡ, which is higher than the accuracy of sentiment analysis methods based on features extracted from the whole image or features extracted from the local regions of image.

Key words: social media, sentiment analysis, image local object detection, deep learning, neural network

摘要： 目前多数图像视觉情感分析方法主要从图像整体构建视觉情感特征表示，然而图像中包含对象的局部区域往往更能突显情感色彩。针对视觉图像情感分析中忽略局部区域情感表示的问题，提出一种嵌入图像整体特征与局部对象特征的视觉情感分析方法。该方法结合整体图像和局部区域以挖掘图像中的情感表示，首先利用对象探测模型定位图像中包含对象的局部区域，然后通过深度神经网络抽取局部区域的情感特征，最后用图像整体抽取的深层特征和局部区域特征来共同训练图像情感分类器并预测图像的情感极性。实验结果表明，所提方法在真实数据集TwitterⅠ和TwitterⅡ上的情感分类准确率分别达到了75.81%和78.90%，高于仅从图像整体特征和仅从局部区域特征分析情感的方法。

关键词: 社交媒体, 情感分析, 图像局部对象检测, 深度学习, 神经网络