基于卷积神经网络的图文融合媒体情感预测

doi:10.11772/j.issn.1001-9081.2016.02.0428

计算机应用 ›› 2016, Vol. 36 ›› Issue (2): 428-431.DOI: 10.11772/j.issn.1001-9081.2016.02.0428

• 第三届CCF大数据学术会议(CCF BigData 2015) • 上一篇下一篇

基于卷积神经网络的图文融合媒体情感预测

蔡国永, 夏彬彬

桂林电子科技大学广西可信软件重点实验室, 广西桂林 541004

收稿日期:2015-08-29 修回日期:2015-09-16 出版日期:2016-02-10 发布日期:2016-02-03
通讯作者: 蔡国永(1971-),男,广西河池人,教授,博士,CCF高级会员,主要研究方向:社交媒体数据处理、机器学习、可信软件。
作者简介:夏彬彬(1990-),男,江西赣州人,硕士研究生,主要研究方向:自然语言处理、情感分析。
基金资助:
广西可信软件重点实验室基金资助项目(kx201503);广西研究生教育创新计划资助项目(YCSZ2015147);广西高等学校高水平创新团队及卓越学者计划项目。

Multimedia sentiment analysis based on convolutional neural network

CAI Guoyong, XIA Binbin

Guangxi Key Laboratory of Trusted Software, Guilin University of Electronic Technology, Guilin Guangxi 541004, China

Received:2015-08-29 Revised:2015-09-16 Online:2016-02-10 Published:2016-02-03

摘要/Abstract

摘要： 近年来,用户在社交媒体上越来越多地使用多媒体内容来分享经历和表达情绪。相比单独的文本和图像,融合文本和图像的多媒体内容能够更为充分地揭示用户的真实情感。针对单一文本或图像的情感不明显问题,提出了一种基于卷积神经网络(CNN)的图文融合媒体的情感分析方法。该方法融合图像特征与三个不同级别(词语级、短语级和句子级)的文本特征构建CNN模型,以分析比较不同层次的语义特征对情感预测的影响。在真实数据集上的实验结果表明,通过捕捉文本情感特征和图像情感特征之间的内部联系,可以更准确地实现对图文融合媒体情感的预测。

关键词: 社交媒体, 多媒体, 情感分析, 卷积神经网络

Abstract: In recent years, more and more multimedia contents were used on social media to share users' experiences and emotions. Compared to single text or image, the complementation of text and image can further fully reveal the real emotion of users. Concerning the sentiment shortage of single text or image, a method based on Convolutional Neural Network (CNN) was proposed for multimedia sentiment analysis. In order to explore the influence of semantic representation in different level, image features were combined with different level (word-level, phrase-level and sentence-level) text features to construct CNN. The experimental results on two real-world datasets demonstrate that the proposed method gets more accurate prediction on multimedia sentiment analysis by capturing the internal relations between text and image.

Key words: social media, multimedia, sentiment analysis, Convolutional Neural Network(CNN)

中图分类号:

TP391

蔡国永, 夏彬彬. 基于卷积神经网络的图文融合媒体情感预测[J]. 计算机应用, 2016, 36(2): 428-431.

CAI Guoyong, XIA Binbin. Multimedia sentiment analysis based on convolutional neural network[J]. Journal of Computer Applications, 2016, 36(2): 428-431.

参考文献

[1] DING X, LIU T, DUAN J, et al. Mining user consumption intention from social media using domain adaptive convolutional neural network[C]//Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press, 2015: 2389-2395.
[2] BOLLEN J, MAO H, ZENG X. Twitter mood predicts the stock market[J]. Journal of Computational Science, 2011, 2(1): 1-8.
[3] ZHANG X, FUEHRES H, GLOOR P A. Predicting stock market indicators through Twitter "I hope it is not as bad as I fear"[J]. Procedia——Social and Behavioral Sciences, 2011, 26: 55-62.
[4] YOU Q, LUO J, JIN H, et al. Robust image sentiment analysis using progressively trained and domain transferred deep networks [C/OL]//Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press, 2015: 381-388.
[5] BORTH D, JI R, CHEN T, et al. Large-scale visual sentiment ontology and detectors using adjective noun pairs[C]//MM '13: Proceedings of the 21st ACM International Conference on Multimedia. New York: ACM, 2013: 223-232.
[6] 李书玲,刘蓉,张鎏钦,等.基于改进型 SVM 算法的语音情感识别[J].计算机应用,2013,33(7):1938-1941. (LI S L, LIU R, ZHANG L Q, et al. Speech emotion recognition algorithm based on modified SVM[J]. Journal of Computer Applications, 2013, 33(7): 1938-1941.)
[7] JIANG Y G, XU B, XUE X. Predicting emotions in user-generated videos[C]//Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press, 2014: 73-79.
[8] TURNEY P D. Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews[C]//ACL '02: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2002: 417-424.
[9] PANG B, LEE L, VAITHYANATHAN S. Thumbs up?: sentiment classification using machine learning techniques[C]//EMNLP '02: Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2002: 79-86.
[10] 梁军,柴玉梅,原慧斌,等.基于深度学习的微博情感分析[J].中文信息学报,2014,28(5):155-161. (LIANG J, CHAI Y M, YUAN H B, et al. Deep learning for Chinese Micro-blog sentiment analysis[J]. Journal of Chinese Information Processing, 2014, 28(5): 155-161.)
[11] YUAN J, MCDONOUGH S, YOU Q, et al. Sentribute: image sentiment analysis from a mid-level perspective[C]//WISDOM '13: Proceedings of the Second International Workshop on Issues of Sentiment Discovery and Opinion Mining. New York: ACM, 2013: 10.
[12] THELWALL M, BUCKLEY K, PALTOGLOU G, et al. Sentiment strength detection in short informal text[J]. Journal of the American Society for Information Science and Technology, 2010, 61(12): 2544-2558.
[13] WANG M, CAO D, LI L, et al. Microblog sentiment analysis based on cross-media bag-of-words model[C]//ICIMCS '14: Proceedings of International Conference on Internet Multimedia Computing and Service. New York: ACM, 2014: 76.
[14] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//NIPS 2012: Advances in Neural Information Processing Systems 25. Cambridge, MA: MIT Press, 2012: 1097-1105.
[15] ABDEL-HAMID O, MOHAMED A, JIANG H, et al. Applying convolutional neural networks concepts to hybrid NN-HMM model for speech recognition[C]//Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway, NJ: IEEE, 2012: 4277-4280.
[16] KALCHBRENNER N, GREFENSTETTE E, BLUNSOM P. A convolutional neural network for modelling sentences [J/OL]. Eprint Arxiv, arXiv:1404.2188, 2014 [2015-06--22]. http://arxiv.org/abs/1404.2188.
[17] KIM Y. Convolutional neural networks for sentence classification [J/OL]. Eprint Arxiv, arXiv:1408.5882, 2014 [2015-07-02]. http://arxiv.org/abs/1408.5882.
[18] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space [J/OL]. Eprint Arxiv, arXiv:1301.3781, 2013 [2015-06-13]. http://arxiv.org/abs/1301.3781.
[19] XU C, CETINTAS S, LEE K-C, et al. Visual sentiment prediction with deep convolutional neural networks [J/OL]. Eprint Arxiv, arXiv:1411.5731, 2014 [2015-06-13]. http://arxiv.org/abs/1411.5731.
[20] HINTON G E, SRIVASTAVA N, KRIZHEVSKY A, et al. Improving neural networks by preventing co-adaptation of feature detectors[J]. Eprint Arxiv, arXiv:1207.0580, 2012 [2015-06-13]. http://arxiv.org/abs/1207.0580.

基于卷积神经网络的图文融合媒体情感预测

Multimedia sentiment analysis based on convolutional neural network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	宋中山, 梁家锐, 郑禄, 刘振宇, 帖军. 基于双向门控尺度特征融合的遥感场景分类[J]. 计算机应用, 2021, 41(9): 2726-2735.
[2]	李康康, 张静. 基于注意力机制的多层次编码和解码的图像描述模型[J]. 计算机应用, 2021, 41(9): 2504-2509.
[3]	张永斌, 常文欣, 孙连山, 张航. 基于字典的域名生成算法生成域名的检测方法[J]. 计算机应用, 2021, 41(9): 2609-2614.
[4]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[5]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[6]	牟长宁, 王海鹏, 周丕宇, 侯鑫行. 基于图卷积神经网络的串联质谱从头测序[J]. 计算机应用, 2021, 41(9): 2773-2779.
[7]	王贺兵, 张春梅. 基于非对称卷积-压缩激发-次代残差网络的人脸关键点检测[J]. 计算机应用, 2021, 41(9): 2741-2747.
[8]	秦斌斌, 彭良康, 卢向明, 钱江波. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 41(8): 2330-2337.
[9]	黄程程, 董霄霄, 李钊. 基于二维Winograd算法的深流水线5×5卷积方法[J]. 计算机应用, 2021, 41(8): 2258-2264.
[10]	曾祥银, 郑伯川, 刘丹. 基于深度卷积神经网络和聚类的左右轨道线检测[J]. 计算机应用, 2021, 41(8): 2324-2329.
[11]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287.
[12]	吴则举, 焦翠娟, 陈亮. 基于改进Faster R-CNN的轮胎缺陷检测方法[J]. 计算机应用, 2021, 41(7): 1939-1946.
[13]	杨粟, 欧阳智, 杜逆索. 基于相关度距离的无监督并行哈希图像检索[J]. 计算机应用, 2021, 41(7): 1902-1907.
[14]	武光利, 李雷霆, 郭振洲, 王成祥. 基于改进的双向长短期记忆网络的视频摘要生成模型[J]. 计算机应用, 2021, 41(7): 1908-1914.
[15]	高钦泉, 黄炳城, 刘文哲, 童同. 基于改进CenterNet的竹条表面缺陷检测方法[J]. 计算机应用, 2021, 41(7): 1933-1938.