Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (2): 307-317.DOI: 10.11772/j.issn.1001-9081.2020060923

Special Issue: 人工智能 综述

• Artificial intelligence •     Next Articles

Survey of sentiment analysis based on image and text fusion

MENG Xiangrui1, YANG Wenzhong1, WANG Ting2   

  1. 1. College of Information Science and Engineering, Xinjiang University, Urumqi Xinjiang 830046, China;
    2. School of Software, Xinjiang University, Urumqi Xinjiang 830046, China
  • Received:2020-06-30 Revised:2020-10-03 Online:2020-12-18 Published:2021-02-10
  • Supported by:
    This work is partially supported by the Key Project of Natural Science Foundation of China (U1435215), the National Natural Science Foundation of China (U1603115), the National Key Research and Development Program of China (2017YFC0820702-3), the Sichuan Science and Technology Program (WA2018-YY007).


孟祥瑞1, 杨文忠1, 王婷2   

  1. 1. 新疆大学 信息科学与工程学院, 乌鲁木齐 830046;
    2. 新疆大学 软件学院, 乌鲁木齐 830046
  • 通讯作者: 杨文忠
  • 作者简介:孟祥瑞(1996-),女,吉林长春人,硕士研究生,主要研究方向:自然语言处理、情感分析;杨文忠(1971-),男,河南南阳人,副教授,博士,CCF会员,主要研究方向:网络舆情、情报分析、信息安全、无线传感器网络;王婷(1996-),女,新疆阿克苏人,硕士研究生,主要研究方向:自然语言处理、文本情感分析。
  • 基金资助:

Abstract: With the continuous improvement of information technology, the amount of image-text data with orientation on various social platforms is growing rapidly, and the sentiment analysis with image and text fusion is widely concerned. The single sentiment analysis method can no longer meet the demand of multi-modal data. Aiming at the technical problems of image and text sentiment feature extraction and fusion, firstly, the widely used image and text emotional analysis datasets were listed, and the extraction methods of text features and image features were introduced. Then, the current fusion modes of image features and text features were focused on and the problems existing in the process of image-text sentiment analysis were briefly described. Finally, the research directions of sentiment analysis in the future were summarized and prospected for. In order to have a deeper understanding of image-text fusion technology, literature research method was adopted to review the study of image-text sentiment analysis, which is helpful to compare the differences between different fusion methods and find more valuable research schemes.

Key words: image and text fusion, sentiment analysis, feature fusion, machine learning, social media

摘要: 随着信息化技术的不断提升,各类社交平台上带有倾向性的图文数据量快速增长,图文融合的情感分析受到广泛关注,单一的情感分析方法不再能够满足多模态数据的需求。针对图文情感特征提取与融合的技术难题,首先,列举了目前应用较广的图文情感分析数据集,介绍了文本特征和图片特征的提取方式;然后,重点研究了当前图文特征融合方式,简述了在图文情感分析过程中存在的问题;最后,针对未来情感分析的研究方向进行了总结与展望。为深入了解图文融合技术,采用文献调研方法对图文情感分析的研究进行综述,有助于比较不同融合方法之间的区别,发现更具价值的研究方案。

关键词: 图文融合, 情感分析, 特征融合, 机器学习, 社交媒体

CLC Number: