基于微博文本的词对主题演化模型

doi:10.11772/j.issn.1001-9081.2017.05.1407

摘要/Abstract

摘要： 针对传统主题模型忽略了微博短文本和文本动态演化的问题，提出了基于微博文本的词对主题演化（BToT）模型，并根据所提模型对数据集进行主题演化分析。BToT模型在文本生成过程中引入连续的时间变量具体描述时间维度上的主题动态演化，同时在文档中构成主题共享的“词对”结构，扩充了短文本特征。采用Gibbs采样方法对BToT参数进行估计，根据获得的主题-时间分布参数对主题进行演化分析。在真实微博数据集上进行验证，结果表明，BToT模型可以描述微博数据集中潜在的主题演化规律，获得的困惑度评价系数低于潜在狄利克雷分配（LDA）、词对主题模型（BTM）和主题演化模型（ToT）。

关键词: 特征值稀疏, 主题演化模型, 动态演化, Gibbs采样, 微博

Abstract: Aiming at the problem that the traditional topic model ignore short text and dynamic evolution of microblog, a Biterm Topic over Time (BToT) model based on microblog text was proposed, and the subject evolution analysis was carried out by the proposed model. A continuous time variable was introduced to describe the dynamic evolution of the topic in the time dimension during the process of text generation in the BToT model, and the "Biterm" structure of the topic sharing in the document was formed to extend short text feature. The Gibbs sampling method was used to estimate the parameters of BToT, and the topic evaluation was analyzed by topic-time distributed parameters. The experimental results on real microblog datasets show that BToT can characterize the latent topic evolution and has lower perplexity than Latent Dirichlet Allocation (LDA), Biterm Topic Model (BTM) and Topic over Time (ToT).

Key words: feature sparsity, theme evolution model, dynamic evolution, Gibbs sampling, microblog

中图分类号:

TP391.1

史庆伟, 刘雨诗, 张丰田. 基于微博文本的词对主题演化模型[J]. 计算机应用, 2017, 37(5): 1407-1412.

SHI Qingwei, LIU Yushi, ZHANG Fengtian. Biterm topic evolution model of microblog[J]. Journal of Computer Applications, 2017, 37(5): 1407-1412.

参考文献

[1] 张剑锋,夏云庆,姚建民.微博文本处理研究综述[J].中文信息学报,2012,26(4):21-27.(ZHANG J F, XIA Y Q, YAO J M. A review towards microtext processing[J]. Journal of Chinese Information Processing, 2012,26(4):21-27.)
[2] 吕超镇,姬东鸿,吴飞飞. 基于LDA特征扩展的短文本分类[J].计算机工程与应用, 2015, 51(4):123-127.(LYU C Z, JI D H, WU F F. Short text classification based on expanding feature of LDA[J]. Computer Engineering and Applications, 2015, 51(4):123-127.)
[3] 张晨逸,孙建伶,丁轶群. 基于MB-LDA模型的微博主题挖掘[J].计算机研究与发展, 2011, 48(10):1795-1802. (ZHANG C Y, SUN J L, DING Y Q. Topic mining for microblog based on MB-LDA model[J]. Journal of Computer Research and Development, 2011, 48(10):1795-1802.)
[4] 唐晓波,向坤. 基于LDA模型和微博热度的热点挖掘[J].图书情报工作, 2014, 58(5):58-63.(TANG X B,XIANG K. Hotspot mining based on LDA model and microblog heat[J]. Library and Information Service, 2014, 58(5):58-63.)
[5] XU T,OARD D W. Wikipedia-based topic clustering for microblogs[J].Proceedings of the American Society for Information Science and Technology,2011,48(1):1-10.
[6] YAN X, GUO J, LAN Y, et al. A biterm topic model for short texts[C]//Proceedings of the 22nd International Conference on World Wide Web. New York:ACM, 2013:1445-1456.
[7] BLEI D, LAFFERTY J. Dynamic topic model[C]//Proceedings of the 23rd ICML International Conference on Machine Learning, New York:ACM, 2006:113-120.
[8] WANG C, BLEI D, HECKERMAN D. Continuous time dynamic topic models[C]//Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence. Corvallis:AUAI Press, 2008:579-586.
[9] 刘晓鸣. 社区问答系统中的专家发现方法研究[D].大连:大连理工大学,2013.(LIU X P. Finding experts in community question answering[D]. Dalian:Dalian University of Technology, 2013.)
[10] 马海平. 基于概率生成模型的相似度建模技术研究及应用[D].合肥:中国科学技术大学,2013.(MA H P. A study of probability generation model based similarity modeling techniques and its applications[D]. Hefei:University of Science and Technology of China, 2013.)
[11] 罗远胜. 跨语言信息检索中双语主题模型及算法研究[D].南昌:江西财经大学,2013.(LUO Y S. Research on bilingual topic model and its algorithm in cross-language information retrieval[D]. Nanchang:Jiangxi University of Finance and Economics, 2013.)
[12] WANG X, MCCALLUM A. Topics over time:a non-Markov continuous-time model of topical trends[C]//Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM, 2006:424-433.
[13] OWEN C B. Parameter estimation for the beta distribution[D]. Provo:Brigham Young University, 2008.
[14] 刘书奎,吴子燕,张玉兵.基于Gibbs抽样的马尔科夫蒙特卡罗方法在结构物理参数识别及损伤定位中的研究[J]. 振动与冲击, 2011, 30(10):203-207.(LIU S K, WU Z Y, ZHANG Y B. Identification of physical parameters and damage locating with Markov chain Monte Carlo method based on Gibbs sampling[J]. Journal of Vibration and Shock, 2011, 30(10):203-207.)
[15] 马跃渊,徐勇勇.Gibbs抽样算法及软件设计的初步研究[J]. 计算机应用与软件, 2005, 22(2):124-126.(MA Y Y, XU Y Y. An initial study on the algorithm and the software of Gibbs sampling[J]. Computer Applications and Software, 2005, 22(2):124-126.)
[16] 张小平,周雪忠,黄厚宽,等.一种改进的LDA主题模型[J]. 北京交通大学学报, 2010, 34(2):111-114.(ZHANG X P, ZHOU X Z, HUANG H K, et al. An improved LDA topic model[J]. Journal of Beijing Jiaotong University, 2010, 34(2):111-114.)
[17] 荀静, 刘培玉, 杨玉珍,等. 基于潜在狄利克雷分布模型的多文档情感摘要[J]. 计算机应用, 2014, 34(6):1636-1640.(XUN J, LIU P Y, YANG Y Z, et al. Multi-document sentiment summarization based on latent Dirichlet allocation model[J]. Joutnal of Computers Applications,2014,34(6):1636-1364.)
[18] 徐戈,王厚峰.自然语言处理中主题模型的发展[J].计算机学报,2011, 34(8):1423-1436.(XU G, WANG H F. The development of topic models in natural language processing[J]. Chinese Journal of Computers, 2014, 34(6):1636-1364.)
[19] HAN X, STIBOR T. Efficient collapsed Gibbs sampling for latent Dirichlet allocation[J].Journal of Machine Learning Research, 2010, 13:63-78.
[20] HOFFMAN M D, BLEI D M,WANG C, et al. Stochastic variational inference[J]. Journal of Machine Learning Research, 2013, 14(1):1303-1347.
[21] HEINRICH G. Parameter estimation for text analysis[EB/OL].[2013-04-25].http://www.Arbylon.net/publications/textest2.pdf.
[22] 汲剑锐.马尔科夫链应用的一些探讨[D].武汉:华中师范大学, 2012.(JI J R. The discussion about the applications of Markov chains[D]. Wuhan:Central China Normal University, 2012.)
[23] 石晶,范猛,李万龙.基于LDA模型的主题分析[J].自动化学报,2009,35(12):1586-1593.(SHI J, FAN M, LI W L. Text segmentation based on model LDA[J]. Acta Automatica Sinica, 2009,35(12):1586-1593.)
[24] 史庆伟, 李艳妮, 郭朋亮.科技文献中作者研究兴趣动态发现[J].计算机应用, 2013, 33(11):3080-3083.(SHI Q W, LI Y N, GUO P L. Dynamic finding of authors' research interests in scientific literature[J]. Journal of Computer Applications, 2013, 33(11):3080-3083.)
[25] 曹娟,张勇东,李锦涛,等.一种基于密度的自适应最优LDA模型选择方法[J]. 计算机学报, 2008,31(10):1780-1787.(CAO J, ZHANG Y D, LI J T, et al. A method of adaptively selecting best LDA model based on density[J]. Chinese Journal of Computers, 2008, 31(10):1780-1787.) SHI Qingwei, born in 1973, Ph. D., associate professor. His research interests include intelligent information processing.LIU Yushi, born in 1993, M. S. candidate. Her research interests include intelligent information processing.ZHANG Fengtian, born in 1991, M. S. candidate. His research interests include big data, cloud computing.