Tag recommendation method combining network structure information and text content

doi:10.11772/j.issn.1001-9081.2020081275

Abstract

Abstract: Recommending appropriate tags for texts is an effective way to better organize and use the text content. At present, most tag recommendation methods mainly recommend tags by mining the text content. However, most of the data information does not exist independently, for example, the co-occurrence of words between texts in a corpus can form a complex network structure. Previous studies have shown that the network structure information between texts and the text content information can summarize the semantics of the same text from two different perspectives, and the information extracted from two aspects can complement and explain each other. Based on this, a tag recommendation method was proposed to simultaneously model the network structure information of text and the content information of text. Firstly, Graph Convolutional neural Network(GCN) was used to extract the structure information of the network between texts, then Recurrent Neural Network(RNN) was used to extract the text content information, and finally the attention mechanism was used to recommend tags by combining the network structure information between texts and the text content information. Compared with baseline methods, such as tag recommendation method based on GCN and tag recommendation method with Topical attention-based Long Short-Term Memory(TLSTM) neural network, the proposed tag recommendation method with attention mechanism combining network structure information and text content information has better performance. For example, on the Mathematics Stack Exchange dataset, the precision, recall and F1 of the proposed method are improved by 2.3%, 3.8%, and 7.0% respectively compared with the optimal baseline method.

Key words: tag recommendation, Recurrent Neural Network (RNN), Graph Convolutional neural Network (GCN), attention mechanism, network structure information, text content

摘要： 为文本推荐合适的标签是更好地组织和使用文本内容的一项有效手段，目前大部分标签推荐方法主要通过挖掘文本内容来进行推荐。然而，大部分数据信息并非独立存在，如语料库中的文本间的词共现关系可形成复杂的网络结构。以往研究表明，文本间的网络结构信息和文本内容信息可以分别从两个不同的角度对同一文本的语义进行概括，并且从两方面提取的信息可以互为补充和解释。基于此，提出一种同时对文本网络结构信息和文本内容信息进行建模的标签推荐方法。该方法首先使用图卷积神经网络（GCN）提取文本间网络的结构信息，然后使用循环神经网络（RNN）提取文本内容信息，最后使用注意力机制结合文本间网络结构信息和文本内容信息进行标签的推荐。与基于图卷积神经网络（GCN）的标签推荐方法、基于主题注意力的长短时记忆（TLSTM）神经网络的标签推荐方法等基线方法相比，提出的使用注意力机制结合网络结构信息与文本内容信息的标签推荐方法具有更好的性能。如在Mathematics Stack Exchange数据集上所提方法的准确率、召回率和F1值相较最优基线方法分别提高了2.3%、3.8%、7.0%。

CLC Number:

TP391

CHE Bingqian, ZHOU Dong. Tag recommendation method combining network structure information and text content[J]. Journal of Computer Applications, 2021, 41(4): 976-983.

车冰倩, 周栋. 融合网络结构信息及文本内容的标签推荐方法[J]. 计算机应用, 2021, 41(4): 976-983.

References

[1] 王梦恬, 魏晶晶, 廖祥文, 等. 融合评论标签的个性化推荐算法[J]. 计算机科学与探索,2016,10(10):1429-1438.(WANG M T,WEI J J,LIAO X W,et al. Personalized recommendation algorithm fusing comment tag[J]. Journal of Frontiers of Computer Science and Technology,2016,10(10):1429-1438.)
[2] 孙松涛, 何炎祥. 基于CNN特征空间的微博多标签情感分类[J]. 工程科学与技术,2017,49(3):162-169.(SUN S T,HE Y X. Multi-label emotion classification for microblog based on CNN feature space[J]. Advanced Engineering Sciences,2017,49(3):162-169.)
[3] 吴磊, 岳峰, 王含茹, 等. 一种融合科研人员标签的学术论文推荐方法[J]. 计算机科学,2020,47(2):51-57.(WU L,YUE F, WANG H R, et al. Academic paper recommendation method combined with researcher tag[J]. Computer Science,2020,47(2):51-57.)
[4] BELÉM F M,ALMEIDA J M,GONÇALVES M A. A survey on tag recommendation methods[J]. Journal of the Association for Information Science and Technology,2017,68(4):830-844.
[5] PUJARI M, KANAWATI R. Tag recommendation by link prediction based on supervised machine learning[C]//Proceedings of the 6th International Conference on Weblogs and Social Media. Palo Alto,CA:AAAI Press,2012:547-550.
[6] MARTINS E F,BELÉM F M,ALMEIDA J M,et al. On cold start for associative tag recommendation[J]. Journal of the Association for Information Science and Technology,2016,67(1):83-105.
[7] KRESTEL R,FANKHAUSER P,NEJDL W. Latent Dirichlet allocation for tag recommendation[C]//Proceedings of the 3rd ACM Conference on Recommender Systems. New York:ACM,2009:61-68.
[8] RAMAGE D,HALL D,NALLAPATI R,et al. Labeled LDA:a supervised topic model for credit attribution in multi-labeled corpora[C]//Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. Stroudsburg,PA:Association for Computational Linguistics,2009,248-256.
[9] WU Y,YAO Y,XU F,et al. Tag2Word:using tags to generate words for content based tag recommendation[C]//Proceedings of the 25th ACM International Conference on Information and Knowledge Management. New York:ACM,2016:2287-2292.
[10] WU Y,XI S,YAO Y,et al. Guiding supervised topic modeling for content based tag recommendation[J]. Neurocomputing, 2018,314:479-489.
[11] LECUN Y,BENGIO Y,HINTON G. Deep learning[J]. Nature, 2015,521(7553):436-444.
[12] WESTON J, CHOPRA S, ADAMS K. #TagSpace:semantic embeddings from hashtags[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2014,1822-1827.
[13] ZHANG Q, WANG J, HUANG H, et al. Hashtag recommendation for multimodal microblog using co-attention network[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. Palo Alto, CA:AAAI Press,2017:3420-3426.
[14] GUO Z,ZHANG Z M,ZHU S,et al. A two-level topic model towards knowledge discovery from citation networks[J]. IEEE Transactions on Knowledge and Data Engineering,2014,26(4):780-794.
[15] WEST J D, WESLEY-SMITH I, BERGSTROM C T. A recommendation system based on hierarchical clustering of an article-level citation network[J]. IEEE Transactions on Big Data, 2016,2(2):113-123.
[16] KIPF T N,WELLING M. Semi-supervised classification with graph convolutional networks[EB/OL].[2020-05-20]. https://arxiv.org/pdf/1609.02907.pdf.
[17] SHI X,HUANG H,ZHAO S,et al. Tag recommendation by wordlevel tag sequence modeling[C]//Proceedings of the 2019 International Conference on Database Systems for Advanced Applications,LNCS 11448. Cham:Springer,2019:420-424.
[18] FENG W,WANG J. Incorporating heterogeneous information for personalized tag recommendation in social tagging systems[C]//Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM,2012:1276-1284.
[19] FANG X, PAN R, CAO G, et al. Personalized tag recommendation through nonlinear tensor factorization using Gaussian kernel[C]//Proceedings of the 29th AAAI Conference on Artificial Intelligence. Palo Alto, CA:AAAI Press, 2015:439-445
[20] ZHAO W, GUAN Z Y, LIU Z. Ranking on heterogeneous manifolds for tag recommendation in social tagging services[J]. Neurocomputing,2015,148:521-534.
[21] SI X,SUN M. Tag-LDA for scalable real-time tag recommendation[J]. Journal of Information and Computational Systems,2009,6(2):1009-1016.
[22] DING Z,QIU X,ZHANG Q,et al. Learning topical translation model for microblog hashtag suggestion[C]//Proceedings of the 23rd International Joint Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2013:2078-2084.
[23] GODIN F,SLAVKOVIKJ V,DE NEVE W,et al. Using topic models for Twitter hashtag recommendation[C]//Proceedings of the 22nd International Conference on World Wide Web s Steering Committee. New York:ACM,2013:593-596.
[24] GONG Y,ZHANG Q. Hashtag recommendation using attentionbased convolutional neural network[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2016:2782-2788.
[25] LI Y,LIU T,JIANG J,et al. Hashtag recommendation with topical attention-based LSTM[C]//Proceedings of the 26th International Conference on Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2016:3019-3029.
[26] HUANG H, ZHANG Q, GONG Y, et al. Hashtag recommendation using end-to-end memory networks with hierarchical attention[C]//Proceedings of the 26th International Conference on Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2016:943-952.
[27] LI J,XU H,HE X,et al. Tweet modeling with LSTM recurrent neural networks for hashtag recommendation[C]//Proceedings of the 2016 International Joint Conference on Neural Networks. Piscataway:IEEE,2016:1570-1577.
[28] TANG S, YAO Y, ZHANG S, et al. An integral tag recommendation model for textual content[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2019:5109-5116.
[29] SHI M, TANG Y, LIU J. TA-BLSTM:tag attention-based bidirectional long short-term memory for service recommendation in mashup creation[C]//Proceedings of the 2019 International Joint Conference on Neural Networks. Piscataway:IEEE,2019:1-8.
[30] YAO L,MAO C,LUO Y. Graph convolutional networks for text classification[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2019:7370-7377.
[31] BENGIO Y, DUCHARME R, VINCENT P, et al. A neural probabilistic language model[J]. Journal of Machine Learning Research,2003,3:1137-1155.
[32] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. Red Hook, NY:Curran Associates Inc.,2013:3111-3119.
[33] HOCHREITER S,SCHMIDHUBER J. Long short-term memory[J]. Neural Computation,1997,9(8):1735-1780.
[34] CHUNG J,GULCEHRE C,CHO K,et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[EB/OL].[2020-06-03]. https://arxiv.org/pdf/1412.3555.pdf.
[35] XU M,JIN R,ZHOU Z. Speedup matrix completion with side information:application to multi-label learning[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. Red Hook,NY:Curran Associates Inc., 2013:2301-2309.
[36] KINGMA D P, BA J L. Adam:a method for stochastic optimization[EB/OL].[2020-06-03]. https://arxiv.org/pdf/1412.6980.pdf.