Multi-intention recognition model with combination of syntactic feature and convolution neural network
YANG Chunni1, FENG Chaosheng1,2
1. School of Computer Science, Sichuan Normal University, Chengdu Sichuan 610101, China; 2. School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu Sichuan 610054, China
Abstract：Multi-Intention (MI) recognition of short texts is a problem in Spoken Language Understanding (SLU). The effective features of short texts are difficult to extract in classification problems because of sparse features of short texts and few words containing many information. To solve the problem, by combining syntactic features and Convolution Neural Network (CNN), a multi-intention recognition model was proposed. Firstly, the sentence was syntactically analyzed to determine whether it contains multi-intention. Secondly, the number of intentions and matrix of distance were calculated by using Term Frequency-Inverse Document Frequency (TF-IDF) and word embedding. Then the matrix of distance was acted as the input of CNN model to classify intentions. Finally, the emotional polarity of each intention was judged to return to the user's true intentions. The experiment was carried out by using the real data of the existing intelligent customer service system. The experimental results show that, the single classification precision of the combination model of syntactic features and CNN is 93.5% in 10 intentions, which is 1.4 percentage points higher than the original CNN model without syntactic features. And in mutil-intention recognition, the classification precision is 30 percentage points higher than others.
杨春妮, 冯朝胜. 结合句法特征和卷积神经网络的多意图识别模型[J]. 计算机应用, 2018, 38(7): 1839-1845.
YANG Chunni, FENG Chaosheng. Multi-intention recognition model with combination of syntactic feature and convolution neural network. Journal of Computer Applications, 2018, 38(7): 1839-1845.
[1] AGGARWAL C C, ZHAIC X. A survey of text classification algorithms[M]//Mining Text Data. Berlin:Springer, 2012:163-222. [2] KIM Y. Convolutional neural networks for sentence classification[C]//EMNLP 2014:Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2014:1746-1751. [3] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//NIPS'13:Proceedings of the 26th International Conference on Neural Information Processing Systems. West Chester, OH:Curran Associates Inc., 2013:3111-3119. [4] RAVUNI S, STOLCKE A. Recurrent neural network and LSTM models for lexical utterance classification[C]//ISCA2015:Proceedings of the 42nd International Symposium on Computer Architecture. Piscataway, NJ:IEEE, 2015:135-139. [5] LAI S W, XU L H, LIU K, et al. Recurrent convolutional neural networks for text classification[C]//Proceedings of the 2015 Twenty-Ninth AAAI Conference on Artificial Intelligence. Menlo Park, CA:AAAI, 2015:2267-2273. [6] ZHOU C T, SUN C L, LIU Z Y, et al. A C-LSTM neural network for text classification[EB/OL].[2017-11-12]. https://arxiv.org/pdf/1511.08630.pdf. [7] JOULIN A, GRAVE E, BOJANOWSKI P, et al. Bag of tricks for efficient text classification[C]//Proceedings of the 2017 Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2017:427-431. [8] 孟欣,左万利.基于word embedding的短文本特征扩展与分类[J].小型微型计算机系统,2017,38(8):1712-1717.(MENG X, ZUO W L. Short text expansion and classification based on word embedding[J]. Journal of Chinese Computer Systems, 2017, 38(8):1712-1717.) [9] 张猛.基于LDA的短文本分类中特征扩展方法的研究[D].北京:中国地质大学,2017.(ZHANG M. Feature extension method for short-text classification based on LDA[D]. Beijing:China University of Geosciences, 2017.) [10] 余本功,张连彬.基于CP-CNN的中文短文本分类研究[J].计算机应用研究,2018,35(4):1001-1004.(YU B G, ZHANG L B. Chinese short text classification based on CP-CNN[J]. Application Research of Computers, 2018, 35(4):1001-1004.) [11] 卢玲,杨武,杨有俊,等.结合语义扩展和卷积神经网络的中文短文本分类方法[J].计算机应用,2017,37(12):3498-3503.(LU L, YANG W, YANG Y J, et al. Chinese short text calssification method by combining semantic expansion and convolutional neural network[J]. Journal of Computer Applications, 2017, 37(12):3498-3503.) [12] MADJAROV G, KOCEV D, GJORGJVIKJ D, et al. An extensive experimental comparison of methods for multi-label learning[J]. Pattern Recognition, 2012, 45(9):3084-3104. [13] BOUTELL M R, LUO J, SHEN X, et al. Learning multi-label scene classification[J]. Pattern Recognition, 2004, 37(9):1757-1771. [14] BRINKER K. Multilabel classification via calibrated label ranking[J]. Machine Learning, 2008, 73(2):133-153. [15] TSOUMAKAS G, VLANHAVAS I. Random k-labelsets:an ensemble method for multilabel classification[C]//ECML 2007:Proceedings of the 18th European Conference on Machine Learning. Berlin:Springer, 2007:406-417. [16] ZHANG M L, ZHOU Z H. ML-KNN:a lazy learning approach to multi-label learning[J]. Pattern Recognition, 2007, 40(7):2038-2048. [17] ELISSEEFF A, WESTON J. A kernel method for multi-labelled classification[C]//NIPS'01:Proceedings of the 14th International Conference on Neural Information Processing Systems:Natural and Synthetic. Cambridge, MA:MIT Press, 2001:681-687. [18] ZHANG M L, ZHANG K. Multi-label learning by exploiting label dependency[C]//Proceedings of the 2010 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM, 2010:999-1008. [19] WU F, WANG Z, ZHANG Z, et al. Weakly semi-supervised deep learning for multi-label image annotation[J]. IEEE Transactions on Big Data, 2017, 1(3):109-122. [20] CHEN G, YE D, XING Z, et al. Ensemble application of convolutional and recurrent neural networks for multi-label text categorization[C]//Proceedings of the 2017 International Joint Conference on Neural Networks. Piscataway, NJ:IEEE, 2017:2377-2383. [21] 韩栋,王春华,肖敏.结合旋转森林和AdaBoost分类器的多标签文本分类方法[J/OL].计算机应用研究,2018,35(12)[2017-12-08]. http://www.arocmag.com/article/02-2018-12-014.html.(HAN D, WANG C H, XIAO M. Multi-label text classification method based on rotating forest and AdaBoost classifier[J/OL]. Application Research of Computers, 2018, 35(12)[2017-12-08]. http://www.arocmag.com/article/02-2018-12-014.html.) [22] 张晶,李裕,李培培.基于随机子空间的多标签类属特征提取算法[J/OL].计算机应用研究,2019,36(2) (2017-05-12)[2018-01-19]. http://www.arocmag.com/article/02-2019-02-012.html.(ZHANG J, LI Y, LI P P. Multi-label label-specific feature extraction algorithm based on random subspace[J/OL]. Application Research of Computers, 2019, 36(2) (2017-05-12)[2018-01-19]. http://www.arocmag.com/article/02-2019-02-012.html.) [23] 孙松涛,何炎祥.基于CNN特征空间的微博多标签情感分类[J].工程科学与技术,2017,49(3):162-169.(SUN S T, HE Y X. Multi-label emotion classification for microblog based on CNN feature space[J]. Advanced Engineering Science, 2017, 49(3):162-169.) [24] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout:a simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research, 2014, 15(1):1929-1958. [25] VORONSOV M A, SIVOKON V. Stochastic parallel-gradient-descent technique for high-resolution wave-front phase-distortion correction[J]. Journal of the Optical Society of America A, 1998, 15(10):2745-2758.
2018, Vol. 38 
