基于改进的多层BLSTM的中文分词和标点预测

doi:10.11772/j.issn.1001-9081.2017112631

计算机应用 ›› 2018, Vol. 38 ›› Issue (5): 1278-1282.DOI: 10.11772/j.issn.1001-9081.2017112631

基于改进的多层BLSTM的中文分词和标点预测

李雅昆, 潘晴, Everett X. WANG

广东工业大学信息工程学院, 广州 510006

收稿日期:2017-11-06 修回日期:2017-12-01 发布日期:2018-05-24 出版日期:2018-05-10
通讯作者: 潘晴
作者简介:李雅昆(1989-),男,河南新乡人,硕士研究生,主要研究方向:深度学习、自然语言处理;潘晴(1975-),男,江苏宜兴人,教授,博士,主要研究方向:图像处理、机器学习;Everett X.WANG (1961-),男,美籍华人,教授,博士,主要研究方向:卫星导航、最优控制和复杂系统的动态建模、机器学习。

Joint Chinese word segmentation and punctuation prediction based on improved multilayer BLSTM network

LI Yakun, PAN Qing, WANG Feng

School of Information Engineering, Guangdong University of Technology, Guangzhou Guangdong 510006, China

Received:2017-11-06 Revised:2017-12-01 Online:2018-05-24 Published:2018-05-10
Contact: 潘晴

摘要/Abstract

摘要： 目前主流的序列标注问题是基于循环神经网络（RNN）实现的。针对RNN和序列标注问题进行研究，提出了一种改进型的多层双向长短时记忆（BLSTM）网络，该网络每层的BLSTM都有一次信息融合，输出包含更多的上下文信息。另外找到一种基于序列标注的可以并行执行中文分词和标点预测的联合任务方法。在公开的数据集上的实验结果表明，所提出的改进型的多层BLSTM网络模型性能优越，提升了中文分词和标点预测的分类精度；在需要完成中文分词和标点预测两项任务时，联合任务方法能够大幅地降低系统复杂度；新的模型及基于该模型的联合任务方法也可应用到其他序列标注任务中。

关键词: 中文分词, 标点预测, 序列标注, 双向长短时记忆网络

Abstract: The current mainstream sequence labeling is based on Recurrent Neural Network (RNN). Aiming at the problem of RNN and sequence labeling, an improved multilayer Bi-direction Long Short Term Memory (BLSTM) network for sequence labeling was proposed. Each layer of BLSTM had an operation of information fusion, and the output contained more contextual information. In addition, a method to perform Chinese word segmentation and punctuation prediction jointly was proposed. Experiments on the public datasets show that the improved multilayer BLSTM network model can improve the classification accuracy of Chinese segmentation and punctuation prediction. In the case of two tasks that need to be accomplished, the joint task method can greatly reduce the complexity of the system, and the new model and the joint task method can also be applied to solve other sequence labeling problems.

Key words: Chinese word segmentation, punctuation prediction, sequence labeling, Bi-directional Long Short Term Memory (BLSTM) network

中图分类号:

TP391.1

李雅昆, 潘晴, Everett X. WANG. 基于改进的多层BLSTM的中文分词和标点预测[J]. 计算机应用, 2018, 38(5): 1278-1282.

LI Yakun, PAN Qing, WANG Feng. Joint Chinese word segmentation and punctuation prediction based on improved multilayer BLSTM network[J]. Journal of Computer Applications, 2018, 38(5): 1278-1282.

参考文献

[1] XUE N, CONVERSE S P. Combining classifiers for Chinese word segmentation[C]//Proceedings of the 1st SIGHAN Workshop on Chinese Language. Stroudsburg, PA:Association for Computational Linguistics, 2002:57-63.
[2] 翟凤文, 赫枫龄, 左万利. 字典与统计相结合的中文分词方法[J]. 小型微型计算机系统, 2006, 27(9):1766-1771.(ZHAI F W, HE F L, ZUO W L. Chinese word segmentation based on dictionary and statistics[J]. MNI-MICRO Systems, 2006, 27(9):1766-1771.)
[3] KNESER R, NEY H. Improved backing-off for N-gram language modeling[C]//Proceedings of the 1995 IEEE International Conference on Acoustics, Speech and Signal Processing. Washington, DC:IEEE Computer Society, 1995:181-184.
[4] GOLDBERG Y, LEVY O. word2vec explained:deriving Mikolov et al.'s negative-sampling word-embedding method[EB/OL].[2017-06-20]. http://www.inf.ed.ac.uk/teaching/courses/nlu/reading/skipgram-derivation.pdf.
[5] 万建成, 杨春花. 书面汉语的全切分分词算法模型[J]. 小型微型计算机系统, 2003, 24(7):1247-1251.(WAN J C, YANG C H. An algorithm model of word omni-segmentation for written Chinese[J]. MNI-MICRO Systems, 2003, 24(7):1247-1251.)
[6] ZHAO H, HUANG C N, LI M, et al. An improved Chinese word segmentation system with conditional random field[EB/OL].[2017-06-20].http://acl.ldc.upenn.edu/W/W06/W06-0127.pdf.
[7] 任智慧, 徐浩煜, 封松林,等. 基于LSTM网络的序列标注中文分词法[J]. 计算机应用研究, 2017, 34(5):1321-1324.(REN Z H,XU H Y,FENG S L, et al. Sequence labeling Chinese word segmentation method based on LSTM networks[J]. Application Research of Computers, 2017, 34(5):1321-1324.)
[8] RATNAPARKHI A. A maximum entropy part-of-speech tagger[C]//Proceedings of the 1996 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 1996:133-142.
[9] LAFFERTY J D, MCCALLUM A, PEREIRA F C N. Conditional random fields:probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning. San Francisco, CA:Morgan Kaufmann, 2001:282-289.
[10] PENG F, FENG F, MCCALLUM A. Chinese segmentation and new word detection using conditional random fields[C]//Proceedings of the 20th International Conference on Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2004:562-568.
[11] ZHENG X, CHEN H, XU T. Deep learning for Chinese word segmentation and POS tagging[C]//Proceedings of the 2013 Conference on Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2013:647-657.
[12] PEI W, GE T, CHANG B. Max-margin tensor neural network for Chinese word segmentation[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2014:293-303.
[13] GERS F A, SCHMIDHUBER J. Recurrent nets that time and count[C]//Proceedings of the 2000 IEEE-INNS-ENNS International Joint Conference on Neural Networks. Washington, DC:IEEE Computer Society, 2000:189-194.
[14] CHO K, MERRIENBOER B V, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[EB/OL].[2017-06-20]. http://www.statnlp.org/wp-content/uploads/2016/02/rnn.pdf.
[15] CHEN X, QIU X, ZHU C, et al. Long short-term memory neural networks for Chinese word segmentation[EB/OL].[2017-06-20]. http://www.emnlp2015.org/proceedings/EMNLP/pdf/EMNLP141.pdf.
[16] GREFF K, SRIVASTAVA R K, KOUTNIK J, et al. LSTM:a search space odyssey[J]. IEEE Transactions on Neural Networks & Learning Systems, 2015, 28(10):2222-2232.
[17] YAO Y, HUANG Z. Bi-directional LSTM recurrent neural network for Chinese word segmentation[C]//Proceedings of the 23rd International Conference on Neural Information Processing. Berlin:Springer, 2016:345-353.
[18] 胡婕, 张俊驰. 双向循环网络中文分词模型[J]. 小型微型计算机系统, 2017, 38(3):522-526.(HU J, ZHANG J C. Bidirectional recurrent networks for Chinese word segmentation[J]. Journal of Chinese Computer Systems, 2017, 38(3):522-526.)
[19] 黄积杨. 基于双向LSTMN神经网络的中文分词研究分析[D]. 南京:南京大学, 2016.(HUANG J Y. Chinese word segmentation analysis based on bidirectional LSTM recurrent neural network[D]. Nanjing:Nanjing University, 2016.)
[20] JING H, ZWEIG G. Maximum entropy model for punctuation annotation from speech[C]//Proceedings of the 7th International Conference on Spoken Language Processing.[S.l.]:DBLP, 2002:917-920.
[21] SHRIBERG E, SHRIBERG E, SHRIBERG E, et al. Using conditional random fields for sentence boundary detection in speech[C]//Proceedings of the 43rd Annual Meeting on Association for Computational Linguistic. Stroudsburg, PA:Association for Computational Linguistics, 2005:451-458.
[22] ZHANG Y, CLARK S. Joint word segmentation and POS tagging using a single perceptron[C]//Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2008:888-896.
[23] WANG A, KAN M Y. Mining informal language from Chinese microtext:joint word recognition and segmentation[EB/OL].[2017-06-20]. http://www.comp.nus.edu.sg/~kanmy/papers/acl2013.pdf.
[24] WU K, WANG X, ZHOU N, et al. Joint Chinese word segmentation and punctuation prediction using deep recurrent neural network for social media data[C]//Proceedings of the 2015 International Conference on Asian Language Processing. Piscataway, NJ:IEEE, 2016:41-44.
[25] QIAN X, LIU Y. Joint Chinese word segnentation, POS tagging and parsing[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Stroudsburg, PA:Association for Computational Linguistics, 2012:501-511.
[26] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
[27] SIGHAN. Second International Chinese word segmentation Bakeoff data[EB/OL].[2005-11-18].http://sighan.cs.uchicago.edu/bakeoff2005/
[28] 并行执行中文分词和标点预测的Python程序[CP/OL].[2017-05-06].https://github.com/camel2000.(Python programs that perform Chinese word segmentation and punctuation prediction in parallel[CP/OL].[2017-05-06].https://github.com/camel2000.)
[29] SIGHAN.Second International Chinese Word Segmentation Bakeoff Result Summary[EB/OL].[2005-11-18].http://sighan.cs.uchicago.edu/bakeoff2005/data/results.php.htm.

基于改进的多层BLSTM的中文分词和标点预测

Joint Chinese word segmentation and punctuation prediction based on improved multilayer BLSTM network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	夏飞, 陈帅琦, 华珉, 蒋碧鸿. 基于改进BERT的电力领域中文分词方法[J]. 《计算机应用》唯一官方网站, 2023, 43(12): 3711-3718.
[2]	左亚尧, 陈皓宇, 陈致然, 洪嘉伟, 陈坤. 融合多语义特征的命名实体识别方法[J]. 《计算机应用》唯一官方网站, 2022, 42(7): 2001-2008.
[3]	李昕, 贾韬. 基于组蛋白修饰数据预测基因差异性表达的深度融合模型[J]. 《计算机应用》唯一官方网站, 2022, 42(11): 3404-3412.
[4]	赵小虎, 李晓. 基于多特征提取的图像语义描述算法[J]. 计算机应用, 2021, 41(6): 1640-1646.
[5]	邱宁佳, 王晓霞, 王鹏, 王艳春. 融合语法规则的双通道中文情感模型分析[J]. 计算机应用, 2021, 41(2): 318-323.
[6]	李灿, 杨雅婷, 马玉鹏, 董瑞. 基于语种相似性挖掘的神经机器翻译语料库扩充方法[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3145-3150.
[7]	吴赛赛, 梁晓贺, 谢能付, 周爱莲, 郝心宁. 面向领域实体关系联合抽取的标注方法[J]. 计算机应用, 2021, 41(10): 2858-2863.
[8]	董永峰, 刘超, 王利琴, 李英双. 融合多跳关系路径信息的关系推理方法[J]. 计算机应用, 2021, 41(10): 2799-2805.
[9]	陈佳沣, 滕冲. 基于强化学习的实体关系联合抽取模型[J]. 计算机应用, 2019, 39(7): 1918-1924.
[10]	闻畅, 刘宇, 顾进广. 基于注意力机制的双向长短时记忆网络模型突发事件演化关系抽取[J]. 计算机应用, 2019, 39(6): 1646-1651.
[11]	严红, 陈兴蜀, 王文贤, 王海舟, 殷明勇. 基于深度神经网络的法语命名实体识别模型[J]. 计算机应用, 2019, 39(5): 1288-1292.
[12]	张晨, 钱涛, 姬东鸿. 基于神经网络的微博情绪识别与诱因抽取联合模型[J]. 计算机应用, 2018, 38(9): 2464-2468.
[13]	黄念娥, 黄河, 王儒敬. 本体与条件随机场结合的涉农商品名称抽取与类别标注[J]. 计算机应用, 2017, 37(1): 233-238.
[14]	刘春丽, 李晓戈, 刘睿, 范贤, 杜丽萍. 基于表示学习的中文分词[J]. 计算机应用, 2016, 36(10): 2794-2798.
[15]	张聪品赵理莉吴长茂. 基于字词分类的层次分词方法研究[J]. 计算机应用, 2010, 30(8): 2034-2037.