1. The National Joint Engineering Laboratory of Internet Applied Technology of Mines(China University of Mining and Technology), Xuzhou Jiangsu 221008, China; 2. School of Information and Control Engineering, China University of Mining and Technology, Xuzhou Jiangsu 221008, China; 3. IoT Perception Mine Research Center, China University of Mining and Technology, Xuzhou Jiangsu 221008, China
Abstract：In view of the problems of term nesting, polysemy and error propagation between extraction subtask tasks, a deep attention model framework was proposed. First, the annotation strategy was used to jointly learn two sub tasks of knowledge extraction for solving the problem of error propagation. Second, a projection method combining multiple word vector information was proposed to alleviate the polysemy problem in term extraction in coal mine field. Third, a deep feature extraction network was designed, and a deep attention model and two model enhancement schemes were proposed to fully extract the semantic information. Finally, the classification layer of the model was analyzed to simplify the model to the maximum extent under the premise of ensuring the extraction effect. Experimental results show that, compared with the best model of coding-decoding structure, the proposed model has the F1-score increased by 1.5 percentage points and the model training speed improved by nearly 3 times. The proposed model can effectively complete two knowledge extraction subtasks which are term extraction and term relationship extraction in coal mine field.
[1] 张海楠,伍大勇,刘悦,等. 基于深度神经网络的中文命名实体识别[J]. 中文信息学报, 2017, 31(4):28-35. ZHANG H N, WU D Y, LIU Y, et al. Chinese Named Entity Recognition based on deep neural network[J]. Journal of Chinese Information Processing, 2017, 31(4):28-35. [2] BIKEL D M, SCHWARTZ R, WEISCHEDEL R M. An algorithm that learns what's in a name[J]. Machine Learning, 1999, 34(1/2/3):211-231. [3] 张玥杰,徐智婷,薛向阳. 融合多特征的最大熵汉语命名实体识别模型[J]. 计算机研究与发展, 2008, 45(6):1004-1010. (ZHANG Y J, XU Z T, XUE X Y. Fusion of multiple features for Chinese named entity recognition based on maximum entropy model[J]. Journal of Computer Research and Development, 2008, 45(6):1004-1010.) [4] ARTALEJO J R, LOPEZ-HERRERO M J. The SIS and SIR stochastic epidemic models:a maximum entropy approach[J]. Theoretical Population Biology, 2011, 80(4):256-264. [5] SONG S, ZHANG N, HUANG H. Named entity recognition based on conditional random fields[J]. Cluster Computing, 2019, 22(S3):5195-5206. [6] LU J, YE M, TANG Z, et al. A novel method for Chinese named entity recognition based on character vector[C]//Proceedings of the 11th International Conference on Collaborative Computing:Networking, Applications and Worksharing, LNICST 163. Cham:Springer, 2015:141-150. [7] DONG C, ZHANG J, ZONG C, et al. Character-based LSTMCRF with radical-level features for Chinese named entity recognition[C]//Proceedings of the 5th CCF Conference on Natural Language Processing and Chinese Computing, and 24th International Conference on Computer Processing of Oriental Languages, LNCS 10102. Cham:Springer, 2016:239-250. [8] 王博冉,林夏,朱晓东,等. Lattice LSTM神经网络法中文医学文本命名实体识别模型研究[J]. 中国卫生信息管理杂志, 2019, 16(1):84-88. (WANG B R, LIN X, ZHU X D, et al. Chinese Name language Entity Recognition (NER) using lattice LSTM in medical language[J]. Chinese Journal of Health Information Management, 2019, 16(1):84-88.) [9] 柏兵,侯霞,石松. 基于CRF和BI-LSTM的命名实体识别方法[J]. 北京信息科技大学学报, 2018, 33(6):27-33. (BAI B, HOU X, SHI S. Named entity recognition method based on CRF and BI-LSTM[J]. Journal of Beijing University of Information Technology, 2018, 33(6):27-33.) [10] 李明扬,孔芳. 融入自注意力机制的社交媒体命名实体识别[J]. 清华大学学报(自然科学版), 2019, 59(6):461-467. (LI M Y, KONG F. Combined self-attention mechanism for named entity recognition in social media[J]. Journal of Tsinghua University (Natural Science Edition), 2019, 59(6):461-467.) [11] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st Conference on Neural Information and Processing Systems. Red Hook, NY:Curran Associates Inc., 2017:6000-6010. [12] ZHENG S, WANG F, BAO H, et al. Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2017:1227-1236. [13] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[EB/OL].[2019-11-12].https://arxiv.org/pdf/1301.3781.pdf. [14] JOULIN A, GRAVE E, BOJANOWSKI P, et al. Bag of tricks for efficient text classification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2017:427-431. [15] PENNINGTON J, SOCHER R, MANNING C. Glove:global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2014:1532-1543. [16] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2016:770-778. [17] JOZEFOWICZ R, ZAREMBA W, SUTSKEVER I. An empirical exploration of recurrent network architectures[C]//Proceedings of the 32nd International Conference on International Conference on Machine Learning. New York:JMLR.org, 2015:2342-2350. [18] GORMLEY M R, YU M, DREDZE M. Improved relation extraction with feature-rich compositional embedding models[C]//Proceedings of the 2015 Conference on Empirical Method in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2015:1774-1784. [19] TANG J, QU M, WANG M, et al. LINE:large-scale information network embedding[C]//Proceedings of the 24th International Conference on World Wide Web. Republic and Canton of Geneva:International World Wide Web Conferences Steering Committee, 2015:1067-1077. [20] HOFFMANN R, ZHANG C, LING X, et al. Knowledge-based weak supervision for information extraction of overlapping relations[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg, PA:Association for Computational Linguistics, 2011:541-550. [21] LI Q, JI H. Incremental joint extraction of entity mentions and relations[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2014:402-412.
2020, Vol. 40 
