融入句法感知表示进行句法增强的语义解析

doi:10.11772/j.issn.1001-9081.2020111863

摘要/Abstract

摘要： 在自然语言处理（NLP）中，句法信息是完整句子中词汇与词汇之间的句法结构关系或者依存关系，是一种重要且有效的参考信息。语义解析任务是将自然语言语句直接转化成语义完整的、计算机可执行的语言。在以往的语义解析研究中，少有采用输入源的句法信息来提高端到端语义解析效率的工作。为了进一步提高端到端语义解析模型的准确率和效率，提出一种利用输入端句法依存关系信息来提高模型效率的语义解析方法。该方法的基本思路是先对一个端到端的依存关系解析器进行预训练；然后将该解析器的中间表示作为句法感知表示，与原有的字词嵌入表示拼接到一起以产生新的输入嵌入表示，并将得到的输入嵌入表示用于端到端语义解析模型；最后采用转导融合学习方式进行模型融合。实验对比了所提模型和基准模型Transformer以及过去十年的相关工作。实验结果表明，在ATIS、GEO、JOBS数据集上，融入依存句法信息感知表示以及转导融合学习的语义解析模型分别实现了89.1%、90.7%、91.4%的最佳准确率，全面超过了Transformer，验证了引入句法依存关系信息的有效性。

关键词: 句法感知, 语义解析, 深度学习, 自然语言处理, 语言模型

Abstract: Syntactic information, which is syntactic structure relations or dependency relations between words of a complete sentence, is an important and effective reference in Natural Language Processing (NLP). The task of semantic parsing is to directly transform natural language sentences into semantically complete and computer-executable languages. In previous semantic parsing studies, there are few efforts on improving the efficiency of end-to-end semantic parsing by using syntactic information from input sources. To further improve the accuracy and efficiency of the end-to-end semantic parsing model, a semantic parsing method was proposed to utilize the source-side dependency relation information of syntax to improve the model efficiency. As the basic idea of the method, an end-to-end dependency relation parser was pre-trained firstly. Then, the middle representation of the parser was used as syntax-aware representation, which was spliced with the original word embedding representation to generate a new input embedding representation, and this obtained input embedding representation was used in the end-to-end semantic parsing model. Finally, the model fusion was carried out by the transductive fusion learning. In the experiments, the proposed model was compared with the baseline model Transformer and the related works in the past decade. Experimental results show that, on ATIS, GEO and JOBS datasets, the semantic parsing model integrating dependency syntax-aware representation and transductive fusion learning achieves the best accuracy of 89.1%, 90.7%, and 91.4% respectively, which exceeds the performance of the Transformer. It verifies the effectiveness of introducing the dependency relation information of syntax.

Key words: syntax-aware, semantic parsing, deep learning, Natural Language Processing (NLP), language model

中图分类号:

TP391

谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495.

XIE Defeng, JI Jianmin. Syntax-enhanced semantic parsing with syntax-aware representation[J]. Journal of Computer Applications, 2021, 41(9): 2489-2495.

参考文献

[1] 吉建民. 提高ASP效率的若干途径及服务机器人上应用[D]. 合肥:中国科学技术大学,2010:2.(JI J M. Several ways to improve efficiency of answer set programming and its application in service robots[D]. Hefei:University of Science and Technology of China, 2010:2.)
[2] 柴伟. 短语结构句法分析综述[J]. 电脑知识与技术,2020,16(16):26-27,30.(CHAI W. A review of the constituency parsing[J]. Computer Knowledge and Technology,2020,16(16):26-27,30.)
[3] BAHDANAU D,CHO K,BENGIO Y. Neural machine translation by jointly learning to align and translate[EB/OL]. (2016-05-19)[2020-09-10]. http://www.arxiv.org/pdf/1409.0473.pdf.
[4] CHO K,VAN MERRIËNBOER B,GÜLÇEHRE Ç,et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics,2014:1724-1734.
[5] ERIGUCHI A, HASHIMOTO K, TSURUOKA Y. Tree-tosequence attentional neural machine translation[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2016:823-833.
[6] CHEN H D,HUANG S J,CHIANG D,et al. Improved neural machine translation with a syntax-aware encoder and decoder[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2017:1936-1945.
[7] ZHANG M S,LI Z H,FU G H,et al. Syntax-enhanced neural machine translation with syntax-aware word representations[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg, PA:Association for Computational Linguistics,2019:1151-1161.
[8] REDDY S,LAPATA M,STEEDMAN M. Large-scale semantic parsing without question-answer pairs[J]. Transactions of the Association for Computational Linguistics,2014,2:377-392.
[9] REDDY S, TÄCKSTRÖM O, PETROV S, et al. Universal semantic parsing[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics,2017:89-101.
[10] REDDY S,TÄCKSTRÖM O,COLLINS M,et al. Transforming dependency structures to logical forms for semantic parsing[J]. Transactions of the Association for Computational Linguistics, 2016,4:127-140.
[11] BERANT J,CHOU A,FROSTIG R,et al. Semantic parsing on freebase from question-answer pairs[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2013:1533-1544.
[12] MATUSZEK C,FITZGERALD N,ZETTLEMOYER L,et al. A joint model of language and perception for grounded attribute learning[C]//Proceedings of the 29th International Conference on Machine Learning. Madison,WI:Omnipress,2012:1671-1678.
[13] ZETTLEMOYER L S,COLLINS M. Learning to map sentences to logical form:structured classification with probabilistic categorical grammars[C]//Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence. Arlington,VA:AUAI Press,2005:658-666.
[14] WANG A,KWIATKOWSKI T,ZETTLEMOYER L. Morphosyntactic lexical generalization for CCG semantic parsing[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg,PA:Association for Computational Linguistics,2014:1284-1295.
[15] DONG L,LAPATA M. Language to logical form with neural attention[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2016:33-43.
[16] MALHOTRA P,VIG L,SHROFF G,et al. Long short term memory networks for anomaly detection in time series[C]//Proceedings of the 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges,Belgium:[s. n.],2015:89-94.
[17] ZAREMBA W,SUTSKEVER I,VINYALS O. Recurrent neural network regularization[EB/OL]. (2015-02-19)[2020-09-11]. https://arxiv.org/pdf/1409.2329.pdf.
[18] MNIH V,HEESS N,GRAVES A,et al. Recurrent models of visual attention[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge:MIT Press,2014:2204-2212.
[19] XU K,WU L F,WANG Z G,et al. Exploiting rich syntactic information for semantic parsing with graph-to-sequence model[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg,PA:Association for Computational Linguistics,2018:918-924.
[20] CHEN B,SUN L,HAN X P. Sequence-to-action:end-to-end semantic graph generation for semantic parsing[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2018:766-777.
[21] SUN Y B,TANG D Y,DUAN N,et al. Semantic parsing with syntax-and table-aware SQL generation[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2018:361-372.
[22] DONG L, LAPATA M. Coarse-to-fine decoding for neural semantic parsing[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg,PA:Association for Computational Linguistics,2018:731-742.
[23] YIN P C,NEUBIG G. TRANX:a transition-based neural abstract syntax parser for semantic parsing and code generation[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing:System Demonstrations. Stroudsburg, PA:Association for Computational Linguistics, 2018:7-12.
[24] ZHANG X,HE S Z,LIU K,et al. AdaNSP:uncertainty-driven adaptive decoding in neural semantic parsing[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2019:4265-4270.
[25] VASWANI A,SHAZEER N,PARMAR N,et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook,NY:Curran Associates Inc.,2017:6000-6010.
[26] SUN Z Y,ZHU Q H,XIONG Y F,et al. TreeGen:a tree-based transformer architecture for code generation[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto, CA:AAAI Press,2020:8984-8991.
[27] DOZAT T,MANNING C D. Simpler but more accurate semantic dependency parsing[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2018:484-490.
[28] DOZAT T,MANNING C D. Deep biaffine attention for neural dependency parsing[EB/OL]. (2017-03-10)[2020-09-12]. https://arxiv.org/pdf/1611.01734.pdf.
[29] WANG Y R,WU L J,XIA Y C,et al. Transductive ensemble learning for neural machine translation[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2020:6291-6298.
[30] MANNING C D,SURDEANU M,BAUER J,et al. The Stanford CoreNLP natural language processing toolkit[C]//Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics:System Demonstrations. Stroudsburg, PA:Association for Computational Linguistics,2014:55-60.
[31] ZETTLEMOYER L,COLLINS M. Online learning of relaxed CCG grammars for parsing to logical form[C]//Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Stroudsburg, PA:Association for Computational Linguistics, 2007:678-687.
[32] KWIATKOWSKI T,ZETTLEMOYER L,GOLDWATER S,et al. Lexical generalization in CCG grammar induction for semantic parsing[C]//Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics,2011:1512-1523.
[33] LIANG P,JORDAN M I,KLEIN D. Learning dependency-based compositional semantics[J]. Computational Linguistics,2013,29(2):389-446.
[34] KWIATKOWSKI T,CHOI E,ARTZI Y,et al. Scaling semantic parsers with on-the-fly ontology matching[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics,2013:1545-1556.
[35] ZHAO K,HUANG L. Type-driven incremental semantic parsing with polymorphism[C]//Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg,PA:Association for Computational Linguistics,2015:1416-1421.
[36] JIA R,LIANG P. Data recombination for neural semantic parsing[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg,PA:Association for Computational Linguistics,2016:12-22.