Chinese natural language interface based on paraphrasing

doi:10.11772/j.issn.1001-9081.2016.05.1290

Abstract

Abstract: In this paper, a novel method for Chinese Natural Language Interface of Database (NLIDB) based on Chinese paraphrase was proposed to solve the problems of traditional methods based on syntactic parsing which cannot obtain high accuracy and need a lot of manual label training corpus. First, key entities of user statements in databases were extracted, and candidate tree sets and their tree expressions were generated. Then most relevant semantic expressions were filtered by paraphrase classifier which was obtained from the Internet Q&A training corpus. Finally, candidate trees were translated into Structured Query Language (SQL). F1 score was respectively 83.4% and 90% on data sets of Chinese America Geography (GeoQueries880) and Questions about Restaurants (RestQueries250) by using the proposed method, better than syntactic based method. The experimental results demonstrate that the NLIDB based on paraphrase can handle the semantic gaps between users and databases better.

Key words: Natural Language Interface of DataBase (NLIDB), word vector, paraphrase, natural language expression, machine learning

摘要： 针对传统以句法分析为主的数据库自然语言接口系统识别用户语义准确率不高,且需要大量人工标注训练语料的问题,提出了一种基于复述的中文自然语言接口(NLIDB)实现方法。首先提取用户语句中表征数据库实体词,建立候选树集及对应的形式化自然语言表达;其次由网络问答语料训练得到的复述分类器筛选出语义最相近的表达;最后将相应的候选树转换为结构化查询语句(SQL)。实验表明该方法在美国地理问答语料(GeoQueries880)、餐饮问答语料(RestQueries250)上的F1值分别达到83.4%、90%,均优于句法分析方法。通过对比实验结果发现基于复述方法的数据库自然语言接口系统能更好地处理用户与数据库的语义鸿沟问题。

关键词: 数据库自然语言接口, 词向量, 复述, 自然语言表达, 机器学习

CLC Number:

TP391.1

ZHANG Junchi, HU Jie, LIU Mengchi. Chinese natural language interface based on paraphrasing[J]. Journal of Computer Applications, 2016, 36(5): 1290-1295.

张俊驰, 胡婕, 刘梦赤. 基于复述的中文自然语言接口[J]. 计算机应用, 2016, 36(5): 1290-1295.

References

[1] RODOLFO A, JUAN J, MARCO A, et al. Natural language interfaces to databases:an analysis of the state of the art[C]//Recent Advances on Hybrid Intelligent Systems. Berlin:Springer, 2013, 451:463-480.
[2] AHMAD R, KHAN M, ALI R. Efficient transformation of natural language query to SQL for Urdu[C]//Proceedings of the 2nd Conference on Language and Technology.[S.l.]:Society for Natural Language Processing, 2009:53-60.
[3] POPESCU A, ARMANASU A, ETZIONI O, et al. Modern natural language interfaces to databases:composing statistical parsing with semantic tractability[C]//Proceedings of the 20th International Conference on Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2004:141-147.
[4] 孟小峰, 王珊. 中文数据库自然语言查询系统NCHIQL设计与实现[J]. 计算机研究与发展, 2001, 38(9):1080-1086. (MENG X F, WANG S. Design and implementation of a Chinese natural language interface to database (NCHIQL)[J]. Computer Research and Development, 2001, 38(9):1080-1086.)
[5] RODOLFO A, JUAN J, MARCO A. Semantic model for improving the performance of natural language interfaces to databases[C]//Proceedings of the 10th Mexican International Conference on Advances in Artificial Intelligence, LNCS 7094. Berlin:Springer-Verlag, 2011:277-290.
[6] 许龙飞, 杨晓昀, 唐世渭. 基于受限汉语的数据库自然语言接口技术研究[J]. 软件学报, 2002, 13(4):537-544.(XU L F, YANG X Y, TANG S W. Study on a database natural language interface technique based on restrictive Chinese[J]. Journal of Software, 2002, 13(4):537-544.)
[7] MINOCK M, OLOFSSON P, NÄSLUND A. Towards building robust natural language interfaces to databases[C]//Proceedings of the 13th International Conference on Natural Language and Information Systems:Applications of Natural Language to Information Systems. Berlin:Springer-Verlag, 2008, 5039:187-198.
[8] WARREN D, PEREIRA F. An efficient easily adaptable system for interpreting natural language queries[J]. Computational Linguistics, 1982, 8(3/4):110-122.
[9] WEISCHEDEL R. A hybrid approach to representation in the Janus natural language processor[C]//Proceedings of the 27th Annual Meeting on Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 1989:193-202.
[10] GIORDANI A, MOSCHITTI A. Automatic generation and reranking of SQL-derived answers to NL questions[C]//Proceedings of the 2nd International Workshop on Trustworthy Eternal Systems via Evolving Software, Data and Knowledge, Volume 379 of the series Communications in Computer and Information Science. Berlin:Springer-Verlag, 2013:59-76.
[11] LI F, JAGADISH H. Constructing an interactive natural language interface for relational databases[J]. Proceedings of the VLDB Endowment, 2014, 8(1):73-84
[12] POON H. Grounded unsupervised semantic parsing[C]//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2013:1-10.
[13] GIORDANI A, MOSCHITTI A. Generating SQL queries using natural language syntactic dependencies and metadata[C]//Proceedings of the 17th International Conference on Applications of Natural Language to Information Systems, LNCS 7337. Berlin:Springer, 2012:164-170.
[14] BERANT J, CHOU A, FROSTIG R, et al. Semantic parsing on freebase from question-answer pairs[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2013:1533-1544.
[15] CAI Q, YATES A. Large-scale semantic parsing via schema matching and lexicon extension[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2013:423-433.
[16] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Proceedings of the Advances in Neural Information Processing Systems. Nevada:NIPS, 2013:3111-3119.
[17] 曹勇刚, 曹羽中, 金茂忠, 等. 面向信息检索的自适应中文分词系统[J]. 软件学报, 2006, 17(3):356-363. (CAO Y G, CAO Y Z, JIN M Z, et al. Information retrieval oriented adaptive Chinese word segmentation system[J]. Journal of Software, 2006, 17(3):356-363.)
[18] ESPAÑA-BOQUERA S, CASTRO-BLEDA M, ZAMORA-MARTÍNEZ F, et al. Efficient viterbi algorithms for lexical tree based models[C]//Proceedings of the 2007 International Conference on Advances in Nonlinear Speech Processing. Berlin:Springer-Verlag, 2007, 4885:179-187.
[19] SIMITSIS A, IOANNIDIS Y. DBMSs should talk back too[C]//Proceedings of the 4th Biennal Conference on Innovative Data Systems Research.[S.l.]:arXiv, 2009:62-70.
[20] KOUTRIKA G, SIMITSIS A, IOANNIDIS Y E. Explaining structured queries in natural language[C]//Proceedings of the 2010 IEEE 26th International Conference on Data Engineering. Piscataway, NJ:IEEE, 2010:333-344.
[21] BARZILAY R, MCKEOWN K. Extracting paraphrases from a parallel corpus[C]//Proceedings of the 39th Annual Meeting on Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2002:50-57.
[22] 赵世奇, 刘挺, 李生. 复述技术研究[J]. 软件学报, 2009, 20(8):2124-2137.(ZHAO S Q, LIU T, LI S. Research on paraphrasing technology[J]. Journal of Software, 2009, 20(8):2124-2137.)
[23] ZHANG D. Question classification using support vector machines[C]//Proceedings of the 26th Annual International ACM SIGIR Conference on Research & Development in Information Retrieval. New York:ACM, 2003:26-32.
[24] CHANG P, TSENG H, JURAFSKY D, et al. Discriminative reordering with Chinese grammatical relations features[C]//Proceedings of the 3rd Workshop on Syntax and Structure in Statistical Translation. Stroudsburg, PA:Association for Computational Linguistics, 2009:51-59.
[25] ZHANG Y, NIVRE J. Transition-based dependency parsing with rich non-local features[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg, PA:Association for Computational Linguistics, 2011:188-193.
[26] SHEN L, JOSHI A K. An SVM based voting algorithm with application to parse reranking[C]//Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL. Stroudsburg, PA:Association for Computational Linguistics, 2003:9-16.
[27] POPESCU A M, ETZIONI O, KAUTZ H. Towards a theory of natural language interfaces to databases[C]//Proceedings of the 8th International Conference on Intelligent User Interfaces. New York:ACM, 2003:149-157.