Review of event causality extraction based on deep learning

doi:10.11772/j.issn.1001-9081.2020071080

Abstract

Abstract: Causality extraction is a kind of relation extraction task in Natural Language Processing (NLP), which mines event pairs with causality from text by constructing event graph, and play important role in applications of finance, security, biology and other fields. Firstly, the concepts such as event extraction and causality were introduced, and the evolution of mainstream methods and the common datasets of causality extraction were described. Then, the current mainstream causality extraction models were listed. Based on the detailed analysis of pipeline based models and joint extraction models, the advantages and disadvantages of various methods and models were compared. Furthermore, the experimental performance and related experimental data of the models were summarized and analyzed. Finally, the research difficulties and future key research directions of causality extraction were given.

Key words: causality, Natural Language Processing (NLP), relation extraction, event pair, deep learning

摘要： 因果关系抽取是自然语言处理（NLP）中的一种关系抽取任务，它通过构造事件图来挖掘文本中具有因果关系的事件对，已经在金融、安全、生物等领域的应用中发挥重要作用。首先，介绍了事件抽取和因果关系等概念，并介绍了因果关系抽取主流方法的演变和常用数据集；然后，列举了当前主流的因果关系抽取模型，并且在分别对基于流水线的模型和联合抽取模型进行详细分析的基础上，对比了各种方法和模型的优缺点；此外，对各模型的实验性能及相关实验数据进行了归纳分析；最后，给出了当前的因果关系抽取的研究难点和未来的重点研究方向。

关键词: 因果关系, 自然语言处理, 关系抽取, 事件对, 深度学习

CLC Number:

TP391

WANG Zhujun, WANG Shi, LI Xueqing, ZHU Junwu. Review of event causality extraction based on deep learning[J]. Journal of Computer Applications, 2021, 41(5): 1247-1255.

王朱君, 王石, 李雪晴, 朱俊武. 基于深度学习的事件因果关系抽取综述[J]. 《计算机应用》唯一官方网站, 2021, 41(5): 1247-1255.

References

[1] 郭笑丹. 基于贝叶斯网络的民航突发事件因果关系分析方法研究[D]. 天津:中国民航大学,2018:1-57.(GUO X D. Research on causal reasoning method of domain ontology based on Bayesian network[D]. Tianjin:Civil Aviation University of China,2018:1-57.)
[2] 刘加新. 生物医学领域的实体因果关系抽取[D]. 苏州:苏州大学,2018:1-86. (LIU J X. Causal relation extraction between biomedical entities[D]. Suzhou:Soochow University,2018:1-86.)
[3] PEARL J,MACKENZIE D. The book of why:the new science of cause and effect[J]. Science,2018,361(6405):855-855.
[4] 刘挺. 从知识图谱到事理图谱[R]. 哈尔滨:哈尔滨工业大学社会计算与信息检索研究中心,2017. (LIU T. From knowledge graph to event evolutionary graph[R]. Harbin:Research Center for Social Computing and Information Retrieval of Harbin Institute of Technology,2017.)
[5] CHANG D S,CHOI K S. Incremental cue phrase learning and bootstrapping method for causality extraction using cue phrase and word pair probabilities[J]. Information Processing and Management,2006,42(3):662-678.
[6] 付剑锋, 刘宗田, 刘炜, 等. 基于层叠条件随机场的事件因果关系抽取[J]. 模式识别与人工智能,2011,24(4):567-573.(FU J F,LIU Z T,LIU W,et al. Event causality extraction based on cascaded conditional random fields[J]. Pattern Recognition and Artificial Intelligence,2011,24(4):567-573.)
[7] 冯冲, 康丽琪, 石戈, 等. 融合对抗学习的因果关系抽取[J]. 自动化学报,2018,44(5):811-818.(FENG C,KANG L Q,SHI G,et al. Causality extraction with GAN[J]. Acta Automatica Sinica,2018,44(5):811-818.)
[8] 许晶航, 左万利, 梁世宁, 等. 基于图注意力网络的因果关系抽取[J]. 计算机研究与发展,2020,57(1):159-174.(XU J H, ZUO W L,LIANG S N,et al. Causal relation extraction based on graph attention networks[J]. Journal of Computer Research and Development,2020,57(1):159-174.)
[9] 庄传志, 靳小龙, 朱伟建, 等. 基于深度学习的关系抽取研究综述[J]. 中文信息学报,2019,33(12):1-18.(ZHUANG C Z,JIN X L,ZHU J W,et al. Deep learning based relation extraction:a survey[J]. Journal of Chinese Information Processing,2019,33(12):1-18.)
[10] KHOO C S G,KORNFILT J,ODDY R N,et al. Automatic extraction of cause-effect information from newspaper text without knowledge-based inferencing[J]. Literary and Linguistic Computing,1998,13(4):177-186.
[11] KHOO C S G,CHAN S,NIU Y. Extracting causal knowledge from a medical database using graphical patterns[C]//Proceedings of 38th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2000:336-343.
[12] GIRJU R,MOLDOVAN D. Text mining for causal relations[C]//Proceedings of the 15th International Florida Artificial Intelligence Research Society Conference. Palo Alto,CA:AAAI Press,2002:360-364.
[13] ITTOO A,BOUMA G. Extracting explicit and implicit causal relations from sparse,domain-specific texts[C]//Proceedings of 16th International Conference on Applications of Natural Language to Information Systems,LNCS 6716. Berlin:Springer,2011:52-63.
[14] GIRJU R. Automatic detection of causal relations for question answering[C]//Proceedings of the ACL 2003 Workshop on Multilingual Summarization and Question Answering. Stroudsburg, PA:Association for Computational Linguistics, 2003:76-83.
[15] LUO Z,SHA Y,ZHU K Q,et al. Commonsense causal reasoning between short texts[C]//Proceedings of the 15th International Conference on Principles of Knowledge Representation and Reasoning. Palo Alto,CA:AAAI Press,2016:421-430.
[16] DE SILVA T N,XIAO Z,ZHAO R,et al. Causal relation identification using convolutional neural networks and knowledge based features[J]. World Academy of Science,Engineering and Technology:International Journal of Mechanical and Mechatronics Engineering,2017,11(6):703-708.
[17] KRUENGKRAI C, TORISAWA K, HASHIMOTO C, et al. Improving event causality recognition with multiple background knowledge sources using multi-column convolutional neural networks[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2017:3466-3473.
[18] LI P,MAO K. Knowledge-oriented convolutional neural network for causal relation extraction from natural language texts[J]. Expert Systems with Applications,2019,115:512-523.
[19] DASGUPTA T,SAHA R,DEY L,et al. Automatic extraction of causal relations from text using linguistically informed deep neural networks[C]//Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue. Stroudsburg, PA:Association for Computational Linguistics,2018:306-316.
[20] AGUILAR J,BELLER C,MCNAMEE P,et al. A comparison of the events and relations across ACE, ERE, TAC-KBP, and FrameNet annotation standards[C]//Proceedings of the 2nd Workshop on EVENTS:Definition,Detection,Coreference,and Representation. Stroudsburg,PA:Association for Computational Linguistics,2014:45-53.
[21] HENDRICKX I,KIM S N,KOZAREVA Z,et al. SemEval-2010 Task 8:multi-way classification of semantic relations between pairs of nominals[C]//Proceedings of the 2009 Workshop on Semantic Evaluations:Recent Achievements and Future Directions. Stroudsburg, PA:Association for Computational Linguistics,2009:94-99.
[22] CHEN Y,XU L,LIU K,et al. Event extraction via dynamic multi-pooling convolutional neural networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics/the 7th International Joint Conference on Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics,2015:167-176.
[23] GHAEINI R,FERN X,HUANG L,et al. Event nugget detection with forward-backward recurrent neural networks[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2016:369-373.
[24] FENG X,QIN B,LIU T. A language-independent neural network for event detection[J]. SCIENCE CHINA Information Sciences, 2018,61(9):No. 092106.
[25] DUAN S, HE R, ZHAO W. Exploiting document level information to improve event detection via recurrent neural networks[C/OL]//Proceedings of the 8th International Joint Conference on Natural Language Processing.[2020-06-12]. https://www.aclweb.org/anthology/I17-1036.pdf.
[26] ZHAO Y, JIN X, WANG Y, et al. Document embedding enhanced event detection with hierarchical and supervised attention[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg,PA:Association for Computational Linguistics,2018:414-419.
[27] YANG S, FENG D, QIAO L, et al. Exploring pre-trained language models for event extraction and generation[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2019:5284-5294.
[28] ZHANG X,ZHAO J,LECUN Y. Character-level convolutional networks for text classification[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge:MIT Press,2015:649-657.
[29] LIU C,SUN W,CHAO W,et al. Convolution neural network for relation extraction[C]//Proceedings of the 9th International Conference on Advanced Data Mining and Applications,LNCS 8347. Berlin:Springer,2013:231-242.
[30] ZENG D, LIU K, LAI S, et al. Relation classification via convolutional deep neural network[C]//Proceedings of the 25th International Conference on Computational Linguistics:Technical Papers. Stroudsburg, PA:Association for Computational Linguistics,2014:2335-2344.
[31] ZHANG D,WANG D. Relation classification via recurrent neural network[EB/OL].[2020-08-05]. https://arxiv.org/pdf/1508.01006.pdf.
[32] HOU X, ZHAO Z, LIU T, et al. Classifying relation via bidirectional recurrent neural network based on local information[C]//Proceedings of the 18th Asia-Pacific Web Conference, LNCS 9931. Cham:Springer,2016:420-430.
[33] XU Y,MOU L,LI G,et al. Classifying relations via long short term memory networks along shortest dependency paths[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg,PA:Association for Computational Linguistics,2015:1785-1794.
[34] ZHANG S,ZHENG D,HU X,et al. Bidirectional long short-term memory networks for relation classification[C/OL]//Proceedings of the 29th Pacific Asia Conference on Language,Information and Computation.[2020-08-05]. https://www.aclweb.org/anthology/Y15-1009.pdf.
[35] ZHOU P,SHI W,TIAN J,et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2016:207-212.
[36] MNIH V,HEESS N,GRAVES A,et al. Recurrent models of visual attention[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge:MIT Press,2014:2204-2212.
[37] XIAO M,LIU C. Semantic relation classification via hierarchical recurrent neural network with attention[C]//Proceedings of the 26th International Conference on Computational Linguistics:Technical Papers. Stroudsburg, PA:Association for Computational Linguistics,2016:1254-1263.
[38] CHEN Y,WANG K,YANG W,et al. A multi-channel deep neural network for relation extraction[J]. IEEE Access,2020,8:13195-13203.
[39] HUANG Z,XU W,YU K. Bidirectional LSTM-CRF models for sequence tagging[EB/OL].[2020-08-09]. https://arxiv.org/pdf/1508.01991v1.pdf.
[40] LAFFERTY J,MCCALLUM A,PEREIRA F C N. Conditional random fields:probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning. San Francisco:Morgan Kaufmann Publishers Inc.,2001:282-289.
[41] ZHANG Y,QI P,MANNING C D. Graph convolution over pruned dependency trees improves relation extraction[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg,PA:Association for Computational Linguistics,2018:2205-2215.
[42] ZHU H,LIN Y,LIU Z,et al. Graph neural networks with generated parameters for relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2019:1331-1339.
[43] ZHENG S,HAO Y,LU D,et al. Joint entity and relation extraction based on a hybrid neural network[J]. Neurocomputing, 2017,257:59-66.
[44] ZENG X,ZENG D,HE S,et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2018:506-514.
[45] PANG Y,LIU J,LIU L,et al. A deep neural network model for joint entity and relation extraction[J]. IEEE Access,2019,7:179143-179150.
[46] BAI C,PAN L,LUO S,et al. Joint extraction of entities and relations by a novel end-to-end model with a double-pointer module[J]. Neurocomputing,2020,377:325-333.
[47] VASWANI A,SHAZEER N,PARMAR N,et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook,NY:Curran Associates Inc.,2017:6000-6010.
[48] ZHENG S,WANG F,BAO H,et al. Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2017:1227-1236.
[49] DAI D,XIAO X,LYU Y,et al. Joint extraction of entities and overlapping relations using position-attentive sequence labeling[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2019:6300-6308.
[50] LI Z,LI Q,ZOU X,et al. Causality extraction based on selfattentive BiLSTM-CRF with transferred embeddings[J]. Neurocomputing,2021,423:207-219.
[51] SUN C,GONG Y,WU Y,et al. Joint type inference on entities and relations via graph convolutional networks[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2019:1361-1370.
[52] FU T J,LI P H,MA W Y. GraphRel:modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2019:1409-1418.
[53] WEI Z,SU J,WANG Y,et al. A novel cascade binary tagging framework for relational triple extraction[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2020:1476-1488.