Complex causal relationship extraction based on prompt enhancement and bi-graph attention network

doi:10.11772/j.issn.1001-9081.2023101486

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (10): 3081-3089.DOI: 10.11772/j.issn.1001-9081.2023101486

• Artificial intelligence • Previous Articles Next Articles

Complex causal relationship extraction based on prompt enhancement and bi-graph attention network

Jinke DENG¹^,², Wenjie DUAN¹^,², Shunxiang ZHANG¹^,²(), Yuqing WANG¹^,², Shuyu LI¹^,², Jiawei LI¹^,²

^1.School of Computer Science and Engineering，Anhui University of Science and Technology，Huainan Anhui 232001，China
^2.Institute of Artificial Intelligence，Hefei Comprehensive National Science Center，Hefei Anhui 230088，China

Received:2023-11-02 Revised:2024-01-10 Accepted:2024-01-19 Online:2024-10-15 Published:2024-10-10
Contact: Shunxiang ZHANG
About author:DENG Jinke， born in 2001， M. S. candidate. His research interests include data mining， information extraction.
DUAN Wenjie， born in 2000， M. S. candidate. His research interests include sentiment analysis.
WANG Yuqing， born in 2000， M. S. candidate. Her research interests include data mining.
LI Shuyu， born in 1999， M. S. candidate. Her research interests include data mining.
LI Jiawei， born in 1999， M. S. candidate. His research interests include sentiment analysis.
Supported by:
National Natural Science Foundation of China(62076006);University Synergy Innovation Program of Anhui Province(GXXT-2021-008)

基于提示增强与双图注意力网络的复杂因果关系抽取

邓金科¹^,², 段文杰¹^,², 张顺香¹^,²(), 汪雨晴¹^,², 李书羽¹^,², 李嘉伟¹^,²

^1.安徽理工大学计算机科学与工程学院，安徽淮南 232001
^2.合肥综合性国家科学中心人工智能研究院，合肥 230088

通讯作者: 张顺香
作者简介:邓金科（2001—），男，安徽亳州人，硕士研究生，CCF会员，主要研究方向：数据挖掘、信息抽取
段文杰（2000—），男，安徽宿州人，硕士研究生，CCF会员，主要研究方向：情感分析
张顺香（1970—），男，安徽无为人，教授，博士，主要研究方向：Web挖掘、语义搜索、关系抽取、复杂网络分析 sxzhang@aust.edu.cn
汪雨晴（2000—），女，安徽蚌埠人，硕士研究生，CCF会员，主要研究方向：数据挖掘
李书羽（1999—），女，安徽铜陵人，硕士研究生，主要研究方向：数据挖掘
李嘉伟（1999—），男，安徽宣城人，硕士研究生，主要研究方向：情感分析。
基金资助:
国家自然科学基金资助项目(62076006);安徽高校协同创新项目(GXXT?2021?008)

Abstract

Abstract:

A complex causal relationship extraction model based on prompt enhancement and Bi-Graph ATtention network （BiGAT） — PE-BiGAT （Prompt Enhancement and Bi-Graph Attention Network） was proposed to address the issues of insufficient external information and information transmission forgetting caused by the high density and long sentence patterns of complex causal sentences. Firstly， the result entities from the sentence were extracted and combined with the prompt learning template to form the prompt information， and the prompt information was enhanced through an external knowledge base. Then， the prompt information was input into the BiGAT， the attention layer was combined with syntax and semantic dependency graphs， and the biaffine attention mechanism was used to alleviate feature overlapping and enhance the model’s perception of relational features. Finally， all causal entities in the sentence were predicted iteratively by the classifier， and all causal pairs in the sentence were analyzed through a scoring function. Experimental results on SemEval-2010 task 8 and AltLex datasets show that compared with RPA-GCN （Relationship Position and Attention?Graph Convolutional Network）， the proposed model improves the F1 score by 1.65 percentage points， with 2.16 and 4.77 percentage points improvements in chain causal and multi-causal sentences， which confirming that the proposed model has an advantage in dealing with complex causal sentences.

Key words: complex causal relationship extraction, prompt enhancement, Bi-Graph Attention Network (BiGAT), biaffine attention, scoring function

摘要：

针对复杂因果句实体密度高、句式冗长等特点导致的外部信息不足和信息传递遗忘问题，提出一种基于提示增强与双图注意力网络（BiGAT）的复杂因果关系抽取模型PE-BiGAT（Prompt Enhancement and Bi-Graph Attention Network）。首先，抽取句子中的结果实体并与提示学习模板组成提示信息，再通过外部知识库增强提示信息；其次，将提示信息输入BiGAT，同时结合关注层与句法和语义依存图，并利用双仿射注意力机制缓解特征重叠的情况，增强模型对关系特征的感知能力；最后，用分类器迭代预测句子中的所有因果实体，并通过评分函数分析句子中所有的因果对。在SemEval-2010 task 8和AltLex数据集上的实验结果表明，与RPA-GCN（Relationship Position and Attention-Graph Convolutional Network）相比，所提模型的F1值提高了1.65个百分点，其中在链式因果和多因果句中分别提高了2.16和4.77个百分点，验证了所提模型在处理复杂因果句时更具优势。

关键词: 复杂因果关系抽取, 提示增强, 双图注意力网络, 双仿射注意力, 评分函数

CLC Number:

TP391.1

Jinke DENG, Wenjie DUAN, Shunxiang ZHANG, Yuqing WANG, Shuyu LI, Jiawei LI. Complex causal relationship extraction based on prompt enhancement and bi-graph attention network[J]. Journal of Computer Applications, 2024, 44(10): 3081-3089.

邓金科, 段文杰, 张顺香, 汪雨晴, 李书羽, 李嘉伟. 基于提示增强与双图注意力网络的复杂因果关系抽取[J]. 《计算机应用》唯一官方网站, 2024, 44(10): 3081-3089.

Figures/Tables 12

References 34

1	XU J， CHEN Y， QIN Y， et al. A feature combination-based graph convolutional neural network model for relation extraction［J］. Symmetry， 2021， 13（8）： No.1458.
2	冯冲，康丽琪，石戈，等. 融合对抗学习的因果关系抽取［J］. 自动化学报， 2018， 44（5）： 811-818.
	FENG C， KANG L Q， SHI G， et al. Causality extraction with GAN［J］. Acta Automatica Sinica， 2018， 44（5）： 811-818.
3	田生伟，周兴发，禹龙，等. 基于双向LSTM的维吾尔语事件因果关系抽取［J］. 电子与信息学报， 2018， 40（1）： 200-208.
	TIAN S W， ZHOU X F， YU L， et al. Causal relation extraction of Uyghur events based on bidirectional long short-term memory model［J］. Journal of Electronics & Information Technology， 2018， 40（1）： 200-208.
4	XU Y， MOU L， LI G， et al. Classifying relations via long short term memory networks along shortest dependency paths［C］// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2015： 1785-1794.
5	TAI K S， SOCHER R， MANNING C D. Improved semantic representations from tree-structured long short-term memory networks［C］// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg： ACL， 2015： 1556-1566.
6	VELIČKOVIĆ P， CUCURULL G， CASANOVA A， et al. Graph attention networks［EB/OL］. （2018-02-04）［2024-01-08］. .
7	SHAN Y， CHE C， WEI X， et al. Bi-graph attention network for aspect category sentiment classification［J］. Knowledge-Based Systems， 2022， 258： No.109972.
8	HENDRICKX I， KIM S N， KOZAREVA Z， et al. SemEval-2010 Task 8： multi-way classification of semantic relations between pairs of nominals［C］// Proceedings of the 5th International Workshop on Semantic Evaluation. Stroudsburg： ACL， 2010： 33-38.
9	HIDEY C， McKEOWN K. Identifying causal relations using parallel Wikipedia articles［C］// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2016： 1424-1433.
10	ZHAO S， WANG Q， MASSUNG S， et al. Constructing and embedding abstract event causality networks from text snippets［C］// Proceedings of the 10th ACM International Conference on Web Search and Data Mining. New York： ACM， 2017： 335-344.
11	GARCIA D， EDF-DER， IMA-TIEM. COATIS， an NLP system to locate expressions of actions connected by causality links［C］// Proceedings of the 1997 International Conference on Knowledge Engineering and Knowledge Management， LNCS 1319. Berlin： Springer， 1997： 347-352.
12	KRUENGKRAI C， TORISAWA K， HASHIMOTO C， et al. Improving event causality recognition with multiple background knowledge sources using multi-column convolutional neural networks［C］// Proceedings of the 31st AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2017： 3466-3473.
13	LIN Z， KAN M Y， NG H T. Recognizing implicit discourse relations in the penn discourse treebank［C］// Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2009： 343-351.
14	LI Z， LI Q， ZOU X， et al. Causality extraction based on self-attentive BiLSTM-CRF with transferred embeddings［J］. Neurocomputing， 2021， 423： 207-219.
15	陈克正，郭晓然，钟勇，等. 基于负训练和迁移学习的关系抽取方法［J］. 计算机应用， 2023， 43（8）： 2426-2430.
	CHEN K Z， GUO X R， ZHONG Y， et al. Relation extraction method based on negative training and transfer learning［J］. Journal of Computer Applications， 2023， 43（8）： 2426-2430.
16	GRAVES A， SCHMIDHUBER J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures［J］. Neural Networks， 2005， 18（5/6）： 602-610.
17	HOCHREITER S， SCHMIDHUBER J. Long short-term memory［J］. Neural Computation， 1997， 9（8）： 1735-1780.
18	CHEN Y， WAN W， HU J， et al. Complex causal extraction of fusion of entity location sensing and graph attention networks［J］. Information， 2022， 13（8）： No.364.
19	KIPF T N， WELLING M. Semi-supervised classification with graph convolutional networks［EB/OL］. （2017-02-22）［2024-01-08］. .
20	LI X， YIN F， SUN Z， et al. Entity-relation extraction as multi-turn question answering［C］// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2019： 1340-1350.
21	ZHOU G， MA W， GONG Y， et al. Nested causality extraction on traffic accident texts as question answering［C］// Proceedings of the 2021 CCF International Conference on Natural Language Processing and Chinese Computing， LNCS 13029. Cham： Springer， 2021： 354-362.
22	CAO Q， HAO X， REN H， et al. Graph attention network based detection of causality for textual emotion-cause pair［J］. World Wide Web， 2022， 26（4）： 1731-1745.
23	FEI H， REN Y， JI D. Boundaries and edges rethinking： an end-to-end neural model for overlapping entity relation extraction［J］. Information Processing and Management， 2020， 57（6）： No.102311.
24	张勇，高大林，巩敦卫，等. 用于关系抽取的注意力图长短时记忆神经网络［J］. 智能系统学报， 2021， 16（3）：518-527.
	ZHANG Y， GAO D L， GONG D W， et al. Attention graph long short-term memory neural network for relation extraction［J］. CAAI Transactions on Intelligent Systems， 2021， 16（3）： 518-527.
25	李志欣，孙亚茹，唐素勤，等. 双路注意力引导图卷积网络的关系抽取［J］. 电子学报， 2021， 49（2）： 315-323.
	LI Z X， SUN Y R， TANG S Q， et al. Dual attention guided graph convolutional networks for relation extraction［J］. Acta Electronica Sinica， 2021， 49（2）： 315-323.
26	DEVLIN J， CHANG M W， LEE K， et al. BERT： pre-training of deep bidirectional transformers for language understanding［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long and Short Papers）. Stroudsburg： ACL， 2019： 4171-4186.
27	WEI Z， SU J， WANG Y， et al. A novel cascade binary tagging framework for relational triple extraction［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2020： 1476-1488.
28	SPEER R， HAVASI C. Representing general relational knowledge in ConceptNet 5［C］// Proceedings of the 8th International Conference on Language Resources and Evaluation. ［S.l.］： European Language Resources Association， 2012： 3679-3686.
29	QI P， ZHANG Y， ZHANG Y， et al. Stanza： a Python natural language processing Toolkit for many human languages［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics： System Demonstrations. Stroudsburg： ACL， 2020： 101-108.
30	DOZAT T， MANNING C D. Deep biaffine attention for neural dependency parsing［EB/OL］. （2017-03-10）［2024-01-08］. .
31	HUANG Z， XU W， YU K. Bidirectional LSTM-CRF models for sequence tagging［EB/OL］. （2015-08-09）［2024-01-08］. .
32	CUI L， ZHANG Y. Hierarchically-refined label attention network for sequence labeling［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg： ACL， 2019： 4115-4128.
33	ZHANG Y， ZHONG V， CHEN D， et al. Position-aware attention and supervised data improve slot filling［C］// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2017： 35-45.
34	ZHANG Y， QI P， MANNING C D. Graph convolution over pruned dependency trees improves relation extraction［C］// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2018： 2205-2215.

模型	总体			单一因果关系			链式因果关系			多因果关系
模型	P	R	F1	P	R	F1	P	R	F1	P	R	F1
LSTM	32.45	31.96	32.04	39.16	39.58	39.37	22.92	23.54	23.23	17.86	20.32	19.01
LSTM+CRF	38.72	38.12	38.18	44.15	44.62	44.39	31.25	33.28	32.23	25.00	25.46	25.23
BiLSTM	70.69	70.23	70.31	79.58	79.65	79.62	56.25	56.71	56.48	55.36	54.98	55.17
BiLSTM+CRF	72.74	72.36	72.27	79.16	79.82	79.49	62.50	62.89	62.69	58.93	58.46	58.69
BiLSTM-LAN	78.13	78.55	78.30	81.66	81.53	81.60	72.92	72.33	72.62	71.43	71.68	71.55
PA-LSTM	82.40	82.50	82.61	87.50	87.46	87.48	75.00	75.32	75.16	73.21	73.65	73.43
C-GCN	84.44	84.58	84.73	87.50	87.12	87.31	80.21	80.84	80.52	78.57	78.15	78.36
RPA-GCN	85.46	85.56	85.67	87.50	87.31	87.41	83.33	83.52	83.42	83.36	81.36	80.86
PE-BiGAT	87.76	86.88	87.32	89.17	89.23	89.20	85.42	85.74	85.58	85.71	85.56	85.63

模型	总体			单一因果关系			链式因果关系			多因果关系
模型	P	R	F1	P	R	F1	P	R	F1	P	R	F1
LSTM	32.45	31.96	32.04	39.16	39.58	39.37	22.92	23.54	23.23	17.86	20.32	19.01
LSTM+CRF	38.72	38.12	38.18	44.15	44.62	44.39	31.25	33.28	32.23	25.00	25.46	25.23
BiLSTM	70.69	70.23	70.31	79.58	79.65	79.62	56.25	56.71	56.48	55.36	54.98	55.17
BiLSTM+CRF	72.74	72.36	72.27	79.16	79.82	79.49	62.50	62.89	62.69	58.93	58.46	58.69
BiLSTM-LAN	78.13	78.55	78.30	81.66	81.53	81.60	72.92	72.33	72.62	71.43	71.68	71.55
PA-LSTM	82.40	82.50	82.61	87.50	87.46	87.48	75.00	75.32	75.16	73.21	73.65	73.43
C-GCN	84.44	84.58	84.73	87.50	87.12	87.31	80.21	80.84	80.52	78.57	78.15	78.36
RPA-GCN	85.46	85.56	85.67	87.50	87.31	87.41	83.33	83.52	83.42	83.36	81.36	80.86
PE-BiGAT	87.76	86.88	87.32	89.17	89.23	89.20	85.42	85.74	85.58	85.71	85.56	85.63

模型	C标签			E标签			O标签
模型	P	R	F1	P	R	F1	P	R	F1
LSTM	55.32	63.57	59.16	60.46	60.19	60.32	95.32	94.23	94.77
LSTM+CRF	61.45	71.38	66.04	63.54	69.15	66.23	96.87	96.52	96.69
BiLSTM	83.21	84.61	83.90	86.42	88.64	87.52	98.28	99.63	98.95
BiLSTM+CRF	83.64	83.55	83.59	88.31	88.43	88.37	98.41	98.70	98.55
BiLSTM-LAN	84.36	84.12	84.24	88.45	88.79	88.62	98.36	98.11	98.23
PA-LSTM	84.65	85.21	84.93	89.16	88.05	88.60	97.54	97.44	97.49
C-GCN	88.72	88.13	88.42	90.32	90.47	90.39	98.13	98.32	98.22
RPA-GCN	89.33	89.18	89.25	91.56	91.02	91.29	98.34	97.95	98.14
PE-BiGAT	91.52	90.54	91.03	92.31	91.64	91.97	98.82	98.71	98.76

模型	C标签			E标签			O标签
模型	P	R	F1	P	R	F1	P	R	F1
LSTM	55.32	63.57	59.16	60.46	60.19	60.32	95.32	94.23	94.77
LSTM+CRF	61.45	71.38	66.04	63.54	69.15	66.23	96.87	96.52	96.69
BiLSTM	83.21	84.61	83.90	86.42	88.64	87.52	98.28	99.63	98.95
BiLSTM+CRF	83.64	83.55	83.59	88.31	88.43	88.37	98.41	98.70	98.55
BiLSTM-LAN	84.36	84.12	84.24	88.45	88.79	88.62	98.36	98.11	98.23
PA-LSTM	84.65	85.21	84.93	89.16	88.05	88.60	97.54	97.44	97.49
C-GCN	88.72	88.13	88.42	90.32	90.47	90.39	98.13	98.32	98.22
RPA-GCN	89.33	89.18	89.25	91.56	91.02	91.29	98.34	97.95	98.14
PE-BiGAT	91.52	90.54	91.03	92.31	91.64	91.97	98.82	98.71	98.76

数据集	单一因果关系数	链式因果关系数	多因果关系数
共计	3 054	652	214
Train set	2 576	458	102
Validation set	238	98	56
Test set	240	96	56

Complex causal relationship extraction based on prompt enhancement and bi-graph attention network

基于提示增强与双图注意力网络的复杂因果关系抽取

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 34

Related Articles 1

Recommended Articles

Metrics

模型	P	R	F1值
w/o syn	85.69	85.14	85.41
w/o sem	85.28	85.79	85.53
w/o kno	83.73	83.15	83.44
w/o mpt	81.25	81.94	81.59
PE-BiGAT	87.76	86.88	87.32