基于数据增强和弱监督对抗训练的中文事件检测

doi:10.11772/j.issn.1001-9081.2021081521

《计算机应用》唯一官方网站 ›› 2022, Vol. 42 ›› Issue (10): 2990-2995.DOI: 10.11772/j.issn.1001-9081.2021081521

所属专题：人工智能

基于数据增强和弱监督对抗训练的中文事件检测

罗萍¹, 丁玲¹, 杨雪², 向阳¹

^1.同济大学电子与信息工程学院，上海 201804
^2.软通动力信息技术（集团）有限公司，河北廊坊 065000

收稿日期:2021-08-26 修回日期:2021-12-03 接受日期:2021-12-06 发布日期:2022-01-07 出版日期:2022-10-10
通讯作者: 向阳
作者简介:第一联系人：罗萍（1997—），女，安徽黄山人，硕士研究生，主要研究方向：自然语言处理、信息抽取、事件抽取
丁玲（1995—），女，山东淄博人，博士研究生，CCF会员，主要研究方向：自然语言处理、信息抽取、事件抽取
杨雪（1985—），女，河北廊坊人，主要研究方向：企业数字化、智慧城市
向阳（1962—），男，上海人，教授，博士，CCF会员，主要研究方向：机器学习、数据挖掘、自然语言处理。tjdxxiangyang@gmail.com
基金资助:
国家自然科学基金资助项目(72071145)

Chinese event detection based on data augmentation and weakly supervised adversarial training

Ping LUO¹, Ling DING¹, Xue YANG², Yang XIANG¹

^1.College of Electronics and Information Engineering，Tongji University，Shanghai 201804，China
^2.iSoftStone Information Technology （Group） Company Limited，Langfang Hebei 065000，China

Received:2021-08-26 Revised:2021-12-03 Accepted:2021-12-06 Online:2022-01-07 Published:2022-10-10
Contact: Yang XIANG
About author:LUO Ping， born in 1997， M. S. candidate. Her research interests include natural language processing， information extraction， event extraction.
DING Ling， born in 1995， Ph. D. candidate. Her research interests include natural language processing， information extraction， event extraction.
YANG Xue， born in 1985. Her research interests include enterprise digitalization， smart city.
XIANG Yang， born in 1962， Ph. D. ， professor. His research interests include machine learning， data mining， natural language processing.
Supported by:
National Natural Science Foundation of China(72071145)

摘要/Abstract

摘要：

当前的事件检测模型严重依赖于人工标注的数据，在标注数据规模有限的情况下，事件检测任务中基于完全监督方法的深度学习模型经常会出现过拟合的问题，而基于弱监督学习的使用自动标注数据代替耗时的人工标注数据的方法又常常依赖于复杂的预定义规则。为了解决上述问题，就中文事件检测任务提出了一种基于BERT的混合文本对抗训练（BMAD）方法。所提方法基于数据增强和对抗学习设定了弱监督学习场景，并采用跨度抽取模型来完成事件检测任务。首先，为改善数据不足的问题，采用回译、Mix-Text等数据增强方法来增强数据并为事件检测任务创建弱监督学习场景；然后，使用一种对抗训练机制进行噪声学习，力求最大限度地生成近似真实样本的生成样本，并最终提高整个模型的鲁棒性。在广泛使用的真实数据集自动文档抽取（ACE）2005上进行实验，结果表明相较于NPN、TLNN、HCBNN等算法，所提方法在F1分数上获取了至少0.84个百分点的提升。

关键词: 信息抽取, 中文事件检测, 数据增强, 弱监督学习, 对抗训练

Abstract:

The existing event detection models rely heavily on human-annotated data， and supervised deep learning models for event detection task often suffer from over-fitting when there is only limited labeled data. Methods of replacing time-consuming human annotation data with auto-labeled data typically rely on sophisticated pre-defined rules. To address these issues， a BERT （Bidirectional Encoder Representations from Transformers） based Mix-text ADversarial training （BMAD） method for Chinese event detection was proposed. In the proposed method， a weakly supervised learning scene was set on the basis of data augmentation and adversarial learning， and a span extraction model was used to solve event detection task. Firstly， to relieve the problem of insufficient data， various data augmentation methods such as back-translation and Mix-Text were applied to augment data and create weakly supervised learning scene for event detection. And then an adversarial training mechanism was applied to learn with noise and improve the robustness of the whole model. Several experiments were conducted on commonly used real-world dataset Automatic Context Extraction （ACE） 2005. The results show that compared with algorithms such as Nugget Proposal Network （NPN）， Trigger-aware Lattice Neural Network （TLNN） and Hybrid-Character-Based Neural Network （HCBNN）， the proposed method has the F1 score improved by at least 0.84 percentage points.

Key words: information extraction, Chinese event detection, data augmentation, weakly supervised learning, adversarial training

中图分类号:

TP182

罗萍, 丁玲, 杨雪, 向阳. 基于数据增强和弱监督对抗训练的中文事件检测[J]. 计算机应用, 2022, 42(10): 2990-2995.

Ping LUO, Ling DING, Xue YANG, Yang XIANG. Chinese event detection based on data augmentation and weakly supervised adversarial training[J]. Journal of Computer Applications, 2022, 42(10): 2990-2995.

图/表 3

参考文献 44

1	贺瑞芳，段绍杨. 基于多任务学习的中文事件抽取联合模型［J］. 软件学报， 2019， 30（4）：1015-1030.
	HE R F， DUAN S Y. Joint Chinese event extraction based multi-task learning［J］. Journal of Software， 2019， 30（4）： 1015-1030.
2	YANG H， CHUA T S， WANG S G， et al. Structured use of external knowledge for event-based open domain question answering［C］// Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2003：33-40. 10.1145/860435.860444
3	BASILE P， CAPUTO A， SEMERARO G， et al. Time event extraction to boost an information retrieval system［M］// LAI C， GIULIANI A， SEMERARO G. Information Filtering and Retrieval， SCI 668. Cham： Springer International Publishing， 2017 ：1-12.
4	CHENG P X， ERK K. Implicit argument prediction with event knowledge［C］// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2018： 831-840. 10.18653/v1/n18-1076
5	AHN D. The stages of event extraction［C］// Proceedings of the 2006 Workshop on Annotating and Reasoning about Time and Events. Stroudsburg， PA： Association for Computational Linguistics， 2006： 1-8. 10.3115/1629235.1629236
6	JI H， GRISHMAN R. Refining event extraction through cross-document inference［C］// Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg， PA： Association for Computational Linguistics， 2008： 254-262. 10.3115/1564169
7	LI Q， JI H， HUANG L. Joint event extraction via structured prediction with global features［C］// Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2013： 73-82.
8	ARAKI J， MITAMURA T. Joint event trigger identification and event coreference resolution with structured perceptron［C］// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2015： 2074-2080. 10.18653/v1/d15-1247
9	NGUYEN T H， GRISHMAN R. Event detection and domain adaptation with convolutional neural networks［C］// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing （Volume 2： Short Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2015： 365-371. 10.3115/v1/p15-2060
10	GHAEINI R， FERN X， HUANG L， et al. Event nugget detection with forward-backward recurrent neural networks［C］// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics （Volume 2： Short Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2016： 369-373. 10.18653/v1/p16-2060
11	WADDEN D， WENNBERG U， LUAN Y， et al. Entity， relation， and event extraction with contextualized span representations［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2019： 5784-5789. 10.18653/v1/d19-1585
12	CAO P F， CHEN Y B， ZHAO J， et al. Incremental event detection via knowledge consolidation networks［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2020： 707-717. 10.18653/v1/2020.emnlp-main.52
13	WANG Z Q， WANG X Z， HAN X， et al. CLEVE： contrastive pre-training for event extraction［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2021： 6283-6297. 10.18653/v1/2021.acl-long.491
14	XIE Q Z， DAI Z H， HOVY E， et al. Unsupervised data augmentation for consistency training［C/OL］// Proceedings of the 34th Conference on Neural Information Processing Systems. ［2021-04-29］..
15	ABDULMUMIN I， GALADANCI B S， ISA A. Iterative batch back-translation for neural machine translation： a conceptual model［EB/OL］. （2019-11-26）［2021-10-10］.. 10.1007/s10590-021-09284-y
16	PATWARDHAN S， RILOFF E. A unified model of phrasal and sentential evidence for information extraction［C］// Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2009： 151-160. 10.3115/1699510.1699530
17	LIAO S S， GRISHMAN R. Using document level cross-event inference to improve event extraction［C］// Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： Association for Computational Linguistics， 2010： 789-797.
18	McCLOSKY D， SURDEANU M， MANNING C D. Event extraction as dependency parsing［C］// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg， PA： Association for Computational Linguistics， 2011： 1626-1635.
19	HONG Y， ZHANG J F， MA B， et al. Using cross-entity inference to improve event extraction［C］// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg， PA： Association for Computational Linguistics， 2011： 1127-1136.
20	HUANG R H， RILOFF E. Modeling textual cohesion for event extraction［C］// Proceedings of the 26th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2012： 1664-1670.
21	LI Q， JI H， HONG Y， et al. Constructing information networks using one single model［C］// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2014： 1846-1851. 10.3115/v1/d14-1198
22	CHEN Y B， XU L H， LIU K， et al. Event extraction via dynamic multi-pooling convolutional neural networks［C］// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2015： 167-176. 10.3115/v1/p15-1017
23	NGUYEN T H， GRISHMAN R. Modeling skip-grams for event detection with convolutional neural networks［C］// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2016： 886-891. 10.18653/v1/d16-1085
24	LIU S L， CHEN Y B， LIU K， et al. Exploiting argument information to improve event detection via supervised attention mechanisms［C］// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2017： 1789-1798. 10.18653/v1/p17-1164
25	LIU S B， CHENG R， YU X M， et al. Exploiting contextual information via dynamic memory network for event detection［C］// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2018： 1030-1035. 10.18653/v1/d18-1127
26	YAN H R， JIN X L， MENG X B， et al. Event detection with multi-order graph convolution and aggregated attention［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2019： 5766-5770. 10.18653/v1/d19-1582
27	WANG Z H， GUO Y， WANG J H. Empower Chinese event detection with improved atrous convolution neural networks［J］. Neural Computing and Applications， 2021， 33（11）： 5805-5820. 10.1007/s00521-020-05360-1
28	CHEN Y B， LIU S L， ZHANG X， et al. Automatically labeled data generation for large scale event extraction［C］// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2017： 409-419. 10.18653/v1/p17-1038
29	ARAKI J， MITAMURA T. Open-domain event detection using distant supervision［C］// Proceedings of the 27th International Conference on Computational Linguistics. Stroudsburg， PA： Association for Computational Linguistics， 2018： 878-891.
30	ZENG Y， FENG Y S， MA R， et al. Scale up event extraction learning via automatic training data generation［C］// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2018： 6045-6052. 10.1609/aaai.v32i1.12030
31	HUANG L F， JI H. Semi-supervised new event type induction and event detection［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2020： 718-724. 10.18653/v1/2020.emnlp-main.53
32	SHAO Z H， SHANG L F， LIU Q， et al. A mutual information maximization approach for the spurious solution problem in weakly supervised question answering［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2021： 4111-4124. 10.18653/v1/2021.acl-long.318
33	GOOGFELLOW I J， POUGET-ABADIE J， MIRZA M， et al. Generative adversarial nets［C］// Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge： MIT Press， 2014：2672-2680.
34	HONG Y， ZHOU W X， ZHANG J L， et al. Self-regulation： employing a generative adversarial network to improve event detection［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2018： 515-526. 10.18653/v1/p18-1048
35	WANG X Z， HAN X， LIU Z Y， et al. Adversarial training for weakly supervised event detection［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long and Short Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2019： 998-1008. 10.18653/v1/n18-2
36	MA X Y， SHEN Y L， FANG G F， et al. Adversarial self-supervised data-free distillation for text classification［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2020： 6182-6192. 10.18653/v1/2020.emnlp-main.499
37	DEVLIN J， CHANG M W， LEE K， et al. BERT： pre-training of deep bidirectional trans-formers for language understanding［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long and Short Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2019： 4171-4186. 10.18653/v1/n18-2
38	YU J T， BOHNET B， POESIO M. Named entity recognition as dependency parsing［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： Association for Computational Linguistics， 2020： 6470-6476. 10.18653/v1/2020.acl-main.577
39	CHEN J A， YANG Z C， YANG D Y. MixText： linguistically-informed interpolation of hidden space for semi-supervised text classification［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： Association for Computational Linguistics， 2020： 2147-2157. 10.18653/v1/2020.acl-main.194
40	CHEN J A， WANG Z H， TIAN R， et al. Local additivity based data augmentation for semi-supervised NER［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2020： 1241-1251. 10.18653/v1/2020.emnlp-main.95
41	FENG X C， QIN B， LIU T. A language-independent neural network for event detection［J］. Science China （Information Sciences）， 2018， 61（9）： No.92106. 10.1007/s11432-017-9359-x
42	LIN H Y， LU Y J， HAN X P， et al. Nugget proposal networks for Chinese event detection［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2018： 1565-1574. 10.18653/v1/p18-1145
43	DING N， LI Z R， LIU Z Y， et al. Event detection with trigger-aware lattice neural network［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. PA： Association for Computational Linguistics， 2019： 347-356. 10.18653/v1/d19-1033
44	XI X Y， ZHANG T， YE W， et al. A hybrid character representation for Chinese event detection［C］// Proceedings of the 2019 International Joint Conference on Neural Networks. Piscataway： IEEE， 2019： 1-8. 10.1109/ijcnn.2019.8851786

模型	P	R	F1
HNN^［41］	77.10	53.10	63.00
NPN^［42］	60.90	69.30	64.80
TLNN^［43］	64.45	71.47	67.78
HCBNN^［44］	66.40	76.00	70.90
BMAD	73.94	69.67	71.74

模型	P	R	F1
HNN^［41］	77.10	53.10	63.00
NPN^［42］	60.90	69.30	64.80
TLNN^［43］	64.45	71.47	67.78
HCBNN^［44］	66.40	76.00	70.90
BMAD	73.94	69.67	71.74

模型	P	R	F1
Baseline	71.78	65.66	68.59
Baseline + Semi	72.80	66.42	69.46
Baseline + Mix	75.49	67.17	71.09
Baseline + Semi + Mix	72.97	69.67	71.28
BMAD （Baseline + Semi + Mix + Adv）	73.94	69.67	71.74

模型	P	R	F1
Baseline	71.78	65.66	68.59
Baseline + Semi	72.80	66.42	69.46
Baseline + Mix	75.49	67.17	71.09
Baseline + Semi + Mix	72.97	69.67	71.28
BMAD （Baseline + Semi + Mix + Adv）	73.94	69.67	71.74

[1]	杨莹, 郝晓燕, 于丹, 马垚, 陈永乐. 面向图神经网络模型提取攻击的图数据生成方法[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2483-2492.
[2]	游新冬, 问英姿, 佘鑫鹏, 吕学强. 面向煤矿机电设备领域的三元组抽取方法[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2026-2033.
[3]	毛典辉, 李学博, 刘峻岭, 张登辉, 颜文婧. 基于并行异构图和序列注意力机制的中文实体关系抽取模型[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2018-2025.
[4]	吴锦富, 柳毅. 基于随机噪声和自适应步长的快速对抗训练方法[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1807-1815.
[5]	沈君凤, 周星辰, 汤灿. 基于改进的提示学习方法的双通道情感分析模型[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1796-1806.
[6]	汪炅, 唐韬韬, 贾彩燕. 无负采样的正样本增强图对比学习推荐方法PAGCL[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1485-1492.
[7]	朱子蒙, 李志新, 郇战, 陈瑛, 梁久祯. 基于三元中心引导的弱监督视频异常检测[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1452-1457.
[8]	郭洁, 林佳瑜, 梁祖红, 罗孝波, 孙海涛. 基于知识感知和跨层次对比学习的推荐方法[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1121-1127.
[9]	郭安迪, 贾真, 李天瑞. 基于伪实体数据增强的高精准率医学领域实体关系抽取[J]. 《计算机应用》唯一官方网站, 2024, 44(2): 393-402.
[10]	宋逸飞, 柳毅. 基于数据增强和标签噪声的快速对抗训练方法[J]. 《计算机应用》唯一官方网站, 2024, 44(12): 3798-3807.
[11]	胡新荣, 陈静雪, 黄子键, 王帮超, 姚迅, 刘军平, 朱强, 杨捷. 基于图卷积网络的掩码数据增强[J]. 《计算机应用》唯一官方网站, 2024, 44(11): 3335-3344.
[12]	陈彤, 位纪伟, 何仕远, 宋井宽, 杨阳. 基于自适应攻击强度的对抗训练方法[J]. 《计算机应用》唯一官方网站, 2024, 44(1): 94-100.
[13]	张小艳, 段正宇. 基于句级别GAN的跨语言零资源命名实体识别模型[J]. 《计算机应用》唯一官方网站, 2023, 43(8): 2406-2411.
[14]	蔡引江, 许光俊, 马喜波. 图结构表示下的药物数据增强方法[J]. 《计算机应用》唯一官方网站, 2023, 43(4): 1136-1141.
[15]	许亮, 张春, 张宁, 田雪涛. 融合多Prompt模板的零样本关系抽取模型[J]. 《计算机应用》唯一官方网站, 2023, 43(12): 3668-3675.

基于数据增强和弱监督对抗训练的中文事件检测

Chinese event detection based on data augmentation and weakly supervised adversarial training

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 3

参考文献 44

相关文章 15

编辑推荐

Metrics