Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (7): 2011-2017.DOI: 10.11772/j.issn.1001-9081.2023081183
• Artificial intelligence • Previous Articles Next Articles
Yuan TANG1,2,3, Yanping CHEN1,2,3(), Ying HU1,2,3, Ruizhang HUANG1,2,3, Yongbin QIN1,2,3
Received:
2023-09-03
Revised:
2023-10-13
Accepted:
2023-10-17
Online:
2024-07-18
Published:
2024-07-10
Contact:
Yanping CHEN
About author:
TANG Yuan, born in 1999, M. S. candidate. Her research interests include natural language processing, information extraction.Supported by:
唐媛1,2,3, 陈艳平1,2,3(), 扈应1,2,3, 黄瑞章1,2,3, 秦永彬1,2,3
通讯作者:
陈艳平
作者简介:
唐媛(1999—),女,四川遂宁人,硕士研究生,主要研究方向:自然语言处理、信息抽取;基金资助:
CLC Number:
Yuan TANG, Yanping CHEN, Ying HU, Ruizhang HUANG, Yongbin QIN. Relation extraction model based on multi-scale hybrid attention convolutional neural networks[J]. Journal of Computer Applications, 2024, 44(7): 2011-2017.
唐媛, 陈艳平, 扈应, 黄瑞章, 秦永彬. 基于多尺度混合注意力卷积神经网络的关系抽取模型[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2011-2017.
Add to citation manager EndNote|Ris|BibTeX
URL: https://www.joca.cn/EN/10.11772/j.issn.1001-9081.2023081183
数据集 | 样本数 | 关系数 | ||
---|---|---|---|---|
训练集 | 测试集 | 验证集 | ||
SemEval | 8 000 | 2 717 | — | 19 |
TACRED | 68 124 | 22 631 | 25 509 | 42 |
Re-TACRED | 58 465 | 13 418 | 19 584 | 40 |
SciERC | 3 219 | 974 | 455 | 7 |
Tab. 1 Dataset information
数据集 | 样本数 | 关系数 | ||
---|---|---|---|---|
训练集 | 测试集 | 验证集 | ||
SemEval | 8 000 | 2 717 | — | 19 |
TACRED | 68 124 | 22 631 | 25 509 | 42 |
Re-TACRED | 58 465 | 13 418 | 19 584 | 40 |
SciERC | 3 219 | 974 | 455 | 7 |
参数 | 设置 |
---|---|
批次大小 | 16 |
迭代次数 | 20 |
随机失活率 | 0.5 |
学习率 | 1×10-3 |
多尺度卷积核大小 | [ |
词向量维度 | 768 |
特征图维度 | 64 |
Tab. 2 Parameters setting
参数 | 设置 |
---|---|
批次大小 | 16 |
迭代次数 | 20 |
随机失活率 | 0.5 |
学习率 | 1×10-3 |
多尺度卷积核大小 | [ |
词向量维度 | 768 |
特征图维度 | 64 |
数据集 | 模型 | 精确率 | 召回率 | F1值 |
---|---|---|---|---|
SemEval | R-BERT[ | — | — | 89.25 |
RE-DMP[ | — | — | 89.65 | |
A-GCN[ | — | — | 89.85 | |
BERT-CNN(基线) | 90.25 | 89.53 | 89.87 | |
本文模型 | 91.69 | 89.06 | 90.32 | |
TACRED | WGCN[ | 71.30 | 66.10 | 68.60 |
R-BERT[ | — | — | 69.40 | |
BERT-CNN(基线) | 71.70 | 68.96 | 70.31 | |
本文模型 | 72.98 | 68.63 | 70.74 | |
Re-TACRED | SpanBERT[ | — | — | 85.30 |
BERT-CNN(基线) | 84.79 | 85.84 | 85.31 | |
本文模型 | 86.93 | 84.53 | 85.71 | |
SciERC | REBEL[ | — | — | 86.30 |
BERT-CNN(基线) | 87.85 | 90.26 | 89.04 | |
本文模型 | 89.80 | 89.52 | 89.66 |
Tab. 3 Model comparison results on different dataset
数据集 | 模型 | 精确率 | 召回率 | F1值 |
---|---|---|---|---|
SemEval | R-BERT[ | — | — | 89.25 |
RE-DMP[ | — | — | 89.65 | |
A-GCN[ | — | — | 89.85 | |
BERT-CNN(基线) | 90.25 | 89.53 | 89.87 | |
本文模型 | 91.69 | 89.06 | 90.32 | |
TACRED | WGCN[ | 71.30 | 66.10 | 68.60 |
R-BERT[ | — | — | 69.40 | |
BERT-CNN(基线) | 71.70 | 68.96 | 70.31 | |
本文模型 | 72.98 | 68.63 | 70.74 | |
Re-TACRED | SpanBERT[ | — | — | 85.30 |
BERT-CNN(基线) | 84.79 | 85.84 | 85.31 | |
本文模型 | 86.93 | 84.53 | 85.71 | |
SciERC | REBEL[ | — | — | 86.30 |
BERT-CNN(基线) | 87.85 | 90.26 | 89.04 | |
本文模型 | 89.80 | 89.52 | 89.66 |
各模块的影响 | F1值/% |
---|---|
-通道注意力-空间注意力 | 89.32 |
-空间注意力 | 90.09 |
-通道注意力 | 90.06 |
通道空间并行 | 90.10 |
先空间后通道 | 90.19 |
先通道后空间 | 90.32 |
Tab. 4 Ablation experimental results
各模块的影响 | F1值/% |
---|---|
-通道注意力-空间注意力 | 89.32 |
-空间注意力 | 90.09 |
-通道注意力 | 90.06 |
通道空间并行 | 90.10 |
先空间后通道 | 90.19 |
先通道后空间 | 90.32 |
模型 | F1/% | 训练时间/s | 测试时间/s |
---|---|---|---|
BERT-CNN(基线) | 89.87 | 110 | 9 |
本文模型 | 90.32 | 158 | 14 |
Tab. 5 Comparative experimental results of running time and performance
模型 | F1/% | 训练时间/s | 测试时间/s |
---|---|---|---|
BERT-CNN(基线) | 89.87 | 110 | 9 |
本文模型 | 90.32 | 158 | 14 |
1 | XU K, REDDY S, FENG Y, et al. Question answering on freebase via relation extraction and textual evidence [C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: ACL, 2016: 2326-2336. |
2 | DISTIAWAN B, WEIKUM G, QI J, et al. Neural relation extraction for knowledge base enrichment [C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2019: 229-240. |
3 | SUN K, ZHANG R, MENSAH S, et al. Aspect-level sentiment analysis via convolution over dependency tree [C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg: ACL, 2019: 5679-5688. |
4 | MA Y, HIRAOKA T, OKAZAKI N. Joint entity and relation extraction based on table labeling using convolutional neural networks [C]// Proceedings of the 6th Workshop on Structured Prediction for NLP. Stroudsburg: ACL, 2022: 11-21. |
5 | ZENG D, LIU K, CHEN Y, et al. Distant supervision for relation extraction via piecewise convolutional neural networks [C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2015: 1753-1762. |
6 | 李子茂,张玥,尹帆,等.基于自注意力与分段卷积神经网络的实体关系抽取[J].中南民族大学学报(自然科学版), 2022, 41(3): 326-332. |
LI Z M, ZHANG Y, YIN F, et al. Entity relation extraction based on self-attention and piecewise convolutional neural network [J]. Journal of South-Central Minzu University(Natural Science Edition), 2022, 41(3): 326-332. | |
7 | VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Proceedings of the 31 st International Conference on Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc., 2017: 6000-6010. |
8 | MNIH V, HEESS N, GRAVES A. Recurrent models of visual attention [C]// Proceedings of the 27th International Conference on Neural Information Processing Systems: Volume 2. Cambridge: MIT Press, 2014: 2204-2212. |
9 | RINK B, HARABAGIU S. UTD: classifying semantic relations by combining lexical and semantic resources [C]// Proceedings of the 5th International Workshop on Semantic Evaluation. Stroudsburg: ACL, 2010: 256-259. |
10 | YANG Y, TONG Y, MA S, et al. A position encoding convolutional neural network based on dependency tree for relation classification [C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL 2016: 65-74. |
11 | QUAN C, HUA L, SUN X, et al. Multichannel convolutional neural network for biological relation extraction [J]. BioMed Research International, 2016, 2016: No.1850404. |
12 | CHEN Y, WANG K, YANG W, et al. A multi-channel deep neural network for relation extraction [J]. IEEE Access, 2020, 8: 13195-13203. |
13 | LEE J, SEO S, CHOI Y S. Semantic relation classification via bidirectional LSTM networks with entity-aware attention using latent entity typing [J]. Symmetry, 2019, 11(6): No.785. |
14 | 闫雄,段跃兴,张泽华.采用自注意力机制CNN融合的实体关系抽取[J].计算机工程与科学, 2020, 42(11): 2059-2066. |
YAN X, DUAN Y X, ZHANG Z H. Entity relationship extraction fusing self-attention mechanism and CNN [J]. Computer Engineering and Science, 2020, 42(11): 2059-2066. | |
15 | SHEN Y, HUANG X. Attention-based convolutional neural network for semantic relation extraction [C]// Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers. [S.l.]: The COLING 2016 Organizing Committee, 2016: 2526-2536. |
16 | TIAN Y, CHEN G, SONG Y, et al. Dependency-driven relation extraction with attentive graph convolutional networks [C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Stroudsburg: ACL, 2021: 4458-4471. |
17 | 隗昊,唐焕玲,周爱,等.基于双路分段注意力神经张量网络的临床文本关系抽取[J].电子学报, 2023, 51(3): 658-665. |
WEI H, TANG H L, ZHOU A, et al. Clinical relation via dual piecewise attention neural tensor network [J]. Acta Electronica Sinica, 2023, 51(3): 658-665. | |
18 | ZHOU L, WANG T, QU H, et al. A weighted GCN with logical adjacency matrix for relation extraction [C]// Proceedings of the 24th European Conference on Artificial Intelligence. Amsterdam: IOS Press, 2020: 2314-2321. |
19 | HENDRICKX I, KIM S N, KOZAREVA Z, et al. SemEval-2010 Task 8: multi-way classification of semantic relations between pairs of nominals[C]// Proceedings of the 5th International Workshop on Semantic Evaluation. Stroudsburg: ACL, 2010:28-33-38. |
20 | ZHANG Y, ZHONG V, CHEN D, et al. Position-aware attention and supervised data improve slot filling [C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2017: 35-45. |
21 | STOICA G, PLATANIOS E A, PÓCZOS B. Re-tacred: addressing shortcomings of the tacred dataset [C]// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press, 2021: 13843-13850. |
22 | LUAN Y, HE L, OSTENDORF M, et al. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction [C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2018: 3219-3232. |
23 | ZHANG Y, QI P, MANNING C D. Graph convolution over pruned dependency trees improves relation extraction [C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2018: 2205-2215. |
24 | LI B, YU D, YE W, et al. Sequence generation with label augmentation for relation extraction [C]// Proceedings of the 37th AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI Press, 2023: 13043-13050. |
25 | WU S, HE Y. Enriching pre-trained language model with entity information for relation classification [C]// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York: ACM, 2019: 2361-2364. |
26 | TIAN Y, SONG Y, XIA F. Improving relation extraction through syntax-induced pre-training with dependency masking [C]// Findings of the Association for Computational Linguistics: ACL 2022. Stroudsburg: ACL, 2022: 1875-1886. |
27 | JOSHI M, CHEN D, LIU Y, et al. SpanBERT: improving pre-training by representing and predicting spans [J]. Transactions of the Association for Computational Linguistics, 2020, 8: 64-77. |
28 | HUGUET CABOT P L, NAVIGLI R. REBEL: relation extraction by end-to-end language generation [C]// Findings of the Association for Computational Linguistics. Stroudsburg: ACL, 2021: 2370-2381. |
29 | LI J, FEI H, LIU J, et al. Unified named entity recognition as word-word relation classification [C]// Proceedings of the 36th AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI Press, 2022: 10965-10973. |
[1] | Yubo ZHAO, Liping ZHANG, Sheng YAN, Min HOU, Mao GAO. Relation extraction between discipline knowledge entities based on improved piecewise convolutional neural network and knowledge distillation [J]. Journal of Computer Applications, 2024, 44(8): 2421-2429. |
[2] | Tong CHEN, Fengyu YANG, Yu XIONG, Hong YAN, Fuxing QIU. Construction method of voiceprint library based on multi-scale frequency-channel attention fusion [J]. Journal of Computer Applications, 2024, 44(8): 2407-2413. |
[3] | Chenqian LI, Jun LIU. Ultrasound carotid plaque segmentation method based on semi-supervision and multi-scale cascaded attention [J]. Journal of Computer Applications, 2024, 44(8): 2604-2610. |
[4] | Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625. |
[5] | Dianhui MAO, Xuebo LI, Junling LIU, Denghui ZHANG, Wenjing YAN. Chinese entity and relation extraction model based on parallel heterogeneous graph and sequential attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2018-2025. |
[6] | Chao WEI, Yanping CHEN, Kai WANG, Yongbin QIN, Ruizhang HUANG. Relation extraction method based on mask prompt and gated memory network calibration [J]. Journal of Computer Applications, 2024, 44(6): 1713-1719. |
[7] | Quan YUAN, Changping CHEN, Ze CHEN, Linfeng ZHAN. Twice attention mechanism distantly supervised relation extraction based on BERT [J]. Journal of Computer Applications, 2024, 44(4): 1080-1085. |
[8] | Boyue WANG, Yingxiang LI, Jiandan ZHONG. Segmentation network for day and night ground-based cloud images based on improved Res-UNet [J]. Journal of Computer Applications, 2024, 44(4): 1310-1316. |
[9] | Andi GUO, Zhen JIA, Tianrui LI. High-precision entity and relation extraction in medical domain based on pseudo-entity data augmentation [J]. Journal of Computer Applications, 2024, 44(2): 393-402. |
[10] | Mengmeng CHEN, Zhiwei QIAO. Sparse reconstruction of CT images based on Uformer with fused channel attention [J]. Journal of Computer Applications, 2023, 43(9): 2948-2954. |
[11] | Meijia LIANG, Xinwu LIU, Xiaopeng HU. Small target detection algorithm for train operating environment image based on improved YOLOv3 [J]. Journal of Computer Applications, 2023, 43(8): 2611-2618. |
[12] | Kezheng CHEN, Xiaoran GUO, Yong ZHONG, Zhenping LI. Relation extraction method based on negative training and transfer learning [J]. Journal of Computer Applications, 2023, 43(8): 2426-2430. |
[13] | Menglin HUANG, Lei DUAN, Yuanhao ZHANG, Peiyan WANG, Renhao LI. Prompt learning based unsupervised relation extraction model [J]. Journal of Computer Applications, 2023, 43(7): 2010-2016. |
[14] | Jingsheng LEI, Kaijun LA, Shengying YANG, Yi WU. Joint entity and relation extraction based on contextual semantic enhancement [J]. Journal of Computer Applications, 2023, 43(5): 1438-1444. |
[15] | Kai ZHANG, Zhengchu QIN, Yue LIU, Xinyi QIN. Multi-learning behavior collaborated knowledge tracing model [J]. Journal of Computer Applications, 2023, 43(5): 1422-1429. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||