Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (4): 1080-1085.DOI: 10.11772/j.issn.1001-9081.2023040490
• Artificial intelligence • Previous Articles
Quan YUAN1,2, Changping CHEN1,2(), Ze CHEN1,2, Linfeng ZHAN1,2
Received:
2023-05-04
Revised:
2023-07-03
Accepted:
2023-07-10
Online:
2023-12-04
Published:
2024-04-10
Contact:
Changping CHEN
About author:
YUAN Quan, born in 1976, M. S., senior engineer. His research interests include big data, natural language processing.袁泉1,2, 陈昌平1,2(), 陈泽1,2, 詹林峰1,2
通讯作者:
陈昌平
作者简介:
袁泉(1976—),男,湖南邵阳人,正高级工程师,硕士,主要研究方向:大数据、自然语言处理CLC Number:
Quan YUAN, Changping CHEN, Ze CHEN, Linfeng ZHAN. Twice attention mechanism distantly supervised relation extraction based on BERT[J]. Journal of Computer Applications, 2024, 44(4): 1080-1085.
袁泉, 陈昌平, 陈泽, 詹林峰. 基于BERT的两次注意力机制远程监督关系抽取[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1080-1085.
Add to citation manager EndNote|Ris|BibTeX
URL: http://www.joca.cn/EN/10.11772/j.issn.1001-9081.2023040490
数据集 | 关系种类数 | 样本总数 | 测试集样本总数 | 测试集 |
---|---|---|---|---|
NYT-10d | 58 | 694 000 | 172 000 | Distant sup |
NYT-10m | 25 | 474 000 | 9 740 | Manual |
Tab. 1 Dataset details
数据集 | 关系种类数 | 样本总数 | 测试集样本总数 | 测试集 |
---|---|---|---|---|
NYT-10d | 58 | 694 000 | 172 000 | Distant sup |
NYT-10m | 25 | 474 000 | 9 740 | Manual |
参数名 | 符号 | 参数值 |
---|---|---|
词向量维度 | Embedding_dim | 768 |
学习率 | Lr | 10-5,2×10-5 |
句子最大长度 | Max_length | 512 |
批处理数 | Batch_size | 16,32,64 |
Dropout | Dropout | 0.5 |
Tab. 2 Model hyperparameters
参数名 | 符号 | 参数值 |
---|---|---|
词向量维度 | Embedding_dim | 768 |
学习率 | Lr | 10-5,2×10-5 |
句子最大长度 | Max_length | 512 |
批处理数 | Batch_size | 16,32,64 |
Dropout | Dropout | 0.5 |
方法 | AUC | P@M |
---|---|---|
文献[ | 10.7 | 49.2 |
PCNN-ATT | 34.1 | 69.4 |
TARE | 38.9 | 71.5 |
Tab. 3 Experimental results of different methods on NYT-10d dataset
方法 | AUC | P@M |
---|---|---|
文献[ | 10.7 | 49.2 |
PCNN-ATT | 34.1 | 69.4 |
TARE | 38.9 | 71.5 |
方法 | AUC | F1 | P@M |
---|---|---|---|
PCNN-ATT | 41.9 | 32.0 | 68.6 |
DISTRE | 35.7 | 31.4 | 65.1 |
CIL | 56.0 | 34.3 | 75.9 |
TARE | 54.1 | 38.3 | 87.2 |
Tab. 4 Experimental results of different methods on NYT-10m dataset
方法 | AUC | F1 | P@M |
---|---|---|---|
PCNN-ATT | 41.9 | 32.0 | 68.6 |
DISTRE | 35.7 | 31.4 | 65.1 |
CIL | 56.0 | 34.3 | 75.9 |
TARE | 54.1 | 38.3 | 87.2 |
模型 | NYT-10m | NYT-10d | |||
---|---|---|---|---|---|
AUC | F1 | P@M | AUC | P@M | |
TARE | 54.1 | 38.4 | 87.3 | 38.9 | 71.5 |
No Sentence-attention | 53.0 | 35.3 | 86.2 | 37.3 | 70.2 |
No self-attention | 51.3 | 32.4 | 85.3 | 34.7 | 69.4 |
Tab. 5 Ablation experiment results on NYT-10m and NYT-10d dataset
模型 | NYT-10m | NYT-10d | |||
---|---|---|---|---|---|
AUC | F1 | P@M | AUC | P@M | |
TARE | 54.1 | 38.4 | 87.3 | 38.9 | 71.5 |
No Sentence-attention | 53.0 | 35.3 | 86.2 | 37.3 | 70.2 |
No self-attention | 51.3 | 32.4 | 85.3 | 34.7 | 69.4 |
1 | ALT C, GABRYSZAK A, HENNIG L. Probing linguistic features of sentence-level representations in neural relation extraction[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020:1534-1545. 10.18653/v1/2020.acl-main.140 |
2 | TANG Y, HUANG J, WANG G, et al. Orthogonal relation transforms with graph context modeling for knowledge graph embedding[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020:2713-2722. 10.18653/v1/2020.acl-main.241 |
3 | WANG C, JIANG H. Explicit utilization of general knowledge in machine reading comprehension[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2019:2263-2272. 10.18653/v1/p19-1219 |
4 | SAXENA A, TRIPATHI A, TALUKDAR P. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020:4498-4507. 10.18653/v1/2020.acl-main.412 |
5 | WEI Z, SU J, WANG Y, et al. A novel cascade binary tagging framework for relational triple extraction[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020:1476-1488. 10.18653/v1/2020.acl-main.136 |
6 | ALT C, HÜBER M, HENNIG L. Fine-tuning pre-trained transformer language models to distantly supervised relation extraction[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2019:1388-1398. 10.18653/v1/p19-1134 |
7 | VEYSEH A P B, DERNONCOURT F, DOU D J, et al. Exploiting the syntax-model consistency for neural relation extraction[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020:8021-8032. 10.18653/v1/2020.acl-main.715 |
8 | MINTZ M, BILLS S, SNOW R, et al. Distant supervision for relation extraction without labeled data[C]// Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. Stroudsburg: ACL, 2009:1003-1011. 10.3115/1690219.1690287 |
9 | ZENG D, LIU K, LAI S, et al. Relation classification via convolutional deep neural network[C]// Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers. Stroudsburg: ACL, 2014:2335-2344. |
10 | QIN P, XU W, WANG W Y. DSGAN: generative adversarial training for distant supervision relation extraction[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: ACL, 2018:496-505. 10.18653/v1/p18-1046 |
11 | HOFFMANN R, ZHANG C, LING X, et al. Knowledge-based weak supervision for information extraction of overlapping relations[C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Stroudsburg: ACL, 2011:541-550. |
12 | RITTER A, ZETTLEMOYER L, MAUSAM, et al. Modeling missing data in distant supervision for information extraction[J]. Transactions of the Association for Computational Linguistics, 2013, 1:367-378. 10.1162/tacl_a_00234 |
13 | SURDEANU M, TIBSHIRANI J, NALLAPATI R, et al. Multi-instance multi-label learning for relation extraction[C]// Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Stroudsburg: ACL, 2012:455-465. |
14 | ZENG D, LIU K, CHEN Y, et al. Distant supervision for relation extraction via piecewise convolutional neural networks[C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2015:1753-1762. 10.18653/v1/d15-1203 |
15 | VASHISHTH S, JOSHI R, PRAYAGA S S, et al. RESIDE: improving distantly-supervised neural relation extraction using side information[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2018:1257-1266. 10.18653/v1/d18-1157 |
16 | CHEN T, SHI H, TANG S, et al. CIL: contrastive instance learning framework for distantly supervised relation extraction[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Stroudsburg: ACL, 2021:6191-6200. 10.18653/v1/2021.acl-long.483 |
17 | 王佳宇,李楹,马春梅,等. 融合实体信息的图卷积神经网络的短文本分类模型[J].天津师范大学学报(自然科学版),2023,43(1):67-72. |
WANG J Y, LI Y, MA C M, et al. Short text classification based on graph convolutional neural networks with entity information[J]. Journal Tianjin Normal University (Natural Science Edition), 2023,43(1):67-72. | |
18 | 唐焕玲,卫红敏,王育林, 等. 结合LDA与Word2vec的文本语义增强方法[J].计算机工程与应用,2022,58(13):135-145. 10.3778/j.issn.1002-8331.2112-0491 |
TANG H L, WEI H M, WANG Y L, et al. Text semantic enhancement method combining LDA and Word2vec[J]. Computer Engineering and Applications, 2022,58(13):135-145. 10.3778/j.issn.1002-8331.2112-0491 | |
19 | ZHOU P, SHI W, TIAN J, et al. Attention-based bidirectional long short-term memory networks for relation classification[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Stroudsburg: ACL, 2016:207-212. 10.18653/v1/p16-2034 |
20 | YE Z-X, LING Z-H. Distant supervision relation extraction with intra-bag and inter-bag attentions[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Stroudsburg: ACL, 2019:2810-2819. 10.18653/v1/n19-1288 |
21 | LI D, ZHANG T, HU N, et al. HiCLRE: a hierarchical contrastive learning framework for distantly supervised relation extraction[C]// Findings of the Association for Computational Linguistics: ACL 2022. Stroudsburg: ACL, 2022:2567-2578. 10.18653/v1/2022.findings-acl.202 |
22 | RATHORE V, BADOLA K, SINGLA P, et al. PARE: a simple and strong baseline for monolingual and multilingual distantly supervised relation extraction[C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Stroudsburg: ACL, 2022:340-354. 10.18653/v1/2022.acl-short.38 |
23 | RIEDEL S, YAO L, McCALLUM A. Modeling relations and their mentions without labeled text[C]// Proceedings of the 2010 Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Cham:Springer,2010:148-163. 10.1007/978-3-642-15939-8_10 |
24 | BHARTIYA A, BADOLA K, MAUSAM. DiS-ReX: a multilingual dataset for distantly supervised relation extraction[C]// Proceeding of the 60th Annual Meeting of the Association for Computational Linguistics(Volume 2: Short Papers). Stroudsburg: ACL, 2022:849-863. 10.18653/v1/2022.acl-short.95 |
25 | LIN Y, SHEN S, LIU Z, et al. Neural relation extraction with selective attention over instances[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: ACL, 2016:2124-2133. 10.18653/v1/p16-1200 |
[1] | Bin XIAO, Yun GAN, Min WANG, Xingpeng ZHANG, Zhaoxing WANG. Network abnormal traffic detection based on port attention and convolutional block attention module [J]. Journal of Computer Applications, 2024, 44(4): 1027-1034. |
[2] | Haihan WANG, Yan ZHU. Offensive speech detection with irony mechanism [J]. Journal of Computer Applications, 2024, 44(4): 1065-1071. |
[3] | Tao SUN, Zhangtian DUAN, Haonan ZHU, Peihao GUO, Heli SUN. Social event recommendation method based on unexpectedness metric [J]. Journal of Computer Applications, 2024, 44(3): 760-766. |
[4] | Yongfeng DONG, Jiaming BAI, Liqin WANG, Xu WANG. Chinese named entity recognition combining prior knowledge and glyph features [J]. Journal of Computer Applications, 2024, 44(3): 702-708. |
[5] | Zijie HUANG, Yang OU, Degang JIANG, Cailing GUO, Bailin LI. Lightweight deep learning algorithm for weld seam surface quality detection of traction seat [J]. Journal of Computer Applications, 2024, 44(3): 983-988. |
[6] | Rui JIANG, Wei LIU, Cheng CHEN, Tao LU. Asymmetric unsupervised end-to-end image deraining network [J]. Journal of Computer Applications, 2024, 44(3): 922-930. |
[7] | Aiguo SHANG, Xinjuan ZHU. Joint approach of intent detection and slot filling based on multi-task learning [J]. Journal of Computer Applications, 2024, 44(3): 690-695. |
[8] | Yuliang ZHENG, Yunhua CHEN, Weijie BAI, Pinghua CHEN. Vehicle target detection by fusing event data and image frames [J]. Journal of Computer Applications, 2024, 44(3): 931-937. |
[9] | Kui ZHAO, Huiqi QIU, Xu LI, Zhifei XU. Real-time pulmonary nodule detection algorithm combining attention and multipath fusion [J]. Journal of Computer Applications, 2024, 44(3): 945-952. |
[10] | Xinran LUO, Tianrui LI, Zhen JIA. Chinese medical named entity recognition based on self-attention mechanism and lexicon enhancement [J]. Journal of Computer Applications, 2024, 44(2): 385-392. |
[11] | Fuqin DENG, Huifeng GUAN, Chaoen TAN, Lanhui FU, Hongmin WANG, Tinlun LAM, Jianmin ZHANG. Multi-robot reinforcement learning path planning method based on request-response communication mechanism and local attention mechanism [J]. Journal of Computer Applications, 2024, 44(2): 432-438. |
[12] | Andi GUO, Zhen JIA, Tianrui LI. High-precision entity and relation extraction in medical domain based on pseudo-entity data augmentation [J]. Journal of Computer Applications, 2024, 44(2): 393-402. |
[13] | Weichao DANG, Lei ZHANG, Gaimei GAO, Chunxia LIU. Weakly supervised action localization method with snippet contrastive learning [J]. Journal of Computer Applications, 2024, 44(2): 548-555. |
[14] | Ziqi HUANG, Jianpeng HU. Entity category enhanced nested named entity recognition in automotive domain [J]. Journal of Computer Applications, 2024, 44(2): 377-384. |
[15] | Li’an CHEN, Yi GUO. Text sentiment analysis model based on individual bias information [J]. Journal of Computer Applications, 2024, 44(1): 145-151. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||