Entity relation extraction method for guidelines of cardiovascular disease based on bidirectional encoder representation from transformers

doi:10.11772/j.issn.1001-9081.2020061008

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (1): 145-149.DOI: 10.11772/j.issn.1001-9081.2020061008

Special Issue: 第八届中国数据挖掘会议(CCDM 2020)

• China Conference on Data Mining 2020 (CCDM 2020) • Previous Articles Next Articles

Entity relation extraction method for guidelines of cardiovascular disease based on bidirectional encoder representation from transformers

WU Xiaoping¹, ZHANG Qiang¹, ZHAO Fang², JIAO Lin²

1. School of Computer Science, Wuhan University, Wuhan Hubei 430072, China;
2. Department of Cardiovascular Disease, Zhongnan Hospital of Wuhan University, Wuhan Hubei 430070, China

Received:2020-05-31 Revised:2020-09-09 Online:2021-01-10 Published:2020-09-17
Supported by:
This work is partially supported by the Scientific Research Project of Hubei Provincial Health Committee (WJ2019M208).

基于BERT的心血管医疗指南实体关系抽取方法

武小平¹, 张强¹, 赵芳², 焦琳²

1. 武汉大学计算机学院, 武汉 430072;
2. 武汉大学中南医院心血管内科, 武汉 430070

通讯作者: 赵芳
作者简介:武小平(1974-),男,湖北钟祥人,副教授,博士,CCF会员,主要研究方向:知识图谱、物联网大数据;张强(1995-),男,江西上饶人,硕士研究生,主要研究方向:知识图谱;赵芳(1977-),女,湖北鄂州人,副主任医师,博士,主要研究方向:心血管疾病发病机制、心血管疾病大数据管理;焦琳(1989-),女,湖北武汉人,住院医师,硕士,主要研究方向:心血管疾病大数据管理。
基金资助:
湖北省卫生健康委员会面上项目（WJ2019M208）。

Abstract

Abstract: Entity relation extraction is a critical basic step of question answering, knowledge graph construction and information extraction in the medical field. In view of the fact that there is no open dataset available in the process of building knowledge graph specialized for cardiovascular disease, a professional training set for entity relation extraction of specialized cardiovascular disease knowledge graph was constructed by collecting some medical guidelines for cardiovascular disease and performing the corresponding professional labeling of the categories of entities and relations. Based on this dataset, firstly, Bidirectional Encoder Representation from Transformers and Convolutional Neural Network (BERT-CNN) model was proposed to realize the relation extraction in Chinese corpus. Then, the improved Bidirectional Encoder Representation from Transformers and Convolutional Neural Networks based on whole word mask (BERT(wwm)-CNN) model was proposed to improve the performance of relation extraction in Chinese corpus, according to the fact that word instead of character is the fundamental unit in Chinese. Experimental results show that, the improved BERT(wwm)-CNN model has the accuracy of 0.85, the recall of 0.80 and the F₁ value of 0.83 on the constructed relation extraction dataset, which are better than those of the comparison models, Bidirectional Encoder Representation from Transformers and Long Short Term Memory (BERT-LSTM) and BERT-CNN, verifying the superiority of the improved BERT(wwm)-CNN.

Key words: entity relation extraction, cardiovascular disease, Bidirectional Encoder Representation from Transformers (BERT) network, Convolutional Neural Network (CNN), knowledge graph

摘要： 实体关系抽取是医疗领域知识问答、知识图谱构建及信息抽取的重要基础环节之一。针对在心血管专病知识图谱构建的过程中尚无公开数据集可用的情况，收集了心血管疾病领域的医疗指南并进行相应的实体和关系类别的专业标注，构建了心血管专病知识图谱实体关系抽取的专业数据集。基于该数据集，首先提出双向变形编码器卷积神经网络（BERT-CNN）模型以实现中文语料中的关系抽取，然后根据中文语义中主要以词而不是字为基本单位的特性，提出了改进的基于全词掩模的双向变形编码器卷积神经网络（BERT（wwm）-CNN）模型用于提升在中文语料中关系抽取的性能。实验结果表明，改进的BERT（wwm）-CNN在所构建的关系抽取数据集上准确率达到0.85，召回率达到0.80，F₁值达到0.83，优于对比的基于双向变形编码器长短期记忆网络（BERT-LSTM）模型和BERT-CNN模型，验证了改进网络模型的优势。

关键词: 实体关系抽取, 心血管疾病, 双向变形编码器网络, 卷积神经网络, 知识图谱

CLC Number:

TP399

WU Xiaoping, ZHANG Qiang, ZHAO Fang, JIAO Lin. Entity relation extraction method for guidelines of cardiovascular disease based on bidirectional encoder representation from transformers[J]. Journal of Computer Applications, 2021, 41(1): 145-149.

武小平, 张强, 赵芳, 焦琳. 基于BERT的心血管医疗指南实体关系抽取方法[J]. 计算机应用, 2021, 41(1): 145-149.

References

[1] SHI P,LIN J. Simple BERT models for relation extraction and semantic role labeling[EB/OL].[2019-06-10]. https://arxiv.org/pdf/1904.05255v1.pdf.
[2] DEVLIN J,CHANG M,LEE K,et al. BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg, PA:Association for Computational Linguistics,2019:4171-4186.
[3] CUI Y,CHE W,LIU T,et al. Pre-training with whole word masking for Chinese BERT[EB/OL].[2019-10-29]. https://arxiv.org/pdf/1906.08101.pdf.
[4] AONE C, HALVERSON L, HAMPTON T, et al. SRA:description of the IE2 system used for MUC-7[EB/OL].[1998-04-29]. https://www.aclweb.org/anthology/M98-1012.pdf.
[5] ZHOU G,SU J,ZHANG J,et al. Exploring various knowledge in relation extraction[C]//Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2005:427-434.
[6] 鄂海红, 张文静, 肖思琪, 等. 深度学习实体关系抽取研究综述[J]. 软件学报,2019,30(6):1793-1818.(E H H,ZHANG W J, XIAO S Q,et al. Survey of entity relationship extraction based on deep learning[J]. Journal of Software,2019,30(6):1793-1818.)
[7] SOCHER R, HUVAL B, MANNING C D, et al. Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Stroudsburg, PA:Association for Computational Linguistics,2012:1201-1211.
[8] HASHIMOTO K, MIWA M, TSURUOKA Y, et al. Simple customization of recursive neural networks for semantic relation classification[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics,2013:1372-1376.
[9] BAHDANAU D,CHO K,BENGIO Y. Neural machine translation by jointly learning to align and translate[EB/OL].[2019-05-19]. https://arxiv.org/pdf/1409.0473v1.pdf.
[10] ZENG D, LIU K, LAI S, et al. Relation classification via convolutional deep neural network[C]//Proceedings of the 25th International Conference on Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2014:2335-2344.
[11] SAHU S,ANAND A,ORUGANTY K,et al. Relation extraction from clinical texts using domain invariant convolutional neural network[C]//Proceedings of the 15th Workshop on Biomedical Natural Language Processing. Stroudsburg,PA:Association for Computational Linguistics,2016:206-215.
[12] CHIKKA V R, KARLAPALEM K. A hybrid deep learning approach for medical relation extraction[EB/OL].[2018-01-26]. https://arxiv.org/pdf/1806.11189.pdf.
[13] NGUYEN D Q,VERSPOOR K. Convolutional neural networks for chemical-disease relation extraction are improved with characterbased word embeddings[C]//Proceedings of the 17th Workshop on Biomedical Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics,2018:129-136.
[14] VASWANI A,SHAZEER N,PARMAR N,et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook,NY:Curran Associates Inc.,2017:5998-6008.
[15] LI W,GAO S,ZHOU H,et al. The automatic text classification method based on BERT and feature union[C]//Proceedings of the IEEE 25th International Conference on Parallel and Distributed Systems. Piscataway:IEEE,2019:774-777.
[16] ZHANG W,JIANG S,ZHAO S,et al. A BERT-BiLSTM-CRF model for Chinese electronic medical records named entity recognition[C]//Proceedings of the 12th International Conference on Intelligent Computation Technology and Automation. Piscataway:IEEE,2019:166-169.
[17] 蔡国永, 林强, 任凯琪. 基于域对抗网络和BERT的跨领域文本情感分析[J]. 山东大学学报(工学版),2020, 50(1):1-7,20. (CAI G Y,LIN Q,REN K Q. Cross-domain text sentiment classification based on domain-adversarial network and BERT[J]. Journal of Shandong University(Engineering Science),2020,50(1):1-7,20.)
[18] 杨锦锋, 于秋滨, 关毅, 等. 电子病历命名实体识别和实体关系抽取研究综述[J]. 自动化学报,2014,40(8):1537-1562. (YANG J F,YU Q B,GUAN Y,et al. An overview of research on electronic medical record oriented named entity recognition and entity relation extraction[J]. Acta Automatica Sinica,2014,40(8):1537-1562.)
[19] DUAN D H,HUANG M X,MU Y Z. Research for building high performance communication service based on Netty protocol in smart health[C]//ICSPS 2017:Proceedings of the 9th International Conference on Signal Processing Systems. New York:ACM,2017:18-21.
[20] 李纲, 潘荣清, 毛进, 等. 整合BiLSTM-CRF网络和词典资源的中文电子病历实体识别[J]. 现代情报,2020,40(4):3-12,58. (LI G,PAN R Q,MAO J,et al. Entity recognition of Chinese electronic medical records based on BiLSTM-CRF network and dictionary resources[J]. Journal of Modern Information,2020,40(4):3-12,58.)
[21] WU Y,SCHUSTER M,CHEN Z,et al. Google's neural machine translation system:bridging the gap between human and machine translation[EB/OL].[2019-10-08]. https://arxiv.org/pdf/1609.08144.pdf.
[22] CHE W,LI Z,LIU T. LTP:a Chinese language technology platform[C]//Proceedings of the 23rd International Conference on Computational Linguistics:Demonstrations. Stroudsburg, PA:Association for Computational Linguistics,2010:13-16.
[23] ZHOU P,SHI W,TIAN J,et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2016:207-212.

Entity relation extraction method for guidelines of cardiovascular disease based on bidirectional encoder representation from transformers

基于BERT的心血管医疗指南实体关系抽取方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	LIU Yaxuan, ZHONG Yong. Joint extraction method of entities and relations based on subject attention [J]. Journal of Computer Applications, 2021, 41(9): 2517-2522.
[2]	WANG Hebing, ZHANG Chunmei. Facial landmark detection based on ResNeXt with asymmetric convolution and squeeze excitation [J]. Journal of Computer Applications, 2021, 41(9): 2741-2747.
[3]	LI Kangkang, ZHANG Jing. Multi-layer encoding and decoding model for image captioning based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(9): 2504-2509.
[4]	ZHANG Yongbin, CHANG Wenxin, SUN Lianshan, ZHANG Hang. Detection method of domains generated by dictionary-based domain generation algorithm [J]. Journal of Computer Applications, 2021, 41(9): 2609-2614.
[5]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[6]	XU Jianglang, LI Linyan, WAN Xinjun, HU Fuyuan. Indoor scene recognition method combined with object detection [J]. Journal of Computer Applications, 2021, 41(9): 2720-2725.
[7]	YU Dunhui, WAN Peng, WANG She. Entity association query system based on enterprise knowledge graph construction [J]. Journal of Computer Applications, 2021, 41(9): 2510-2516.
[8]	SONG Zhongshan, LIANG Jiarui, ZHENG Lu, LIU Zhenyu, TIE Jun. Remote sensing scene classification based on bidirectional gated scale feature fusion [J]. Journal of Computer Applications, 2021, 41(9): 2726-2735.
[9]	ZENG Xiangyin, ZHENG Bochuan, LIU Dan. Detection of left and right railway tracks based on deep convolutional neural network and clustering [J]. Journal of Computer Applications, 2021, 41(8): 2324-2329.
[10]	CAO Yuhong, XU Hai, LIU Sun'ao, WANG Zixiao, LI Hongliang. Review of deep learning-based medical image segmentation [J]. Journal of Computer Applications, 2021, 41(8): 2273-2287.
[11]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[12]	HUANG Chengcheng, DONG Xiaoxiao, LI Zhao. Deep pipeline 5×5 convolution method based on two-dimensional Winograd algorithm [J]. Journal of Computer Applications, 2021, 41(8): 2258-2264.
[13]	TIAN Ling, ZHANG Jinchuan, ZHANG Jinhao, ZHOU Wangtao, ZHOU Xue. Knowledge graph survey： representation， construction， reasoning and knowledge hypergraph theory [J]. Journal of Computer Applications, 2021, 41(8): 2161-2186.
[14]	WU Guangli, LI Leiting, GUO Zhenzhou, WANG Chengxiang. Video summarization generation model based on improved bi-directional long short-term memory network [J]. Journal of Computer Applications, 2021, 41(7): 1908-1914.
[15]	TAN Daoqiang, ZENG Cheng, QIAO Jinxia, ZHANG Jun. Shadow detection method based on hybrid attention model [J]. Journal of Computer Applications, 2021, 41(7): 2076-2081.