Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (2): 433-439.DOI: 10.11772/j.issn.1001-9081.2021020334
Special Issue: 人工智能
• Artificial intelligence • Previous Articles Next Articles
Yi ZHANG, Shuangsheng WANG(), Bin HE, Peiming YE, Keqiang LI
Received:
2021-03-08
Revised:
2021-04-29
Accepted:
2021-04-30
Online:
2022-02-11
Published:
2022-02-10
Contact:
Shuangsheng WANG
About author:
ZHANG Yi, born in 1970, M. S., professor. His research interests include educational informatization, deep learning, machine learning.Supported by:
通讯作者:
王爽胜
作者简介:
张毅(1970—),男,重庆人,教授,硕士,主要研究方向:教育信息化、深度学习、机器学习;基金资助:
CLC Number:
Yi ZHANG, Shuangsheng WANG, Bin HE, Peiming YE, Keqiang LI. Named entity recognition method of elementary mathematical text based on BERT[J]. Journal of Computer Applications, 2022, 42(2): 433-439.
张毅, 王爽胜, 何彬, 叶培明, 李克强. 基于BERT的初等数学文本命名实体识别方法[J]. 《计算机应用》唯一官方网站, 2022, 42(2): 433-439.
Add to citation manager EndNote|Ris|BibTeX
URL: https://www.joca.cn/EN/10.11772/j.issn.1001-9081.2021020334
实体类别 | 标注符号 | 实体描述 | 示例 |
---|---|---|---|
角 | Angle | 角、二面角 | ∠ABC |
三角形 | Triangle | 直角三角形、锐角三角形 | 三角形BCD |
数列 | sequence | 数列、等差数列、等比数列 | 数列 |
点 | Point | 圆点、动点 | 点M(2,3) |
向量 | Vector | 向量、单位向量 | 向量 |
Tab. 1 Representation and examples of some entities
实体类别 | 标注符号 | 实体描述 | 示例 |
---|---|---|---|
角 | Angle | 角、二面角 | ∠ABC |
三角形 | Triangle | 直角三角形、锐角三角形 | 三角形BCD |
数列 | sequence | 数列、等差数列、等比数列 | 数列 |
点 | Point | 圆点、动点 | 点M(2,3) |
向量 | Vector | 向量、单位向量 | 向量 |
环境 | 配置 |
---|---|
操作系统 | Windows 10 |
CPU | Intel Core i7-8700k @3.7 GHz |
GPU | NVIDIA GeForce RTX 2080Ti(11 GB) |
Python | 3.7.1 |
TensorFlow | 1.13.1 |
内存 | 64 GB |
Tab. 2 Experimental environment
环境 | 配置 |
---|---|
操作系统 | Windows 10 |
CPU | Intel Core i7-8700k @3.7 GHz |
GPU | NVIDIA GeForce RTX 2080Ti(11 GB) |
Python | 3.7.1 |
TensorFlow | 1.13.1 |
内存 | 64 GB |
参数 | 值 | 参数 | 值 |
---|---|---|---|
Transformer层数 | 12 | Batch_size | 16 |
隐藏层维度 | 768 | attention_size | 128 |
优化器 | Adam | dropout | 0.5 |
学习率 | 0.000 05 | clip | 5.0 |
Lstm_dim | 128 |
Tab. 3 Model parameters
参数 | 值 | 参数 | 值 |
---|---|---|---|
Transformer层数 | 12 | Batch_size | 16 |
隐藏层维度 | 768 | attention_size | 128 |
优化器 | Adam | dropout | 0.5 |
学习率 | 0.000 05 | clip | 5.0 |
Lstm_dim | 128 |
模型 | 准确率 | 召回率 | F1值 |
---|---|---|---|
CRF | 67.06 | 47.61 | 55.68 |
IDCNN-CRF | 86.38 | 87.62 | 87.00 |
BiLSTM-CRF | 89.96 | 89.28 | 89.62 |
BiLSTM- Attention -CRF | 89.08 | 91.04 | 90.05 |
BiLSTM-IDCNN-CRF | 91.37 | 90.19 | 90.78 |
BERT-BiLSTM-CRF | 91.07 | 94.34 | 92.68 |
BERT-BiLSTM-IDCNN-CRF | 92.89 | 94.95 | 93.91 |
BERT-BiLSTM-IDCNN-Attention-CRF | 93.64 | 94.28 | 93.96 |
Tab. 4 Comparison of named entity recognition results of different models
模型 | 准确率 | 召回率 | F1值 |
---|---|---|---|
CRF | 67.06 | 47.61 | 55.68 |
IDCNN-CRF | 86.38 | 87.62 | 87.00 |
BiLSTM-CRF | 89.96 | 89.28 | 89.62 |
BiLSTM- Attention -CRF | 89.08 | 91.04 | 90.05 |
BiLSTM-IDCNN-CRF | 91.37 | 90.19 | 90.78 |
BERT-BiLSTM-CRF | 91.07 | 94.34 | 92.68 |
BERT-BiLSTM-IDCNN-CRF | 92.89 | 94.95 | 93.91 |
BERT-BiLSTM-IDCNN-Attention-CRF | 93.64 | 94.28 | 93.96 |
实体 | P | R | F1 |
---|---|---|---|
角 | 97.75 | 96.02 | 96.88 |
圆 | 66.67 | 66.67 | 66.67 |
锥体 | 100.00 | 100.00 | 100.00 |
方程 | 56.25 | 75.00 | 64.29 |
函数 | 94.12 | 96.55 | 95.32 |
线 | 89.41 | 94.41 | 91.84 |
点 | 81.65 | 95.70 | 88.12 |
四边形 | 100.00 | 100.00 | 100.00 |
数列 | 96.33 | 95.45 | 95.89 |
集合 | 97.62 | 100.00 | 98.80 |
面 | 100.00 | 88.89 | 94.12 |
三角形 | 100.00 | 100.00 | 100.00 |
向量 | 94.74 | 91.76 | 93.23 |
Tab. 5 Recognition results of BERT-BiLSTM-IDCNN-CRF to each entity
实体 | P | R | F1 |
---|---|---|---|
角 | 97.75 | 96.02 | 96.88 |
圆 | 66.67 | 66.67 | 66.67 |
锥体 | 100.00 | 100.00 | 100.00 |
方程 | 56.25 | 75.00 | 64.29 |
函数 | 94.12 | 96.55 | 95.32 |
线 | 89.41 | 94.41 | 91.84 |
点 | 81.65 | 95.70 | 88.12 |
四边形 | 100.00 | 100.00 | 100.00 |
数列 | 96.33 | 95.45 | 95.89 |
集合 | 97.62 | 100.00 | 98.80 |
面 | 100.00 | 88.89 | 94.12 |
三角形 | 100.00 | 100.00 | 100.00 |
向量 | 94.74 | 91.76 | 93.23 |
1 | 乐娟,赵玺.基于HMM的京剧机构命名实体识别算法[J].计算机工程, 2013, 39(6): 266-271, 286. 10.3969/j.issn.1000-3428.2013.06.059 |
LE J, ZHAO X. Algorithm of Beijing Opera organization names entity recognition based on HMM[J]. Computer Engineering, 2013, 39(6): 266-271, 286. 10.3969/j.issn.1000-3428.2013.06.059 | |
2 | 程健一,关毅,何彬.基于SVM和CRF双层分类器的英文电子病历去隐私化[J].智能计算机与应用, 2016, 6(6): 17-19, 24. 10.3969/j.issn.2095-2163.2016.06.005 |
CHENG J Y, GUAN Y, HE B. De-identification on electronic medical records using a two tier classifier based on SVM and CRF[J]. Intelligent Computer and Application, 2016, 6(6): 17-19, 24. 10.3969/j.issn.2095-2163.2016.06.005 | |
3 | 何彬,关毅.基于字级别条件随机场的医学实体识别[J].智能计算机与应用, 2019, 9(2): 130-134, 142. 10.3969/j.issn.2095-2163.2019.02.030 |
HE B, GUAN Y. Character-based CRF for medical entity recognition[J]. Intelligent Computer and Application, 2019, 9(2): 130-134, 142. 10.3969/j.issn.2095-2163.2019.02.030 | |
4 | HAMMERTON J. Named entity recognition with long short-term memory[C]// Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL. Stroudsburg, PA: Association for Computational Linguistics, 2003: 172-175. 10.3115/1119176.1119202 |
5 | LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition[C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2016: 260-270. 10.18653/v1/n16-1030 |
6 | TANG P, YANG P L, SHI Y, et al. Recognizing Chinese judicial named entity using BiLSTM-CRF[J]. Journal of Physics: Conference Series, 2020, 1592: No.012040. 10.1088/1742-6596/1592/1/012040 |
7 | 肖瑞,胡冯菊,裴卫.基于BiLSTM-CRF的中医文本命名实体识别[J].世界科学技术:中医药现代化, 2020, 22(7): 2504-2510. |
XIAO R, HU F J, PEI W. Chinese medicine text named entity recognition based on BiLSTM-CRF[J]. Modernization of Traditional Chinese Medicine — World Science and Technology, 2020, 22(7): 2504-2510. | |
8 | 李丽双,郭元凯.基于CNN-BLSTM-CRF模型的生物医学命名实体识别[J].中文信息学报, 2018, 32(1): 116-122. 10.3969/j.issn.1003-0077.2018.01.015 |
LI L S, GUO Y K. Biomedical named entity recognition with CNN-BLSTM-CRF[J]. Journal of Chinese Information Processing, 2018, 32(1): 116-122. 10.3969/j.issn.1003-0077.2018.01.015 | |
9 | LUO L, YANG Z H, YANG P, et al. An attention-based BiLSTM-CRF approach to document-level chemical named entity recognition[J]. Bioinformatics, 2018, 34(8): 1381-1388. 10.1093/bioinformatics/btx761 |
10 | 张晗,郭渊博,李涛.结合GAN与BiLSTM-Attention-CRF的领域命名实体识别[J].计算机研究与发展, 2019, 56(9): 1851-1858. 10.7544/issn1000-1239.2019.20180733 |
ZHANG H, GUO Y B, LI T. Domain named entity recognition combining GAN and BiLSTM-Attention-CRF[J]. Journal of Computer Research and Development, 2019, 56(9): 1851-1858. 10.7544/issn1000-1239.2019.20180733 | |
11 | STRUBELL E, VERGA P, BELANGER D, et al. Fast and accurate entity recognition with iterated dilated convolutions[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2017: 2670-2680. 10.18653/v1/d17-1283 |
12 | 吕江海,杜军平,周南,等.基于膨胀卷积迭代与注意力机制的实体名识别方法[J].计算机工程, 2021, 47(1): 58-65, 71. 10.19678/j.issn.1000-3428.0055986 |
LYU J H, DU J P, ZHOU N, et al. Entity name recognition method based on dilated convolution iterative and attention mechanism[J]. Computer Engineering, 2021, 47(1): 58-65, 71. 10.19678/j.issn.1000-3428.0055986 | |
13 | ZHANG Y, YANG J. Chinese NER using lattice LSTM[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2018: 1554-1564. 10.18653/v1/p18-1144 |
14 | PETERS M E, NEUMANN M, IYYER M, et al. Deep contextualized word representations[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics, Volume 1(Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2018: 2227-2237. 10.18653/v1/n18-1202 |
15 | RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding with unsupervised learning[EB/OL]. [2021-03-01]. . |
16 | DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1(Long and Short Papers). Stroudsburg, PA: Association for Computational Linguistics, 2019: 4171-4186. 10.18653/v1/n19-1423 |
17 | SOUZA F, NOGUEIRA R, LOTUFO R. Portuguese named entity recognition using BERT-CRF[EB/OL]. (2020-02-27) [2021-03-01]. . |
18 | 张秋颖,傅洛伊,王新兵.基于BERT-BiLSTM-CRF的学者主页信息抽取[J].计算机应用研究, 2020, 37(S1): 47-49. 10.1007/978-981-16-1843-7_14 |
ZHANG Q Y, FU L Y, WANG X B. Information extraction from scholar homepage based on BERT-BiLSTM-CRF[J]. Application Research of Computers, 2020, 37(S1): 47-49. 10.1007/978-981-16-1843-7_14 | |
19 | 王月,王孟轩,张胜,等.基于BERT的警情文本命名实体识别[J].计算机应用, 2020, 40(2): 535-540. 10.11772/j.issn.1001-9081.2019101717 |
WANG Y, WANG M X, ZHANG S, et al. Alarm text named entity recognition based on BERT[J]. Journal of Computer Applications, 2020, 40(2): 535-540. 10.11772/j.issn.1001-9081.2019101717 | |
20 | VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc., 2017: 6000-6010. 10.1016/s0262-4079(17)32358-8 |
21 | SZEGEDY C, IOFFE S, VANHOUCKE V, et al. Inception-v4, inception-ResNet and the impact of residual connections on learning[C]// Proceedings of the 31st AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI Press, 2017: 4278-4284. |
22 | BA J L, KIROS J R, HINTON G E. Layer normalization[EB/OL]. (2016-07-21) [2021-03-08]. . |
23 | 谢腾,杨俊安,刘辉.基于BERT-BiLSTM-CRF模型的中文实体识别[J].计算机系统应用,2020,29(07):48-55. |
XIE T, YANG J A, LIU H. Chinese entity recognition based on BERT-BiLSTM-CRF model [J]. Computer Systems & Applications, 2020, 29(7): 48-55. | |
24 | YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. (2016-04-30) [2021-03-08]. . 10.4236/psych.2020.1110096 |
25 | 杨文明,褚伟杰.在线医疗问答文本的命名实体识别[J].计算机系统应用, 2019, 28(2): 8-14. |
YANG W M, CHU W J. Named entity recognition of online medical questions answering text[J]. Computer Systems & Applications, 2019, 28(2): 8-14. |
[1] | Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974. |
[2] | Liting LI, Bei HUA, Ruozhou HE, Kuang XU. Multivariate time series prediction model based on decoupled attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2732-2738. |
[3] | Zhiqiang ZHAO, Peihong MA, Xinhong HEI. Crowd counting method based on dual attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2886-2892. |
[4] | Huanliang SUN, Siyi WANG, Junling LIU, Jingke XU. Help-seeking information extraction model for flood event in social media data [J]. Journal of Computer Applications, 2024, 44(8): 2437-2445. |
[5] | Kaipeng XUE, Tao XU, Chunjie LIAO. Multimodal sentiment analysis network with self-supervision and multi-layer cross attention [J]. Journal of Computer Applications, 2024, 44(8): 2387-2392. |
[6] | Yuqing WANG, Guangli ZHU, Wenjie DUAN, Shuyu LI, Ruotong ZHOU. Sentiment classification model of psychological counseling text based on attention over attention mechanism [J]. Journal of Computer Applications, 2024, 44(8): 2393-2399. |
[7] | Pengqi GAO, Heming HUANG, Yonghong FAN. Fusion of coordinate and multi-head attention mechanisms for interactive speech emotion recognition [J]. Journal of Computer Applications, 2024, 44(8): 2400-2406. |
[8] | Zhonghua LI, Yunqi BAI, Xuejin WANG, Leilei HUANG, Chujun LIN, Shiyu LIAO. Low illumination face detection based on image enhancement [J]. Journal of Computer Applications, 2024, 44(8): 2588-2594. |
[9] | Shangbin MO, Wenjun WANG, Ling DONG, Shengxiang GAO, Zhengtao YU. Single-channel speech enhancement based on multi-channel information aggregation and collaborative decoding [J]. Journal of Computer Applications, 2024, 44(8): 2611-2617. |
[10] | Li LIU, Haijin HOU, Anhong WANG, Tao ZHANG. Generative data hiding algorithm based on multi-scale attention [J]. Journal of Computer Applications, 2024, 44(7): 2102-2109. |
[11] | Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199. |
[12] | Dahai LI, Zhonghua WANG, Zhendong WANG. Dual-branch low-light image enhancement network combining spatial and frequency domain information [J]. Journal of Computer Applications, 2024, 44(7): 2175-2182. |
[13] | Wenliang WEI, Yangping WANG, Biao YUE, Anzheng WANG, Zhe ZHANG. Deep learning model for infrared and visible image fusion based on illumination weight allocation and attention [J]. Journal of Computer Applications, 2024, 44(7): 2183-2191. |
[14] | Wu XIONG, Congjun CAO, Xuefang SONG, Yunlong SHAO, Xusheng WANG. Handwriting identification method based on multi-scale mixed domain attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2225-2232. |
[15] | Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||