Named entity recognition model based on global information fusion and multi-dimensional relation perception

doi:10.11772/j.issn.1001-9081.2024050675

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (5): 1511-1519.DOI: 10.11772/j.issn.1001-9081.2024050675

• Artificial intelligence • Previous Articles

Named entity recognition model based on global information fusion and multi-dimensional relation perception

Jie HU¹^,²^,³, Shuaixing WU¹, Zhilan CAO¹^,²^,³(), Yan ZHANG¹^,²^,³

^1.School of Computer Science，Hubei University，Wuhan Hubei 430062，China
^2.Hubei Key Laboratory of Big Data Intelligent Analysis and Application （Hubei University），Wuhan Hubei 430062，China
^3.Engineering Research Center of Hubei Province in Intelligent Government Affairs and Application of Artificial Intelligence （Hubei University），Wuhan Hubei 430062，China

Received:2024-05-27 Revised:2024-08-09 Accepted:2024-08-30 Online:2024-09-05 Published:2025-05-10
Contact: Zhilan CAO
About author:HU Jie， born in 1977， Ph. D.， professor. Her research interests include complex semantic big data management， natural language processing.
WU Shuaixing， born in 2000， M. S. candidate. His research interests include natural language processing.
CAO Zhilan， born in 1971， M. S.， lecturer. Her research interests include natural language processing.
ZHANG Yan， born in 1974， Ph. D.， professor. His research interests include software engineering， information security.
Supported by:
National Natural Science Foundation of China(61977021)

基于全域信息融合和多维关系感知的命名实体识别模型

胡婕¹^,²^,³, 武帅星¹, 曹芝兰¹^,²^,³(), 张龑¹^,²^,³

^1.湖北大学计算机学院，武汉 430062
^2.大数据智能分析与行业应用湖北省重点实验室（湖北大学），武汉 430062
^3.智慧政务与人工智能应用湖北省工程研究中心（湖北大学），武汉 430062

通讯作者: 曹芝兰
作者简介:胡婕（1977—），女，湖北汉川人，教授，博士，主要研究方向：复杂语义大数据管理、自然语言处理
武帅星（2000—），男，河南安阳人，硕士研究生，主要研究方向：自然语言处理
曹芝兰（1971—），女，湖北麻城人，讲师，硕士，主要研究方向：自然语言处理
张龑（1974—），男，湖北宜昌人，教授，博士，CCF会员，主要研究方向：软件工程、信息安全。
基金资助:
国家自然科学基金资助项目(61977021)

Abstract

Abstract:

The existing Named Entity Recognition （NER） models based on Bidirectional Long Short-Term Memory （BiLSTM） network are difficult to fully understand the global semantics of text and capture the complex relationships between entities. Therefore， an NER model based on global information fusion and multi-dimensional relation perception was proposed. Firstly， BERT （Bidirectional Encoder Representations from Transformers） was used to obtain vector representation of the input sequence， and BiLSTM was combined to further learn context information of the input sequence. Secondly， a global information fusion mechanism composed of gradient stabilization layer and feature fusion module was proposed. With the former one， the model was able to maintain stable gradient propagation and update as well as optimize representation of the input sequence. In the latter one， the forward and backward representations of BiLSTM were integrated to obtain more comprehensive feature representation. Thirdly， a multi-dimensional relation perception structure was constructed to learn correlations between words in different subspaces in order to capture complex entity relationships in documents. In addition， the adaptive focus loss function was used to adjust the weights of different entity types dynamically to improve the recognition performance of the model for minority entities. Finally， experiments were conducted on 7 public datasets for the proposed model and 11 baseline models. The results show that all of the F1 values of the proposed model are higher than those of the comparison models， validating the comprehensive performance of the proposed model.

Key words: Named Entity Recognition (NER), global information fusion mechanism, gradient stabilization layer, multi-dimensional relation perception, adaptive focus loss

摘要：

现有的基于双向长短时记忆（BiLSTM）网络的命名实体识别（NER）模型难以全面理解文本的整体语义以及捕捉复杂的实体关系。因此，提出一种基于全域信息融合和多维关系感知的NER模型。首先，通过BERT （Bidirectional Encoder Representations from Transformers）获取输入序列的向量表示，并结合BiLSTM进一步学习输入序列的上下文信息。其次，提出由梯度稳定层和特征融合模块组成的全域信息融合机制：前者使模型保持稳定的梯度传播并更新优化输入序列的表示，后者则融合BiLSTM的前后向表示获取更全面的特征表示。接着，构建多维关系感知结构学习不同子空间单词的关联性，以捕获文档中复杂的实体关系。此外，使用自适应焦点损失函数动态调整不同类别实体的权重，提高模型对少数类实体的识别性能。最后，在7个公开数据集上将所提模型和11个基线模型进行对比，实验结果表明所提模型的F1值均优于对比模型，可见该模型的综合性较优。

关键词: 命名实体识别, 全域信息融合机制, 梯度稳定层, 多维关系感知, 自适应焦点损失

CLC Number:

TP391.1

Jie HU, Shuaixing WU, Zhilan CAO, Yan ZHANG. Named entity recognition model based on global information fusion and multi-dimensional relation perception[J]. Journal of Computer Applications, 2025, 45(5): 1511-1519.

胡婕, 武帅星, 曹芝兰, 张龑. 基于全域信息融合和多维关系感知的命名实体识别模型[J]. 《计算机应用》唯一官方网站, 2025, 45(5): 1511-1519.

Figures/Tables 7

References 25

1	邓依依，邬昌兴，魏永丰，等. 基于深度学习的命名实体识别综述［J］. 中文信息学报， 2021， 35（9）：30-45.
	DENG Y Y， WU C X， WEI Y F， et al. A survey on named entity recognition based on deep learning［J］. Journal of Chinese Information Processing， 2021， 35（9）： 30-45.
2	冯艳红，于红，孙庚，等. 基于BLSTM的命名实体识别方法［J］. 计算机科学， 2018， 45（2）：261-268.
	FENG Y H， YU H， SUN G， et al. Named entity recognition method based on BLSTM［J］. Computer Science， 2018， 45（2）： 261-268.
3	LIU Y， MENG F， ZHANG J， et al. GCDT： a global context enhanced deep transition architecture for sequence labeling［C］// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2019： 2431-2441.
4	SANTURKAR S， TSIPRAS D， ILYAS A， et al. How does batch normalization help optimization？［C］// Proceedings of the 32nd International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2018： 2488-2498.
5	范西朋，刘云飞，李盛阳，等. 基于MRC动态数据生成的命名实体识别方法［J］. 中文信息学报， 2023， 37（6）：104-114.
	FAN X P， LIU Y F， LI S Y， et al. A MRC dynamic data generation method for NER tasks［J］. Journal of Chinese Information Processing， 2023， 37（6）： 104-114.
6	ZHANG Y， WEI X S， ZHOU B， et al. Bag of tricks for long-tailed visual recognition with deep convolutional neural networks［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2021： 3447-3455.
7	BIKEL D M， MILLER S， SCHWARTZ R， et al. Nymble： a high-performance learning name-finder［C］// Proceedings of the 5th Conference on Applied Natural Language Processing. Stroudsburg： ACL， 1997： 194-201.
8	BIKEL D M， SCHWARTZ R， WEISCHEDEL R M. An algorithm that learns what's in a name［J］. Machine Learning， 1999， 34（1/2/3）： 211-231.
9	LAFFERTY J， McCALLUM A， PEREIRA F C N. Conditional random fields： probabilistic models for segmenting and labeling sequence data［C］// Proceedings of the 18th International Conference on Machine Learning. San Francisco： Morgan Kaufmann Publishers Inc.， 2001： 282-289.
10	SCHUSTER M， PALIWAL K K. Bidirectional recurrent neural networks［J］. IEEE Transactions on Signal Processing， 1997， 45（11）： 2673-2681.
11	ZHANG Y， YANG J. Chinese NER using lattice LSTM［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2018： 1554-1564.
12	LI X， YAN H， QIU X， et al. FLAT： Chinese NER using flat-lattice Transformer［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2020： 6836-6842.
13	MA R， PENG M， ZHANG Q， et al. Simplify the usage of lexicon in Chinese NER［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2020： 5951-5960.
14	蔡宇翔，骆妲，甘洋镭，等.基于跨度边界感知的嵌套命名实体识别［J］.软件学报，2024，35（11）：5149-5162.
	CAI Y X， LUO D， GAN Y L， et al. Nested named entity recognition based on span boundary perception［J］. Journal of Software， 2024， 35（11）： 5149-5162.
15	YU J， BOHNET B， POESIO M. Named entity recognition as dependency parsing［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2020： 6470-6476.
16	SHEN Y， MA X， TAN Z， et al. Locate and label： a two-stage identifier for nested named entity recognition［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg： ACL， 2021： 2782-2794.
17	ZHU E， LI J. Boundary smoothing for named entity recognition［C］// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2022： 7096-7108.
18	HUANG Z， XU W， YU K. Bidirectional LSTM-CRF models for sequence tagging［EB/OL］. ［2023-09-29］..
19	YAN H， GUI T， DAI J， et al. A unified generative framework for various NER subtasks［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg： ACL， 2021： 5808-5822.
20	YUAN Z， TAN C， HUANG S， et al. Fusing heterogeneous factors with triaffine mechanism for nested named entity recognition［C］// Findings of the Association for Computational Linguistics： ACL 2022. Stroudsburg： ACL， 2022： 3174-3186.
21	LI J， FEI H， LIU J， et al. Unified named entity recognition as word-word relation classification［C］// Proceedings of the 36th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2022： 10965-10973.
22	SHEN Y， SONG K， TAN X， et al. DiffusionNER： boundary diffusion for named entity recognition［C］// Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2023： 3875-3890.
23	WANG S， SUN X， LI X， et al. GPT-NER： named entity recognition via large language models［EB/OL］. ［2024-05-07］..
24	LIU R， WEI J， JIA C， et al. Modulating language models with emotions［C］// Findings of the Association for Computational Linguistics： ACL-IJCNLP 2021. Stroudsburg： ACL， 2021： 4332-4339.
25	LI X， WANG W， WU L， et al. Generalized focal loss： learning qualified and distributed bounding boxes for dense object detection［C］// Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2020： 21002-21012.

信息	集合划分	样本数
信息	集合划分	OntoNotes 4.0	MSRA	Resume NER	Weibo NER	CoNLL 2003	GENIA	CADEC
句子数	训练集	15 736	46 471	3 819	1 350	17 291	15 023	5 340
	验证集	4 306	4 376	463	270	3 453	1 669	1 097
	测试集	4 351	4 376	477	270	3 453	1 854	1 160
实体数	训练集	13 372	74 703	13 438	1 855	29 441	45 144	4 428
	验证集	6 950	6 181	1 497	379	5 648	5 365	898
	测试集	7 684	6 181	1 630	409	5 648	5 506	990

信息	集合划分	样本数
信息	集合划分	OntoNotes 4.0	MSRA	Resume NER	Weibo NER	CoNLL 2003	GENIA	CADEC
句子数	训练集	15 736	46 471	3 819	1 350	17 291	15 023	5 340
	验证集	4 306	4 376	463	270	3 453	1 669	1 097
	测试集	4 351	4 376	477	270	3 453	1 854	1 160
实体数	训练集	13 372	74 703	13 438	1 855	29 441	45 144	4 428
	验证集	6 950	6 181	1 497	379	5 648	5 365	898
	测试集	7 684	6 181	1 630	409	5 648	5 506	990

数据集	评价指标	Lattice-LSTM	FLAT	SoftLexicon	W2NER	Boundary Smooth	DiffusionNER	Biaffine	BARTNER	Locate and Label	Triaffine	GPT-NER	本文模型
OntoNotes 4.0	P	76.35	—	83.41	82.31	81.65	—	—	—	—	—	—	84.05
	R	71.56	—	82.21	83.36	84.03	—	—	—	—	—	—	82.68
	F₁	73.88	81.82	82.81	83.08	82.83	—	—	—	—	—	—	83.36
MSRA	P	93.58	—	95.75	96.12	96.37	95.71	—	—	—	—	—	96.37
	R	92.79	—	95.10	96.08	96.15	94.11	—	—	—	—	—	96.34
	F₁	93.18	96.09	95.42	96.10	96.26	94.91	—	—	—	—	—	96.36
Resume NER	P	94.81	—	96.08	96.96	96.63	—	—	—	—	—	—	96.69
	R	94.11	—	96.13	96.35	96.69	—	—	—	—	—	—	96.81
	F₁	94.46	95.86	96.11	96.65	96.66	—	—	—	—	—	—	96.75
Weibo NER	P	53.04	—	70.94	70.84	70.16	—	—	—	—	—	—	76.07
	R	62.25	—	67.02	73.87	75.36	—	—	—	—	—	—	72.95
	F₁	58.79	68.55	70.50	72.32	72.66	—	—	—	—	—	—	74.48
CoNLL 2003	P	—	—	—	92.71	93.61	92.99	92.46	92.61	—	—	88.54	93.83
	R	—	—	—	93.44	93.68	92.56	92.67	93.87	—	—	91.40	94.16
	F₁	—	—	—	93.07	93.65	92.78	92.55	93.24	—	—	89.97	93.99
GENIA	P	—	—	—	83.10	—	82.10	78.20	78.57	80.19	80.42	61.38	82.54
	R	—	—	—	79.76	—	80.97	78.20	79.30	80.89	82.06	66.74	82.10
	F₁	—	—	—	81.39	—	81.53	78.20	78.93	80.54	81.23	64.06	82.32
CADEC	P	—	—	—	74.09	—	—	—	70.08	—	—	—	79.77
	R	—	—	—	72.35	—	—	—	71.21	—	—	—	68.89
	F₁	—	—	—	73.21	—	—	—	70.64	—	—	—	73.93

数据集	评价指标	Lattice-LSTM	FLAT	SoftLexicon	W2NER	Boundary Smooth	DiffusionNER	Biaffine	BARTNER	Locate and Label	Triaffine	GPT-NER	本文模型
OntoNotes 4.0	P	76.35	—	83.41	82.31	81.65	—	—	—	—	—	—	84.05
	R	71.56	—	82.21	83.36	84.03	—	—	—	—	—	—	82.68
	F₁	73.88	81.82	82.81	83.08	82.83	—	—	—	—	—	—	83.36
MSRA	P	93.58	—	95.75	96.12	96.37	95.71	—	—	—	—	—	96.37
	R	92.79	—	95.10	96.08	96.15	94.11	—	—	—	—	—	96.34
	F₁	93.18	96.09	95.42	96.10	96.26	94.91	—	—	—	—	—	96.36
Resume NER	P	94.81	—	96.08	96.96	96.63	—	—	—	—	—	—	96.69
	R	94.11	—	96.13	96.35	96.69	—	—	—	—	—	—	96.81
	F₁	94.46	95.86	96.11	96.65	96.66	—	—	—	—	—	—	96.75
Weibo NER	P	53.04	—	70.94	70.84	70.16	—	—	—	—	—	—	76.07
	R	62.25	—	67.02	73.87	75.36	—	—	—	—	—	—	72.95
	F₁	58.79	68.55	70.50	72.32	72.66	—	—	—	—	—	—	74.48
CoNLL 2003	P	—	—	—	92.71	93.61	92.99	92.46	92.61	—	—	88.54	93.83
	R	—	—	—	93.44	93.68	92.56	92.67	93.87	—	—	91.40	94.16
	F₁	—	—	—	93.07	93.65	92.78	92.55	93.24	—	—	89.97	93.99
GENIA	P	—	—	—	83.10	—	82.10	78.20	78.57	80.19	80.42	61.38	82.54
	R	—	—	—	79.76	—	80.97	78.20	79.30	80.89	82.06	66.74	82.10
	F₁	—	—	—	81.39	—	81.53	78.20	78.93	80.54	81.23	64.06	82.32
CADEC	P	—	—	—	74.09	—	—	—	70.08	—	—	—	79.77
	R	—	—	—	72.35	—	—	—	71.21	—	—	—	68.89
	F₁	—	—	—	73.21	—	—	—	70.64	—	—	—	73.93

模型	OntoNotes 4.0	Weibo NER	CoNLL 2003	Resume NER	MSRA	GENIA	CADEC
本文模型	83.36	74.48	93.99	96.75	96.36	82.32	73.93
-BERT	58.09	52.31	47.49	93.74	74.89	39.27	39.30
-梯度稳定层	82.63	73.21	93.33	96.46	96.23	81.94	73.50
-特征融合	82.96	74.05	93.69	96.42	96.20	81.83	73.54
-多维关系感知结构	82.74	73.66	93.66	96.43	96.26	82.02	73.61
-自适应焦点损失函数	82.68	73.80	93.73	96.66	96.24	82.13	73.73

Named entity recognition model based on global information fusion and multi-dimensional relation perception

基于全域信息融合和多维关系感知的命名实体识别模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 25

Related Articles 15

Recommended Articles

Metrics

[1]	Biqing ZENG, Guangbin ZHONG, James Zhiqing WEN. Few-shot named entity recognition based on decomposed fuzzy span [J]. Journal of Computer Applications, 2025, 45(5): 1504-1510.
[2]	Xueqiang LYU, Tao WANG, Xindong YOU, Ge XU. HTLR： named entity recognition framework with hierarchical fusion of multi-knowledge [J]. Journal of Computer Applications, 2025, 45(1): 40-47.
[3]	Huanliang SUN, Siyi WANG, Junling LIU, Jingke XU. Help-seeking information extraction model for flood event in social media data [J]. Journal of Computer Applications, 2024, 44(8): 2437-2445.
[4]	Youren YU, Yangsen ZHANG, Yuru JIANG, Gaijuan HUANG. Chinese named entity recognition model incorporating multi-granularity linguistic knowledge and hierarchical information [J]. Journal of Computer Applications, 2024, 44(6): 1706-1712.
[5]	Yongfeng DONG, Jiaming BAI, Liqin WANG, Xu WANG. Chinese named entity recognition combining prior knowledge and glyph features [J]. Journal of Computer Applications, 2024, 44(3): 702-708.
[6]	Xiaoyan ZHANG, Zhengyu DUAN. Cross-lingual zero-resource named entity recognition model based on sentence-level generative adversarial network [J]. Journal of Computer Applications, 2023, 43(8): 2406-2411.
[7]	Jingsheng LEI, Kaijun LA, Shengying YANG, Yi WU. Joint entity and relation extraction based on contextual semantic enhancement [J]. Journal of Computer Applications, 2023, 43(5): 1438-1444.
[8]	Jie HU, Yan HU, Mengchi LIU, Yan ZHANG. Chinese named entity recognition based on knowledge base entity enhanced BERT model [J]. Journal of Computer Applications, 2022, 42(9): 2680-2685.
[9]	Guanyou XU, Weisen FENG. Python named entity recognition model based on transformer [J]. Journal of Computer Applications, 2022, 42(9): 2693-2700.
[10]	Yayao ZUO, Haoyu CHEN, Zhiran CHEN, Jiawei HONG, Kun CHEN. Named entity recognition method combining multiple semantic features [J]. Journal of Computer Applications, 2022, 42(7): 2001-2008.
[11]	Yi ZHANG, Shuangsheng WANG, Bin HE, Peiming YE, Keqiang LI. Named entity recognition method of elementary mathematical text based on BERT [J]. Journal of Computer Applications, 2022, 42(2): 433-439.
[12]	Lanlan ZENG, Yisong WANG, Panfeng CHEN. Named entity recognition based on BERT and joint learning for judgment documents [J]. Journal of Computer Applications, 2022, 42(10): 3011-3017.
[13]	Yue WANG, Mengxuan WANG, Sheng ZHANG, Wen DU. Alarm text named entity recognition based on BERT [J]. Journal of Computer Applications, 2020, 40(2): 535-540.
[14]	YAN Hong, CHEN Xingshu, WANG Wenxian, WANG Haizhou, YIN Mingyong. Recognition model for French named entities based on deep neural network [J]. Journal of Computer Applications, 2019, 39(5): 1288-1292.
[15]	ZHOU Xiang, LI Shaobo, YANG Guanci. Entity recognition of clothing commodity attributes [J]. Journal of Computer Applications, 2015, 35(7): 1945-1949.