Abstract:

To address the problem that relevant models typically only model characters and relevant vocabulary without fully utilizing the unique glyph structure information and entity type information of Chinese characters， a model that integrates prior knowledge and glyph features for Named Entity Recognition （NER） task was proposed. Firstly， the input sequence was encoded using a Transformer combined with Gaussian attention mechanism， and the Chinese definitions of entity types were obtained from Chinese Wikipedia. Bidirectional Gated Recurrent Unit （BiGRU） was used to encode the entity type information as prior knowledge， which was combined with the character representation using an attention mechanism. Secondly， Bidirectional Long Short-Term Memory （BiLSTM） network was used to encode the long-distance dependency relationship of the input sequence， and a glyph encoding table was used to obtain traditional Chinese characters’ Cangjie codes and simplified Chinese characters’ modern Wubi codes. Then， Convolutional Neural Network （CNN） was used to extract glyph feature representations， and the traditional and simplified glyph feature representations were combined with different weights， which were then combined with the character representation encoded by BiLSTM using a gating mechanism. Finally， decoding was performed using Conditional Random Field （CRF） to obtain a sequence of named entity annotations. Experiment results on the colloquial dataset Weibo， the small dataset Boson， and the large dataset PeopleDaily show that， compared with the baseline model MECT （Multi-metadata Embedding based Cross-Transformer）， the proposed model has the F1 value increased by 2.47， 1.20， and 0.98 percentage points， respectively， proving the effectiveness of the proposed model.

Key words: Named Entity Recognition (NER), attention mechanism, Convolutional Neural Network (CNN), Bidirectional Long Short-Term Memory (BiLSTM), Conditional Random Field (CRF)

摘要：

针对命名实体识别（NER）任务中相关模型通常仅对字符及相关词汇进行建模，未充分利用汉字特有的字形结构信息和实体类型信息的问题，提出一种融合先验知识和字形特征的命名实体识别模型。首先，采用结合高斯注意力机制的Transformer对输入序列进行编码，并从中文维基百科中获取实体类型的中文释义，采用双向门控循环单元（BiGRU）编码实体类型信息作为先验知识，利用注意力机制将它与字符表示进行组合；其次，采用双向长短时记忆（BiLSTM）网络编码输入序列的远距离依赖关系，通过字形编码表获得繁体的仓颉码和简体的现代五笔码，采用卷积神经网络（CNN）提取字形特征表示，并根据不同权重组合繁体与简体字形特征，利用门控机制将它与经过BiLSTM编码后的字符表示进行组合；最后，使用条件随机场（CRF）解码，得到命名实体标注序列。在偏口语化的数据集Weibo、小型数据集Boson和大型数据集PeopleDaily上的实验结果表明，与基线模型MECT（Multi-metadata Embedding based Cross-Transformer）相比，所提模型的F1值别提高了2.47、1.20和0.98个百分点，验证了模型的有效性。

关键词: 命名实体识别, 注意力机制, 卷积神经网络, 双向长短时记忆, 条件随机场