Abstract:

With the deepening of the network in the Medical Named Entity Recognition （MNER） problem， the recognition accuracy and computing power requirements of the deep learning-based recognition models are unbalanced. Aiming at this problem， a medical named entity recognition model CasSAttMNER （Cascade Self-Attention Medical Named Entity Recognition） based on deep auto-encoding was proposed. Firstly， a depth difference balance strategy between encoding and decoding was used in the model， and the distilled Transformer language model RBT6 was used as the encoder to reduce the encoding depth and the computing power requirements for training and application. Then， Bidirectional Long Short-Term Memory （BiLSTM） network and Conditional Random Field （CRF） were used to propose a cascaded multi-task dual decoder to complete entity mention sequence labeling and entity class determination. Finally， based on the self-attention mechanism， the model design was optimized by effectively representing the implicit decoding information between the entity classes and the entity mentions. Experimental results show that the F value measurements of CasSAttMNER on two Chinese medical entity datasets can reach 0.943 9 and 0.945 7， which are 3 percentage points and 8 percentage points higher than those of the baseline model， respectively， verifying that this model further improves the decoder performance.

Key words: named entity recognition, auto-encoding network, Bidirectional Long Short-Term Memory (BiLSTM) network, attention mechanism, multi-task

摘要：

针对在医疗命名实体识别（MNER）问题中随着网络加深，基于深度学习的识别模型出现的识别精度与算力要求不平衡的问题，提出一种基于深度自编码的医疗命名实体识别模型CasSAttMNER。首先，使用编码与解码间深度差平衡策略，以经过蒸馏的Transformer语言模型RBT6作为编码器以减小编码深度以及降低对训练和应用上的算力要求；然后，使用双向长短期记忆（BiLSTM）网络和条件随机场（CRF）提出了级联式多任务双解码器，从而完成实体提及序列标注与实体类别判断；最后，基于自注意力机制在实体类别中增加实体提及过程抽取的隐解码信息，以此来优化模型设计。实验结果表明，CasSAttMNER在两个中文医疗实体数据集上的F值度量可分别达到0.943 9和0.945 7，较基线模型分别提高了3个百分点和8个百分点，验证了该模型更进一步地提升了解码器性能。

关键词: 命名实体识别, 自编码网络, 双向长短期记忆网络, 注意力机制, 多任务