关键词: 敏感信息, 数据集构建, 数据增强, BERT, 命名实体识别

Abstract:

Named Entity Recognition （NER） for sensitive information is a key technology of privacy protection. However， the existing NER methods face challenges in the sensitive information domain due to the scarcity of relevant datasets and the traditional techniques have problems such as low accuracy and poor portability. To address these issues， firstly， a sensitive information NER dataset， SenResume， was constructed by crawling and manually annotating text corpora containing sensitive information from the Internet. Secondly， a data augmentation model — Entity-based Masked Language Modeling （E-MLM） was proposed to utilize whole-word masking technique to generate new data samples， and expand the dataset to enhance data diversity. Thirdly， a RoBERTa-ResBiLSTM-CRF model was introduced， which combined the Robustly optimized Bidirectional Encoder Representations from Transformers approach with Whole Word Masking （RoBERTa-WWM） to extract contextual features for generating high-quality word vector representations， while ResBiLSTM （Residual Bidirectional Long Short-Term Memory） was employed to enhance text features. Finally， a multi-layer residual network was applied to improve training efficiency and model stability， and Conditional Random Field （CRF） was used for global decoding to enhance the accuracy of sequence labeling. Experimental results demonstrate that E-MLM improves dataset quality significantly， and the proposed NER model achieves the optimal performance on both the original and 1x augmented datasets， with F1 scores of 96.16% and 97.84%， respectively. It can be seen that the introduction of E-MLM and residual networks contribute to improvements in the accuracy of sensitive information NER.

Key words: sensitive information, dataset construction, data enhancement, Bidirectional Encoder Representations from Transformers (BERT), Named Entity Recognition (NER)