Abstract:

Aiming at the problem of category vocabulary noise and label noise in weakly-supervised text classification tasks， a weakly-supervised text classification model with label semantic enhancement was proposed. Firstly， the category vocabulary was denoised on the basis of the contextual semantic representation of the words in order to construct a highly accurate category vocabulary. Then， a word category prediction task based on MASK mechanism was constructed to fine-tune the pre-training model BERT （Bidirectional Encoder Representations from Transformers）， so as to learn the relationship between words and categories. Finally， a self-training module with label semantics introduced was used to make full use of all data information and reduce the impact of label noise in order to achieve word-level to sentence-level semantic conversion， thereby accurately predicting text sequence categories. Experimental results show that compared with the current state-of-the-art weakly-supervised text classification model LOTClass （Label-name-Only Text Classification）， the proposed method improves the classification accuracy by 5.29， 1.41 and 1.86 percentage points respectively on the public datasets THUCNews， AG News and IMDB.

Key words: weakly-supervised text classification, BERT (Bidirectional Encoder Representations from Transformers), MASK mechanism, label semantics, label noise, self-training

摘要：

针对弱监督文本分类任务中存在的类别词表噪声和标签噪声问题，提出了一种标签语义增强的弱监督文本分类模型。首先，基于单词上下文语义表示对类别词表去噪，从而构建高度准确的类别词表；然后，构建基于MASK机制的词类别预测任务对预训练模型BERT进行微调，以学习单词与类别的关系；最后，利用引入标签语义的自训练模块来充分利用所有数据信息并减少标签噪声的影响，以实现词级到句子级语义的转换，从而准确预测文本序列类别。实验结果表明，与目前最先进的弱监督文本分类模型LOTClass相比，所提方法在THUCNews、AG News和IMDB公开数据集上，分类准确率分别提高了5.29、1.41和1.86个百分点。

关键词: 弱监督文本分类, BERT, MASK机制, 标签语义, 标签噪声, 自训练