Abstract:

Traditional data augmentation techniques， such as synonym substitution， random insertion， and random deletion， may change the original semantics of text and even result in the loss of critical information. Moreover， data in text classification tasks typically have both textual and label parts. However， traditional data augmentation methods only focus on the textual part. To address these issues， a Label Confusion incorporated Data Augmentation （LCDA） technique was proposed for providing a comprehensive enhancement of data from both textual and label aspects. In terms of text， by enhancing the text through random insertion and replacement of punctuation marks and completing end-of-sentence punctuation marks， textual diversity was increased with all textual information and sequence preserved. In terms of labels， simulated label distribution was generated using a label confusion approach， and used to replace the traditional one-hot label distribution， so as to better reflect the relationships among instances and labels as well as between labels. In experiments conducted on few-shot datasets constructed from THUCNews （TsingHua University Chinese News） and Toutiao Chinese news datasets， the proposed technique was combined with TextCNN， TextRNN， BERT （Bidirectional Encoder Representations from Transformers）， and RoBERTa-CNN （Robustly optimized BERT approach Convolutional Neural Network） text classification models. The experimental results indicate that compared to those before enhancement， all models demonstrate significant performance improvements. Specifically， on 50-THU， a dataset constructed on THUCNews dataset， the accuracies of four models combing LCDA technique are improved by 1.19， 6.87， 3.21， and 2.89 percentage points， respectively， compared to those before enhancement， and by 0.78， 7.62， 1.75， and 1.28 percentage points， respectively， compared to those of the four models combining softEDA （Easy Data Augmentation with soft labels） method. By both textual and label processing results， model accuracy is enhanced by LCDA technique significantly， particularly in application scenarios characterized by limited data availability.

Key words: data augmentation, text classification, label confusion, Chinese news topic, pre-trained model

摘要：

传统数据增强技术，如同义词替换、随机插入和随机删除等，可能改变文本的原始语义，甚至导致关键信息丢失。此外，在文本分类任务中，数据通常包含文本部分和标签部分，然而传统数据增强方法仅针对文本部分。为解决这些问题，提出一种结合标签混淆的数据增强（LCDA）技术，从文本和标签这2个基本方面入手，为数据提供全面的强化。在文本方面，通过对文本进行标点符号随机插入和替换以及句末标点符号补齐等增强，在保留全部文本信息和顺序的同时增加文本的多样性；在标签方面，采用标签混淆方法生成模拟标签分布替代传统的one-hot标签分布，以更好地反映实例和标签与标签之间的关系。在THUCNews（TsingHua University Chinese News）和Toutiao这2个中文新闻数据集构建的小样本数据集上分别结合TextCNN、TextRNN、BERT（Bidirectional Encoder Representations from Transformers）和RoBERTa-CNN（Robustly optimized BERT approach Convolutional Neural Network）文本分类模型的实验结果表明，与增强前相比，性能均得到显著提升。其中，在由THUCNews数据集构造的50-THU数据集上，4种模型结合LCDA技术后的准确率相较于增强前分别提高了1.19、6.87、3.21和2.89个百分点；相较于softEDA（Easy Data Augmentation with soft labels）方法增强的模型分别提高了0.78、7.62、1.75和1.28个百分点。通过在文本和标签这2个维度的处理结果可知，LCDA技术能显著提升模型的准确率，在数据量较少的应用场景中表现尤为突出。

关键词: 数据增强, 文本分类, 标签混淆, 中文新闻主题, 预训练模型