Abstract:

Multi-Label Text Classification （MLTC） of power customer service work orders has important significance for enhancing service efficiency and user satisfaction. Aiming at the problems of insufficient modeling of label relationships and class imbalance in MLTC of power customer service work orders， an MLTC method of power customer service work orders integrating feature enhancement and contrastive learning was proposed. Firstly， the text features of customer service work orders were extracted by using a pre-trained language model. Then， an innovative text feature enhancement method was developed by integrating global encoding module of multi-head attention mechanism and local encoding module of Convolutional Neural Network （CNN）. Finally， an MLTC framework of contrastive learning enhanced K-Nearest Neighbor （KNN） algorithm was introduced， positive samples were generated by using the R-Drop （Regularized Dropout） method， while negative samples were reweighted， and supervised contrastive learning loss function was incorporated during training to enhance the quality of neighbors retrieved by the KNN mechanism during inference， thereby effectively mitigating the negative impact of sample imbalance. Experimental results indicate that the proposed method achieves a micro-averaged F1 score of 92.17% on the power customer service work order dataset， surpassing the BERT （Bidirectional Encoder Representations from Transformers） model by 1.62 percentage points. Additionally， the proposed method achieves the micro-averaged F1 scores of 75.2% and 88.5%， respectively， on the public MLTC datasets AAPD and RCV1-V2， demonstrating the application value of improving work order processing accuracy and the service effectiveness in complex MLTC tasks.

Key words: Multi-Label Text Classification (MLTC), power customer service work order, contrastive learning, feature enhancement, pre-trained language model

摘要：

电力客服工单多标签文本分类（MLTC）在提升服务效率与用户满意度方面具有重要意义。针对电力客服工单MLTC中的标签关系建模不足与类别不平衡问题，提出一种融合特征增强和对比学习的电力客服工单MLTC方法。首先，通过预训练语言模型提取客服工单文本特征；其次，结合多头注意力机制的全局编码与卷积神经网络（CNN）的局部编码模块，设计一种文本特征增强方法，以有效捕捉电力工单文本中的重要信息并提升特征表达能力；最后，引入对比学习改进的K最近邻（KNN）算法的MLTC框架，采用R-Drop （Regularized Dropout）方法生成正样本，而对负样本重新加权，并在训练中结合监督对比学习损失函数提高KNN机制推理期间检索到的邻居的质量，从而有效地缓解样本不平衡带来的负面影响。实验结果表明，所提方法在电力客服工单数据集上的微平均F1值为92.17%，较BERT（Bidirectional Encoder Representations from Transformers）模型提高了1.62个百分点；同时，所提方法在MLTC公共数据集AAPD和RCV1-V2上分别取得了75.2%和88.5%的微平均F1值，不仅在提升工单处理准确性和服务效率方面展现出较高的应用价值，而且在复杂MLTC任务中具备有效性。

关键词: 多标签文本分类, 电力客服工单, 对比学习, 特征增强, 预训练语言模型