关键词: 多标签文本分类, 电力客服工单, 对比学习, 特征增强, 预训练语言模型

Abstract:

Multi-Label Text Classification （MLTC） of power customer service work orders has important significance for enhancing service efficiency and user satisfaction. Aiming at the problems of insufficient modeling of label relationships and class imbalance in MLTC of power customer service work orders， an MLTC method of power customer service work orders integrating feature enhancement and contrastive learning was proposed. Firstly， the text features of customer service work orders were extracted by using a pre-trained language model. Then， an innovative text feature enhancement method was developed by integrating global encoding module of multi-head attention mechanism and local encoding module of Convolutional Neural Network （CNN）. Finally， an MLTC framework of contrastive learning enhanced K-Nearest Neighbor （KNN） algorithm was introduced， positive samples were generated by using the R-Drop （Regularized Dropout） method， while negative samples were reweighted， and supervised contrastive learning loss function was incorporated during training to enhance the quality of neighbors retrieved by the KNN mechanism during inference， thereby effectively mitigating the negative impact of sample imbalance. Experimental results indicate that the proposed method achieves a micro-averaged F1 score of 92.17% on the power customer service work order dataset， surpassing the BERT （Bidirectional Encoder Representations from Transformers） model by 1.62 percentage points. Additionally， the proposed method achieves the micro-averaged F1 scores of 75.2% and 88.5%， respectively， on the public MLTC datasets AAPD and RCV1-V2， demonstrating the application value of improving work order processing accuracy and the service effectiveness in complex MLTC tasks.

Key words: Multi-Label Text Classification (MLTC), power customer service work order, contrastive learning, feature enhancement, pre-trained language model