Hierarchical multi-label classification model for public complaints with long-tailed distribution

doi:10.11772/j.issn.1001-9081.2024010085

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (1): 82-89.DOI: 10.11772/j.issn.1001-9081.2024010085

• Artificial intelligence • Previous Articles Next Articles

Hierarchical multi-label classification model for public complaints with long-tailed distribution

Xin LIU¹(), Dawei YANG¹, Changheng SHAO², Haiwen WANG¹, Mingjiang PANG¹, Yanru LI¹

^1.Qingdao Institute of Software，College of Computer Science and Technology，China University of Petroleum （East China），Qingdao Shandong 266580，China
^2.College of Computer Science and Technology，Qingdao University，Qingdao Shandong 266071，China

Received:2024-01-25 Revised:2024-03-25 Accepted:2024-03-27 Online:2024-05-09 Published:2025-01-10
Contact: Xin LIU
About author:YANG Dawei， born in 1997， M. S. candidate. His research interests include natural language processing， deep learning.
SHAO Changheng， born in 1986， Ph. D. His research interests include big data， smart city.
WANG Haiwen， born in 1993， M. S. candidate. His research interests include blockchain， natural language processing.
PANG Mingjiang， born in 2000， M. S. candidate. His research interests include natural language processing.
LI Yanru， born in 1998， M. S. candidate. Her research interests include natural language processing， federated learning.
Supported by:
National Natural Science Foundation of China(62071491);Shandong Provincial Natural Science Foundation(ZR2020MF045)

面向长尾分布的民众诉求层次多标签分类模型

刘昕¹(), 杨大伟¹, 邵长恒², 王海文¹, 庞铭江¹, 李艳茹¹

^1.中国石油大学（华东）青岛软件学院、计算机科学与技术学院，山东青岛 266580
^2.青岛大学计算机科学技术学院，山东青岛 266071

通讯作者: 刘昕
作者简介:杨大伟（1997—），男，江苏扬州人，硕士研究生，主要研究方向：自然语言处理、深度学习；
邵长恒（1986—），男，山东枣庄人，博士，主要研究方向：大数据、智慧城市；
王海文（1993—），男，山东淄博人，硕士研究生，主要研究方向：区块链、自然语言处理；
庞铭江（2000—），男，山东济南人，硕士研究生，主要研究方向：自然语言处理；
李艳茹（1998—），女，山东济南人，硕士研究生，主要研究方向：自然语言处理、联邦学习。
基金资助:
国家自然科学基金资助项目(62071491);山东省自然科学基金资助项目(ZR2020MF045)

Abstract

Abstract:

Swift response to public complaints is an important measure to realize intelligent social governance and improve people’s satisfaction. It is particularly crucial to analyze public complaints accurately to match work order processing departments intelligently， and to realize swift response and efficient handling of public complaints. However， the vague description of complaints， confusion of categories and imbalance of proportion in public complaint data lead to difficulties in analyzing categories of complaints， thus reducing the efficiency and accuracy of intelligent order dispatching. To solve the above problems， a hierarchical multi-label classification model （HMCHotline） for complaints with encoder-decoder structure was proposed. Firstly， the fine-grained keyword prior knowledge in complaint domain was introduced into the text encoder to suppress noise interference， and the spatio-temporal information in complaints was fused to improve the discriminant ability of semantic features. Secondly， the label hierarchy was used to generate label embeddings with hierarchy-awareness and semantic-awareness， and a label decoder based on the Transformer model was constructed to decode labels using the semantic features from the complaints and label features. At the same time，the dynamic label table strategy was introduced based on the hierarchical dependency to limit the decoding range of labels for solving the problem of label inconsistency. Finally， the Softmax grouping strategy was used to divide the label categories with the similar size into the same group for Softmax operation， which alleviated the problem of low classification accuracy caused by the long-tailed distribution of labels. Experimental results on Hotline， RCV1 （Reuters Corpus Volume I） -v2 and WOS （Web Of Science） datasets show that compared with Hierarchy-aware label semantics Matching network （HiMatch）， the proposed model improves the Micro-F1 by 1.65， 2.06 and 0.43 percentage points respectively， proving the effectiveness of the proposed model.

Key words: swift response to public complaints, intelligent order dispatching, hierarchical multi-label classification, priori knowledge, long-tailed distribution, encoder-decoder

摘要：

接诉即办是实现社会治理智能化、提高人民满意度的重要举措，其中精准分析民众诉求智能匹配工单处理部门，实现诉求的快速响应、高效办理尤为关键；然而，民众诉求数据中的诉求描述不清晰、类别混淆且比例失衡会导致诉求类别分析困难，影响了智能派单的效率与准确性。针对上述问题，提出编解码器结构的诉求层次多标签分类模型（HMCHotline）。首先，在文本编码器中引入诉求领域中的细粒度关键词先验知识以抑制噪声干扰，并融合诉求的时空信息提高语义特征的判别力；其次，利用标签层次结构生成具有层次与语义感知的标签嵌入，并构建基于Transformer模型的标签解码器，利用诉求的语义特征和标签嵌入进行标签解码；同时，在标签的层级依赖关系基础上引入动态标签表策略限制标签的解码范围，以解决标签不一致问题；最后，采用Softmax分组策略将样本数量相近的标签类别分为同组进行Softmax操作，从而缓解由标签长尾分布导致的分类准确率低的问题。在Hotline、RCV1 （Reuters Corpus Volume I）-v2和WOS （Web Of Science）数据集上的实验结果表明，相较于层次感知的标签语义匹配网络（HiMatch），所提模型的Micro-F1分别提高了1.65、2.06和0.43个百分点，验证了模型的有效性。

关键词: 接诉即办, 智能派单, 层次多标签分类, 先验知识, 长尾分布, 编解码器

CLC Number:

TP391

Xin LIU, Dawei YANG, Changheng SHAO, Haiwen WANG, Mingjiang PANG, Yanru LI. Hierarchical multi-label classification model for public complaints with long-tailed distribution[J]. Journal of Computer Applications, 2025, 45(1): 82-89.

刘昕, 杨大伟, 邵长恒, 王海文, 庞铭江, 李艳茹. 面向长尾分布的民众诉求层次多标签分类模型[J]. 《计算机应用》唯一官方网站, 2025, 45(1): 82-89.

Figures/Tables 9

References 27

1	雷斌，蓝羽石，黎茂林，等.智慧社会体系化建设总体构想与发展建议［J］.中国工程科学， 2023， 25（3）： 219-229.
	LEI B， LAN Y S， LI M L， et al. Overall conception and development suggestions for the systematic construction of smart society ［J］. Strategic Study of CAE， 2023， 25（3）： 219-229.
2	贾经冬，张敏南，赵祥，等.接诉即办智能派单业务调度算法研究［J］.计算机科学， 2023， 50（11A）： No.230300029.
	JIA J D， ZHANG M N， ZHAO X， et al. Study on scheduling algorithm of intelligent order dispatching ［J］. Computer Science， 2023， 50（11A）： No.230300029.
3	陈锋. “下交群评”：迈向人民主体性的基层治理——以北京市平谷区接诉即办改革为例［J］.北京工业大学学报（社会科学版）， 2024， 24（3）： 29-39.
	CHEN F. “Downward transfer and mass evaluation”： towards grass-roots governance of people's subjectivity — take the work of in the reform of immediate handling of complaints in Pinggu district of Beijing as an example ［J］. Journal of Beijing University of Technology （Social Sciences Edition）， 2024， 24（3）： 29-39.
4	肖琳，陈博理，黄鑫，等.基于标签语义注意力的多标签文本分类［J］.软件学报， 2020， 31（4）： 1079-1089.
	XIAO L， CHEN B L， HUANG X， et al. Multi-label text classification method based on label semantic information ［J］. Journal of Software， 2020， 31（4）： 1079-1089.
5	黄伟，刘贵全. MSML-BERT模型的层级多标签文本分类方法研究［J］.计算机工程与应用， 2022， 58（15）： 191-201.
	HUANG W， LIU G Q. Study on hierarchical multi-label text classification method of MSML-BERT model ［J］. Computer Engineering and Applications， 2022， 58（15）： 191-201.
6	MAO Y， TIAN J， HAN J， et al. Hierarchical text classification with reinforced label assignment ［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg： ACL， 2019： 445-455.
7	吕学强，彭郴，张乐，等.融合BERT与标签语义注意力的文本多标签分类方法［J］.计算机应用， 2022， 42（1）： 57-63.
	LYU X Q， PENG C， ZHANG L， et al. Text multi-label classification method incorporating BERT and label semantic attention ［J］. Journal of Computer Applications， 2022， 42（1）： 57-63.
8	JOHNSON R， ZHANG T. Effective use of word order for text categorization with convolutional neural networks ［C］// Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg： ACL， 2015： 103-112.
9	WANG Z， WANG P， HUANG L， et al. Incorporating hierarchy into text encoder： a contrastive learning approach for hierarchical text classification ［C］// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2022： 7109-7119.
10	ROJAS K R， BUSTAMANTE G， ONCEVAY A， et al. Efficient strategies for hierarchical text classification： external knowledge and auxiliary tasks ［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2020： 2252-2257.
11	WEHRMANN J， CERRI R， BARROS R. Hierarchical multi-label classification networks ［C］// Proceedings of the 35th International Conference on Machine Learning. New York： JMLR.org， 2018： 5075-5084.
12	SHIMURA K， LI J， FUKUMOTO F. HFT-CNN： learning hierarchical category structure for multi-label short text categorization ［C］// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2018： 811-816.
13	ZHOU J， MA C， LONG D， et al. Hierarchy-aware global model for hierarchical text classification ［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2020： 1106-1117.
14	CHEN H， MA Q， LIN Z， et al. Hierarchy-aware label semantics matching network for hierarchical text classification ［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg： ACL， 2021： 4370-4379.
15	DENG Z， PENG H， HE D， et al. HTCInfoMax： a global model for hierarchical text classification via information maximization ［C］// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg： ACL， 2021： 3259-3265.
16	JIANG T， WANG D， SUN L， et al. Exploiting global and local hierarchies for hierarchical text classification ［C］// Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2022： 4030-4039.
17	YU C， SHEN Y， MAO Y. Constrained sequence-to-tree generation for hierarchical text classification ［C］// Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2022： 1865-1869.
18	ZHAO W， ZHAO H. Hierarchical long-tailed classification based on multi-granularity knowledge transfer driven by multi-scale feature fusion ［J］. Pattern Recognition， 2024， 145： No.109842.
19	ZHAO J， LI J， FUKUMOTO F. Hierarchy-aware bilateral-branch network for imbalanced hierarchical text classification ［C］// Proceedings of the 2023 International Conference on Database and Expert Systems Applications， LNCS 14147. Cham： Springer， 2023： 143-157.
20	ZHAO X， LI Z， ZHANG X， et al. An interactive fusion model for hierarchical multi-label text classification ［C］// Proceedings of the 11th CCF International Conference on Natural Language Processing and Chinese Computing， LNCS 13552. Cham： Springer， 2022： 168-178.
21	WANG B， HU X， LI P， et al. Cognitive structure learning model for hierarchical multi-label text classification ［J］. Knowledge-Based Systems， 2021， 218： No.106876.
22	SUN Y， QIU H， ZHENG Y， et al. SIFRank： a new baseline for unsupervised keyphrase extraction based on pre-trained language model ［J］. IEEE Access， 2020， 8： 10896-10906.
23	DEVLIN J， CHANG M W， LEE K， et al. BERT： pre-training of deep bidirectional Transformers for language understanding ［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1（Long and Short Papers）. Stroudsburg： ACL， 2019： 4171-4186.
24	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need ［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2017： 6000-6010.
25	WILLIAMS R J， ZIPSER D. A learning algorithm for continually running fully recurrent neural networks ［J］. Neural Computation， 1989， 1（2）： 270-280.
26	LEWIS D D， YANG Y， ROSE T G， et al. RCV1： a new benchmark collection for text categorization research ［J］. Journal of Machine Learning Research， 2004， 5： 361-397.
27	KOWSARI K， BROWN D E， HEIDARYSAFA M， et al. HDLTex： hierarchical deep learning for text classification ［C］// Proceedings of the 16th IEEE International Conference on Machine Learning and Applications. Piscataway： IEEE， 2017： 364-371.

数据集	标签数	深度	平均标签深度	样本数
数据集	标签数	深度	平均标签深度	训练集	验证集	测试集
Hotline	1 568	5	3.82	528 595	176 198	176 198
RCV1-v2^［26］	103	4	3.24	20 833	2 316	781 265
WOS^［27］	141	2	2.00	30 070	7 518	9 397

数据集	标签数	深度	平均标签深度	样本数
数据集	标签数	深度	平均标签深度	训练集	验证集	测试集
Hotline	1 568	5	3.82	528 595	176 198	176 198
RCV1-v2^［26］	103	4	3.24	20 833	2 316	781 265
WOS^［27］	141	2	2.00	30 070	7 518	9 397

模型	Hotline				RCV1-v2^［26］				WOS^［27］
模型	Micro-F1	C-Micro-F1	Macro-F1	C-Macro-F1	Micro-F1	C-Micro-F1	Macro-F1	C-Macro-F1	Micro-F1	C-Micro-F1	Macro-F1	C-Macro-F1
TextRCNN^［13］	73.81	72.98	58.72	58.04	81.57	—	59.25	—	83.55	—	76.99	—
HiAGM^［13］	76.62	75.23	61.13	60.56	83.96	83.05	63.35	59.64	85.82	85.35	80.28	79.84
HTCinfoMax^［15］	76.54	75.16	60.89	59.94	83.51	—	62.71	—	85.58	—	80.05	—
HiMatch^［14］	77.16	75.92	61.28	60.33	84.73	83.49	64.11	60.64	86.20	85.61	80.53	79.32
BERT^［14］	77.23	76.09	61.34	60.45	85.65	—	67.02	—	85.63	—	79.07	—
HiAGM-BERT^［17］	77.79	77.03	62.98	62.15	85.58	—	67.93	—	86.04	—	80.19	—
HiMatch-BERT^［14］	78.05	76.79	63.14	62.39	86.33	85.25	68.66	67.15	86.70	85.74	81.06	79.86
HMCHotline	78.81	78.81	63.76	63.76	86.79	86.79	69.54	69.54	86.63	86.63	80.87	80.87

模型	Hotline				RCV1-v2^［26］				WOS^［27］
模型	Micro-F1	C-Micro-F1	Macro-F1	C-Macro-F1	Micro-F1	C-Micro-F1	Macro-F1	C-Macro-F1	Micro-F1	C-Micro-F1	Macro-F1	C-Macro-F1
TextRCNN^［13］	73.81	72.98	58.72	58.04	81.57	—	59.25	—	83.55	—	76.99	—
HiAGM^［13］	76.62	75.23	61.13	60.56	83.96	83.05	63.35	59.64	85.82	85.35	80.28	79.84
HTCinfoMax^［15］	76.54	75.16	60.89	59.94	83.51	—	62.71	—	85.58	—	80.05	—
HiMatch^［14］	77.16	75.92	61.28	60.33	84.73	83.49	64.11	60.64	86.20	85.61	80.53	79.32
BERT^［14］	77.23	76.09	61.34	60.45	85.65	—	67.02	—	85.63	—	79.07	—
HiAGM-BERT^［17］	77.79	77.03	62.98	62.15	85.58	—	67.93	—	86.04	—	80.19	—
HiMatch-BERT^［14］	78.05	76.79	63.14	62.39	86.33	85.25	68.66	67.15	86.70	85.74	81.06	79.86
HMCHotline	78.81	78.81	63.76	63.76	86.79	86.79	69.54	69.54	86.63	86.63	80.87	80.87

模型	整体			头部		中部		尾部
模型	Macro-F1	Macro-P	Macro-R	Macro-P	Macro-R	Macro-P	Macro-R	Macro-P	Macro-R
TextRCNN^［13］	73.81	70.58	77.13	75.12	80.57	72.85	78.22	43.51	61.35
HiAGM^［13］	76.62	72.33	79.48	79.65	82.53	74.46	80.53	44.82	63.40
HTCinfoMax^［15］	76.54	72.09	79.56	80.93	82.52	74.61	80.95	42.77	62.34
HiMatch^［14］	77.16	73.98	80.18	81.41	83.43	74.80	80.87	43.02	65.57
BERT^［14］	77.23	73.79	81.21	81.57	83.75	74.98	80.96	44.35	66.21
HiAGM-BERT^［17］	77.79	74.16	81.67	81.65	83.58	75.14	81.13	44.20	65.27
HiMatch-BERT^［14］	78.05	74.79	82.03	82.09	84.78	76.22	81.17	45.98	67.08
HMCHotline	78.81	75.02	82.42	81.86	84.55	78.16	82.41	51.58	70.14

Hierarchical multi-label classification model for public complaints with long-tailed distribution

面向长尾分布的民众诉求层次多标签分类模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 27

Related Articles 13

Recommended Articles

Metrics

模型	Micro-F1	Macro-F1
BERT	77.23	61.34
BERT-Keyword	77.91	61.89
BERT-Date	77.30	61.39
BERT-Location	77.25	61.36
BERT-TEPK	77.94	61.93

模型	RCV1-v2^［26］	WOS^［27］	Hotline
HiAGM^［13］	0.91	0.47	1.39
HiMatch^［14］	1.24	0.59	1.24
HiMatch-BERT^［14］	1.08	0.96	1.26
HMCHotline	0.00	0.00	0.00

[1]	Zhigang XU, Chuang ZHANG. Multi-level color restoration of mural image based on gated positional encoding [J]. Journal of Computer Applications, 2024, 44(9): 2931-2937.
[2]	Rui JIANG, Wei LIU, Cheng CHEN, Tao LU. Asymmetric unsupervised end-to-end image deraining network [J]. Journal of Computer Applications, 2024, 44(3): 922-930.
[3]	Weina DONG, Jia LIU, Xiaozhong PAN, Lifeng CHEN, Wenquan SUN. High-capacity robust image steganography scheme based on encoding-decoding network [J]. Journal of Computer Applications, 2024, 44(3): 772-779.
[4]	Guolong YUAN, Yujin ZHANG, Yang LIU. Image tampering forensics network based on residual feedback and self-attention [J]. Journal of Computer Applications, 2023, 43(9): 2925-2931.
[5]	Zhirong HOU, Xiaodong FAN, Hua ZHANG, Xiaonan MA. J-SGPGN： paraphrase generation network based on joint learning of sequence and graph [J]. Journal of Computer Applications, 2023, 43(5): 1365-1371.
[6]	Lu CHEN, Daoxi CHEN, Yiming LU, Weizhong LU. Handwritten mathematical expression recognition model based on attention mechanism and encoder-decoder [J]. Journal of Computer Applications, 2023, 43(4): 1297-1302.
[7]	Hexuan HU, Huachao SUI, Qiang HU, Ye ZHANG, Zhenyun HU, Nengwu MA. Runoff forecast model based on graph attention network and dual-stage attention mechanism [J]. Journal of Computer Applications, 2022, 42(5): 1607-1615.
[8]	LIU Shize, QIN Yanjun, WANG Chenxing, SU Lin, KE Qixue, LUO Haiyong, SUN Yi, WANG Baohui. Traffic flow prediction algorithm based on deep residual long short-term memory network [J]. Journal of Computer Applications, 2021, 41(6): 1566-1572.
[9]	DU Xixi, CHENG Hua, FANG Yiquan. Reinforced automatic summarization model based on advantage actor-critic algorithm [J]. Journal of Computer Applications, 2021, 41(3): 699-705.
[10]	DING Xiangguo, SANG Jitao. Joint extraction of entities and relations based on relation-adaptive decoding [J]. Journal of Computer Applications, 2021, 41(1): 29-35.
[11]	YANG Wenxia, WANG Meng, ZHANG Liang. Semantic face image inpainting based on U-Net with dense blocks [J]. Journal of Computer Applications, 2020, 40(12): 3651-3657.
[12]	JIA Ruiming, QIU Zhenzhi, CUI Jiali, WANG Yiding. Deep multi-scale encoder-decoder convolutional network for blind deblurring [J]. Journal of Computer Applications, 2019, 39(9): 2552-2557.
[13]	ZHENG Yanbin, LI Bo, AN Deyu, LI Na. Multi-Agent path planning algorithm based on hierarchical reinforcement learning and artificial potential field [J]. Journal of Computer Applications, 2015, 35(12): 3491-3496.

模型	Micro-F1	Macro-F1
BERT-TEPK	77.94	61.93
BERT-TEPK-HSI	78.19	62.14
BERT-TEPK-SI	78.31	62.27
BERT-TEPK-LEHSA	78.38	62.32

模型	Micro-F1	Macro-F1
BERT-TEPK	77.94	61.93
BERT-TEPK-HSI	78.19	62.14
BERT-TEPK-SI	78.31	62.27
BERT-TEPK-LEHSA	78.38	62.32