Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (1): 159-166.DOI: 10.11772/j.issn.1001-9081.2023010029
Special Issue: 人工智能
• Artificial intelligence • Previous Articles Next Articles
Xiao GUO1,2, Yanping CHEN1,2(), Ruixue TANG1,2,3, Ruizhang HUANG1,2, Yongbin QIN1,2
Received:
2023-01-11
Revised:
2023-03-18
Accepted:
2023-03-28
Online:
2023-06-06
Published:
2024-01-10
Contact:
Yanping CHEN
About author:
GUO Xiao, born in 1998, M. S. candidate. His research interests include natural language processing, information extraction.Supported by:
郭晓1,2, 陈艳平1,2(), 唐瑞雪1,2,3, 黄瑞章1,2, 秦永彬1,2
通讯作者:
陈艳平
作者简介:
郭晓(1998—),男,山西阳泉人,硕士研究生,CCF会员,主要研究方向:自然语言处理、信息抽取;基金资助:
CLC Number:
Xiao GUO, Yanping CHEN, Ruixue TANG, Ruizhang HUANG, Yongbin QIN. Multi-task learning model for charge prediction with action words[J]. Journal of Computer Applications, 2024, 44(1): 159-166.
郭晓, 陈艳平, 唐瑞雪, 黄瑞章, 秦永彬. 融合行为词的罪名预测多任务学习模型[J]. 《计算机应用》唯一官方网站, 2024, 44(1): 159-166.
Add to citation manager EndNote|Ris|BibTeX
URL: https://www.joca.cn/EN/10.11772/j.issn.1001-9081.2023010029
罪名类型 | 数据规模 | 行为词数 |
---|---|---|
故意杀人罪 | 242 | 2 493 |
故意伤害罪 | 115 | 1 052 |
贩卖毒品罪 | 21 | 200 |
运输毒品罪 | 13 | 101 |
假冒注册商标罪 | 26 | 173 |
危险驾驶罪 | 13 | 83 |
走私罪 | 97 | 823 |
抢劫罪 | 12 | 130 |
盗窃罪 | 16 | 136 |
诈骗罪 | 51 | 481 |
受贿罪 | 12 | 86 |
走私、贩卖、运输、制造毒品罪 | 81 | 626 |
Tab. 1 Number distribution of action words and charges
罪名类型 | 数据规模 | 行为词数 |
---|---|---|
故意杀人罪 | 242 | 2 493 |
故意伤害罪 | 115 | 1 052 |
贩卖毒品罪 | 21 | 200 |
运输毒品罪 | 13 | 101 |
假冒注册商标罪 | 26 | 173 |
危险驾驶罪 | 13 | 83 |
走私罪 | 97 | 823 |
抢劫罪 | 12 | 130 |
盗窃罪 | 16 | 136 |
诈骗罪 | 51 | 481 |
受贿罪 | 12 | 86 |
走私、贩卖、运输、制造毒品罪 | 81 | 626 |
参数 | 设定值 | 参数 | 设定值 |
---|---|---|---|
Batch size | 4 | 学习率 | 5×10-6 |
Epoch | 100 | 学习率预热值 | 0.1 |
Dropout | 0.5 | GCN层数 | 2 |
字向量维度 | 1 024 | 输入序列长度 | 512 |
权重衰减率 | 0.01 |
Tab. 2 Parameter setting
参数 | 设定值 | 参数 | 设定值 |
---|---|---|---|
Batch size | 4 | 学习率 | 5×10-6 |
Epoch | 100 | 学习率预热值 | 0.1 |
Dropout | 0.5 | GCN层数 | 2 |
字向量维度 | 1 024 | 输入序列长度 | 512 |
权重衰减率 | 0.01 |
模型 | 行为词识别 | 罪名预测 | ||
---|---|---|---|---|
P | R | F | ||
BERT-Attention[ | 79.33 | 81.12 | 80.22 | 80.00 |
BERT-BiGRU[ | 78.65 | 82.45 | 80.50 | 80.71 |
BERT-CNN[ | 81.18 | 84.48 | 82.80 | 82.14 |
本文模型 | 82.69 | 83.85 | 83.27 | 84.29 |
Tab. 3 Experimental result comparison among different models
模型 | 行为词识别 | 罪名预测 | ||
---|---|---|---|---|
P | R | F | ||
BERT-Attention[ | 79.33 | 81.12 | 80.22 | 80.00 |
BERT-BiGRU[ | 78.65 | 82.45 | 80.50 | 80.71 |
BERT-CNN[ | 81.18 | 84.48 | 82.80 | 82.14 |
本文模型 | 82.69 | 83.85 | 83.27 | 84.29 |
方法 | ||
---|---|---|
故意杀人罪 | 故意伤害罪 | |
文本语义 | 82.47 | 57.17 |
行为词语义 | 84.85 | 60.00 |
Tab. 4 Experiment results of confusible charge prediction
方法 | ||
---|---|---|
故意杀人罪 | 故意伤害罪 | |
文本语义 | 82.47 | 57.17 |
行为词语义 | 84.85 | 60.00 |
模型 | 行为词识别 | ||
---|---|---|---|
P | R | F | |
BERT-IDCNN-CRF[ | 80.08 | 80.89 | 80.48 |
BERT-BIGRU-CRF[ | 81.57 | 81.83 | 81.70 |
BERT-BILSTM-Attention[ | 83.94 | 80.34 | 82.10 |
本文模型 | 82.69 | 83.85 | 83.27 |
Tab. 5 Comparative experiment results of action word recognition with single task models
模型 | 行为词识别 | ||
---|---|---|---|
P | R | F | |
BERT-IDCNN-CRF[ | 80.08 | 80.89 | 80.48 |
BERT-BIGRU-CRF[ | 81.57 | 81.83 | 81.70 |
BERT-BILSTM-Attention[ | 83.94 | 80.34 | 82.10 |
本文模型 | 82.69 | 83.85 | 83.27 |
模型 | 罪名预测Fmicro |
---|---|
TextRNN-Attention[ | 70.71 |
FastText[ | 77.14 |
TextCNN[ | 80.71 |
本文模型 | 84.29 |
Tab. 6 Comparative experiment results of predict charge with single task models
模型 | 罪名预测Fmicro |
---|---|
TextRNN-Attention[ | 70.71 |
FastText[ | 77.14 |
TextCNN[ | 80.71 |
本文模型 | 84.29 |
模型 | 行为词识别 | 罪名预测Fmicro | ||
---|---|---|---|---|
P | R | F | ||
本文模型 | 82.69 | 83.85 | 83.27 | 84.29 |
-相关性构建 | 82.00 | 83.15 | 82.57 | 82.14 |
-边界识别 | 76.65 | 81.67 | 79.08 | 83.57 |
Tab. 7 Ablation experiment results
模型 | 行为词识别 | 罪名预测Fmicro | ||
---|---|---|---|---|
P | R | F | ||
本文模型 | 82.69 | 83.85 | 83.27 | 84.29 |
-相关性构建 | 82.00 | 83.15 | 82.57 | 82.14 |
-边界识别 | 76.65 | 81.67 | 79.08 | 83.57 |
1 | WEI B, KUANG K, SUN C, et al. A full-process intelligent trial system for smart court [J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23: 186-206. 10.1631/fitee.2100041 |
2 | KORT F. Predicting Supreme Court decisions mathematically: A quantitative analysis of the “right to counsel” cases [J]. American Political Science Review, 1957, 51(1): 1-12. 10.2307/1951767 |
3 | NAGEL S S. Applying correlation analysis to case prediction [J]. Texas Law Review, 1963, 42: 1006. |
4 | 于游,付钰,吴晓平.中文文本分类方法综述[J].网络与信息安全学报, 2019, 5(5): 1-8. 10.11959/j.issn.2096-109x.2019045 |
YU Y, FU Y, WU X P. Summary of text classification methods [J]. Chinese Journal of Network and Information Security, 2019, 5(5): 1-8. 10.11959/j.issn.2096-109x.2019045 | |
5 | 白昌前,代晓,张岸.基于数据增强和改进BERT的罪名预测[J].电脑与信息技术, 2023, 31(1): 37-40. 10.3969/j.issn.1005-1228.2023.01.012 |
BAI C Q, DAI X, ZHANG A. Crime prediction based on data enhancement and improved BERT [J]. Computer and Information Technology, 2023, 31(1): 37-40. 10.3969/j.issn.1005-1228.2023.01.012 | |
6 | 彭韬,杨亮,张琍,等.联合多源分析的罪名预测研究[J].计算机工程与应用, 2023, 59(4): 290-296. 10.3778/j.issn.1002-8331.2108-0339 |
PENG T, YANG L, ZHANG L, et al. Research on charge prediction based on multi-source joint analysis [J]. Computer Engineering and Applications, 2023, 59(4): 290-296. 10.3778/j.issn.1002-8331.2108-0339 | |
7 | HU Z, LI X, TU C, et al. Few-shot charge prediction with discriminative legal attributes [C]// Proceedings of the 27th International Conference on Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2018: 487-498. |
8 | 刘宗林,张梅山,甄冉冉,等.融入罪名关键词的法律判决预测多任务学习模型[J].清华大学学报(自然科学版), 2019, 59(7): 497-504. |
LIU Z L, ZHANG M S, ZHEN R R, et al. Multi-task learning model for legal judgment predictions with charge keywords [J]. Journal of Tsinghua University (Science and Technology), 2019, 59(7): 497-504. | |
9 | 李婷,秦永彬,黄瑞章,等.基于神经网络的中文谓语动词识别研究[J].数据采集与处理, 2020, 35(3): 582-590. |
LI T, QIN Y B, HUANG R Z, et al. Research on Chinese predicate verb recognition based on neural network [J]. Journal of Data Acquisition & Processing, 2020, 35(3): 582-590. | |
10 | DEVLIN J, CHANG M-W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding [EB/OL]. [2019-05-24]. . 10.18653/v1/n18-2 |
11 | SCHUSTER M, PALIWAL K K. Bidirectional recurrent neural networks [J]. IEEE Transactions on Signal Processing, 1997, 45(11): 2673-2681. 10.1109/78.650093 |
12 | KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks [EB/OL]. [2019-09-09]. . 10.48550/arXiv.1609.02907 |
13 | ULMER S S. Quantitative analysis of judicial processes: Some practical and theoretical applications [J]. Law and Contemporary Problems, 1963, 28(1): 164-184. 10.2307/1190728 |
14 | KEOWN R. Mathematical models for legal prediction [J]. Computer/Law Journal, 1980, 2: 829-862. 10.1016/0270-0255(80)90042-1 |
15 | LIU C-L, CHANG C-T, J-H HO. Case instance generation and refinement for case-based criminal summary judgments in Chinese [J]. Journal of Information Science and Engineering, 2004, 20: 783-800. 10.1145/1047788.1047840 |
16 | BIJALWAN V, KUMARI V, KUMARI P, et al. KNN based machine learning approach for text and document mining [J]. International Journal of Database Theory and Application, 2014, 7(1): 61-70. 10.14257/ijdta.2014.7.1.06 |
17 | O-M SULEA, ZAMPIERI M, MALMASI S, et al. Exploring the use of text classification in the legal domain [EB/OL]. [2017-10-25]. . 10.1109/icds47004.2019.8942343 |
18 | ZHANG H, BERG A C, MAIRE M, et al. SVM-KNN: Discriminative nearest neighbor classification for visual category recognition [C]// Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2006: 2126-2136. |
19 | LIU Y-H, CHEN Y-L, W-L HO. Predicting associated statutes for legal problems [J]. Information Processing & Management, 2015, 51(1): 194-211. 10.1016/j.ipm.2014.07.003 |
20 | LIN W-C, T-T KUO, CHANG T-J, et al. Exploiting machine learning models for Chinese legal documents labeling, case classification, and sentencing prediction [J]. Computational Linguistic and Chinese Language Processing, 2012, 17(4): 49-68. |
21 | LAFFERTY J D, McCALLUM A, PEREIRA F C. Conditional random fields: Probabilistic models for segmenting and labeling sequence data [C]// Proceedings of the 18th International Conference on Machine Learning. San Francisco: Morgan Kaufmann Publishers Inc., 2001: 282-289. |
22 | LUO B, FENG Y, XU J, et al. Learning to predict charges for criminal cases with legal basis [EB/OL]. [2017-07-28]. . 10.18653/v1/d17-1289 |
23 | LONG S, TU C, LIU Z, et al. Automatic judgment prediction via legal reading comprehension [C]// Proceedings of the 2019 China National Conference on Chinese Computational Linguistics. Cham: Springer, 2019: 558-572. 10.1007/978-3-030-32381-3_45 |
24 | YE H, JIANG X, LUO Z, et al. Interpretable charge predictions for criminal cases: Learning to generate court views from fact descriptions [EB/OL]. [2018-02-23]. . 10.18653/v1/n18-1168 |
25 | 赵琪珲,李大鹏,高天寒,等.基于图注意力网络的案件罪名预测方法: CP-GAT [J].东北大学学报(自然科学版), 2021, 42(12): 1681-1687. |
ZHAO Q H, LI D P, GAO T H, et al. A charge prediction method based on graph attention network: CP-GAT [J]. Journal of Northeastern University (Natural Science), 2021, 42(12): 1681-1687. | |
26 | 孙倩,秦永彬,黄瑞章,等.结合案件要素序列的罪名预测方法[J].大数据, 2021, 7(6): 30-40. 10.11959/j.issn.2096-0271.2021058 |
SUN Q, QIN Y B, HUANG R Z, et al. Charge prediction method combined with case elements sequence [J]. Big Data Research, 2021, 7(6): 30-40. 10.11959/j.issn.2096-0271.2021058 | |
27 | 倪晴超,殷聪珏,赵冬华.基于BERT和关键词的属性-罪名分类[J].计算机应用, 2021, 41(S2): 36-40. |
NI Q C, YIN C J, ZHAO D H. Classification of attributes and charges based on BERT and keywords [J]. Journal of Computer Applications, 2021, 41(S2): 36-40. | |
28 | 王卓越,陈彦光,邢铁军,等.基于多任务学习的多罪名案件信息联合抽取[J].计算机工程与应用, 2023, 59(2): 178-184. 10.3778/j.issn.1002-8331.2108-0344 |
WANG Z Y, CHEN Y G, XING T J, et al. Joint entity and relation extraction for multi-crime legal documents with multi-task learning [J]. Computer Engineering and Applications, 2023, 59(2): 178-184. 10.3778/j.issn.1002-8331.2108-0344 | |
29 | HENDRYCKS D, GIMPEL K. Gaussian Error Linear Units (GELUs) [EB/OL]. [2020-07-08]. . |
30 | GIRSHICK R. Fast R-CNN [C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 1440-1448. 10.1109/iccv.2015.169 |
31 | CHEN Y, JIN W, QIN Y, et al. Annotation of Chinese predicate heads and relevant elements [EB/OL]. [2021-04-01]. . |
32 | 毛明毅,吴晨,钟义信,等.加入自注意力机制的BERT命名实体识别模型[J].智能系统学报, 2020, 15(4): 772-779. 10.11992/tis.202003003 |
MAO M Y, WU C, ZHONG Y X, et al. BERT named entity recognition model with self-attention mechanism [J]. CAAI Transactions on Intelligent Systems, 2020, 15(4): 772-779. 10.11992/tis.202003003 | |
33 | YU Q, WANG Z, JIANG K. Research on text classification based on BERT-BiGRU model [J]. Journal of Physics: Conference Series, 2021, 1746: 012019. 10.1088/1742-6596/1746/1/012019 |
34 | 史振杰,董兆伟,庞超逸,等.基于BERT-CNN的电商评论情感分析[J].智能计算机与应用, 2020, 10(2): 7-11. 10.3969/j.issn.2095-2163.2020.02.002 |
SHI Z J, DONG Z W, PANG C Y, et al. Sentiment analysis of e-commerce reviews based on BERT-CNN [J]. Intelligent Computer and Applications, 2020, 10(2): 7-11. 10.3969/j.issn.2095-2163.2020.02.002 | |
35 | VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc., 2017: 6000-6010. |
36 | CAI X, SUN E, LEI J. Research on application of named entity recognition of electronic medical records based on BERT-IDCNN-CRF model [C]// Proceedings of the 6th International Conference on Graphics and Signal Processing. New York: ACM, 2022: 80-85. 10.1145/3561518.3561531 |
37 | QIN Q, ZHAO S, LIU C. A BERT-BiGRU-CRF model for entity recognition of Chinese electronic medical records [J]. Complexity, 2021, 2021: 6631837. 10.1155/2021/6631837 |
38 | 倪健,陈鹏兴.基于Bert-BiLSTM-Attention的互联网金融实体识别方法[J].信息与电脑, 2021, 33(20): 58-61. 10.3969/j.issn.1003-9767.2021.20.018 |
NI J, CHEN P X. Internet financial entity recognition method based on BiLSTM-attention [J]. China Computer & Communication, 2021, 33(20): 58-61. 10.3969/j.issn.1003-9767.2021.20.018 | |
39 | LIU P, QIU X, HUANG X. Recurrent neural network for text classification with multi-task learning [EB/OL]. [2016-05-17]. . 10.18653/v1/d16-1012 |
40 | JOULIN A, GRAVE E, BOJANOWSKI P, et al. Bag of tricks for efficient text classification [EB/OL]. [2016-08-09]. . 10.18653/v1/e17-2068 |
41 | ZHANG Y, WALLACE B. A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification [EB/OL]. [2016-04-06]. . 10.18653/v1/d16-1076 |
[1] | Wei LI, Ling CHEN, Xiuyuan XU, Min ZHU, Jixiang GUO, Kai ZHOU, Hao NIU, Yuchen ZHANG, Shanye YI, Yi ZHANG, Fengming LUO. Interstitial lung disease segmentation algorithm based on multi-task learning [J]. Journal of Computer Applications, 2024, 44(4): 1285-1293. |
[2] | Aiguo SHANG, Xinjuan ZHU. Joint approach of intent detection and slot filling based on multi-task learning [J]. Journal of Computer Applications, 2024, 44(3): 690-695. |
[3] | Yudan SONG, Jing WANG, Xuehui WANG, Zhaoyang MA, Youfang LIN. Sleep physiological time series classification method based on adaptive multi-task learning [J]. Journal of Computer Applications, 2024, 44(2): 654-662. |
[4] | Cunyi LIAO, Yi ZHENG, Weijin LIU, Huan YU, Shouyin LIU. Decoupling-fusing algorithm for multiple tasks with autonomous driving environment perception [J]. Journal of Computer Applications, 2024, 44(2): 424-431. |
[5] | Jianhui HE, Chunlong HU, Xin SHU. Multi-task age estimation method based on multi-peak label distribution learning [J]. Journal of Computer Applications, 2023, 43(5): 1578-1583. |
[6] | Ying CHEN, Jiong YU, Jiaying CHEN, Xusheng DU. Cross-layer data sharing based multi-task model [J]. Journal of Computer Applications, 2022, 42(5): 1447-1454. |
[7] | Qiming RUAN, Yi GUO, Nan ZHENG, Yexiang WANG. Customs declaration good classification algorithm based on hierarchical multi-task BERT [J]. Journal of Computer Applications, 2022, 42(1): 71-77. |
[8] | WU Guoliang, XU Jining. Chinese emergency event extraction method based on named entity recognition task feedback enhancement [J]. Journal of Computer Applications, 2021, 41(7): 1891-1896. |
[9] | YAO Jie, CHENG Chunling, HAN Jing, LIU Zheng. Anomaly detection method based on multi-task temporal convolutional network in cloud workflow [J]. Journal of Computer Applications, 2021, 41(6): 1701-1708. |
[10] | ZHANG Sun, YIN Chunyong. Sequential multimodal sentiment analysis model based on multi-task learning [J]. Journal of Computer Applications, 2021, 41(6): 1631-1639. |
[11] | FU Ying, WANG Hongling, WANG Zhongqing. Scientific paper summarization model using macro discourse structure [J]. Journal of Computer Applications, 2021, 41(10): 2864-2870. |
[12] | RUAN Canhua, LIN Jiaxiang. Multi-task Logistic survival prediction method for time-dependent time-to-event data [J]. Journal of Computer Applications, 2020, 40(5): 1284-1290. |
[13] | CAO Jinmeng, NI Rongrong, YANG Biao. Crowd counting using multi-scale multi-task convolutional neural network [J]. Journal of Computer Applications, 2019, 39(1): 199-204. |
[14] | CHENG Jin, WANG Jian. Endpoint prediction method for steelmaking based on multi-task learning [J]. Journal of Computer Applications, 2017, 37(3): 889-895. |
[15] | OUYANG Ning, MA Yutao, LIN Leping. Multi-pose face reconstruction and recognition based on multi-task learning [J]. Journal of Computer Applications, 2017, 37(3): 896-900. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||