Violent crime hierarchy algorithm by joint modeling of improved hierarchical attention network and TextCNN

doi:10.11772/j.issn.1001-9081.2023030270

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (2): 403-410.DOI: 10.11772/j.issn.1001-9081.2023030270

Special Issue: 人工智能

• Artificial intelligence • Previous Articles Next Articles

Violent crime hierarchy algorithm by joint modeling of improved hierarchical attention network and TextCNN

Jiawei ZHANG¹^,², Guandong GAO²^,³(), Ke XIAO¹^,⁴, Shengzun SONG⁵

^1.College of Information Science and Technology，Hebei Agricultural University，Baoding Hebei 071000，China
^2.The Centre of Data Science and Intelligent Correction Technology，The National Police University for Criminal Justice，Baoding Hebei 071000，China
^3.Department of Information Management，The National Police University for Criminal Justice，Baoding Hebei 071000，China
^4.Hebei Key Laboratory of Agricultural Big Data （Hebei Agricultural University），Baoding Hebei 071000，China
^5.Department of Penology，The National Police University for Criminal Justice，Baoding Hebei 071000，China

Received:2023-03-16 Revised:2023-05-09 Accepted:2023-05-11 Online:2023-05-23 Published:2024-02-10
Contact: Guandong GAO
About author:ZHANG Jiawei， born in 1998， M. S. candidate. His research interests include natural language processing， criminal psychological portrait.
XIAO Ke， born in 1980， Ph. D.， professor. Her research interests include agricultural engineering， internet of things， machine vision.
SONG Shengzun， born in 1967， M. S.， professor. Her research interests include criminal psychology， prisoner’s emotional management， affective computing.
Supported by:
Social Science Foundation of Hebei Province(HB21ZZ002)

基于改进分层注意网络和TextCNN联合建模的暴力犯罪分级算法

张家伟¹^,², 高冠东²^,³(), 肖珂¹^,⁴, 宋胜尊⁵

^1.河北农业大学信息科学与技术学院, 河北保定 071000
^2.中央司法警官学院数据科学与智能矫正技术研究中心, 河北保定 071000
^3.中央司法警官学院信息管理系, 河北保定 071000
^4.河北省农业大数据重点实验室(河北农业大学), 河北保定 071000
^5.中央司法警官学院监狱学学院, 河北保定 071000

通讯作者: 高冠东
作者简介:张家伟（1998—），男，河北邯郸人，硕士研究生，主要研究方向：自然语言处理、犯罪心理画像
肖珂（1980—），女，四川内江人，教授，博士，CCF会员，主要研究方向：农业工程、物联网、机器视觉
宋胜尊（1967—），女，河北深州人，教授，硕士，主要研究方向：犯罪心理学、服刑人员情绪管理、情感计算。
基金资助:
河北省社会科学基金资助项目(HB21ZZ002)

Abstract

Abstract:

A text classification method in Natural Language Processing （NLP） was introduced into the field of criminal psychology to scientifically and intelligently grade the violent tendencies of prisoners. A Criminal semantic Convolutional Hierarchical Attention Network （CCHA-Net） based on the joint modeling of two channels of improved HAN （Hierarchy Attention Network） and TextCNN （Text Convolutional Neural Network） was proposed to complete the violent criminal temperament grade by separately mining the semantic information of crime facts and basic information of prisoners. Firstly， Focal Loss was used to simultaneously replace the Cross-Entropy function in both channels to optimize the sample size imbalance problem. Secondly， in the two-channel input layer， positional encoding was simultaneously introduced to improve the perception of positional information. The HAN channel was improved by using max-pooling to construct salient vectors. Finally， global average pooling was used to replace the fully connected method in all output layers to avoid overfitting. Experimental results show that compared with 17 related baseline models such as AC-BiLSTM （Attention-based Bidirectional Long Short-Term Memory with Convolution layer） and Support Vector Machine （SVM）， the indicators of CCHA-Net reach the best， the micro-average F1 （Micro_F1） is 99.57%， and the Area Under the Curve （AUC） under the macro-average and the micro-average are 99.45% and 99.89%， respectively， which are 4.08， 5.59 and 0.74 percentage points higher than those of the suboptimal AC-BiLSTM. It can be verified that the violent criminal temperament grade task can be effectively performed by CCHA-Net.

Key words: deep learning, text classification, Convolutional Neural Network (CNN), Hierarchical Attention Network (HAN), hierarchy of violent crime, temperament type

摘要：

为了科学、智能地对服刑人员的暴力倾向分级，将自然语言处理（NLP）中的文本分类方法引入犯罪心理学领域，提出一种基于改进分层注意网络（HAN）与TextCNN（Text Convolutional Neural Network）两通道联合建模的犯罪语义卷积分层注意网络（CCHA-Net），通过分别挖掘犯罪事实与服刑人员基本情况的语义信息，完成暴力犯罪气质分级。首先，采用Focal Loss同时替代两通道中的Cross-Entropy函数，优化样本数量不均衡问题。其次，在两通道输入层中，同时引入位置编码，改进对位置信息的感知能力；改进HAN通道，采用最大池化构建显著向量。最后，输出层都采用全局平均池化替代全连接方法，以避免过拟合。实验结果表明，与AC-BiLSTM（Attention-based Bidirectional Long Short-Term Memory with Convolution layer）、支持向量机（SVM）等17种相关基线模型相比，CCHA-Net各项指标均最优，微平均F1（Micro_F1）为99.57%，宏平均和微平均下的曲线下面积（AUC）分别为99.45%和99.89%，相较于次优的AC-BiLSTM提高了4.08、5.59和0.74个百分点，验证了CCHA-Net能有效胜任暴力犯罪气质分级任务。

关键词: 深度学习, 文本分类, 卷积神经网络, 分层注意网络, 暴力犯罪分级, 气质类型

CLC Number:

TP391.1

Jiawei ZHANG, Guandong GAO, Ke XIAO, Shengzun SONG. Violent crime hierarchy algorithm by joint modeling of improved hierarchical attention network and TextCNN[J]. Journal of Computer Applications, 2024, 44(2): 403-410.

张家伟, 高冠东, 肖珂, 宋胜尊. 基于改进分层注意网络和TextCNN联合建模的暴力犯罪分级算法[J]. 《计算机应用》唯一官方网站, 2024, 44(2): 403-410.

Figures/Tables 7

References 39

1	WARD T， DURRANT R， DIXON L. The classification of crime： towards pluralism［J］. Aggression and Violent Behavior， 2020， 59： 10474. 10.1016/j.avb.2020.101474
2	WARD T， CAETER E. The classification of offending and crime related problems： a functional perspective［J］. Psychology， Crime & Law， 2019， 25（6）： 542-560. 10.1080/1068316x.2018.1557182
3	MO C-Y， JIN J， JIN P. Relationship between teachers’ teaching modes and students’ temperament and learning motivation in confucian culture during the COVID-19 pandemic［J］. Frontiers in Psychology， 2022， 13： 865445. 10.3389/fpsyg.2022.865445
4	BOJANOWSKA A， ZALEWSKA A M. Happy temperament？ four types of stimulation control linked to four types of subjective well-being［J］. Journal of Happiness Studies： An Interdisciplinary Forum on Subjective Well-Being， 2017， 18（5）： 1403-1423. 10.1007/s10902-016-9777-2
5	DELISI M， VAUGHN M G. Foundation for a temperament-based theory of antisocial behavior and criminal justice system involvement［J］. Journal of Criminal Justice， 2014， 42（1）：10-25. 10.1016/j.jcrimjus.2013.11.001
6	马皑，宋业臻. 情感计算技术如何推动犯罪风险评估工具的发展？［J］. 心理科学， 2021， 44（1）： 52-59.
	MA A， SONG Y Z. How does emotional computing technology promote the development of crime risk assessment tools？［J］. Journal of Psychological Science， 2021， 44（1）： 52-59.
7	马皑，孙晓，宋业臻. 犯罪危险性智能化评估的理论与实践［M］. 北京：中国法制出版社， 2020： 36-50.
	MA A， SUN X， SONG Y Z. Theory and Practice of Intelligent Assessment of Criminal Dangerousness［M］. Beijing： China Legal Publishing House， 2020： 36-50.
8	MALHOTRA A， JINDAL R. Deep learning techniques for suicide and depression detection from online social media： a scoping review［J］. Applied Soft Computing， 2022， 130： 109713. 10.1016/j.asoc.2022.109713
9	MAHAJAN R， MAHAJAN R， SHARMA E， et al. “Are we tweeting our real selves？” personality prediction of Indian Twitter users using deep learning ensemble model［J］. Computers in Human Behavior， 2022， 128： 107101. 10.1016/j.chb.2021.107101
10	ZHANG N， TAN Y-A， YANG C， et al. Deep learning feature exploration for Android malware detection［J］. Applied Soft Computing， 2021， 102： 107069. 10.1016/j.asoc.2020.107069
11	QUOC TRAN K， TRONG NGUYEN A， HOANG P G， et al. Vietnamese hate and offensive detection using PhoBERT-CNN and social media streaming data［J］. Neural Computing and Applications， 2023， 35： 573-594. 10.1007/s00521-022-07745-w
12	孙新，唐正，赵永妍，等. 基于层次混合注意力机制的文本分类模型［J］. 中文信息学报， 2021， 35（2）： 69-77. 10.3969/j.issn.1003-0077.2021.02.007
	SUN X， TANG Z， ZHAO Y Y， et al. Hierarchical networks with mixed attention for text classification［J］. Journal of Chinese Information Processing， 2021， 35（2）： 69-77. 10.3969/j.issn.1003-0077.2021.02.007
13	MUMCUOĞLU E， ÖZTÜRK C E， OZAKTAS H M， et al. Natural language processing in law： prediction of outcomes in the higher courts of Turkey［J］. Information Processing & Management， 2021， 58（5）：102684. 10.1016/j.ipm.2021.102684
14	AHMAD S， ASGHAR M Z， ALOTAIBI F M， et al. A hybrid CNN+ BILSTM deep learning-based DSS for efficient prediction of judicial case decisions［J］. Expert Systems with Applications， 2022， 209： 118318. 10.1016/j.eswa.2022.118318
15	NI Y， BARZMAN D， BACHTEL A， et al. Finding warning markers： leveraging natural language processing and machine learning technologies to detect risk of school violence［J］. International Journal of Medical Informatics， 2020， 139： 104137. 10.1016/j.ijmedinf.2020.104137
16	KALCHBRENNER N， GREFENSTETTE E， BLUNSOM P. A convolutional neural network for modelling sentences ［C］// Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2014： 655-665. 10.3115/v1/p14-1062
17	ZAREMBA W， SUTSKEVER I， VINYALS O. Recurrent neural network regularization ［EB/OL］. （2015-02-19）［2022-03-10］. . 10.48550/arXiv.1409.2329
18	罗浩然，杨青.基于情感词典和堆叠残差的双向长短期记忆网络的情感分析［J］.计算机应用，2022，42（4）： 1099-1107. 10.11772/j.issn.1001-9081.2021071179
	LUO H R， YANG Q. Sentiment analysis based on sentiment lexicon and stacked residual Bi-LSTM network［J］. Journal of Computer Applications， 2022， 42（4）： 1199-1107. 10.11772/j.issn.1001-9081.2021071179
19	YANG Z， YANG D， DYER C， et al. Hierarchical attention networks for document classification ［C］// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies. San Diego： NAACL， 2016： 1480-1489. 10.18653/v1/n16-1174
20	M-S BAEK， PARK W， PARK J， et al. Smart policing technique with crime type and risk score prediction based on machine learning for early awareness of risk situation［J］. IEEE Access， 2021， 9： 131906-131915. 10.1109/access.2021.3112682
21	SADIQ S， MEHMOOD A， ULLAH S， et al. Aggression detection through deep neural model on twitter［J］. Future Generation Computer Systems， 2021， 114： 120-129. 10.1016/j.future.2020.07.050
22	VASWANI A， SHAZEER N， PARMER N， et al. Attention is all you need ［EB/OL］. （2017-12-16）［2022-03-12］. .
23	LIN T-Y， GOYAL P， GIRSHICK R， et al. Focal loss for dense object detection ［EB/OL］. （2018-02-07）［2022-06-10］. . 10.1109/iccv.2017.324
24	COVER T， HART P. Nearest neighbor pattern classification［J］. IEEE Transactions on Information Theory， 1967， 13（1）： 21-27. 10.1109/tit.1967.1053964
25	NIGAM K， McCALLUM A K， THRUN S， et al. Text classification from labeled and unlabeled documents using EM［J］. Machine Learning， 2000， 39： 103-134. 10.1023/a:1007692713085
26	MCCALLUM A， NIGAM K. A comparison of event models for naive Bayes text classification ［C］// Proceedings of the 1998 AAAI/ICML Workshop on Learning for Text Categorization. Menlo Park： AAAI Press， 1998： 41-48.
27	PONTE J M， CROFT W B. A language modeling approach to information retrieval ［C］// Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 1998： 275-281. 10.1145/290941.291008
28	QUINLAN J R. Induction of decision trees［J］. Machine Learning， 1986， 1： 81-106. 10.1007/bf00116251
29	HO T K. Random decision forests ［C］// Proceedings of 3rd International Conference on Document Analysis and Recognition. Piscataway： IEEE， 1995， 1： 278-282. 10.1109/icdar.1995.598929
30	CORTE C， VAPNIK V. Support-vector networks［J］. Machine Learning， 1995， 20： 273-297. 10.1007/bf00994018
31	CHEN T， GUESTRIN C. XGBoost： a scalable tree boosting system ［C］// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2016： 785-794. 10.1145/2939672.2939785
32	HOSMER D W， Jr， LEMESHOW S， STURDIVANT R X. Applied Logistic Regression［M］. Hoboken： John Wiley & Sons， 2013： 398. 10.1002/9781118548387
33	HOCHREITER S， SCHMIDHUBER J. Long short-term memory［J］. Neural Computation， 1997， 9（8）： 1735-1780. 10.1162/neco.1997.9.8.1735
34	SCHUSTER M， PALIWAL K K. Bidirectional recurrent neural networks［J］. IEEE Transactions on Signal Processing， 1997， 45（11）： 2673-2681. 10.1109/78.650093
35	CHO K， VAN MERRIËNBOER B， GULCEHRE C， et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation ［C］// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsberg： ACL， 2014： 1724-1734. 10.3115/v1/d14-1179
36	CHUNG J， GULCEHRE C， CHO K， et al. Empirical evaluation of gated recurrent neural networks on sequence modeling ［EB/OL］. （2014-12-11）［2022-09-10］.. 10.1007/978-3-030-89929-5_3
37	ZHOU P， SHI W， TIAN J， et al. Attention-based bidirectional long short-term memory networks for relation classification ［C］// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics （Volume 2： Short Papers）. Stroudsberg： ACL， 2016： 207-212. 10.18653/v1/p16-2034
38	ZHOU C， SUN C， LIU Z， et al. A C-LSTM neural network for text classification ［EB/OL］. （2015-11-27）［2022-09-26］. .
39	LIU G， GUO J. Bidirectional LSTM with attention mechanism and convolutional layer for text classification［J］. Neurocomputing， 2019， 337： 325-338. 10.1016/j.neucom.2019.01.078

参数	参数值
Kernels size	3，4，5
Number of kernels	128
Number of Bi-GRU Layers	1
Number of Bi-GRU hidden units	64
Batch size	128
Epoch	50
Loss function	Focal Loss
Learning rate	0.001

参数	参数值
Kernels size	3，4，5
Number of kernels	128
Number of Bi-GRU Layers	1
Number of Bi-GRU hidden units	64
Batch size	128
Epoch	50
Loss function	Focal Loss
Learning rate	0.001

类型	训练集样本数	验证集样本数	测试集样本数	总计
总计	2 800	933	932	4 665
胆汁质	1 340	446	446	2 232
多血质	1 178	393	392	1 963
粘液质	279	93	93	465
抑郁质	3	1	1	5

类型	训练集样本数	验证集样本数	测试集样本数	总计
总计	2 800	933	932	4 665
胆汁质	1 340	446	446	2 232
多血质	1 178	393	392	1 963
粘液质	279	93	93	465
抑郁质	3	1	1	5

类型	模型	Acc	Macro_P	Macro_R	Macro_F1	Macro_AUC	Micro_P	Micro_R	Micro_F1	Micro_AUC
犯罪事实语义建模	HAN	90.45	64.72	67.87	66.05	86.89	90.45	90.45	90.45	97.40
	HAN+Focal Loss	91.52	67.68	69.70	68.61	88.77	91.52	91.52	91.52	98.72
	HAN+Focal Loss+位置编码	92.60	67.74	68.87	68.27	94.48	92.60	92.60	92.60	98.88
	HAN+Focal Loss+位置编码+显著向量	96.57	71.82	72.14	71.96	97.88	96.57	96.57	96.57	99.61
	HAN+Focal Loss+位置编码+ 显著向量+GAP	97.10	72.69	71.79	72.21	98.09	97.10	97.10	97.10	99.51
犯罪基本情况语义建模	TextCNN	92.60	69.33	68.62	68.96	85.52	92.60	92.60	92.60	99.10
	TextCNN+Focal Loss	93.45	70.92	69.81	70.30	89.48	93.45	93.45	93.45	99.42
	TextCNN+Focal Loss+位置编码	94.21	71.05	69.12	70.00	94.13	94.21	94.21	94.21	99.52
	TextCNN+Focal Loss+位置编码+GAP	95.17	71.65	71.17	71.38	94.32	95.17	95.17	95.17	99.38
联合建模	CCHA-Net	99.57	74.75	74.62	74.68	99.45	99.57	99.57	99.57	99.89

Violent crime hierarchy algorithm by joint modeling of improved hierarchical attention network and TextCNN

基于改进分层注意网络和TextCNN联合建模的暴力犯罪分级算法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 39

Related Articles 15

Recommended Articles

Metrics

类型	模型	Acc	Macro_P	Macro_R	Macro_F1	Macro_AUC	Micro_P	Micro_R	Micro_F1	Micro_AUC
传统机器学习模型	KNN	83.49	66.31	58.16	60.94	80.92	83.49	83.49	83.49	94.50
	MNB	90.46	69.88	65.02	66.93	88.46	90.46	90.46	90.46	98.69
	GNB	73.85	57.57	46.48	48.60	81.41	73.85	73.85	73.85	92.72
	BNB	89.39	69.74	58.18	60.88	86.96	89.39	89.39	89.39	98.56
	DT	88.32	67.80	64.47	65.89	79.63	88.32	88.32	88.32	92.21
	RF	90.35	69.76	66.48	67.85	92.32	90.35	90.35	90.35	98.80
	SVM	95.28	72.16	71.53	71.79	90.15	95.28	95.28	95.28	99.50
	XGBoost	93.78	71.06	65.87	67.83	81.41	93.78	93.78	93.78	98.77
	LR	95.18	72.32	71.24	71.72	93.11	95.18	95.18	95.18	99.51
深度学习模型	LSTM	83.69	58.32	63.31	59.97	86.71	83.69	83.69	83.69	94.11
	Bi-LSTM	85.41	59.09	63.55	60.40	81.35	85.41	85.41	85.41	94.89
	GRU	88.84	62.28	65.13	63.46	80.79	88.84	88.84	88.84	96.84
	Bi-GRU	91.31	65.41	64.62	64.95	92.93	91.31	91.31	91.31	96.84
	Att-BiLSTM	92.17	69.77	70.02	69.77	85.71	92.17	92.17	92.17	98.27
	C-LSTM	93.78	71.38	70.47	70.82	81.36	93.78	93.78	93.78	98.34
	CNN-BiLSTM	94.53	72.02	71.09	71.45	89.81	94.53	94.53	94.53	98.40
	AC-BiLSTM	95.49	71.66	71.60	71.57	93.86	95.49	95.49	95.49	99.15
本文模型	CCHA-Net	99.57	74.75	74.62	74.68	99.45	99.57	99.57	99.57	99.89

模型	浮点运算量/FLOPs	参数量/10³
TextCNN通道	3.506	196.992
HAN通道	41.410	62.208
CCHA-Net	44.916	259.200

[1]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.
[2]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[3]	Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918.
[4]	Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703.
[5]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[6]	Yun LI, Fuyou WANG, Peiguang JING, Su WANG, Ao XIAO. Uncertainty-based frame associated short video event detection method [J]. Journal of Computer Applications, 2024, 44(9): 2903-2910.
[7]	Hong CHEN, Bing QI, Haibo JIN, Cong WU, Li’ang ZHANG. Class-imbalanced traffic abnormal detection based on 1D-CNN and BiGRU [J]. Journal of Computer Applications, 2024, 44(8): 2493-2499.
[8]	Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557.
[9]	Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625.
[10]	Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650.
[11]	Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318.
[12]	Yangyi GAO, Tao LEI, Xiaogang DU, Suiyong LI, Yingbo WANG, Chongdan MIN. Crowd counting and locating method based on pixel distance map and four-dimensional dynamic convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2233-2242.
[13]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.
[14]	Xun SUN, Ruifeng FENG, Yanru CHEN. Monocular 3D object detection method integrating depth and instance segmentation [J]. Journal of Computer Applications, 2024, 44(7): 2208-2215.
[15]	Zheng WU, Zhiyou CHENG, Zhentian WANG, Chuanjian WANG, Sheng WANG, Hui XU. Deep learning-based classification of head movement amplitude during patient anaesthesia resuscitation [J]. Journal of Computer Applications, 2024, 44(7): 2258-2263.