Handwriting identification method based on multi-scale mixed domain attention mechanism

doi:10.11772/j.issn.1001-9081.2023071018

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (7): 2225-2232.DOI: 10.11772/j.issn.1001-9081.2023071018

• Multimedia computing and computer simulation • Previous Articles Next Articles

Handwriting identification method based on multi-scale mixed domain attention mechanism

Wu XIONG¹^,², Congjun CAO¹^,²(), Xuefang SONG¹^,², Yunlong SHAO¹^,², Xusheng WANG¹^,²

^1.Faculty of Printing，Packaging Engineering and Digital Media Technology，Xi’an University of Technology，Xi’an Shaanxi 710054，China
^2.Printing & Packaging Engineering Technology Research Center of Shaanxi Province，Xi’an Shaanxi 710054，China

Received:2023-07-27 Revised:2023-09-19 Accepted:2023-09-25 Online:2023-10-26 Published:2024-07-10
Contact: Congjun CAO
About author:XIONG Wu， born in 1998， M. S. candidate. His research interests include handwriting identification， data hiding.
SONG Xuefang， born in 1996， M. S. candidate. Her research interests include deep learning， handwriting identification.
SHAO Yunlong， born in 2000， M. S. candidate. His research interests include deep learning， few-shot object detection.
WANG Xusheng， born in 1988， Ph. D.， associate professor. His research interests include deep learning， big data mining and analysis.
First author contact:CAO Congjun， born in 1970， Ph. D.， professor. Her research interests include color management and image reproduction， printing quality measurement and control and information traceability.
Supported by:
Key Scientific Research Base Project of Shaanxi Province(2023 HBGC-18)

基于多尺度混合域注意力机制的笔迹鉴别方法

熊武¹^,², 曹从军¹^,²(), 宋雪芳¹^,², 邵云龙¹^,², 王旭升¹^,²

^1.西安理工大学印刷包装与数字媒体学院，西安 710054
^2.陕西省印刷包装工程技术研究中心，西安 710054

通讯作者: 曹从军
作者简介:熊武（1998—），男，陕西商洛人，硕士研究生，主要研究方向：笔迹鉴别、信息隐藏；
宋雪芳（1996—），女，河南商丘人，硕士研究生，主要研究方向：深度学习、笔迹鉴别；
邵云龙（2000—），男，安徽合肥人，硕士研究生，主要研究方向：深度学习、小样本目标检测；
王旭升（1988—），男，山西运城人，副教授，博士，主要研究方向：深度学习、大数据挖掘与分析。
第一联系人：曹从军（1970—），女，陕西西安人，教授，博士，主要研究方向：色彩管理与图像复现、印刷质量测控与信息追溯；
基金资助:
陕西省重点科研基地项目(2023 HBGC-18)

Abstract

Abstract:

In the task of handwriting identification， the large area of image is background， handwriting information is sparse， key information is difficult to capture， and personal handwriting signature style has slight changes and handwriting imitated is highly similar， as well as there is few public Chinese handwriting datasets. By improving attention mechanism and Siamese network model， a handwriting identification method based on Multi-scale and Mixed Domain Attention mechanism （MMDANet） was proposed. Firstly， a maximum pooling layer was connected in parallel to the effective channel attention module， and to extend the number of channels of two-dimensional strip pooling module to three dimensions. The improved effective channel attention module and strip pooling module were fused to generate a Mixed Domain Module （MDM）， thereby solving the problems that large area of handwriting image is background， handwriting information is sparse and detailed features are difficult to extract. Secondly， the Path Aggregation Network （PANet） feature pyramid was used to extract features at multiple scales to capture the subtle differences between true and false handwriting， and the comparison loss of Siamese network and Additive Margin Softmax （AM-Softmax） loss were weightedly fused for training to increase the discrimination between categories and solve the problem of personal handwriting style variation and high similarity between true and false handwriting. Finally， a Chinese Handwriting Dataset （CHD） with a total sample size of 8 000 was self-made. The accuracy of the proposed method on the Chinese dataset CHD reached 84.25%； and compared with the suboptimal method Two-stage Siamese Network （Two-stage SiamNet）， the proposed method increased the accuracy by 4.53%， 1.02% and 1.67% respectively on three foreign language datasets Cedar， Bengla and Hindi. The experimental results show that the MMDANet can more accurately capture the subtle differences between true and false handwriting， and complete complex handwriting identification tasks.

Key words: handwriting identification, Siamese network, attention mechanism, multi-scale, mixed domain

摘要：

针对笔迹鉴别任务中图像大面积是背景、笔迹信息稀疏、关键性信息难以捕捉，并且个人笔迹签名风格具有微小变化而刻意模仿的笔迹高度相似，以及公开的中文笔迹数据集的匮乏的问题，通过对注意力机制和孪生网络模型进行改进，提出一种基于多尺度混合域注意力机制的笔迹鉴别方法（MMDANet）。首先，在有效通道注意力模块上并联一个最大池化层，并将二维条带池化模块的通道数扩展到三维，将改进的有效通道注意力模块和条带池化模块融合生成混合域模块（MDM），解决了笔迹图像大面积是背景、笔迹信息稀疏、细节特征难以提取的问题；其次，利用PANet特征金字塔进行多尺度提取特征，捕获真伪笔迹间的细微差异，采用孪生网络的对比损失与AM-Softmax损失加权融合进行训练，增加类别间的区分度，解决个人笔迹风格变化和真伪笔迹高度相似的问题；最后自制了总体样本数为8 000的中文笔迹数据集（CHD）。所提方法在自制中文数据集CHD上的准确率达到了84.25%，且相较于次优的Two-stage SiamNet方法，所提方法在3个外文数据集Cedar、Bengla和Hindi上准确率分别提升了4.53%、1.02%和1.67%。实验结果表明，MMDANet可以更准确地捕获真伪笔迹的细微差异，完成复杂的笔迹鉴别任务。

关键词: 笔迹鉴别, 孪生网络, 注意力机制, 多尺度, 混合域

CLC Number:

TP391.43

Wu XIONG, Congjun CAO, Xuefang SONG, Yunlong SHAO, Xusheng WANG. Handwriting identification method based on multi-scale mixed domain attention mechanism[J]. Journal of Computer Applications, 2024, 44(7): 2225-2232.

熊武, 曹从军, 宋雪芳, 邵云龙, 王旭升. 基于多尺度混合域注意力机制的笔迹鉴别方法[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2225-2232.

Figures/Tables 12

References 24

1	杨维斌，房斌，尚赵伟，等.基于复小波的脱线手写体笔迹鉴别［J］.计算机应用， 2009， 29（6）： 1696-1698.
	YANG W B， FANG B， SHANG Z W， et al. Handwriting-based writer identification with complex wavelet transform ［J］. Journal of Computer Applications， 2009， 29（6）： 1696-1698.
2	马小晴，桑庆兵.基于LBP和深度学习的手写签名识别算法［J］.量子电子学报， 2017， 34（1）： 23-31.
	MA X Q， SANG Q B. Handwritten signature verification algorithm based on LBP and deep learning ［J］. Acta Electronica Sinica， 2017， 34（1）： 23-31.
3	王执竞，尹凡，陈庆虎.手写笔迹的表面法向量提取与鉴别系统［J］.计算机仿真， 2022， 39（4）： 156-160.
	WANG Z J， YIN F， CHEN Q H. Surface normal vector extraction and identification system of handwriting ［J］. Computer Simulation， 2022， 39（4）： 156-160.
4	HAFEMANN L G， SABOURIN R， OLIVEIRA L S. Learning features for off-line handwritten signature verification using deep convolutional neural networks ［J］. Pattern Recognition， 2017， 70： 163-176.
5	OOI S Y， TEOH A B J， PANG Y H， et al. Image-based hand-written signature verification using hybrid methods of discrete Radon transform， principal component analysis and probabilistic neural network ［J］. Applied Soft Computing， 2016， 40： 274-282.
6	CHEN S， SRIHARI S. A new off-line signature verification method based on graph ［C］// Proceedings of the 18th International Conference on Pattern Recognition. Piscataway： IEEE， 2016： 869-872.
7	KALERA M K， SRIHARI S N， XU A. Offline signature verification and identification using distance statistics ［J］. International Journal of Pattern Recognition and Artificial Intelligence， 2004， 18（7）： 1339-1360.
8	CHEN S， SRIHARI S. Use of exterior contours and shape features in off-line signature verification ［C］// Proceedings of the 8th International Conference on Document Analysis and Recognition. Piscataway： IEEE， 2005， 2： 1280-1284.
9	KUMAR R， SHARMA J D， CHANDA B. Writer-independent off-line signature verification using surroundedness feature ［J］. Pattern Recognition Letters， 2012， 33（3）： 301-308.
10	PAL S， ALAEI A， PAL U， et al. Performance of an off-line signature verification method based on texture features on a large indic-script signature dataset ［C］// Proceedings of the 2016 12th IAPR Workshop on Document Analysis Systems. Piscataway： IEEE， 2016： 72-77.
11	MRIDHA M F， OHI A Q， ALI M A， et al. BanglaWriting： a multi-purpose off-line Bangla handwriting dataset ［J］. Data in Brief， 2021， 34： 106633.
12	WU D， LUO X， SHANG M， et al. A data-characteristic-aware latent factor model for Web services QoS prediction ［J］. IEEE Transactions on Knowledge and Data Engineering， 2022， 34（6）： 2525-2538.
13	DEY S， DUTTA A， TOLEDO J I， et al. SigNet： convolutional Siamese network for writer independent off-line signature verification ［EB/OL］. （2017-09-30）［2023-07-23］. .
14	LI C， LIN F， WANG Z， et al. DeepHSV： user-independent off-line signature verification using two-channel CNN ［C］// Proceedings of the 2019 International Conference on Document Analysis and Recognition. Piscataway： IEEE， 2019： 166-171.
15	SHAIKH M A， DUAN T， CHAUHAN M， et al. Attention based writer independent verification ［C］// Proceedings of the 2020 17th International Conference on Frontiers in Handwriting Recognition. Piscataway： IEEE， 2020： 373-379.
16	XIAO W， DING Y. A two-stage Siamese network model for off-line handwritten signature verification ［J］. Symmetry， 2022， 14（6）： 1216.
17	SHAIKH M A， CHAUHAN M， CHU J， et al. Hybrid feature learning for handwriting verification ［C］// Proceedings of the 2018 16th International Conference on Frontiers in Handwriting Recognition. Piscataway： IEEE， 2018： 187-192.
18	CHU J， SHAIKH M A， CHAUHAN M， et al. Writer verification using CNN feature extraction ［C］// Proceedings of the 2018 16th International Conference on Frontiers in Handwriting Recognition. Piscataway： IEEE， 2018： 181-186.
19	KOCH G， ZEMEL R， SALAKHUTDINOV R. Siamese neural networks for one-shot image recognition ［C/OL］// Proceedings of the 32nd International Conference on Machine Learning.［2023-07-01］. .
20	KRISHNA V A， REDDY A， NAGAJYOTHI D. Signature recognition using Siamese neural networks ［C］// Proceedings of the 2021 IEEE International Conference on Mobile Networks and Wireless Communications. Piscataway： IEEE， 2021： 1-4.
21	WANG Q， WU B， ZHU P， et al. ECA-Net： efficient channel attention for deep convolutional neural networks ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 11531-11539.
22	HUANG G， SUN Y， LIU Z， et al. Deep networks with stochastic depth ［C］// Proceedings of the 14th European Conference on Computer Vision. Cham： Springer， 2016： 646-661.
23	LIN T-Y， DOLLÁR P， GIRSHICK R， et al. Feature pyramid networks for object detection ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 936-944.
24	WANG F， CHENG J， LIU W， et al. Additive margin Softmax for face verification ［J］. IEEE Signal Processing Letters， 2018， 25（7）： 926-930.

数据集	方法	准确率	FAR	FRR
Cedar	Texture Feature^*	92.10	7.70	8.20
	Two-stage SiamNet	95.66	4.20	6.78
	SigNet	100.00	0.00	0.00
	DeepHSV	100.00	0.00	0.00
	MMDANet	100.00	0.00	0.00
Bengla	Texture Feature^*	76.68	24.19	24.47
	SigNet	86.11	13.56	13.89
	DeepHSV	88.08	11.64	11.92
	Two-stage SiamNet	90.64	8.52	8.86
	MMDANet	91.57	8.21	8.43
Hindi	Texture Feature^*	75.53	24.67	24.94
	SigNet	84.64	15.07	15.36
	DeepHSV	86.60	13.06	13.34
	Two-stage SiamNet	88.98	11.24	11.47
	MMDANet	90.47	9.18	9.53

数据集	方法	准确率	FAR	FRR
Cedar	Texture Feature^*	92.10	7.70	8.20
	Two-stage SiamNet	95.66	4.20	6.78
	SigNet	100.00	0.00	0.00
	DeepHSV	100.00	0.00	0.00
	MMDANet	100.00	0.00	0.00
Bengla	Texture Feature^*	76.68	24.19	24.47
	SigNet	86.11	13.56	13.89
	DeepHSV	88.08	11.64	11.92
	Two-stage SiamNet	90.64	8.52	8.86
	MMDANet	91.57	8.21	8.43
Hindi	Texture Feature^*	75.53	24.67	24.94
	SigNet	84.64	15.07	15.36
	DeepHSV	86.60	13.06	13.34
	Two-stage SiamNet	88.98	11.24	11.47
	MMDANet	90.47	9.18	9.53

数据集	方法	准确率	FAR	FRR
Cedar	VGG	97.02	2.92	3.04
	VGG + IECA	98.78	1.22	1.34
	VGG + MSPM	100.00	0.00	0.00
	VGG + MDM	100.00	0.00	0.00
	VGG + PANet	100.00	0.00	0.00
	VGG + MDM + PANet	100.00	0.00	0.00
Bengla	VGG	81.90	18.10	18.10
	VGG + IECA	83.78	16.22	16.22
	VGG + MSPM	86.60	13.40	13.40
	VGG + MDM	89.96	11.01	9.07
	VGG + PANet	88.46	11.54	11.54
	VGG + MDM + PANet	91.57	8.21	8.43
Hindi	VGG	79.80	21.07	21.20
	VGG + IECA	82.48	17.21	17.52
	VGG + MSPM	86.37	13.35	13.63
	VGG + MDM	88.98	11.02	11.32
	VGG + PANet	87.46	12.23	12.54
	VGG + MDM + PANet	90.47	9.18	9.53

数据集	方法	准确率	FAR	FRR
Cedar	VGG	97.02	2.92	3.04
	VGG + IECA	98.78	1.22	1.34
	VGG + MSPM	100.00	0.00	0.00
	VGG + MDM	100.00	0.00	0.00
	VGG + PANet	100.00	0.00	0.00
	VGG + MDM + PANet	100.00	0.00	0.00
Bengla	VGG	81.90	18.10	18.10
	VGG + IECA	83.78	16.22	16.22
	VGG + MSPM	86.60	13.40	13.40
	VGG + MDM	89.96	11.01	9.07
	VGG + PANet	88.46	11.54	11.54
	VGG + MDM + PANet	91.57	8.21	8.43
Hindi	VGG	79.80	21.07	21.20
	VGG + IECA	82.48	17.21	17.52
	VGG + MSPM	86.37	13.35	13.63
	VGG + MDM	88.98	11.02	11.32
	VGG + PANet	87.46	12.23	12.54
	VGG + MDM + PANet	90.47	9.18	9.53

特征提取网络	准确率	FAR	FRR
VGG	70.10	29.60	30.20
VGG + IECA	78.06	20.96	21.84
VGG + MSPM	79.41	20.16	21.02
VGG + MDM	82.16	17.72	17.96
VGG + PANet	80.76	18.12	20.36
VGG + MDM + PANet	84.25	15.47	16.03

Handwriting identification method based on multi-scale mixed domain attention mechanism

基于多尺度混合域注意力机制的笔迹鉴别方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 24

Related Articles 15

Recommended Articles

Metrics

训练集	测试集
训练集	Cedar	Bengla	Hindi	CHD
Cedar	100.00	64.80	57.37	50.50
Bengla	80.45	91.57	68.48	51.47
Hindi	75.87	66.79	90.47	53.56
CHD	55.67	56.78	55.74	84.25

MDM模块数	不同数据集中的准确率/%
MDM模块数	CHD	Hindi	Bengla
0	70.10	79.80	81.90
1	80.74	84.90	86.10
2	81.06	86.40	89.45
3	82.16	88.98	89.96

[1]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[2]	Yan RONG, Jiawen LIU, Xinlei LI. Adaptive hybrid network for affective computing in student classroom [J]. Journal of Computer Applications, 2024, 44(9): 2919-2930.
[3]	Liting LI, Bei HUA, Ruozhou HE, Kuang XU. Multivariate time series prediction model based on decoupled attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2732-2738.
[4]	Zhiqiang ZHAO, Peihong MA, Xinhong HEI. Crowd counting method based on dual attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2886-2892.
[5]	Kaipeng XUE, Tao XU, Chunjie LIAO. Multimodal sentiment analysis network with self-supervision and multi-layer cross attention [J]. Journal of Computer Applications, 2024, 44(8): 2387-2392.
[6]	Pengqi GAO, Heming HUANG, Yonghong FAN. Fusion of coordinate and multi-head attention mechanisms for interactive speech emotion recognition [J]. Journal of Computer Applications, 2024, 44(8): 2400-2406.
[7]	Tong CHEN, Fengyu YANG, Yu XIONG, Hong YAN, Fuxing QIU. Construction method of voiceprint library based on multi-scale frequency-channel attention fusion [J]. Journal of Computer Applications, 2024, 44(8): 2407-2413.
[8]	Zhonghua LI, Yunqi BAI, Xuejin WANG, Leilei HUANG, Chujun LIN, Shiyu LIAO. Low illumination face detection based on image enhancement [J]. Journal of Computer Applications, 2024, 44(8): 2588-2594.
[9]	Chenqian LI, Jun LIU. Ultrasound carotid plaque segmentation method based on semi-supervision and multi-scale cascaded attention [J]. Journal of Computer Applications, 2024, 44(8): 2604-2610.
[10]	Shangbin MO, Wenjun WANG, Ling DONG, Shengxiang GAO, Zhengtao YU. Single-channel speech enhancement based on multi-channel information aggregation and collaborative decoding [J]. Journal of Computer Applications, 2024, 44(8): 2611-2617.
[11]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[12]	Dianhui MAO, Xuebo LI, Junling LIU, Denghui ZHANG, Wenjing YAN. Chinese entity and relation extraction model based on parallel heterogeneous graph and sequential attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2018-2025.
[13]	Wei LI, Xiaorong ZHANG, Peng CHEN, Qing LI, Changqing ZHANG. Crowd counting algorithm with multi-scale fusion based on normal inverse Gamma distribution [J]. Journal of Computer Applications, 2024, 44(7): 2243-2249.
[14]	Li LIU, Haijin HOU, Anhong WANG, Tao ZHANG. Generative data hiding algorithm based on multi-scale attention [J]. Journal of Computer Applications, 2024, 44(7): 2102-2109.
[15]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.