Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (7): 2225-2232.DOI: 10.11772/j.issn.1001-9081.2023071018

• Multimedia computing and computer simulation • Previous Articles     Next Articles

Handwriting identification method based on multi-scale mixed domain attention mechanism

Wu XIONG1,2, Congjun CAO1,2(), Xuefang SONG1,2, Yunlong SHAO1,2, Xusheng WANG1,2   

  1. 1.Faculty of Printing,Packaging Engineering and Digital Media Technology,Xi’an University of Technology,Xi’an Shaanxi 710054,China
    2.Printing & Packaging Engineering Technology Research Center of Shaanxi Province,Xi’an Shaanxi 710054,China
  • Received:2023-07-27 Revised:2023-09-19 Accepted:2023-09-25 Online:2023-10-26 Published:2024-07-10
  • Contact: Congjun CAO
  • About author:XIONG Wu, born in 1998, M. S. candidate. His research interests include handwriting identification, data hiding.
    SONG Xuefang, born in 1996, M. S. candidate. Her research interests include deep learning, handwriting identification.
    SHAO Yunlong, born in 2000, M. S. candidate. His research interests include deep learning, few-shot object detection.
    WANG Xusheng, born in 1988, Ph. D., associate professor. His research interests include deep learning, big data mining and analysis.
    First author contact:CAO Congjun, born in 1970, Ph. D., professor. Her research interests include color management and image reproduction, printing quality measurement and control and information traceability.
  • Supported by:
    Key Scientific Research Base Project of Shaanxi Province(2023 HBGC-18)

基于多尺度混合域注意力机制的笔迹鉴别方法

熊武1,2, 曹从军1,2(), 宋雪芳1,2, 邵云龙1,2, 王旭升1,2   

  1. 1.西安理工大学 印刷包装与数字媒体学院,西安 710054
    2.陕西省印刷包装工程技术研究中心,西安 710054
  • 通讯作者: 曹从军
  • 作者简介:熊武(1998—),男,陕西商洛人,硕士研究生,主要研究方向:笔迹鉴别、信息隐藏;
    宋雪芳(1996—),女,河南商丘人,硕士研究生,主要研究方向:深度学习、笔迹鉴别;
    邵云龙(2000—),男,安徽合肥人,硕士研究生,主要研究方向:深度学习、小样本目标检测;
    王旭升(1988—),男,山西运城人,副教授,博士,主要研究方向:深度学习、大数据挖掘与分析。
    第一联系人:曹从军(1970—),女,陕西西安人,教授,博士,主要研究方向:色彩管理与图像复现、印刷质量测控与信息追溯;
  • 基金资助:
    陕西省重点科研基地项目(2023 HBGC-18)

Abstract:

In the task of handwriting identification, the large area of image is background, handwriting information is sparse, key information is difficult to capture, and personal handwriting signature style has slight changes and handwriting imitated is highly similar, as well as there is few public Chinese handwriting datasets. By improving attention mechanism and Siamese network model, a handwriting identification method based on Multi-scale and Mixed Domain Attention mechanism (MMDANet) was proposed. Firstly, a maximum pooling layer was connected in parallel to the effective channel attention module, and to extend the number of channels of two-dimensional strip pooling module to three dimensions. The improved effective channel attention module and strip pooling module were fused to generate a Mixed Domain Module (MDM), thereby solving the problems that large area of handwriting image is background, handwriting information is sparse and detailed features are difficult to extract. Secondly, the Path Aggregation Network (PANet) feature pyramid was used to extract features at multiple scales to capture the subtle differences between true and false handwriting, and the comparison loss of Siamese network and Additive Margin Softmax (AM-Softmax) loss were weightedly fused for training to increase the discrimination between categories and solve the problem of personal handwriting style variation and high similarity between true and false handwriting. Finally, a Chinese Handwriting Dataset (CHD) with a total sample size of 8 000 was self-made. The accuracy of the proposed method on the Chinese dataset CHD reached 84.25%; and compared with the suboptimal method Two-stage Siamese Network (Two-stage SiamNet), the proposed method increased the accuracy by 4.53%, 1.02% and 1.67% respectively on three foreign language datasets Cedar, Bengla and Hindi. The experimental results show that the MMDANet can more accurately capture the subtle differences between true and false handwriting, and complete complex handwriting identification tasks.

Key words: handwriting identification, Siamese network, attention mechanism, multi-scale, mixed domain

摘要:

针对笔迹鉴别任务中图像大面积是背景、笔迹信息稀疏、关键性信息难以捕捉,并且个人笔迹签名风格具有微小变化而刻意模仿的笔迹高度相似,以及公开的中文笔迹数据集的匮乏的问题,通过对注意力机制和孪生网络模型进行改进,提出一种基于多尺度混合域注意力机制的笔迹鉴别方法(MMDANet)。首先,在有效通道注意力模块上并联一个最大池化层,并将二维条带池化模块的通道数扩展到三维,将改进的有效通道注意力模块和条带池化模块融合生成混合域模块(MDM),解决了笔迹图像大面积是背景、笔迹信息稀疏、细节特征难以提取的问题;其次,利用PANet特征金字塔进行多尺度提取特征,捕获真伪笔迹间的细微差异,采用孪生网络的对比损失与AM-Softmax损失加权融合进行训练,增加类别间的区分度,解决个人笔迹风格变化和真伪笔迹高度相似的问题;最后自制了总体样本数为8 000的中文笔迹数据集(CHD)。所提方法在自制中文数据集CHD上的准确率达到了84.25%,且相较于次优的Two-stage SiamNet方法,所提方法在3个外文数据集Cedar、Bengla和Hindi上准确率分别提升了4.53%、1.02%和1.67%。实验结果表明,MMDANet可以更准确地捕获真伪笔迹的细微差异,完成复杂的笔迹鉴别任务。

关键词: 笔迹鉴别, 孪生网络, 注意力机制, 多尺度, 混合域

CLC Number: