In the task of handwriting identification， the large area of image is background， handwriting information is sparse， key information is difficult to capture， and personal handwriting signature style has slight changes and handwriting imitated is highly similar， as well as there is few public Chinese handwriting datasets. By improving attention mechanism and Siamese network model， a handwriting identification method based on Multi-scale and Mixed Domain Attention mechanism （MMDANet） was proposed. Firstly， a maximum pooling layer was connected in parallel to the effective channel attention module， and to extend the number of channels of two-dimensional strip pooling module to three dimensions. The improved effective channel attention module and strip pooling module were fused to generate a Mixed Domain Module （MDM）， thereby solving the problems that large area of handwriting image is background， handwriting information is sparse and detailed features are difficult to extract. Secondly， the Path Aggregation Network （PANet） feature pyramid was used to extract features at multiple scales to capture the subtle differences between true and false handwriting， and the comparison loss of Siamese network and Additive Margin Softmax （AM-Softmax） loss were weightedly fused for training to increase the discrimination between categories and solve the problem of personal handwriting style variation and high similarity between true and false handwriting. Finally， a Chinese Handwriting Dataset （CHD） with a total sample size of 8 000 was self-made. The accuracy of the proposed method on the Chinese dataset CHD reached 84.25%； and compared with the suboptimal method Two-stage Siamese Network （Two-stage SiamNet）， the proposed method increased the accuracy by 4.53%， 1.02% and 1.67% respectively on three foreign language datasets Cedar， Bengla and Hindi. The experimental results show that the MMDANet can more accurately capture the subtle differences between true and false handwriting， and complete complex handwriting identification tasks.