Abstract:

In current international society， as the international language， English characters appear in many public occasions， as well as the Chinese pinyin characters in Chinese environment. When these characters appear in the image， especially in the image with complex style， it is difficult to edit and modify them directly. In order to solve the problems， an image character editing method based on improved character generation network named Font Adaptive Neural network （FANnet） was proposed. Firstly， the salience detection algorithm based on Histogram Contrast （HC） was used to improve the Character Adaptive Detection （CAD） model to accurately extract the image characters selected by the user. Secondly， the binary image of the target character that was almost consistent with the font of the source character was generated by using FANnet. Then， the color of source characters were transferred to target characters effectively by the proposed Colors Distribute-based Local （CDL） transfer model based on color complexity discrimination. Finally， the target editable characters that were highly consistent with the font structure and color change of the source character were generated， so as to achieve the purpose of character editing. Experimental results show that， on MSRA-TD500， COCO-Text and ICDAR datasets， the average values of Structural SIMilarity（SSIM）， Peak Signal-to-Noise Ratio （PSNR） and Normalized Root Mean Square Error （NRMSE） of the proposed method are 0.776 5， 18.321 1 dB and 0.435 8 respectively， which are increased by 18.59%，14.02% and decreased by 2.97% comparing with those of Scene Text Editor using Font Adaptive Neural Network（STEFANN） algorithm respectively， and increased by 30.24%，23.92% and decreased by 4.68% comparing with those of multi-modal few-shot font style transfer model named Multi-Content GAN（MC-GAN） algorithm（with 1 input character）respectively. For the image characters with complex font structure and color gradient distribution in real scene， the editing effect of the proposed method is also good. The proposed method can be applied to image reuse， image character computer automatic error correction and image text information restorage.

Key words: Font Adaptive Neural network (FANnet), image character editing, Histogram Contrast (HC), salience detection, color transfer, font structure

摘要：

在当今国际化的社会，作为国际通用语言的英文字符及中文环境下的拼音字符出现在众多公共场合。当这些字符出现在图像中时，尤其在风格复杂的图像中时，难以直接对其进行编辑修改。针对上述问题，提出了一种改进文字生成网络（FANnet）的图像字符编辑方法。首先，利用基于直方图对比度（HC）的显著性检测算法改进自适应字符检测（CAD）模型，准确提取出用户所选择的图像字符；接着，根据FANnet，生成与源字符字体几乎一致的目标字符的二值图；然后，通过所提出的局部颜色分布（CDL）迁移模型，迁移源字符颜色至目标字符；最后，生成与源字符字体结构和颜色变化均高度一致的目标可编辑修改字符，从而达到字符编辑目的。实验结果表明，在MSRA-TD500、COCO-Text和ICDAR数据集上，所提方法的结构相似性（SSIM）、峰值信噪比（PSNR）和归一化均方根误差（NRMSE）平均值分别为0.776 5、18.321 1 dB和0.435 8，相较于基于字体自适应神经网络的场景文本编辑器（STEFANN）算法分别提高了18.59%、14.02%和降低了2.97%，相较于多模态小样本字体迁移模型MC-GAN算法（输入1个字符时）分别提高了30.24%、23.92%和降低了4.68%；而且针对字体结构和颜色渐变分布比较复杂的实际场景图像字符，所提方法的编辑效果也较好。该方法可以应用于图像重利用、图像字符计算机自动纠错和图像文本信息重存储。

关键词: 体自适应神经网络, 图像字符编辑, 直方图对比度, 显著性检测, 颜色迁移, 字体结构