Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (7): 2048-2053.DOI: 10.11772/j.issn.1001-9081.2020081184

Special Issue: 多媒体计算与计算机仿真

• Multimedia computing and computer simulation • Previous Articles     Next Articles

Image colorization algorithm based on foreground semantic information

WU Lidan1,2, XUE Yuyang3, TONG Tong1,2,4, DU Min2, GAO Qinquan1,2,4   

  1. 1. College of Physics and Information Engineering, Fuzhou University, Fuzhou Fujian 350108, China;
    2. Key Laboratory of Medical Instrumentation & Pharmaceutical Technology of Fujian Province(Fuzhou University), Fuzhou Fujian 350108, China;
    3. Department of Computer Science, University of Tsukuba, Tsukuba 3058577, Japan;
    4. Imperial Vision Technology Company Limited, Fuzhou Fujian 350001, China
  • Received:2020-08-10 Revised:2020-12-11 Online:2021-07-10 Published:2020-12-29
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61901120), the Science and Technology Major Project of Fujian Province (2019YZ016006).

基于前景语义信息的图像着色算法

吴丽丹1,2, 薛雨阳3, 童同1,2,4, 杜民2, 高钦泉1,2,4   

  1. 1. 福州大学 物理与信息工程学院, 福州 350108;
    2. 福建省医疗器械与医药技术重点实验室(福州大学), 福州 350108;
    3. 筑波大学 计算机科学学院, 筑波 3058577, 日本;
    4. 福建帝视信息科技有限公司, 福州 350001
  • 通讯作者: 童同
  • 作者简介:吴丽丹(1996-),女,福建龙岩人,硕士研究生,主要研究方向:计算机视觉、图像处理;薛雨阳(1994-),男,福建福州人,博士研究生,主要研究方向:计算机视觉、图像处理;童同(1986-),男,安徽安庆人,教授,博士,主要研究方向:计算机视觉、医学影像处理、脑疾病辅助诊断;杜民(1955-),女,福建泉州人,教授,博士,主要研究方向:传感技术、生物医学仪器;高钦泉(1986-),男,福建福州人,副教授,博士,主要研究方向:计算机视觉、增强现实、医学影像处理。
  • 基金资助:
    国家自然科学基金资助项目(61901120);福建省科技厅重大专项(2019YZ016006)。

Abstract: An image can be divided into foreground part and background part, while the foreground is often the visual center. Due to the large categories and complex situations of foreground part, the image colorization is difficult, thus the foreground part of an image may suffer from poor colorization and detail loss problems. To solve these problems, an image colorization algorithm based on foreground semantic information was proposed to improve the image colorization effect and achieve the purpose of natural overall image color and rich content color. First, the foreground network was used to extract the low-level features and high-level features of the foreground part. Then these features were integrated into the foreground subnetwork to eliminate the influence of background color information and emphasize the foreground color information. Finally, the network was continuously optimized by the generation loss and pixel-level color loss, so as to guide the generation of high-quality images. Experimental results show that after introducing the foreground semantic information, the proposed algorithm improves Peak Signal-to-Noise Ratio (PSNR) and Learned Perceptual Image Patch Similarity (LPIPS), effectively solving the problems of dull color, detail loss and low contrast in the colorization of the central visual regions; compared with other algorithms, the proposed algorithm achieves a more natural colorization effect on the overall image and a significant improvement on the content part.

Key words: image colorization, feature fusion, grayscale image, foreground semantic information

摘要: 图像可分为前景部分与背景部分,而前景往往是视觉中心。在图像着色任务上,由于前景的类别多且情况复杂,着色困难,以至于图像中的前景部分会存在着色暗淡和细节丢失等问题。针对这些问题,提出了基于前景语义信息的图像着色算法,以改善图像着色效果,达到图像整体颜色自然、内容颜色丰富的目的。首先利用前景子网提取前景部分的低级特征和高级特征;然后将这些特征融合到全景子网训练中,以排除背景颜色信息影响并强调前景颜色信息;最后用生成损失和像素级别的颜色损失来不断优化网络,指导生成高质量图像。实验结果表明,引入前景语义信息后,所提算法在峰值信噪比(PSNR)和感知相似度(LPIPS)上有所提升,可有效改善视觉中心区域着色中的色泽暗淡、细节丢失、对比度低等问题;相比其他算法,该算法在图像整体上取得了更自然的着色效果,在内容部分上取得了显著的改进。

关键词: 图像着色, 特征融合, 灰度图像, 前景语义信息

CLC Number: