Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (8): 2378-2385.DOI: 10.11772/j.issn.1001-9081.2019122146

• Virtual reality and multimedia computing • Previous Articles     Next Articles

Single image shadow removal method based on multistage generative adversarial network

ZHANG Shuping1, WU Wen1, WAN Yi2   

  1. 1. Department of Information Engineering, Xinjiang Institute of Technology, Aksu Xinjiang 843100, China;
    2. School of Electrical and Electronic Engineering, Wenzhou University, Wenzhou Zhejiang 325035, China
  • Received:2019-12-23 Revised:2020-03-24 Online:2020-08-10 Published:2020-05-14
  • Supported by:
    This work is partially supported by the Technology Plan Public Project of Zhejiang Province in China(LGG18F040002), the Natural Science Foundation of Zhejiang Province (LY19F020035).

基于多阶段生成对抗网络的单幅图像阴影去除方法

张淑萍1, 吴文1, 万毅2   

  1. 1. 新疆理工学院 信息工程系, 新疆 阿克苏 843100;
    2. 温州大学 电气与电子工程学院, 浙江 温州 325035
  • 通讯作者: 万毅(1974-),男,浙江温州人,教授,博士,主要研究方向:系统可行性、智能检测,1119764335@qq.com
  • 作者简介:张淑萍(1974-),女,甘肃武威人,讲师,硕士,主要研究方向:图像处理、深度学习;吴文(1994-),男,湖北武汉人,硕士,主要研究方向:图像处理、深度学习。
  • 基金资助:
    浙江省基础公益研究计划项目(LGG18F040002);浙江省自然科学基金资助项目(LY19F020035)。

Abstract: Traditional deep learning shadow removal methods often change the pixels in non-shadow areas and cannot obtain results with smooth boundary transition. In order to solve these problems, a new multistage shadow removal framework based on Generative Adversarial Network (GAN) was proposed. Firstly, shadow mask and shadow matte of the input image were generated by multitask driven generator via shadow detection subnet and shadow matter generation subnet respectively. Secondly, under the guidance of shadow mask and shadow matte, an umbra module and a penumbra module were designed respectively to remove different types of shadows successively. Thirdly, a new compose loss function dominated by least squares loss was created to obtain a better result. Compared with state-of-the-art shadow removal methods based on deep learning, the proposed method has the Balanced Error Rate (BER) averagely reduced by 4.39%, the Structural SIMilarity index (SSIM) averagely improved by 0.44%, and the Root Mean Square Error (RMSE) averagely reduced by 13.32%. Experimental results show that the boundary transition of shadow removal result of the proposed method is smoother.

Key words: deep learning, Generative Adversarial Network (GAN), image processing, shadow removal, shadow detection

摘要: 传统的深度学习阴影去除方法常常会改变非阴影区域的像素且无法得到边界过渡自然的阴影去除结果。为了解决该问题,基于生成对抗网络(GAN)提出一种新颖的多阶段阴影去除框架。首先,多任务驱动的生成器分别通过阴影检测子网和蒙版生成子网为输入图像生成相应的阴影掩膜和阴影蒙版;其次,在阴影掩膜和阴影蒙版的引导下,分别设计全影模块和半影模块,分阶段去除图像中不同类型的阴影;然后,以最小二乘损失为主导构建一种新的组合损失函数以得到更好的结果。与最新的深度学习阴影去除方法相比,在筛选数据集上,所提方法的平衡误差率(BER)减小约4.39%,结构相似性(SSIM)提高约0.44%,像素均方根误差(RMSE)减小约13.32%。实验结果表明该方法得到的阴影去除结果边界过渡更加平滑。

关键词: 深度学习, 生成对抗网络, 图像处理, 阴影去除, 阴影检测

CLC Number: