基于多阶段生成对抗网络的单幅图像阴影去除方法

doi:10.11772/j.issn.1001-9081.2019122146

计算机应用 ›› 2020, Vol. 40 ›› Issue (8): 2378-2385.DOI: 10.11772/j.issn.1001-9081.2019122146

• 虚拟现实与多媒体计算 • 上一篇下一篇

基于多阶段生成对抗网络的单幅图像阴影去除方法

张淑萍¹, 吴文¹, 万毅²

1. 新疆理工学院信息工程系, 新疆阿克苏 843100;
2. 温州大学电气与电子工程学院, 浙江温州 325035

收稿日期:2019-12-23 修回日期:2020-03-24 出版日期:2020-08-10 发布日期:2020-05-14
通讯作者: 万毅(1974-),男,浙江温州人,教授,博士,主要研究方向:系统可行性、智能检测,1119764335@qq.com
作者简介:张淑萍(1974-),女,甘肃武威人,讲师,硕士,主要研究方向:图像处理、深度学习;吴文(1994-),男,湖北武汉人,硕士,主要研究方向:图像处理、深度学习。
基金资助:
浙江省基础公益研究计划项目（LGG18F040002）；浙江省自然科学基金资助项目（LY19F020035）。

Single image shadow removal method based on multistage generative adversarial network

ZHANG Shuping¹, WU Wen¹, WAN Yi²

1. Department of Information Engineering, Xinjiang Institute of Technology, Aksu Xinjiang 843100, China;
2. School of Electrical and Electronic Engineering, Wenzhou University, Wenzhou Zhejiang 325035, China

Received:2019-12-23 Revised:2020-03-24 Online:2020-08-10 Published:2020-05-14
Supported by:
This work is partially supported by the Technology Plan Public Project of Zhejiang Province in China(LGG18F040002), the Natural Science Foundation of Zhejiang Province (LY19F020035).

摘要/Abstract

摘要： 传统的深度学习阴影去除方法常常会改变非阴影区域的像素且无法得到边界过渡自然的阴影去除结果。为了解决该问题，基于生成对抗网络（GAN）提出一种新颖的多阶段阴影去除框架。首先，多任务驱动的生成器分别通过阴影检测子网和蒙版生成子网为输入图像生成相应的阴影掩膜和阴影蒙版；其次，在阴影掩膜和阴影蒙版的引导下，分别设计全影模块和半影模块，分阶段去除图像中不同类型的阴影；然后，以最小二乘损失为主导构建一种新的组合损失函数以得到更好的结果。与最新的深度学习阴影去除方法相比，在筛选数据集上，所提方法的平衡误差率（BER）减小约4.39%，结构相似性（SSIM）提高约0.44%，像素均方根误差（RMSE）减小约13.32%。实验结果表明该方法得到的阴影去除结果边界过渡更加平滑。

关键词: 深度学习, 生成对抗网络, 图像处理, 阴影去除, 阴影检测

Abstract: Traditional deep learning shadow removal methods often change the pixels in non-shadow areas and cannot obtain results with smooth boundary transition. In order to solve these problems, a new multistage shadow removal framework based on Generative Adversarial Network (GAN) was proposed. Firstly, shadow mask and shadow matte of the input image were generated by multitask driven generator via shadow detection subnet and shadow matter generation subnet respectively. Secondly, under the guidance of shadow mask and shadow matte, an umbra module and a penumbra module were designed respectively to remove different types of shadows successively. Thirdly, a new compose loss function dominated by least squares loss was created to obtain a better result. Compared with state-of-the-art shadow removal methods based on deep learning, the proposed method has the Balanced Error Rate (BER) averagely reduced by 4.39%, the Structural SIMilarity index (SSIM) averagely improved by 0.44%, and the Root Mean Square Error (RMSE) averagely reduced by 13.32%. Experimental results show that the boundary transition of shadow removal result of the proposed method is smoother.

Key words: deep learning, Generative Adversarial Network (GAN), image processing, shadow removal, shadow detection

中图分类号:

TP391.41

张淑萍, 吴文, 万毅. 基于多阶段生成对抗网络的单幅图像阴影去除方法[J]. 计算机应用, 2020, 40(8): 2378-2385.

ZHANG Shuping, WU Wen, WAN Yi. Single image shadow removal method based on multistage generative adversarial network[J]. Journal of Computer Applications, 2020, 40(8): 2378-2385.

参考文献

[1] GUO R, DAI Q Y, HOIEM D. Paired regions for shadow detection and removal[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2013, 35(12):2956-2967.
[2] KHAN S H, BENNAMOUN M, SOHEL F, et al. Automatic shadow detection and removal from a single image[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(3):431-446.
[3] VICENTE T F Y, HOAI M, SAMMARAS D. Leave-one-out kernel optimization for shadow detection and removal[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(3):682-695.
[4] 廖斌,吴文.区域配对引导的光照传播视频阴影去除方法[J].计算机应用,2019,39(2):556-563. (LIAO B, WU W. Video shadow removal using region matching guided by illumination transfer[J]. Journal of Computer Applications, 2019, 39(2):556-563.)
[5] ZHANG L, ZHU Y, LIAO B, et al. Video shadow removal using spatio-temporal illumination transfer[J]. Computer Graphics Forum, 2017, 36(7):125-134.
[6] SHEN L, CHUA T W, LEMAN K. Shadow optimization from structured deep edge detection[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2015:2067-2074.
[7] LIU F, GLEICHER M. Texture-consistent shadow removal[C]//Proceedings of the 10th European Conference on Computer Vision, LNCS 5305. Berlin:Springer, 2008:437-450.
[8] FINLAYSON G D, HORDLEY S D, DREW M S. Removing shadows from images[C]//Proceedings of the 7th European Conference on Computer Vision, LNCS 2353. Berlin:Springer, 2002:823-836.
[9] FINLAYSON G D, HORDLEY S D, LU C, et al. On the removal of shadows from images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(1):59-68.
[10] MOHAN A, TUMBLIN J, CHOUDHURY P. Editing soft shadows in a digital photograph[J]. IEEE Computer Graphics and Applications, 2007, 27(2):23-31.
[11] ZHANG L, YAN Q, ZHU Y, et al. Effective shadow removal via multi-scale image decomposition[J]. The Visual Computer, 2019, 35(6/7/8):1091-1104.
[12] ARBEL E, HEL-OR H. Shadow removal using intensity surfaces and texture anchor points[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(6):1202-1216.
[13] BARRON J T, MALIK J. Shape, illumination, and reflectance from shading[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(8):1670-1687.
[14] QU L, TIAN J, HE S, et al. DeshadowNet:a multi-context embedding deep network for shadow removal[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:2308-2316.
[15] WANG J, LI X, YANG J. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2018:1788-1797.
[16] 廖斌,谭道强,吴文. 基于衰减式生成对抗网络的单幅图像阴影去除[J]. 计算机应用, 2019, 39(9):2712-2718. (LIAO B, TAN D Q, WU W. Single image shadow removal based on attenuated generative adversarial networks[J]. Journal of Computer Applications, 2019, 39(9):2712-2718.)
[17] HU X, JIANG Y, FU C W, et al. Mask-ShadowGAN:learning to remove shadows from unpaired data[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2019:2472-2481.
[18] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge:MIT Press, 2014:2672-2680.
[19] MIRZA M, OSINDERO S. Conditional generative adversarial nets[EB/OL].[2019-12-25]. https://arxiv.org/pdf/1411.1784.pdf.
[20] ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:5967-5976.
[21] FAN H, HAN M, LI J. Image shadow removal using end-to-end deep convolutional neural networks[J]. Applied Sciences, 2019, 9(5):No.1009.
[22] JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-time style transfer and super-resolution[C]//Proceedings of the 14th European Conference on Computer Vision, LNCS 9906. Berlin:Springer, 2016:694-711.
[23] CHO D,TAI Y W,KWEON I S. Deep convolutional neural network for natural image matting using initial alpha mattes[J]. IEEE Transactions on Image Processing,2019,28(3):1054-1067.
[24] LOFFE S, SZEGEDY C. Batch normalization:accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on Machine Learning. New York:JMLR.org, 2015:448-456.
[25] MAO X, LI Q, XIE H, et al. Least squares generative adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2017:2813-2821.
[26] WONGSUPHASAWAT K, SMILKOV D, WEXLER J, et al. Visualizing dataflow graphs of deep learning models in TensorFlow[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 24(1):1-12.

基于多阶段生成对抗网络的单幅图像阴影去除方法

Single image shadow removal method based on multistage generative adversarial network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495.
[2]	代雨柔, 杨庆, 张凤荔, 周帆. 基于自监督学习的社交网络用户轨迹预测模型[J]. 计算机应用, 2021, 41(9): 2545-2551.
[3]	郑志强, 胡鑫, 翁智, 王雨禾, 程曦. 基于改进DenseNet的牛眼图像特征提取方法[J]. 计算机应用, 2021, 41(9): 2780-2784.
[4]	陈成瑞, 孙宁, 何世彪, 廖勇. 面向C-V2X通信的基于深度学习的联合信道估计与均衡算法[J]. 计算机应用, 2021, 41(9): 2687-2693.
[5]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[6]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[7]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287.
[8]	秦斌斌, 彭良康, 卢向明, 钱江波. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 41(8): 2330-2337.
[9]	管其杰, 张挺, 李德亚, 周绍景, 杜奕. 基于多分辨率生成对抗网络的空间数据不确定性重建方法[J]. 计算机应用, 2021, 41(8): 2306-2311.
[10]	孙潇, 徐金东. 基于级联生成对抗网络的遥感图像去雾方法[J]. 计算机应用, 2021, 41(8): 2440-2444.
[11]	何正海, 线岩团, 王蒙, 余正涛. 融合句法指导与字符注意力机制的案情阅读理解方法[J]. 计算机应用, 2021, 41(8): 2427-2431.
[12]	李亚芳, 梁烨, 冯韦玮, 祖宝开, 康玉健. 基于社区优化的深度网络嵌入方法[J]. 计算机应用, 2021, 41(7): 1956-1963.
[13]	汤桂花, 孙磊, 毛秀青, 戴乐育, 胡永进. 基于深度对齐网络的生成对抗网络伪造人脸检测[J]. 计算机应用, 2021, 41(7): 1922-1927.
[14]	侯笑晗, 金国栋, 谭力宁, 薛远亮. 基于自适应和最优特征的合成孔径雷达舰船检测方法[J]. 计算机应用, 2021, 41(7): 2150-2155.
[15]	高钦泉, 黄炳城, 刘文哲, 童同. 基于改进CenterNet的竹条表面缺陷检测方法[J]. 计算机应用, 2021, 41(7): 1933-1938.