基于增强生成器条件生成对抗网络的单幅图像去雾

doi:10.11772/j.issn.1001-9081.2021010092

摘要/Abstract

摘要：

大气中烟雾等粒子的存在会导致肉眼捕获场景的能见度降低。大多数传统的去雾方法都是预期估计雾霾场景的透射率、大气光，并利用大气散射模型恢复无雾图像。这些方法尽管取得了显著进展，但由于过分依赖苛刻的先验条件，在缺乏相应先验条件下的去雾效果并不理想。因此，提出一种端到端的一体化除雾网络，使用增强生成器的条件生成对抗网络（CGAN）直接恢复无雾图像。生成器端以U-Net作为基础架构，通过“整合-加强-减去”的促进策略，用一个简单有效的增强解码器，增强解码器中特征的恢复。另外，加入了多尺度结构相似性（MS-SSIM）损失函数，增强图像的边缘细节恢复。在合成数据集和真实数据集上的实验中，该模型的峰值信噪比（PSNR）和结构相似性（SSIM）明显优于传统的暗通道先验（DCP）、一体化除雾网络（AOD-Net）、渐进式特征融合网络（PFFNet）、条件Wasserstein生成对抗网络（CWGAN）去雾模型。实验结果表明，相较于对比算法，所提网络能够恢复出更接近于地面真相的无雾图像，除雾效果更优。

关键词: 深度学习, 图像去雾, 生成对抗网络, 增强解码器, 多尺度结构相似性损失函数

Abstract:

The presence of particles such as smoke in the atmosphere can lead to reduced visibility in scenes captured by the naked eye. Most traditional dehazing methods estimate the transmissivity and atmospheric light of the haze scene， and restore the image without haze by using atmospheric scattering model. Although these methods have made significant progresses， due to excessive reliance on harsh prior conditions， the dehazing effect in the absence of corresponding prior conditions is not ideal. Therefore， an end-to-end integrated dehazing network was proposed， in which the Conditional Generative Adversarial Network （CGAN） with enhanced generator was used to directly restore the image without haze. In the generator side， U-Net was used as the basic structure， and a simple and effective enhanced decoder was used through the promotion strategy of “integration-enhance-subtraction” to enhance the recovery of features in the decoder. In addition， the Multi-Scale Structural SIMilarity （MS-SSIM） loss function was added to enhance the restoration of the edge details of the image. In experiments on synthetic and real datasets， the model was significantly better than the traditional dehazing models such as Dark Channel Prior （DCP）， All-in-One Dehazing Network （AOD-Net）， Progressive Feature Fusion Network （PFFNet） and Conditional Wasserstein Generative Adversarial Network （CWGAN） in Peak Signal-to-Noise Ratio （PSNR） and Structural Similarity （SSIM）. Experimental results show that compared with the comparison algorithms， the proposed network can recover images without haze closer to the ground truth with better dehazing effect.

Key words: deep learning, image dehazing, Generative Adversarial Network (GAN), enhanced decoder, Multi-Scale Structural SIMilarity (MS-SSIM) loss function

中图分类号:

TP391.41

赵扬, 李波. 基于增强生成器条件生成对抗网络的单幅图像去雾[J]. 计算机应用, 2021, 41(12): 3686-3691.

Yang ZHAO, Bo LI. Single image dehazing based on conditional generative adversarial network with enhanced generator[J]. Journal of Computer Applications, 2021, 41(12): 3686-3691.

图/表 7

图1 生成器的结构

Fig.1 Structure of generator

图2 SOS增强模块

Fig. 2 SOS boosted module

图3 鉴别器的结构

Fig. 3 Structure of discriminator

表1 在合成数据集改进前后U-Net以及损失函数的比较

Tab.1 Comparison of U-Net and loss function before and after improvement on synthetic datasets

模型	SOTS		NTIRE18
模型	PSNR/dB	SSIM	PSNR/dB	SSIM
原始U-Net	22.953 2	0.908 4	16.426 0	0.695 1
SSIM Loss	23.156 7	0.912 6	16.452 0	0.675 4
增强U-Net+ MS-SSIM Loss	23.860 6	0.917 3	16.474 4	0.712 0

表2 在合成数据集上本文方法与其他方法的定量比较

Tab.2 Quantitative comparison of the proposed method and other methods on synthetic datasets

方法	SOTS		NTIRE18
方法	PSNR/dB	SSIM	PSNR/dB	SSIM
DCP	18.583 5	0.794 1	14.439 6	0.571 9
AOD-Net	19.037 1	0.843 5	15.183 4	0.632 4
PFFNet	20.882 6	0.825 5	15.996 5	0.617 1
CWGAN	21.769 9	0.871 4	16.442 9	0.686 1
本文方法	23.860 6	0.917 3	16.474 4	0.712 0

图4 合成数据集上本文方法与其他方法的定量对比结果图

Fig.4 Result images of quantitative comparison of the proposed method and other methods on synthetic datasets

图5 真实数据集上本文方法与其他方法的定性对比结果图

Fig.5 Result images of qualitative comparison of the proposed method and other methods on real datasets

参考文献 22

1	FATTAL R. Single image dehazing［J］. ACM Transactions on Graphics， 2008， 27（3）： 1-9. 10.1145/1360612.1360671
2	RONNEBERGER O， FISCHER P， BROX T. U-net： convolutional networks for biomedical image segmentation［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS9351. Cham： Springer， 2015： 234-241.
3	ZHANG H， SINDAGI V， PATEL V M. Multi-scale single image dehazing using perceptual pyramid deep network［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2018： 1015-101509. 10.1109/cvprw.2018.00135
4	CHEN D D， HE M M， FAN Q N， et al. Gated context aggregation network for image dehazing and deraining［C］// Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision. Piscataway： IEEE， 2019：1375-1383. 10.1109/wacv.2019.00151
5	DONG H， PAN J S， XIANG L， et al. Multi-scale boosted dehazing network with dense feature fusion［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020：2154-2164. 10.1109/CVPR42600.2020.00223
6	HE K M， SUN J， TANG X O. Single image haze removal using dark channel prior［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2011， 33（12）： 2341-2353. 10.1109/tpami.2010.168
7	ZHU Q S， MAI J M， SHAO L. A fast single image haze removal algorithm using color attenuation prior［J］. IEEE Transactions on Image Processing， 2015， 24（11）：3522-3533. 10.1109/tip.2015.2446191
8	DUDHANE A， MURALA S. Cardinal color fusion network for single image haze removal［J］. Machine Vision and Applications， 2019， 30（2）：231-242. 10.1007/s00138-019-01014-y
9	LI B Y， PENG X L， WANG Z Y， et al. AOD-Net： all-in-one dehazing network［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 4780-4788. 10.1109/iccv.2017.511
10	MEI K F， JIANG A W， LI J C， et al. Progressive feature fusion network for realistic image dehazing［C］// Proceedings of the 2018 Asian Conference on Computer Vision， LNCS11361. Cham： Springer， 2019：203-215.
11	EBENEZER J P， JOSHUA P， DAS B， et al. Single image haze removal using conditional Wasserstein generative adversarial networks［C］// Proceedings of the 7th European Signal Processing Conference. Piscataway： IEEE， 2019：1903-1908. 10.23919/eusipco.2019.8902992
12	ARJOVSKY M， CHINTALA S， BOTTOU L. Wasserstein generative adversarial networks［C］// Proceedings of the 34th International Conference on Machine Learning. New York： JMLR.org， 2017：214-223.
13	GOODFELLOW I J， POUGET-ABADIE J， MIRZA M， et al. Generative adversarial nets［C］// Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge： MIT Press， 2014：2672-2680.
14	MIRZA M， OSINDERO S. Conditional generative adversarial nets［EB/OL］. （2014-11-06）［2020-10-23］..
15	LEDIG C， THEIS L， HUSZÁR F， et al. Photo-realistic single image super-resolution using a generative adversarial network［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017：105-114. 10.1109/cvpr.2017.19
16	ZHANG H， SINDAGI V， PATELV M. Image deraining using a conditional generative adversarial network［J］. IEEE Transactions on Circuits and Systems for Video Technology， 2020， 30（11）：3943-3956. 10.1109/tcsvt.2019.2920407
17	LIM B， SON S， KIM H， et al. Enhanced deep residual networks for single image super-resolution［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2017：1132-1140. 10.1109/cvprw.2017.151
18	ROMANO Y， ELAD M. Boosting of image denoising algorithms［J］. SIAM Journal on Imaging Sciences， 2015， 8（2）：1187-1219. 10.1137/140990978
19	ISOLA P， ZHU J Y， ZHOU T H， et al. Image-to-image translation with conditional adversarial networks［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017：5967-5976. 10.1109/cvpr.2017.632
20	WANG Z， SIMONCELLI E P， BOVIK A C. Multiscale structural similarity for image quality assessment［C］// Proceedings of the 37th Asilomar Conference on Signals， Systems and Computers. Piscataway： IEEE， 2003：1398-1402. 10.1109/acssc.2003.1292216
21	SILBERMAN N， HOIEM D， KOHLI P， et al. Indoor segmentation and support inference from RGBD images［C］// Proceedings of the 2012 European Conference on Computer Vision， LNCS7576. Berlin： Springer， 2012：746-760.
22	LI B Y， REN W Q， FENG D P， et al. Benchmarking single-image dehazing and beyond［J］. IEEE Transactions on Image Processing， 2019， 28（1）： 492-505. 10.1109/tip.2018.2867951

[1]	陈成瑞, 孙宁, 何世彪, 廖勇. 面向C-V2X通信的基于深度学习的联合信道估计与均衡算法[J]. 计算机应用, 2021, 41(9): 2687-2693.
[2]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[3]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[4]	郑志强, 胡鑫, 翁智, 王雨禾, 程曦. 基于改进DenseNet的牛眼图像特征提取方法[J]. 计算机应用, 2021, 41(9): 2780-2784.
[5]	谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495.
[6]	代雨柔, 杨庆, 张凤荔, 周帆. 基于自监督学习的社交网络用户轨迹预测模型[J]. 计算机应用, 2021, 41(9): 2545-2551.
[7]	管其杰, 张挺, 李德亚, 周绍景, 杜奕. 基于多分辨率生成对抗网络的空间数据不确定性重建方法[J]. 计算机应用, 2021, 41(8): 2306-2311.
[8]	孙潇, 徐金东. 基于级联生成对抗网络的遥感图像去雾方法[J]. 计算机应用, 2021, 41(8): 2440-2444.
[9]	何正海, 线岩团, 王蒙, 余正涛. 融合句法指导与字符注意力机制的案情阅读理解方法[J]. 计算机应用, 2021, 41(8): 2427-2431.
[10]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 《计算机应用》唯一官方网站, 2021, 41(8): 2273-2287.
[11]	秦斌斌, 彭良康, 卢向明, 钱江波. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 41(8): 2330-2337.
[12]	汤桂花, 孙磊, 毛秀青, 戴乐育, 胡永进. 基于深度对齐网络的生成对抗网络伪造人脸检测[J]. 计算机应用, 2021, 41(7): 1922-1927.
[13]	杜炎, 吕良福, 焦一辰. 基于模糊推理的模糊原型网络[J]. 计算机应用, 2021, 41(7): 1885-1890.
[14]	李亚芳, 梁烨, 冯韦玮, 祖宝开, 康玉健. 基于社区优化的深度网络嵌入方法[J]. 计算机应用, 2021, 41(7): 1956-1963.
[15]	王月, 江逸茗, 兰巨龙. 基于改进三元组网络和K近邻算法的入侵检测[J]. 计算机应用, 2021, 41(7): 1996-2002.