Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (9): 2858-2864.DOI: 10.11772/j.issn.1001-9081.2021081379
• Multimedia computing and computer simulation • Previous Articles
Received:
2021-08-02
Revised:
2021-11-08
Accepted:
2021-11-25
Online:
2022-01-07
Published:
2022-09-10
Contact:
Xianfu BAO
About author:
QIANG Zanxia, born in 1972, Ph. D., associate professor. Her research interests include pattern recognition, artificial intelligence, computer vision.
Supported by:
通讯作者:
鲍先富
作者简介:
强赞霞(1972—),女,河南项城人,副教授,博士,CCF会员,主要研究方向:模式识别、人工智能、计算机视觉;
基金资助:
CLC Number:
Zanxia QIANG, Xianfu BAO. Residual attention deraining network based on convolutional long short-term memory[J]. Journal of Computer Applications, 2022, 42(9): 2858-2864.
强赞霞, 鲍先富. 基于卷积长短期记忆的残差注意力去雨网络[J]. 《计算机应用》唯一官方网站, 2022, 42(9): 2858-2864.
Add to citation manager EndNote|Ris|BibTeX
URL: http://www.joca.cn/EN/10.11772/j.issn.1001-9081.2021081379
特征尺寸 | CLSTM数 | RCAB数 | RG数 |
---|---|---|---|
1 | 1(核大小:3×3) | 4 | 10 |
1/2 | 1(核大小:3×3) | 4 | 10 |
1/4 | 1(核大小:3×3) | 4 | 10 |
Tab. 1 GN structure design
特征尺寸 | CLSTM数 | RCAB数 | RG数 |
---|---|---|---|
1 | 1(核大小:3×3) | 4 | 10 |
1/2 | 1(核大小:3×3) | 4 | 10 |
1/4 | 1(核大小:3×3) | 4 | 10 |
数据集名 | 测试集名 | 训练集样本数 | 测试集样本数 |
---|---|---|---|
合计 | — | 15 400 | 2 800 |
Rain14000[ | Test2000 | 12 000 | 2 000 |
Rain1800[ | Rain1800 | 1 800 | 0 |
Rain800[ | Test100 | 700 | 100 |
Rain100H[ | Rain100H | 0 | 100 |
Rain100L[ | Rain100L | 0 | 100 |
Rain1200[ | Test300 | 900 | 300 |
Real200[ | Real200 | 0 | 200 |
Tab. 2 Composition of deraining datasets
数据集名 | 测试集名 | 训练集样本数 | 测试集样本数 |
---|---|---|---|
合计 | — | 15 400 | 2 800 |
Rain14000[ | Test2000 | 12 000 | 2 000 |
Rain1800[ | Rain1800 | 1 800 | 0 |
Rain800[ | Test100 | 700 | 100 |
Rain100H[ | Rain100H | 0 | 100 |
Rain100L[ | Rain100L | 0 | 100 |
Rain1200[ | Test300 | 900 | 300 |
Real200[ | Real200 | 0 | 200 |
模型 | 结构 | PSNR/dB | SSIM | 推理时间/s |
---|---|---|---|---|
Model1 | t=2,B=4,C=10 | 25.19 | 0.782 | 0.245 |
Model2 | t=0,B=4,C=10 | 23.61 | 0.727 | 0.156 |
Model3 | t=1,B=4,C=1 | 25.15 | 0.791 | 0.164 |
Model4 | t=1,B=4,C=5 | 27.31 | 0.831 | 0.184 |
Model5 | t=1,B=4,C=10 | 29.06 | 0.896 | 0.194 |
Model6 | t=1,B=4,C=15 | 27.98 | 0.856 | 0.213 |
Model7 | t=1,B=3,C=10 | 25.61 | 0.812 | 0.182 |
Model8 | t=1,B=5,C=10 | 24.40 | 0.829 | 0.206 |
Tab. 3 Experimental results of ablation
模型 | 结构 | PSNR/dB | SSIM | 推理时间/s |
---|---|---|---|---|
Model1 | t=2,B=4,C=10 | 25.19 | 0.782 | 0.245 |
Model2 | t=0,B=4,C=10 | 23.61 | 0.727 | 0.156 |
Model3 | t=1,B=4,C=1 | 25.15 | 0.791 | 0.164 |
Model4 | t=1,B=4,C=5 | 27.31 | 0.831 | 0.184 |
Model5 | t=1,B=4,C=10 | 29.06 | 0.896 | 0.194 |
Model6 | t=1,B=4,C=15 | 27.98 | 0.856 | 0.213 |
Model7 | t=1,B=3,C=10 | 25.61 | 0.812 | 0.182 |
Model8 | t=1,B=5,C=10 | 24.40 | 0.829 | 0.206 |
模型 | PSNR/dB | SSIM | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Test2000 | Test100 | Rain100H | Rain100L | Test300 | Real200 | Test2000 | Test100 | Rain100H | Rain100L | Test300 | Real200 | |
DerainNet[ | 24.31 | 22.70 | 14.90 | 27.00 | 23.40 | 25.90 | 0.86 | 0.81 | 0.59 | 0.88 | 0.84 | 0.81 |
RESCAN[ | 31.29 | 25.00 | 26.40 | 29.80 | 30.50 | 30.40 | 0.90 | 0.84 | 0.79 | 0.88 | 0.88 | 0.96 |
PreNet[ | 31.75 | 24.80 | 26.80 | 32.40 | 31.40 | 30.50 | 0.92 | 0.85 | 0.86 | 0.95 | 0.91 | 0.91 |
MSPFN[ | 32.82 | 27.50 | 28.70 | 32.40 | 32.40 | 30.20 | 0.93 | 0.88 | 0.86 | 0.93 | 0.92 | 0.90 |
GADN | 33.61 | 29.90 | 29.10 | 33.10 | 34.80 | 32.40 | 0.95 | 0.90 | 0.89 | 0.94 | 0.93 | 0.93 |
Tab. 4 Comparison of rain removal effect
模型 | PSNR/dB | SSIM | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Test2000 | Test100 | Rain100H | Rain100L | Test300 | Real200 | Test2000 | Test100 | Rain100H | Rain100L | Test300 | Real200 | |
DerainNet[ | 24.31 | 22.70 | 14.90 | 27.00 | 23.40 | 25.90 | 0.86 | 0.81 | 0.59 | 0.88 | 0.84 | 0.81 |
RESCAN[ | 31.29 | 25.00 | 26.40 | 29.80 | 30.50 | 30.40 | 0.90 | 0.84 | 0.79 | 0.88 | 0.88 | 0.96 |
PreNet[ | 31.75 | 24.80 | 26.80 | 32.40 | 31.40 | 30.50 | 0.92 | 0.85 | 0.86 | 0.95 | 0.91 | 0.91 |
MSPFN[ | 32.82 | 27.50 | 28.70 | 32.40 | 32.40 | 30.20 | 0.93 | 0.88 | 0.86 | 0.93 | 0.92 | 0.90 |
GADN | 33.61 | 29.90 | 29.10 | 33.10 | 34.80 | 32.40 | 0.95 | 0.90 | 0.89 | 0.94 | 0.93 | 0.93 |
1 | CRESWELL A, WHITE T, DUMOULIN V, et al. Generative adversarial networks: an overview[J]. IEEE Signal Processing Magazine, 2018, 35(1): 53-65. 10.1109/msp.2017.2765202 |
2 | REYNOLDS D A, QUATIERI T F, DUNN R B. Speaker verification using adapted Gaussian mixture models[J]. Digital Signal Processing, 2000, 10(1/2/3):19-41. 10.1006/dspr.1999.0361 |
3 | KANG L W, LIN C W, FU Y H. Automatic single-image-based rain streaks removal via image decomposition[J]. IEEE Transactions on Image Processing, 2012, 21(4): 1742-1755. 10.1109/tip.2011.2179057 |
4 | LUO Y, XU Y, JI H. Removing rain from a single image via discriminative sparse coding[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 3397-3405. 10.1109/iccv.2015.388 |
5 | LI Y, TAN R T, GUO X J, et al. Rain streak removal using layer priors[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 2736-2744. 10.1109/cvpr.2016.299 |
6 | QIAN R, TAN R T, YANG W H, et al. Attentive generative adversarial network for raindrop removal from a single image[C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 2482-2491. 10.1109/cvpr.2018.00263 |
7 | CHEN B H, HUANG S C, KUO S Y. Error-optimized sparse representation for single image rain removal[J]. IEEE Transactions on Industrial Electronics, 2017, 64(8): 6573-6581. 10.1109/tie.2017.2682036 |
8 | FU X Y, HUANG J B, DING X H, et al. Clearing the skies: a deep network architecture for single-image rain removal[J]. IEEE Transactions on Image Processing, 2017, 26(6): 2944-2956. 10.1109/tip.2017.2691802 |
9 | BARZEGAR S, SHARIFI A, MANTHOURI M. Super-resolution using lightweight detailnet network[J]. Multimedia Tools and Applications, 2020, 79(1/2): 1119-1136. 10.1007/s11042-019-08218-4 |
10 | ZHANG H, SINDAGI V, PATEL V M. Image de-raining using a conditional generative adversarial network[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(11): 3943-3956. 10.1109/tcsvt.2019.2920407 |
11 | JIANG K, WANG Z Y, YI P, et al. Multi-scale progressive fusion network for single image deraining[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 8343-8352. 10.1109/cvpr42600.2020.00837 |
12 | 邱永茹,姚光乐,冯杰,等. 基于半监督学习的单幅图像去雨算法[J]. 计算机应用, 2022, 42(5): 1577-1582. 10.11772/j.issn.1001-9081.2021030492 |
QIU Y R, YAO G L, FENG J, et al. Single image de-raining algorithm based on semi-supervised learning[J]. Journal of Computer Applications, 2022, 42(5): 1577-1582. 10.11772/j.issn.1001-9081.2021030492 | |
13 | LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian pyramid networks for fast and accurate super-resolution[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern recognition. Piscataway: IEEE, 2017: 5835-5843. 10.1109/cvpr.2017.618 |
14 | ZHANG H, PATEL V M. Density-aware single image de-raining using a multi-stream dense network[C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 695-704. 10.1109/cvpr.2018.00079 |
15 | SHI X J, CHEN Z R, WANG H, et al. Convolutional LSTM network: a machine learning approach for precipitation nowcasting[C]// Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2015: 802-810. |
16 | GERS F A, SCHMIDHUBER J, CUMMINS F. Learning to forget: continual prediction with LSTM[J]. Neural Computation, 2000, 12(10): 2451-2471. 10.1162/089976600300015015 |
17 | ZHANG Y L, LI K P, LI K, et al. Image super-resolution using very deep residual channel attention networks[C]// Proceedings of the 2018 European Conference on Computer Vision, LNCS 11211. Cham: Springer, 2018: 294-310. |
18 | FU X Y, HUANG J B, ZENG D L, et al. Removing rain from single images via a deep detail network[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 1715-1723. 10.1109/cvpr.2017.186 |
19 | YANG W H, TAN R T, FENG J S, et al. Deep joint rain detection and removal from a single image[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 1685-1694. 10.1109/cvpr.2017.183 |
20 | LI X, WU J L, LIN Z C, et al. Recurrent squeeze-and-excitation context aggregation net for single image deraining[C]// Proceedings of the 2018 European Conference on Computer Vision, LNCS 11211. Cham: Springer, 2018: 262-277. |
21 | REN D E, ZUO W M, HU Q H, et al. Progressive image deraining networks: a better and simpler baseline[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 3932-3941. 10.1109/cvpr.2019.00406 |
22 | BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[EB/OL]. (2020-04-23) [2021-07-25].. |
[1] | Zefang HAN, Xiong ZHANG, Hong SHANGGUAN, Xinglong HAN, Jing HAN, Gang FENG, Xueying CUI. Artifacts sensing generative adversarial network for low-dose CT denoising [J]. Journal of Computer Applications, 2022, 42(7): 2301-2310. |
[2] | Wentao MAO, Guifang WU, Chao WU, Zhi DOU. Animation video generation model based on Chinese impressionistic style transfer [J]. Journal of Computer Applications, 2022, 42(7): 2162-2169. |
[3] | Jia LI, Yuanlin ZHENG, Kaiyang LIAO, Haojie LOU, Shiyu LI, Zehao CHEN. No-reference image quality assessment algorithm based on saliency deep features [J]. Journal of Computer Applications, 2022, 42(6): 1957-1964. |
[4] | Yuming ZHAO, Shenkai GU. Adversarial attack defense model with residual dense block self-attention mechanism and generative adversarial network [J]. Journal of Computer Applications, 2022, 42(3): 921-929. |
[5] | Yimin CAO, Lei CAI, Jingyang GAO. Gene data generation method based on generative adversarial network [J]. Journal of Computer Applications, 2022, 42(3): 783-790. |
[6] | Dingkang YANG, Shuai HUANG, Shunli WANG, Peng ZHAI, Yidan LI, Lihua ZHANG. EE-GAN:facial expression recognition method based on generative adversarial network and network integration [J]. Journal of Computer Applications, 2022, 42(3): 750-756. |
[7] | GUAN Qijie, ZHANG Ting, LI Deya, ZHOU Shaojing, DU Yi. Indefinite reconstruction method of spatial data based on multi-resolution generative adversarial network [J]. Journal of Computer Applications, 2021, 41(8): 2306-2311. |
[8] | SUN Xiao, XU Jindong. Remote sensing image dehazing method based on cascaded generative adversarial network [J]. Journal of Computer Applications, 2021, 41(8): 2440-2444. |
[9] | TANG Guihua, SUN Lei, MAO Xiuqing, DAI Leyu, HU Yongjin. Generative adversarial network synthesized face detection based on deep alignment network [J]. Journal of Computer Applications, 2021, 41(7): 1922-1927. |
[10] | WANG Xianwu, ZHANG Ting, JI Xin, DU Yi. 3D shale digital core reconstruction method based on deep convolutional generative adversarial network with gradient penalty [J]. Journal of Computer Applications, 2021, 41(6): 1805-1811. |
[11] | LI Yanzhi, FAN Yong, GAO Lin. Anomaly detection of oil drilling water flow based on shape flow [J]. Journal of Computer Applications, 2021, 41(6): 1842-1848. |
[12] | SUN Heli, SUN Yuzhu, ZHANG Xiaoyun. Event description generation based on generative adversarial network [J]. Journal of Computer Applications, 2021, 41(5): 1256-1261. |
[13] | GUO Maozu, YANG Qiannan, ZHAO Lingling. Image generation based on conditional-Wassertein generative adversarial network [J]. Journal of Computer Applications, 2021, 41(5): 1432-1437. |
[14] | OU Lili, SHAO Fengjing, SUN Rencheng, SUI Yi. Cerebral infarction image recognition based on semi-supervised method [J]. Journal of Computer Applications, 2021, 41(4): 1221-1226. |
[15] | DUAN Youxiang, ZHANG Hanxiao, SUN Qifeng, SUN Youkai. Image super-resolution reconstruction algorithm based on Laplacian pyramid generative adversarial network [J]. Journal of Computer Applications, 2021, 41(4): 1020-1026. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||