Underwater image super-resolution reconstruction method based on deep learning

doi:10.11772/j.issn.1001-9081.2019020353

Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (9): 2738-2743.DOI: 10.11772/j.issn.1001-9081.2019020353

• Virtual reality and multimedia computing • Previous Articles Next Articles

Underwater image super-resolution reconstruction method based on deep learning

CHEN Longbiao, CHEN Yuzhang, WANG Xiaochen, ZOU Peng, HU Xuemin

School of Computer Science and Information Engineering, Hubei University, Wuhan Hubei 430062, China

Received:2019-03-05 Revised:2019-05-05 Online:2019-09-10 Published:2019-06-03
Supported by:
This work is partially supported by Youth Science Foundation of National Natural Science Foundation of China (61806076), the Student's Platform for Innovation and Entrepreneurship Training Program of Hubei Province (201710512051, 201810512051).

基于深度学习的水下图像超分辨率重建方法

陈龙彪, 谌雨章, 王晓晨, 邹鹏, 胡学敏

湖北大学计算机与信息工程学院, 武汉 430062

通讯作者: 谌雨章
作者简介:陈龙彪(1997-),男,湖北咸宁人,主要研究方向:深度学习、图像处理;谌雨章(1984-),男,湖北武汉人,副教授,博士,主要研究方向:光电探测、图像处理;王晓晨(1998-),男,河南郑州人,主要研究方向:深度学习、软件工程;邹鹏(1997-),男,湖北鄂州人,主要研究方向:图像处理、深度学习;胡学敏(1985-),男,湖南岳阳人,副教授,博士,主要研究方向:计算机视觉、智能系统。
基金资助:
国家自然科学基金青年科学基金资助项目（61806076）；湖北省大学生创新创业训练计划基金资助项目（201710512051，201810512051）。

Abstract

Abstract:

Due to the characteristics of water itself and the absorption and scattering of light by suspended particles in the water, a series of problems, such as low Signal-to-Noise Ratio (SNR) and low resolution, exist in underwater images. Most of the traditional processing methods include image enhancement, restoration and reconstruction rely on degradation model and have ill-posed algorithm problem. In order to further improve the effects and efficiency of underwater image restoration algorithm, an improved image super-resolution reconstruction method based on deep convolutional neural network was proposed. An Improved Dense Block structure (IDB) was introduced into the network of the method, which can effectively solve the gradient disappearance problem of deep convolutional neural network and improve the training speed at the same time. The network was used to train the underwater images before and after the degradation by registration and obtained the mapping relation between the low-resolution image and the high-resolution image. The experimental results show that on a self-built underwater image training set, the underwater image reconstructed by the deep convolutional neural network with IDB has the Peak Signal-to-Noise Ratio (PSNR) and Structural SIMilarity (SSIM) improved by 0.38 dB and 0.013 respectively, compared with SRCNN (an image Super-Resolution method using Conventional Neural Network) and proposed method can effectively improve the reconstruction quality of underwater images.

Key words: Convolutional Neural Network (CNN), super-resolution reconstruction, Signal-to-Noise Ratio (SNR), underwater image processing, mapping

摘要：

由于水体本身的特性以及水中悬浮颗粒对光的吸收和散射作用，水下图像普遍存在信噪比（SNR）低、分辨率低等一系列问题，但大部分方法传统处理方法包含图像增强、复原及重建，都依赖退化模型，并存在算法病态性问题。为进一步提高水下图像恢复算法的效果和效率，提出了一种改进的基于深度卷积神经网络的图像超分辨率重建方法。该方法网络中引入了改良的密集块结构（IDB），能在有效解决深度卷积神经网络梯度弥散问题的同时提高训练速度。该网络对经过配准的退化前后的水下图像进行训练，得到水下低分辨率图像和高分辨率图像之间的一个映射关系。实验结果表明，在基于自建的水下图像作为训练集上，较卷积神经网络的单帧图像超分辨率重建算法（SRCNN），使用引入了改良的密集块结构（IDB）的深度卷积神经网络对水下图像进行重建，重建图像的峰值信噪比（PSNR）提升达到0.38 dB，结构相似度（SSIM）提升达到0.013，能有效地提高水下图像的重建质量。

关键词: 卷积神经网络, 超分辨率重建, 信噪比, 水下图像处理, 映射

CLC Number:

CHEN Longbiao, CHEN Yuzhang, WANG Xiaochen, ZOU Peng, HU Xuemin. Underwater image super-resolution reconstruction method based on deep learning[J]. Journal of Computer Applications, 2019, 39(9): 2738-2743.

陈龙彪, 谌雨章, 王晓晨, 邹鹏, 胡学敏. 基于深度学习的水下图像超分辨率重建方法[J]. 计算机应用, 2019, 39(9): 2738-2743.

References

[1] CHEN Y, YANG W, TAN H, et al. Image enhancement for LD based imaging in turbid water[J]. Optik, 2016, 127(2):517-521.
[2] 谌雨章,叶婷,程超杰,et al.水下湍流成像退化及优化恢复研究[J].光电工程,2018,45(12):55-65.(CHEN Y Z, YE T, CHENG C J, et al. Degradation and optimal recovery of underwater turbulent imaging[J]. Opto-Electronic Engineering, 2018, 45(12):55-65.)
[3] SCHETTINI R, CORCHS S. Underwater image processing:state of the art of restoration and image enhancement methods[J]. EURASIP Journal on Advances in Signal Processing, 2010(3):1-15.
[4] 杨爱萍,郑佳,王建,等.基于颜色失真去除与暗通道先验的水下图像复原[J].电子与信息学报,2015,37(11):2541-2547.(YANG A P, ZHENG J, WANG J, et al. Underwater image restoration based on color cast removal and dark channel prior[J]. Journal of Electronics and Information Technology, 2015, 37(11):2541-2547.)
[5] 王鑫,朱行成,宁晨,等.融合暗原色先验和稀疏表示的水下图像复原[J].电子与信息学报,2018, 40(2):264-271.(WANG X, ZHU X C, NING C, et al. Combination of dark-channel prior with sparse representation for underwater image restoration[J]. Journal of Electronics and Information Technology, 2018, 40(2):264-271.)
[6] 郭相凤,贾建芳,杨瑞峰,等.基于水下图像光学成像模型的清晰化算法[J].计算机应用,2012,32(10):2836-2839.(GUO X F, JIA J F, YANG R F, et al. Visibility enhancing algorithm based on optical imaging model for underwater images[J]. Journal of Computer Applications, 2012, 32(10):2836-2839.)
[7] QUEVEDO E, DELORY E, CALLIC G M, et al. Underwater video enhancement using multi-camera super-resolution[J]. Optics Communications, 2017, 404:94-102.
[8] LI J, LI Y. Underwater image restoration algorithm for free-ascending deep-sea tripods[J]. Optics and Laser Technology, 2019, 110:129-134.
[9] 张颢,范新南,李敏,等.基于光学成像模型的水下图像超分辨率重构[J].计算机与现代化,2017(4):7-13.(ZHANG H, FAN X N, LI M, et al. Underwater image super-resolution reconstruction based on optical imaging model[J]. Computer and Modernization, 2017(4):7-13.)
[10] LU H, LI Y, NAKASHIMA S, et al. Underwater image super-resolution by descattering and fusion[J]. IEEE Access, 2017,5:670-679.
[11] NAKAGAWA Y, KIHARA K, TADOH R, et al. Super resolving of the depth map for 3D reconstruction of underwater terrain using kinect[C]//Proceedings of the 2016 IEEE 22nd International Conference on Parallel and Distributed Systems. Piscataway, NJ:IEEE, 2016, 1:1237-1240.
[12] LU H, LI Y, UEMURA T, et al. Low illumination underwater light field images reconstruction using deep convolutional neural networks[J]. Future Generation Computer Systems, 2018, 82:142-148.
[13] PEREZ J, ATTANASIO A C, NECHYPORENKO N, et al. A deep learning approach for underwater image enhancement[C]//Proceedings of the 2017 International Work-Conference on the Interplay Between Natural and Artificial Computation, LNCS 10338. Berlin:Springer, 2017:183-192.
[14] DONG C, LOY C C, HE K, et al. Learning a deep convolutional network for image super-resolution[C]//Proceedings of the 2014 European Conference on Computer Vision, LNCS 8692. Berlin:Springer, 2014:184-199.
[15] 张清博,张晓晖,韩宏伟.一种基于深度卷积神经网络的水下光电图像质量优化方法[J].光学学报,2018,38(11):88-96.(ZHANG Q B, ZHANG X H, HAN H W. Optimization of underwater photoelectric image quality based on deep convolutional neural networks[J]. Acta Optica Sinica, 2018, 38(11):88-96.)
[16] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the 2017 Computer Vision and Pattern Recognition Workshops. Washington, DC:IEEE Computer Society, 2017:136-144.
[17] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2016:1646-1654.
[18] HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9):1904-1916.
[19] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2016, 1:770-778.
[20] TONG T, LI G, LIU X, et al. Image super-resolution using dense skip connections[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Washington, DC:IEEE Computer Society, 2017:4809-4817.
[21] WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4):600-612.
[22] TIMOFTE R, de SMET V, van GOOL L. A+:adjusted anchored neighborhood regression for fast super-resolution[C]//Proceedings of the 2014 Asian Conference on Computer Vision, LNCS 9006. Berlin:Springer, 2014:111-126.
[23] HUANG J-B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2015, 1:5197-5206.

Underwater image super-resolution reconstruction method based on deep learning

基于深度学习的水下图像超分辨率重建方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	WANG Hebing, ZHANG Chunmei. Facial landmark detection based on ResNeXt with asymmetric convolution and squeeze excitation [J]. Journal of Computer Applications, 2021, 41(9): 2741-2747.
[2]	SONG Zhongshan, LIANG Jiarui, ZHENG Lu, LIU Zhenyu, TIE Jun. Remote sensing scene classification based on bidirectional gated scale feature fusion [J]. Journal of Computer Applications, 2021, 41(9): 2726-2735.
[3]	LI Kangkang, ZHANG Jing. Multi-layer encoding and decoding model for image captioning based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(9): 2504-2509.
[4]	ZHANG Yongbin, CHANG Wenxin, SUN Lianshan, ZHANG Hang. Detection method of domains generated by dictionary-based domain generation algorithm [J]. Journal of Computer Applications, 2021, 41(9): 2609-2614.
[5]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[6]	XU Jianglang, LI Linyan, WAN Xinjun, HU Fuyuan. Indoor scene recognition method combined with object detection [J]. Journal of Computer Applications, 2021, 41(9): 2720-2725.
[7]	AN Xin, YANG Haijiao, LI Jianhua, REN Fuji. Dynamic mapping method for heterogeneous multi-core system under thermal safety constraint [J]. Journal of Computer Applications, 2021, 41(9): 2631-2638.
[8]	ZENG Xiangyin, ZHENG Bochuan, LIU Dan. Detection of left and right railway tracks based on deep convolutional neural network and clustering [J]. Journal of Computer Applications, 2021, 41(8): 2324-2329.
[9]	CAO Yuhong, XU Hai, LIU Sun'ao, WANG Zixiao, LI Hongliang. Review of deep learning-based medical image segmentation [J]. Journal of Computer Applications, 2021, 41(8): 2273-2287.
[10]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[11]	HUANG Chengcheng, DONG Xiaoxiao, LI Zhao. Deep pipeline 5×5 convolution method based on two-dimensional Winograd algorithm [J]. Journal of Computer Applications, 2021, 41(8): 2258-2264.
[12]	WU Guangli, LI Leiting, GUO Zhenzhou, WANG Chengxiang. Video summarization generation model based on improved bi-directional long short-term memory network [J]. Journal of Computer Applications, 2021, 41(7): 1908-1914.
[13]	LU Guoqing, SUN Hao. Two-dimensional mapping of swarm robot based on random walk [J]. Journal of Computer Applications, 2021, 41(7): 2121-2127.
[14]	YANG Su, OUYANG Zhi, DU Nisuo. Unsupervised parallel hash image retrieval based on correlation distance [J]. Journal of Computer Applications, 2021, 41(7): 1902-1907.
[15]	GAO Qinquan, HUANG Bingcheng, LIU Wenzhe, TONG Tong. Bamboo strip surface defect detection method based on improved CenterNet [J]. Journal of Computer Applications, 2021, 41(7): 1933-1938.