计算机应用 ›› 2019, Vol. 39 ›› Issue (10): 2899-2904.DOI: 10.11772/j.issn.1001-9081.2019040614

• 人工智能 • 上一篇    下一篇

基于深度学习的彩色以及近红外图像去马赛克

谢长江, 杨晓敏, 严斌宇, 芦璐   

  1. 四川大学 电子信息学院, 成都 610065
  • 收稿日期:2019-04-15 修回日期:2019-06-20 出版日期:2019-10-10 发布日期:2019-08-21
  • 通讯作者: 严斌宇
  • 作者简介:谢长江(1994-),男,四川巴中人,硕士研究生,主要研究方向:图像处理、机器学习;杨晓敏(1980-),女,四川成都人,教授,博士,主要研究方向:图像处理、机器学习;严斌宇(1975-),男,四川成都人,副教授,博士,主要研究方向:移动传感器网络、网络安全;芦璐(1990-),男,四川成都人,博士,主要研究方向:自适应信号处理、核方法、分布式估计。

RGB-NIR image demosaicing based on deep learning

XIE Changjiang, YANG Xiaomin, YAN Binyu, LU Lu   

  1. College of Electronics and Information Engineering, Sichuan University, Chengdu Sichuan 610065, China
  • Received:2019-04-15 Revised:2019-06-20 Online:2019-10-10 Published:2019-08-21

摘要: 单传感器捕获的彩色-近红外(RGB-NIR)图像存在光谱干扰,从而导致重建出的标准彩色图像(RGB)图像与近红外(NIR)图像存在色彩失真以及细节信息模糊。针对这个问题提出一种基于深度学习的去马赛克方法,通过引入跳远连接与稠密连接解决了梯度消失和梯度弥散问题,使得网络更容易训练,并且提升了网络的拟合能力。首先,用浅层特征提取层提取了马赛克图像的像素相关性以及通道相关性等低级特征;然后,将得到的浅层特征图输入到连续多个的残差稠密块以提取专门针对去马赛克的高级语义特征;其次,为充分利用低级特征与高级特征,将多个残差稠密块提取到的特征进行组合;最后,通过全局跳远连接恢复最终的RGB-NIR图像。在深度学习框架Tensorflow上使用公共的图像与视觉表示组(IVRG)数据集、有植被的户外多光谱图像(OMSIV)数据集和森林(Forest)三个公开数据集进行实验。实验结果表明,所提方法优于基于多级自适应残差插值、基于卷积卷积和神经神经网络以及基于深度残差U型网络的主流的RGB-NIR图像去马赛克方法。

关键词: 彩色-近红外图像, 去马赛克, 残差稠密网络, 跳远连接, 稠密连接

Abstract: Spectral interference in Red Green Blue-Near InfRared (RGB-NIR) images captured by single sensor results in colour distortion and detail information ambiguity of the reconstructed standard Red Green Blue (RBG) and Near InfRared (NIR) images. To resolve this problem, a demosaicing method based on deep learning was proposed. In this method, the grandient dppearance and dispersion problems were solved by introducing long jump connection and dense connection, the network was easier to be trained, and the fitting ability of the network was improved. Firstly, the low-level features such as pixel correlation and channel correlation of the mosaic image were extracted by the shallow feature extraction layer. Secondly, the obtained shallow feature graph was input into successive and multiple residual dense blocks to extract the high-level semantic features aiming at the demosaicing. Thirdly, to make full use of the low-level features and high-level features, the features extracted by multiple residual dense blocks were combined. Finally, the RGB-NIR image was reconstructed by the global long jump connection. Experiments were performed on the deep learning framework Tensorflow using three public data sets, the Common Image and Visual Representation Group (IVRG) dataset, the Outdoor Multi-Spectral Images with Vegetation (OMSIV) dataset, and the Forest dataset. The experimental results show that the proposed method is superior to the RGB-NIR image demosaicing methods based on multi-level adaptive residual interpolation, convolutional neural network and deep residual U-shaped network.

Key words: Red Green Blue-Near InfRared (RGB-NIR) image, demosaicing, residual dense network, long jump connection, dense connection

中图分类号: