Color image demosaicking network based on inter-channel correlation and enhanced information distillation

doi:10.11772/j.issn.1001-9081.2021010127

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (1): 245-251.DOI: 10.11772/j.issn.1001-9081.2021010127

• Multimedia computing and computer simulation • Previous Articles Next Articles

Color image demosaicking network based on inter-channel correlation and enhanced information distillation

Hengxin LI¹, Kan CHANG¹^,²(), Yufei TAN¹^,³, Mingyang LING¹, Tuanfa QIN¹^,²

^1.School of Computer，Electronics and Information，Guangxi University，Nanning Guangxi 530004，China
^2.Guangxi Key Laboratory of Multimedia Communications and Network Technology （Guangxi University），Nanning Guangxi 530004，China
^3.College of Electronic Engineering，Guangxi Normal University，Guilin Guangxi 541004，China

Received:2021-01-22 Revised:2021-03-05 Accepted:2021-03-17 Online:2022-01-11 Published:2022-01-10
Contact: Kan CHANG
About author:LI Hengxin， born in 1996， M. S. candidate. His research interests include color image demosaicking， super-resolution.
CHANG Kan， born in 1983， Ph. D.， associate professor. His research interests include multimedia communication， compressive sensing， sparse representation.
TAN Yufei， born in 1993， M. S. His research interests include color image restoration.
LING Mingyang， born in 1998， M. S. candidate. Her research interests include color image restoration.
QIN Tuanfa， born in 1966， Ph. D.， professor. His research interests include media communication， network coding， image and video retrieval.
Supported by:
National Natural Science Foundation of China(61761005)

应用通道间相关性及增强信息蒸馏的彩色图像去马赛克网络

李恒鑫¹, 常侃¹^,²(), 谭宇飞¹^,³, 凌铭阳¹, 覃团发¹^,²

^1.广西大学计算机与电子信息学院, 南宁 530004
^2.广西多媒体通信与网络技术重点实验室(广西大学), 南宁 530004
^3.广西师范大学电子工程学院, 广西桂林, 541004

通讯作者: 常侃
作者简介:李恒鑫（1996—），男，江西抚州人，硕士研究生，主要研究方向：图像去马赛克、超分辨率
常侃（1983—），男，湖南衡阳人，副教授，博士，CCF会员，主要研究方向：多媒体通信、压缩感知、稀疏表示
谭宇飞（1993—），男，广西桂林人，硕士，主要研究方向：图像复原
凌铭阳（1998—），女，广西钦州人，硕士研究生，主要研究方向：图像复原
覃团发（1966—），男，教授，博士，主要研究方向：媒体通信、网络编码、图像和视频检索。

Abstract

Abstract:

In commercial digital cameras， due to the limitation of Complementary Metal Oxide Semiconductor （CMOS） sensors， there is only one color channel information for each pixel in the sampled image. Therefore， the Color image DeMosaicking （CDM） algorithm is required to restore the full-color images. However， most of the existing Convolutional Neural Network （CNN）-based CDM algorithms cannot achieve satisfactory performance with relatively low computational complexity and small network parameter number. To solve this problem， a CDM network based on Inter-channel Correlation and Enhanced Information Distillation （ICEID） was proposed. Firstly， to fully utilize the inter-channel correlation of the color image， an inter-channel guided reconstruction structure was designed to obtain the initial CDM result. Secondly， an Enhanced Information Distillation Module （EIDM）， which can effectively extract and refine features from image with relatively small parameter number， was presented to enhance the reconstructed full-color image in high efficiency. Experimental results demonstrate that compared with many state-of-the-art CDM methods， the proposed algorithm achieves significant improvement in both objective quality and subjective quality， and has relatively low computational complexity and small network parameter number.

Key words: Color image DeMosaicking (CDM), inter-channel correlation, information distillation, residual learning, Convolutional Neural Network (CNN)

摘要：

在商用数码相机中，由于CMOS传感器的限制，在采样得到的图像中的每个像素位置仅有一个色彩通道的信息，因此，需要采用彩色图像去马赛克（CDM）算法来恢复全彩图像。然而，现有的基于卷积神经网络（CNN）的CDM算法不能以较低的计算复杂度和网络参数量取得令人满意的性能。针对这个问题，提出一种应用通道间相关性和增强信息蒸馏（ICEID）的彩色图像去马赛克网络。首先，为了充分利用彩色图像的通道间相关性，提出了一种通道间的引导重建结构来生成初始CDM结果；其次，提出一种增强信息蒸馏模块（EIDM）来以相对较低的参数量有效地提取和精炼图像特征，从而高效地优化重建的全彩图像。实验结果表明，与主流CDM算法相比，所提算法不仅在客观质量与主观质量上均获得了明显提升，而且具有较低的计算复杂度和网络参数量。

关键词: 彩色图像去马赛克, 通道间相关性, 信息蒸馏, 残差学习, 卷积神经网络

CLC Number:

TP751

Hengxin LI, Kan CHANG, Yufei TAN, Mingyang LING, Tuanfa QIN. Color image demosaicking network based on inter-channel correlation and enhanced information distillation[J]. Journal of Computer Applications, 2022, 42(1): 245-251.

李恒鑫, 常侃, 谭宇飞, 凌铭阳, 覃团发. 应用通道间相关性及增强信息蒸馏的彩色图像去马赛克网络[J]. 《计算机应用》唯一官方网站, 2022, 42(1): 245-251.

Figures/Tables 13

References 20

1	BAYER B E. Color imaging array： US， 3971065［P］. 1976-07-20.
2	LONGERE P， ZHANG X M， DELAHUNT P B， et al. Perceptual assessment of demosaicing algorithm performance［J］. Proceedings of the IEEE， 2002， 90（1）： 123-132. 10.1109/5.982410
3	ARÀNDIGA F. A nonlinear algorithm for monotone piecewise bicubic interpolation［J］. Applied Mathematics and Computation， 2016， 272（Pt 1）： 100-113. 10.1016/j.amc.2015.08.027
4	McKINLEY S， LEVINE M. Cubic spline interpolation［EB/OL］. ［2020-12-03］..
5	KIKU D， MONNO Y， TANAKA M， et al. Residual interpolation for color image demosaicking［C］// Proceedings of the 2013 IEEE International Conference on Image Processing. Piscataway： IEEE， 2013： 2304-2308. 10.1109/icip.2013.6738475
6	KIKU D， MONNO Y， TANAKA M， et al. Minimized-Laplacian residual interpolation for color image demosaicking［C］// Proceedings of the SPIE 9023， Digital Photography X. Bellingham， WA： SPIE， 2014： No.90230L. 10.1117/12.2038425
7	MONNO Y， KIKU D， TANAKA M， et al. Adaptive residual interpolation for color image demosaicking［C］// Proceedings of the 2015 IEEE International Conference on Image Processing. Piscataway： IEEE， 2015： 3861-3865. 10.1109/icip.2015.7351528
8	黄丽丽，肖亮，韦志辉. 彩色图像去马赛克的非局部稀疏表示方法［J］. 电子学报， 2014， 42（2）： 272-279. 10.3969/j.issn.0372-2112.2014.02.010
	HUANG L L， XIAO L， WEI Z H. A nonlocal sparse representation method for color demosaicking［J］. Acta Electronica Sinica， 2014， 42（2）： 272-279. 10.3969/j.issn.0372-2112.2014.02.010
9	ZHANG L， WU X L， BUADES A， et al. Color demosaicking by local directional interpolation and nonlocal adaptive thresholding［J］. Journal of Electronic Imaging， 2011， 20（2）： No.023016. 10.1117/1.3600632
10	CHANG K， DING P L K， LI B X. Color image demosaicking using inter-channel correlation and nonlocal self-similarity［J］. Signal Processing： Image Communication， 2015， 39（Pt A）： 264-279. 10.1016/j.image.2015.10.003
11	GHARBI M， CHAURASIA G， PARIS S， et al. Deep joint demosaicking and denoising［J］. ACM Transactions on Graphics， 2016， 35（6）： No.191. 10.1145/2980179.2982399
12	TAN R J， ZHANG K， ZUO W M， et al. Color image demosaicking via deep residual learning［EB/OL］. ［2020-12-03］..
13	TAN D S， CHEN W Y， HUA K L. DeepDemosaicking： adaptive image demosaicking via multiple deep fully convolutional networks［J］. IEEE Transactions on Image Processing， 2018， 27（5）： 2408-2419. 10.1109/tip.2018.2803341
14	CUI K， JIN Z， STEINBACH E. Color image demosaicking using a 3-stage convolutional neural network structure［C］// Proceedings of the 25th IEEE International Conference on Image Processing. Piscataway： IEEE， 2018： 2177-2181. 10.1109/icip.2018.8451020
15	谢长江，杨晓敏，严斌宇，等. 基于深度学习的彩色以及近红外图像去马赛克［J］. 计算机应用， 2019， 39（10）： 2899-2904.
	XIE C J， YANG X M， YAN B Y， et al. RGB-NIR image demosaicing based on deep learning［J］. Journal of Computer Applications， 2019， 39（10）： 2899-2904.
16	CHANG K， LI M H， DING P L K， et al. Accurate single image super-resolution using multi-path wide-activated residual network［J］. Signal Processing， 2020， 172： No.107567. 10.1016/j.sigpro.2020.107567
17	HUI Z， WANG X M， GAO X B. Fast and accurate single image super-resolution via information distillation network［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 723-731. 10.1109/cvpr.2018.00082
18	SHI W Z， CABALLERO J， HUSZÁR F， et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1874-1883. 10.1109/cvpr.2016.207
19	ZHANG Y L， LI K P， LI K， et al. Image super-resolution using very deep residual channel attention networks［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS11211. Cham： Springer， 2018： 294-310. 10.1007/978-3-030-01234-2_18
20	AGUSTSSON E， TIMOFTE R. NTIRE 2017 Challenge on single image super-resolution： dataset and study［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2017： 1122-1131. 10.1109/cvprw.2017.150

实验网络	参数量/10³	CPSNR/dB	SSIM
Base	471	39.09	0.989 3
w/Bayer	495	39.17	0.989 3
w/ID	481	39.15	0.989 4
w/EID	495	39.18	0.989 6

实验网络	参数量/10³	CPSNR/dB	SSIM
Base	471	39.09	0.989 3
w/Bayer	495	39.17	0.989 3
w/ID	481	39.15	0.989 4
w/EID	495	39.18	0.989 6

图像序号	ARI^［7］		DJDD^［11］		DRL^［12］		3-stage^［14］		ICEID
图像序号	CPSNR/dB	SSIM	CPSNR/dB	SSIM	CPSNR/dB	SSIM	CPSNR/dB	SSIM	CPSNR/dB	SSIM
平均值	37.46	0.964 3	37.76	0.986 6	38.73	0.988 2	38.78	0.988 3	39.18	0.989 5
1	29.70	0.923 6	30.03	0.968 6	31.12	0.972 6	31.00	0.971 9	31.34	0.974 3
2	39.20	0.972 1	35.44	0.980 4	35.88	0.981 3	35.97	0.981 9	36.38	0.983 9
3	40.11	0.973 3	34.97	0.989 6	35.97	0.991 8	36.14	0.991 8	36.81	0.992 3
4	39.91	0.964 7	37.86	0.996 1	40.21	0.997 1	40.16	0.997 0	40.63	0.997 3
5	40.68	0.951 0	35.49	0.985 1	36.71	0.987 8	36.42	0.987 2	36.77	0.988 4
6	38.91	0.956 2	38.69	0.990 2	40.64	0.992 8	40.61	0.992 7	41.07	0.993 9
7	39.14	0.958 4	40.81	0.992 5	40.78	0.992 6	41.30	0.992 9	41.81	0.993 3
8	35.53	0.966 9	40.58	0.990 5	41.06	0.991 2	41.28	0.991 5	41.36	0.991 7
9	34.65	0.961 7	38.65	0.988 3	39.65	0.989 5	39.74	0.989 8	40.03	0.990 8
10	36.34	0.966 1	39.65	0.990 6	40.61	0.991 6	40.63	0.991 7	41.06	0.992 7
11	35.19	0.945 4	40.67	0.989 9	41.50	0.990 9	41.53	0.990 8	41.78	0.991 4
12	34.84	0.972 8	40.07	0.988 3	40.99	0.988 8	40.95	0.989 1	41.49	0.991 1
13	38.19	0.989 5	41.34	0.986 1	41.80	0.986 5	41.94	0.987 6	42.10	0.990 0
14	35.59	0.962 9	39.58	0.987 6	39.86	0.987 3	39.96	0.987 6	40.34	0.989 5
15	39.45	0.972 3	39.70	0.984 2	40.10	0.984 9	40.15	0.985 0	40.54	0.986 5
16	39.66	0.978 8	35.36	0.986 1	36.79	0.988 9	36.44	0.988 3	37.26	0.989 7
17	39.53	0.978 6	34.12	0.978 8	35.85	0.985 8	36.07	0.986 2	36.53	0.987 6
18	37.63	0.963 0	36.69	0.985 3	37.57	0.986 6	37.68	0.986 7	38.05	0.987 5

图像序号	ARI^［7］		DJDD^［11］		DRL^［12］		3-stage^［14］		ICEID
图像序号	CPSNR/dB	SSIM	CPSNR/dB	SSIM	CPSNR/dB	SSIM	CPSNR/dB	SSIM	CPSNR/dB	SSIM
平均值	37.46	0.964 3	37.76	0.986 6	38.73	0.988 2	38.78	0.988 3	39.18	0.989 5
1	29.70	0.923 6	30.03	0.968 6	31.12	0.972 6	31.00	0.971 9	31.34	0.974 3
2	39.20	0.972 1	35.44	0.980 4	35.88	0.981 3	35.97	0.981 9	36.38	0.983 9
3	40.11	0.973 3	34.97	0.989 6	35.97	0.991 8	36.14	0.991 8	36.81	0.992 3
4	39.91	0.964 7	37.86	0.996 1	40.21	0.997 1	40.16	0.997 0	40.63	0.997 3
5	40.68	0.951 0	35.49	0.985 1	36.71	0.987 8	36.42	0.987 2	36.77	0.988 4
6	38.91	0.956 2	38.69	0.990 2	40.64	0.992 8	40.61	0.992 7	41.07	0.993 9
7	39.14	0.958 4	40.81	0.992 5	40.78	0.992 6	41.30	0.992 9	41.81	0.993 3
8	35.53	0.966 9	40.58	0.990 5	41.06	0.991 2	41.28	0.991 5	41.36	0.991 7
9	34.65	0.961 7	38.65	0.988 3	39.65	0.989 5	39.74	0.989 8	40.03	0.990 8
10	36.34	0.966 1	39.65	0.990 6	40.61	0.991 6	40.63	0.991 7	41.06	0.992 7
11	35.19	0.945 4	40.67	0.989 9	41.50	0.990 9	41.53	0.990 8	41.78	0.991 4
12	34.84	0.972 8	40.07	0.988 3	40.99	0.988 8	40.95	0.989 1	41.49	0.991 1
13	38.19	0.989 5	41.34	0.986 1	41.80	0.986 5	41.94	0.987 6	42.10	0.990 0
14	35.59	0.962 9	39.58	0.987 6	39.86	0.987 3	39.96	0.987 6	40.34	0.989 5
15	39.45	0.972 3	39.70	0.984 2	40.10	0.984 9	40.15	0.985 0	40.54	0.986 5
16	39.66	0.978 8	35.36	0.986 1	36.79	0.988 9	36.44	0.988 3	37.26	0.989 7
17	39.53	0.978 6	34.12	0.978 8	35.85	0.985 8	36.07	0.986 2	36.53	0.987 6
18	37.63	0.963 0	36.69	0.985 3	37.57	0.986 6	37.68	0.986 7	38.05	0.987 5

图像序号	ARI^［7］		DJDD^［11］		DRL^［12］		3-stage^［14］		ICEID
图像序号	CPSNR/dB	SSIM	CPSNR/dB	SSIM	CPSNR/dB	SSIM	CPSNR/dB	SSIM	CPSNR/dB	SSIM
平均值	39.30	0.981 6	41.31	0.994 9	41.95	0.995 6	42.17	0.995 5	42.67	0.996 2
1	38.80	0.986 9	40.87	0.996 3	41.58	0.996 9	41.90	0.997 0	43.01	0.997 7
2	36.68	0.970 5	41.07	0.990 8	41.62	0.992 8	41.81	0.992 8	42.20	0.993 8
3	42.46	0.987 8	43.81	0.996 2	44.99	0.997 0	45.11	0.996 7	45.43	0.997 2
4	39.06	0.980 6	42.10	0.994 0	43.18	0.995 2	43.38	0.994 9	43.96	0.995 6
5	38.27	0.989 0	39.67	0.996 8	40.78	0.997 5	40.90	0.997 4	41.38	0.997 8
6	40.10	0.988 2	41.46	0.996 3	42.15	0.997 0	42.37	0.996 9	43.11	0.997 4
7	42.04	0.988 6	43.50	0.996 7	44.61	0.997 3	44.72	0.997 1	44.95	0.997 5
8	35.43	0.981 6	38.15	0.995 2	38.77	0.995 8	39.08	0.995 9	39.94	0.996 6
9	41.87	0.969 4	44.14	0.994 7	44.39	0.995 2	44.66	0.994 9	44.99	0.995 4
10	41.94	0.981 7	43.54	0.995 1	43.99	0.995 6	44.21	0.995 4	44.47	0.995 9
11	39.68	0.984 4	41.66	0.995 2	42.20	0.995 9	42.42	0.995 9	42.92	0.996 6
12	42.95	0.986 8	44.27	0.995 8	44.84	0.996 4	45.18	0.996 2	45.54	0.996 8
13	35.22	0.981 5	36.96	0.994 6	37.18	0.995 1	37.50	0.995 3	38.43	0.996 4
14	37.87	0.984 8	39.20	0.995 0	40.10	0.996 1	40.19	0.996 0	40.84	0.996 7
15	36.52	0.969 6	41.01	0.993 5	41.24	0.994 1	41.44	0.994 2	41.97	0.995 1
16	43.40	0.989 3	44.96	0.996 7	45.60	0.997 1	45.76	0.996 9	46.15	0.997 4
17	41.63	0.987 1	42.62	0.995 5	43.01	0.996 0	43.23	0.995 8	43.60	0.996 2
18	36.83	0.976 3	38.35	0.993 3	38.64	0.993 8	38.94	0.993 7	39.56	0.994 5
19	40.74	0.983 1	42.01	0.994 7	42.33	0.995 2	42.77	0.995 1	43.49	0.995 7
20	35.55	0.976 3	42.03	0.994 9	42.66	0.996 0	42.96	0.995 6	43.03	0.996 0
21	39.33	0.973 2	41.16	0.994 8	41.50	0.995 3	41.70	0.995 1	42.30	0.995 7
22	38.32	0.973 7	39.51	0.992 1	40.39	0.993 0	40.47	0.992 9	40.80	0.993 8
23	43.27	0.981 8	42.88	0.994 5	44.38	0.995 5	44.32	0.995 2	43.93	0.995 9
24	35.35	0.985 2	36.40	0.995 1	36.68	0.995 4	36.96	0.995 3	38.07	0.996 4

Color image demosaicking network based on inter-channel correlation and enhanced information distillation

应用通道间相关性及增强信息蒸馏的彩色图像去马赛克网络

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 20

Related Articles 15

Recommended Articles

Metrics

[1]	Yun LI, Fuyou WANG, Peiguang JING, Su WANG, Ao XIAO. Uncertainty-based frame associated short video event detection method [J]. Journal of Computer Applications, 2024, 44(9): 2903-2910.
[2]	Hong CHEN, Bing QI, Haibo JIN, Cong WU, Li’ang ZHANG. Class-imbalanced traffic abnormal detection based on 1D-CNN and BiGRU [J]. Journal of Computer Applications, 2024, 44(8): 2493-2499.
[3]	Dongwei WANG, Baichen LIU, Zhi HAN, Yanmei WANG, Yandong TANG. Deep network compression method based on low-rank decomposition and vector quantization [J]. Journal of Computer Applications, 2024, 44(7): 1987-1994.
[4]	Yangyi GAO, Tao LEI, Xiaogang DU, Suiyong LI, Yingbo WANG, Chongdan MIN. Crowd counting and locating method based on pixel distance map and four-dimensional dynamic convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2233-2242.
[5]	Mengyuan HUANG, Kan CHANG, Mingyang LING, Xinjie WEI, Tuanfa QIN. Progressive enhancement algorithm for low-light images based on layer guidance [J]. Journal of Computer Applications, 2024, 44(6): 1911-1919.
[6]	Jianjing LI, Guanfeng LI, Feizhou QIN, Weijun LI. Multi-relation approximate reasoning model based on uncertain knowledge graph embedding [J]. Journal of Computer Applications, 2024, 44(6): 1751-1759.
[7]	Wenshuo GAO, Xiaoyun CHEN. Point cloud classification network based on node structure [J]. Journal of Computer Applications, 2024, 44(5): 1471-1478.
[8]	Min SUN, Qian CHENG, Xining DING. CBAM-CGRU-SVM based malware detection method for Android [J]. Journal of Computer Applications, 2024, 44(5): 1539-1545.
[9]	Tianhua CHEN, Jiaxuan ZHU, Jie YIN. Bird recognition algorithm based on attention mechanism [J]. Journal of Computer Applications, 2024, 44(4): 1114-1120.
[10]	Lijun XU, Hui LI, Zuyang LIU, Kansong CHEN, Weixuan MA. 3D-GA-Unet： MRI image segmentation algorithm for glioma based on 3D-Ghost CNN [J]. Journal of Computer Applications, 2024, 44(4): 1294-1302.
[11]	Jie WANG, Hua MENG. Image classification algorithm based on overall topological structure of point cloud [J]. Journal of Computer Applications, 2024, 44(4): 1107-1113.
[12]	Jingxian ZHOU, Xina LI. UAV detection and recognition based on improved convolutional neural network and radio frequency fingerprint [J]. Journal of Computer Applications, 2024, 44(3): 876-882.
[13]	Ruifeng HOU, Pengcheng ZHANG, Liyuan ZHANG, Zhiguo GUI, Yi LIU, Haowen ZHANG, Shubin WANG. Iterative denoising network based on total variation regular term expansion [J]. Journal of Computer Applications, 2024, 44(3): 916-921.
[14]	Yongfeng DONG, Jiaming BAI, Liqin WANG, Xu WANG. Chinese named entity recognition combining prior knowledge and glyph features [J]. Journal of Computer Applications, 2024, 44(3): 702-708.
[15]	Jiawei ZHANG, Guandong GAO, Ke XIAO, Shengzun SONG. Violent crime hierarchy algorithm by joint modeling of improved hierarchical attention network and TextCNN [J]. Journal of Computer Applications, 2024, 44(2): 403-410.

算法	参数量/10³	FLOPs/10⁹
ARI^［7］	—	—
DJDD^［11］	561	264
DRL^［12］	229	423
3-stage^［14］	1 793	3 309
本文算法	495	227

算法	参数量/10³	FLOPs/10⁹
ARI^［7］	—	—
DJDD^［11］	561	264
DRL^［12］	229	423
3-stage^［14］	1 793	3 309
本文算法	495	227