Progressive enhancement algorithm for low-light images based on layer guidance

doi:10.11772/j.issn.1001-9081.2023060736

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (6): 1911-1919.DOI: 10.11772/j.issn.1001-9081.2023060736

Special Issue: 多媒体计算与计算机仿真

• Multimedia computing and computer simulation • Previous Articles Next Articles

Progressive enhancement algorithm for low-light images based on layer guidance

Mengyuan HUANG¹, Kan CHANG¹^,²(), Mingyang LING¹, Xinjie WEI¹, Tuanfa QIN¹^,²

^1.School of Computer and Electronic Information，Guangxi University，Nanning Guangxi 530004，China
^2.Guangxi Key Laboratory of Multimedia Communications and Network Technology （Guangxi University），Nanning Guangxi 530004，China

Received:2023-06-09 Revised:2023-08-08 Accepted:2023-08-10 Online:2023-08-30 Published:2024-06-10
Contact: Kan CHANG
About author:HUANG Mengyuan， born in 1997， M. S. candidate. Her research interests include computer vision， exposure correction.
LING Mingyang， born in 1998， Ph. D. candidate. Her research interests include super-resolution reconstruction.
WEI Xinjie， born in 1996， M. S. candidate. His research interests include low-light image enhancement.
QIN Tuanfa， born in 1966， Ph. D.， professor. His research interests include multimedia communication， network coding， image and video retrieval.
Supported by:
National Natural Science Foundation of China(62171145)

基于层间引导的低光照图像渐进增强算法

黄梦源¹, 常侃¹^,²(), 凌铭阳¹, 韦新杰¹, 覃团发¹^,²

^1.广西大学计算机与电子信息学院, 南宁 530004
^2.广西多媒体通信与网络技术重点实验室(广西大学), 南宁 530004

通讯作者: 常侃
作者简介:黄梦源（1997—），女（壮族），广西南宁人，硕士研究生，主要研究方向：计算机视觉、曝光调整
凌铭阳（1998—），女（壮族），广西钦州人，博士研究生，主要研究方向：超分辨率重建
韦新杰（1996—），男（壮族），广西贵港人，硕士研究生，主要研究方向：低光照图像增强
覃团发（1966—），男，广西宾阳人，教授，博士，主要研究方向：多媒体通信、网络编码、图像和视频检索。
基金资助:
国家自然科学基金资助项目(62171145)

Abstract

Abstract:

The quality of low-light images is poor and Low-Light Image Enhancement （LLIE） aims to improve the visual quality. Most of LLIE algorithms focus on enhancing luminance and contrast， while neglecting details. To solve this issue， a Progressive Enhancement algorithm for low-light images based on Layer Guidance （PELG） was proposed， which enhanced algorithm images to a suitable illumination level and reconstructed clear details. First， to reduce the task complexity and improve the efficiency， the image was decomposed into several frequency components by Laplace Pyramid （LP） decomposition. Secondly， since different frequency components exhibit correlation， a Transformer-based fusion model and a lightweight fusion model were respectively proposed for layer guidance. The Transformer-based model was applied between the low-frequency and the lowest high-frequency components. The lightweight model was applied between two neighbouring high-frequency components. By doing so， components were enhanced in a coarse-to-fine manner. Finally， the LP was used to reconstruct the image with uniform brightness and clear details. The experimental results show that， the proposed algorithm achieves the Peak Signal-to-Noise Ratio （PSNR） 2.3 dB higher than DSLR （Deep Stacked Laplacian Restorer） on LOL（LOw-Light dataset）-v1 and 0.55 dB higher than UNIE （Unsupervised Night Image Enhancement） on LOL-v2. Compared with other state-of-the-art LLIE algorithms， the proposed algorithm has shorter runtime and achieves significant improvement in objective and subjective quality， which is more suitable for real scenes.

Key words: Low-Light Image Enhancement (LLIE), Laplace Pyramid (LP), feature fusion, Convolutional Neural Network (CNN), Transformer

摘要：

低光照图像的图像质量通常较低，低光照图像增强（LLIE）旨在提高这类图像的视觉质量。针对现有的LLIE算法大多专注增强亮度和对比度、忽略细节增强的问题，提出一个基于层间引导的低光照图像渐进增强算法（PELG），兼顾图像亮度和细节增强。首先，使用拉普拉斯金字塔（LP）降低任务复杂度，提高算法效率；其次，利用各频率分量间的相关性，在低频和高频分量之间构建基于Transformer的层间引导融合模块，在各高频分量之间构建轻量级的层间引导融合模块，有效精炼金字塔较低层增强信息指导较高层处理图像，实现基于层间引导的渐进增强；最后，通过LP重建亮度均匀、细节清晰的增强图像。实验结果表明，所提算法的峰值信噪比（PSNR）在LOL（LOw-Light dataset）-v1上比DSLR（Deep Stacked Laplacian Restorer）高2.3 dB，在LOL-v2上比UNIE（Unsupervised Night Image Enhancement）高0.55 dB；与其他基于深度学习的LLIE算法相比，所提算法运行速度快，增强结果在客观和主观质量上均获得明显提升，更适用于实际场景。

关键词: 低光照图像增强, 拉普拉斯金字塔, 特征融合, 卷积神经网络, Transformer

CLC Number:

TP391.9

Mengyuan HUANG, Kan CHANG, Mingyang LING, Xinjie WEI, Tuanfa QIN. Progressive enhancement algorithm for low-light images based on layer guidance[J]. Journal of Computer Applications, 2024, 44(6): 1911-1919.

黄梦源, 常侃, 凌铭阳, 韦新杰, 覃团发. 基于层间引导的低光照图像渐进增强算法[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1911-1919.

Figures/Tables 13

Fig.1 Results of LP decomposition of low-/normal-light images

Fig.2 Overall structure of PELG model

Fig.3 Structure of LEM network

Tab.1 Performance comparison among various algorithms on different datasets

算法	LOL-v1数据集					LOL-v2数据集
算法	PSNR/dB	SSIM	LPIPS	$Δ E *$	LOE	PSNR/dB	SSIM	LPIPS	$Δ E *$	LOE
MBLLEN^［11］	18.76	0.881 3	0.091 1	14.08	410	19.12	0.846 7	0.153 3	23.18	458
RetinexNet^［12］	17.84	0.840 1	0.375 9	18.15	645	15.88	0.780 3	0.324 6	25.48	595
KinD^［13］	20.38	0.891 8	0.108 1	12.84	347	18.63	0.843 5	0.279 0	14.72	445
EnlightenGAN^［4］	19.15	0.852 0	0.120 2	14.11	483	16.40	0.812 2	0.166 7	19.01	445
Zero-DCE^［5］	16.33	0.700 6	0.247 5	20.92	413	18.43	0.705 5	0.133 9	17.51	580
DRBN^［8］	18.80	0.891 3	0.100 8	14.55	511	20.14	0.8853	0.236 0	11.66	569
DSLR^［14］	21.64	0.841 5	0.208 0	12.86	573	18.60	0.824 2	0.0943	14.77	596
MSEC^［15］	19.02	0.849 9	0.213 9	15.35	489	18.74	0.805 7	0.316 5	14.23	588
UNIE^［6］	21.52	0.837 2	0.176 0	12.21	326	20.85	0.813 5	0.217 1	11.87	350
PairLIE^［7］	19.51	0.820 1	0.171 9	13.87	347	18.03	0.790 9	0.205 3	15.28	493
PELG	23.94	0.9061	0.0926	10.47	244	21.40	0.880 6	0.105 9	11.63	258

Tab.1 Performance comparison among various algorithms on different datasets

算法	LOL-v1数据集					LOL-v2数据集
算法	PSNR/dB	SSIM	LPIPS	$Δ E *$	LOE	PSNR/dB	SSIM	LPIPS	$Δ E *$	LOE
MBLLEN^［11］	18.76	0.881 3	0.091 1	14.08	410	19.12	0.846 7	0.153 3	23.18	458
RetinexNet^［12］	17.84	0.840 1	0.375 9	18.15	645	15.88	0.780 3	0.324 6	25.48	595
KinD^［13］	20.38	0.891 8	0.108 1	12.84	347	18.63	0.843 5	0.279 0	14.72	445
EnlightenGAN^［4］	19.15	0.852 0	0.120 2	14.11	483	16.40	0.812 2	0.166 7	19.01	445
Zero-DCE^［5］	16.33	0.700 6	0.247 5	20.92	413	18.43	0.705 5	0.133 9	17.51	580
DRBN^［8］	18.80	0.891 3	0.100 8	14.55	511	20.14	0.8853	0.236 0	11.66	569
DSLR^［14］	21.64	0.841 5	0.208 0	12.86	573	18.60	0.824 2	0.0943	14.77	596
MSEC^［15］	19.02	0.849 9	0.213 9	15.35	489	18.74	0.805 7	0.316 5	14.23	588
UNIE^［6］	21.52	0.837 2	0.176 0	12.21	326	20.85	0.813 5	0.217 1	11.87	350
PairLIE^［7］	19.51	0.820 1	0.171 9	13.87	347	18.03	0.790 9	0.205 3	15.28	493
PELG	23.94	0.9061	0.0926	10.47	244	21.40	0.880 6	0.105 9	11.63	258

Tab.2 Comparisons of model size， FLOPs and runtime

算法	参数量/10⁶	浮点运算量/10⁹	运行时间/ms
MBLLEN^［11］	0.45	73.09	32.7
RetinexNet^［12］	0.44	141.26	25.6
KinD^［13］	8.02	119.55	16.2
EnlightenGAN^［4］	8.64	65.78	12.0
Zero-DCE^［5］	0.08	20.76	3.4
DRBN^［8］	0.56	42.41	21.6
DSLR^［14］	14.93	23.56	12.4
MSEC^［15］	7.02	38.40	9.7
UNIE^［6］	8.64	65.70	9.9
PairLIE^［7］	0.34	89.39	13.3
PELG	0.71	26.28	8.1

Fig.4 Visual effect comparison of various LLIE algorithms on LOL-v1 dataset

Fig.5 Visual comparison of various LLIE methods on LOL-v2 dataset

Tab.3 Influence of LP and layer guidance

算法	LP	层间引导	LEM	PSNR/dB	SSIM	参数量/10⁶	浮点运算量/10⁹	运算时间/ms
变种算法1	×	×	×	18.44	0.767 0	0.74	194.18	22.5
变种算法2	√	×	×	20.56	0.892 6	0.77	21.15	5.8
变种算法3	√	√	×	21.17	0.902 3	0.75	22.11	6.2
变种算法4	√	×	√	22.75	0.894 3	0.73	29.72	9.0
变种算法5	√	√	√	22.90	0.900 8	0.71	26.12	8.3
完整算法	√	√	√	23.94	0.906 1	0.71	26.28	8.1

Fig.6 Enhancement results of various variant algorithms

Tab.4 Influence of decomposition layers in LP

高频分量层数n	PSNR/ dB	SSIM	参数量/ 10⁶	浮点运算量/ 10⁹	运算时间/ ms
1	22.45	0.893 5	0.58	93.09	16.8
2	23.03	0.898 0	0.65	39.65	10.3
3	23.94	0.906 1	0.71	26.28	8.1
4	23.16	0.895 1	0.77	22.94	9.6
5	22.06	0.886 7	0.84	22.11	10.9

Fig.7 Enhancement results of various LP decomposition layers

Tab.5 Influence of RAB number in low-frequency layer

低频层RAB数	PSNR /dB	SSIM	参数量/ 10⁶	浮点运算量/ 10⁹	运算时间/ ms
2	22.83	0.899 5	0.56	25.68	7.1
3	23.45	0.906 0	0.64	25.99	7.7
4	23.94	0.906 1	0.71	26.28	8.1
5	23.81	0.905 6	0.78	26.59	8.5
6	23.79	0.906 1	0.86	26.89	8.9

Tab.6 Influence of RAB number in DRM

DRM中RAB数	PSNR/ dB	SSIM	参数量/ 10⁶	浮点运算量/ 10⁹	运算时间/ ms
1	22.87	0.888 2	0.47	11.70	4.2
2	23.56	0.902 8	0.60	19.00	6.7
3	23.94	0.906 1	0.71	26.28	8.1
4	23.84	0.909 7	0.82	33.57	10.6
5	23.75	0.904 5	0.93	40.87	13.0

References 26

1	LEE C， LEE C， KIM C-S. Contrast enhancement based on layered difference representation of 2D histograms ［J］. IEEE Transactions on Image Processing， 2013， 22 （12）： 5372-5384.
2	PIERRE F， J-F AUJOL， BUGEAU A， et al. Hue-preserving perceptual contrast enhancement ［C］// Proceedings of the 2016 IEEE International Conference on Image Processing. Piscataway： IEEE， 2016： 4067-4071.
3	GUO X， LI Y， LING H. LIME： low-light image enhancement via illumination map estimation ［J］. IEEE Transactions on Image Processing， 2017， 26（2）： 982-993.
4	JIANG Y， GONG X， LIU D， et al. EnlightenGAN： deep light enhancement without paired supervision ［J］. IEEE Transactions on Image Processing， 2021， 30： 2340-2349.
5	GUO C， LI C， GUO J， et al. Zero-reference deep curve estimation for low-light image enhancement ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 1777-1786.
6	JIN Y， YANG W， TAN T R. Unsupervised night image enhancement： when layer decomposition meets light-effects suppression ［C］// Proceedings of the 17th European Conference on Computer Vision. Berlin： Springer， 2022： 404-421.
7	FU Z， YANG Y， TU X， et al. Learning a simple low-light image enhancer from paired low-light instances ［C］// Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2023： 22252-22261.
8	YANG W， WANG S， FANG Y， et al. From fidelity to perceptual quality： a semi-supervised approach for low-light image enhancement ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 3060-3069.
9	祖佳贞，周永霞，陈乐. 结合注意力的双分支残差低光照图像增强［J］. 计算机应用， 2023， 43（4）： 1240-1247.
	ZU J Z， ZHOU Y X， CHEN L. Dual-branch residual low-light image enhancement combined with attention［J］. Journal of Computer Applications， 2023， 43（4）： 1240-1247.
10	谌贵辉，林瑾瑜，李跃华，等. 注意力机制下的多阶段低照度图像增强网络［J］. 计算机应用， 2023， 43（2）： 552-559.
	CHEN G H， LIN J Y， LI Y H， et al. Multi-stage low-illuminance image enhancement network based on attention mechanism［J］. Journal of Computer Applications， 2023， 43（2）： 552-559.
11	LV F， LU F， WU J， et al. MBLLEN： low-light image/video enhancement using CNNs ［C］// Proceedings of the 2018 British Machine Vision Conference. Birmingham： British Machine Vision Association， 2018： 220.
12	WEI C， WANG W， YANG W， et al. Deep Retinex decomposition for low-light enhancement ［C］// Proceedings of the 2018 British Machine Vision Conference. Birmingham： British Machine Vision Association， 2018： 155.
13	ZHANG Y， ZHANG J， GUO X. Kindling the darkness： a practical low-light image enhancer ［C］// Proceedings of the 27th ACM International Conference on Multimedia. New York： ACM， 2019： 1632-1640.
14	LIM S， KIM W. DSLR： deep stacked Laplacian restorer for low-light image enhancement ［J］. IEEE Transactions on Multimedia， 2021， 23： 4272-4284.
15	AFIFI M， DERPANIS K G， OMMER B， et al. Learning multi-scale photo exposure correction ［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 9157-9167.
16	BURT P， ADELSON E. The Laplacian pyramid as a compact image code ［J］. IEEE Transactions on Communications， 1983， 31（4）： 532-540.
17	LAND E H， McCANN J J. Lightness and Retinex theory ［J］. The Journal of the Optical Society of America， 1971， 61（1）： 1-11.
18	DABOV K， FOI A， KATKOVNIK V， et al. Color image denoising via sparse 3D collaborative filtering with grouping constraint in luminance-chrominance space ［C］// Proceedings of the 2007 IEEE International Conference on Image Processing. Piscataway： IEEE， 2007： 313-316.
19	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778.
20	WOO S， PARK J， LEE J-Y， et al. CBAM： convolutional block attention module ［C］// Proceedings of the 15th European Conference on Computer Vision. Berlin： Springer， 2018： 3-19.
21	ZAMIR S W， ARORA A， KHAN S， et al. Restormer： efficient transformer for high-resolution image restoration ［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 5718-5729.
22	YAN Q， GONG D， WANG P， et al. SharpFormer： learning local feature preserving global representations for image deblurring ［J］. IEEE Transactions on Image Processing， 2023， 32： 2857-2866.
23	GAO G， XU Z， LI J， et al. CTCNet： a CNN-Transformer cooperation network for face image super-resolution［J］. IEEE Transactions on Image Processing， 2023， 32：1978-1991.
24	ZHANG R， ISOLA P， EFROS A A， et al. The unreasonable effectiveness of deep features as a perceptual metric ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 586-595.
25	HOVIS J. Color vision： perspectives from different disciplines ［J］. Optometry and Vision Science， 1999， 76（1）： 10.
26	YING Z， LI G， REN Y， et al. A new low-light image enhancement algorithm using camera response model ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops. Piscataway： IEEE， 2017： 3015-3022.

[1]	Jiepo FANG, Chongben TAO. Hybrid internet of vehicles intrusion detection system for zero-day attacks [J]. Journal of Computer Applications, 2024, 44(9): 2763-2769.
[2]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.
[3]	Xin YANG, Xueni CHEN, Chunjiang WU, Shijie ZHOU. Short-term traffic flow prediction of urban highway based on variant residual model and Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2947-2951.
[4]	Jieru JIA, Jianchao YANG, Shuorui ZHANG, Tao YAN, Bin CHEN. Unsupervised person re-identification based on self-distilled vision Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2893-2902.
[5]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[6]	Yun LI, Fuyou WANG, Peiguang JING, Su WANG, Ao XIAO. Uncertainty-based frame associated short video event detection method [J]. Journal of Computer Applications, 2024, 44(9): 2903-2910.
[7]	Jinjin LI, Guoming SANG, Yijia ZHANG. Multi-domain fake news detection model enhanced by APK-CNN and Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2674-2682.
[8]	Liehong REN, Lyuwen HUANG, Xu TIAN, Fei DUAN. Multivariate long-term series forecasting method with DFT-based frequency-sensitive dual-branch Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2739-2746.
[9]	Yuwei DING, Hongbo SHI, Jie LI, Min LIANG. Image denoising network based on local and global feature decoupling [J]. Journal of Computer Applications, 2024, 44(8): 2571-2579.
[10]	Kaili DENG, Weibo WEI, Zhenkuan PAN. Industrial defect detection method with improved masked autoencoder [J]. Journal of Computer Applications, 2024, 44(8): 2595-2603.
[11]	Fan YANG, Yao ZOU, Mingzhi ZHU, Zhenwei MA, Dawei CHENG, Changjun JIANG. Credit card fraud detection model based on graph attention Transformation neural network [J]. Journal of Computer Applications, 2024, 44(8): 2634-2642.
[12]	Hong CHEN, Bing QI, Haibo JIN, Cong WU, Li’ang ZHANG. Class-imbalanced traffic abnormal detection based on 1D-CNN and BiGRU [J]. Journal of Computer Applications, 2024, 44(8): 2493-2499.
[13]	Dongwei WANG, Baichen LIU, Zhi HAN, Yanmei WANG, Yandong TANG. Deep network compression method based on low-rank decomposition and vector quantization [J]. Journal of Computer Applications, 2024, 44(7): 1987-1994.
[14]	Yangyi GAO, Tao LEI, Xiaogang DU, Suiyong LI, Yingbo WANG, Chongdan MIN. Crowd counting and locating method based on pixel distance map and four-dimensional dynamic convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2233-2242.
[15]	Dahai LI, Zhonghua WANG, Zhendong WANG. Dual-branch low-light image enhancement network combining spatial and frequency domain information [J]. Journal of Computer Applications, 2024, 44(7): 2175-2182.

Progressive enhancement algorithm for low-light images based on layer guidance

基于层间引导的低光照图像渐进增强算法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 26

Related Articles 15

Recommended Articles

Metrics