Multi-stage low-illuminance image enhancement network based on attention mechanism

doi:10.11772/j.issn.1001-9081.2022010093

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (2): 552-559.DOI: 10.11772/j.issn.1001-9081.2022010093

• Multimedia computing and computer simulation • Previous Articles

Multi-stage low-illuminance image enhancement network based on attention mechanism

Guihui CHEN¹, Jinyu LIN¹(), Yuehua LI², Zhongbing LI¹, Yuli WEI¹, Kai LU¹

^1.School of Electrical Engineering and Information，Southwest Petroleum University，Chengdu Sichuan 610500，China
^2.Department of Mechanical and Electrical Engineering，Nanchong Vocational and Technical College，Nanchong Sichuan 637131，China

Received:2022-01-25 Revised:2022-04-13 Accepted:2022-04-18 Online:2022-04-21 Published:2023-02-10
Contact: Jinyu LIN
About author:CHEN Guihui， born in 1971， M. S.， professor. His research interests include digital image processing.
LI Yuehua， born in 1971， M. S.， lecturer. His research interests include physics， electrical automatization.
LI Zhongbing， born in 1985， Ph. D.， lecturer. His research interests include digital image processing.
WEI Yuli， born in 1999， M. S. candidate. His research interests include big data analysis.
LU Kai， born in 1997， M. S. candidate. His research interests include digital image processing.
Supported by:
Nanchong City-University Science and Technology Strategic Cooperation Project(21SXHZ020)

注意力机制下的多阶段低照度图像增强网络

谌贵辉¹, 林瑾瑜¹(), 李跃华², 李忠兵¹, 魏钰力¹, 卢凯¹

^1.西南石油大学电气信息学院，成都 610500
^2.南充职业技术学院机电工程系，四川南充 637131

通讯作者: 林瑾瑜
作者简介:谌贵辉（1971—），男，四川成都人，教授，硕士，主要研究方向：数字图像处理
李跃华（1971—），男，四川蓬安人，讲师，硕士，主要研究方向：物理、电气自动化
李忠兵（1985—），男，湖北武汉人，讲师，博士，主要研究方向：数字图像处理
魏钰力（1999—），男，四川绵阳人，硕士研究生，主要研究方向：大数据分析
卢凯（1997—），男，四川成都人，硕士研究生，主要研究方向：数字图像处理。
基金资助:
南充市市校科技战略合作项目(21SXHZ020)

Abstract

Abstract:

A multi-stage low-illuminance image enhancement network based on attention mechanism was proposed to solve the problem that the details of low-illuminance images are lost due to the overlapping of image contents and large brightness differences in some regions during the enhancement process of low-illuminance images. At the first stage， an improved multi-scale fusion module was used to perform preliminary image enhancement. At the second stage， the enhanced image information of the first stage was cascaded with the input of this stage， and the result was used as the input of the multi-scale fusion module in this stage. At the third stage， the enhanced image information of the second stage was cascaded with the input of the this stage， and the result was used as the input of the multi-scale fusion module in this stage. In this way， with the use of multi-stage fusion， not only the brightness of the image was improved adaptively， but also the details were retained adaptively. Experimental results on open datasets LOL and SICE show that compared to the algorithms and networks such as MSR （Multi-Scale Retinex） algorithm， gray Histogram Equalization （HE） algorithm and RetinexNet （Retina cortex Network）， the proposed network has the value of Peak Signal-to-Noise Ratio （PSNR） 11.0% to 28.9% higher， and the value of Structural SIMilarity （SSIM） increased by 6.8% to 46.5%. By using multi-stage method and attention mechanism to realize low-illuminance image enhancement， the proposed network effectively solves the problems of image content overlapping and large brightness difference， and the images obtained by this network are more detailed and subjective recognizable with clearer textures.

Key words: low-illumination image, attention mechanism, multi-stage, multi-scale, adaptive

摘要：

对于低照度图像增强过程中，因图像内容重叠且部分区域亮度差异较大导致的图像细节丢失的问题，提出一个注意力机制下的多阶段低照度图像增强网络。第一阶段利用改进的多尺度融合模块对图像进行初步增强；第二阶段利用第一阶段增强后的图像信息与本阶段的输入进行级联，并将其结果作为该阶段多尺度融合模块的输入；第三阶段利用第二阶段增强后的图像信息与该阶段的输入级联，并将其结果作为该阶段多尺度融合模块的输入。这样利用多阶段的方式完成自适应的亮度提升和细节的保留。在公开数据集LOL和SICE上的实验结果表明，相较于MSR算法、灰度直方图均衡化（HE）算法和RetinexNet等算法和网络，所提网络的峰值信噪比（PSNR）的数值提高了11.0%~28.9%，结构相似性（SSIM）的数值提高了6.8%~46.5%。所提网络利用多阶段和注意力机制实现低照度图像增强，有效解决了图像内容重叠和亮度差异大的问题，得到的图像细节更丰富，纹理更清晰，主观辨识度更高。

关键词: 低照度图像, 注意力机制, 多阶段, 多尺度, 自适应

CLC Number:

TP391.4

Guihui CHEN, Jinyu LIN, Yuehua LI, Zhongbing LI, Yuli WEI, Kai LU. Multi-stage low-illuminance image enhancement network based on attention mechanism[J]. Journal of Computer Applications, 2023, 43(2): 552-559.

谌贵辉, 林瑾瑜, 李跃华, 李忠兵, 魏钰力, 卢凯. 注意力机制下的多阶段低照度图像增强网络[J]. 《计算机应用》唯一官方网站, 2023, 43(2): 552-559.

Figures/Tables 14

Fig.1 Network architecture of MMCA-Net

Fig.2 Principle of bilinear interpolation method

Fig.3 Coordinate Attention-UNet module

Fig.4 Coordinate Attention module

Fig.5 Dataset image pairs

Fig.6 Loss function changes during training process

Fig.7 Subjective images of UNet network and CAU network

Tab.1 PSNR and SSIM values of UNet and CAU networks

网络	PSNR/dB	SSIM
UNet网络	11.98	0.30
CAU网络	13.52	0.35

Fig.8 Subjective images with different attention mechanism modules introduced

Tab.2 PSNR and SSIM values of multi-scale fusion networks with different attention mechanisms

网络	PSNR/dB	SSIM
SENet网络	12.63	0.31
CBAM网络	13.33	0.34
CA网络	13.52	0.35

Fig.9 Subjective images of Coordinate-Attention UNet network at different stages

Tab.3 PSNR and SSIM values after CAU network processing and number of network parameters at different stages

网络	PSNR/dB	SSIM	参数量/10⁶
单阶段CAU网络	13.52	0.35	34.5
两阶段CAU网络	18.46	0.58	69.3
三阶段CAU网络	21.56	0.63	103.7
四阶段CAU网络			135.6

Fig.10 Original images and effect images enhanced by different methods

Tab.4 PSNR and SSIM values and number of network parameters of different methods

方法	PSNR/dB	SSIM	参数量/10⁶
HE	16.86	0.49
MSR	17.10	0.54
RetinexNet	16.72	0.43	0.87
DSLR	19.43	0.59	23.40
MMCA-Net	21.56	0.63	103.70

References 23

1	刘佳敏，何宁，尹晓杰. 基于Retinex-UNet算法的低照度图像增强［J］. 计算机工程与应用， 2020， 56（22）： 211-216. 10.3778/j.issn.1002-8331.1910-0239
	LIU J M， HE N， YIN X J. Low illumination image enhancement based on RetiNex-UNet algorithm［J］. Computer Engineering and Applications， 2020， 56（22）： 211-216. 10.3778/j.issn.1002-8331.1910-0239
2	LAND E H， McCANN J J. Lightness and retinex theory［J］. Journal of the Optical Society of America， 1971， 61 （1）： 1-11. 10.1364/josa.61.000001
3	JOBSON D J， RAHMAN Z， WOODELL G A. Properties and performance of a center/surround retinex［J］. IEEE Transactions on Image Processing， 1997， 6（3）： 451-462. 10.1109/83.557356
4	RAHMAN Z， OBSON D J， WOODELL G A. Multiscale retinex for color rendition and dynamic range compression［C］// Proceedings of the SPIE 2847， Applications of Digital Image Processing XIX. Bellingham， WA： SPIE， 1996： No.258224. 10.1117/12.258224
5	陈灿灿. 基于Retinex理论的低照度彩色图像增强算法研究［D］. 沈阳：沈阳航空航天大学， 2018： 6-45.
	CHEN C C. Low illumination color image enhancement algorithm research based on retinex theory［D］. Shenyang： Shenyang Aerospace University， 2018：6-45.
6	卢玮，高涛，王翠翠，等. Retinex理论下基于融合思想的低照度彩色图像增强算法［J］. 科学技术与工程， 2019， 19（13）： 151-157. 10.3969/j.issn.1671-1815.2019.13.024
	LU W， GAO T， WANG C C， et al. Low illumination color image enhancement algorithm based on fusion idea under Retinex theory［J］. Science Technology and Engineering， 2019， 19 （13）： 151-157. 10.3969/j.issn.1671-1815.2019.13.024
7	ABDULLAH-ALWADUD M， KABIR M H， AKBER DEWAN M A， et al. A dynamic histogram equalization for image contrast enhancement［J］. IEEE Transactions on Consumer Electronics， 2007， 53（2）： 593-600. 10.1109/tce.2007.381734
8	IBRAHIM H， KONG N S P. Brightness preserving dynamic histogram equalization for image contrast enhancement［J］. IEEE Transactions on Consumer Electronics， 2007， 53（4）： 1752-1758. 10.1109/tce.2007.4429280
9	LORE K G， AKINTAYO A， SARKAR S. LLNet： a deep autoencoder approach to natural low-light image enhancement［J］. Pattern Recognition， 2017， 61： 650-662. 10.1016/j.patcog.2016.06.008
10	TAO L， ZHU C， XIANG G Q， et al. LLCNN： a convolutional neural network for low-light image enhancement［C］// Proceedings of the 2017 IEEE Visual Communications and Image Processing. Piscataway： IEEE， 2017： 1-4. 10.1109/vcip.2017.8305143
11	WEI C， WANG W J， YANG W H， et al. Deep Retinex decomposition for low-light enhancement［C］// Proceedings of the 2018 British Machine Vision Conference. Durham： BMVA Press， 2018： No.155. 10.48550/arXiv.1808.04560
12	WANG W J， WEI C， YANG W H， et al. GLADNet： low-light enhancement network with global awareness［C］// Proceedings of the 13th IEEE International Conference on Automatic Face and Gesture Recognition. Piscataway： IEEE， 2018：751-755. 10.1109/fg.2018.00118
13	JIANG Y F， GONG X Y， LIU D， et al. EnlightenGAN： deep light enhancement without paired supervision［J］. IEEE Transactions on Image Processing， 2021， 30： 2340-2349. 10.1109/tip.2021.3051462
14	LIM S， KIM W. DSLR： deep stacked Laplacian restorer for low-light image enhancement［J］. IEEE Transactions on Multimedia， 2021， 23： 4272-4284. 10.1109/tmm.2020.3039361
15	ZAMIR S W， ARORA A， KHAN S， et al. Multi-stage progressive image restoration［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 14816-14826. 10.1109/cvpr46437.2021.01458
16	RONNEBERGER O， FISCHER P， BROX T. U-net： convolutional networks for biomedical image segmentation［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS 9351. Cham： Springer， 2015： 234-241.
17	HOU Q B， ZHOU D Q， FENG J S. Coordinate attention for efficient mobile network design［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 13708-13717. 10.1109/cvpr46437.2021.01350
18	JIE H， LI S， ALBANIE S， et al. Squeeze-and-excitation networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2020， 42（8）： 2011-2023. 10.1109/tpami.2019.2913372
19	WOO S A， PARK J B， LEE J C. CBAM： convolutional block attention module［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11211. Cham： Springer， 2018： 3-19.
20	DIEDERIK K， JIMMY B. Adam： a method for stochastic optimization［EB/OL］. （2017-01-30）［2021-12-03］..
21	WANG Z， BOVIK A C， SHEIKH H R， et al. Image quality assessment： from error visibility to structural similarity［J］. IEEE Transactions on Image Processing， 2004， 13（4）： 600-612. 10.1109/tip.2003.819861
22	CHENG H D， SHI X J. A simple and effective histogram equalization approach to image enhancement［J］. Digital Signal Processing， 2004， 14（2）：158-170. 10.1016/j.dsp.2003.07.002
23	CAI J R， GU S H， ZHANG L. Learning a deep single image contrast enhancer from multi-exposure images［J］. IEEE Transactions on Image Processing， 2018， 27（4）： 2049-2062. 10.1109/tip.2018.2794218

[1]	Cong LIU, Genshun WAN, Jianqing GAO, Zhonghua FU. End-to-end speech recognition method based on prosodic features [J]. Journal of Computer Applications, 2023, 43(2): 380-384.
[2]	Gang CHEN, Yongwei LIAO, Zhenguo YANG, Wenying LIU. Image inpainting algorithm of multi-scale generative adversarial network based on multi-feature fusion [J]. Journal of Computer Applications, 2023, 43(2): 536-544.
[3]	Ming XU, Linhao LI, Qiaoling QI, Liqin WANG. Abductive reasoning model based on attention balance list [J]. Journal of Computer Applications, 2023, 43(2): 349-355.
[4]	Xuesen MA, Xuemei XU, Gonghui JIANG, Yan QIAO, Tianbao ZHOU. Hybrid adaptive particle swarm optimization algorithm for workflow scheduling [J]. Journal of Computer Applications, 2023, 43(2): 474-483.
[5]	Xiaomeng SHAO, Meng ZHANG. Temporal convolutional knowledge tracing model with attention mechanism [J]. Journal of Computer Applications, 2023, 43(2): 343-348.
[6]	Ping WANG, Nan CHEN, Lei LU. Fall detection algorithm based on scene prior and attention guidance [J]. Journal of Computer Applications, 2023, 43(2): 529-535.
[7]	Yaling XUN, Linqing WANG, Jianghui CAI, Haifeng YANG. Partial periodic pattern incremental mining of time series data based on multi-scale [J]. Journal of Computer Applications, 2023, 43(2): 391-397.
[8]	Zihao GUO, Lele DONG, Zhijian QU. Arthropod object detection method based on improved Faster RCNN [J]. Journal of Computer Applications, 2023, 43(1): 88-97.
[9]	Kewen LI, Jiantao YANG, Zongchao HUANG. Improved YOLOv3 target detection based on boundary limit point features [J]. Journal of Computer Applications, 2023, 43(1): 81-87.
[10]	Zhijun SHEN, Lina MU, Jing GAO, Yuanhang SHI, Zhiqiang LIU. Review of fine-grained image categorization [J]. Journal of Computer Applications, 2023, 43(1): 51-60.
[11]	Renbiao WU, Zhenchi ZHANG, Yunfei JIA, Han QIAO. Adaptive scheduling strategy based on deadline under cloud platform [J]. Journal of Computer Applications, 2023, 43(1): 176-184.
[12]	Zeqiang SUN, Bingcai CHEN, Xiaobo CUI, Lei WANG, Yanuo LU. Strip steel surface defect detection by YOLOv5 algorithm fusing frequency domain attention mechanism and decoupled head [J]. Journal of Computer Applications, 2023, 43(1): 242-249.
[13]	Honggang YANG, Jiejie CHEN, Mengfei XU. Bilinear involution neural network for image classification of fundus diseases [J]. Journal of Computer Applications, 2023, 43(1): 259-264.
[14]	Yuchao LYU, Xi JIANG, Yinghao XU, Xijun ZHU. Improved brachial plexus nerve segmentation method based on multi-scale feature fusion [J]. Journal of Computer Applications, 2023, 43(1): 273-279.
[15]	Jun ZHANG, Pengli WU, Lukui SHI, Jin SHI, Bin PAN. Deep learning model for multi-station temperature prediction combined with MOD11A1 and surface meteorological station data [J]. Journal of Computer Applications, 2023, 43(1): 321-328.

Multi-stage low-illuminance image enhancement network based on attention mechanism

注意力机制下的多阶段低照度图像增强网络

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 23

Related Articles 15

Recommended Articles

Metrics