Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (7): 2175-2182.DOI: 10.11772/j.issn.1001-9081.2023070933

• Multimedia computing and computer simulation • Previous Articles     Next Articles

Dual-branch low-light image enhancement network combining spatial and frequency domain information

Dahai LI, Zhonghua WANG(), Zhendong WANG   

  1. School of Information Engineering,Jiangxi University of Science and Technology,Ganzhou Jiangxi 341000,China
  • Received:2023-07-13 Revised:2023-09-16 Accepted:2023-09-20 Online:2023-10-26 Published:2024-07-10
  • Contact: Zhonghua WANG
  • About author:LI Dahai, born in 1975, Ph. D., associate professor. His research interests include intelligent optimization algorithms, deep learning.
    WANG Zhendong, born in 1982, Ph. D., associate professor. His research interests include wireless sensor network, artificial intelligence, cybersecurity.
    First author contact:WANG Zhonghua, born in 1999, M. S. candidate. His research interests include computer vision.
  • Supported by:
    National Natural Science Foundation of China(61562037);Science Foundation of Jiangxi University of Science and Technology(205200100013)

结合空间域和频域信息的双分支低光照图像增强网络

李大海, 王忠华(), 王振东   

  1. 江西理工大学 信息工程学院,江西 赣州 341000
  • 通讯作者: 王忠华
  • 作者简介:李大海(1975—),男,山东乳山人,副教授,博士,CCF会员,主要研究方向:智能优化算法、深度学习;
    王振东(1982—),男,湖北随州人,副教授,博士,主要研究方向:无线传感器网络、人工智能、网络安全。
    第一联系人:王忠华(1999—),男,江西赣州人,硕士研究生,主要研究方向:计算机视觉;
  • 基金资助:
    国家自然科学基金资助项目(61562037);江西理工大学校级基金资助项目(205200100013)

Abstract:

To address the problems of blurred texture details and color distortion in low-light image enhancement, an end-to-end lightweight dual-branch network by combining spatial and frequency information, named SAFNet, was proposed. Transformer-based spatial block and frequency block were adopted by SAFNet to process spatial information and Fourier transformed frequency information of input image in spatial and frequency branchs, respectively. Attention mechanism was also applied in SAFNet to fuse features captured from spatial and frequency branchs adaptively to obtain final enhanced image. Furthermore, a frequency-domain loss function for frequency information was added into joint loss function, in order to constrain SAFNet on both spatial and frequency domains. Experiments on public datasets LOL and LSRW were conducted to evaluate the performance of SAFNet. Experimed results show that SAFNet achieved 0.823, 0.114 in metrics of Structural SIMilarity (SSIM) and Learned Perceptual Image Patch Similarity (LPIPS) on LOL, respectively, and 17.234 dB, 0.550 in Peak Signal-to-Noise Ratio (PSNR) and SSIM on LSRW. SAFNet achieve supreme performance than evaluated mainstream methods, such as LLFormer (Low-Light Transformer), IAT (Illumination Adaptive Transformer), and KinD (Kindling the Darkness) ++ with only 0.07×106 parameters. On DarkFace dataset, the average precision of human face detection is increased from 52.6% to 72.5% by applying SAFNet as preprocessing step. Above experimental results illustrate that SAFNet can effectively enhance low-light images quality and improve performance of low-light face detection for downstream tasks significantly.

Key words: low-light image enhancement, spatial domain, frequency domain, Transformer, attention mechanism, dual-branch network

摘要:

针对低光照图像增强中纹理细节模糊和颜色失真的问题,从空间域和频域信息结合的角度出发,提出一个端到端的轻量级双分支网络(SAFNet)。SAFNet使用基于Transformer的空间域处理模块和频域处理模块在空间域分支和频域分支分别对图像的空间域信息和傅里叶变换后的频域信息进行处理,并通过注意力机制引导两个分支的特征进行自适应融合,得到最终增强的图像。此外,针对频域信息提出一个频域损失函数作为联合损失函数的一部分,通过联合损失函数在空间域和频域都对SAFNet进行约束。在公开数据集LOL和LSRW上进行实验,在LOL上,SAFNet在客观指标结构相似性(SSIM)和学习感知图像块相似度(LPIPS)两项指标上分别达到0.823和0.114;在LSRW上,峰值信噪比(PSNR)和SSIM分别达到17.234 dB和0.550,均优于LLFormer (Low-Light Transformer)、IAT (Illumination Adaptive Transformer)、 KinD (Kindling the Darkness)++等主流方法,且网络参数量仅为0.07×106;在DarkFace数据集上,使用SAFNet作为预处理步骤对待检测图像进行增强,可以使人脸检测平均精确率从52.6%提升至72.5%。实验结果表明,SAFNet能有效提高低光照图像的质量,并能显著改善下游任务低光照人脸检测的性能。

关键词: 低光照图像增强, 空间域、频域信息, Transformer, 注意力机制, 双分支网络

CLC Number: