Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (8): 2391-2395.DOI: 10.11772/j.issn.1001-9081.2019010158

• Virtual reality and multimedia computing • Previous Articles     Next Articles

Self-attention network based image super-resolution

OUYANG Ning1,2, LIANG Ting2, LIN Leping1,2   

  1. 1. Key Laboratory of Cognitive Radio and Information Processing of Ministry of Education(Guilin University of Electronic Technology), Guilin Guangxi 541004, China;
    2. School of Information and Communication, Guilin University of Electronic Technology, Guilin Guangxi 541004, China
  • Received:2019-01-23 Revised:2019-03-15 Online:2019-04-15 Published:2019-08-10
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61661017), the China Postdoctoral Science Foundation (2016M602923XB), the Fundation of the Key Laboratory of Cognitive Radio and Information Processing of Ministry of Education (CRKL160104, CRKL150103, 2011KF11), the Natural Science Foundation of Guangxi (2017GXNSFBA198212, 2014GXNSFDA118035, 2016GXNSFAA38014), the Scientific and Technological Innovation Ability and Condition Construction Plan of Guangxi (1598025-21), the Innovation Project of GUET Graduate Education (2016YJCXB02), the Scientific and Technological Bureau of Guilin (20150103-6).


欧阳宁1,2, 梁婷2, 林乐平1,2   

  1. 1. 认知无线电与信息处理省部共建教育部重点实验室(桂林电子科技大学), 广西 桂林 541004;
    2. 桂林电子科技大学 信息与通信学院, 广西 桂林 541004
  • 通讯作者: 林乐平
  • 作者简介:欧阳宁(1972-),男,湖南宁远人,教授,硕士,主要研究方向:数字图像处理、智能信息处理;梁婷(1993-),女,广西梧州人,硕士研究生,主要研究方向:模式识别、深度学习;林乐平(1980-),女,广西桂平人,副教授,博士,主要研究方向:机器学习、智能信息处理、图像信号处理。
  • 基金资助:

Abstract: Concerning the recovery problem of high-frequency information like texture details in image super-resolution reconstruction, an image super-resolution reconstruction method based on self-attention network was proposed. Two reconstruction stages were used to gradually restore the image accuracy from-coarse-to-fine. In the first stage, firstly, a Low-Resolution (LR) image was taken as the input through a Convolutional Neural Network (CNN), and a High-Resolution (HR) image was output with coarse precision; then, the coarse HR image was used as the input and a finer HR image was produced. In the second stage, the correlation of all positions between features was calculate by the self-attention module, and the global dependencies of features were captured to enhance texture details. Experimental results on the benchmark datasets show that, compared with the state-of-the-art deep neural networks based super-resolution algorithms, the proposed algorithm not only has the best visual effect, but also has the Peak Signal-to-Noise Ratio (PSNR) improved averagely by 0.1dB and 0.15dB on Set5 and BDSD100. It indicates that the network can enhance the global representation ability of features to reconstruct high quality images.

Key words: deep convolutional neural network, from-coarse-to-fine, self-attention, global dependency, super-resolution

摘要: 针对图像超分辨率重建中纹理细节等高频信息恢复的问题,提出一种基于自注意力网络的图像超分辨率重建方法。该网络框架利用两个重建阶段逐步地将图像的精确度从粗到细进行恢复。在第一阶段中,首先将低分辨率(LR)图像作为输入通过一个卷积神经网络(CNN),并输出一个粗精度的高分辨率(HR)图像;然后将粗精度图像作为输入并产生更加精细的高分辨率图像。在第二阶段中,使用自注意力模块计算特征之间所有位置的关联性,通过捕捉特征的全局依赖关系来提高纹理细节的恢复能力。在基准数据集上的实验结果表明,与现有基于深度神经网路的超分辨率重建算法相比,所提算法不仅图像视觉效果最好,而且在数据集Set5和BDSD100上的峰值信噪比(PSNR)平均提高了0.1 dB、0.15 dB,表明该网络可以通过增强特征的全局表达能力来重建出高质量图像。

关键词: 深度卷积神经网络, 从粗到细, 自注意力, 全局依赖关系, 超分辨率

CLC Number: