《计算机应用》唯一官方网站 ›› 2025, Vol. 45 ›› Issue (12): 3978-3986.DOI: 10.11772/j.issn.1001-9081.2024111684

• 多媒体计算与计算机仿真 • 上一篇    下一篇

基于信息补偿的全景图像超分辨率重建网络

范宇, 陈纯毅, 胡小娟, 李延风, 于海洋, 张日培, 刘云彪   

  1. 长春理工大学 计算机科学技术学院,长春 130022
  • 收稿日期:2024-12-02 修回日期:2025-03-23 接受日期:2025-04-01 发布日期:2025-04-08 出版日期:2025-12-10
  • 通讯作者: 陈纯毅
  • 作者简介:范宇(1994—),女,内蒙古阿尔山人,博士研究生,主要研究方向:计算机视觉、图像处理
    陈纯毅(1981—),男,重庆人,教授,博士,主要研究方向:真实感三维图形绘制、计算机仿真
    胡小娟(1985—),女,山东淄博人,讲师,博士,主要研究方向:计算机仿真
    李延风(1985—),女,河北唐山人,副教授,博士,主要研究方向:机器学习、计算机视觉
    于海洋(1989—),男,吉林长春人,副教授,博士,主要研究方向:光传输仿真
    张日培(1992—),男,辽宁铁岭人,博士,主要研究方向:图像处理
    刘云彪(1994—),男,吉林榆树人,博士研究生,主要研究方向:计算机视觉、图像处理。
  • 基金资助:
    国家自然科学基金资助项目(U19A2063);国家自然科学基金资助项目(62305030);吉林省科技发展计划项目(20230201080GX);吉林省科技发展计划项目(20240602123RC);吉林省教育厅科学技术研究项目(JJKH20240945KJ)

Information compensation-based panoramic image super-resolution reconstruction network

Yu FAN, Chunyi CHEN, Xiaojuan HU, Yanfeng LI, Haiyang YU, Ripei ZHANG, Yunbiao LIU   

  1. School of Computer Science and Technology,Changchun University of Science and Technology,Changchun Jilin 130022,China
  • Received:2024-12-02 Revised:2025-03-23 Accepted:2025-04-01 Online:2025-04-08 Published:2025-12-10
  • Contact: Chunyi CHEN
  • About author:FAN Yu, born in 1994, Ph. D. candidate. Her research interests include computer vision, image processing.
    CHEN Chunyi, born in 1981, Ph. D., professor. His research interests include photorealistic 3D rendering, computer simulation.
    HU Xiaojuan, born in 1985, Ph. D., lecturer. Her research interests include computer simulation.
    LI Yanfeng, born in 1985, Ph. D., associate professor. Her research interests include machine learning, computer vision.
    YU Haiyang, born in 1989, Ph. D., associate professor. His research interests include optical transmission simulation.
    ZHANG Ripei, born in 1992, Ph. D. His research interests include image processing.
    LIU Yunbiao, born in 1994, Ph. D. candidate. His research interests include computer vision,image processing.
  • Supported by:
    National Natural Science Foundation of China(U19A2063);Jilin Provincial Development Program of Science and Technology(20230201080GX);Science and Technology Research Program of Education Department of Jilin Province(JJKH20240945KJ)

摘要:

全景图像因投影形式特殊,存在严重的几何扭曲。现有2D图像超分辨率网络未考虑全景图像的几何扭曲特性,因此并不适用于全景图像的超分辨重建。与2D超分辨网络不同,全景图像超分辨模型需要关注不同纬度区域的特征差异,而且需要解决对不同尺度特征捕获不足和上下文信息未充分学习等问题。针对上述问题,提出一种基于信息补偿的全景图像超分辨率重建网络(ICPSnet)。首先,根据全景图像的几何特性引入位置感知机制,通过计算每个像素在纬度方向上的位置权重增强模型对不同纬度区域的关注;其次,为了解决不同尺度特征捕获不足的问题,设计一种跨尺度协同注意力(CSCA)模块,该模块利用不同感受野的多核卷积注意力机制获取丰富的跨尺度特征;此外,设计信息补偿(IC)块,通过改进空洞空间金字塔池化(ASPP),增强网络的上下文信息学习能力,从而提高重建图像质量。在2种基准数据集ODI-SR和SUN360上的实验结果表明,在放大因子为4倍、8倍时,ICPSnet的加权球面均匀信噪比(WS-PSNR)比当前最先进的OSRT(Omnidirectional image Super-Resolution Transformer)分别提高了0.14 dB、0.64 dB和0.25 dB、0.26 dB。可见,相较于其他网络,ICPSnet在视觉效果上表现更加优异,它重建的图像能够更好地展现高纬度区域的纹理细节。

关键词: 全景图像, 超分辨, 信息补偿, 位置感知, 注意力机制

Abstract:

Panoramic images, due to their unique projection format, suffer from severe geometric distortions. The existing 2D image super-resolution networks fail to account for the geometric distortion characteristics of panoramic images, making them unsuitable for super-resolution reconstruction of such images. Unlike 2D super-resolution networks, panoramic image super-resolution models must focus on the feature differences across different latitude regions and address issues such as insufficient feature capture at different scales and insufficient learning of contextual information. To address the above issues, an Information Compensation-based Panoramic image Super-resolution reconstruction network (ICPSnet) was proposed. Firstly, based on the geometric characteristics of panoramic images, a position awareness mechanism was introduced to calculate the position weight of each pixel in the latitude direction, thereby enhancing the model’s attention to different latitude regions. Secondly, to address the insufficient feature extraction issue at diverse scales, a Cross-Scale Collaborative Attention (CSCA) module was designed, which utilized a multi-kernel convolutional attention mechanism of different receptive fields to obtain rich cross-scale features. Additionally, to improve quality of the reconstructed image, an Information Compensation (IC) block was designed to enhance the network’s ability to learn contextual information by improving the Atrous Spatial Pyramid Pooling (ASPP). Experimental results on two benchmark datasets, ODI-SR and SUN360, show that when the amplification factor is 4 and 8, ICPSnet improves the Weighted-to-Spherically-uniform Peak Signal-to-Noise Ratio (WS-PSNR) by 0.14 dB, 0.64 dB, and 0.25 dB, 0.26 dB, respectively, compared to current state-of-the-art OSRT (Omnidirectional image Super-Resolution Transformer). It can be seen that compared to other networks, ICPSnet has superior visual performance with reconstructed images better representing the texture details of high-latitude regions.

Key words: panoramic image, super-resolution, Information Compensation (IC), position awareness, attention mechanism

中图分类号: