《计算机应用》唯一官方网站 ›› 2022, Vol. 42 ›› Issue (6): 1941-1949.DOI: 10.11772/j.issn.1001-9081.2021040594

• 多媒体计算与计算机仿真 • 上一篇    

基于形状自适应非局部回归和非局部梯度正则的深度图像超分辨

张莹莹1, 任超2, 朱策1()   

  1. 1.电子科技大学 信息与通信工程学院,成都 611731
    2.四川大学 电子信息学院,成都 610065
  • 收稿日期:2021-04-15 修回日期:2021-05-27 接受日期:2021-05-27 发布日期:2022-06-22 出版日期:2022-06-10
  • 通讯作者: 朱策
  • 作者简介:张莹莹(1989—),女,山东泰安人,博士研究生,主要研究方向:图像超分辨率
    任超(1988—),男,四川南充人,副研究员,博士,主要研究方向:图像处理
  • 基金资助:
    国家自然科学基金资助项目(62020106011)

Depth image super-resolution based on shape-adaptive non-local regression and non-local gradient regularization

Yingying ZHANG1, Chao REN2, Ce ZHU1()   

  1. 1.School of Information and Communication Engineering,University of Electronic Science and Technology of China,Chengdu Sichuan 611731,China
    2.College of Electronics and Informatiom Engineering,Sichuan University,Chengdu Sichuan 610065,China
  • Received:2021-04-15 Revised:2021-05-27 Accepted:2021-05-27 Online:2022-06-22 Published:2022-06-10
  • Contact: Ce ZHU
  • About author:ZHANG Yingying,born in 1989,Ph. D. candidate. Her research interests include image super-resolution.
    REN Chao,born in 1988,Ph. D.,associate research fellow. His research interests include image prosessing.
  • Supported by:
    National Natural Science Foundation of China(62020106011)

摘要:

针对深度图像分辨率低、深度不连续性模糊问题,提出一种基于形状自适应非局部回归和非局部梯度正则的深度图像超分辨方法。为了探究深度图像非局部相似块之间的相关性,提出了形状自适应的非局部回归。该方法对每个像素点提取其形状自适应块,并根据形状自适应块构建目标像素的相似像素组;然后针对相似像素组中的每个像素,结合同场景的高分辨率彩色图像获得非局部权重,从而构建非局部回归先验。为了保持深度图像的边缘信息,对图像梯度的非局部性进行探究。不同于总变分(TV)正则化对所有像素点梯度的零均值拉普拉斯分布假设,该方法利用深度图像梯度的非局部相似性,用非局部块估计特定像素点的梯度均值,并用学习到的均值来拟合各像素点的梯度分布。实验结果表明,相较于基于边缘不一致性评价模型(EIEM),所提方法在Middlebury数据集上的2倍和4倍上采样率的平均绝对值差(MAD)分别下降了41.1%和40.8%。

关键词: 深度图像, 超分辨, 形状自适应, 非局部自相似, 非局部梯度

Abstract:

To deal with the low resolution of depth images and blurring depth discontinuities, a depth image super-resolution method based on shape-adaptive non-local regression and non-local gradient regularization was proposed. To explore the correlation between non-local similar patches of depth image, a shape-adaptive non-local regression method was proposed. The shape-adaptive self-similarity patch was extracted for each pixel, and a similar pixel group for the target pixel was constructed according to its shape-adaptive patch. Then for each pixel in the similar pixel group, a non-local weight was obtained with the assistant of the high-resolution color image of the same scene, thereby constructing the non-local regression prior. To maintain the edge information of the depth image, the non-locality of the gradient of the depth image was explored. Different from the Total Variation (TV) regularization which assumed that all pixels obeyed Laplacian distribution with zero mean value, through non-local similarity of the depth image, the gradient mean value of specific pixel was estimated by non-local patches, and the gradient distribution of each pixel was fit by using the learned mean value. Experimental results show that compared with Edge Inconsistency Evaluation Model (EIEM) on Middlebury datasets, the proposed method decreases Mean Absolute Difference (MAD) of 41.1% and 40.8% respectively.

Key words: depth image, super-resolution, shape-adaptive, non-local self-similarity, non-local gradient

中图分类号: