Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (11): 3362-3367.DOI: 10.11772/j.issn.1001-9081.2020122065

• Multimedia computing and computer simulation • Previous Articles     Next Articles

Super-resolution and multi-view fusion based on magnetic resonance image inter-layer interpolation

Meng LI1,2,3, Pinle QIN1,2,3, Jianchao ZENG1,2,3(), Junbo LI1,2,3   

  1. 1.Shanxi Medical Imaging and Data Analysis Engineering Research Center (North University of China),Taiyuan Shanxi 030051,China
    2.School of Data Science and Technology,North University of China,Taiyuan Shanxi 030051,China
    3.Shanxi Medical Imaging Artificial Intelligence Engineering Technology Research Center (North University of China),Taiyuan Shanxi 030051,China
  • Received:2020-12-31 Revised:2021-04-23 Accepted:2021-05-18 Online:2021-04-23 Published:2021-11-10
  • Contact: Jianchao ZENG
  • About author:LI Meng,born in 1995,M. S. candidate. His research interests include machine learning,computer vision,medical image analysis
    QIN Pinle,born in 1978,Ph. D.,professor. His research interests include medical image analysis,big data,machine vision
    ZENG Jianchao,born in 1963,Ph. D.,professor. His research interests include maintenance decision and health management for complex systems
    LI Junbo,born in 1996,M. S. candidate. His research interests include machine learning,computer vision,medical image analysis.
  • Supported by:
    the Innovation Project of Graduate Education in Shanxi Province(2020SY381)


李萌1,2,3, 秦品乐1,2,3, 曾建潮1,2,3(), 李俊伯1,2,3   

  1. 1.山西省医学影像与数据分析工程研究中心(中北大学),太原 030051
    2.中北大学 大数据学院,太原 030051
    3.山西省医学影像人工智能工程技术研究中心(中北大学),太原 030051
  • 通讯作者: 曾建潮
  • 作者简介:李萌(1995—),男,山东济宁人,硕士研究生,主要研究方向:机器学习、计算机视觉、医学影像分析
    秦品乐(1978—),男,山西 长治人,教授,博士,CCF会员,主要研究方向:医学影像分析、大数据、机器视觉
    曾建潮(1963—),男,山西太原人,教授,博士生导师,博士, CCF会员,主要研究方向:复杂系统的维护决策和健康管理
    李俊伯(1996—),男,山西运城人,硕士研究生,主要研究方向:机器学习、计算机 视觉、医学影像分析。
  • 基金资助:


The high resolution in Magnetic Resonance (MR) image slices and low resolution between the slices lead to the lack of medical diagnostic significance of MR in the coronal and sagittal planes. In order to solve the problem, a medical image processing algorithm based on inter-layer interpolation and multi-view fusion network was proposed. Firstly, the inter-layer interpolation module was introduced to cut the MR volume data from three-dimensional data into two-dimensional images along the coronal and sagittal directions. Then, after the feature extraction on the coronal and sagittal planes, the weights were dynamically calculated by the spatial matrix filter and used for upsampling factor with any size to magnify the image. Finally, the results of the coronal and sagittal images obtained in the inter-layer interpolation module were aggregated into three-dimensional data and then cut into two-dimensional images along the axial direction. The obtained two-dimensional images were fused in pairs and corrected by the axial direction data. Experimental results show that, compared with other super-resolution algorithms, the proposed algorithm has improved the Peak Signal-to-Noise Ratio (PSNR) by about 1 dB in ×2, ×3, and ×4 scales. It can be seen that the proposed algorithm can effectively improve the quality of image reconstruction.

Key words: super-resolution, neural network, inter-layer interpolation, brain Magnetic Resonance (MR) image, multi-view fusion


针对磁共振(MR)图像切片内分辨率高而切片间分辨率低,导致MR在冠状面和矢状面上缺乏医学诊断意义的问题,提出了一种基于层间插值及多视角融合网络的医学图像处理算法。首先,引入了层间插值模块,用来将MR体数据沿冠状和矢状方向从三维数据切割成二维图像;然后,在分别对冠状面和矢状面进行特征提取之后,通过空间矩阵滤波器动态计算权重用于任意大小的上采样因子放大图像;最后,将冠状图和矢状图在层间插值模块中得到的结果聚合成三维数据后再次沿轴状方向切割成二维图像,对得到的二维图像两两进行融合并通过轴状方向数据进行修正。实验结果表明,所提算法相较于其他超分辨率算法在×2、×3、×4尺度下的峰值信噪比(PSNR)均有1 dB左右的提升,可见所提算法有效提升了图像的重建质量。

关键词: 超分辨率, 神经网络, 层间插值, 脑部磁共振影像, 多视角融合

CLC Number: