Research on super-resolution reconstruction algorithm of medical images based on void convolution

doi:10.11772/j.issn.1001-9081.2023030381

Abstract

Abstract: Abstract: To solve the problems of blurred image details and insufficient utilization of global information in existing medical image super-resolution reconstruction, a medical image super-resolution reconstruction algorithm based on void convolution and improved hybrid attention mechanism is proposed. The method combines depth-separable convolution with the idea of cavity convolution to extract features at different scales of images using different size perceptual fields to enhance feature representation. The edge channel attention mechanism is introduced to fuse the edge information while extracting the high frequency features of the image to improve the reconstruction accuracy of the model. Considering the special characteristics of medical images, the L1 loss and perceptual loss function are mixed as the overall loss function in order to make the reconstructed image effect more consistent with human visual perception. The results show that the proposed model outperforms the comparison algorithms in terms of PSNR and SSIM indexes, enhances the effect and texture features of the image, and provides a more complete restoration of the overall image structure.

Key words: Keywords: super-resolution reconstruction, medical image, depthwise separable convolution, dilated convolution, attention mechanism

摘要： 摘要: 为解决现有医学图像超分辨率重建中存在的图像细节模糊、全局信息利用不充分等问题，提出一种基于空洞卷积与改进的混合注意力机制的医学图像超分辨率重建算法。该方法将深度可分离卷积与空洞卷积思想相结合，使用不同大小感受野对图像进行不同尺度的特征提取，增强特征表达能力。引入边缘通道注意力机制，在提取图像高频特征的同时融合边缘信息，提高模型的重建精度。考虑到医学图像的特殊性，为使重建后的图像效果更加符合人类视觉感观，混合L1损失与感知损失函数作为整体损失函数。并在所用数据集上与SRCNN、VDSR等传统超分算法进行对比实验，结果表明，所提模型在PSNR与SSIM指标上优于对比算法，增强了图像的效果与纹理特征，对图像整体结构还原更加完整。

关键词: 关键词: 超分辨率重建, 医学图像, 深度可分离卷积, 空洞卷积, 注意力机制

CLC Number:

中图分类号:TP391.1

李众王雅婧马巧梅. 基于空洞卷积的医学图像超分辨率重建算法研究[J]. 《计算机应用》唯一官方网站, 2023, 43(9): 2940-2947.

Figures/Tables 15

References 27

1	CHRISTENSEN-JEFFRIES K， COUTURE O， DAYTON P A， et al. Super-resolution ultrasound imaging［J］. Ultrasound in Medicine and Biology， 2020， 46（4）： 865-891. 10.1016/j.ultrasmedbio.2019.11.013
2	BÄTZ M， EICHENSEER A， SEILER J， et al. Hybrid super-resolution combining example-based single-image and interpolation-based multi-image reconstruction approaches［C］// Proceeding of the 2015 IEEE International Conference on Image Processing. Piscataway： IEEE， 2015： 58-62. 10.1109/icip.2015.7350759
3	李现国，冯欣欣，李建雄. 多尺度残差网络的单幅图像超分辨率重建［J］. 计算机工程与应用， 2021， 57（7）：215-221. 10.3778/j.issn.1002-8331.2001-0073
	LI X G， FENG X X， LI J X. Single image super-resolution reconstruction based on multi-scale residual network［J］. Computer Engineering and Applications， 2021， 57（7）： 215-221. 10.3778/j.issn.1002-8331.2001-0073
4	DONG C， LOY C C， HE K M，et al ． Image super-resolution using deep convolutional networks［J］．IEEE Transactions on Pattern Analysis and Machine Intelligence， 2016， 38 （2）： 295-307． 10.1109/tpami.2015.2439281
5	KIM J， LEE J K， LEE K M. Accurate image super-resolution using very deep convolutional networks［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE，2016：1646-1654. 10.1109/cvpr.2016.182
6	LIM B，SON S， KIM H，et al. Enhanced deep residual networks for single image super-resolution［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE，2017：1132-1440. 10.1109/cvprw.2017.151
7	CHOLLET F. Xception： deep learning with depthwise separable convolutions［C］∥ Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017：1800-1807. 10.1109/cvpr.2017.195
8	CHEN L C， PAPANDREOU G， KOKKINOS I， et al. Deep Lab： semantic image segmentation with deep convolutional nets， atrous convolution， and fully connected CRFs［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2018，40（4）：834-848. 10.1109/tpami.2017.2699184
9	胡雪影，郭海儒，朱蓉. 基于混合深度卷积网络的图像超分辨率重建［J］. 计算机应用， 2020， 40（7）：2069-2076. 10.11772/j.issn.1001-9081.2019122149
	HU X Y， GUO H R， ZHU R. Image super-resolution reconstruction based on hybrid deep convolutional network［J］. Journal of Computer Applications， 2020， 40（7）： 2069-2076. 10.11772/j.issn.1001-9081.2019122149
10	WOO S， PARK J， LEE J Y， et al. CBAM： convolutional block attention module［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11211. Cham： Springer，2018： 3-19.
11	张晔，刘蓉，刘明，等. 基于多通道注意力机制的图像超分辨率重建网络［J］. 计算机应用，2022，42（5）：1563-1569. 10.11772/j.issn.1001-9081.2021030498
	ZHANG Y， LIU R， LIU M， et al. Image super-resolution reconstruction network based on multi-channel attention mechanism［J］. Journal of Computer Applications， 2022， 42（5）： 1563-1569. 10.11772/j.issn.1001-9081.2021030498
12	ZHANG Y N， AN M Q. Deep learning-and transfer learning-based super resolution reconstruction from single medical image［J］. Journal of Healthcare Engineering， 2017， 2017：No.5859727. 10.1155/2017/5859727
13	WANG Y K， TENG Q Z， HE X H， et al. CT-image of rock samples super resolution using 3D convolutional neural network［J］. Computers and Geosciences， 2019， 133： No.104314. 10.1016/j.cageo.2019.104314
14	高媛，王晓晨，秦品乐，等. 基于深度可分离卷积和宽残差网络的医学影像超分辨率重建［J］. 计算机应用， 2019， 39（9）：2731-2737. 10.11772/j.issn.1001-9081.2019030413
	GAO Y， WANG X C， QIN P L， et al. Medical image super-resolution reconstruction based on deep separable convolution and wide residual network［J］. Journal of Computer Applications， 2019， 39（9）： 2731-2737. 10.11772/j.issn.1001-9081.2019030413
15	CHEN L H， YANG X M， JEON G， et al. A trusted medical image super-resolution method based on feedback adaptive weighted dense network［J］. Artificial Intelligence in Medicine，2020，106：No.101857. 10.1016/j.artmed.2020.101857
16	QIU D F， ZHENG L X， ZHU J Q， et al. Multiple improved residual networks for medical image super-resolution［J］. Future Generation Computer Systems， 2021，116： 200-208. 10.1016/j.future.2020.11.001
17	HE K M， ZHANG X Y， REN SH Q，et al.Deep residual learning for image recognition［C］// Proceeding of the 2016 IEEE Conference on Computer Vision and Pattern Recongnition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
18	ENGEL J， KOLTUN V， CREMERS D. Direct sparse odometry ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2018，40（3）：611-625. 10.1109/tpami.2017.2658577
19	CANNY J. A computational approach to edge detection［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 1986，PAMI-8（6）： 679-698. 10.1109/tpami.1986.4767851
20	ZHAO H， GALLO O， FROSIO I， et al. Loss functions for image restoration with neural networks［J］. IEEE Transactions on Computational Imaging， 2017， 3（1）： 47-57. 10.1109/tci.2016.2644865
21	SHEN H. Towards a mathematical understanding of the difficulty in learning with feedforward neural networks ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 811-820. 10.1109/cvpr.2018.00091
22	ZHANG R， ISOLA P， EFROS A A， et al. The unreasonable effectiveness of deep features as a perceptual metric［C］∥ Proceeding of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018：586-595. 10.1109/cvpr.2018.00068
23	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition ［EB/OL］. ［2023-01-17］. .
24	KEYS R. Cubic convolution interpolation for digital image processing［J］. IEEE Transactions on Acoustics， Speech， and Signal Processing， 1981， 29（6）： 1153-1160. 10.1109/tassp.1981.1163711
25	DONG C， LOY C C， TANG X. Accelerating the super-resolution convolutional neural network ［EB/OL］. ［2022-12-18］. . 10.1007/978-3-319-46475-6_25
26	ZHANG Y L， LI K P， LI K， et al. Image super-resolution using very deep residual channel attention networks［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11211. Cham： Springer， 2018： 286-301.
27	李俊珠，郑华，雷帅，等. 基于多尺度注意力残差网络的图像超分辨率重建［J］. 计算机系统应用， 2022， 31（5）：324-330.
	LI J Z， ZHENG H， LEI S， et al. Image super-resolution reconstruction based on multi-scale attention residual network［J］. Computer Systems and Applications， 2022， 31（5）：324-330.

模块	PSNR/dB	SSIM
DSC	29.02	0.796 3
DSC+CA	30.94	0.883 2
DSC+SAM	29.81	0.800 0
DSC+ECAM	31.02	0.879 5
DSC+ISAM	30.27	0.877 1
DSC+ECAM+ISAM	31.35	0.901 3

模块	PSNR/dB	SSIM
DSC	29.02	0.796 3
DSC+CA	30.94	0.883 2
DSC+SAM	29.81	0.800 0
DSC+ECAM	31.02	0.879 5
DSC+ISAM	30.27	0.877 1
DSC+ECAM+ISAM	31.35	0.901 3

测试集	算法	不同放大倍数下的PSNR/dB			不同放大倍数下的SSIM
测试集	算法	2倍	3倍	4倍	2倍	3倍	4倍
测试集A	Bicubic	24.88	22.97	22.15	0.789 5	0.705 0	0.609 8
	SRCNN	28.16	26.51	26.70	0.860 0	0.794 5	0.795 8
	FSRCNN	26.69	24.34	24.71	0.803 5	0.693 7	0.705 4
	VDSR	30.12	27.56	27.78	0.898 0	0.824 5	0.818 5
	RCAN	30.44	28.47	27.93	0.891 2	0.835 1	0.813 7
	文献［27］算法	30.07	29.10	27.62	0.887 1	0.821 4	0.815 4
	FAWDN	31.11	28.95	28.44	0.923 1	0.832 2	0.819 1
	MIRN	30.57	29.03	27.87	0.895 7	0.825 0	0.810 6
	本文算法	31.35	29.31	28.74	0.901 3	0.847 8	0.816 9
测试集B	Bicubic	30.15	27.41	24.93	0.789 5	0.808 8	0.719 2
	SRCNN	34.05	30.07	27.43	0.860 0	0.864 8	0.784 4
	FSRCNN	29.51	26.21	24.80	0.803 5	0.749 5	0.689 4
	VDSR	36.20	30.80	28.06	0.898 0	0.879 6	0.806 3
	RCAN	35.42	31.54	30.64	0.902 5	0.863 6	0.827 2
	文献［27］算法	35.92	33.21	30.01	0.910 0	0.891 0	0.829 5
	FAWDN	35.37	32.74	30.58	0.915 2	0.890 2	0.815 0
	MIRN	35.58	33.43	30.29	0.903 8	0.887 6	0.828 8
	本文算法	35.83	33.68	30.78	0.901 3	0.897 5	0.831 1

测试集	算法	不同放大倍数下的PSNR/dB			不同放大倍数下的SSIM
测试集	算法	2倍	3倍	4倍	2倍	3倍	4倍
测试集A	Bicubic	24.88	22.97	22.15	0.789 5	0.705 0	0.609 8
	SRCNN	28.16	26.51	26.70	0.860 0	0.794 5	0.795 8
	FSRCNN	26.69	24.34	24.71	0.803 5	0.693 7	0.705 4
	VDSR	30.12	27.56	27.78	0.898 0	0.824 5	0.818 5
	RCAN	30.44	28.47	27.93	0.891 2	0.835 1	0.813 7
	文献［27］算法	30.07	29.10	27.62	0.887 1	0.821 4	0.815 4
	FAWDN	31.11	28.95	28.44	0.923 1	0.832 2	0.819 1
	MIRN	30.57	29.03	27.87	0.895 7	0.825 0	0.810 6
	本文算法	31.35	29.31	28.74	0.901 3	0.847 8	0.816 9
测试集B	Bicubic	30.15	27.41	24.93	0.789 5	0.808 8	0.719 2
	SRCNN	34.05	30.07	27.43	0.860 0	0.864 8	0.784 4
	FSRCNN	29.51	26.21	24.80	0.803 5	0.749 5	0.689 4
	VDSR	36.20	30.80	28.06	0.898 0	0.879 6	0.806 3
	RCAN	35.42	31.54	30.64	0.902 5	0.863 6	0.827 2
	文献［27］算法	35.92	33.21	30.01	0.910 0	0.891 0	0.829 5
	FAWDN	35.37	32.74	30.58	0.915 2	0.890 2	0.815 0
	MIRN	35.58	33.43	30.29	0.903 8	0.887 6	0.828 8
	本文算法	35.83	33.68	30.78	0.901 3	0.897 5	0.831 1

算法	运行时间/s	算法	运行时间/s
Bicubic	1.071	文献［27］算法	3.056
SRCNN	1.654	FAWDN	5.631
FSRCNN	1.241	MIRN	4.944
VDSR	2.062	本文算法	3.527
RCAN	2.453