CT three-dimensional reconstruction algorithm based on super-resolution network

doi:10.11772/j.issn.1001-9081.2021020219

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (2): 584-591.DOI: 10.11772/j.issn.1001-9081.2021020219

• Multimedia computing and computer simulation • Previous Articles Next Articles

CT three-dimensional reconstruction algorithm based on super-resolution network

Junbo LI¹^,²^,³, Pinle QIN¹^,²^,³, Jianchao ZENG¹^,²^,³(), Meng LI¹^,²^,³

^1.Shanxi Medical Imaging and Data Analysis Engineering Research Center （North University of China），Taiyuan Shanxi 030051，China
^2.College of Data Science，North University of China，Taiyuan Shanxi 030051，China
^3.Shanxi Medical Imaging Artificial Intelligence Engineering Technology Research Center （North University of China），Taiyuan Shanxi 030051，China

Received:2021-02-04 Revised:2021-05-11 Accepted:2021-05-13 Online:2022-02-11 Published:2022-02-10
Contact: Jianchao ZENG
About author:LI Junbo， born in 1996， M. S. candidate. His research interests include machine learning， computer vision， medical image analysis.
QIN Pinle， born in 1978， Ph. D.， professor. His research interests include medical image， big data， machine vision.
ZENG Jianchao， born in 1963， Ph. D.， professor. His research interests include medical image， maintenance decision of complex system.
LI Meng， born in 1995， M. S. candidate. His research interests include machine learning， computer vision， medical image analysis.
Supported by:
Construction Project of Engineering Technology Research Center of Shanxi Province(201805D121008)

基于超分辨率网络的CT三维重建算法

李俊伯¹^,²^,³, 秦品乐¹^,²^,³, 曾建潮¹^,²^,³(), 李萌¹^,²^,³

^1.山西省医学影像与数据分析工程研究中心(中北大学), 太原 030051
^2.中北大学大数据学院, 太原 030051
^3.山西省医学影像人工智能工程技术研究中心(中北大学), 太原 030051

通讯作者: 曾建潮
作者简介:李俊伯（1996—），男，山西运城人，硕士研究生，主要研究方向：机器学习、计算机视觉、医学影像分析；
秦品乐（1978—），男，山西长治人，教授，博士，CCF会员，主要研究方向：医学影像分析、大数据、机器视觉；
曾建潮（1963—），男，山西太原人，教授，博士，CCF会员，主要研究方向：医学影像、复杂系统的维护决策；
李萌（1995—），男，山东济宁人，硕士研究生，主要研究方向：机器学习、计算机视觉、医学影像分析。
基金资助:
山西省工程技术研究中心建设项目(201805D121008)

Abstract

Abstract:

Computed Tomography （CT） three-dimensional reconstruction technique improves the quality of three-dimensional model by upsampling volume data， and reduces the jagged edges， streak artifacts and discontinuous surface in the model， so as to improve the accuracy of disease diagnosis in clinical medicine. A CT three-dimensional reconstruction algorithm based on super-resolution network was proposed to solve the problem that the model after CT three-dimensional reconstruction remains unclear enough in the past. The network model is a Double Loss Refinement Network （DLRNET）， and the three-dimensional reconstruction of abdominal CT was performed by uniaxial super-resolution. The optimization learning module was introduced at the end of the network model， and besides the calculation of the loss between the baseline image and super-resolution image， the loss between the roughly reconstructed image in the network model and the baseline image was also calculated. In this way， with the force of optimization learning and double loss， the results closer to the baseline image were produced by the network. Then， spatial pyramid pooling and channel attention mechanism were introduced into the feature extraction module to learn the features of vascular tissues with different thickness degrees and scales. Finally， the upsampling method was used to dynamically generate the convolution kernel set， so that a single network model was able to complete the upsampling tasks with different scaling factors. Experimental results show that compared with Residual Channel Attention Network （RCAN）， the proposed network model improves the Peak Signal-to-Noise Ratio （PSNR） by 0.789 dB on average under 2， 3， and 4 scaling factors， showing that the network model effectively improves the quality of CT three-dimensional model， recovers the continuous detail features of vascular tissues to some extent， and has practicability.

Key words: deep learning, three-dimensional reconstruction, super-resolution, Computed Tomography (CT), optimization learning

摘要：

计算机断层扫描（CT）三维重建技术通过上采样体数据来提高三维模型质量，减轻模型中的锯齿状边缘、条纹状伪影和不连续表面等现象，从而提高临床医学中疾病诊断的准确率。针对以往CT三维重建后模型仍然不够清晰的问题，提出一种基于超分辨率网络的CT三维重建算法。网络模型为具有双重损失的优化学习纵轴超分辨率重建网络（DLRNet），通过单轴超分辨率进行腹部CT三维重建。网络末端引入优化学习模块，且除计算基准图与超分辨率图像的损失外，还计算网络内部粗略重建图像与基准图的损失，这样一来，优化学习与双重损失能使网络产生更接近于基准图的结果。随后在特征提取模块引入空间特征金字塔池化和通道注意力机制，加权细化学习了不同粗细以及规模不一的血管组织的特征。最后使用动态生成卷积核组的方法进行上采样使得单一网络模型可应对不同缩放因子的上采样任务。实验结果表明，相较于通道注意力的方法RCAN（Residual Channel Attention Network），所提网络模型在2、3、4倍缩放因子下的峰值信噪比（PSNR）平均提高0.789 dB。可见所提网络模型有效提升了CT三维模型的质量，一定程度上恢复了血管组织的连续细节特征，同时具备了实用性。

关键词: 深度学习, 三维重建, 超分辨率, 计算机断层扫描, 优化学习

CLC Number:

TP183

Junbo LI, Pinle QIN, Jianchao ZENG, Meng LI. CT three-dimensional reconstruction algorithm based on super-resolution network[J]. Journal of Computer Applications, 2022, 42(2): 584-591.

李俊伯, 秦品乐, 曾建潮, 李萌. 基于超分辨率网络的CT三维重建算法[J]. 《计算机应用》唯一官方网站, 2022, 42(2): 584-591.

Figures/Tables 13

References 23

1	KHAN U， YASIN A， ABID M， et al. A methodological review of 3D reconstruction techniques in tomographic imaging［J］. Journal of Medical Systems， 2018， 42（10）： No.190. 10.1007/s10916-018-1042-2
2	LV S Q， CHEN Y M， LI Z Y， et al. Application of time-frequency domain transform to three-dimensional interpolation of medical images［J］. Journal of Computational Biology， 2017， 24（11）： 1112-1124. 10.1089/cmb.2017.0038
3	AFSHAR P， SHAHROUDNEJAD A， MOHAMMADI A， et al. CARISI： convolutional autoencoder-based inter-slice interpolation of brain tumor volumetric images ［C］// Proceedings of the 25th IEEE International Conference on Image Processing. Piscataway： IEEE， 2018： 1458-1462. 10.1109/icip.2018.8451759
4	CHAO Z， KIM H J. Slice interpolation of medical images using enhanced fuzzy radial basis function neural networks［J］. Computers in Biology and Medicine， 2019， 110： 66-78. 10.1016/j.compbiomed.2019.05.013
5	郭钧锋，蔡元龙，王玉平.医学图像三维重建的插值方法研究［J］. CT理论与应用研究， 1994， 3（4）： 7-11.
	GUO J F， CAI Y L， WANG Y P. Investigation of interpolation methods for medical 3D reconstruction［J］. CT Theory and Applications， 1994， 3（4）： 7-11.
6	徐艳蕾，赵继印，李敏，等.基于顺序形态学的医学图像插值算法的研究［J］.电子学报， 2010， 38（5）： 1002-1007. 10.1007/978-3-642-13199-8_2
	XU Y L， ZHAO J Y， LI M， et al. The interpolation arithmetic study of medical image based on the order morphology［J］. Acta Electronica Sinica， 2010， 38（5）： 1002-1007. 10.1007/978-3-642-13199-8_2
7	LIU S Q， XU D G， ZHOU S K， et al. 3D anisotropic hybrid network： transferring convolutional features from 2D images to 3D anisotropic volumes ［C］// Proceedings of the 2018 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS11071. Cham： Springer， 2018： 851-858.
8	朱付平，田捷，林瑶，等.基于Level Set方法的医学图像分割［J］.软件学报， 2002， 13（9）： 1866-1872. 10.3969/j.issn.1007-0249.2003.06.017
	ZHU F P， TIAN J， LIN Y， et al. Medical image segmentation based on Level Set method［J］. Journal of Software， 2002， 13（9）： 1866-1872. 10.3969/j.issn.1007-0249.2003.06.017
9	GREVERA G J， UDUPA J K. Shape-based interpolation of multidimensional grey-level images［J］. IEEE Transactions on Medical Imaging， 1996， 15（6）： 881-892. 10.1109/42.544506
10	PENG C， LIN W A， LIAO H F， et al. Deep slice interpolation via marginal super-resolution， fusion and refinement［EB/OL］. （2019-08-15）［2020-12-22］. . 10.1109/cvpr42600.2020.00777
11	DONG C， LOY C C， HE K M， et al. Image super-resolution using deep convolutional networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2016， 38（2）： 295-307. 10.1109/tpami.2015.2439281
12	YANG J C， WRIGHT J， HUANG T S， et al. Image super-resolution via sparse representation［J］. IEEE Transactions on Image Processing， 2010， 19（11）： 2861-2873. 10.1109/tip.2010.2050625
13	WANG Z H， CHEN J， HOI S C H. Deep learning for image super-resolution： a survey［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021， 43（10）： 3365-3387. 10.1109/tpami.2020.2982166
14	SHI W Z， CABALLERO J， HUSZÁR F， et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1874-1883. 10.1109/cvpr.2016.207
15	LIM B， SON S， KIM H， et al. Enhanced deep residual networks for single image super-resolution ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2017： 1132-1140. 10.1109/cvprw.2017.151
16	LEDIG C， THEIS L， HUSZÁR F， et al. Photo-realistic single image super-resolution using a generative adversarial network ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 105-114. 10.1109/cvpr.2017.19
17	高媛，刘志，秦品乐，等.基于深度残差生成对抗网络的医学影像超分辨率算法［J］.计算机应用， 2018， 38（9）： 2689-2695. 10.11772/j.issn.1001-9081.2018030574
	GAO Y， LIU Z， QIN P L， et al.Medical image super-resolution algorithm based on deep residual generation adversarial network［J］. Journal of Computer Applications， 2018， 38（9）： 2689-2695. 10.11772/j.issn.1001-9081.2018030574
18	ZHANG Y L， TIAN Y P， KONG Y， et al. Residual dense network for image super-resolution ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 2472-2481. 10.1109/cvpr.2018.00262
19	ZHANG Y L， LI K P， LI K， et al. Image super-resolution using very deep residual channel attention networks ［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS11211. Cham： Springer， 2018： 294-310. 10.1007/978-3-030-01234-2_18
20	HU X C， MU H Y， ZHANG X Y， et al. Meta-SR： a magnification-arbitrary network for super-resolution ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 1575-1584. 10.1109/cvpr.2019.00167
21	LIU J， ZHANG W J， TANG Y T， et al. Residual feature aggregation network for image super-resolution ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 2356-2365. 10.1109/cvpr42600.2020.00243
22	HU J， SHEN L， SUN G. Squeeze-and-excitation networks ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7132-7141. 10.1109/cvpr.2018.00745
23	KEYS R. Cubic convolution interpolation for digital image processing［J］. IEEE Transactions on Acoustics， Speech， and Signal Processing， 1981， 29（6）： 1153-1160. 10.1109/tassp.1981.1163711

卷积核量	补丁尺寸	SSIM	PSNR/dB	耗时/s
32	28×28	0.908 1	34.238	0.04
64	14×14	0.920 2	34.602	0.06
	28×28	0.928 3	35.962
	36×36	0.910 6	34.330
128	28×28	0.928 8	35.969	0.12
128	28×28	0.928 8	35.969	0.12

卷积核量	补丁尺寸	SSIM	PSNR/dB	耗时/s
32	28×28	0.908 1	34.238	0.04
64	14×14	0.920 2	34.602	0.06
	28×28	0.928 3	35.962
	36×36	0.910 6	34.330
128	28×28	0.928 8	35.969	0.12
128	28×28	0.928 8	35.969	0.12

算法	参数量	PSNR/dB			SSIM
算法	参数量	×2	×3	×4	×2	×3	×4
Bicubic	—	28.337	25.433	23.181	0.842 1	0.807 4	0.766 9
Antialias	—	28.590	25.540	23.514	0.861 0	0.829 7	0.791 1
EDSR+VMeta	4.32×10⁷	33.549	30.372	27.871	0.929 8	0.877 6	0.842 0
RDN+VMeta	2.24×10⁷	34.728	31.547	28.904	0.921 2	0.878 2	0.842 5
RCAN+VMeta	1.61×10⁷	35.146	31.685	29.330	0.924 6	0.880 1	0.843 7
DLRNet	3.43×10⁷	35.962	32.463	30.103	0.928 3	0.881 9	0.844 5

算法	参数量	PSNR/dB			SSIM
算法	参数量	×2	×3	×4	×2	×3	×4
Bicubic	—	28.337	25.433	23.181	0.842 1	0.807 4	0.766 9
Antialias	—	28.590	25.540	23.514	0.861 0	0.829 7	0.791 1
EDSR+VMeta	4.32×10⁷	33.549	30.372	27.871	0.929 8	0.877 6	0.842 0
RDN+VMeta	2.24×10⁷	34.728	31.547	28.904	0.921 2	0.878 2	0.842 5
RCAN+VMeta	1.61×10⁷	35.146	31.685	29.330	0.924 6	0.880 1	0.843 7
DLRNet	3.43×10⁷	35.962	32.463	30.103	0.928 3	0.881 9	0.844 5

算法	参数量	PSNR/dB			SSIM
算法	参数量	×2	×3	×4	×2	×3	×4
RDN+VM-Meta	2.24×10⁷	34.728	31.547	29.904	0.921 2	0.878 2	0.842 5
MASPP+VM-Meta	2.37×10⁷	35.181	31.890	30.243	0.925 3	0.880 0	0.843 8
CA+MASPP+VM-Meta	2.92×10⁷	35.529	32.211	30.528	0.926 1	0.880 5	0.844 0
CA+MASPP+VM-Meta	3.04×10⁷	35.643	32.305	30.590	0.926 6	0.880 8	0.844 0
CA+MASPP+VM-Meta+CRM	3.43×10⁷	35.962	32.563	30.779	0.928 3	0.881 9	0.844 5

CT three-dimensional reconstruction algorithm based on super-resolution network

基于超分辨率网络的CT三维重建算法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 23

Related Articles 15

Recommended Articles

Metrics

[1]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.
[2]	Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703.
[3]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[4]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[5]	Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918.
[6]	Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557.
[7]	Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625.
[8]	Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650.
[9]	Zheng WU, Zhiyou CHENG, Zhentian WANG, Chuanjian WANG, Sheng WANG, Hui XU. Deep learning-based classification of head movement amplitude during patient anaesthesia resuscitation [J]. Journal of Computer Applications, 2024, 44(7): 2258-2263.
[10]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[11]	Zhi ZHANG, Xin LI, Naifu YE, Kaixi HU. DKP： defending against model stealing attacks based on dark knowledge protection [J]. Journal of Computer Applications, 2024, 44(7): 2080-2086.
[12]	Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318.
[13]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.
[14]	Xun SUN, Ruifeng FENG, Yanru CHEN. Monocular 3D object detection method integrating depth and instance segmentation [J]. Journal of Computer Applications, 2024, 44(7): 2208-2215.
[15]	Xiaolu WANG, Wangfei QIAN. Gait recognition method based on two-branch convolutional network [J]. Journal of Computer Applications, 2024, 44(6): 1965-1971.