Sinogram inpainting for sparse-view cone-beam computed tomography image reconstruction based on residual encoder-decoder generative adversarial network

doi:10.11772/j.issn.1001-9081.2022050773

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (6): 1950-1957.DOI: 10.11772/j.issn.1001-9081.2022050773

Special Issue: 多媒体计算与计算机仿真

• Multimedia computing and computer simulation • Previous Articles Next Articles

Sinogram inpainting for sparse-view cone-beam computed tomography image reconstruction based on residual encoder-decoder generative adversarial network

Xin JIN¹^,², Yangchuan LIU², Yechen ZHU², Zijian ZHANG³, Xin GAO¹^,²()

^1.Division of Life Sciences and Medicine，School of Biomedical Engineering （Suzhou），University of Science and Technology of China，Suzhou Jiangsu 215009，China
^2.Suzhou Institute of Biomedical Engineering and Technology，Chinese Academy of Sciences，Suzhou Jiangsu 215163，China
^3.Department of Oncology，Xiangya Hospital，Central South University，Changsha Hunan 410008，China

Received:2022-05-31 Revised:2022-08-31 Accepted:2022-09-12 Online:2023-06-08 Published:2023-06-10
Contact: Xin GAO
About author:JIN Xin， born in 1996， Ph. D. candidate. His research interests include medical image processing.
LIU Yangchuan， born in 1987， Ph. D.， associate research fellow. His research interests include X-ray Computed Tomography （CT）.
ZHU Yechen， born in 1993， M. S.， assistant research fellow. His research interests include CT image processing and artifact reconstruction.
ZHANG Zijian， born in 1986， M. S.， technician in charge. His research interests include medical image processing.
Supported by:
National Natural Science Foundation of China(81871439);Key Research and Development Program of Shandong Province(2021SFGC0104);Key Research and Development Program of Jiangsu Province(BE2021663);Suzhou Science and Technology Program(SJC20211014)

基于残差编解码-生成对抗网络的正弦图修复的稀疏角度锥束CT图像重建

靳鑫¹^,², 刘仰川², 朱叶晨², 张子健³, 高欣¹^,²()

^1.中国科学技术大学生物医学工程学院(苏州) 生命科学与医学部, 江苏苏州 215009
^2.中国科学院苏州生物医学工程技术研究所, 江苏苏州 215163
^3.中南大学湘雅医院肿瘤科, 长沙 410008

通讯作者: 高欣
作者简介:靳鑫（1996—），男，河南南阳人，博士研究生，主要研究方向：医学图像处理
刘仰川（1987—），男，山东济宁人，副研究员，博士，主要研究方向：X射线断层成像（CT）
朱叶晨（1993—），男，江苏苏州人，助理研究员，硕士，主要研究方向：CT图像重建及伪影校正
张子健（1986—），男，湖南郴州人，主管技师，硕士，主要研究方向：医学图像处理
高欣（1975—），男，吉林吉林人，研究员，博士，主要研究方向：低剂量锥束CT、基于智能计算的精准医疗、手术导航及机器人Email：xingaosam@163.com。
基金资助:
国家自然科学基金资助项目(81871439);山东省重点研发计划项目(2021SFGC0104);江苏省重点研发计划项目(BE2021663);苏州市科技计划项目(SJC20211014)

Abstract

Abstract:

Sparse-view projection can reduce the scan does and scan time of Cone-Beam Computed Tomography （CBCT） effectively but brings a lot of streak artifacts to the reconstructed images. Sinogram inpainting can generate projection data for missing angles and improve the quality of reconstructed images. Based on the above， a Residual Encoder-Decoder Generative Adversarial Network （RED-GAN） was proposed for sinogram inpainting to reconstruct sparse-view CBCT images. In this network， the U-Net generator in Pix2pixGAN （Pix2pix Generative Adversarial Network） was replaced with the Residual Encoder-Decoder （RED） module. In addition， the conditional discriminator based on PatchGAN （Patch Generative Adversarial Network） was used to distinguish between the repaired sinograms from the real sinograms， thereby further improving the network performance. After the network training using real CBCT projection data， the proposed network was tested under 1/2， 1/3 and 1/4 sparse-view sampling conditions， and compared with linear interpolation method， Residual Encoder-Decoder Convolutional Neural Network （RED-CNN） and Pix2pixGAN. Experimental results indicate that the sinogram inpainting results of RED-GAN are better than those of the comparison methods under all the three conditions. Under the 1/4 sparse-view sampling condition， the proposed network has the most obvious advantages. In the sinogram domain， the proposed network has the Root Mean Square Error （RMSE） decreased by 7.2%， Peak Signal-to-Noise Ratio （PSNR） increased by 1.5% and Structural Similarity （SSIM） increased by 1.4%； in the reconstructed image domain， the proposed network has the RMSE decreased by 5.4%， PSNR increased by 1.6% and SSIM increased by 1.0%. It can be seen that RED-GAN is suitable for high-quality CBCT reconstruction and has potential application value in the field of fast low-dose CBCT scanning.

Key words: Cone-Beam Computed Tomography (CBCT), sparse-view, sinogram inpainting, Generative Adversarial Network (GAN), deep learning

摘要：

稀疏投影可有效缩短锥束CT（CBCT）扫描剂量和扫描时间，但会导致重建图像中出现大量条状伪影。正弦图修复可以生成缺失角度的投影数据，并提高重建图像质量。基于这些，提出了一种用于稀疏角度CBCT重建的正弦图修复的残差编解码-生成对抗网络（RED-GAN）。该网络利用残差编解码结构（RED）模块替换Pix2pixGAN（Pix2pix Generative Adversarial Network）中的U-Net生成器，并利用基于PatchGAN（Patch Generative Adversarial Network）的条件判别器鉴别修复后的正弦图和真实正弦图，从而进一步提升网络性能。利用真实CBCT投影数据进行网络训练后，分别在1/2、1/3、1/4稀疏采样条件下测试所提网络，并把RED-GAN与线性插值法、残差编解码-卷积神经网络（RED-CNN）和Pix2pixGAN对比。实验结果表明，RED-GAN的正弦图修复结果在3种条件下均优于对比方法，并在1/4稀疏采样条件下所提网络的优势最为明显。在正弦图域中，RED-GAN的均方根误差（RMSE）下降了7.2%，峰值信噪比（PSNR）上升了1.5%，结构相似性（SSIM）上升了1.4%；在重建图像域中，RMSE下降了5.4%，PSNR上升了1.6%，SSIM上升了1.0%。可见，RED-GAN适用于高质量的稀疏角度CBCT重建，在快速低剂量CBCT扫描领域具有潜在的应用价值。

关键词: 锥束CT, 稀疏角度, 正弦图修复, 生成对抗网络, 深度学习

CLC Number:

TP391.4

Xin JIN, Yangchuan LIU, Yechen ZHU, Zijian ZHANG, Xin GAO. Sinogram inpainting for sparse-view cone-beam computed tomography image reconstruction based on residual encoder-decoder generative adversarial network[J]. Journal of Computer Applications, 2023, 43(6): 1950-1957.

靳鑫, 刘仰川, 朱叶晨, 张子健, 高欣. 基于残差编解码-生成对抗网络的正弦图修复的稀疏角度锥束CT图像重建[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1950-1957.

Figures/Tables 10

References 21

1	MOHAN K A， VENKATAKRISHNAN S V， GIBBS J W， et al. TIMBIR： a method for time-space reconstruction from interlaced views ［J］. IEEE Transactions on Computational Imaging， 2015， 1（2）： 96-111. 10.1109/tci.2015.2431913
2	KALKE M， SILTANEN S. Sinogram interpolation method for sparse-angle tomography ［J］. Applied Mathematics， 2014， 5（3）：423-441.
3	KIM H G， YOO H. Image enhancement for computed tomography using directional interpolation for sparsely-sampled sinogram［J］. Optik， 2018， 166： 227-235. 10.1016/j.ijleo.2018.03.139
4	KÖSTLER H， PRÜMMER M， RÜDE U， et al. Adaptive variational sinogram interpolation of sparsely sampled CT data ［C］// Proceedings of the 18th International Conference on Pattern Recognition. Piscataway： IEEE， 2006， 3： 778-781. 10.1109/icpr.2006.225
5	HERRAIZ J L， ESPANA S， VICENTE E， et al. Frequency selective signal extrapolation for compensation of missing data in sinograms［C］// Proceedings of the 2008 IEEE Nuclear Science Symposium/ Medical Imaging Conference/ 16th International Workshop on Room-Temperature Semiconductor X- and Gamma-ray Detectors’ Conference Record. Piscataway： IEEE， 2008： 4299-4302. 10.1109/nssmic.2008.4774234
6	李昆鹏，张鹏程，上官宏，等. 基于卷积神经网络的时频域CT重建算法［J］. 计算机应用， 2022， 42（4）： 1308-1316. 10.11772/j.issn.1001-9081.2021050876
	LI K P， ZHANG P C， SHANGGUAN H， et al. Time-frequency domain CT reconstruction algorithm based on convolutional neural network ［J］. Journal of Computer Applications， 2022， 42（4）：1308-1316. 10.11772/j.issn.1001-9081.2021050876
7	LEE H， LEE J， KIM H， et al. Deep-neural-network-based sinogram synthesis for sparse-view CT image reconstruction［J］. IEEE Transactions on Radiation and Plasma Medical Sciences， 2019， 3（2）： 109-119. 10.1109/trpms.2018.2867611
8	HAN Y， YE J C. Framing U-Net via deep convolutional framelets： application to sparse-view CT ［J］. IEEE Transactions on Medical Imaging， 2018， 37（6）： 1418-1429. 10.1109/tmi.2018.2823768
9	张艳娇，乔志伟. 基于多残差UNet的CT图像高精度稀疏重建［J］. 计算机应用， 2021， 41（10）：2964-2969. 10.11772/j.issn.1001-9081.2020121985
	ZHANG Y J， QIAO Z W. High-precision sparse reconstruction of CT images based on multiply residual UNet［J］. Journal of Computer Applications， 2021， 41（10）：2964-2969. 10.11772/j.issn.1001-9081.2020121985
10	BERTRAM M， WIEGERT J， SCHAFER D， et al. Directional view interpolation for compensation of sparse angular sampling in cone-beam CT ［J］. IEEE Transactions on Medical Imaging， 2009， 28（7）： 1011-1022. 10.1109/tmi.2008.2011550
11	ZHANG H， SONKE J J. Directional sinogram interpolation for sparse angular acquisition in cone-beam computed tomography［J］. Journal of X-Ray Science and Technology， 2013， 21（4）： 481-496. 10.3233/xst-130401
12	ODENA A， DUMOULIN V， OLAH C. Deconvolution and checkerboard artifacts ［J/OL］. Distill， 2016 ［2022-10-27］..
13	CHEN H， ZHANG Y， KALRA M K， et al. Low-dose CT with a residual encoder-decoder convolutional neural network ［J］. IEEE Transactions on Medical Imaging， 2017， 36（12）： 2524-2535. 10.1109/tmi.2017.2715284
14	JIANG Z R， CHEN Y X， ZHANG Y W， et al. Augmentation of CBCT reconstructed from under-sampled projections using deep learning ［J］. IEEE Transactions on Medical Imaging， 2019， 38（11）： 2705-2715. 10.1109/tmi.2019.2912791
15	LIAO H F， HUO Z M， SEHNERT W J， et al. Adversarial sparse-view CBCT artifact reduction［C］// Proceedings of the 2018 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS 11070. Cham： Springer， 2018： 154-162.
16	MAO X D， LI Q， XIE H R， et al. Least squares generative adversarial networks［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2813-2821. 10.1109/iccv.2017.304
17	YI X， BABYN P. Sharpness-aware low-dose CT denoising using conditional generative adversarial network ［J］. Journal of Digital Imaging， 2018， 31（5）： 655-669. 10.1007/s10278-018-0056-0
18	DAI X B， BAI J N， LIU T L， et al. Limited-view cone-beam CT reconstruction based on an adversarial autoencoder network with joint loss ［J］. IEEE Access， 2019， 7： 7104-7116. 10.1109/access.2018.2890135
19	PATHAK D， KRÄHENBÜHL P， DONAHUE J， et al. Context encoders： feature learning by inpainting ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 2536-2544. 10.1109/cvpr.2016.278
20	ISOLA P， ZHU J Y， ZHOU T H， et al. Image-to-image translation with conditional adversarial networks ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 5967-5976. 10.1109/cvpr.2017.632
21	DER SARKISSIAN H， LUCKA F， van EIJNATTEN M， et al. A cone-beam X-ray computed tomography data collection designed for machine learning ［J］. Scientific Data， 2019， 6： No.215. 10.1038/s41597-019-0235-y

稀疏采样因子η	真实数据比例	修复方法	RMSE	PSNR/dB	SSIM
4	1/4	LI	111.6	44.8	0.795 8
		RED-CNN	103.2	45.3	0.814 8
		Pix2pixGAN	81.4	47.3	0.837 6
		RED-GAN	75.5	48.0	0.8492
3	1/3	LI	95.6	46.2	0.814 6
		RED-CNN	85.8	47.0	0.833 6
		Pix2pixGAN	84.4	47.0	0.821 0
		RED-GAN	84.3	47.2	0.8360
2	1/2	LI	81.0	48.0	0.838 2
		RED-CNN	69.3	49.2	0.859 9
		Pix2pixGAN	79.0	48.1	0.837 4
		RED-GAN	68.5	49.3	0.8633

稀疏采样因子η	真实数据比例	修复方法	RMSE	PSNR/dB	SSIM
4	1/4	LI	111.6	44.8	0.795 8
		RED-CNN	103.2	45.3	0.814 8
		Pix2pixGAN	81.4	47.3	0.837 6
		RED-GAN	75.5	48.0	0.8492
3	1/3	LI	95.6	46.2	0.814 6
		RED-CNN	85.8	47.0	0.833 6
		Pix2pixGAN	84.4	47.0	0.821 0
		RED-GAN	84.3	47.2	0.8360
2	1/2	LI	81.0	48.0	0.838 2
		RED-CNN	69.3	49.2	0.859 9
		Pix2pixGAN	79.0	48.1	0.837 4
		RED-GAN	68.5	49.3	0.8633

稀疏采样因子η	真实数据比例	修复方法	RMSE	PSNR/dB	SSIM
4	1/4	LI	0.102 0	25.9	0.846 2
		RED-CNN	0.004 8	28.9	0.884 2
		Pix2pixGAN	0.003 7	31.0	0.902 0
		RED-GAN	0.0035	31.5	0.9108
3	1/3	LI	0.004 3	30.3	0.903 2
		RED-CNN	0.003 8	31.4	0.919 2
		Pix2pixGAN	0.003 8	31.4	0.910 2
		RED-GAN	0.0037	31.5	0.9204
2	1/2	LI	0.003 4	31.8	0.922 3
		RED-CNN	0.002 8	33.6	0.941 0
		Pix2pixGAN	0.003 1	32.7	0.927 3
		RED-GAN	0.0027	33.7	0.9420

稀疏采样因子η	真实数据比例	修复方法	RMSE	PSNR/dB	SSIM
4	1/4	LI	0.102 0	25.9	0.846 2
		RED-CNN	0.004 8	28.9	0.884 2
		Pix2pixGAN	0.003 7	31.0	0.902 0
		RED-GAN	0.0035	31.5	0.9108
3	1/3	LI	0.004 3	30.3	0.903 2
		RED-CNN	0.003 8	31.4	0.919 2
		Pix2pixGAN	0.003 8	31.4	0.910 2
		RED-GAN	0.0037	31.5	0.9204
2	1/2	LI	0.003 4	31.8	0.922 3
		RED-CNN	0.002 8	33.6	0.941 0
		Pix2pixGAN	0.003 1	32.7	0.927 3
		RED-GAN	0.0027	33.7	0.9420

[1]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.
[2]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[3]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[4]	Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918.
[5]	Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703.
[6]	Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557.
[7]	Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625.
[8]	Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650.
[9]	Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318.
[10]	Li LIU, Haijin HOU, Anhong WANG, Tao ZHANG. Generative data hiding algorithm based on multi-scale attention [J]. Journal of Computer Applications, 2024, 44(7): 2102-2109.
[11]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.
[12]	Xun SUN, Ruifeng FENG, Yanru CHEN. Monocular 3D object detection method integrating depth and instance segmentation [J]. Journal of Computer Applications, 2024, 44(7): 2208-2215.
[13]	Zheng WU, Zhiyou CHENG, Zhentian WANG, Chuanjian WANG, Sheng WANG, Hui XU. Deep learning-based classification of head movement amplitude during patient anaesthesia resuscitation [J]. Journal of Computer Applications, 2024, 44(7): 2258-2263.
[14]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[15]	Zhi ZHANG, Xin LI, Naifu YE, Kaixi HU. DKP： defending against model stealing attacks based on dark knowledge protection [J]. Journal of Computer Applications, 2024, 44(7): 2080-2086.

Sinogram inpainting for sparse-view cone-beam computed tomography image reconstruction based on residual encoder-decoder generative adversarial network

基于残差编解码-生成对抗网络的正弦图修复的稀疏角度锥束CT图像重建

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 21

Related Articles 15

Recommended Articles

Metrics