Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (9): 2911-2918.DOI: 10.11772/j.issn.1001-9081.2023091332
• Multimedia computing and computer simulation • Previous Articles Next Articles
Xiyuan WANG1, Zhancheng ZHANG1(), Shaokang XU2, Baocheng ZHANG3, Xiaoqing LUO4, Fuyuan HU1
Received:
2023-09-27
Revised:
2023-12-28
Accepted:
2024-01-08
Online:
2024-01-31
Published:
2024-09-10
Contact:
Zhancheng ZHANG
About author:
WANG Xiyuan, born in 1999, M. S. candidate. His research interests include medical image, image registration.Supported by:
王熙源1, 张战成1(), 徐少康2, 张宝成3, 罗晓清4, 胡伏原1
通讯作者:
张战成
作者简介:
王熙源(1999—),男,江苏南京人,硕士研究生,CCF会员,主要研究方向:医学影像、图像配准基金资助:
CLC Number:
Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation[J]. Journal of Computer Applications, 2024, 44(9): 2911-2918.
王熙源, 张战成, 徐少康, 张宝成, 罗晓清, 胡伏原. 面向手术导航3D/2D配准的无监督跨域迁移网络[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2911-2918.
Add to citation manager EndNote|Ris|BibTeX
URL: https://www.joca.cn/EN/10.11772/j.issn.1001-9081.2023091332
序号 | 网络 | mTRE/mm | 训练样本 | |
---|---|---|---|---|
冠状面 | 矢状面 | |||
1 | ResNet(源域) | 5.42 | 4.75 | DRR |
2 | UCDTN(源域) | 1.32 | 1.06 | DRR |
3 | ResNet(跨域1) | 37.42 | 24.71 | DRR |
4 | UCDTN(跨域1) | 21.80 | 18.81 | DRR |
5 | ResNet(跨域2) | 14.33 | 12.57 | DRR,伪Xray |
6 | UCDTN(跨域2) | 9.71 | 8.39 | DRR,伪Xray |
7 | UCDTN(域迁移) | 3.18 | 2.13 | DRR,伪Xray |
Tab. 1 Effective analysis of different networks
序号 | 网络 | mTRE/mm | 训练样本 | |
---|---|---|---|---|
冠状面 | 矢状面 | |||
1 | ResNet(源域) | 5.42 | 4.75 | DRR |
2 | UCDTN(源域) | 1.32 | 1.06 | DRR |
3 | ResNet(跨域1) | 37.42 | 24.71 | DRR |
4 | UCDTN(跨域1) | 21.80 | 18.81 | DRR |
5 | ResNet(跨域2) | 14.33 | 12.57 | DRR,伪Xray |
6 | UCDTN(跨域2) | 9.71 | 8.39 | DRR,伪Xray |
7 | UCDTN(域迁移) | 3.18 | 2.13 | DRR,伪Xray |
感知深度d | ||
---|---|---|
1 | 8.21 | 0.023 |
2 | 6.08 | 0.016 |
3 | 4.62 | 0.007 |
4 | 5.33 | 0.009 |
Tab. 2 mTRE and MAE of for perceptual domains with different depths
感知深度d | ||
---|---|---|
1 | 8.21 | 0.023 |
2 | 6.08 | 0.016 |
3 | 4.62 | 0.007 |
4 | 5.33 | 0.009 |
实验序号 | 方法 | ||
---|---|---|---|
1 | 预训练( | 6.63 | 0.017 |
2 | 预训练( | 4.21 | 0.008 |
3 | 预训练( | 8.27 | 0.022 |
Tab. 3 Effectiveness comparison of auxiliary domain
实验序号 | 方法 | ||
---|---|---|---|
1 | 预训练( | 6.63 | 0.017 |
2 | 预训练( | 4.21 | 0.008 |
3 | 预训练( | 8.27 | 0.022 |
方法序号 | 损失函数 | ||
---|---|---|---|
1 | 19.84 | 0.117 | |
2 | 8.62 | 0.023 | |
3 | 4.43 | 0.018 | |
4 | 6.21 | 0.015 | |
5 | 2.74 | 0.005 |
Tab. 4 Performance of different losses
方法序号 | 损失函数 | ||
---|---|---|---|
1 | 19.84 | 0.117 | |
2 | 8.62 | 0.023 | |
3 | 4.43 | 0.018 | |
4 | 6.21 | 0.015 | |
5 | 2.74 | 0.005 |
方法 | 耗时/s | ||
---|---|---|---|
Opt-GO[ | 12.74 | 0.064 | 21.5 |
Opt-GC[ | 11.21 | 0.051 | 17.6 |
Opt-NGI[ | 12.29 | 0.062 | 23.1 |
姿态编码[ | 23.83 | 0.191 | 1.7 |
MLP[ | 32.17 | 0.244 | 3.1 |
姿态编码+Opt-GC | 8.73 | 0.027 | 8.6 |
UCDTN | 2.33 | 0.004 | 1.8 |
Tab. 5 Registration performance comparison among different methods
方法 | 耗时/s | ||
---|---|---|---|
Opt-GO[ | 12.74 | 0.064 | 21.5 |
Opt-GC[ | 11.21 | 0.051 | 17.6 |
Opt-NGI[ | 12.29 | 0.062 | 23.1 |
姿态编码[ | 23.83 | 0.191 | 1.7 |
MLP[ | 32.17 | 0.244 | 3.1 |
姿态编码+Opt-GC | 8.73 | 0.027 | 8.6 |
UCDTN | 2.33 | 0.004 | 1.8 |
1 | MARKELJ P, TOMAŽEVIČ D, LIKAR B, et al. A review of 3D/2D registration methods for image-guided interventions [J]. Medical Image Analysis, 2012, 16(3): 642-661. |
2 | BRENNER D J, HALL E J. Computed tomography — an increasing source of radiation exposure [J]. The New England Journal of Medicine, 2007, 357(22): 2277-2284. |
3 | MACKIE T R, KAPATOES J, RUCHALA K, et al. Image guidance for precise conformal radiotherapy [J]. International Journal of Radiation Oncology, Biology, Physics, 2003, 56(1): 89-105. |
4 | UNBERATH M, GAO C, HU Y, et al. The impact of machine learning on 2D/3D registration for image-guided interventions: a systematic review and perspective [J]. Frontiers in Robotics and AI, 2021, 8: 716007. |
5 | VARNAVAS A, CARRELL T, PENNEY G. Increasing the automation of a 2D-3D registration system [J]. IEEE Transactions on Medical Imaging, 2013, 32(2): 387-399. |
6 | OUADAH S, JACOBSON M, STAYMAN J W, et al. Correction of patient motion in cone-beam CT using 3D-2D registration [J]. Physics in Medicine and Biology, 2017, 62(23): 8813-8831. |
7 | OTAKE Y, WANG A S, STAYMAN J W, et al. Robust 3D-2D image registration: application to spine interventions and vertebral labeling in the presence of anatomical deformation [J]. Physics in Medicine and Biology, 2013, 58(23): 8535-8553. |
8 | GOUVEIA A R, METZ C, FREIRE L, et al. Registration-by-regression of coronary CTA and X-ray angiography [J]. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 2017, 5(3): 208-220. |
9 | C-R CHOU, FREDERICK B, MAGERAS G, et al. 2D/3D image registration using regression learning [J]. Computer Vision and Image Understanding, 2013, 117(9): 1095-1106. |
10 | MIAO S, WANG Z J, LIAO R. A CNN regression approach for real-time 2D/3D registration [J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1352-1363. |
11 | LIAO R, MIAO S, DE TOURNEMIRE P, et al. An artificial agent for robust image registration [J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2017, 31(1): 4168-4175. |
12 | MIAO S, PIAT S, FISCHER P, et al. Dilated FCN for multi-agent 2D/3D medical image registration [J]. Proceedings of the AAAI Conference on Artificial Intelligence,2018, 32(1): 4694-4701. |
13 | 徐少康, 张战成, 姚浩男, 等. 基于姿态编码器的2D/3D脊椎医学图像实时配准方法[J]. 计算机应用, 2023, 43(2): 589-594. |
XU S K, ZHANG Z C, YAO H N, et al. 2D/3D spine medical image real-time registration method based on pose encoder[J]. Journal of Computer Applications, 2023, 43(2): 589-594. | |
14 | BALAKRISHNAN G, ZHAO A, SABUNCU M R, et al. VoxelMorph: a learning framework for deformable medical image registration [J]. IEEE Transactions on Medical Imaging, 2019, 38(8): 1788-1800. |
15 | MOK T C W, CHUNG A C S. Fast symmetric diffeomorphic image registration with convolutional neural networks [C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 4643-4652. |
16 | 张家岗, 李达平, 杨晓东, 等. 基于深度卷积特征光流的形变医学图像配准算法 [J]. 计算机应用, 2020, 40(6): 1799-1805. |
ZHANG J G, LI D P, YANG X D, et al. Deformable medical image registration algorithm based on deep convolution feature optical flow [J]. Journal of Computer Applications, 2020, 40(6): 1799-1805. | |
17 | 王丽芳, 王雁丽, 蔺素珍, 等. 基于改进的Zernike矩的局部描述符与图割离散优化的非刚性多模态脑部图像配准[J]. 计算机应用, 2019, 39(2): 582-588. |
WANG L F, WANG Y L, LIN S Z, et al. Non-rigid multi-modal brain image registration by using improved Zernike moment based local descriptor and graph cuts discrete optimization [J]. Journal of Computer Applications, 2019, 39(2): 582-588. | |
18 | CHEN J, FREY E C, HE Y, et al. TransMorph: Transformer for unsupervised medical image registration [J]. Medical Image Analysis, 2022, 82: 102615. |
19 | QIN Z, YU H, WANG C, et al. Geometric Transformer for fast and robust point cloud registration [C]// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 11133-11142. |
20 | MOK T C W, CHUNG A C S. Affine medical image registration with coarse-to-fine vision transformer [C]// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 20803-20812. |
21 | MENG C, LI Y, XU Y, et al. A weakly supervised framework for 2D/3D vascular registration oriented to incomplete 2D blood vessels[J]. IEEE Transactions on Medical Robotics and Bionics, 2022, 4(2): 381-390. |
22 | GAO C, KILLEEN B D, HU Y, et al. Synthetic data accelerates the development of generalizable learning-based algorithms for X-ray image analysis [J]. Nature Machine Intelligence, 2023, 5(3): 294-308. |
23 | ZHENG S, YANG X, WANG Y, et al. Unsupervised cross-modality domain adaptation network for X-ray to CT registration[J]. IEEE Journal of Biomedical and Health Informatics, 2022, 26(6): 2637-2647. |
24 | LIAO H, LIN W-A, ZHANG J, et al. Multiview 2D/3D rigid registration via a point-of-interest network for tracking and triangulation [C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 12630-12639. |
25 | LI P, PEI Y, GUO Y, et al. Non-rigid 2D-3D registration using convolutional autoencoders [C]// Proceedings of the 2020 IEEE 17th International Symposium on Biomedical Imaging. Piscataway: IEEE, 2020: 700-704. |
26 | ZHANG Y. An unsupervised 2D-3D deformable registration network (2D3D-RegNet) for cone-beam CT estimation [J]. Physics in Medicine and Biology, 2021, 66(7): 074001. |
27 | HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778. |
28 | ZHU J-Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks [C]// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 2242-2251. |
29 | PASZKE A, GROSS S, MASSA F, et al. PyTorch: an imperative style, high-performance deep learning library [C]// Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2019: 8026-8037. |
30 | YONG H, HUANG J, HUA X, et al. Gradient centralization: a new optimization technique for deep neural networks [C]// Proceedings of the 16th European Conference on Computer Vision. Cham: Springer, 2020: 635-652. |
31 | LÖFFLER M T, SEKUBOYINA A, JACOB A, et al. A vertebral segmentation dataset with fracture grading [J]. Radiology: Artificial Intelligence, 2020, 2(4): e190138. |
32 | FITZPATRICK J M, WEST J B. The distribution of target registration error in rigid-body point-based registration [J]. IEEE Transactions on Medical Imaging, 2001, 20(9): 917-927. |
33 | YUAN L, CHEN Y, WANG T, et al. Tokens-to-Token ViT: training vision Transformers from scratch on ImageNet [C]// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 538-547. |
34 | DE SILVA T, UNERI A, KETCHA M D, et al. 3D-2D image registration for target localization in spine surgery: investigation of similarity metrics providing robustness to content mismatch [J]. Physics in Medicine and Biology, 2016, 61(8): 3009-3025. |
[1] | Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969. |
[2] | Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877. |
[3] | Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974. |
[4] | Liting LI, Bei HUA, Ruozhou HE, Kuang XU. Multivariate time series prediction model based on decoupled attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2732-2738. |
[5] | Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703. |
[6] | Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557. |
[7] | Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625. |
[8] | Fan YANG, Yao ZOU, Mingzhi ZHU, Zhenwei MA, Dawei CHENG, Changjun JIANG. Credit card fraud detection model based on graph attention Transformation neural network [J]. Journal of Computer Applications, 2024, 44(8): 2634-2642. |
[9] | Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650. |
[10] | Zheng WU, Zhiyou CHENG, Zhentian WANG, Chuanjian WANG, Sheng WANG, Hui XU. Deep learning-based classification of head movement amplitude during patient anaesthesia resuscitation [J]. Journal of Computer Applications, 2024, 44(7): 2258-2263. |
[11] | Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072. |
[12] | Zhi ZHANG, Xin LI, Naifu YE, Kaixi HU. DKP: defending against model stealing attacks based on dark knowledge protection [J]. Journal of Computer Applications, 2024, 44(7): 2080-2086. |
[13] | Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318. |
[14] | Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199. |
[15] | Xun SUN, Ruifeng FENG, Yanru CHEN. Monocular 3D object detection method integrating depth and instance segmentation [J]. Journal of Computer Applications, 2024, 44(7): 2208-2215. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||