Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (9): 2911-2918.DOI: 10.11772/j.issn.1001-9081.2023091332

• Multimedia computing and computer simulation • Previous Articles     Next Articles

Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation

Xiyuan WANG1, Zhancheng ZHANG1(), Shaokang XU2, Baocheng ZHANG3, Xiaoqing LUO4, Fuyuan HU1   

  1. 1.School of Electronic and Information Engineering,Suzhou University of Science and Technology,Suzhou Jiangsu 215009,China
    2.Shanghai Jirui Medical Technology Company Limited,Shanghai 201799,China
    3.Department of Orthopaedics,General Hospital of Central Theater Command,Wuhan Hubei 430070,China
    4.School of Artificial Intelligence and Computer Science,Jiangnan University,Wuxi Jiangsu 214122,China
  • Received:2023-09-27 Revised:2023-12-28 Accepted:2024-01-08 Online:2024-01-31 Published:2024-09-10
  • Contact: Zhancheng ZHANG
  • About author:WANG Xiyuan, born in 1999, M. S. candidate. His research interests include medical image, image registration.
    XU Shaokang, born in 1997, M. S. candidate. His research interests include image registration, medical imaging.
    ZHANG Baocheng, born in 1983, M. S., associate chief physician. His research interests include traumatology and orthopaedics, bone repair materials, artificial blood vessels.
    LUO Xiaoqing, born in 1980, Ph. D., professor. Her research interests include image fusion, medical imaging.
    HU Fuyuan, born in 1978, Ph. D., professor. His research interests include image processing, pattern recognition.
  • Supported by:
    National Natural Science Foundation of China(61772237);“Six Talent Peaks” Project in Jiangsu Province(XYDXX-030)

面向手术导航3D/2D配准的无监督跨域迁移网络

王熙源1, 张战成1(), 徐少康2, 张宝成3, 罗晓清4, 胡伏原1   

  1. 1.苏州科技大学 电子与信息工程学院, 江苏 苏州 215009
    2.上海极睿医疗科技有限公司, 上海 201799
    3.中部战区总医院 骨科, 武汉 430070
    4.江南大学 人工智能与计算机学院, 江苏 无锡 214122
  • 通讯作者: 张战成
  • 作者简介:王熙源(1999—),男,江苏南京人,硕士研究生,CCF会员,主要研究方向:医学影像、图像配准
    徐少康(1997—),男,江苏淮安人,硕士研究生,主要研究方向:图像配准、医学影像
    张宝成(1983—),男,山西平遥人,副主任医师,硕士,主要研究方向:创伤骨科、骨修复材料、人造血管
    罗晓清(1980—),女,江西景德镇人,教授,博士,主要研究方向:图像融合、医学影像
    胡伏原(1978—),男,湖南岳阳人,教授,博士,CCF会员,主要研究方向:图像处理、模式识别。
  • 基金资助:
    国家自然科学基金资助项目(61772237);江苏省“六大人才高峰”项目(XYDXX-030)

Abstract:

3D/2D registration is a key technique for intraoperative guidance. In existing deep learning based registration methods, image features were extracted through the network to regress the corresponding pose transformation parameters. This kind of method relies on real samples and their corresponding 3D labels for training, however, this part of expert-annotated medical data is scarce. In the alternative solution, the network was trained with Digital Reconstructed Radiography (DRR) images, which struggled to keep the original accuracy on Xray images due to the differences of image features across domains. For the above problems, an Unsupervised Cross-Domain Transfer Network (UCDTN) based on self-attention was designed. Without relying on Xray images and their 3D spatial labels as the training samples, the correspondence between the image features captured in the source domain and spatial transformations were migrated to the target domain. The public features were used to reduce the disparity of features between domains to minimize the negative impact of cross-domain. Experimental results show that the mTRE (mean Registration Target Error) of the result predicted by UCDTN is 2.66 mm, with a 70.61% reduction compared to the model without cross-domain transfer training, indicating the effectiveness of UCDTN in cross-domain registration tasks.

Key words: image registration, surgical navigation, cross-domain transfer, deep learning, self-attention

摘要:

3D/2D配准是手术导航的关键技术,现有基于深度学习的配准方法通过网络提取图像特征,继而回归出相应的姿态变换参数。此类方法依赖于真实的样本以及对应的3D标签用于训练,然而这部分专家标注的医疗数据十分稀缺。替代的方案用数字重建放射影像(DRR)图像训练网络,由于图像特征跨域的差异,在X射线图像上难以保持原有的配准精度。针对上述问题,设计基于自注意力的无监督跨域迁移网络(UCDTN),无须依赖X射线图像与其对应的3D空间标签作为训练样本,将源域所捕获的图像特征与空间变换间的对应关系迁移到目标域,借助公共特征减小域间特征的差距、降低跨域所带来的负面影响。实验结果表明,UCDTN预测结果的平均配准误差(mTRE)为2.66 mm;与未经跨域迁移训练的模型相比,mTRE指标降低了70.61%,验证了UCDTN在跨域配准任务上的有效性。

关键词: 图像配准, 手术导航, 跨域迁移, 深度学习, 自注意力

CLC Number: