Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Parallel medical image registration model based on convolutional neural network and Transformer
Xin ZHAO, Xinjie LI, Jian XU, Buyun LIU, Xiang BI
Journal of Computer Applications    2024, 44 (12): 3915-3921.   DOI: 10.11772/j.issn.1001-9081.2023121828
Abstract232)   HTML5)    PDF (2554KB)(129)       Save

Medical image registration models aim to establish the correspondence of anatomical positions between images. The traditional image registration method obtains the deformation field through continuous iteration, which is time-consuming and has low accuracy. The deep neural networks not only achieve end-to-end generation of deformation fields, thereby speeding up the generation of deformation fields, but also further improve the accuracy of image registration. However, all of the current deep learning registration models use single Convolutional Neural Network (CNN) or Transformer architecture, and have the problems such as the inability to fully utilize the advantages of the combination of CNN and Transformer, resulting in insufficient registration accuracy, and the inability to maintain the original topology effectively after image registration. To solve these problems, a parallel medical image registration model based on CNN and Transformer — PPCTNet (Parallel Processing of CNN and Transformer Network) was proposed. Firstly, the model was constructed using Swin Transformer, which currently has the excellent registration accuracy, and LOCV-Net (Lightweight attentiOn-based ConVolutional Network), a very lightweight CNN. Then, the feature information extracted by Swin Transformer and LOCV-Net were fully integrated by designing a fusion strategy, so that the model not only had the local feature extraction capability of CNN and the long-distance dependency capability of Transformer, but also had the advantage of being lightweight. Finally, based on the brain Magnetic Resonance Imaging (MRI) dataset, PPCTNet was compared with 10 classical image alignment models. The results show that compared to the currently excellent registration model TransMorph (hybrid Transformer-ConvNet network for image registration), PPCTNet has the highest registration accuracy 0.5 percentage points higher, and the folding rate of deformation field 1.56 percentage points reduced, maintaining the topological structures of the registered images. Besides, compared with TransMorph, PPCTNet has the parameters reduced by 10.39×106, and the computational cost reduced by 278×109, which reflects the lightweight advantage of PPCTNet.

Table and Figures | Reference | Related Articles | Metrics