Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (12): 3528-3534.DOI: 10.11772/j.issn.1001-9081.2019050937

• Artificial intelligence • Previous Articles     Next Articles

Medical image fusion algorithm based on generative adversarial residual network

GAO Yuan, WU Fan, QIN Pinle, WANG Lifang   

  1. School of Big Data Science, North University of China, Taiyuan Shanxi 030051, China
  • Received:2019-06-03 Revised:2019-07-16 Online:2019-12-10 Published:2019-07-25
  • Contact: 吴帆

生成对抗残差网络的医学图像融合算法

高媛, 吴帆, 秦品乐, 王丽芳   

  1. 中北大学 大数据学院, 太原 030051
  • 作者简介:高媛(1972-),女,山西太原人,副教授,硕士,主要研究方向:图像处理、人工智能;吴帆(1995-),女,山西吕梁人,硕士研究生,主要研究方向:深度学习、图像融合;秦品乐(1978-),男,山西长治人,副教授,博士,主要研究方向:大数据、机器视觉、三维重建;王丽芳(1977-),女,山西长治人,副教授,博士,主要研究方向:机器视觉、大数据。

Abstract: In the traditional medical image fusion, it is necessary to manually set the fusion rules and parameters by using prior knowledge, which leads to the uncertainty of fusion effect and the lack of detail expression. In order to solve the problems, a Computed Tomography (CT)/Magnetic Resonance (MR) image fusion algorithm based on improved Generative Adversarial Network (GAN) was proposed. Firstly, the network structures of generator and discriminator were improved. In the design of generator network, residual block and fast connection were used to deepen the network structure, so as to better capture the deep image information. Then, the down-sampling layer of the traditional network was removed to reduce the information loss during image transmission, and the batch normalization was changed to the layer normalization to better retain the source image information, and the depth of the discriminator network was increased to improve the network performance. Finally, the CT image and the MR image were connected and input into the generator network to obtain the fused image, and the network parameters were continuously optimized through the loss function, and the model most suitable for medical image fusion was trained to generate the high-quality image. The experimental results show that, the proposed algorithm is superior to Discrete Wavelet Transformation (DWT) algorithm, NonSubsampled Contourlet Transform (NSCT) algorithm, Sparse Representation (SR) algorithm and Sparse Representation of classified image Patches (PSR) algorithm on Mutual Information (MI), Information Entropy (IE) and Structural SIMilarity (SSIM). The final fused image has rich texture and details. At the same time, the influence of human factors on the stability of the fusion effect is avoided.

Key words: medical image fusion, Generative Adversarial Network (GAN), Computed Tomography (CT)/ Magnetic Resonance (MRI), layer normalization, residual block

摘要: 针对传统医学图像融合中需要依靠先验知识手动设置融合规则和参数,导致融合效果存在不确定性、细节表现力不足的问题,提出了一种基于改进生成对抗网络(GAN)的脑部计算机断层扫描(CT)/磁共振(MR)图像融合算法。首先,对生成器和判别器两个部分的网络结构进行改进,在生成器网络的设计中采用残差块和快捷连接以加深网络结构,更好地捕获深层次的图像信息;然后,去掉常规网络中的下采样层,以减少图像传输过程中的信息损失,并将批量归一化改为层归一化,以更好地保留源图像信息,增加判别器网络的深度以提高网络性能;最后,连接CT图像和MR图像,将其输入到生成器网络中得到融合图像,通过损失函数不断优化网络参数,训练出最适合医学图像融合的模型来生成高质量的图像。实验结果表明,与当前表现优良的基于离散小波变换(DWT)算法、基于非下采样剪切波变换(NSCT)算法、基于稀疏表示(SR)算法和基于图像分类块稀疏表示(PSR)算法对比,所提算法在互信息(MI)、信息熵(IE)、结构相似性(SSIM)上均表现良好,最终的融合图像纹理和细节丰富,同时避免了人为因素对融合效果稳定性的影响。

关键词: 医学图像融合, 生成对抗网络, 计算机断层成像(CT)/磁共振(MR), 层归一化, 残差块

CLC Number: