Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (9): 3017-3025.DOI: 10.11772/j.issn.1001-9081.2024081188

• Multimedia computing and computer simulation • Previous Articles    

Medical image segmentation network with content-guided multi-angle feature fusion

Fang WANG, Jing HU(), Rui ZHANG, Wenting FAN   

  1. College of Computer Science and Technology,Taiyuan University of Science and Technology,Taiyuan Shanxi 030024,China
  • Received:2024-08-21 Revised:2024-10-14 Accepted:2024-10-21 Online:2024-11-07 Published:2025-09-10
  • Contact: Jing HU
  • About author:WANG Fang, born in 1989, M. S., lecturer. Her research interests include graphic and image processing, deep learning.
    ZHANG Rui, born in 1987, Ph. D., associate professor. His research interests include intelligent information processing.
    FAN Wenting, born in 1988, M. S., lecturer. Her research interests include natural language processing, computer vision.
  • Supported by:
    Natural Science Foundation of Shanxi Province(20210302123216)

内容引导下多角度特征融合医学图像分割网络

王芳, 胡静(), 张睿, 范文婷   

  1. 太原科技大学 计算机科学与技术学院,太原 030024
  • 通讯作者: 胡静
  • 作者简介:王芳(1989—),女,山西太原人,讲师,硕士,主要研究方向:图形图像处理、深度学习
    张睿(1987—),男,山西太原人,副教授,博士,CCF会员,主要研究方向:智能信息处理
    范文婷(1988—),女,山西朔州人,讲师,硕士,主要研究方向:自然语言处理、计算机视觉。
  • 基金资助:
    山西省自然科学基金资助项目(20210302123216)

Abstract:

In view of the lack of traditional image segmentation algorithms to guide Convolutional Neural Network (CNN) for segmentation in the current field of medical image segmentation, a medical image segmentation Network with Content-Guided Multi-Angle Feature Fusion (CGMAFF-Net) was proposed. Firstly, grayscale images and Otsu threshold segmentation images were used to generate lesion region guidance maps through a Transformer-based micro U-shaped feature extraction module, and Adaptive Combination Weighting (ACW) was used to weight them to the original medical images for initial guidance. Then, Residual Network (ResNet) was employed to extract downsampled features from the weighted medical images, and a Multi-Angle Feature Fusion (MAFF) module was used to fuse feature maps at 1/16 and 1/8 scales. Finally, Reverse Attention (RA) was applied to upsample and restore the feature map size gradually, so as to predict key lesion regions. Experimental results on CVC-ClinicDB, Kvasir-SEG, and ISIC 2018 datasets demonstrate that compared to the existing best-performing segmentation multiscale spatial reverse attention network MSRAformer, CGMAFF-Net increases the mean Intersection over Union (mIoU) by 0.97, 0.78, and 0.11 percentage points, respectively; compared to the classic network U-Net, CGMAFF-Net improves the mIoU by 2.66, 8.94, and 1.69 percentage points, respectively, fully verifying the effectiveness and advancement of CGMAFF-Net.

Key words: medical image segmentation, transfer learning, multi-angle feature fusion, adaptive combination weighting, Transformer

摘要:

针对当前医学图像分割领域缺乏使用传统图像分割算法引导卷积神经网络(CNN)进行分割的问题,提出内容引导下多角度特征融合医学图像分割网络(CGMAFF-Net)。首先,利用灰度图以及Otsu阈值分割图像通过基于Transformer的小微U型特征提取模块生成病变区域引导图,并使用自适应组合赋权(ACW)将它们赋权于原始医学图像以进行初始引导;其次,使用残差网络(ResNet)对赋权后的医学图像进行下采样特征提取,并使用多角度特征融合(MAFF)模块对1/16和1/8的特征图进行特征融合;最后,使用反向注意力(RA)上采样并逐步还原特征图的大小,从而实现对关键病变区域的预测。在CVC-ClinicDB、Kvasir-SEG和ISIC 2018数据集上的实验结果表明,与目前分割性能最好的多尺度空间反向注意力网络MSRAformer相比,CGMAFF-Net的平均交并比(mIoU)分别提升了0.97、0.78和0.11个百分点;与经典网络U-Net相比,CGMAFF-Net的mIoU则分别提升了2.66、8.94和1.69个百分点,充分验证了CGMAFF-Net的有效性与先进性。

关键词: 医学图像分割, 迁移学习, 多角度特征融合, 自适应组合赋权, Transformer

CLC Number: