Abstract: In view of the lack of traditional image segmentation algorithms to guide convolutional neural network (CNN) for segmentation in the current field of medical image segmentation, a medical image segmentation network with content-guided multi-angle feature fusion (CGMAFF-Net) was proposed. First, grayscale images and Otsu threshold segmentation images were used to generate lesion region guidance maps through a transformer-based micro U-shaped feature extraction module, and adaptive combination weighting (ACW) was used to weight them onto the original medical images for initial guidance. Then, the residual neural network (ResNet) was employed to extract downsampled features from the weighted medical images, and a multi-angle feature fusion (MAFF) module was used to fuse feature maps at 1/16 and 1/8 scales. Finally, reverse attention (RA) was applied to upsample and gradually restore the feature map size to predict key lesion regions. Experiments on the CVC-ClinicDB, Kvasir-SEG, and ISIC 2018 datasets demonstrated that, compared to the best-performing segmentation model MSRAformer, the mean intersection over union (mIoU) increased by 0.97, 0.78, and 0.11 percentage points, respectively. Compared to the classic network UNet, mIoU increased by 2.66, 8.94, and 1.69 percentage points, respectively, fully proving the effectiveness and advancement of CGMAFF-Net.

Key words: Keywords: medical image segmentation, transfer learning, multi-angle feature fusion, adaptive combination weighting, transformer

摘要： 针对当前医学图像分割领域缺乏使用传统图像分割算法引导卷积神经网络(Convolutional Neural Network，CNN)进行分割的问题，提出内容引导下多角度特征融合医学图像分割网络(Medical Image Segmentation Network with Content-Guided Multi-Angle Feature Fusion，CGMAFF-Net)。首先，利用灰度图以及Otsu阈值分割图像使用基于Transformer的小微U型特征提取模块生成病变区域引导图，并使用自适应组合赋权(Adaptive Combination Weighting，ACW)将其赋权于原始医学图像进行初始引导；其次，残差网络(Residual Neural Network，ResNet)对赋权后的医学图像进行下采样特征提取，同时使用多角度特征融合(Multi-Angle Feature Fusion，MAFF)模块对1/16和1/8的特征图进行特征融合；最后，使用反向注意(Reverse Attention, RA)上采样逐步还原特征图大小，实现对关键病变区域的预测。在CVC-ClinicDB、Kvasir-SEG和ISIC 2018数据集上的实验表明，与分割性能最好的MSRAformer相比，均交并比(mean intersection over union，mIoU)分别提升了0.97、0.78和0.11个百分点。与经典网络UNet相比，mIoU则分别提升了2.66、8.94和1.69个百分点，充分证明了CGMAFF-Net的有效性与先进性。

关键词: 关键词: 医学图像分割, 迁移学习, 多角度特征融合, 自适应组合赋权, Transformer