关键词: 直肠癌, 疗效预测, 注意力机制, 深度学习, 医学图像融合

Abstract: Traditional methods for predicting efficacy in patients with locally advanced rectal cancer (LARC) often rely on single-modality medical imaging information, clinical data, or pathological information. To address this limitation, a novel multimodal medical image fusion network framework based on attention mechanisms was proposed to predict efficacy. This framework comprises a feature extraction module, a multi-scale feature fusion module, a multi-modal medical image fusion module, and an attention-based decision module. Initially, multi-scale deep features are extracted using the feature extraction and multi-scale feature fusion modules. These deep features are then integrated through the multi-modal medical image fusion module. Finally, the attention-based decision module combines clinical features, radiomics features, and deep features to predict treatment efficacy. Experimental results demonstrate that the proposed method improves accuracy by 5 percentage points, recall by 6 percentage points, and AUC by 4 percentage points on the experimental dataset compared to single-modality methods, thereby validating the significant enhancement in rectal cancer treatment efficacy prediction achieved by this method.

Key words: rectal cancer, efficacy prediction, attention mechanism, deep learning, medical image fusion