Abstract: The use of multimodal data can capture more comprehensive pathological features and is crucial for Alzheimer's Disease (AD) diagnosis. However, due to various limitations, case data combining multiple modalities is scarce, resulting in a large number of single-modal samples being ineffectively utilized. To address this issue, a cross-modal data generation method was proposed, and based on this method, an Alzheimer's disease diagnostic model GMF-Net that integrated multimodal features was designed. Firstly, missing Structural Connectivity Networks (SCNs) were generated from structural Magnetic Resonance Imaging (sMRI) using a dual-discriminator Cross-Modal Generation (CMG)module for the dataset completion. Secondly, a dual-path feature extraction approach was designed to capture features from both sMRI and SCN data. Finally, a Cross-Modal Attention Aggregation (CMAA) module was used to deeply fuse the extracted spatial texture features and structural connection features to further capture the complex correlations between modalities, and then achieved Alzheimer's disease classification. Experimental results demonstrate that the classification accuracy of NC/AD, NC/MCI, and MCI/AD reaches 94.69%, 82.05%, and 91.25%, respectively, on a publicly available dataset. The effectiveness of the multimodal data and the fusion mechanism is further validated by comparative experiments.

Key words: Alzheimer's Disease (AD), structural Magnetic Resonance Imaging (sMRI), Structural Connection Network (SCN), Generate Adversarial Network (GAN), multimodal fusion

摘要： 多模态数据的使用可以捕获更全面的病理特征，在阿尔兹海默症(AD)诊断中至关重要。然而受各种因素限制，兼具多种模态的病例数据较少，导致大量单模态样本不能被有效利用。针对这一问题，提出一种跨模态数据生成方法，并在此基础上设计基于多模态特征融合的阿尔兹海默症诊断模型GMF-Net。首先，通过双鉴别器的跨模态生成模块(CMG)，利用结构性磁共振成像(sMRI)生成缺失的结构连接网络(SCN)，用于数据集的补全；其次，针对sMRI与SCN的数据特性，设计双路径的特征提取分别获取二者特征；最后，采用跨模态注意力聚合(CMAA)模块，将提取到的空间纹理特征与结构连接特征进行深度融合，进一步捕捉模态间的复杂相关性，实现阿尔兹海默症的分类。实验结果表明，所提方法在公开数据集ADNI上NC/AD、NC/MCI和MCI/AD的分类准确率分别达到了94.69%、82.05%和91.25%，对比实验结果则进一步验证了多模态数据的使用及融合机制的有效性。

关键词: 阿尔兹海默症, 结构性磁共振成像, 结构连接网络, 生成对抗网络, 多模态融合