《计算机应用》唯一官方网站 ›› 2026, Vol. 46 ›› Issue (9): 2761-2768.DOI: 10.11772/j.issn.1001-9081.2025080959
• 人工智能 • 上一篇
收稿日期:2025-08-22
修回日期:2025-10-03
接受日期:2025-10-14
发布日期:2026-09-16
出版日期:2026-09-10
通讯作者:
刘赏
作者简介:刘赏(1977—),女,河北辛集人,教授,博士,主要研究方向:模式识别、数字图像处理基金资助:
Shang LIU(
), Zhaosen TANG, Hongyue LIU, Linfang DONG, Jin ZHOU
Received:2025-08-22
Revised:2025-10-03
Accepted:2025-10-14
Online:2026-09-16
Published:2026-09-10
Contact:
Shang LIU
About author:LIU Shang,born in 1977, Ph. D., professor. Her research interestsinclude pattern recognition, digital image processing.摘要:
现有研究在进行模态缺失场景下的多模态情感分析时,在缺失模态生成过程中常忽略模态间的相关性,导致恢复数据与原始数据的语义不一致,同时基于扩散模型的缺失模态生成会带来较大的计算开销。针对以上问题,提出一种共享语义条件下面向模态缺失的多模态情感分析模型(MM-SSC)。首先,设计共享潜空间映射模块,利用矢量量化变分自编码器(VQ-VAE)的共享潜空间有效捕捉多模态分布,确保模态间共享语义;其次,给出跨模态一致性约束方法,以学习每种模态潜空间内的互信息,促进模态间语义信息的完善,增强跨模态一致性;再次,设计缺失模态重建与对齐模块,以重建与细化缺失模态,同时降低重建计算开销;最后,引入多模态融合与预测模块,融合重建模态与可用模态,用于一致性的情感分析。实验结果显示,在固定模态缺失的情况下,相较于不完全多模态扩散情感识别(IMDer)模型,所提模型在CMU-MOSI数据集上的F1得分和七分类准确率(ACC7)指标分别平均提升了0.4与0.7个百分点,在CMU-MOSEI数据集上的F1得分和ACC7分别平均提升了0.9与0.3个百分点;在随机模态缺失的情况下,相比IMDer模型,所提模型在CMU-MOSI数据集上的F1得分和ACC7都平均提升了2.2个百分点,在CMU-MOSEI数据集上的F1得分和ACC7分别平均提升了0.8与0.4个百分点。MM-SSC能有效应对模态缺失场景下的多模态情感分析任务。
中图分类号:
刘赏, 汤兆森, 刘鸿月, 董林芳, 周金. 共享语义条件下面向模态缺失的多模态情感分析模型[J]. 计算机应用, 2026, 46(9): 2761-2768.
Shang LIU, Zhaosen TANG, Hongyue LIU, Linfang DONG, Jin ZHOU. Multimodal sentiment analysis model for missing modalities under shared semantic conditions[J]. Journal of Computer Applications, 2026, 46(9): 2761-2768.
| 模态组合 | DCCAE | MCTN | MMIN | GCNet | IMDer | MM-SSC | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | |
| 平均 | 64.8 | 24.4 | 69.9 | 31.3 | 71.4 | 31.6 | 72.8 | 32.1 | 75.3 | 35.5 | 75.7 | 36.2 |
| 76.5 | 28.3 | 79.2 | 41.0 | 83.8 | 41.6 | 83.6 | 42.3 | 84.7 | 44.8 | 85.7 | 46.9 | |
| 51.1 | 17.1 | 54.4 | 16.5 | 54.0 | 15.5 | 55.7 | 16.9 | 60.8 | 22.2 | 61.1 | 22.5 | |
| 42.1 | 16.9 | 54.5 | 16.5 | 51.5 | 15.5 | 54.5 | 16.6 | 62.2 | 22.0 | 62.3 | 22.5 | |
| 76.8 | 30.0 | 81.2 | 42.1 | 83.9 | 42.0 | 84.2 | 43.4 | 85.4 | 45.3 | 85.8 | 47.7 | |
| 77.0 | 30.2 | 81.0 | 43.2 | 84.0 | 42.3 | 84.4 | 43.4 | 85.3 | 45.0 | 85.7 | 47.1 | |
| 52.5 | 17.4 | 57.4 | 16.8 | 58.5 | 19.5 | 61.9 | 17.2 | 63.4 | 23.8 | 63.3 | 18.5 | |
| 77.4 | 31.2 | 81.5 | 43.4 | 84.4 | 44.8 | 85.1 | 44.9 | 85.6 | 45.3 | 86.1 | 48.1 | |
表1 CMU-MOSI数据集上固定模态缺失下的实验结果 (%)
Tab. 1 Experimental results on CMU-MOSI dataset with fixed modality missing
| 模态组合 | DCCAE | MCTN | MMIN | GCNet | IMDer | MM-SSC | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | |
| 平均 | 64.8 | 24.4 | 69.9 | 31.3 | 71.4 | 31.6 | 72.8 | 32.1 | 75.3 | 35.5 | 75.7 | 36.2 |
| 76.5 | 28.3 | 79.2 | 41.0 | 83.8 | 41.6 | 83.6 | 42.3 | 84.7 | 44.8 | 85.7 | 46.9 | |
| 51.1 | 17.1 | 54.4 | 16.5 | 54.0 | 15.5 | 55.7 | 16.9 | 60.8 | 22.2 | 61.1 | 22.5 | |
| 42.1 | 16.9 | 54.5 | 16.5 | 51.5 | 15.5 | 54.5 | 16.6 | 62.2 | 22.0 | 62.3 | 22.5 | |
| 76.8 | 30.0 | 81.2 | 42.1 | 83.9 | 42.0 | 84.2 | 43.4 | 85.4 | 45.3 | 85.8 | 47.7 | |
| 77.0 | 30.2 | 81.0 | 43.2 | 84.0 | 42.3 | 84.4 | 43.4 | 85.3 | 45.0 | 85.7 | 47.1 | |
| 52.5 | 17.4 | 57.4 | 16.8 | 58.5 | 19.5 | 61.9 | 17.2 | 63.4 | 23.8 | 63.3 | 18.5 | |
| 77.4 | 31.2 | 81.5 | 43.4 | 84.4 | 44.8 | 85.1 | 44.9 | 85.6 | 45.3 | 86.1 | 48.1 | |
| 模态组合 | DCCAE | MCTN | MMIN | GCNet | IMDer | MM-SSC | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | |
| 平均 | 70.2 | 44.6 | 72.5 | 46.8 | 73.5 | 47.1 | 73.7 | 47.1 | 75.3 | 48.5 | 76.2 | 48.8 |
| 79.5 | 47.0 | 82.8 | 50.2 | 82.4 | 51.4 | 83.2 | 51.2 | 84.5 | 52.5 | 85.4 | 53.5 | |
| 57.2 | 40.1 | 57.1 | 41.6 | 60.0 | 40.7 | 61.6 | 41.7 | 63.6 | 42.6 | 64.2 | 42.1 | |
| 53.8 | 40.9 | 54.5 | 41.4 | 59.5 | 40.4 | 60.3 | 41.1 | 60.6 | 41.7 | 62.2 | 41.7 | |
| 80.4 | 47.1 | 83.2 | 50.4 | 83.4 | 51.2 | 84.4 | 51.1 | 85.0 | 53.1 | 85.8 | 53.8 | |
| 80.0 | 47.4 | 83.3 | 50.7 | 83.3 | 52.0 | 84.4 | 51.3 | 85.1 | 53.1 | 85.5 | 53.7 | |
| 59.2 | 41.6 | 62.7 | 42.1 | 61.9 | 41.8 | 57.2 | 42.0 | 63.5 | 42.8 | 64.6 | 42.7 | |
| 81.2 | 48.2 | 84.2 | 51.2 | 84.2 | 52.4 | 85.1 | 51.5 | 85.1 | 53.4 | 86.0 | 53.8 | |
表2 CMU-MOSEI数据集上固定模态缺失下的实验结果 (%)
Tab. 2 Experimental results on CMU-MOSEI dataset with fixed modality missing
| 模态组合 | DCCAE | MCTN | MMIN | GCNet | IMDer | MM-SSC | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | F1 | ACC7 | |
| 平均 | 70.2 | 44.6 | 72.5 | 46.8 | 73.5 | 47.1 | 73.7 | 47.1 | 75.3 | 48.5 | 76.2 | 48.8 |
| 79.5 | 47.0 | 82.8 | 50.2 | 82.4 | 51.4 | 83.2 | 51.2 | 84.5 | 52.5 | 85.4 | 53.5 | |
| 57.2 | 40.1 | 57.1 | 41.6 | 60.0 | 40.7 | 61.6 | 41.7 | 63.6 | 42.6 | 64.2 | 42.1 | |
| 53.8 | 40.9 | 54.5 | 41.4 | 59.5 | 40.4 | 60.3 | 41.1 | 60.6 | 41.7 | 62.2 | 41.7 | |
| 80.4 | 47.1 | 83.2 | 50.4 | 83.4 | 51.2 | 84.4 | 51.1 | 85.0 | 53.1 | 85.8 | 53.8 | |
| 80.0 | 47.4 | 83.3 | 50.7 | 83.3 | 52.0 | 84.4 | 51.3 | 85.1 | 53.1 | 85.5 | 53.7 | |
| 59.2 | 41.6 | 62.7 | 42.1 | 61.9 | 41.8 | 57.2 | 42.0 | 63.5 | 42.8 | 64.6 | 42.7 | |
| 81.2 | 48.2 | 84.2 | 51.2 | 84.2 | 52.4 | 85.1 | 51.5 | 85.1 | 53.4 | 86.0 | 53.8 | |
| 缺失率 | DCCAE | MCTN | MMIN | GCNet | IMDer | MM-SSC | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | |
| 平均 | 66.3 | 25.2 | 70.1 | 34.8 | 73.1 | 35.8 | 75.1 | 36.7 | 79.3 | 40.5 | 81.5 | 42.7 |
| 0.0 | 77.4 | 31.2 | 81.5 | 43.4 | 84.4 | 44.8 | 85.1 | 44.9 | 85.6 | 45.3 | 86.1 | 48.1 |
| 0.1 | 74.7 | 28.1 | 78.5 | 39.8 | 81.8 | 41.2 | 82.3 | 42.1 | 84.8 | 44.8 | 85.7 | 47.5 |
| 0.2 | 71.9 | 27.6 | 75.7 | 38.5 | 79.1 | 38.9 | 79.5 | 40.0 | 83.4 | 44.3 | 84.1 | 46.4 |
| 0.3 | 66.7 | 25.8 | 71.2 | 35.5 | 76.2 | 36.9 | 77.2 | 38.2 | 81.0 | 42.5 | 83.0 | 46.1 |
| 0.4 | 62.8 | 24.2 | 67.6 | 32.9 | 71.6 | 34.9 | 74.4 | 36.6 | 78.5 | 39.7 | 80.1 | 41.0 |
| 0.5 | 61.3 | 23.0 | 64.8 | 31.2 | 66.5 | 32.2 | 69.8 | 39.9 | 75.9 | 37.9 | 79.5 | 39.8 |
| 0.6 | 58.5 | 20.9 | 62.5 | 29.7 | 64.0 | 29.1 | 66.7 | 29.8 | 74.0 | 35.8 | 77.6 | 38.0 |
| 0.7 | 57.4 | 20.6 | 59.0 | 27.5 | 61.0 | 28.4 | 65.4 | 28.1 | 71.2 | 33.4 | 75.5 | 34.6 |
表3 CMU-MOSI数据集上随机模态缺失下的实验结果
Tab. 3 Experimental results on CMU-MOSI dataset with random modality missing
| 缺失率 | DCCAE | MCTN | MMIN | GCNet | IMDer | MM-SSC | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | |
| 平均 | 66.3 | 25.2 | 70.1 | 34.8 | 73.1 | 35.8 | 75.1 | 36.7 | 79.3 | 40.5 | 81.5 | 42.7 |
| 0.0 | 77.4 | 31.2 | 81.5 | 43.4 | 84.4 | 44.8 | 85.1 | 44.9 | 85.6 | 45.3 | 86.1 | 48.1 |
| 0.1 | 74.7 | 28.1 | 78.5 | 39.8 | 81.8 | 41.2 | 82.3 | 42.1 | 84.8 | 44.8 | 85.7 | 47.5 |
| 0.2 | 71.9 | 27.6 | 75.7 | 38.5 | 79.1 | 38.9 | 79.5 | 40.0 | 83.4 | 44.3 | 84.1 | 46.4 |
| 0.3 | 66.7 | 25.8 | 71.2 | 35.5 | 76.2 | 36.9 | 77.2 | 38.2 | 81.0 | 42.5 | 83.0 | 46.1 |
| 0.4 | 62.8 | 24.2 | 67.6 | 32.9 | 71.6 | 34.9 | 74.4 | 36.6 | 78.5 | 39.7 | 80.1 | 41.0 |
| 0.5 | 61.3 | 23.0 | 64.8 | 31.2 | 66.5 | 32.2 | 69.8 | 39.9 | 75.9 | 37.9 | 79.5 | 39.8 |
| 0.6 | 58.5 | 20.9 | 62.5 | 29.7 | 64.0 | 29.1 | 66.7 | 29.8 | 74.0 | 35.8 | 77.6 | 38.0 |
| 0.7 | 57.4 | 20.6 | 59.0 | 27.5 | 61.0 | 28.4 | 65.4 | 28.1 | 71.2 | 33.4 | 75.5 | 34.6 |
| 缺失率 | DCCAE | MCTN | MMIN | GCNet | IMDer | MM-SSC | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | |
| 平均 | 71.2 | 45.0 | 76.1 | 46.9 | 75.4 | 48.2 | 76.8 | 48.3 | 79.8 | 50.6 | 80.6 | 51.0 |
| 0.0 | 81.2 | 48.2 | 84.2 | 51.2 | 84.2 | 52.4 | 85.1 | 51.5 | 85.1 | 53.4 | 86.0 | 53.8 |
| 0.1 | 78.3 | 46.9 | 81.6 | 49.8 | 81.3 | 50.6 | 82.1 | 51.2 | 84.6 | 53.1 | 85.6 | 53.5 |
| 0.2 | 75.4 | 46.3 | 78.7 | 48.6 | 78.8 | 49.6 | 79.9 | 50.2 | 82.4 | 52.0 | 83.5 | 52.4 |
| 0.3 | 72.2 | 45.6 | 76.2 | 47.4 | 75.5 | 48.1 | 76.8 | 49.2 | 80.7 | 51.3 | 81.1 | 51.8 |
| 0.4 | 70.0 | 44.0 | 74.1 | 45.6 | 72.6 | 47.5 | 74.9 | 48.0 | 78.1 | 50.0 | 80.4 | 50.4 |
| 0.5 | 66.4 | 43.3 | 72.6 | 45.1 | 70.7 | 46.7 | 73.2 | 46.7 | 77.4 | 49.2 | 78.1 | 50.0 |
| 0.6 | 63.2 | 42.9 | 71.1 | 43.8 | 70.3 | 45.6 | 72.1 | 45.1 | 75.5 | 48.5 | 77.0 | 47.5 |
| 0.7 | 62.6 | 42.5 | 70.5 | 43.6 | 69.5 | 44.8 | 70.4 | 44.5 | 74.6 | 47.6 | 73.0 | 48.3 |
表4 CMU-MOSEI数据集上随机模态缺失下的实验结果
Tab. 4 Experimental results on CMU-MOSEI dataset with random modality missing
| 缺失率 | DCCAE | MCTN | MMIN | GCNet | IMDer | MM-SSC | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | F1/% | ACC7/% | |
| 平均 | 71.2 | 45.0 | 76.1 | 46.9 | 75.4 | 48.2 | 76.8 | 48.3 | 79.8 | 50.6 | 80.6 | 51.0 |
| 0.0 | 81.2 | 48.2 | 84.2 | 51.2 | 84.2 | 52.4 | 85.1 | 51.5 | 85.1 | 53.4 | 86.0 | 53.8 |
| 0.1 | 78.3 | 46.9 | 81.6 | 49.8 | 81.3 | 50.6 | 82.1 | 51.2 | 84.6 | 53.1 | 85.6 | 53.5 |
| 0.2 | 75.4 | 46.3 | 78.7 | 48.6 | 78.8 | 49.6 | 79.9 | 50.2 | 82.4 | 52.0 | 83.5 | 52.4 |
| 0.3 | 72.2 | 45.6 | 76.2 | 47.4 | 75.5 | 48.1 | 76.8 | 49.2 | 80.7 | 51.3 | 81.1 | 51.8 |
| 0.4 | 70.0 | 44.0 | 74.1 | 45.6 | 72.6 | 47.5 | 74.9 | 48.0 | 78.1 | 50.0 | 80.4 | 50.4 |
| 0.5 | 66.4 | 43.3 | 72.6 | 45.1 | 70.7 | 46.7 | 73.2 | 46.7 | 77.4 | 49.2 | 78.1 | 50.0 |
| 0.6 | 63.2 | 42.9 | 71.1 | 43.8 | 70.3 | 45.6 | 72.1 | 45.1 | 75.5 | 48.5 | 77.0 | 47.5 |
| 0.7 | 62.6 | 42.5 | 70.5 | 43.6 | 69.5 | 44.8 | 70.4 | 44.5 | 74.6 | 47.6 | 73.0 | 48.3 |
| 模型 | 参数量/106 | GPU浮点数运算量/GFLOPs |
|---|---|---|
| IMDer | 89.79 | 230.7 |
| MM-SSC | 88.46 | 137.9 |
表5 模型在CMU-MOSI数据集上的训练开销对比
Tab. 5 Training overhead comparison of models on CMU-MOSI dataset
| 模型 | 参数量/106 | GPU浮点数运算量/GFLOPs |
|---|---|---|
| IMDer | 89.79 | 230.7 |
| MM-SSC | 88.46 | 137.9 |
| 模型 | 不同缺失率下的训练时间/s | 平均训练 时间/s | |||
|---|---|---|---|---|---|
| VMR=0.1 | VMR=0.3 | VMR=0.5 | VMR=0.7 | ||
| IMDer | 49.48 | 128.85 | 200.96 | 266.96 | 161.56 |
| MM-SSC | 13.86 | 15.79 | 16.03 | 16.11 | 15.44 |
表6 不同模型在CMU-MOSI数据集上的训练时间对比
Tab. 6 Training time comparison of different models on CMU-MOSI dataset
| 模型 | 不同缺失率下的训练时间/s | 平均训练 时间/s | |||
|---|---|---|---|---|---|
| VMR=0.1 | VMR=0.3 | VMR=0.5 | VMR=0.7 | ||
| IMDer | 49.48 | 128.85 | 200.96 | 266.96 | 161.56 |
| MM-SSC | 13.86 | 15.79 | 16.03 | 16.11 | 15.44 |
| CMC | SMR | CMU-MOSI | CMU-MOSEI | ||
|---|---|---|---|---|---|
| F1/% | MAE | F1/% | MAE | ||
| 79.8 | 0.862 | 78.7 | 0.625 | ||
| √ | 81.2 | 0.828 | 79.5 | 0.605 | |
| √ | 80.9 | 0.806 | 80.0 | 0.620 | |
| √ | √ | 82.5 | 0.799 | 80.4 | 0.596 |
表7 CMU-MOSI和CMU-MOSEI数据集上的消融实验结果
Tab. 7 Ablation experimental results on CMU-MOSI and CMU-MOSEI datasets
| CMC | SMR | CMU-MOSI | CMU-MOSEI | ||
|---|---|---|---|---|---|
| F1/% | MAE | F1/% | MAE | ||
| 79.8 | 0.862 | 78.7 | 0.625 | ||
| √ | 81.2 | 0.828 | 79.5 | 0.605 | |
| √ | 80.9 | 0.806 | 80.0 | 0.620 | |
| √ | √ | 82.5 | 0.799 | 80.4 | 0.596 |
| [1] | 夏鑫雨,朱磊,聂秀山,等. 典型概念驱动的模态缺失深度跨模态检索[J]. 计算机辅助设计与图形学学报, 2025, 37(3): 519-532. |
| Xia Xinyu, Zhu Lei, Nie Xiushan, et al. Typical concept-driven modality-missing deep cross-modal retrieval[J]. Journal of Computer-Aided Design and Computer Graphics, 2025, 37(3): 519-532. | |
| [2] | Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models[C]// NeurIPS 2020. Red Hook: Curran Associates Inc., 2020: 6840-6851. |
| [3] | Van den Oord A, Vinyals O, Kavukcuoglu K. Neural discrete representation learning [C]// NeurIPS 2017. Red Hook: Curran Associates Inc., 2017: 6309-6318. |
| [4] | Zhao J, Li R, Jin Q. Missing modality imagination network for emotion recognition with uncertain missing modalities [C]// ACL-IJCNLP 2021 (Volume 1: Long Papers). Stroudsburg: ACL, 2021: 2608-2618. |
| [5] | Zuo H, Liu R, Zhao J, et al. Exploiting modality-invariant feature for robust multimodal emotion recognition with missing modalities[C]// ICASSP 2023. Piscataway: IEEE, 2023: 1-5. |
| [6] | Zeng J, Liu T, Zhou J. Tag-assisted multimodal sentiment analysis under uncertain missing modalities [C]// SIGIR 2022. New York: ACM, 2022: 1545-1554. |
| [7] | Yuan Z, Li W, Xu H, et al. Transformer-based feature reconstruction network for robust multimodal sentiment analysis[C]// MM 2021. New York: ACM, 2021: 4400-4407. |
| [8] | Lian Z, Chen L, Sun L, et al. GCNet: graph completion network for incomplete multimodal learning in conversation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(7): 8419-8432. |
| [9] | Wang Y, Li Y, Cui Z. Incomplete multimodality-diffused emotion recognition [C]// NeurIPS 2023. Red Hook: Curran Associates Inc., 2023: 17117-17128. |
| [10] | Song Y, Sohl-Dickstein J, Kingma D P, et al. Score-based generative modeling through stochastic differential equations [PP/OL]. V2. arXiv (2021-02-10) [2025-05-16].. |
| [11] | 罗渊贻,吴锐,刘家锋,等. 面向情感语义不一致的多模态情感分析方法[J]. 计算机研究与发展, 2025, 62(2): 374-382. |
| Luo Yuanyi, Wu Rui, Liu Jiafeng, et al. Multimodal sentiment analysis method for sentimental semantic inconsistency [J]. Journal of Computer Research and Development, 2025, 62(2): 374-382. | |
| [12] | Radhakrishnan A, Friedman S F, Khurshid S, et al. Cross-modal autoencoder framework learns holistic representations of cardiovascular state [J]. Nature Communications, 2023, 14: No.2436. |
| [13] | Sadok S, Leglaive S, Girin L, et al. A multimodal dynamical variational autoencoder for audiovisual speech representation learning [J]. Neural Networks, 2024, 172: No.106120. |
| [14] | Ashuach T, Gabitto M I, Koodli R V, et al. MultiVI: deep generative model for the integration of multimodal data [J]. Nature Methods, 2023, 20(8): 1222-1231. |
| [15] | Wu Y, Song T, Wu Z, et al. CodeBrain: imputing any brain MRI via instance-specific scalar-quantized codes [PP/OL]. V2. arXiv (2025-03-09) [2025-05-23].. |
| [16] | Redekop E, Pleasure M, Wang Z, et al. Codebook VQ-VAE approach for prostate cancer diagnosis using multiparametric MRI[C]// CVPRW 2024. Piscataway: IEEE, 2024: 2365-2372. |
| [17] | Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional Transformers for language understanding [C]// NAACL-HLT 2019, Volume 1 (Long and Short Papers). Stroudsburg: ACL, 2019: 4171-4186. |
| [18] | Stöckli S, Schulte-Mecklenbeck M, Borer S, et al. Facial expression analysis with AFFDEX and FACET: a validation study[J]. Behavior Research Methods, 2018, 50(4): 1446-1460. |
| [19] | Degottex G, Kane J, Drugman T, et al. COVAREP: a collaborative voice analysis repository for speech technologies [C]// ICASSP 2014. Piscataway: IEEE, 2014: 960-964. |
| [20] | Zadeh A, Zellers R, Pincus E, et al. Multimodal sentiment intensity analysis in videos: facial gestures and verbal messages[J]. IEEE Intelligent Systems, 2016, 31(6): 82-88. |
| [21] | Bagher Zadeh A, Liang P P, Poria S, et al. Multimodal language analysis in the wild: CMU-MOSEI dataset and interpretable dynamic fusion graph [C]// ACL (Volume 1: Long Papers). Stroudsburg: ACL, 2018: 2236-2246. |
| [22] | Wang W, Arora R, Livescu K, et al. On deep multi-view representation learning [J]. Proceedings of Machine Learning Research, 2015, 37: 1083-1092. |
| [23] | Pham H, Liang P P, Manzini T, et al. Found in translation: learning robust joint representations by cyclic translations between modalities [J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 6892-6899. |
| [1] | 刘明, 沈东奇, 孟子洋. 双分支结构下多层次特征融合的点云配准网络[J]. 《计算机应用》唯一官方网站, 2026, 46(8): 2584-2593. |
| [2] | 刘凤春, 邵馨莹, 张春英, 王立亚, 任静. 多尺度特征优化的单目深度估计框架FCMdepth[J]. 《计算机应用》唯一官方网站, 2026, 46(8): 2603-2611. |
| [3] | 刘威, 李维刚, 田志强. 面向点云分类与分割的层次化旋转不变几何结构的表征学习方法[J]. 《计算机应用》唯一官方网站, 2026, 46(8): 2594-2602. |
| [4] | 刘新亮, 徐雨时, 李杜白, 任延昭. 基于知识图谱的问答方法综述[J]. 《计算机应用》唯一官方网站, 2026, 46(8): 2394-2410. |
| [5] | 郭晓金, 隋旭洋, 周柯男. 基于深度学习的STAR⁃RIS辅助混合场通信系统信道估计[J]. 《计算机应用》唯一官方网站, 2026, 46(8): 2548-2554. |
| [6] | 刘晶, 赵邵泽, 刘鑫刚, 牛浩哲, 季海鹏. 跨工况下基于改进CGAN的钛合金显微组织数据生成方法[J]. 《计算机应用》唯一官方网站, 2026, 46(7): 2373-2382. |
| [7] | 凌妙根, 井瑞, 方巍. 可解释性深度学习在热带气旋预报中的应用研究综述[J]. 《计算机应用》唯一官方网站, 2026, 46(7): 2318-2326. |
| [8] | 杜秀丽, 高星, 张校毓, 潘成胜, 邹启杰. 基于密集时空可变形注意力的视频快照压缩成像重建方法[J]. 《计算机应用》唯一官方网站, 2026, 46(7): 2288-2296. |
| [9] | 武湘怡, 叶海良, 曹飞龙. 基于平滑-锐化图卷积的点云补全方法[J]. 《计算机应用》唯一官方网站, 2026, 46(7): 2267-2276. |
| [10] | 邱晓红, 于子祥, 胡玉杰. 高频引导的扩散模型故障诊断样本增广方法[J]. 《计算机应用》唯一官方网站, 2026, 46(7): 2119-2129. |
| [11] | 杜艺, 续明进, 孔佳仪, 王力瑶, 赵晨. 基于YOLOv11的低秩自适应参数高效微调算法[J]. 《计算机应用》唯一官方网站, 2026, 46(6): 1738-1745. |
| [12] | 熊珍凯, 徐梦军, 孙胤胤, 王鑫. 基于改进YOLOv8的复杂天气环境下海面船舶检测算法[J]. 《计算机应用》唯一官方网站, 2026, 46(6): 1998-2006. |
| [13] | 何运平, 王雷春, 宋芮芮, 卢祥凤, 魏金香, 刘小萌. 基于对比不变性和强化特定性的双通道多模态情感分析模型[J]. 《计算机应用》唯一官方网站, 2026, 46(6): 1767-1775. |
| [14] | 姚力挽, 刘海龙, 曾张帆. 基于频域驱动及扩散融合的声纳图像增强算法[J]. 《计算机应用》唯一官方网站, 2026, 46(6): 1947-1955. |
| [15] | 何丽丽, 曹勐, 张磊, 潘洪军, 刘义, 孙成心. 基于Kolmogorov-Arnold网络与扩散Transformer的手语生成模型[J]. 《计算机应用》唯一官方网站, 2026, 46(6): 1801-1810. |
| 阅读次数 | ||||||
|
全文 |
|
|||||
|
摘要 |
|
|||||