关键词: 单模态弱监督, 对比学习, 门控集成, 多模态, 有害信息检测

Abstract: The proliferation of multimodal harmful information on social media not only undermines public interests but also severely disrupts social order, highlighting the urgent need for effective detection methods. Existing approaches have predominantly relied on pre-trained models to extract and integrate multimodal features, often neglecting the limitations of general semantics in harmful information detection and the complex, dynamic combinations of harmful content. To address these issues, a multimodal harmful content detection framework based on weakly Supervised modality semantic enhancement (weak-S) was introduced. Weakly supervised modality information was utilized to facilitate the harmful semantic alignment of multimodal features, and a low-rank bilinear pooling-based multimodal gated integration mechanism was designed to differentiate the contributions of various information sources. Experimental results show that the proposed method achieves F1-score improvements of 2.2 and 3.2 percentage points on the HarmP and MultiOFF benchmark datasets, respectively, outperforming SOTA (State-Of-The-Art) models and validating the significance of weakly supervised modality semantics in multimodal harmful information detection. Additionally, the method delivers a 1-percentage-point improvement in generalization performance for multimodal exaggeration detection tasks.

Key words: unimodal weak supervision, contrastive learning, gated integration, multimodal, harmful content detection