关键词: 身体动作情感识别, 对比语言-图像预训练模型, 隐空间学习, 提示学习, 多标签分类

Abstract:

The key to body movement emotion recognition lies in extracting emotional features existed in human body movements. To solve the problems of poor emotional feature learning capability and difficulty in improving emotion recognition accuracy in existing models， a body movement emotion recognition method based on Emotional Latent Space Learning （ELSL） and Contrastive Language-Image Pre-training （CLIP） model was proposed. Firstly， CLIP model was introduced to improve the emotional feature learning capability of the model. Secondly， for the fine-grained multi-label emotion classification task， ELSL method was proposed. By learning discriminative mappings from emotional latent space to various subspaces， the subtle differences between emotion categories and the feature information beneficial to the classification of each emotion category in various emotional subspaces. Experiments were carried out on real-world open scenarios-oriented Body Language Dataset （BoLD） The results demonstrate that the proposed method makes use of the advantages of CLIP model and latent space learning in feature learning effectively， leading to significant performance improvement. In specific， compared to Movement Analysis Network （MANet）， the proposed method has a 1.08 percentage points increase in mean Average Precision （mAP） and a 1.32 percentage points improvement in mean Area Under Receiver Operating Characteristic Curve （mRA）.

Key words: body movement emotion recognition, Contrastive Language-Image Pre-training （CLIP） model, latent space learning, prompt learning, multi-label classification