Abstract: Speech emotion recognition is an important and challenging task in human-computer interaction systems. To address the issues of single-feature representation and weak feature interaction in current speech emotion recognition systems, a multi-input interactive attention network is proposed. The network consists of two sub-networks, namely the Specific Feature Coordinate Residual Attention Network and the Shared Feature Multi-Head Attention Network. The former utilizes Res2Net and coordinate attention modules to learn specific features extracted from raw speech and generate multi-scale feature representations, enhancing the model's ability to represent emotion-related information. The latter combines features obtained from the forward network to form shared features, which are then input into the multi-head attention module. The multi-head attention mechanism allows for simultaneous attention to relevant information in different feature subspaces, enhancing the interaction among features and capturing highly discriminative characteristics. Through the synergistic effect of the two sub-networks mentioned above, the model can increase the diversity of features and improve the interaction capability among features. During the training process, a dual-loss function is applied for joint supervision, aiming to make the samples of the same class more compact and the samples of different classes more separated. The experimental results demonstrate that the proposed model achieves a weighted average accuracy of 91.43% on the EMO-DB database and 76.33% on the IEMOCAP database. Compared to other state-of-the-art models, it exhibits superior classification performance.

Key words: Speech emotion recognition, coordinate attention mechanism, multi-head attention mechanism, specific feature learning, shared feature learning

摘要： 语音情感识别是人机交互系统中一项重要且充满挑战性的任务。针对目前语音情感识别系统中存在特征单一和特征间交互性较弱问题，提出了多输入交互注意力网络。该网络由特定特征坐标残差注意力网络和共享特征多头注意力网络两个子网络组成。前者利用Res2Net和坐标注意力模块学习从原始语音中获取的特定特征，并生成多尺度特征表示，增强模型对情感相关信息的表征能力；后者融合前向网络所获取的特征，组成共享特征，并经BLSTM输入至多头注意力模块，能够同时关注不同特征子空间中的相关信息，增强特征之间的交互性，以捕获判别性强的特征。通过两个子网络间的协同作用，能够使模型增加特征的多样性，并提高特征之间的交互能力。在训练过程中，应用双损失函数共同监督，使同类样本更紧凑，不同类样本更分离。实验表明：所提模型在EMO-DB和IEMOCAP语料库上分别取得了91.43和76.33的加权平均精度，相较于其他主流模型，具有更好的分类性能。

关键词: 语音情感识别，坐标注意力机制，多头注意力机制，特定特征学习，共享特征学习