Abstract:

In response to the problems that traditional Convolutional Neural Networks （CNNs） have difficulty in capturing global emotional information in Speech Emotion Recognition （SER） due to limited receptive fields， and that global self-attention mechanisms have excessive computational overhead on resource-constrained devices such as mobile terminals， a lightweight and high-precision SER model suitable for edge deployment — a Global-Aware Fusion network with Binary Multi-scale Feature Representation （GAF-BMFR） was proposed. In this network， a Binary Neural Network （BNN） was used as the backbone to reduce model complexity. The core parts include the following ones： a Binary Multi-Scale Block （BMSB） was designed， in which parallel 3×3 and 5×5 binary convolutions were used to extract fine-grained and large-receptive-field features synchronously in the time-frequency domain； a lightweight Global-Aware Block （GAB） was introduced， which utilized fully connected layers and 1×1 convolutions to replace the high-cost self-attention mechanism， thereby achieving global correlation modeling and adaptive selection of cross-scale features. Furthermore， a union loss function of Class-Balanced Focal， Log-Margin， and Multi-Scale binary Regularization （CB-FLM-MSR） was designed to alleviate problems such as class imbalance and ambiguous emotional boundaries. Experimental results on public datasets show that GAF-BMFR has superior performance with low resource consumption. Taking the script part of the IEMOCAP dataset as an example， compared to the typical Attention Pooling Convolutional Neural Network （APCNN） model， GAF-BMFR has the Weighted Accuracy （WA） improved by 25.23 percentage points， and the Unweighted Accuracy （UA） improved by 24.18 percentage points. Concurrently， GAF-BMFR has the number of model parameters reduced by approximately 24.04% significantly， compared to its full-precision version. It can be seen that GAF-BMFR model enhances the accuracy and robustness of SER effectively while reducing the number of parameters and computational overhead significantly， providing a feasible lightweight solution for deploying efficient affective computing applications in resource-constrained scenarios.

Key words: Speech Emotion Recognition (SER), binarization, multi-scale feature, Global-Aware Block (GAB), model compression, edge deployment

摘要：

针对传统卷积神经网络（CNN）在语音情感识别（SER）中因感受野受限而难以捕获全局情感信息，以及全局自注意力机制在移动端等资源受限设备上计算开销过大的问题，提出一种适用于边缘部署的轻量级、高精度的SER模型——基于二值多尺度特征表示的全局感知融合网络（GAF-BMFR）。该网络以二值神经网络（BNN）为骨架以降低模型复杂度，核心部分包括：设计二值多尺度模块（BMSB），通过并行的3×3与5×5二值卷积同步提取时频域的细粒度与宽视野特征；引入轻量级的全局感知模块（GAB），利用全连接层与1×1卷积替代高成本的自注意力机制，实现跨尺度特征的全局关联建模与自适应选择。此外，设计类均衡焦点?对数边距?多尺度二值正则化（CB-FLM-MSR）联合损失函数，缓解类别不平衡与情感边界模糊等问题。在公开数据集上的实验结果表明，GAF-BMFR的性能优越且资源消耗低。以IEMOCAP数据集的剧本部分为例，与典型的注意力池化卷积神经网络（APCNN）模型相比，GAF-BMFR的加权准确率（WA）提升了25.23个百分点，未加权准确率（UA）提升了24.18个百分点；同时，GAF-BMFR的参数量相较于它的全精度版本显著降低了约24.04%。可见，GAF-BMFR在显著降低参数量与计算开销的同时，有效提升了SER的准确率与鲁棒性，为在资源受限场景下部署高效的情感计算应用提供了一个可行的轻量级解决方案。

关键词: 语音情感识别, 二值化, 多尺度特征, 全局感知模块, 模型压缩, 边缘部署