In response to the problems that traditional Convolutional Neural Networks （CNNs） have difficulty in capturing global emotional information in Speech Emotion Recognition （SER） due to limited receptive fields， and that global self-attention mechanisms have excessive computational overhead on resource-constrained devices such as mobile terminals， a lightweight and high-precision SER model suitable for edge deployment — a Global-Aware Fusion network with Binary Multi-scale Feature Representation （GAF-BMFR） was proposed. In this network， a Binary Neural Network （BNN） was used as the backbone to reduce model complexity. The core parts include the following ones： a Binary Multi-Scale Block （BMSB） was designed， in which parallel 3×3 and 5×5 binary convolutions were used to extract fine-grained and large-receptive-field features synchronously in the time-frequency domain； a lightweight Global-Aware Block （GAB） was introduced， which utilized fully connected layers and 1×1 convolutions to replace the high-cost self-attention mechanism， thereby achieving global correlation modeling and adaptive selection of cross-scale features. Furthermore， a union loss function of Class-Balanced Focal， Log-Margin， and Multi-Scale binary Regularization （CB-FLM-MSR） was designed to alleviate problems such as class imbalance and ambiguous emotional boundaries. Experimental results on public datasets show that GAF-BMFR has superior performance with low resource consumption. Taking the script part of the IEMOCAP dataset as an example， compared to the typical Attention Pooling Convolutional Neural Network （APCNN） model， GAF-BMFR has the Weighted Accuracy （WA） improved by 25.23 percentage points， and the Unweighted Accuracy （UA） improved by 24.18 percentage points. Concurrently， GAF-BMFR has the number of model parameters reduced by approximately 24.04% significantly， compared to its full-precision version. It can be seen that GAF-BMFR model enhances the accuracy and robustness of SER effectively while reducing the number of parameters and computational overhead significantly， providing a feasible lightweight solution for deploying efficient affective computing applications in resource-constrained scenarios.