关键词: 说话人确认, 说话人嵌入, Transformer, 自注意力机制, 特征融合

Abstract: The Transformer based on the self-attention mechanism has demonstrated superior performance in most Natural Language Processing (NLP) tasks. However, previous studies have indicated that the Transformer is not highly competitive when applied to speaker verification, particularly in terms of local modeling capability, lightweight structure, and real-time inference performance. To address these issues, an end-to-end speaker verification model Deep Treatment Fusion-Transformer (DTF-Transformer) based on an improved Transformer was proposed from three aspects. First, a simplified multi-scale attention mechanism was employed in place of multi-head attention to enhance the model's local modeling ability and reduce the number of parameters. Second, a lightweight Feed Forward Network (FFN) was designed to further reduce parameters and accelerate inference. Finally, a fusion mechanism was applied to features at different depths to improve the model’s ability to represent and generalize deep features. Experimental results on the VoxCeleb and CN-Celeb public benchmark datasets demonstrate that, compared to the popular ResNet34 and ECAPA-TDNN networks, the DTF-Transformer reduces the Equal Error Rate (EER) by 14% and 23% on the VoxCeleb-O test set, and by 14% and 15% on the CN-Celeb(E) test set, respectively. Furthermore, the DTF-Transformer is more lightweight and offers better inference speed without sacrificing accuracy.

Key words: speaker verification, speaker embedding, Transformer, self-attention mechanism, feature fusion