Abstract:

In Query-by-Example Spoken Term Detection （QbE-STD）， the Acoustic Word Embedding （AWE） speech information extracted by Convolutional Neural Network （CNN） or Recurrent Neural Network （RNN） is limited. To better represent speech content and improve model performance， an acoustic word embedding model based on Bi-directional Long Short-Term Memory （Bi-LSTM） and convolutional-Transformer was proposed. Firstly， Bi-LSTM was utilized for extracting features， modeling speech sequences and improving the model learning ability by superposition. Secondly， to learn local information while capturing global information， CNN and Transformer encoder were connected in parallel to form convolutional-Transformer， which taking full advantages in feature extraction to aggregate more efficient information and improving the discrimination of embeddings. Under the constraint of contrast loss， the Average Precision （AP） of the proposed model reaches 94.36%， which is 1.76% higher than that of the Bi-LSTM model based on attention. The experimental results show that the proposed model can effectively improve model performance and better perform QbE-STD.

Key words: Convolutional Neural Network (CNN), Acoustic Word Embedding (AWE), speech information, Query-by-Example Spoken Term Detection (QbE-STD), Recurrent Neural Network (RNN)

摘要：

示例查询语音关键词检测中，卷积神经网络（CNN）或者循环神经网络（RNN）提取到的声学词嵌入语音信息有限，为更好地表示语音内容以及改善模型的性能，提出一种基于双向长短时记忆（Bi-LSTM）和卷积Transformer的声学词嵌入模型。首先，使用Bi-LSTM提取特征、对语音序列进行建模，并通过叠加方式来提高模型的学习能力；其次，为了能在捕获全局信息的同时学习到局部信息，将CNN和Transformer编码器并联连接组成卷积Transformer，充分利用它在特征提取上的优势，聚合更多有效的信息，提高嵌入的区分性。在对比损失约束下，所提模型平均精度达到了94.36%，与基于注意力的Bi-LSTM模型相比，平均精度提高了1.76%。实验结果表明，所提模型可以有效改善模型性能，更好地实现示例查询语音关键词检测。

关键词: 卷积神经网络, 声学词嵌入, 语音信息, 示例查询语音关键词检测, 循环神经网络