Abstract:

In order to further improve the feature recognition and extraction efficiency of transfer learning， reduce negative transfer and enhance the learning performance of the model， a transfer learning model based on improved Domain Separation Network （DSN） — AMCN-DSN （Attention Mechanism Capsule Network-DSN） was proposed. Firstly， the extraction and reconstruction of feature information in the source and target domains were accomplished by using Multi-Head Attention CapsNet （MHAC）， the feature information was filtered effectively based on the attention mechanism， and the capsule network was adopted to improve the extraction quality of deep information. Secondly， a dynamic adversarial factor was introduced to optimize the reconstruction loss function， so that the reconstructor was able to dynamically measure the relative importance of the source and target domain information to improve the robustness and convergence speed of transfer learning. Finally， a multi-head self-attention mechanism was incorporated into the classifier to enhance the semantic understanding of the public features and improve the classification performance. In the sentiment analysis experiments， compared to other transfer learning models， the proposed model can transfer the learned knowledge to tasks with less data but high similarity with the least degradation of classification performance and good transfer performance. In the intent recognition experiments， the proposed model improves the precision， recall and F1 score by 4.5%， 4.3% and 4.4% respectively， compared to the model with suboptimal classification performance — Capsule Network improved Domain Adversarial Neural Network （DANN+CapsNet） model， showing certain advantages of the proposed model in dealing with small data problems and personalization problems. In comparison with DSN， AMCN-DSN has the F1 scores on the target domain in the above-mentioned two types of experiments improved by 6.0% and 12.4% respectively， further validating the effectiveness of the improved model.

Key words: transfer learning, Domain Separation Network (DSN), capsule network, attention mechanism, Natural Language Processing (NLP)

摘要：

为进一步提高迁移学习的特征识别和提取效率、减少负迁移并增强模型的学习性能，提出了一种基于改进领域分离网络（DSN）的迁移学习模型AMCN-DSN（Attention Mechanism Capsule Network-DSN）。首先，使用融合多头注意力机制的胶囊网络（MHAC）完成源域和目标域特征信息的提取与重构，基于注意力机制有效筛选特征信息，并利用胶囊网络提高深层信息的提取质量；其次，引入动态对抗因子优化重构损失函数，使重构器可动态衡量源域与目标域信息的相对重要性，从而增强迁移学习的鲁棒性和提升收敛速度；最后，在分类器中融入多头自注意力机制，以强化对公有特征的语义理解并提高分类性能。在情感分析实验中，相较于其他迁移学习模型，所提模型能够将学习到的知识迁移到数据量少但相似性高的任务中，分类性能的下降幅度最小，迁移表现较好；在意图识别实验中，相较于分类性能次优的胶囊网络改进领域对抗神经网络（DANN+CapsNet）模型，所提模型的精确度、召回率和F1值分别提升了4.5%、4.3%和4.4%，表明所提模型在处理小数据问题和个性化问题上具有一定优势。与DSN相比，AMCN-DSN在上述两类实验目标域上的F1值分别提高了6.0%和12.4%，进一步验证了改进模型的有效性。

关键词: 迁移学习, 领域分离网络, 胶囊网络, 注意力机制, 自然语言处理