Abstract:

In text matching tasks， the complexity and diversity of textual data often lead to issues of lacking robustness during training. Traditional methods to address the lack of robustness， such as data augmentation and regularization， can be effective， but are often only applicable to specific types of noise or disturbances， and require a lot of computational resources. Therefore， a method based on Meta-Weight network （MW-net） — Meta-Weight network improved by the Covariance matrix （CovMW-net） was proposed. Firstly， the weight parameters and loss functions were adjusted by learning adaptively， thereby realizing rapid and reasonable weight distribution. Then， by controlling the weights of samples， the impacts of samples on training effects were magnified or diminished， and ultimately the training robustness was enhanced. The meta-learning framework of MW-net was inherited by CovMW-net， thereby saving computational resources. At the same time， by CovMW-net， through incorporating covariance matrices， deep feature extraction for samples in each category was conducted， and the covariance matrices of these features were calculated to measure minority class data， thereby mitigating the negative impacts of long-tail distributions caused by random sampling from meta-datasets in MW-net. Experimental results on the Clothing1M dataset show that CovMW-net outperforms the original method MW-net by 0.86 percentage points in accuracy and outperforms all comparative methods. In addition， on the Large-scale Chinese Question Matching Corpus （LCQMC） and Baidu Question-answer matching dataset （BQ）， CovMW-net has the accuracy improvements between 4 and 6 percentage points mostly compared to the baseline. It can be seen that CovMW-net is effective in dealing with biases in meta-datasets and is feasible for application in research on the robustness of text matching.

Key words: text matching, robustness, meta-weight, covariance matrix, sample reweighting

摘要：

在文本匹配任务中，文本数据的复杂性与多样性使训练时常暴露出鲁棒性欠佳的问题。传统的解决文本鲁棒性不足的手段，诸如数据增强和正则化等，虽能发挥一定作用，但这些方法大多仅适用于特定类型的噪声或扰动，并且对计算资源的需求较高。因此，提出一种基于元权重网络（MW-net）的方法——协方差矩阵改进的元权重网络（CovMW-net）。首先，通过自适应学习调整权重参数与损失函数，从而迅速实现较合理的权重分配。其次，借助对样本权重的调控，放大或缩小样本在训练过程中对训练效果的影响，最终实现提升训练鲁棒性的目的。CovMW-net继承MW-net的元学习框架，进而节约计算资源。同时，它融合协方差矩阵，针对每个类别的样本开展深度特征提取，计算这些特征的协方差矩阵，并以此度量少数类数据，进而削减MW-net因为元数据集随机取样而产生长尾分布所造成的负面影响。在Clothing1M数据集上的实验结果表明，CovMW-net在准确率上超过原始方法MW-net 0.86个百分点，并优于所有对比方法。此外，在大规模中文问题匹配语料库（LCQMC）和百度问答匹配数据集（BQ）上CovMW-net的准确率相较于baseline提升大多在4~6个百分点。可见，CovMW-net在处理元数据集存在偏差时具备有效性，且应用于文本匹配鲁棒性研究时具有可行性。

关键词: 文本匹配, 鲁棒性, 元权重, 协方差矩阵, 样本重加权