关键词: 文本匹配, 鲁棒性, 元权重, 协方差矩阵, 样本重加权

Abstract:

In text matching tasks， the complexity and diversity of textual data often lead to issues of lacking robustness during training. Traditional methods to address the lack of robustness， such as data augmentation and regularization， can be effective， but are often only applicable to specific types of noise or disturbances， and require a lot of computational resources. Therefore， a method based on Meta-Weight network （MW-net） — Meta-Weight network improved by the Covariance matrix （CovMW-net） was proposed. Firstly， the weight parameters and loss functions were adjusted by learning adaptively， thereby realizing rapid and reasonable weight distribution. Then， by controlling the weights of samples， the impacts of samples on training effects were magnified or diminished， and ultimately the training robustness was enhanced. The meta-learning framework of MW-net was inherited by CovMW-net， thereby saving computational resources. At the same time， by CovMW-net， through incorporating covariance matrices， deep feature extraction for samples in each category was conducted， and the covariance matrices of these features were calculated to measure minority class data， thereby mitigating the negative impacts of long-tail distributions caused by random sampling from meta-datasets in MW-net. Experimental results on the Clothing1M dataset show that CovMW-net outperforms the original method MW-net by 0.86 percentage points in accuracy and outperforms all comparative methods. In addition， on the Large-scale Chinese Question Matching Corpus （LCQMC） and Baidu Question-answer matching dataset （BQ）， CovMW-net has the accuracy improvements between 4 and 6 percentage points mostly compared to the baseline. It can be seen that CovMW-net is effective in dealing with biases in meta-datasets and is feasible for application in research on the robustness of text matching.

Key words: text matching, robustness, meta-weight, covariance matrix, sample reweighting