Abstract:

As a current leading Chinese Spelling Correction （CSC） model， ReLM （Rephrasing Language Model） has insufficient feature representation in complex semantic scenarios. To address this issue， an ReLM enhanced with deep semantic features， namely FeReLM （Feature-enhanced Rephrasing Language Model）， was proposed. In the model， Depthwise Separable Convolution （DSC） technique was used to integrate deep semantic features generated by feature extraction model BGE （BAAI General Embedding） with global features generated by ReLM， thereby enhancing the model’s ability to parse complex contexts and effectively improving the precision in recognizing and correcting spelling errors. Initially， FeReLM was trained on Wang271K dataset， enabling the model to learn deep semantics and complex expressions within sentences continuously. Subsequently， the trained weights were transferred， so that the knowledge learned by the model was applied to new datasets for fine-tuning. Experimental results show that FeReLM outperforms models such as ReLM， MCRSpell （Metric learning of Correct Representation for Chinese Spelling Correction）， and RSpell （Retrieval-augmented Framework for Domain Adaptive Chinese Spelling Check） on ECSpell and MCSC datasets in key metrics such as precision， recall， and F1 score， with improvements ranging from 0.6 to 28.7 percentage points. The effectiveness of the proposed method is confirmed through ablation experiments.

Key words: Natural Language Processing (NLP), feature enhancement, Chinese Spelling Correction (CSC), semantic fusion, text correction, Pre-trained Language Model (PLM)

摘要：

ReLM （Rephrasing Language Model）是当前性能领先的中文拼写纠错（CSC）模型。针对它在复杂语义场景中存在特征表达不足的问题，提出深层语义特征增强的ReLM——FeReLM （Feature-enhanced Rephrasing Language Model）。该模型利用深度可分离卷积（DSC）技术融合特征提取模型BGE（BAAI General Embeddings）生成的深层语义特征与ReLM生成的整体特征，从而有效提升模型对复杂上下文的解析力和拼写错误的识别纠正精度。首先，在Wang271K数据集上训练FeReLM，使模型持续学习句子中的深层语义和复杂表达；其次，迁移训练好的权重，从而将模型学习到的知识应用于新的数据集并进行微调。实验结果表明，在ECSpell和MCSC数据集上与ReLM、MCRSpell （Metric learning of Correct Representation for Chinese Spelling Correction）和RSpell（Retrieval-augmented Framework for Domain Adaptive Chinese Spelling Check）等模型相比，FeReLM的精确率、召回率、F1分数等关键指标的提升幅度可达0.6~28.7个百分点。此外，通过消融实验验证了所提方法的有效性。

关键词: 自然语言处理, 特征增强, 中文拼写纠错, 语义融合, 文本纠错, 预训练语言模型