• •    

基于编辑约束的端到端越南语文本正则化方法

蒋铭1,王琳钦1,赖华2,高盛祥1   

  1. 1. 昆明理工大学
    2. 昆明理工大学信息工程与自动化学院
  • 收稿日期:2024-03-05 修回日期:2024-04-17 发布日期:2024-06-04
  • 通讯作者: 赖华
  • 基金资助:
    国家自然科学基金;云南高新技术产业发展项目;云南省重点研发计划;云南省基础研究计划;云南省学术和技术带头人后备人才

An End-to-End Vietnamese Text Normalization Approach Based on Edit Constraints

  • Received:2024-03-05 Revised:2024-04-17 Online:2024-06-04

摘要: 文本正则化是语音合成文本前端分析任务中不可或缺的步骤。语义歧义性是文本正则化任务面临的主要问题,特别是在非标准词汇,如数字、日期等方面。虽然神经网络系统可以利用上下文解决这些问题,但会产生不可恢复性的错误。因此,本文提出了一种基于编辑约束的端到端文本正则化方法,充分考虑了越南语的语言特点,设计专门用于越南语的标注方法,以提高模型对上下文语义信息的建模能力。同时,本文采用编辑对齐算法,有效地约束非标准词文本的范围,减小解码端搜索空间,从而避免了模型自身局限性所导致的非正则化文本预测错误。实验证明,本研究方法在越南语文本正则化中取得了97%的准确率,并且在中文开源数据集上也取得了显著的效果,验证了该方法在越南语之外的适用性。

关键词: 越南语, 文本正则化, 编辑对齐算法, 语音合成

Abstract: Text normalization is a crucial pre-processing step in text-to-speech synthesis front-end analysis. The Viet-namese language presents challenges related to the semantic ambiguity of non-standard words such as num-bers and dates. Neural text normalization systems can leverage context; however, they suffer from unrecov-erable errors. This study introduces an end-to-end text normalization method grounded in edit constraints. Taking full account of the linguistic characteristics of Vietnamese, we propose a specialized text normalization annotation method for Vietnamese, aiming to enhance the model's contextual semantic information modeling. Additionally, an edit alignment algorithm is applied to effectively restrict the scope of non-standard word text, thereby reducing the search space during decoding. This mitigates text normalization prediction errors arising from inherent model limitations. The experimental results show 97% accuracy in Vietnamese text normali-zation using the proposed method. Moreover, its effectiveness extends to an open-source Chinese dataset, validating the applicability of the method beyond the Vietnamese language.

Key words: Keywords: Vietnamese, Text normalization, Edit alignment algorithm, TTS