Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (7): 1891-1895.DOI: 10.11772/j.issn.1001-9081.2019111974

• Artificial intelligence • Previous Articles     Next Articles

Non-autoregressive method for Uyghur-Chinese neural machine translation

ZHU Xiangrong1,2,3, WANG Lei1,2,3, YANG Yating1,2,3, DONG Rui1,2,3, ZHANG Jun1,3   

  1. 1. The Xinjiang Technical Institute of Physics&Chemistry, Chinese Academy of Sciences, Urumqi Xinjiang 830011, China;
    2. University of Chinese Academy of Sciences, Beijing 100049, China;
    3. Xinjiang Laboratory of Minority Speech and Language Information Processing;(The Xinjiang Technical Institute of Physics&Chemistry, Chinese Academy of Sciences), Urumqi Xinjiang 830011, China
  • Received:2019-11-20 Revised:2020-01-14 Online:2020-07-10 Published:2020-06-29
  • Supported by:
    This work is partially supported by the National Key Research and Development Program of China (2017YFC0820703),the "Light of the West" Talent Training and Introduction Program of Chinese Academy of Sciences (2017-XBQNXZ-A-005).

基于非自回归方法的维汉神经机器翻译

朱相荣1,2,3, 王磊1,2,3, 杨雅婷1,2,3, 董瑞1,2,3, 张俊1,3   

  1. 1. 中国科学院新疆理化技术研究所, 乌鲁木齐 830011;
    2. 中国科学院大学, 北京 100049;
    3. 中国科学院新疆理化技术研究所 新疆民族语音语言信息处理实验室, 乌鲁木齐 830011
  • 通讯作者: 王磊
  • 作者简介:朱相荣(1994-),男,山西吕梁人,硕士研究生,主要研究方向:维汉神经机器翻译、自然语言处理;王磊(1974-),男,新疆乌鲁木齐人,研究员,博士,CCF会员,主要研究方向:自然语言处理、多语种信息处理;杨雅婷(1985-),女,新疆乌鲁木齐人,研究员,博士,CCF会员,主要研究方向:多语种语音识别、机器翻译;董瑞(1985-),男,新疆乌鲁木齐人,副研究员,博士,CCF会员,主要研究方向:维汉神经机器翻译、自然语言处理;张俊(1983-),男,湖南石首人,工程师,硕士,主要研究方向:网络安全、区块链。
  • 基金资助:
    国家重点研发计划项目(2017YFC0820703);中国科学院“西部之光”人才培养引进计划项目(2017-XBQNXZ-A-005)

Abstract: Although the existing autoregressive translation models based on recurrent neural network, convolutional neural network or Transformer have good translation performance, they have the problem of low translation speed due to low decoding parallelism. Therefore, a non-autoregressive model based learning rate optimization strategy was proposed. On the basis of the non-autoregressive sequence model based on iterative optimization, the learning rate adjustment method was changed, which means that warm up was replaced with liner annealing. Firstly, liner annealing was evaluated to be better than warm up; then liner annealing was applied to the non-autoregressive sequence model in order to obtain the optimal balance between translation quality and decoding speed; finally a comparison between this method and the method of autoregressive model was carried out. Experimental results show that compared with the autoregressive model Transformer, when the decoding speed is increased by 2.74 times, this method has the BiLingual Evaluation Understudy (BLEU) score value of translation quality of 41.31, which reached 95.34% of that of the Transformer. It can be seen that the non-autoregressive sequence model of liner annealing can effectively improve the decoding speed under the condition of reducing a little translation quality, which is suitable for the platforms with urgent need for translation speed.

Key words: Uyghur, machine translation, decoding speed, translation quality, non-autoregressive model

摘要: 现有的基于循环神经网络、卷积神经网络和Transformer的自回归翻译模型,虽然都具有良好的翻译性能,但由于解码并行性较低导致了翻译速度慢的问题,针对这个问题提出一种基于非自回归模型的优化学习率策略的方法。在基于迭代优化的非自回归序列模型的基础上,改变学习率调节方法,即把warm up替换为liner annealing方法。首先评估出liner annealing优于warm up方法,然后将liner annealing应用于非自回归序列模型以得到翻译质量和解码速度的最优平衡,最后将该方法与自回归模型的方法作对比。实验结果表明该方法相较于自回归模型Transformer,当解码速度提升1.74倍时,翻译质量的双语评估替换(BLEU)分数值为41.31,可达到Transformer的95.34%。由此可见,采用liner annealing的非自回归序列模型,在降低少许翻译质量的条件下,能够有效地提升解码速度,适用于对翻译速度需求迫切的平台。

关键词: 维吾尔语, 机器翻译, 解码速度, 翻译质量, 非自回归模型

CLC Number: