Reordering table reconstruction model for Chinese-Uyghur machine translation
PAN Yirong1,2,3, LI Xiao1,3, YANG Yating1,3, MI Chenggang1,3, DONG Rui1,3
1. Xinjiang Technical Institute of Physics & Chemistry, Chinese Academy of Sciences, Urumqi Xinjiang 830011, China; 2. University of Chinese Academy of Sciences, Beijing 100049, China; 3. Xinjiang Laboratory of Minority Speech and Language Information Processing, Urumqi Xinjiang 830011, China
Abstract：Focused on the issue that lexicalized reordering models are faced with context independence and sparsity problems in machine translation, a reordering table reconstruction model based on semantic content for reordering orientation and probability prediction was proposed. Firstly, continuous distributed representation approach was employed to acquire the feature vectors of reordering rules. Secondly, Recurrent Neural Networks (RNN) were utilized to predict the reordering orientation and probability of each reordering rule that represented with dense vectors. Finally, the original reordering table was filtered and reconstructed with more reasonable reordering probability distribution for the purpose of improving the reordering information accuracy in reordering model as well as reducing the size of the reordering table to speed up subsequent decoding process. The experimental results show that the reordering table reconstruction model can provide BLEU point gains (+0.39) for Chinese to Uyghur machine translation task.
潘一荣, 李晓, 杨雅婷, 米成刚, 董瑞. 面向汉维机器翻译的调序表重构模型[J]. 计算机应用, 2018, 38(5): 1283-1288.
PAN Yirong, LI Xiao, YANG Yating, MI Chenggang, DONG Rui. Reordering table reconstruction model for Chinese-Uyghur machine translation. Journal of Computer Applications, 2018, 38(5): 1283-1288.
[1] TILLMANN C. A unigram orientation model for statistical machine translation[C]//Proceedings of the 2004 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg, PA:Association for Computational Linguistics, 2004:101-104. [2] GILDEA D, KHUDANPUR S, SARKAR A, et al. A smorgasbord of features for statistical machine translation[C]//Proceedings of the 2004 Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2004:161-168. [3] LI P, LIU Y, SUN M, et al. A neural reordering model for phrase-based translation[C]//Proceedings of the 2014 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg, PA:Association for Computational Linguistics, 2014:1897-1907. [4] 董兴华, 陈丽娟, 周喜, 等. 汉维统计机器翻译中的形态学处理[J]. 计算机工程, 2011, 37(12):150-152.(DONG X H, CHEN L J, ZHOU X, et al. Morphology processing in Chinese-Uyghur statistical machine translation[J]. Computer Engineering, 2011, 37(12):150-152.) [5] 陈丽娟, 张恒, 董兴华, 等. 基于句法调序的汉维统计机器翻译[J]. 计算机工程, 2012, 38(3):169-171.(CHEN L J, ZHANG H, DONG X H, et al. Chinese-Uyghur statistical machine translation based on syntactical reordering[J]. Computer Engineering, 2012, 38(3):169-171.) [6] 孔金英, 李晓, 王磊, 等. 调序规则表的深度过滤研究[J]. 计算机科学与探索, 2017, 11(5):785-793.(KONG J Y, LI X, WANG L, et al. Research of deep filtering lexical reordering table[J]. Journal of Frontiers of Computer Science and Technology, 2017, 11(5):785-793.) [7] 杨南, 李沐. 基于神经网络的统计机器翻译的预调序模型[J]. 中文信息学报, 2016, 30(3):103-110.(YANG N, LI M. A neural pre-reordering model for statistical machine translation[J]. Journal of Chinese Information Processing, 2016, 30(3):103-110.) [8] LI P, LIU Y, SUN M. Recursive autoencoders for ITG-based translation[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. New York:ACM, 2013:567-577. [9] GREEN S, GALLEY M, MANNING C D. Improved models of distortion cost for statistical machine translation[C]//Proceedings of the 2010 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg, PA:Association for Computational Linguistics, 2010:867-875. [10] NGUYEN V V, NGUYEN T P, NGUYEN M L, et al. A model lexicalized hierarchical reordering for phrase based translation[J]. Procedia-Social and Behavioral Sciences, 2011, 27:77-85. [11] HADIWINOTO C, NG H T. A dependency-based neural reordering model for statistical machine translation[C]//Proceedings of the 2017 Conference on Artificial Intelligence. Menlo Park, CA:AAAI Press, 2017:109-115. [12] SIVIC J, ZISSERMAN A. Efficient visual search of videos cast as text retrieval[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2009, 31(4):591-606. [13] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[EB/OL].[2017-05-10]. https://arxiv.org/abs/1301.3781. [14] MIKOLOV T, YIH W, ZWEIG G. Linguistic regularities in continuous space word representations[C]//Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg, PA:Association for Computational Linguistics, 2013:746-751. [15] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[EB/OL].[2017-05-10]. https://arxiv.org/abs/1310.4546. [16] GREFENSTETTE E, DINU G, ZHANG Y Z, et al. Multi-step regression learning for compositional distributional semantics[EB/OL].[2017-05-10]. http://www.anthology.aclweb.org/W/W13/W13-0112.pdf. [17] LE Q V, MIKOLOV T. Distributed representations of sentences and documents[EB/OL].[2017-05-10]. http://proceedings.mlr.press/v32/le14.pdf. [18] GRAVES A, MOHAMED A R, HINTON G. Speech recognition with deep recurrent neural networks[C]//Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway, NJ:IEEE, 2013:6645-6649. [19] GRAVES A. Generating sequences with recurrent neural networks[EB/OL].[2017-05-10]. https://arxiv.org/abs/1308.0850. [20] FINKEL J R, GRENAGER T, MANNING C. Incorporating non-local information into information extraction systems by Gibbs sampling[C]//Proceedings of the 2005 Conference of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2005:363-370. [21] KOEHN P, HOANG H, BIRCH A, et al. Moses:open source toolkit for statistical machine translation[C]//Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions. Stroudsburg, PA:Association for Computational Linguistics, 2007:177-180. [22] OCH F J. GIZA++:Training of statistical translation models[EB/OL].[2017-05-10]. https://www.prhlt.upv.es/~evidal/students/doct/sht/transp/giza2p.pdf. [23] STOLCKE A. SRILM-an extensible language modeling toolkit[EB/OL].[2017-05-10]. http://isca-speech.org/archive/archive_papers/icslp_2002/i02_0901.pdf. [24] PAPINENI K, ROUKOS S, WARD T, et al. BLEU:a method for automatic evaluation of machine translation[C]//Proceedings of the 2002 Conference of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2002:311-318. [25] HUANG X, ALLEVA F, HON H, et al. The SPHINX-Ⅱ speech recognition system:an overview[J]. Computer Speech & Language, 1993, 7(2):137-148. [26] MORIN F, BENGIO Y. Hierarchical probabilistic neural network language model[EB/OL].[2017-05-10]. http://www.iro.umontreal.ca/~lisa/pointeurs/hierarchical-nnlm-aistats05.pdf. [27] YANMIN S, ANDREW K C W, MOHAMED S K. Classification of imbalanced data:a review[J]. International Journal of Pattern Recognition & Artificial Intelligence, 2009, 23(4):687-719. [28] LIANG G, ZHANG C. An efficient and simple under-sampling technique for imbalanced time series classification[C]//CIKM 2012:Proceedings of the 21st ACM International Conference on Information and Knowledge Management. New York:ACM, 2012:2339-2342. [29] YU D J, HU J, TANG Z M, et al. Improving protein-ATP binding residues prediction by boosting SVMs with random under-sampling[J]. Neurocomputing, 2013, 104:180-190.
2018, Vol. 38 
