mlsanityData to 7 Oct 2026

Bridging the Gap between Training and Inference for Neural Machine Translation

arXiv 1906.02448 ↗First posted 6 Jun 2019cs.CL cs.LG stat.MLcited by 123

Not on the map. Why?

Abstract (arXiv)

Neural Machine Translation (NMT) generates target words sequentially in the way of predicting the next word conditioned on the context words. At training time, it predicts with the ground truth words as context while at inference it has to generate the entire sequence from scratch. This discrepancy of the fed context leads to error accumulation among the way. Furthermore, word-level training requires strict matching between the generated sequence and the ground truth sequence which leads to overcorrection over different but reasonable translations. In this paper, we address these issues by sampling context words not only from the ground truth sequence but also from the predicted sequence by the model during training, where the predicted sequence is selected with a sentence-level optimum. Experiment results on Chinese->English and WMT'14 English->German translation tasks demonstrate that our approach can achieve significant improvements on multiple datasets.

Key facts. Cited by 123 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 3; 2021: 32; 2022: 28; 2023: 31; 2024: 12; 2025: 8; 2026: 9.

123 papers.

Sort:

By area:

  1. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  2. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 236
  3. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  4. 2024DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Trainingcited by 137
  5. 2022Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generationcited by 106
  6. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 88
  7. 2020Autoregressive Knowledge Distillation through Imitation Learningcited by 68
  8. 2021Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-trainingcited by 67
  9. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  10. 2022STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translationcited by 57
  11. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
  12. 2023Learning to Generate Better Than Your LLMcited by 49
38 more
  1. 2026Stable On-Policy Distillation through Adaptive Target Reformulationcited by 47
  2. 2021Selective Knowledge Distillation for Neural Machine Translationcited by 42
  3. 2023Beyond Imitation: Leveraging Fine-grained Quality Signals for Alignmentcited by 35
  4. 2021Prevent the Language Model from being Overconfident in Neural Machine Translationcited by 34
  5. 2023On Error Propagation of Diffusion Modelscited by 30
  6. 2025Taming Teacher Forcing for Masked Autoregressive Video Generationcited by 29
  7. 2022Neural Machine Translation with Phrase-Level Universal Visual Representationscited by 26
  8. 2024PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuningcited by 24
  9. 2021KD-VLP: Improving End-to-End Vision-and-Language Pretraining with Object Knowledge Distillationcited by 23
  10. 2021Tail-to-Tail Non-Autoregressive Sequence Prediction for Chinese Grammatical Error Correctioncited by 19
  11. 2021Relating Neural Text Degeneration to Exposure Biascited by 19
  12. 2021Sequence-Level Training for Non-Autoregressive Neural Machine Translationcited by 18
  13. 2021Modeling Bilingual Conversational Characteristics for Neural Chat Translationcited by 18
  14. 2021Rethinking Perturbations in Encoder-Decoders for Fast Trainingcited by 17
  15. 2021Discontinuous Named Entity Recognition as Maximal Clique Discoverycited by 17
  16. 2022GL-RG: Global-Local Representation Granularity for Video Captioningcited by 16
  17. 2021Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translationcited by 15
  18. 2022MSCTD: A Multimodal Sentiment Chat Translation Datasetcited by 15
  19. 2022Go Back in Time: Generating Flashbacks in Stories with Event Temporal Promptscited by 15
  20. 2023Understanding and Bridging the Modality Gap for Speech Translationcited by 15
  21. 2025VideoMAR: Autoregressive Video Generatio with Continuous Tokenscited by 15
  22. 2021The Volctrans GLAT System: Non-autoregressive Translation Meets WMT21cited by 14
  23. 2022Recent Advances in Neural Text Generation: A Task-Agnostic Surveycited by 14
  24. 2021Towards Making the Most of Dialogue Characteristics for Neural Chat Translationcited by 12
  25. 2020Motion Generation Using Bilateral Control-Based Imitation Learning with Autoregressive Learningcited by 11
  26. 2021WeChat Neural Machine Translation Systems for WMT21cited by 11
  27. 2021Scheduled Sampling Based on Decoding Steps for Neural Machine Translationcited by 11
  28. 2026SimCT: Recovering Lost Supervision for Cross-Tokenizer On-Policy Distillationcited by 11
  29. 2020Learning Coupled Policies for Simultaneous Machine Translation using Imitation Learningcited by 10
  30. 2021Confidence-Aware Scheduled Sampling for Neural Machine Translationcited by 10
  31. 2022Nearest Neighbor Knowledge Distillation for Neural Machine Translationcited by 10
  32. 2023Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!cited by 10
  33. 2023Towards Understanding and Improving Knowledge Distillation for Neural Machine Translationcited by 10
  34. 2023Large-Scale and Multi-Perspective Opinion Summarization with Diverse Review Subsetscited by 10
  35. 2024Retrieval is Accurate Generationcited by 10
  36. 2021Modulating Language Models with Emotionscited by 9
  37. 2021Guiding Teacher Forcing with Seer Forcing for Neural Machine Translationcited by 8
  38. 2021Mixed Cross Entropy Loss for Neural Machine Translationcited by 8

50 most cited of 123 All 123 (JSON)