mlsanityData to 14 Oct 2026

Rephrasing the Reference for Non-Autoregressive Machine Translation

arXiv 2211.16863 ↗First posted 30 Nov 2022cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Non-autoregressive neural machine translation (NAT) models suffer from the multi-modality problem that there may exist multiple possible translations of a source sentence, so the reference sentence may be inappropriate for the training when the NAT output is closer to other translations. In response to this problem, we introduce a rephraser to provide a better training target for NAT by rephrasing the reference sentence according to the NAT output. As we train NAT based on the rephraser output rather than the reference sentence, the rephraser output should fit well with the NAT output and not deviate too far from the reference, which can be quantified as reward functions and optimized by reinforcement learning. Experiments on major WMT benchmarks and NAT baselines show that our approach consistently improves the translation quality of NAT. Specifically, our best variant achieves comparable performance to the autoregressive Transformer, while being 14.7 times more efficient in inference.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 44 of its 51 references that are in the map. First posted on arXiv 30 Nov 2022.

Builds on

44 of 51 references.

References · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  6. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  7. 2016Sequence-Level Knowledge Distillationcited by 801
  8. 2015Sequence Level Training with Recurrent Neural Networkscited by 576
  9. 2017Convolutional Sequence to Sequence Learningcited by 552
  10. 2017Non-Autoregressive Neural Machine Translationcited by 463
  11. 2016An Actor-Critic Algorithm for Sequence Predictioncited by 159
  12. 2019Understanding Knowledge Distillation in Non-autoregressive Machine Translationcited by 122
32 more
  1. 2018Fast Decoding in Sequence Models using Discrete Latent Variablescited by 104
  2. 2020Glancing Transformer for Non-Autoregressive Neural Machine Translationcited by 103
  3. 2020Fully Non-autoregressive Neural Machine Translation: Tricks of the Tradecited by 99
  4. 2020Non-Autoregressive Machine Translation with Latent Alignmentscited by 96
  5. 2019FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flowcited by 89
  6. 2013The Optimal Reward Baseline for Gradient-Based Reinforcement Learningcited by 85
  7. 2018A Study of Reinforcement Learning for Neural Machine Translationcited by 74
  8. 2018End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classificationcited by 72
  9. 2020Aligned Cross Entropy for Non-Autoregressive Machine Translationcited by 65
  10. 2019Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posteriorcited by 58
  11. 2021Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 57
  12. 2019Non-Autoregressive Machine Translation with Auxiliary Regularizationcited by 52
  13. 2022Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 44
  14. 2019Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translationcited by 40
  15. 2021Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervisioncited by 40
  16. 2019Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Informationcited by 37
  17. 2020ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translationcited by 34
  18. 2021Non-Autoregressive Translation by Learning Target Categorical Codescited by 25
  19. 2021AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translatecited by 24
  20. 2017Trainable Greedy Decoding for Neural Machine Translationcited by 23
  21. 2020An EM Approach to Non-autoregressive Conditional Sequence Generationcited by 21
  22. 2022One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translationcited by 20
  23. 2019Syntactically Supervised Transformers for Faster Neural Machine Translationcited by 19
  24. 2021Sequence-Level Training for Non-Autoregressive Neural Machine Translationcited by 18
  25. 2019Non-autoregressive Transformer by Position Learningcited by 17
  26. 2020Human-Paraphrased References Improve Neural Machine Translationcited by 16
  27. 2022Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translationcited by 16
  28. 2019Retrieving Sequential Information for Non-Autoregressive Neural Machine Translationcited by 15
  29. 2021Enriching Non-Autoregressive Transformer with Syntactic and SemanticStructures for Neural Machine Translationcited by 13
  30. 2022Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 13
  31. 2021Duplex Sequence-to-Sequence Learning for Reversible Machine Translationcited by 9
  32. 2018Paraphrases as Foreign Languages in Multilingual Neural Machine Translationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 0; 2025: 0; 2026: 1.

3 papers.

  1. 2023Simultaneous Machine Translation with Tailored Referencecited by 9
  2. 2026FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation--Full Versioncited by 2
  3. 2023Optimizing Non-Autoregressive Transformers with Contrastive Learningcited by 1