mlsanityData to 7 Oct 2026

Hint-Based Training for Non-Autoregressive Machine Translation

arXiv 1909.06708 ↗First posted 15 Sep 2019cs.CL cs.LG stat.MLcited by 28

Not on the map. Why?

Abstract (arXiv)

Due to the unparallelizable nature of the autoregressive factorization, AutoRegressive Translation (ART) models have to generate tokens sequentially during decoding and thus suffer from high inference latency. Non-AutoRegressive Translation (NART) models were proposed to reduce the inference time, but could only achieve inferior translation accuracy. In this paper, we proposed a novel approach to leveraging the hints from hidden states and word alignments to help the training of NART models. The results achieve significant improvement over previous NART models for the WMT14 En-De and De-En datasets and are even comparable to a strong LSTM-based ART baseline but one order of magnitude faster in inference.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 12; 2022: 5; 2023: 7; 2024: 4.

28 papers.

Sort:

By area:

  1. 2023Speculative Decoding with Big Little Decodercited by 163
  2. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  3. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  4. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  5. 2021Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 57
  6. 2024A Survey on Transformer Compressioncited by 44
  7. 2021FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognitioncited by 43
  8. 2021AlphaNet: Improved Training of Supernets with Alpha-Divergencecited by 36
  9. 2022One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translationcited by 20
  10. 2021Sequence-Level Training for Non-Autoregressive Neural Machine Translationcited by 18
  11. 2021Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generationcited by 17
  12. 2021Enriching Non-Autoregressive Transformer with Syntactic and SemanticStructures for Neural Machine Translationcited by 13
16 more
  1. 2021How Does Distilled Data Complexity Impact the Quality and Confidence of Non-Autoregressive Machine Translation?cited by 12
  2. 2024Optimized Multi-Token Joint Decoding with Auxiliary Model for LLM Inferencecited by 11
  3. 2021Duplex Sequence-to-Sequence Learning for Reversible Machine Translationcited by 9
  4. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  5. 2022ngram-OAXE: Phrase-Based Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 8
  6. 2023Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generationcited by 8
  7. 2023Non-Autoregressive Math Word Problem Solver with Unified Tree Structurecited by 7
  8. 2021MvSR-NAT: Multi-view Subset Regularization for Non-Autoregressive Machine Translationcited by 5
  9. 2022Non-Autoregressive Neural Machine Translation: A Call for Claritycited by 5
  10. 2021Modeling Coverage for Non-Autoregressive Neural Machine Translationcited by 4
  11. 2023Topic Shift Detection in Chinese Dialogues: Corpus and Benchmarkcited by 4
  12. 2022Hybrid-Regressive Neural Machine Translationcited by 3
  13. 2023Attentive Multi-Layer Perceptron for Non-autoregressive Generationcited by 0
  14. 2023Non-Autoregressive Sentence Orderingcited by 0
  15. 2023AMLNet: Adversarial Mutual Learning Neural Network for Non-AutoRegressive Multi-Horizon Time Series Forecastingcited by 0
  16. 2024NARRepair: Non-Autoregressive Code Generation Model for Automatic Program Repaircited by 0