mlsanityData to 7 Oct 2026

Semi-Autoregressive Training Improves Mask-Predict Decoding

arXiv 2001.08785 ↗First posted 23 Jan 2020cs.CL cs.LG stat.MLcited by 33

Not on the map. Why?

Abstract (arXiv)

The recently proposed mask-predict decoding algorithm has narrowed the performance gap between semi-autoregressive machine translation models and the traditional left-to-right approach. We introduce a new training method for conditional masked language models, SMART, which mimics the semi-autoregressive behavior of mask-predict, producing training examples that contain model predictions as part of their inputs. Models trained with SMART produce higher-quality translations when using mask-predict decoding, effectively closing the remaining performance gap with fully autoregressive models.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 8; 2022: 9; 2023: 12; 2024: 3; 2025: 1.

33 papers.

Sort:

By area:

  1. 2022Continuous diffusion for categorical datacited by 202
  2. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  3. 2023Accelerating Transformer Inference for Translation via Parallel Decodingcited by 123
  4. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  5. 2023Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systemscited by 101
  6. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  7. 2023DINOISER: Diffused Conditional Sequence Learning by Manipulating Noisescited by 50
  8. 2022Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 44
  9. 2021Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translationcited by 27
  10. 2022Lossless Acceleration for Seq2seq Generation with Aggressive Decodingcited by 18
  11. 2021Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialogcited by 17
  12. 2023Candidate Soups: Fusing Candidate Results Improves Translation Quality for Non-Autoregressive Translationcited by 13
21 more
  1. 2023Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translationcited by 11
  2. 2021Duplex Sequence-to-Sequence Learning for Reversible Machine Translationcited by 9
  3. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  4. 2022Uncertainty-Aware Image Captioningcited by 6
  5. 2024Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelinescited by 6
  6. 2021Towards Variable-Length Textual Adversarial Attackscited by 5
  7. 2021MvSR-NAT: Multi-view Subset Regularization for Non-Autoregressive Machine Translationcited by 5
  8. 2023AMOM: Adaptive Masking over Masking for Conditional Masked Language Modelcited by 5
  9. 2023Non-autoregressive Machine Translation with Probabilistic Context-free Grammarcited by 4
  10. 2025Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMscited by 4
  11. 2021Can Latent Alignments Improve Autoregressive Machine Translation?cited by 3
  12. 2022Improving Top-K Decoding for Non-Autoregressive Semantic Parsing via Intent Conditioningcited by 3
  13. 2022Hybrid-Regressive Neural Machine Translationcited by 3
  14. 2022Diffusion Glancing Transformer for Parallel Sequence to Sequence Learningcited by 3
  15. 2023Dynamic Alignment Mask CTC: Improved Mask-CTC with Aligned Cross Entropycited by 2
  16. 2023Optimizing Non-Autoregressive Transformers with Contrastive Learningcited by 1
  17. 2023Encoder-decoder multimodal speaker change detectioncited by 1
  18. 2023Emage: Non-Autoregressive Text-to-Image Generationcited by 1
  19. 2022DiMS: Distilling Multiple Steps of Iterative Non-Autoregressive Transformers for Machine Translationcited by 0
  20. 2024Improving Non-autoregressive Machine Translation with Error Exposure and Consistency Regularizationcited by 0
  21. 2024Leveraging Diverse Modeling Contexts with Collaborating Learning for Neural Machine Translationcited by 0