mlsanityData to 7 Oct 2026

Non-Autoregressive Translation by Learning Target Categorical Codes

arXiv 2103.11405 ↗First posted 21 Mar 2021cs.CL cs.AIcited by 25

Not on the map. Why?

Abstract (arXiv)

Non-autoregressive Transformer is a promising text generation model. However, current non-autoregressive models still fall behind their autoregressive counterparts in translation quality. We attribute this accuracy gap to the lack of dependency modeling among decoder inputs. In this paper, we propose CNAT, which learns implicitly categorical codes as latent variables into the non-autoregressive decoding. The interaction among these categorical codes remedies the missing dependencies and improves the model capacity. Experiment results show that our model achieves comparable or better performance in machine translation tasks, compared with several strong baselines.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 19 of its 29 references that are in the map. First posted on arXiv 21 Mar 2021.

Builds on

19 of 29 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  6. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  7. 2017Non-Autoregressive Neural Machine Translationcited by 457
  8. 2016Deep Biaffine Attention for Neural Dependency Parsingcited by 334
  9. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
  10. 2018Fast Decoding in Sequence Models using Discrete Latent Variablescited by 103
  11. 2020Glancing Transformer for Non-Autoregressive Neural Machine Translationcited by 103
  12. 2019FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flowcited by 89
7 more
  1. 2019Fast Structured Decoding for Sequence Modelscited by 59
  2. 2019Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posteriorcited by 58
  3. 2019Non-Autoregressive Machine Translation with Auxiliary Regularizationcited by 52
  4. 2018Non-Autoregressive Neural Machine Translation with Enhanced Decoder Inputcited by 39
  5. 2019Imitation Learning for Non-Autoregressive Neural Machine Translationcited by 36
  6. 2019Syntactically Supervised Transformers for Faster Neural Machine Translationcited by 19
  7. 2019Non-autoregressive Transformer by Position Learningcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 14; 2023: 3; 2024: 3; 2025: 2; 2026: 1.

25 papers.

Sort:

By area:

  1. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 169
  2. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 113
  3. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  4. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformerscited by 63
  5. 2022Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 44
  6. 2021Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervisioncited by 40
  7. 2022On the Learning of Non-Autoregressive Transformerscited by 27
  8. 2022textitlatent-GLAT: Glancing at Latent Variables for Parallel Text Generationcited by 20
  9. 2022One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translationcited by 20
  10. 2022Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translationcited by 16
  11. 2021DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformercited by 14
  12. 2022Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 13
13 more
  1. 2022Multi-Granularity Optimization for Non-Autoregressive Translationcited by 6
  2. 2022Non-Autoregressive Neural Machine Translation: A Call for Claritycited by 5
  3. 2023Non-autoregressive Machine Translation with Probabilistic Context-free Grammarcited by 4
  4. 2025Curriculum Learning for Biological Sequence Prediction: The Case of De Novo Peptide Sequencingcited by 4
  5. 2022Rephrasing the Reference for Non-Autoregressive Machine Translationcited by 3
  6. 2024Translating Images to Road Network: A Sequence-to-Sequence Perspectivecited by 3
  7. 2023Selective Knowledge Distillation for Non-Autoregressive Neural Machine Translationcited by 2
  8. 2022Helping the Weak Makes You Strong: Simple Multi-Task Learning Improves Non-Autoregressive Translatorscited by 1
  9. 2022Accelerating Antimicrobial Peptide Discovery with Latent Structurecited by 1
  10. 2024What Have We Achieved on Non-autoregressive Translation?cited by 1
  11. 2024Improving Non-autoregressive Machine Translation with Error Exposure and Consistency Regularizationcited by 0
  12. 2025Diffusion Is Your Friend in Show, Suggest and Tellcited by 0
  13. 2026Towards the Holographic Characteristic of LLMs for Efficient Short-text Generationcited by 0