# End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification

arXiv: https://arxiv.org/abs/1811.04719 · HTML: https://mlsanity.com/paper/1811.04719/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2018.

## Built on by (most cited first, 30 of 72)

- [Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation](https://mlsanity.com/paper/2203.16487/) (2022)
- [A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond](https://mlsanity.com/paper/2204.09269/) (2022)
- [Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2106.05093/) (2021)
- [Directed Acyclic Transformer for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2205.07459/) (2022)
- [A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models](https://mlsanity.com/paper/2508.08712/) (2025)
- [A Comparative Study on Non-Autoregressive Modelings for Speech-to-Text Generation](https://mlsanity.com/paper/2110.05249/) (2021)
- [On the Learning of Non-Autoregressive Transformers](https://mlsanity.com/paper/2206.05975/) (2022)
- [Revisiting End-to-End Speech-to-Text Translation From Scratch](https://mlsanity.com/paper/2206.04571/) (2022)
- [CTC Alignments Improve Autoregressive Translation](https://mlsanity.com/paper/2210.05200/) (2022)
- [Investigating the Reordering Capability in CTC-based Non-Autoregressive End-to-End Speech Translation](https://mlsanity.com/paper/2105.04840/) (2021)
- [One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation](https://mlsanity.com/paper/2205.14333/) (2022)
- [Sequence-Level Training for Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/2106.08122/) (2021)
- [Lossless Acceleration for Seq2seq Generation with Aggressive Decoding](https://mlsanity.com/paper/2205.10350/) (2022)
- [Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2210.03953/) (2022)
- [Enriching Non-Autoregressive Transformer with Syntactic and SemanticStructures for Neural Machine Translation](https://mlsanity.com/paper/2101.08942/) (2021)
- [Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2210.05193/) (2022)
- [Candidate Soups: Fusing Candidate Results Improves Translation Quality for Non-Autoregressive Translation](https://mlsanity.com/paper/2301.11503/) (2023)
- [Non-autoregressive Streaming Transformer for Simultaneous Translation](https://mlsanity.com/paper/2310.14883/) (2023)
- [Streaming End-to-End ASR based on Blockwise Non-Autoregressive Models](https://mlsanity.com/paper/2107.09428/) (2021)
- [Non-Autoregressive Machine Translation: It's Not as Fast as it Seems](https://mlsanity.com/paper/2205.01966/) (2022)
- [Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2303.06662/) (2023)
- [ToolACE-MT: Non-Autoregressive Generation for Agentic Multi-Turn Interaction](https://mlsanity.com/paper/2508.12685/) (2025)
- [ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation](https://mlsanity.com/paper/2210.13304/) (2022)
- [Duplex Sequence-to-Sequence Learning for Reversible Machine Translation](https://mlsanity.com/paper/2105.03458/) (2021)
- [Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoring](https://mlsanity.com/paper/2109.04411/) (2021)
- [LegoNN: Building Modular Encoder-Decoder Models](https://mlsanity.com/paper/2206.03318/) (2022)
- [SoftCorrect: Error Correction with Soft Detection for Automatic Speech Recognition](https://mlsanity.com/paper/2212.01039/) (2022)
- [CTC-based Non-autoregressive Speech Translation](https://mlsanity.com/paper/2305.17358/) (2023)
- [DASpeech: Directed Acyclic Transformer for Fast and High-quality Speech-to-Speech Translation](https://mlsanity.com/paper/2310.07403/) (2023)
- [Simultaneous Machine Translation with Tailored Reference](https://mlsanity.com/paper/2310.13588/) (2023)

## Abstract (arXiv)

Autoregressive decoding is the only part of sequence-to-sequence models that prevents them from massive parallelization at inference time. Non-autoregressive models enable the decoder to generate all output symbols independently in parallel. We present a novel non-autoregressive architecture based on connectionist temporal classification and evaluate it on the task of neural machine translation. Unlike other non-autoregressive methods which operate in several steps, our model can be trained end-to-end. We conduct experiments on the WMT English-Romanian and English-German datasets. Our models achieve a significant speedup over the autoregressive models, keeping the translation quality comparable to other non-autoregressive models.