mlsanityData to 7 Oct 2026

Candidate Soups: Fusing Candidate Results Improves Translation Quality for Non-Autoregressive Translation

arXiv 2301.11503 ↗First posted 27 Jan 2023cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

Non-autoregressive translation (NAT) model achieves a much faster inference speed than the autoregressive translation (AT) model because it can simultaneously predict all tokens during inference. However, its translation quality suffers from degradation compared to AT. And existing NAT methods only focus on improving the NAT model's performance but do not fully utilize it. In this paper, we propose a simple but effective method called "Candidate Soups," which can obtain high-quality translations while maintaining the inference speed of NAT models. Unlike previous approaches that pick the individual result and discard the remainders, Candidate Soups (CDS) can fully use the valuable information in the different candidate translations through model uncertainty. Extensive experiments on two benchmarks (WMT'14 EN-DE and WMT'16 EN-RO) demonstrate the effectiveness and generality of our proposed method, which can significantly improve the translation quality of various base models. More notably, our best variant outperforms the AT model on three translation tasks with 7.6 times speedup.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 31 of its 42 references that are in the map. First posted on arXiv 27 Jan 2023.

Builds on

31 of 42 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  6. 2017Non-Autoregressive Neural Machine Translationcited by 457
  7. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  8. 2019Levenshtein Transformercited by 197
  9. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
  10. 2019The Evolved Transformercited by 180
  11. 2019Insertion Transformer: Flexible Sequence Generation via Insertion Operationscited by 123
  12. 2019Understanding Knowledge Distillation in Non-autoregressive Machine Translationcited by 121
19 more
  1. 2020Glancing Transformer for Non-Autoregressive Neural Machine Translationcited by 103
  2. 2020Fully Non-autoregressive Neural Machine Translation: Tricks of the Tradecited by 99
  3. 2020Non-Autoregressive Machine Translation with Latent Alignmentscited by 96
  4. 2019FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flowcited by 89
  5. 2018End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classificationcited by 72
  6. 2020Aligned Cross Entropy for Non-Autoregressive Machine Translationcited by 65
  7. 2019Fast Structured Decoding for Sequence Modelscited by 59
  8. 2019Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posteriorcited by 58
  9. 2021Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 57
  10. 2018Semi-Autoregressive Neural Machine Translationcited by 48
  11. 2021Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervisioncited by 40
  12. 2019Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Informationcited by 37
  13. 2020ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translationcited by 34
  14. 2020Semi-Autoregressive Training Improves Mask-Predict Decodingcited by 33
  15. 2022A Simple Hash-Based Early Exiting Approach For Language Understanding and Generationcited by 32
  16. 2021AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translatecited by 24
  17. 2020Learning to Recover from Multi-Modality Errors for Non-Autoregressive Neural Machine Translationcited by 19
  18. 2020Multi-Task Learning with Shared Encoder for Non-Autoregressive Machine Translationcited by 16
  19. 2020AutoTrans: Automating Transformer Design via Reinforced Architecture Searchcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 3; 2026: 5.

13 papers.

Sort:

By area:

  1. 2023PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domaincited by 40
  2. 2024ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Modelscited by 36
  3. 2024MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuningcited by 19
  4. 2025AMAS: Adaptively Determining Communication Topology for LLM-based Multi-Agent Systemcited by 15
  5. 2023Overview of the PromptCBLUE Shared Task in CHIP2023cited by 14
  6. 2025PARA: Parameter-Efficient Fine-tuning with Prompt Aware Representation Adjustmentcited by 9
  7. 2025FT-MDT: Extracting Decision Trees from Medical Texts via a Novel Low-rank Adaptation Methodcited by 7
  8. 2026MRAG: Benchmarking Retrieval-Augmented Generation for Bio-medicinecited by 4
  9. 2024IAPT: Instruction-Aware Prompt Tuning for Large Language Modelscited by 1
  10. 2026Evaluating ChatGPT on Medical Information Extraction Tasks: Performance, Explainability and Beyondcited by 1
  11. 2026Pursuing Best Industrial Practices for Retrieval-Augmented Generation in the Medical Domaincited by 1
  12. 2026ACL: Aligned Contrastive Learning Improves BERT and Multi-exit BERT Fine-tuningcited by 1
1 more
  1. 2026A Generative Model for Joint Multiple Intent Detection and Slot Fillingcited by 0