mlsanityData to 14 Oct 2026

Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decoding

arXiv 2311.02851 ↗First posted 6 Nov 2023cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

Contemporary translation engines based on the encoder-decoder framework have made significant strides in development. However, the emergence of Large Language Models (LLMs) has disrupted their position by presenting the potential for achieving superior translation quality. To uncover the circumstances in which LLMs excel and explore how their strengths can be harnessed to enhance translation quality, we first conduct a comprehensive analysis to assess the strengths and limitations of various commercial NMT systems and MT-oriented LLMs. Our findings indicate that neither NMT nor MT-oriented LLMs alone can effectively address all the translation issues, but MT-oriented LLMs show promise as a complementary solution to NMT systems. Building upon these insights, we propose Cooperative Decoding (CoDec), which treats NMT systems as a pretranslation model and MT-oriented LLMs as a supplemental solution to handle complex scenarios beyond the capability of NMT alone. Experimental results on the WMT22 test sets and a newly collected test set WebCrawl demonstrate the effectiveness and efficiency of CoDec, highlighting its potential as a robust solution for combining NMT systems with MT-oriented LLMs in the field of machine translation.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 34 of its 50 references that are in the map. First posted on arXiv 6 Nov 2023.

Builds on

34 of 50 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  7. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  8. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  9. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  10. 2020Efficient Transformers: A Surveycited by 790
  11. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  12. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 482
22 more
  1. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 439
  2. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 359
  3. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  4. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  5. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  6. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  7. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
  8. 2022In-context Examples Selection for Machine Translationcited by 172
  9. 2020Detecting Hallucinated Content in Conditional Neural Sequence Generationcited by 152
  10. 2023Accelerating Transformer Inference for Translation via Parallel Decodingcited by 124
  11. 2020BLEU might be Guilty but References are not Innocentcited by 110
  12. 2023The unreasonable effectiveness of few-shot learning for machine translationcited by 91
  13. 2020Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Searchcited by 76
  14. 2023Inference with Reference: Lossless Acceleration of Large Language Modelscited by 65
  15. 2021A Survey on Green Deep Learningcited by 52
  16. 2023TIM: Teaching Large Language Models to Translate with Comparisoncited by 50
  17. 2023BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Modelscited by 42
  18. 2021When Attention Meets Fast Recurrence: Training Language Models with Reduced Computecited by 34
  19. 2020Complete Multilingual Neural Machine Translationcited by 30
  20. 2023Do GPTs Produce Less Literal Translations?cited by 30
  21. 2023Chain-of-Dictionary Prompting Elicits Translation in Large Language Modelscited by 16
  22. 2023On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translationcited by 3

Built on by

Cited by year

Citing papers per year: 2024: 5; 2025: 5.

10 papers.

  1. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  2. 2024G-DIG: Towards Gradient-based Diverse and High-quality Instruction Data Selection for Machine Translationcited by 30
  3. 2025Evaluating o1-Like LLMs: Unlocking Reasoning for Translation through Comprehensive Analysiscited by 26
  4. 2024Guiding Large Language Models to Post-Edit Machine Translation with Error Annotationscited by 22
  5. 2025Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translationcited by 8
  6. 2024LLM-based Translation Inference with Iterative Bilingual Understandingcited by 6
  7. 2024Make Imagination Clearer! Stable Diffusion-based Visual Imagination for Multimodal Machine Translationcited by 5
  8. 2025Automatic Input Rewriting Improves Translation with Large Language Modelscited by 4
  9. 2025Multi-Amateur Contrastive Decoding for Text Generationcited by 3
  10. 2025Combining the Best of Both Worlds: A Method for Hybrid NMT and LLM Translationcited by 2