mlsanityData to 7 Oct 2026

Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation

arXiv 2503.06594 ↗First posted 9 Mar 2025cs.CLcited by 8

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

The field of neural machine translation (NMT) has changed with the advent of large language models (LLMs). Much of the recent emphasis in natural language processing (NLP) has been on modeling machine translation and many other problems using a single pre-trained Transformer decoder, while encoder-decoder architectures, which were the standard in earlier NMT models, have received relatively less attention. In this paper, we explore translation models that are universal, efficient, and easy to optimize, by marrying the world of LLMs with the world of NMT. We apply LLMs to NMT encoding and leave the NMT decoder unchanged. We also develop methods for adapting LLMs to work better with the NMT decoder. Furthermore, we construct a new dataset involving multiple tasks to assess how well the machine translation system generalizes across various tasks. Evaluations on the WMT and our datasets show that results using our method match or surpass a range of baselines in terms of translation quality, but achieve 2.4 ~ 6.5 × inference speedups and a 75% reduction in the memory footprint of the KV cache. It also demonstrates strong generalization across a variety of translation-related tasks.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 84 of its 113 references that are in the map. First posted on arXiv 9 Mar 2025.

Builds on

84 of 113 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Towards Optimal Multi-draft Speculative Decoding26 Feb
  2. 2023Speculative Decoding with Big Little Decoder15 Feb
  3. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 81

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  8. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  10. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  11. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  12. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
38 more
  1. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  2. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  3. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  4. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  5. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  6. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
  7. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  8. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  9. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  10. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  11. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  12. 2023The Falcon Series of Open Language Modelscited by 526
  13. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  14. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  15. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  16. 2019Learning Deep Transformer Models for Machine Translationcited by 309
  17. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  18. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  19. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  20. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 262
  21. 2016Multi30K: Multilingual English-German Image Descriptionscited by 250
  22. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 247
  23. 2024Generative Representational Instruction Tuningcited by 227
  24. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  25. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
  26. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 156
  27. 2023A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Modelscited by 88
  28. 2023Large language models effectively leverage document-level context for literary translation, but critical errors persistcited by 80
  29. 2022Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answeringcited by 71
  30. 2020TICO-19: the Translation Initiative for Covid-19cited by 67
  31. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 60
  32. 2020MLQE-PE: A Multilingual Quality Estimation and Post-Editing Datasetcited by 47
  33. 2024Efficient Prompting Methods for Large Language Models: A Surveycited by 46
  34. 2023BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Modelscited by 42
  35. 2022QALD-9-plus: A Multilingual Dataset for Question Answering over DBpedia and Wikidata Translated by Native Speakerscited by 39
  36. 2022DEMETR: Diagnosing Evaluation Metrics for Translationcited by 36
  37. 2020A Set of Recommendations for Assessing Human-Machine Parity in Language Translationcited by 34
  38. 2023ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedbackcited by 34

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 2.

8 papers.

  1. 2025UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter-Efficient Fine-Tuning of Large Modelscited by 11
  2. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translationcited by 8
  3. 2025NiuTrans.LMT: Toward Inclusive and Scalable Multilingual Machine Translation with LLMscited by 4
  4. 2025VeriLoC: Line-of-Code Level Prediction of Hardware Design Quality from Verilog Codecited by 3
  5. 2026SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Modelscited by 2
  6. 2026RouteLMT: Learned Sample Routing for Hybrid LLM Translation Deploymentcited by 2
  7. 2025LaTeXTrans: Structured LaTeX Translation with Multi-Agent Coordinationcited by 1
  8. 2025Attention2Probability: Attention-Driven Terminology Probability Estimation for Robust Speech-to-Text Systemcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Investigating Decoder-only Large Language Models for Speech-to-text Translation
  2. 2024Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding
  3. 2025Calibrating Translation Decoding with Quality Estimation on LLMs
  4. 2024Choose the Final Translation from NMT and LLM hypotheses Using MBR Decoding: HW-TSC's Submission to the WMT24 General MT Shared Task
  5. 2024A Survey on Efficient Inference for Large Language Models
  6. 2024What do Large Language Models Need for Machine Translation Evaluation?
  7. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation
  8. 2024Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters