mlsanityData to 7 Oct 2026

Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions

arXiv 2305.15083 ↗First posted 24 May 2023cs.CLcited by 55

Not on the map. Why?

Abstract (arXiv)

Large-scale Pretrained Language Models (LLMs), such as ChatGPT and GPT4, have shown strong abilities in multilingual translations, without being explicitly trained on parallel corpora. It is interesting how the LLMs obtain their ability to carry out translation instructions for different languages. In this paper, we present a detailed analysis by finetuning a multilingual pretrained language model, XGLM-7B, to perform multilingual translation following given instructions. Firstly, we show that multilingual LLMs have stronger translation abilities than previously demonstrated. For a certain language, the performance depends on its similarity to English and the amount of data used in the pretraining phase. Secondly, we find that LLMs' ability to carry out translation instructions relies on the understanding of translation instructions and the alignment among different languages. With multilingual finetuning, LLMs could learn to perform the translation task well even for those language pairs unseen during the instruction tuning phase.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Builds on 32 of its 44 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

32 of 44 references.

References · 32

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  8. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  9. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  10. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  11. 2023LIMA: Less Is More for Alignmentcited by 1,080
  12. 2020Language-agnostic BERT Sentence Embeddingcited by 621
20 more
  1. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  2. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  3. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  4. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  5. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  6. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 247
  7. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
  8. 2022In-context Examples Selection for Machine Translationcited by 172
  9. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 172
  10. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 156
  11. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 144
  12. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  13. 2022Broken Neural Scaling Lawscited by 102
  14. 2023The unreasonable effectiveness of few-shot learning for machine translationcited by 91
  15. 2019Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlationscited by 61
  16. 2019The Missing Ingredient in Zero-Shot Neural Machine Translationcited by 59
  17. 2023ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedbackcited by 34
  18. 2020Multilingual BERT Post-Pretraining Alignmentcited by 30
  19. 2019Consistency by Agreement in Zero-shot Neural Machine Translationcited by 22
  20. 2021Improving Pretrained Models for Zero-shot Multi-label Text Classification through Reinforced Label Hierarchy Reasoningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 25; 2025: 17; 2026: 5.

55 papers.

Sort:

By area:

  1. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
  2. 2024Large Language Models Meet NLP: A Surveycited by 102
  3. 2023A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Modelscited by 88
  4. 2023Extrapolating Large Language Models to Non-English by Aligning Languagescited by 69
  5. 2023Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpacacited by 60
  6. 2024Scaling Laws for Downstream Task Performance of Large Language Modelscited by 49
  7. 2023Steering Large Language Models for Machine Translation with Finetuning and In-Context Learningcited by 48
  8. 2024Fine-tuning Large Language Models for Domain-specific Machine Translationcited by 43
  9. 2025Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Studycited by 38
  10. 2023Empowering Cross-lingual Abilities of Instruction-tuned Large Language Models by Translation-following demonstrationscited by 34
  11. 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 32
  12. 2023CodeCoT: Tackling Code Syntax Errors in CoT Reasoning for Code Generationcited by 29
38 more
  1. 2024G-DIG: Towards Gradient-based Diverse and High-quality Instruction Data Selection for Machine Translationcited by 29
  2. 2024Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowlycited by 27
  3. 2024The Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilitiescited by 22
  4. 2024Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixturecited by 22
  5. 2024The Price of Prompting: Profiling Energy Use in Large Language Models Inferencecited by 22
  6. 2024URIEL+: Enhancing Linguistic Inclusion and Usability in a Typological and Multilingual Knowledge Basecited by 21
  7. 2025Seed-X: Building Strong Multilingual Translation LLM with 7B Parameterscited by 21
  8. 2024Multilingual Large Language Models: A Systematic Surveycited by 18
  9. 2024How Much Data is Enough Data? Fine-Tuning Large Language Models for In-House Translation: Performance Evaluation Across Multiple Dataset Sizescited by 17
  10. 2024Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice?cited by 16
  11. 2024Towards Achieving Human Parity on End-to-end Simultaneous Speech Translation via LLM Agentcited by 14
  12. 2023How Vocabulary Sharing Facilitates Multilingualism in LLaMA?cited by 12
  13. 2024Ladder: A Model-Agnostic Framework Boosting LLM-based Machine Translation to the Next Levelcited by 11
  14. 2025Seed LiveInterpret 2.0: End-to-end Simultaneous Speech-to-speech Translation with Your Voicecited by 9
  15. 2024Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translationcited by 8
  16. 2025SinLlama -- A Large Language Model for Sinhalacited by 8
  17. 2025DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimizationcited by 7
  18. 2023On-the-Fly Fusion of Large Language Models and Machine Translationcited by 5
  19. 2024Towards Boosting Many-to-Many Multilingual Machine Translation with Large Language Modelscited by 5
  20. 2024Disce aut Deficere: Evaluating LLMs Proficiency on the INVALSI Italian Benchmarkcited by 5
  21. 2025Provoking Multi-modal Few-Shot LVLM via Exploration-Exploitation In-Context Learningcited by 5
  22. 2024Choose the Final Translation from NMT and LLM hypotheses Using MBR Decoding: HW-TSC's Submission to the WMT24 General MT Shared Taskcited by 4
  23. 2025Understanding LLMs' Cross-Lingual Context Retrieval: How Good It Is And Where It Comes Fromcited by 4
  24. 2025IRLBench: A Multi-modal, Culturally Grounded, Parallel Irish-English Benchmark for Open-Ended LLM Reasoning Evaluationcited by 4
  25. 2026ADEPT: Adaptive Dynamic Early-Exit Process for Transformerscited by 4
  26. 2024Language Modelling Approaches to Adaptive Machine Translationcited by 3
  27. 2024KS-Lottery: Finding Certified Lottery Tickets for Multilingual Language Modelscited by 3
  28. 2026Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translationcited by 3
  29. 2024POMP: Probability-driven Meta-graph Prompter for LLMs in Low-resource Unsupervised Neural Machine Translationcited by 2
  30. 2024Towards Chapter-to-Chapter Context-Aware Literary Translation via Large Language Modelscited by 2
  31. 2025Translation Analytics for Freelancers: I. Introduction, Data Preparation, Baseline Evaluationscited by 2
  32. 2025Fine-Tuning LLMs for Low-Resource Dialect Translation: The Case of Lebanesecited by 2
  33. 2025Investigating Numerical Translation with Large Language Modelscited by 1
  34. 2025BadLingual: A Novel Lingual-Backdoor Attack against Large Language Modelscited by 1
  35. 2025FRAME: Feedback-Refined Agent Methodology for Enhancing Medical Research Insightscited by 1
  36. 2025The Effect of Language Diversity When Fine-Tuning Large Language Models for Translationcited by 1
  37. 2024ACT-MNMT Auto-Constriction Turning for Multilingual Neural Machine Translationcited by 0
  38. 2024Evaluating the Efficacy of Foundational Models: Advancing Benchmarking Practices to Enhance Fine-Tuning Decision-Makingcited by 0

50 most cited of 55 All 55 (JSON)