mlsanityData to 7 Oct 2026

Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis

arXiv 2304.04675 ↗First posted 10 Apr 2023cs.CLcited by 156

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have demonstrated remarkable potential in handling multilingual machine translation (MMT). In this paper, we systematically investigate the advantages and challenges of LLMs for MMT by answering two questions: 1) How well do LLMs perform in translating massive languages? 2) Which factors affect LLMs' performance in translation? We thoroughly evaluate eight popular LLMs, including ChatGPT and GPT-4. Our empirical results show that translation capabilities of LLMs are continually involving. GPT-4 has beat the strong supervised baseline NLLB in 40.91% of translation directions but still faces a large gap towards the commercial translation system like Google Translate, especially on low-resource languages. Through further analysis, we discover that LLMs exhibit new working patterns when used for MMT. First, LLM can acquire translation ability in a resource-efficient way and generate moderate translation even on zero-resource languages. Second, instruction semantics can surprisingly be ignored when given in-context exemplars. Third, cross-lingual exemplars can provide better task guidance for low-resource translation than exemplars in the same language pairs. Code will be released at: https://github.com/NJUNLP/MMT-LLM.

Key facts. Cited by 156 papers in the map (data through 7 Oct 2026). Builds on 38 of its 59 references that are in the map. First posted on arXiv 10 Apr 2023.

Builds on

38 of 59 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  10. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  11. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  12. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
26 more
  1. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  2. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  3. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  4. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  5. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  6. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  7. 2023Larger language models do in-context learning differentlycited by 386
  8. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  9. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  10. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 247
  11. 2022In-context Examples Selection for Machine Translationcited by 172
  12. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 172
  13. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 144
  14. 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
  15. 2023Hallucinations in Large Multilingual Translation Modelscited by 126
  16. 2022Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrationscited by 116
  17. 2023The unreasonable effectiveness of few-shot learning for machine translationcited by 91
  18. 2021Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluationcited by 69
  19. 2023Adaptive Machine Translation with Large Language Modelscited by 58
  20. 2023Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOMcited by 58
  21. 2023PanGu-Sigma: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computingcited by 55
  22. 2023OpenICL: An Open-Source Framework for In-context Learningcited by 51
  23. 2023In-Context Learning with Many Demonstration Examplescited by 39
  24. 2020NeurST: Neural Speech Translation Toolkitcited by 23
  25. 2022Not All Errors are Equal: Learning Text Generation Metrics using Stratified Error Synthesiscited by 18
  26. 2022Robustness of Demonstration-based Learning Under Limited Data Scenariocited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 31; 2024: 54; 2025: 44; 2026: 27.

156 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  2. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  3. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
  4. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
  5. 2023A Rank Stabilization Scaling Factor for Fine-Tuning with LoRAcited by 231
  6. 2025Video models are zero-shot learners and reasonerscited by 208
  7. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 186
  8. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
  9. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Promptingcited by 163
  10. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
  11. 2023SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoningcited by 94
  12. 2023A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Modelscited by 88
38 more
  1. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 82
  2. 2024LLMs are Good Sign Language Translatorscited by 76
  3. 2024Open-LLM-Leaderboard: From Multi-choice to Open-style Questions for LLMs Evaluation, Benchmark, and Arenacited by 76
  4. 2023Extrapolating Large Language Models to Non-English by Aligning Languagescited by 69
  5. 2023BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfercited by 68
  6. 2023BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languagescited by 60
  7. 2023Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructionscited by 55
  8. 2023CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translationcited by 55
  9. 2023TIM: Teaching Large Language Models to Translate with Comparisoncited by 50
  10. 2024Language Model Alignment in Multilingual Trolley Problemscited by 46
  11. 2023Exploring Human-Like Translation Strategy with Large Language Modelscited by 44
  12. 2024BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Modelscited by 44
  13. 2024Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologiescited by 44
  14. 2023BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Modelscited by 42
  15. 2023GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLPcited by 37
  16. 2024X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scalecited by 36
  17. 2023HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translationcited by 35
  18. 2023Empowering Cross-lingual Abilities of Instruction-tuned Large Language Models by Translation-following demonstrationscited by 34
  19. 2023The Perils & Promises of Fact-checking with Large Language Modelscited by 31
  20. 2025Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Executioncited by 25
  21. 2025Adaptive Layer-skipping in Pre-trained LLMscited by 24
  22. 2023ChatGPT MT: Competitive for High- (but not Low-) Resource Languagescited by 23
  23. 2025BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Modelscited by 23
  24. 2024ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaborationcited by 22
  25. 2025Speculative Decoding and Beyond: An In-Depth Survey of Techniquescited by 22
  26. 2025Seed-X: Building Strong Multilingual Translation LLM with 7B Parameterscited by 21
  27. 2024An Investigation of Large Language Models for Real-World Hate Speech Detectioncited by 20
  28. 2026Large Language Model Reasoning Failurescited by 20
  29. 2024Causal Evaluation of Language Modelscited by 18
  30. 2024Multilingual Large Language Models: A Systematic Surveycited by 18
  31. 2024Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Modelscited by 17
  32. 2024How Much Data is Enough Data? Fine-Tuning Large Language Models for In-House Translation: Performance Evaluation Across Multiple Dataset Sizescited by 17
  33. 2024Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Basescited by 17
  34. 2024Whose Morality Do They Speak? Unraveling Cultural Bias in Multilingual Language Modelscited by 17
  35. 2023Chain-of-Dictionary Prompting Elicits Translation in Large Language Modelscited by 16
  36. 2023TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varietiescited by 16
  37. 2023BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Modelscited by 16
  38. 2025Structured Moral Reasoning in Language Models: A Value-Grounded Evaluation Frameworkcited by 16

50 most cited of 156 All 156 (JSON)