mlsanityData to 7 Oct 2026

LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages

arXiv 2407.05975 ↗First posted 8 Jul 2024cs.CL cs.AIcited by 47

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) demonstrate remarkable translation capabilities in high-resource language tasks, yet their performance in low-resource languages is hindered by insufficient multilingual data during pre-training. To address this, we conduct extensive multilingual continual pre-training on the LLaMA series models, enabling translation support across more than 100 languages. Through a comprehensive analysis of training strategies, such as vocabulary expansion and data augmentation, we develop LLaMAX. Remarkably, without sacrificing its generalization ability, LLaMAX achieves significantly higher translation performance compared to existing open-source LLMs (by more than 10 spBLEU points) and performs on-par with specialized translation model (M2M-100-12B) on the Flores-101 benchmark. Extensive experiments indicate that LLaMAX can serve as a robust multilingual foundation model. The code footnoteurlhttps://github.com/CONE-MT/LLaMAX/. and the models footnoteurlhttps://huggingface.co/LLaMAX/. are publicly available.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 49 of its 70 references that are in the map. First posted on arXiv 8 Jul 2024.

Builds on

49 of 70 references.

References · 49

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  9. 2021Program Synthesis with Large Language Modelscited by 3,509
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  11. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  12. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
37 more
  1. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  2. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  3. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  4. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  5. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  6. 2020Aligning AI With Shared Human Valuescited by 706
  7. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  8. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  9. 2023Scaling Data-Constrained Language Modelscited by 391
  10. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  11. 2016FastText.zip: Compressing text classification modelscited by 331
  12. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  13. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
  14. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  15. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  16. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  17. 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 180
  18. 2024How do Large Language Models Handle Multilingualism?cited by 148
  19. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 144
  20. 2023GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4cited by 143
  21. 2023Continual Pre-Training of Large Language Models: How to (re)warm your model?cited by 135
  22. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 121
  23. 2019Neural Machine Translation with Byte-Level Subwordscited by 115
  24. 2024Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilitiescited by 111
  25. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
  26. 2021It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoningcited by 69
  27. 2020TICO-19: the Translation Initiative for Covid-19cited by 67
  28. 2021Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoningcited by 64
  29. 2023PolyLM: An Open Source Polyglot Large Language Modelcited by 64
  30. 2024CroissantLLM: A Truly Bilingual French-English Language Modelcited by 52
  31. 2022Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptationcited by 44
  32. 2024Question Translation Training for Better Multilingual Reasoningcited by 41
  33. 2024Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languagescited by 38
  34. 2022Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translationcited by 21
  35. 2024MindMerger: Efficient Boosting LLM Reasoning in non-English Languagescited by 21
  36. 2023How Vocabulary Sharing Facilitates Multilingualism in LLaMA?cited by 12
  37. 2024KS-Lottery: Finding Certified Lottery Tickets for Multilingual Language Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 32; 2026: 6.

47 papers.

Sort:

By area:

  1. 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMscited by 145
  2. 2024Towards Evaluating and Building Versatile Large Language Models for Medicinecited by 38
  3. 2025Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Studycited by 38
  4. 2024X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scalecited by 36
  5. 2024Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Surveycited by 23
  6. 2025BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Modelscited by 23
  7. 2025Seed-X: Building Strong Multilingual Translation LLM with 7B Parameterscited by 21
  8. 2025AdaSpec: Adaptive Speculative Decoding for Fast, SLO-Aware Large Language Model Servingcited by 18
  9. 2025Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Modelscited by 17
  10. 2025Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Aheadcited by 15
  11. 2025SSR-Zero: Simple Self-Rewarding Reinforcement Learning for Machine Translationcited by 13
  12. 2024The Future of Open Human Feedbackcited by 10
35 more
  1. 2025LayAlign: Enhancing Multilingual Reasoning in Large Language Models via Layer-Wise Adaptive Fusion and Alignment Strategycited by 10
  2. 2025Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting LLMs Across Languages and Resourcescited by 10
  3. 2025Aligning LLMs for Multilingual Consistency in Enterprise Applicationscited by 10
  4. 2024A Survey on Knowledge Organization Systems of Research Fields: Resources and Challengescited by 9
  5. 2025Seed LiveInterpret 2.0: End-to-end Simultaneous Speech-to-speech Translation with Your Voicecited by 9
  6. 2025Bring Your Own Knowledge: A Survey of Methods for LLM Knowledge Expansioncited by 8
  7. 2025Lugha-Llama: Adapting Large Language Models for African Languagescited by 8
  8. 2025A Large Language Model-Empowered Agent for Reliable and Robust Structural Analysiscited by 8
  9. 2024Uhura: A Benchmark for Evaluating Scientific Question Answering and Truthfulness in Low-Resource African Languagescited by 7
  10. 2025Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inferencecited by 6
  11. 2026OrbitFlow: SLO-Aware Long-Context LLM Serving with Fine-Grained KV Cache Reconfigurationcited by 6
  12. 2024Boosting LLM Translation Skills without General Ability Loss via Rationale Distillationcited by 5
  13. 2025TACTIC: Translation Agents with Cognitive-Theoretic Interactive Collaborationcited by 5
  14. 2025MiLiC-Eval: Benchmarking Multilingual LLMs for China's Minority Languagescited by 4
  15. 2025Understanding LLMs' Cross-Lingual Context Retrieval: How Good It Is And Where It Comes Fromcited by 4
  16. 2025Massively Multilingual Adaptation of Large Language Models Using Bilingual Translation Datacited by 4
  17. 2025Evolution of meta's llama models and parameter-efficient fine-tuning of large language models: a surveycited by 4
  18. 2025NiuTrans.LMT: Toward Inclusive and Scalable Multilingual Machine Translation with LLMscited by 4
  19. 2026Culture-Aware Machine Translation in Large Language Models: Benchmarking and Investigationcited by 4
  20. 2024Mufu: Multilingual Fused Learning for Low-Resource Translation with LLMcited by 3
  21. 2024Dictionary Insertion Prompting for Multilingual Reasoning on Multilingual Large Language Modelscited by 3
  22. 2025Small Models, Big Impact: Efficient Corpus and Graph-Based Adaptation of Small Multilingual Language Models for Low-Resource Languagescited by 3
  23. 2025Exploring the Translation Mechanism of Large Language Modelscited by 3
  24. 2025M2PO: Multi-Perspective Multi-Pair Preference Optimization for Machine Translationcited by 3
  25. 2025MCAT: Scaling Many-to-Many Speech-to-Text Translation with MLLMs to 70 Languagescited by 3
  26. 2026Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translationcited by 3
  27. 2025Bridging Language Barriers in Healthcare: A Study on Arabic LLMscited by 2
  28. 2025Mutarjim: Advancing Bidirectional Arabic-English Translation with a Small Language Modelcited by 2
  29. 2025EnAnchored-X2X: English-Anchored Optimization for Many-to-Many Translationcited by 2
  30. 2025Arti-"fickle" Intelligence: Using LLMs as a Tool for Inference in the Political and Social Sciencescited by 1
  31. 2025Evaluating the Impact of Verbal Multiword Expressions on Machine Translationcited by 1
  32. 2025Is Multilingual LLM Watermarking Truly Multilingual? Scaling Robustness to 100+ Languages via Back-Translationcited by 1
  33. 2026VEPO: Variable Entropy Policy Optimization for Low-Resource Language Foundation Modelscited by 0
  34. 2026Leveraging Audio-LLMs to Filter Speech-to-Speech Training Datacited by 0
  35. 2026KItCAT: Knowledge Injection via Input Corruption for Auto-regressive Trainingcited by 0