mlsanityData to 7 Oct 2026

Tower: An Open Multilingual Large Language Model for Translation-Related Tasks

arXiv 2402.17733 ↗First posted 27 Feb 2024cs.CLcited by 205

Not on the map. Why?

Abstract (arXiv)

While general-purpose large language models (LLMs) demonstrate proficiency on multiple tasks within the domain of translation, approaches based on open LLMs are competitive only when specializing on a single task. In this paper, we propose a recipe for tailoring LLMs to multiple tasks present in translation workflows. We perform continued pretraining on a multilingual mixture of monolingual and parallel data, creating TowerBase, followed by finetuning on instructions relevant for translation processes, creating TowerInstruct. Our final model surpasses open alternatives on several tasks relevant to translation workflows and is competitive with general-purpose closed LLMs. To facilitate future research, we release the Tower models, our specialization dataset, an evaluation framework for LLMs focusing on the translation ecosystem, and a collection of model generations, including ours, on our benchmark.

Key facts. Cited by 205 papers in the map (data through 7 Oct 2026). Builds on 59 of its 96 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

59 of 96 references.

References · 59

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  5. 2023Qwen Technical Reportcited by 3,273
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  8. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  9. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  10. 2024Mixtral of Expertscited by 1,785
  11. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  12. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
38 more
  1. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  2. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  3. 2023LIMA: Less Is More for Alignmentcited by 1,080
  4. 2023BloombergGPT: A Large Language Model for Financecited by 923
  5. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  6. 2022Galactica: A Large Language Model for Sciencecited by 736
  7. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  8. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  9. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  10. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  11. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
  12. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  13. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
  14. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  15. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 272
  16. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 262
  17. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 260
  18. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
  19. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 205
  20. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  21. 2023ChipNeMo: Domain-Adapted LLMs for Chip Designcited by 168
  22. 2023GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4cited by 143
  23. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 137
  24. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  25. 2020The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MTcited by 127
  26. 2021High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metricscited by 104
  27. 2023The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluationcited by 95
  28. 2023A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Modelscited by 88
  29. 2023Scaling up COMETKIWI: Unbabel-IST 2023 Submission for the Quality Estimation Shared Taskcited by 88
  30. 2024Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuningcited by 85
  31. 2020TICO-19: the Translation Initiative for Covid-19cited by 67
  32. 2023PolyLM: An Open Source Polyglot Large Language Modelcited by 64
  33. 2023Searching for Needles in a Haystack: On the Role of Incidental Bilingualism in PaLM's Translation Capabilitycited by 62
  34. 2023Leveraging GPT-4 for Automatic Translation Post-Editingcited by 56
  35. 2024CroissantLLM: A Truly Bilingual French-English Language Modelcited by 52
  36. 2019The Effect of Translationese in Machine Translation Test Setscited by 49
  37. 2022Quality-Aware Decoding for Neural Machine Translationcited by 49
  38. 2024Navigating the Metrics Maze: Reconciling Score Magnitudes and Accuraciescited by 40

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 69; 2025: 97; 2026: 39.

205 papers.

Sort:

By area:

  1. 2025PolyGuard: A Multilingual Safety Moderation Tool for 17 Languagescited by 57
  2. 2025MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learningcited by 48
  3. 2024LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languagescited by 47
  4. 2024Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in?cited by 40
  5. 2025Tower+: Bridging Generality and Translation Specialization in Multilingual LLMscited by 40
  6. 2024SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domaincited by 39
  7. 2025Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Studycited by 38
  8. 2024LumberChunker: Long-Form Narrative Document Segmentationcited by 36
  9. 2024X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scalecited by 36
  10. 2024Trans-Tokenization and Cross-lingual Vocabulary Transfers: Language Adaptation of LLMs for Low-Resource NLPcited by 35
  11. 2024Understanding Cross-Lingual Alignment -- A Surveycited by 34
  12. 2024Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Textscited by 31
38 more
  1. 2025EuroBERT: Scaling Multilingual Encoders for European Languagescited by 28
  2. 2025Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMscited by 26
  3. 2024How Good Are LLMs for Literary Translation, Really? Literary Translation Evaluation with Humans and LLMscited by 23
  4. 2024Guiding Large Language Models to Post-Edit Machine Translation with Error Annotationscited by 22
  5. 2024MQM-APE: Toward High-Quality Error Annotation Predictors with Automatic Post-Editing in LLM Translation Evaluatorscited by 22
  6. 2024Poro 34B and the Blessing of Multilingualitycited by 18
  7. 2024xTower: A Multilingual LLM for Explaining and Correcting Translation Errorscited by 17
  8. 2025Multilingual Language Model Pretraining using Machine-translated Datacited by 17
  9. 2024Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice?cited by 16
  10. 2024QUEST: Quality-Aware Metropolis-Hastings Sampling for Machine Translationcited by 16
  11. 2024Meltemi: The first open Large Language Model for Greekcited by 16
  12. 2024MoE-LPR: Multilingual Extension of Large Language Models through Mixture-of-Experts with Language Priors Routingcited by 16
  13. 2024EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Modelscited by 16
  14. 2025Babel: Open Multilingual Large Language Models Serving Over 90% of Global Speakerscited by 16
  15. 2024PrExMe! Large Scale Prompt Exploration of Open Source LLMs for Machine Translation and Summarization Evaluationcited by 15
  16. 2025An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT)cited by 15
  17. 2025Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Trainingcited by 15
  18. 2024Word Alignment as Preference for Machine Translationcited by 14
  19. 2026Tiny Aya: Bridging Scale and Multilingual Depthcited by 14
  20. 2024Improving Minimum Bayes Risk Decoding with Multi-Promptcited by 13
  21. 2024Quality or Quantity? On Data Scale and Diversity in Adapting Large Language Models for Low-Resource Translationcited by 13
  22. 2025Overestimation in LLM Evaluation: A Controlled Large-Scale Study on Data Contamination's Impact on Machine Translationcited by 13
  23. 2025Krikri: Advancing Open Large Language Models for Greekcited by 13
  24. 2025SSR-Zero: Simple Self-Rewarding Reinforcement Learning for Machine Translationcited by 13
  25. 2024Low-Resource Machine Translation through Retrieval-Augmented LLM Prompting: A Study on the Mambai Languagecited by 12
  26. 2024Ladder: A Model-Agnostic Framework Boosting LLM-based Machine Translation to the Next Levelcited by 11
  27. 2024Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappingscited by 11
  28. 2025Multilingual Contextualization of Large Language Models for Document-Level Machine Translationcited by 11
  29. 2024Fairness Definitions in Language Models Explainedcited by 10
  30. 2024Introducing the NewsPaLM MBR and QE Dataset: LLM-Generated High-Quality Parallel Data Outperforms Traditional Web-Crawled Datacited by 10
  31. 2024Pitfalls and Outlooks in Using COMETcited by 10
  32. 2024Beyond Correlation: Interpretable Evaluation of Machine Translation Metricscited by 10
  33. 2024Watching the Watchers: Exposing Gender Disparities in Machine Translation Quality Estimationcited by 10
  34. 2024SpeechQE: Estimating the Quality of Direct Speech Translationcited by 10
  35. 2024Investigating Length Issues in Document-level Machine Translationcited by 10
  36. 2025SwiLTra-Bench: The Swiss Legal Translation Benchmarkcited by 10
  37. 2025Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting LLMs Across Languages and Resourcescited by 10
  38. 2025Conditions for Catastrophic Forgetting in Multilingual Translationcited by 10

50 most cited of 205 All 205 (JSON)