# Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOM

arXiv: https://arxiv.org/abs/2303.01911 · HTML: https://mlsanity.com/paper/2303.01911/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 61 references that are in the map.
- First posted on arXiv 3 Mar 2023.

## Builds on (most cited first, 30 of 39)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts](https://mlsanity.com/paper/2202.01279/) (2022)
- [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021)
- [Prompting Large Language Model for Machine Translation: A Case Study](https://mlsanity.com/paper/2301.07069/) (2023)
- [The Natural Language Decathlon: Multitask Learning as Question Answering](https://mlsanity.com/paper/1806.08730/) (2018)
- [Prompting PaLM for Translation: Assessing Strategies and Performance](https://mlsanity.com/paper/2211.09102/) (2022)
- [Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism](https://mlsanity.com/paper/1601.01073/) (2016)
- [Context-Aware Neural Machine Translation Learns Anaphora Resolution](https://mlsanity.com/paper/1805.10163/) (2018)
- [Multi-task Sequence to Sequence Learning](https://mlsanity.com/paper/1511.06114/) (2015)
- [Language Contamination Helps Explain the Cross-lingual Capabilities of English Pretrained Models](https://mlsanity.com/paper/2204.08110/) (2022)
- [The unreasonable effectiveness of few-shot learning for machine translation](https://mlsanity.com/paper/2302.01398/) (2023)

## Built on by (most cited first, 30 of 59)

- [Lessons from the Trenches on Reproducible Evaluation of Language Models](https://mlsanity.com/paper/2405.14782/) (2024)
- [Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis](https://mlsanity.com/paper/2304.04675/) (2023)
- [Hallucinations in Large Multilingual Translation Models](https://mlsanity.com/paper/2303.16104/) (2023)
- [Cognitive Mirage: A Review of Hallucinations in Large Language Models](https://mlsanity.com/paper/2309.06794/) (2023)
- [BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languages](https://mlsanity.com/paper/2305.18098/) (2023)
- [Steering Large Language Models for Machine Translation with Finetuning and In-Context Learning](https://mlsanity.com/paper/2310.13448/) (2023)
- [Exploring Human-Like Translation Strategy with Large Language Models](https://mlsanity.com/paper/2305.04118/) (2023)
- [Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decoding](https://mlsanity.com/paper/2309.07098/) (2023)
- [Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed?](https://mlsanity.com/paper/2312.12683/) (2023)
- [Improving Machine Translation with Human Feedback: An Exploration of Quality Estimation as a Reward Model](https://mlsanity.com/paper/2401.12873/) (2024)
- [Guiding Large Language Models to Post-Edit Machine Translation with Error Annotations](https://mlsanity.com/paper/2404.07851/) (2024)
- [In-Context Example Selection via Similarity Search Improves Low-Resource Machine Translation](https://mlsanity.com/paper/2408.00397/) (2024)
- [Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMs](https://mlsanity.com/paper/2502.09331/) (2025)
- [Unlocking Memorization in Large Language Models with Dynamic Soft Prompting](https://mlsanity.com/paper/2409.13853/) (2024)
- [Comparing Hallucination Detection Metrics for Multilingual Generation](https://mlsanity.com/paper/2402.10496/) (2024)
- [Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editing](https://mlsanity.com/paper/2410.07054/) (2024)
- [Domain Terminology Integration into Machine Translation: Leveraging Large Language Models](https://mlsanity.com/paper/2310.14451/) (2023)
- [Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translation](https://mlsanity.com/paper/2503.04554/) (2025)
- [Don't Rank, Combine! Combining Machine Translation Hypotheses Using Quality Estimation](https://mlsanity.com/paper/2401.06688/) (2024)
- [The power of Prompts: Evaluating and Mitigating Gender Bias in MT with LLMs](https://mlsanity.com/paper/2407.18786/) (2024)
- [Prefix Text as a Yarn: Eliciting Non-English Alignment in Foundation Language Model](https://mlsanity.com/paper/2404.16766/) (2024)
- [A Preference-driven Paradigm for Enhanced Translation with Large Language Models](https://mlsanity.com/paper/2404.11288/) (2024)
- [Neural Machine Translation Models Can Learn to be Few-shot Learners](https://mlsanity.com/paper/2309.08590/) (2023)
- [Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation](https://mlsanity.com/paper/2504.01919/) (2025)
- [Retrieving Examples from Memory for Retrieval Augmented Neural Machine Translation: A Systematic Comparison](https://mlsanity.com/paper/2404.02835/) (2024)
- [Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs](https://mlsanity.com/paper/2512.16378/) (2025)
- [Perturbation-based QE: An Explainable, Unsupervised Word-level Quality Estimation Method for Blackbox Machine Translation](https://mlsanity.com/paper/2305.07457/) (2023)
- [Insights into Classifying and Mitigating LLMs' Hallucinations](https://mlsanity.com/paper/2311.08117/) (2023)
- [How Multilingual Are Large Language Models Fine-Tuned for Translation?](https://mlsanity.com/paper/2405.20512/) (2024)
- [Split and Rephrase with Large Language Models](https://mlsanity.com/paper/2312.11075/) (2023)

## Abstract (arXiv)

The NLP community recently saw the release of a new large open-access multilingual language model, BLOOM (BigScience et al., 2022) covering 46 languages. We focus on BLOOM's multilingual ability by evaluating its machine translation performance across several datasets (WMT, Flores-101 and DiaBLa) and language pairs (high- and low-resourced). Our results show that 0-shot performance suffers from overgeneration and generating in the wrong language, but this is greatly improved in the few-shot setting, with very good results for a number of language pairs. We study several aspects including prompt design, model sizes, cross-lingual transfer and the use of discursive context.