# How Does Quantization Affect Multilingual LLMs?

arXiv: https://arxiv.org/abs/2407.03211 · HTML: https://mlsanity.com/paper/2407.03211/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 63 references that are in the map.
- First posted on arXiv 3 Jul 2024.

## Builds on (most cited first, 30 of 41)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019)
- [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://mlsanity.com/paper/2402.07827/) (2024)
- [Analyzing Leakage of Personally Identifiable Information in Language Models](https://mlsanity.com/paper/2302.00539/) (2023)
- [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021)
- [The case for 4-bit precision: k-bit Inference Scaling Laws](https://mlsanity.com/paper/2212.09720/) (2022)
- [Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models](https://mlsanity.com/paper/2305.13707/) (2023)
- [Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.16039/) (2023)
- [Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning](https://mlsanity.com/paper/2402.06619/) (2024)
- [What Do Compressed Deep Neural Networks Forget?](https://mlsanity.com/paper/1911.05248/) (2019)
- [Characterising Bias in Compressed Models](https://mlsanity.com/paper/2010.03058/) (2020)
- [Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing](https://mlsanity.com/paper/2306.12929/) (2023)
- [Aya 23: Open Weight Releases to Further Multilingual Progress](https://mlsanity.com/paper/2405.15032/) (2024)
- [Understanding and Mitigating Language Confusion in LLMs](https://mlsanity.com/paper/2406.20052/) (2024)
- [An empirical study of LLaMA3 quantization: from LLMs to MLLMs](https://mlsanity.com/paper/2404.14047/) (2024)
- [Randomness In Neural Network Training: Characterizing The Impact of Tooling](https://mlsanity.com/paper/2106.11872/) (2021)
- [Intriguing Properties of Quantization at Scale](https://mlsanity.com/paper/2305.19268/) (2023)
- [The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation](https://mlsanity.com/paper/2110.03036/) (2021)
- [Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study](https://mlsanity.com/paper/2307.08072/) (2023)
- [AfroBench: How Good are Large Language Models on African Languages?](https://mlsanity.com/paper/2311.07978/) (2023)
- [GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP](https://mlsanity.com/paper/2305.14976/) (2023)

## Built on by (most cited first, 30 of 43)

- [MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment](https://mlsanity.com/paper/2410.05873/) (2024)
- [How Numerical Precision Affects Arithmetical Reasoning Capabilities of LLMs](https://mlsanity.com/paper/2410.13857/) (2024)
- [Tiny Aya: Bridging Scale and Multilingual Depth](https://mlsanity.com/paper/2603.11510/) (2026)
- [KunServe: Parameter-centric Memory Management for Efficient Memory Overloading Handling in LLM Serving](https://mlsanity.com/paper/2412.18169/) (2024)
- [Does quantization affect models' performance on long-context tasks?](https://mlsanity.com/paper/2505.20276/) (2025)
- [Why Do Some Inputs Break Low-Bit LLM Quantization?](https://mlsanity.com/paper/2506.12044/) (2025)
- [Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels](https://mlsanity.com/paper/2605.15208/) (2026)
- [Humans Hallucinate Too: Language Models Identify and Correct Subjective Annotation Errors With Label-in-a-Haystack Prompts](https://mlsanity.com/paper/2505.17222/) (2025)
- [The Devil in the Details: Emergent Misalignment, Format and Coherence in Open-Weights LLMs](https://mlsanity.com/paper/2511.20104/) (2025)
- [Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanations](https://mlsanity.com/paper/2601.00282/) (2026)
- [Aggregation Artifacts in Subjective Tasks Collapse Large Language Models' Posteriors](https://mlsanity.com/paper/2410.13776/) (2024)
- [OWL: Probing Cross-Lingual Recall of Memorized Texts via World Literature](https://mlsanity.com/paper/2505.22945/) (2025)
- [LPASS: Linear Probes as Stepping Stones for vulnerability detection using compressed LLMs](https://mlsanity.com/paper/2505.24451/) (2025)
- [Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models](https://mlsanity.com/paper/2508.18609/) (2025)
- [Through a Compressed Lens: Investigating The Impact of Quantization on Factual Knowledge Recall](https://mlsanity.com/paper/2505.13963/) (2025)
- [Quantized Large Language Models in Biomedical Natural Language Processing: Evaluation and Recommendation](https://mlsanity.com/paper/2509.04534/) (2025)
- [Exploring Large Language Models for Translating Romanian Computational Problems into English](https://mlsanity.com/paper/2501.05601/) (2025)
- [A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLP](https://mlsanity.com/paper/2505.16661/) (2025)
- [FreshTab: Sourcing Fresh Data for Table-to-Text Generation Evaluation](https://mlsanity.com/paper/2510.13598/) (2025)
- [Calibrating Beyond English: Language Diversity for Better Quantized Multilingual LLM](https://mlsanity.com/paper/2601.18306/) (2026)
- [Broken on Arrival: Silently Defective LLM Artifacts in Public Model Registries and How to Catch Them](https://mlsanity.com/paper/2609.05881/) (2026)
- [English K_Quantization of LLMs Does Not Disproportionately Diminish Multilingual Performance](https://mlsanity.com/paper/2503.03592/) (2025)
- [The Uneven Impact of Post-Training Quantization in Machine Translation](https://mlsanity.com/paper/2508.20893/) (2025)
- [Multilinguality at the Edge: Developing Language Models for the Global South](https://mlsanity.com/paper/2604.21637/) (2026)
- [Challenges and Recommendations for LLM-as-a-Judge in Multilingual Settings and for Low-Resource Languages](https://mlsanity.com/paper/2607.02235/) (2026)
- [QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction](https://mlsanity.com/paper/2608.13966/) (2026)
- [Evaluating the Performance of Large Language Models in Competitive Programming: A Multi-Year, Multi-Grade Analysis](https://mlsanity.com/paper/2409.09054/) (2024)
- [Multilingual Collaborative Defense for Large Language Models](https://mlsanity.com/paper/2505.11835/) (2025)
- [Fair-GPTQ: Bias-Aware Quantization for Large Language Models](https://mlsanity.com/paper/2509.15206/) (2025)
- [Exploring Cross-Lingual Knowledge Transfer via Transliteration-Based MLM Fine-Tuning for Critically Low-resource Chakma Language](https://mlsanity.com/paper/2510.09032/) (2025)

## Abstract (arXiv)

Quantization techniques are widely used to improve inference speed and deployment of large language models. While a wide body of work examines the impact of quantization on LLMs in English, none have evaluated across languages. We conduct a thorough analysis of quantized multilingual LLMs, focusing on performance across languages and at varying scales. We use automatic benchmarks, LLM-as-a-Judge, and human evaluation, finding that (1) harmful effects of quantization are apparent in human evaluation, which automatic metrics severely underestimate: a 1.7% average drop in Japanese across automatic tasks corresponds to a 16.0% drop reported by human evaluators on realistic prompts; (2) languages are disparately affected by quantization, with non-Latin script languages impacted worst; and (3) challenging tasks like mathematical reasoning degrade fastest. As the ability to serve low-compute models is critical for wide global adoption of NLP technologies, our results urge consideration of multilingual performance as a key evaluation criterion for efficient models.