mlsanityData to 14 Oct 2026

How Does Quantization Affect Multilingual LLMs?

arXiv 2407.03211 ↗First posted 3 Jul 2024cs.CL cs.LGcited by 43

Not on the map. Why?

Abstract (arXiv)

Quantization techniques are widely used to improve inference speed and deployment of large language models. While a wide body of work examines the impact of quantization on LLMs in English, none have evaluated across languages. We conduct a thorough analysis of quantized multilingual LLMs, focusing on performance across languages and at varying scales. We use automatic benchmarks, LLM-as-a-Judge, and human evaluation, finding that (1) harmful effects of quantization are apparent in human evaluation, which automatic metrics severely underestimate: a 1.7% average drop in Japanese across automatic tasks corresponds to a 16.0% drop reported by human evaluators on realistic prompts; (2) languages are disparately affected by quantization, with non-Latin script languages impacted worst; and (3) challenging tasks like mathematical reasoning degrade fastest. As the ability to serve low-compute models is critical for wide global adoption of NLP technologies, our results urge consideration of multilingual performance as a key evaluation criterion for efficient models.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Builds on 41 of its 63 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

41 of 63 references.

References · 41

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  5. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  7. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  8. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  9. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  10. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  11. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  12. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 295
29 more
  1. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 285
  2. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  3. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  4. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 192
  5. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 190
  6. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 177
  7. 2019What Do Compressed Deep Neural Networks Forget?cited by 171
  8. 2020Characterising Bias in Compressed Modelscited by 167
  9. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 151
  10. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 123
  11. 2024Understanding and Mitigating Language Confusion in LLMscited by 75
  12. 2024An empirical study of LLaMA3 quantization: from LLMs to MLLMscited by 64
  13. 2021Randomness In Neural Network Training: Characterizing The Impact of Toolingcited by 59
  14. 2023Intriguing Properties of Quantization at Scalecited by 49
  15. 2021The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translationcited by 47
  16. 2023Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Studycited by 43
  17. 2023AfroBench: How Good are Large Language Models on African Languages?cited by 43
  18. 2023GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLPcited by 37
  19. 2023Lost in Translation: Large Language Models in Non-English Content Analysiscited by 35
  20. 2023On Evaluating and Mitigating Gender Biases in Multilingual Settingscited by 31
  21. 2024What Makes Quantization for Large Language Models Hard? An Empirical Study from the Lens of Perturbationcited by 23
  22. 2023A Material Lens on Coloniality in NLPcited by 21
  23. 2024Quantifying the Capabilities of LLMs across Scale and Precisioncited by 19
  24. 2024From One to Many: Expanding the Scope of Toxicity Mitigation in Language Modelscited by 18
  25. 2022Intriguing Properties of Compression on Multilingual Modelscited by 15
  26. 2024Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMscited by 12
  27. 2022Robust Distillation for Worst-class Performancecited by 6
  28. 2022Too Brittle To Touch: Comparing the Stability of Quantization and Distillation Towards Developing Lightweight Low-Resource MT Modelscited by 5
  29. 2023On The Fairness Impacts of Hardware Selection in Machine Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 20; 2026: 18.

43 papers.

Sort:

By area:

  1. 2024MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignmentcited by 25
  2. 2024How Numerical Precision Affects Arithmetical Reasoning Capabilities of LLMscited by 24
  3. 2026Tiny Aya: Bridging Scale and Multilingual Depthcited by 15
  4. 2024KunServe: Parameter-centric Memory Management for Efficient Memory Overloading Handling in LLM Servingcited by 11
  5. 2025Does quantization affect models' performance on long-context tasks?cited by 10
  6. 2025Why Do Some Inputs Break Low-Bit LLM Quantization?cited by 9
  7. 2026Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levelscited by 6
  8. 2025Humans Hallucinate Too: Language Models Identify and Correct Subjective Annotation Errors With Label-in-a-Haystack Promptscited by 5
  9. 2025The Devil in the Details: Emergent Misalignment, Format and Coherence in Open-Weights LLMscited by 5
  10. 2026Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanationscited by 5
  11. 2024Aggregation Artifacts in Subjective Tasks Collapse Large Language Models' Posteriorscited by 4
  12. 2025OWL: Probing Cross-Lingual Recall of Memorized Texts via World Literaturecited by 4
31 more
  1. 2025LPASS: Linear Probes as Stepping Stones for vulnerability detection using compressed LLMscited by 4
  2. 2025Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Modelscited by 4
  3. 2025Through a Compressed Lens: Investigating The Impact of Quantization on Factual Knowledge Recallcited by 3
  4. 2025Quantized Large Language Models in Biomedical Natural Language Processing: Evaluation and Recommendationcited by 3
  5. 2025Exploring Large Language Models for Translating Romanian Computational Problems into Englishcited by 2
  6. 2025A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLPcited by 2
  7. 2025FreshTab: Sourcing Fresh Data for Table-to-Text Generation Evaluationcited by 2
  8. 2026Calibrating Beyond English: Language Diversity for Better Quantized Multilingual LLMcited by 2
  9. 2026Broken on Arrival: Silently Defective LLM Artifacts in Public Model Registries and How to Catch Themcited by 2
  10. 2025English K_Quantization of LLMs Does Not Disproportionately Diminish Multilingual Performancecited by 1
  11. 2025The Uneven Impact of Post-Training Quantization in Machine Translationcited by 1
  12. 2026Multilinguality at the Edge: Developing Language Models for the Global Southcited by 1
  13. 2026Challenges and Recommendations for LLM-as-a-Judge in Multilingual Settings and for Low-Resource Languagescited by 1
  14. 2026QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstructioncited by 1
  15. 2024Evaluating the Performance of Large Language Models in Competitive Programming: A Multi-Year, Multi-Grade Analysiscited by 0
  16. 2025Multilingual Collaborative Defense for Large Language Modelscited by 0
  17. 2025Fair-GPTQ: Bias-Aware Quantization for Large Language Modelscited by 0
  18. 2025Exploring Cross-Lingual Knowledge Transfer via Transliteration-Based MLM Fine-Tuning for Critically Low-resource Chakma Languagecited by 0
  19. 2025Compressed Causal Reasoning: Quantization and GraphRAG Effects on Interventional and Counterfactual Accuracycited by 0
  20. 2025Toward Agentic Environments: GenAI and the Convergence of AI, Sustainability, and Human-Centric Spacescited by 0
  21. 2025Comparing Approaches to Automatic Summarization in Less-Resourced Languagescited by 0
  22. 2026K-Quantization and its Impact on Output Performancecited by 0
  23. 2026Beyond English: Uncovering the Multilingual Gap in Vision-Language-Action Modelscited by 0
  24. 2026Studying quantization trade-offs for efficient inference deployment in machine translationcited by 0
  25. 2026Cultural Awareness is Represented but Not Decoded: Tracing Mythological Knowledge across 18 Open-Source LLMscited by 0
  26. 2026The Multilingual Quantization Tax: Structural Collapse and Typological Fragility in Edge SLMscited by 0
  27. 2026Same Request, Different Answer: Quantization Amplifies Cache-Induced Divergence in LLM Servingcited by 0
  28. 2026A Calibrated Instrument for Measuring How Inference Optimizations Affect Output Qualitycited by 0
  29. 2026Quantization Thresholds Replicate, Failure Modes Do Not: A Three-Model Study of Agentic Tool Use in Polish from 8-bit to 2-bitcited by 0
  30. 2026Saying, Not Knowing: Aggressively GGUF-Quantized Small Language Models Still Write Rare Words They Can No Longer Definecited by 0
  31. 2026Lightweight but Linguistically Fair? Quantisation Robustness across Bavarian, German, and Englishcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Evaluating the Generalization Ability of Quantized LLMs: Benchmark, Analysis, and Toolbox
  2. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Models
  3. 2025A Comprehensive Evaluation on Quantization Techniques for Large Language Models
  4. 2025MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation
  5. 2026QuantiBias: Benchmarking Quantization-Induced Bias in LLMs
  6. 2024LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit
  7. 2024The Impact of Quantization on Retrieval-Augmented Generation: An Analysis of Small LLMs
  8. 2025Towards Inclusive NLP: Assessing Compressed Multilingual Transformers across Diverse Language Benchmarks