mlsanityData to 14 Oct 2026

The Uneven Impact of Post-Training Quantization in Machine Translation

arXiv 2508.20893 ↗First posted 28 Aug 2025cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Quantization is essential for deploying large language models (LLMs) on resource-constrained hardware, but its implications for multilingual tasks remain underexplored. We conduct the first large-scale evaluation of post-training quantization (PTQ) on machine translation across 55 languages using five LLMs ranging from 1.7B to 70B parameters. Our analysis reveals that while 4-bit quantization often preserves translation quality for high-resource languages and large models, significant degradation occurs for low-resource and typologically diverse languages, particularly in 2-bit settings. We compare four quantization techniques (AWQ, BitsAndBytes, GGUF, and AutoRound), showing that algorithm choice and model size jointly determine robustness. GGUF variants provide the most consistent performance, even at 2-bit precision. Additionally, we quantify the interactions between quantization, decoding hyperparameters, and calibration languages, finding that language-matched calibration offers benefits primarily in low-bit scenarios. Our findings offer actionable insights for deploying multilingual LLMs for machine translation under quantization constraints, especially in low-resource settings.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 23 of its 35 references that are in the map. First posted on arXiv 28 Aug 2025.

Builds on

23 of 35 references.

References · 23

  1. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  5. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  6. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  7. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  8. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  9. 2019What Do Compressed Deep Neural Networks Forget?cited by 171
  10. 2020Characterising Bias in Compressed Modelscited by 167
  11. 2024Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilitiescited by 111
  12. 2025WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialectscited by 79
11 more
  1. 2024An empirical study of LLaMA3 quantization: from LLMs to MLLMscited by 64
  2. 2023Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMscited by 50
  3. 2023Intriguing Properties of Quantization at Scalecited by 49
  4. 2021The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translationcited by 47
  5. 2024How Does Quantization Affect Multilingual LLMs?cited by 43
  6. 2024What Makes Quantization for Large Language Models Hard? An Empirical Study from the Lens of Perturbationcited by 23
  7. 2024Guardians of the Machine Translation Meta-Evaluation: Sentinel Metrics Fall In!cited by 21
  8. 2024Quantifying the Capabilities of LLMs across Scale and Precisioncited by 19
  9. 2022Intriguing Properties of Compression on Multilingual Modelscited by 15
  10. 2024Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMscited by 12
  11. 2022Too Brittle To Touch: Comparing the Stability of Quantization and Distillation Towards Developing Lightweight Low-Resource MT Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Saying, Not Knowing: Aggressively GGUF-Quantized Small Language Models Still Write Rare Words They Can No Longer Definecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Evaluation on Quantization Techniques for Large Language Models
  2. 2026Calibrating Beyond English: Language Diversity for Better Quantized Multilingual LLM
  3. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Models
  4. 2024Evaluating the Generalization Ability of Quantized LLMs: Benchmark, Analysis, and Toolbox
  5. 2024Post Training Quantization of Large Language Models with Microscaling Formats
  6. 2025Resource-Efficient Language Models: Quantization for Fast and Accessible Inference
  7. 2023Understanding the Impact of Post-Training Quantization on Large Language Models
  8. 2025Layer-Wise High-Impact Parameter Ratio Optimization in Post-Training Quantization for Large Language Models