mlsanityData to 14 Oct 2026

Quantization Effects on Bangla Language Understanding in Large Language Models: A Systematic Evaluation

arXiv 2608.24615 ↗First posted 25 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Post-training quantization lowers the memory footprint of Large Language Models (LLMs) and speeds up inference, which is why it is now common for on-device deployment. Most of what we know about its effects, however, comes from English benchmarks. It is not clear whether the same holds for morphologically complex, low-resource languages such as Bangla, and this gap is what we address here. We evaluate three model families---Qwen-2.5-7B, LLaMA-3.1-8B, and GPT-OSS-20B---in full precision and in three quantized formats (GPTQ-Int8, GPTQ-Q8, GGUF-W8A16) across five Bangla natural language understanding benchmarks (Bangla MMLU, CommonsenseQA-BN, OpenBookQA-BN, PIQA-BN, and BoolQ-BN), using zero-shot evaluation through lm-evaluation-harness. To our knowledge this is the first controlled comparison of quantization formats on Bangla NLU. The three families do not respond the same way: GPT-OSS loses up to 57.35% accuracy on reasoning-heavy tasks under GGUF-W8A16, while Qwen and LLaMA hold steady under GPTQ, and in a few cases the quantized version edges out the full-precision one. BoolQ-BN, a comprehension task, stays stable across all three families regardless of format. Taken together, these results suggest quantization can work well for Bangla deployment, but the choice of architecture and quantization method matters more than the bit width alone. We discuss what this means for practitioners choosing a model to run on constrained hardware.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 18 references that are in the map. First posted on arXiv 25 Aug 2026.

Builds on

12 of 18 references.

References · 12

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  4. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  5. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  6. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  8. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  9. 2024Evaluating the Generalization Ability of Quantized LLMs: Benchmark, Analysis, and Toolboxcited by 11
  10. 2024LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matricescited by 11
  11. 2025Script Fragmentation and Format: What Drives the English-Bengali Performance Gap in Open LLMs?cited by 9
  12. 2025A Comprehensive Evaluation on Quantization Techniques for Large Language Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Models
  2. 2024Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giant
  3. 2025Does quantization affect models' performance on long-context tasks?
  4. 2025The Uneven Impact of Post-Training Quantization in Machine Translation
  5. 2026Which Quantization Should I Use? A Unified Evaluation of llama.cpp Quantization on Llama-3.1-8B-Instruct
  6. 2025Resource-Efficient Language Models: Quantization for Fast and Accessible Inference
  7. 2025Towards Inclusive NLP: Assessing Compressed Multilingual Transformers across Diverse Language Benchmarks
  8. 2024Post Training Quantization of Large Language Models with Microscaling Formats