mlsanityData to 14 Oct 2026

Lightweight but Linguistically Fair? Quantisation Robustness across Bavarian, German, and English

arXiv 2610.18343 ↗First posted 13 Oct 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Post-training quantisation reduces the memory cost of large language models, but it remains unclear whether compression preserves language skills of regional varieties as reliably as their standard-language counterparts. We study this question for Bavarian, Standard German, and English using BAVGROUND, a multiple-choice benchmark, where 206 source questions are aligned across all three languages. Across 14 open-weight 7B--10B instruction models, we compare BF16 with 8-bit GPTQ and two 4-bit settings, GPTQ and AWQ. Eight-bit GPTQ remains close to BF16 and largely preserves the relative behaviour of the three languages, while 4-bit quantisation was found to be much less stable. The two 4-bit methods reduce average accuracy by about 1.4--1.5 percentage points and change 11.9--16.0% of BF16 decisions, with changes concentrated on examples where the BF16 model is already uncertain. This instability does not translate into a consistent Bavarian disadvantage. Although Bavarian, a low-resource dialect, has lower baseline accuracy than German and English, the language most affected at 4 bits varies across models and quantisation methods. Calibration further shapes these cross-language effects: the relative Bavarian--German effect changes with calibration language, while changing the calibration source has little average effect on GPTQ-4 but can affect AWQ-4 much more strongly. Free-form evaluation on five models further shows that 4-bit compression can degrade generated responses even in settings where multiple-choice accuracy changes only modestly, while 8-bit GPTQ remains comparatively stable. Overall, our results show that quantisation robustness for regional language varieties depends on the model, quantisation method, calibration data, and evaluation setting, and cannot be measured by aggregate multilingual accuracy alone.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 28 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

17 of 28 references.

References · 17

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  3. 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
  4. 2019What Do Compressed Deep Neural Networks Forget?cited by 171
  5. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 149
  6. 2023NLPositionality: Characterizing Design Biases of Datasets and Modelscited by 94
  7. 2021The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translationcited by 47
  8. 2024How Does Quantization Affect Multilingual LLMs?cited by 43
  9. 2015Resiliency of Deep Neural Networks under Quantizationcited by 35
  10. 2024Assessing Dialect Fairness and Robustness of Large Language Models in Reasoning Taskscited by 22
  11. 2024Quantifying the Capabilities of LLMs across Scale and Precisioncited by 19
  12. 2024Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giantcited by 14
5 more
  1. 2024Dialectal Toxicity Detection: Evaluating LLM-as-a-Judge Consistency Across Language Varietiescited by 12
  2. 2024Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behindcited by 11
  3. 2025Llama-GENBA-10B: A Trilingual Large Language Model for German, English and Bavariancited by 4
  4. 2025Evaluating the Impact of Post-Training Quantization on Large Language Models for Code Generationcited by 3
  5. 2026BavGround: A Benchmark for Regional Cultural Grounding and Dialect Competence in Bavariancited by 1

Built on by

Not cited yet.