mlsanityData to 14 Oct 2026

The Multilingual Quantization Tax: Structural Collapse and Typological Fragility in Edge SLMs

arXiv 2608.09941 ↗First posted 21 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

While 4-bit weight quantization is critical for deploying Small Language Models (SLMs) on edge devices, evaluations of the resulting performance degradation-the quantization tax-remain overwhelmingly English-centric. We present a zero-shot multilingual evaluation of 4-bit quantization across the Gemma 4 and Qwen 3.5 architectures. Evaluating on eight typo-logically diverse languages using MMLU ProX Lite and GlobalPIQA, we show parameter truncation exposes deep pre-training inequalities. We identify four phenomena: (1) Typological Fragility: low-resource and specific non-Latin scripts suffer representational collapse via architecture-specific double dissociations, failing to generate valid task logits; (2) Home Language Fragility Paradox: foundational pre-training pathways provide limited precision loss protection; (3) Domain-Specific Forgetting: multi-step cross-lingual routing degrades while associative soft-science recall remains robust; and (4) Quantization Resistance: highly saturated, typologically aligned domains resist deterministic degradation, with post-quantization performance gains bounded by statistical noise.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 13 references that are in the map. First posted on arXiv 21 Jun 2026.

Builds on

8 of 13 references.

References · 8

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  5. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  6. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 192
  7. 2025MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluationcited by 79
  8. 2024How Does Quantization Affect Multilingual LLMs?cited by 43

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giant
  2. 2026Language-Conditional Dequantization: Recovering What Quantization Steals from Non-English Languages
  3. 2026EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillation
  4. 2026Low-Bit Recurrent States in Hybrid Language Models
  5. 2025Mixed-Precision Quantization for Language Models: Techniques and Prospects
  6. 2026Accuracy is Not Enough: A Divergence-Based Approach to Evaluate Fidelity Loss in Quantized LLMs
  7. 2026Quantization Degradation in Large Language Models: A Signal-Noise Perspective
  8. 2026The Structure of Quantization Damage in LLMs: Why the Next Bit Should Be Spent Globally