mlsanityData to 7 Oct 2026

You Never Know: Quantization Induces Inconsistent Biases in Vision-Language Foundation Models

arXiv 2410.20265 ↗First posted 26 Oct 2024cs.CV cs.CY cs.LGcited by 2

On the map: Multimodal LLMs / vision-language models → Social bias and fairness in VLMs

Abstract (arXiv)

We study the impact of a standard practice in compressing foundation vision-language models - quantization - on the models' ability to produce socially-fair outputs. In contrast to prior findings with unimodal models that compression consistently amplifies social biases, our extensive evaluation of four quantization settings across three datasets and three CLIP variants yields a surprising result: while individual models demonstrate bias, we find no consistent change in bias magnitude or direction across a population of compressed models due to quantization.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 35 references that are in the map. First posted on arXiv 26 Oct 2024.

Builds on

28 of 35 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024FairDeDup: Detecting and Mitigating Vision-Language Fairness Disparities in Semantic Dataset Deduplication24 Apr
  2. 2023DeAR: Debiasing Vision-Language Models with Additive Residuals18 Mar
  3. 2022A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning22 Mar

Other references · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  9. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  10. 20218-bit Optimizers via Block-wise Quantizationcited by 395
  11. 2019What Do Compressed Deep Neural Networks Forget?cited by 164
  12. 2020Characterising Bias in Compressed Modelscited by 162
13 more
  1. 2019Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Searchcited by 137
  2. 2022FairPrune: Achieving Fairness Through Pruning for Dermatological Disease Diagnosiscited by 59
  3. 2023FACET: Fairness in Computer Vision Evaluation Benchmarkcited by 58
  4. 2022Pruning has a disparate impact on model accuracycited by 43
  5. 2023Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classificationcited by 31
  6. 2022FairGRAPE: Fairness-aware GRAdient Pruning mEthod for Face Attribute Classificationcited by 30
  7. 2021Gender Bias Amplification During Speed-Quality Optimization in Neural Machine Translationcited by 28
  8. 2023ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Modelscited by 19
  9. 2021Stealthy Backdoors as Compression Artifactscited by 17
  10. 2021Qu-ANTI-zation: Exploiting Quantization Artifacts for Achieving Adversarial Outcomescited by 16
  11. 2022Intriguing Properties of Compression on Multilingual Modelscited by 15
  12. 2022The Effect of Model Compression on Fairness in Facial Expression Recognitioncited by 13
  13. 2023Debiased Distillation by Transplanting the Last Layercited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Less Precise Can Be More Reliable: A Systematic Evaluation of Quantization's Impact on VLMs Beyond Accuracycited by 3
  2. 2026Edge Reliability Gap in Vision-Language Models: Quantifying Failure Modes of Compressed VLMs Under Visual Corruptioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Biases Propagate in Encoder-based Vision-Language Models: A Systematic Analysis From Intrinsic Measures to Zero-shot Retrieval Outcomes
  2. 2025How Quantization Shapes Bias in Large Language Models
  3. 2025Data Matters Most: Auditing Social Bias in Contrastive Vision Language Models
  4. 2025Intrinsic Bias is Predicted by Pretraining Data and Correlates with Downstream Performance in Vision-Language Encoders
  5. 2024Images Speak Louder than Words: Understanding and Mitigating Bias in Vision-Language Model from a Causal Mediation Perspective
  6. 2026Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels
  7. 2024Dataset Scale and Societal Consistency Mediate Facial Impression Bias in Vision-Language AI
  8. 2024A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models