mlsanityData to 7 Oct 2026

Fairness and Bias in Multimodal AI: A Survey

arXiv 2406.19097 ↗First posted 27 Jun 2024cs.CLcited by 19

On the map: Multimodal LLMs / vision-language models → Social bias and fairness in VLMs

Abstract (arXiv)

The importance of addressing fairness and bias in artificial intelligence (AI) systems cannot be over-emphasized. Mainstream media has been awashed with news of incidents around stereotypes and other types of bias in many of these systems in recent years. In this survey, we fill a gap with regards to the relatively minimal study of fairness and bias in Large Multimodal Models (LMMs) compared to Large Language Models (LLMs), providing 50 examples of datasets and models related to both types of AI along with the challenges of bias affecting them. We discuss the less-mentioned category of mitigating bias, preprocessing (with particular attention on the first part of it, which we call preuse). The method is less-mentioned compared to the two well-known ones in the literature: intrinsic and extrinsic mitigation methods. We critically discuss the various ways researchers are addressing these challenges. Our method involved two slightly different search queries on two reputable search engines, Google Scholar and Web of Science (WoS), which revealed that for the queries 'Fairness and bias in Large Multimodal Models' and 'Fairness and bias in Large Language Models', 33,400 and 538,000 links are the initial results, respectively, for Scholar while 4 and 50 links are the initial results, respectively, for WoS. For reproducibility and verification, we provide links to the search results and the citations to all the final reviewed papers. We believe this work contributes to filling this gap and providing insight to researchers and other stakeholders on ways to address the challenges of fairness and bias in multimodal and language AI.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 205 of its 337 references that are in the map. First posted on arXiv 27 Jun 2024.

Builds on

205 of 337 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023DeAR: Debiasing Vision-Language Models with Additive Residuals18 Mar
  2. 2022Counterfactually Measuring and Eliminating Social Bias in Vision-Language Pre-training Models3 Jul

Other references · 203

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  9. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  10. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  11. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  12. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
38 more
  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  13. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  14. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  15. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  16. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  17. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  18. 2015VQA: Visual Question Answeringcited by 2,489
  19. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  20. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  21. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  22. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  23. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  24. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  25. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  26. 2018Datasheets for Datasetscited by 1,371
  27. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  28. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  29. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  30. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  31. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  32. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  33. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  34. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  35. 2022Red Teaming Language Models with Language Modelscited by 983
  36. 2021Ethical and social risks of harm from Language Modelscited by 957
  37. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  38. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911

50 most cited of 203 All 203 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 14; 2026: 3.

19 papers.

Sort:

By area:

  1. 2025LLM Agents for Education: Advances and Applicationscited by 59
  2. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challengescited by 46
  3. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  4. 2025Building Trustworthy Multimodal AI: A Review of Fairness, Transparency, and Ethics in Vision-Language Taskscited by 7
  5. 2025Echoes of Power: Investigating Geopolitical Bias in US and China Large Language Modelscited by 5
  6. 2024Measuring Agreeableness Bias in Multimodal Modelscited by 4
  7. 2026Bias Beyond Borders: Political Ideology Evaluation and Steering in Multilingual LLMscited by 4
  8. 2025Findings of MEGA: Maths Explanation with LLMs using the Socratic Method for Active Learningcited by 3
  9. 2025Fairness Evaluation and Inference Level Mitigation in LLMscited by 2
  10. 2024ViLBias: Detecting and Reasoning about Bias in Multimodal Contentcited by 1
  11. 2025The Multimodal Paradox: How Added and Missing Modalities Shape Bias and Performance in Multimodal AIcited by 1
  12. 2025Inference Time Debiasing Concepts in Diffusion Modelscited by 1
7 more
  1. 2025Automated Evaluation of Gender Bias Across 13 Large Multimodal Modelscited by 1
  2. 2025SGM: Safety Glasses for Multimodal Large Language Models via Neuron-Level Detoxificationcited by 1
  3. 2026Order Matters: LVLMs as Judges for Temporal Reasoning in Image Sequencescited by 1
  4. 2025FAIRTOPIA: Envisioning Multi-Agent Guardianship for Disrupting Unfair AI Pipelinescited by 0
  5. 2025Evaluating Perspectival Biases in Cross-Modal Retrievalcited by 0
  6. 2025From the Rock Floor to the Cloud: A Systematic Survey of State-of-the-Art NLP in Battery Life Cyclecited by 0
  7. 2026Counterargument for Critical Thinking as Judged by AI and Humanscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Physics-based phenomenological characterization of cross-modal bias in multimodal models
  2. 2025FairSense-AI: Responsible AI Meets Sustainability
  3. 2024Can Prompt Modifiers Control Bias? A Comparative Analysis of Text-to-Image Generative Models
  4. 2025MALIBU Benchmark: Multi-Agent LLM Implicit Bias Uncovered
  5. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Models
  6. 2025Using LLMs as prompt modifier to avoid biases in AI image generators
  7. 2024Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspective
  8. 2026Benchmarking Bias Mitigation Toward Fairness Without Harm from Vision to LVLMs