# More is Less? A Simulation-Based Approach to Dynamic Interactions between Biases in Multimodal Models

arXiv: https://arxiv.org/abs/2412.17505 · HTML: https://mlsanity.com/paper/2412.17505/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 23 Dec 2024.

## Builds on (most cited first, 26 of 26)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [Towards Understanding and Mitigating Social Biases in Language Models](https://mlsanity.com/paper/2106.13219/) (2021)
- [Gender bias and stereotypes in Large Language Models](https://mlsanity.com/paper/2308.14921/) (2023)
- [Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings](https://mlsanity.com/paper/1904.04047/) (2019)
- [Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review](https://mlsanity.com/paper/2403.02469/) (2024)
- [A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning](https://mlsanity.com/paper/2203.11933/) (2022)
- [Bias and Generalization in Deep Generative Models: An Empirical Study](https://mlsanity.com/paper/1811.03259/) (2018)
- [Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2104.08666/) (2021)
- [CLIPSyntel: CLIP and LLM Synergy for Multimodal Question Summarization in Healthcare](https://mlsanity.com/paper/2312.11541/) (2023)
- [Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspective](https://mlsanity.com/paper/2403.18346/) (2024)
- [Exposing and Correcting the Gender Bias in Image Captioning Datasets and Models](https://mlsanity.com/paper/1912.00578/) (2019)
- [CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?](https://mlsanity.com/paper/2403.04547/) (2024)
- [Multimodal Categorization of Crisis Events in Social Media](https://mlsanity.com/paper/2004.04917/) (2020)
- [Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitment](https://mlsanity.com/paper/2302.10908/) (2023)
- [Review of multimodal machine learning approaches in healthcare](https://mlsanity.com/paper/2402.02460/) (2024)
- [Towards Multimodal Sentiment Analysis Debiasing via Bias Purification](https://mlsanity.com/paper/2403.05023/) (2024)
- [Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response](https://mlsanity.com/paper/2004.11838/) (2020)
- [Bias in Multimodal AI: Testbed for Fair Automatic Recruitment](https://mlsanity.com/paper/2004.07173/) (2020)
- [Multimodal Composite Association Score: Measuring Gender Bias in Generative Multimodal Models](https://mlsanity.com/paper/2304.13855/) (2023)
- [Safety and Fairness for Content Moderation in Generative Models](https://mlsanity.com/paper/2306.06135/) (2023)
- [Bias in Machine Learning Models Can Be Significantly Mitigated by Careful Training: Evidence from Neuroimaging Studies](https://mlsanity.com/paper/2205.13421/) (2022)
- [Discover and Mitigate Multiple Biased Subgroups in Image Classifiers](https://mlsanity.com/paper/2403.12777/) (2024)
- [Bias and Fairness on Multimodal Emotion Detection Algorithms](https://mlsanity.com/paper/2205.08383/) (2022)
- [Rethinking Multimodal Content Moderation from an Asymmetric Angle with Mixed-modality](https://mlsanity.com/paper/2305.10547/) (2023)
- [Formalizing Multimedia Recommendation through Multimodal Deep Learning](https://mlsanity.com/paper/2309.05273/) (2023)

## Built on by (most cited first, 1 of 1)

- [Order Matters: LVLMs as Judges for Temporal Reasoning in Image Sequences](https://mlsanity.com/paper/2608.10908/) (2026)

## Abstract (arXiv)

Multimodal machine learning models, such as those that combine text and image modalities, are increasingly used in critical domains including public safety, security, and healthcare. However, these systems inherit biases from their single modalities. This study proposes a systemic framework for analyzing dynamic multimodal bias interactions. Using the MMBias dataset, which encompasses categories prone to bias such as religion, nationality, and sexual orientation, this study adopts a simulation-based heuristic approach to compute bias scores for text-only, image-only, and multimodal embeddings. A framework is developed to classify bias interactions as amplification (multimodal bias exceeds both unimodal biases), mitigation (multimodal bias is lower than both), and neutrality (multimodal bias lies between unimodal biases), with proportional analyzes conducted to identify the dominant mode and dynamics in these interactions. The findings highlight that amplification (22%) occurs when text and image biases are comparable, while mitigation (11%) arises under the dominance of text bias, highlighting the stabilizing role of image bias. Neutral interactions (67%) are related to a higher text bias without divergence. Conditional probabilities highlight the text's dominance in mitigation and mixed contributions in neutral and amplification cases, underscoring complex modality interplay. In doing so, the study encourages the use of this heuristic, systemic, and interpretable framework to analyze multimodal bias interactions, providing insight into how intermodal biases dynamically interact, with practical applications for multimodal modeling and transferability to context-based datasets, all essential for developing fair and equitable AI models.