# FAIR_XAI: Improving Multimodal Foundation Model Fairness via Explainability for Wellbeing Assessment

arXiv: https://arxiv.org/abs/2604.23786 · HTML: https://mlsanity.com/paper/2604.23786/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 70 references that are in the map.
- First posted on arXiv 26 Apr 2026.

## Builds on (most cited first, 30 of 30)

- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [InterpretML: A Unified Framework for Machine Learning Interpretability](https://mlsanity.com/paper/1909.09223/) (2019)
- [Debiasing Vision-Language Models via Biased Prompts](https://mlsanity.com/paper/2302.00070/) (2023)
- [Systematic Biases in LLM Simulations of Debates](https://mlsanity.com/paper/2402.04049/) (2024)
- [AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognition](https://mlsanity.com/paper/1907.11510/) (2019)
- [Bias in Large Language Models: Origin, Evaluation, and Mitigation](https://mlsanity.com/paper/2411.10915/) (2024)
- [ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features](https://mlsanity.com/paper/2502.04320/) (2025)
- [Markedness in Visual Semantic AI](https://mlsanity.com/paper/2205.11378/) (2022)
- [Responsible AI: Gender bias assessment in emotion recognition](https://mlsanity.com/paper/2103.11436/) (2021)
- [Fairness and Explainability: Bridging the Gap Towards Fair Model Explanations](https://mlsanity.com/paper/2212.03840/) (2022)
- [Focus! Rating XAI Methods and Finding Biases](https://mlsanity.com/paper/2109.15035/) (2021)
- [U-Fair: Uncertainty-based Multimodal Multitask Learning for Fairer Depression Detection](https://mlsanity.com/paper/2501.09687/) (2025)
- [Small but Fair! Fairness for Multimodal Human-Human and Robot-Human Mental Wellbeing Coaching](https://mlsanity.com/paper/2407.01562/) (2024)
- [MER 2025: When Affective Computing Meets Large Language Models](https://mlsanity.com/paper/2504.19423/) (2025)
- [Multimodal Gender Fairness in Depression Prediction: Insights on Data from the USA & China](https://mlsanity.com/paper/2408.04026/) (2024)
- [Machine Learning Fairness for Depression Detection using EEG Data](https://mlsanity.com/paper/2501.18192/) (2025)
- [Underneath the Numbers: Quantitative and Qualitative Gender Fairness in LLMs for Depression Prediction](https://mlsanity.com/paper/2406.08183/) (2024)
- [What People Share With a Robot When Feeling Lonely and Stressed and How It Helps Over Time](https://mlsanity.com/paper/2504.02991/) (2025)
- [MindBenchAI: An Actionable Platform to Evaluate the Profile and Performance of Large Language Models in a Mental Healthcare Context](https://mlsanity.com/paper/2510.13812/) (2025)
- [Robot-Led Vision Language Model Wellbeing Assessment of Children](https://mlsanity.com/paper/2504.02765/) (2025)
- [LEXI: Large Language Models Experimentation Interface](https://mlsanity.com/paper/2407.01488/) (2024)
- [Gender Fairness of Machine Learning Algorithms for Pain Detection](https://mlsanity.com/paper/2506.11132/) (2025)
- [Exploring Causality for HRI: A Case Study on Robotic Mental Well-being Coaching](https://mlsanity.com/paper/2503.11684/) (2025)
- [Automatic Depression Assessment using Machine Learning: A Comprehensive Survey](https://mlsanity.com/paper/2506.18915/) (2025)
- [FairSSL: Fair Multimodal Self-Supervised Learning](https://mlsanity.com/paper/2508.16748/) (2025)
- [Investigating Associational Biases in Inter-Model Communication of Large Generative Models](https://mlsanity.com/paper/2601.22093/) (2026)

## Abstract (arXiv)

In recent years, the integration of multimodal machine learning in wellbeing assessment has offered transformative potential for monitoring mental health. However, with the rapid advancement of Vision-Language Models (VLMs), their deployment in clinical settings has raised concerns due to their lack of transparency and potential for bias. While previous research has explored the intersection of fairness and Explainable AI (XAI), its application to VLMs for wellbeing assessment and depression prediction remains under-explored. This work investigates VLM performance across laboratory (AFAR-BSFT) and naturalistic (E-DAIC) datasets, focusing on diagnostic reliability and demographic fairness. Performance varied substantially across environments and architectures; Phi3.5-Vision achieved 80.4% accuracy on E-DAIC, while Qwen2-VL struggled at 33.9%. Additionally, both models demonstrated a tendency to over-predict depression on AFAR-BSFT. Although bias existed across both architectures, Qwen2-VL showed higher gender disparities, while Phi-3.5-Vision exhibited more racial bias. Our XAI intervention framework yielded mixed results; fairness prompting achieved perfect equal opportunity for Qwen2-VL at a severe accuracy cost on E-DAIC. On AFAR-BSFT, explainability-based interventions improved procedural consistency but did not guarantee outcome fairness, sometimes amplifying racial bias. These results highlight a persistent gap between procedural transparency and equitable outcomes. We analyse these findings and consolidate concrete recommendations for addressing them, emphasising that future fairness interventions must jointly optimise predictive accuracy, demographic parity, and cross-domain generalisation.