# Underneath the Numbers: Quantitative and Qualitative Gender Fairness in LLMs for Depression Prediction

arXiv: https://arxiv.org/abs/2406.08183 · HTML: https://mlsanity.com/paper/2406.08183/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 59 references that are in the map.
- First posted on arXiv 12 Jun 2024.

## Builds on (most cited first, 18 of 18)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- ["Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters](https://mlsanity.com/paper/2310.09219/) (2023)
- [Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey](https://mlsanity.com/paper/2207.07068/) (2022)
- [AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognition](https://mlsanity.com/paper/1907.11510/) (2019)
- [Evaluation of ChatGPT for NLP-based Mental Health Applications](https://mlsanity.com/paper/2303.15727/) (2023)
- [ROBBIE: Robust Bias Evaluation of Large Generative Language Models](https://mlsanity.com/paper/2311.18140/) (2023)
- [Disclosure and Mitigation of Gender Bias in LLMs](https://mlsanity.com/paper/2402.11190/) (2024)
- [How Do Fairness Definitions Fare? Examining Public Attitudes Towards Algorithmic Definitions of Fairness](https://mlsanity.com/paper/1811.03654/) (2018)
- [Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4](https://mlsanity.com/paper/2312.16171/) (2023)
- [Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AI](https://mlsanity.com/paper/2206.10847/) (2022)
- [Probing Explicit and Implicit Gender Bias through LLM Conditional Text Generation](https://mlsanity.com/paper/2311.00306/) (2023)
- [Unveiling Gender Bias in Terms of Profession Across LLMs: Analyzing and Addressing Sociological Implications](https://mlsanity.com/paper/2307.09162/) (2023)
- [Gender Bias in Depression Detection Using Audio Features](https://mlsanity.com/paper/2010.15120/) (2020)
- [Bias Reducing Multitask Learning on Mental Health Prediction](https://mlsanity.com/paper/2208.03621/) (2022)
- [Robots as Mental Well-being Coaches: Design and Ethical Recommendations](https://mlsanity.com/paper/2208.14874/) (2022)

## Built on by (most cited first, 6 of 6)

- [U-Fair: Uncertainty-based Multimodal Multitask Learning for Fairer Depression Detection](https://mlsanity.com/paper/2501.09687/) (2025)
- [Multimodal Gender Fairness in Depression Prediction: Insights on Data from the USA & China](https://mlsanity.com/paper/2408.04026/) (2024)
- [Gender Fairness of Machine Learning Algorithms for Pain Detection](https://mlsanity.com/paper/2506.11132/) (2025)
- [PinLanding: Content-First Keyword Landing Page Generation via Multi-Modal AI for Web-Scale Discovery](https://mlsanity.com/paper/2503.00619/) (2025)
- [Who Owns The Robot?: Four Ethical and Socio-technical Questions about Wellbeing Robots in the Real World through Community Engagement](https://mlsanity.com/paper/2509.02624/) (2025)
- [FAIR_XAI: Improving Multimodal Foundation Model Fairness via Explainability for Wellbeing Assessment](https://mlsanity.com/paper/2604.23786/) (2026)

## Abstract (arXiv)

Recent studies show bias in many machine learning models for depression detection, but bias in LLMs for this task remains unexplored. This work presents the first attempt to investigate the degree of gender bias present in existing LLMs (ChatGPT, LLaMA 2, and Bard) using both quantitative and qualitative approaches. From our quantitative evaluation, we found that ChatGPT performs the best across various performance metrics and LLaMA 2 outperforms other LLMs in terms of group fairness metrics. As qualitative fairness evaluation remains an open research question we propose several strategies (e.g., word count, thematic analysis) to investigate whether and how a qualitative evaluation can provide valuable insights for bias analysis beyond what is possible with quantitative evaluation. We found that ChatGPT consistently provides a more comprehensive, well-reasoned explanation for its prediction compared to LLaMA 2. We have also identified several themes adopted by LLMs to qualitatively evaluate gender fairness. We hope our results can be used as a stepping stone towards future attempts at improving qualitative evaluation of fairness for LLMs especially for high-stakes tasks such as depression detection.