# WellDunn: On the Robustness and Explainability of Language Models and Large Language Models in Identifying Wellness Dimensions

arXiv: https://arxiv.org/abs/2406.12058 · HTML: https://mlsanity.com/paper/2406.12058/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 66 references that are in the map.
- First posted on arXiv 17 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [ERNIE: Enhanced Representation through Knowledge Integration](https://mlsanity.com/paper/1904.09223/) (2019)
- [MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data](https://mlsanity.com/paper/2304.08247/) (2023)
- [A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics](https://mlsanity.com/paper/2310.05694/) (2023)
- [MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare](https://mlsanity.com/paper/2110.15621/) (2021)
- [Deep Gamblers: Learning to Abstain with Portfolio Theory](https://mlsanity.com/paper/1907.00208/) (2019)
- [Deep Learning for Suicide and Depression Identification with Unsupervised Label Correction](https://mlsanity.com/paper/2102.09427/) (2021)
- [Primal-Attention: Self-attention through Asymmetric Kernel SVD in Primal Representation](https://mlsanity.com/paper/2305.19798/) (2023)
- [Learning to Automate Follow-up Question Generation using Process Knowledge for Depression Triage on Reddit Posts](https://mlsanity.com/paper/2205.13884/) (2022)
- [Building Trustworthy NeuroSymbolic AI Systems: Consistency, Reliability, Explainability, and Safety](https://mlsanity.com/paper/2312.06798/) (2023)
- [Building and Using Personal Knowledge Graph to Improve Suicidal Ideation Detection on Social Media](https://mlsanity.com/paper/2012.09123/) (2020)
- [Towards Effective Paraphrasing for Information Disguise](https://mlsanity.com/paper/2311.05018/) (2023)
- [WellXplain: Wellness Concept Extraction and Classification in Reddit Posts for Mental Health Analysis](https://mlsanity.com/paper/2308.13710/) (2023)
- [Augmenting Reddit Posts to Determine Wellness Dimensions impacting Mental Health](https://mlsanity.com/paper/2306.04059/) (2023)

## Built on by (most cited first, 10 of 10)

- [A Survey of Large Language Models in Psychotherapy: Current Landscape and Future Directions](https://mlsanity.com/paper/2502.11095/) (2025)
- [FedMentor: Domain-Aware Differential Privacy for Heterogeneous Federated LLMs in Mental Health](https://mlsanity.com/paper/2509.14275/) (2025)
- [Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions](https://mlsanity.com/paper/2509.02452/) (2025)
- [LLAMADRS: Evaluating Open-Source LLMs on Real Clinical Interviews--To Reason or Not to Reason?](https://mlsanity.com/paper/2501.03624/) (2025)
- [Found in Translation: Measuring Multilingual LLM Consistency as Simple as Translate then Evaluate](https://mlsanity.com/paper/2505.21999/) (2025)
- [Mental Health Equity in LLMs: Leveraging Multi-Hop Question Answering to Detect Amplified and Silenced Perspectives](https://mlsanity.com/paper/2506.18116/) (2025)
- [SymLoc: Symbolic Localization of Hallucination across HaluEval and TruthfulQA](https://mlsanity.com/paper/2511.14172/) (2025)
- [IoT-Based Preventive Mental Health Using Knowledge Graphs and Standards for Better Well-Being](https://mlsanity.com/paper/2406.13791/) (2024)
- [Holistix: A Dataset for Holistic Wellness Dimensions Analysis in Mental Health Narratives](https://mlsanity.com/paper/2507.09565/) (2025)
- [Bag of Tricks or Bag of Myths? Reducing Modeling Complexity with Task Knowledge in Explainable Suicide Risk Assessment](https://mlsanity.com/paper/2609.07766/) (2026)

## Abstract (arXiv)

Language Models (LMs) are being proposed for mental health applications where the heightened risk of adverse outcomes means predictive performance may not be a sufficient litmus test of a model's utility in clinical practice. A model that can be trusted for practice should have a correspondence between explanation and clinical determination, yet no prior research has examined the attention fidelity of these models and their effect on ground truth explanations. We introduce an evaluation design that focuses on the robustness and explainability of LMs in identifying Wellness Dimensions (WDs). We focus on two existing mental health and well-being datasets: (a) Multi-label Classification-based MultiWD, and (b) WellXplain for evaluating attention mechanism veracity against expert-labeled explanations. The labels are based on Halbert Dunn's theory of wellness, which gives grounding to our evaluation. We reveal four surprising results about LMs/LLMs: (1) Despite their human-like capabilities, GPT-3.5/4 lag behind RoBERTa, and MedAlpaca, a fine-tuned LLM on WellXplain fails to deliver any remarkable improvements in performance or explanations. (2) Re-examining LMs' predictions based on a confidence-oriented loss function reveals a significant performance drop. (3) Across all LMs/LLMs, the alignment between attention and explanations remains low, with LLMs scoring a dismal 0.0. (4) Most mental health-specific LMs/LLMs overlook domain-specific knowledge and undervalue explanations, causing these discrepancies. This study highlights the need for further research into their consistency and explanations in mental health and well-being.