mlsanityData to 7 Oct 2026

FedMentalCare: Towards Privacy-Preserving Fine-Tuned LLMs to Analyze Mental Health Status Using Federated Learning Framework

arXiv 2503.05786 ↗First posted 27 Feb 2025cs.CL cs.HC cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

With the increasing prevalence of mental health conditions worldwide, AI-powered chatbots and conversational agents have emerged as accessible tools to support mental health. However, deploying Large Language Models (LLMs) in mental healthcare applications raises significant privacy concerns, especially regarding regulations like HIPAA and GDPR. In this work, we propose FedMentalCare, a privacy-preserving framework that leverages Federated Learning (FL) combined with Low-Rank Adaptation (LoRA) to fine-tune LLMs for mental health analysis. We investigate the performance impact of varying client data volumes and model architectures (e.g., MobileBERT and MiniLM) in FL environments. Our framework demonstrates a scalable, privacy-aware approach for deploying LLMs in real-world mental healthcare scenarios, addressing data security and computational efficiency challenges.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 29 of its 57 references that are in the map. First posted on arXiv 27 Feb 2025.

Builds on

29 of 57 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  11. 2019Advances and Open Problems in Federated Learningcited by 3,021
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
17 more
  1. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  2. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  3. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  4. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  5. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  6. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  7. 2020The Future of Digital Health with Federated Learningcited by 625
  8. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  9. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 476
  10. 2021MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcarecited by 159
  11. 2023DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4cited by 128
  12. 2023Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Modelscited by 49
  13. 2023DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule Graphscited by 40
  14. 2023OphGLM: Training an Ophthalmology Large Language-and-Vision Assistant based on Instructions and Dialoguecited by 34
  15. 2023FedSecurity: Benchmarking Attacks and Defenses in Federated Learning and Federated LLMscited by 19
  16. 2024A Novel Nuanced Conversation Evaluation Framework for Large Language Models in Mental Healthcited by 7
  17. 2023The Potential and Pitfalls of using a Large Language Model such as ChatGPT or GPT-4 as a Clinical Assistantcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2025FedMentor: Domain-Aware Differential Privacy for Heterogeneous Federated LLMs in Mental Healthcited by 5
  2. 2025Menta: A Small Language Model for On-Device Mental Health Predictioncited by 3
  3. 2026Toward Zero-Egress Psychiatric AI: On-Device LLM Deployment for Privacy-Preserving Mental Health Decision Supportcited by 2
  4. 2026OracleZoom: On-Policy Self-Distillation Inspired Reference-Constrained Recursive Image Super Resolutioncited by 1
  5. 2026BEFORE THE FLIP: Measuring Hidden Score Shifts In Quantized Vision Language Models Before The Answer Changes for Visual Question Answeringcited by 1
  6. 2026Towards Privacy-Preserving Mental Health Support with Large Language Modelscited by 0
  7. 2026New Evidence, Same Choice: Testing Physical Experiment Selection in Vision Language Modelscited by 0
  8. 2026From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Healthcited by 0