mlsanityData to 7 Oct 2026

Menta: A Small Language Model for On-Device Mental Health Prediction

arXiv 2512.02716 ↗First posted 2 Dec 2025cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Mental health conditions affect hundreds of millions globally, yet early detection remains limited. While large language models (LLMs) have shown promise in mental health applications, their size and computational demands hinder practical deployment. Small language models (SLMs) offer a lightweight alternative, but their use for social media--based mental health prediction remains largely underexplored. In this study, we introduce Menta, the first optimized SLM fine-tuned specifically for multi-task mental health prediction from social media data. Menta is jointly trained across six classification tasks using a LoRA-based framework, a cross-dataset strategy, and a balanced accuracy--oriented loss. Evaluated against nine state-of-the-art SLM baselines, Menta achieves an average improvement of 15.2% across tasks covering depression, stress, and suicidality compared with the best-performing non--fine-tuned SLMs. It also achieves higher accuracy on depression and stress classification tasks compared to 13B-parameter LLMs, while being approximately 3.25x smaller. Moreover, we demonstrate real-time, on-device deployment of Menta on an iPhone 15 Pro Max, requiring only approximately 3GB RAM. Supported by a comprehensive benchmark against existing SLMs and LLMs, Menta highlights the potential for scalable, privacy-preserving mental health monitoring. Code is available at: https://hong-labs.github.io/menta-project/

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 47 of its 100 references that are in the map. First posted on arXiv 2 Dec 2025.

Builds on

47 of 100 references.

References · 47

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  4. 2024Qwen2 Technical Reportcited by 2,076
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  6. 2025Gemma 3 Technical Reportcited by 1,686
  7. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  8. 2024Phi-4 Technical Reportcited by 675
  9. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  10. 2023The Falcon Series of Open Language Modelscited by 526
  11. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  12. 2025Small Language Models are the Future of Agentic AIcited by 254
35 more
  1. 2021MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcarecited by 159
  2. 2019What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Usecited by 134
  3. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 125
  4. 2024Small Language Models: Survey, Measurements, and Insightscited by 109
  5. 2019Dreaddit: A Reddit Dataset for Stress Analysis in Social Mediacited by 81
  6. 2023MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Modelscited by 63
  7. 2023Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Taskscited by 62
  8. 2023Evaluation of ChatGPT for NLP-based Mental Health Applicationscited by 53
  9. 2023Towards Interpretable Mental Health Analysis with Large Language Modelscited by 53
  10. 2021Deep Learning for Suicide and Depression Identification with Unsupervised Label Correctioncited by 45
  11. 2024Unveiling the Generalization Power of Fine-Tuned Large Language Modelscited by 35
  12. 2024A Survey of Small Language Modelscited by 31
  13. 2025The Rise of Small Language Models in Healthcare: A Comprehensive Surveycited by 18
  14. 2024Comparing Specialised Small and General Large Language Models on Text Classification: 100 Labelled Samples to Achieve Break-Even Performancecited by 17
  15. 2024Small Language Models are Good Too: An Empirical Study of Zero-Shot Classificationcited by 17
  16. 2024Stable Code Technical Reportcited by 16
  17. 2024I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responsescited by 13
  18. 2024Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thoughtcited by 12
  19. 2023FedTherapist: Mental Health Monitoring with User-Generated Linguistic Expressions on Smartphones via Federated Learningcited by 10
  20. 2024SlimLM: An Efficient Small Language Model for On-Device Document Assistancecited by 9
  21. 2025FedMentalCare: Towards Privacy-Preserving Fine-Tuned LLMs to Analyze Mental Health Status Using Federated Learning Frameworkcited by 8
  22. 2022EmoMent: An Emotion Annotated Mental Health Corpus from two South Asian Countriescited by 7
  23. 2025Survey and Experiments on Mental Disorder Detection via Social Media: From Large Language Models and RAG to Agentscited by 6
  24. 2025Interpretable Depression Detection from Social Media Text Using LLM-Derived Embeddingscited by 6
  25. 2025Beyond Scale: Small Language Models are Comparable to GPT-4 in Mental Health Understandingcited by 6
  26. 2024mhGPT: A Lightweight Generative Pre-Trained Transformer for Mental Health Text Analysiscited by 5
  27. 2025Small or Large? Zero-Shot or Finetuned? Guiding Language Model Choice for Specialized Applications in Healthcarecited by 5
  28. 2024Advancing Depression Detection on Social Media Platforms Through Fine-Tuned Large Language Modelscited by 4
  29. 2025A Gold Standard Dataset and Evaluation Framework for Depression Detection and Explanation in Social Media using LLMscited by 4
  30. 2024Detecting mental disorder on social media: a ChatGPT-augmented explainable approachcited by 3
  31. 2024Using LLMs to Aid Annotation and Collection of Clinically-Enriched Data in Bipolar Disorder and Schizophreniacited by 3
  32. 2025HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoringcited by 3
  33. 2025Towards Privacy-aware Mental Health AI Models: Advances, Challenges, and Opportunitiescited by 2
  34. 2025Generating Medically-Informed Explanations for Depression Detection using LLMscited by 1
  35. 2025Pruning Laws for Large Language Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026TRACE: A taxonomy-grounded synthetic dataset for teaching-program generation and session interpretation in Applied Behavior Analysiscited by 0
  2. 2026Titans-as-a-Layer: Test-Time Memory for Conversational Speech Emotion Recognitioncited by 0
  3. 2026Reading Anxiety or Reading the Label? Comparing Fine-Tuned and Frontier Models for Anxiety Detection on Social Mediacited by 0