# Micromodels for Efficient, Explainable, and Reusable Systems: A Case Study on Mental Health

arXiv: https://arxiv.org/abs/2109.13770 · HTML: https://mlsanity.com/paper/2109.13770/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 73 references that are in the map.
- First posted on arXiv 28 Sep 2021.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [K-BERT: Enabling Language Representation with Knowledge Graph](https://mlsanity.com/paper/1909.07606/) (2019)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [InterpretML: A Unified Framework for Machine Learning Interpretability](https://mlsanity.com/paper/1909.09223/) (2019)
- [Finding Universal Grammatical Relations in Multilingual BERT](https://mlsanity.com/paper/2005.04511/) (2020)
- [Depression and Self-Harm Risk Assessment in Online Forums](https://mlsanity.com/paper/1709.01848/) (2017)
- [SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions](https://mlsanity.com/paper/1806.05258/) (2018)
- [Training Classifiers with Natural Language Explanations](https://mlsanity.com/paper/1805.03818/) (2018)
- [Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Tasks](https://mlsanity.com/paper/1801.07772/) (2018)
- [Outlier Detection for Improved Data Quality and Diversity in Dialog Systems](https://mlsanity.com/paper/1904.03122/) (2019)
- [Multi-Task Learning for Mental Health using Social Media Text](https://mlsanity.com/paper/1712.03538/) (2017)
- [Infusing Finetuning with Semantic Dependencies](https://mlsanity.com/paper/2012.05395/) (2020)
- [Gender and Racial Fairness in Depression Research using Social Media](https://mlsanity.com/paper/2103.10550/) (2021)
- [Attention-Based LSTM for Psychological Stress Detection from Spoken Language Using Distant Supervision](https://mlsanity.com/paper/1805.12307/) (2018)
- [A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis](https://mlsanity.com/paper/2004.12150/) (2020)

## Built on by (most cited first, 13 of 13)

- [MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare](https://mlsanity.com/paper/2110.15621/) (2021)
- [Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnaires](https://mlsanity.com/paper/2204.10432/) (2022)
- [Logical Fallacy Detection](https://mlsanity.com/paper/2202.13758/) (2022)
- [Symptom Identification for Interpretable Detection of Multiple Mental Disorders](https://mlsanity.com/paper/2205.11308/) (2022)
- [A Benchmark for Understanding Dialogue Safety in Mental Health Support](https://mlsanity.com/paper/2307.16457/) (2023)
- [Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models](https://mlsanity.com/paper/2305.12544/) (2023)
- [Empathy Detection from Text, Audiovisual, Audio or Physiological Signals: A Systematic Review of Task Formulations and Machine Learning Methods](https://mlsanity.com/paper/2311.00721/) (2023)
- [Then and Now: Quantifying the Longitudinal Validity of Self-Disclosed Depression Diagnoses](https://mlsanity.com/paper/2206.11155/) (2022)
- [A Survey of Cognitive Distortion Detection and Classification in NLP](https://mlsanity.com/paper/2508.09878/) (2025)
- [DECK: Behavioral Tests to Improve Interpretability and Generalizability of BERT Models Detecting Depression from Text](https://mlsanity.com/paper/2209.05286/) (2022)
- [Responsible Evaluation of AI for Mental Health](https://mlsanity.com/paper/2602.00065/) (2026)
- [Predicting Treatment Adherence of Tuberculosis Patients at Scale](https://mlsanity.com/paper/2211.02943/) (2022)
- [Conceptualizing Suicidal Behavior: Utilizing Explanations of Predicted Outcomes to Analyze Longitudinal Social Media Data](https://mlsanity.com/paper/2312.08299/) (2023)

## Abstract (arXiv)

Many statistical models have high accuracy on test benchmarks, but are not explainable, struggle in low-resource scenarios, cannot be reused for multiple tasks, and cannot easily integrate domain expertise. These factors limit their use, particularly in settings such as mental health, where it is difficult to annotate datasets and model outputs have significant impact. We introduce a micromodel architecture to address these challenges. Our approach allows researchers to build interpretable representations that embed domain knowledge and provide explanations throughout the model's decision process. We demonstrate the idea on multiple mental health tasks: depression classification, PTSD classification, and suicidal risk assessment. Our systems consistently produce strong results, even in low-resource scenarios, and are more interpretable than alternative methods.