# MoodCapture: Depression Detection Using In-the-Wild Smartphone Images

arXiv: https://arxiv.org/abs/2402.16182 · HTML: https://mlsanity.com/paper/2402.16182/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 84 references that are in the map.
- First posted on arXiv 25 Feb 2024.

## Builds on (most cited first, 9 of 9)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Rendering of Eyes for Eye-Shape Registration and Gaze Estimation](https://mlsanity.com/paper/1505.05916/) (2015)
- [Topic Modeling Based Multi-modal Depression Detection](https://mlsanity.com/paper/1803.10384/) (2018)
- [Instagram photos reveal predictive markers of depression](https://mlsanity.com/paper/1608.03282/) (2016)
- [What Twitter Profile and Posted Images Reveal About Depression and Anxiety](https://mlsanity.com/paper/1904.02670/) (2019)
- [Social Media Image Analysis for Public Health](https://mlsanity.com/paper/1512.04476/) (2015)
- [Adaptive App Design by Detecting Handedness](https://mlsanity.com/paper/1805.08367/) (2018)

## Built on by (most cited first, 8 of 8)

- [Predicting Trust In Autonomous Vehicles: Modeling Young Adult Psychosocial Traits, Risk-Benefit Attitudes, And Driving Factors With Machine Learning](https://mlsanity.com/paper/2409.08980/) (2024)
- [TimeSRL: Generalizable Time-Series Behavioral Modeling via Semantic RL-Tuned LLMs -- A Case Study in Mental Health](https://mlsanity.com/paper/2605.21295/) (2026)
- [AI-Driven Early Mental Health Screening: Analyzing Selfies of Pregnant Women](https://mlsanity.com/paper/2410.05450/) (2024)
- [Technology as uncharted territory: Contextual integrity and the notion of AI as new ethical ground](https://mlsanity.com/paper/2412.05130/) (2024)
- [Development of Mental Models in Human-AI Collaboration: A Conceptual Framework](https://mlsanity.com/paper/2510.08104/) (2025)
- [Evaluating Federated Learning for Cross-Country Mood Inference from Smartphone Sensing Data](https://mlsanity.com/paper/2602.15478/) (2026)
- [EmoTrack: Clinical-Semantic Modeling for Text-Based Depression Severity Estimation](https://mlsanity.com/paper/2605.22286/) (2026)
- [LongCounsel-8: A Benchmark Suite for Longitudinal Depression Tracking from Multi-Session Counseling Dialogues](https://mlsanity.com/paper/2609.03507/) (2026)

## Abstract (arXiv)

MoodCapture presents a novel approach that assesses depression based on images automatically captured from the front-facing camera of smartphones as people go about their daily lives. We collect over 125,000 photos in the wild from N=177 participants diagnosed with major depressive disorder for 90 days. Images are captured naturalistically while participants respond to the PHQ-8 depression survey question: textit``I have felt down, depressed, or hopeless''. Our analysis explores important image attributes, such as angle, dominant colors, location, objects, and lighting. We show that a random forest trained with face landmarks can classify samples as depressed or non-depressed and predict raw PHQ-8 scores effectively. Our post-hoc analysis provides several insights through an ablation study, feature importance analysis, and bias assessment. Importantly, we evaluate user concerns about using MoodCapture to detect depression based on sharing photos, providing critical insights into privacy concerns that inform the future design of in-the-wild image-based mental health assessment tools.