# A Vector-Quantized Foundation Model for Patient Behavior Monitoring

arXiv: https://arxiv.org/abs/2503.15221 · HTML: https://mlsanity.com/paper/2503.15221/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 19 Mar 2025.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [Handling Incomplete Heterogeneous Data using VAEs](https://mlsanity.com/paper/1807.03653/) (2018)
- [Discovering Discrete Latent Topics with Neural Variational Inference](https://mlsanity.com/paper/1706.00359/) (2017)
- [Self-supervised Learning for Human Activity Recognition Using 700,000 Person-days of Wearable Data](https://mlsanity.com/paper/2206.02909/) (2022)
- [Scaling Wearable Foundation Models](https://mlsanity.com/paper/2410.13638/) (2024)
- [VAEs in the Presence of Missing Data](https://mlsanity.com/paper/2006.05301/) (2020)
- [Discrete Latent Variable Representations for Low-Resource Text Classification](https://mlsanity.com/paper/2006.06226/) (2020)
- [Representation Learning on Variable Length and Incomplete Wearable-Sensory Time Series](https://mlsanity.com/paper/2002.03595/) (2020)
- [Medical data wrangling with sequential variational autoencoders](https://mlsanity.com/paper/2103.07206/) (2021)
- [PyHHMM: A Python Library for Heterogeneous Hidden Markov Models](https://mlsanity.com/paper/2201.06968/) (2022)
- [Change-Point Detection on Hierarchical Circadian Models](https://mlsanity.com/paper/1809.04197/) (2018)
- [Passive detection of behavioral shifts for suicide attempt prevention](https://mlsanity.com/paper/2011.09848/) (2020)

## Abstract (arXiv)

Foundation models have achieved remarkable success across various domains, yet their adoption in healthcare remains limited. While significant advances have been made in medical imaging, genetic biomarkers, and time series from electronic health records, the potential of foundation models for patient behavior monitoring through personal digital devices remains underexplored. The data generated by these devices are inherently heterogeneous, multisource, and often exhibit high rates of missing data, posing unique challenges. This paper introduces a novel foundation model based on a modified vector quantized variational autoencoder, specifically designed to process real-world data from smartphones and wearable devices. We leveraged the discrete latent representation of this model to effectively perform two downstream tasks, suicide risk assessment and emotional state prediction, on different held-out clinical cohorts without the need of fine-tuning. We also highlight the existence of a trade-off between discrete and continuous latent structures, suggesting that hybrid models may be optimal for balancing accuracy across various supervised and unsupervised tasks.