# Neural Mixed Effects for Nonlinear Personalized Predictions

arXiv: https://arxiv.org/abs/2306.08149 · HTML: https://mlsanity.com/paper/2306.08149/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 61 references that are in the map.
- First posted on arXiv 13 Jun 2023.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Challenges in Markov chain Monte Carlo for Bayesian neural networks](https://mlsanity.com/paper/1910.06539/) (2019)
- [Kernel-based Tests for Joint Independence](https://mlsanity.com/paper/1603.00285/) (2016)
- [Integrating Random Effects in Deep Neural Networks](https://mlsanity.com/paper/2206.03314/) (2022)
- [Multi-Source Social Feedback of Online News Feeds](https://mlsanity.com/paper/1801.07055/) (2018)
- [Neural CRF Parsing](https://mlsanity.com/paper/1507.03641/) (2015)
- [Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data](https://mlsanity.com/paper/2106.13213/) (2021)
- [f-SAEM: A fast Stochastic Approximation of the EM algorithm for nonlinear mixed effects models](https://mlsanity.com/paper/1910.12222/) (2019)
- [Mixed Effects Random Forests for Personalised Predictions of Clinical Depression Severity](https://mlsanity.com/paper/2301.09815/) (2023)

## Built on by (most cited first, 6 of 6)

- [Modeling Parkinson's Disease Progression Using Longitudinal Voice Biomarkers: A Comparative Study of Statistical and Neural Mixed-Effects Models](https://mlsanity.com/paper/2507.20058/) (2025)
- [Enabling Mixed Effects Neural Networks for Diverse, Clustered Data Using Monte Carlo Methods](https://mlsanity.com/paper/2407.01115/) (2024)
- [Hamiltonian Monte Carlo Inference of Marginalized Linear Mixed-Effects Models](https://mlsanity.com/paper/2410.24079/) (2024)
- [Machine Learning for ALSFRS-R Score Prediction: Making Sense of the Sensor Data](https://mlsanity.com/paper/2407.08003/) (2024)
- [From Affect Prediction to Affect Forecasting: Evidence for Distinct Information Sources in Longitudinal Text](https://mlsanity.com/paper/2606.16687/) (2026)
- [Deep Neural Mixed Models for Nonlinear Longitudinal Prediction with Subject-Specific Heterogeneity](https://mlsanity.com/paper/2610.15323/) (2026)

## Abstract (arXiv)

Personalized prediction is a machine learning approach that predicts a person's future observations based on their past labeled observations and is typically used for sequential tasks, e.g., to predict daily mood ratings. When making personalized predictions, a model can combine two types of trends: (a) trends shared across people, i.e., person-generic trends, such as being happier on weekends, and (b) unique trends for each person, i.e., person-specific trends, such as a stressful weekly meeting. Mixed effect models are popular statistical models to study both trends by combining person-generic and person-specific parameters. Though linear mixed effect models are gaining popularity in machine learning by integrating them with neural networks, these integrations are currently limited to linear person-specific parameters: ruling out nonlinear person-specific trends. In this paper, we propose Neural Mixed Effect (NME) models to optimize nonlinear person-specific parameters anywhere in a neural network in a scalable manner. NME combines the efficiency of neural network optimization with nonlinear mixed effects modeling. Empirically, we observe that NME improves performance across six unimodal and multimodal datasets, including a smartphone dataset to predict daily mood and a mother-adolescent dataset to predict affective state sequences where half the mothers experience at least moderate symptoms of depression. Furthermore, we evaluate NME for two model architectures, including for neural conditional random fields (CRF) to predict affective state sequences where the CRF learns nonlinear person-specific temporal transitions between affective states. Analysis of these person-specific transitions on the mother-adolescent dataset shows interpretable trends related to the mother's depression symptoms.