# Deep Generalised Mixed Models: a Novel Neural Network Structure for Analysing Hierarchical Data

arXiv: https://arxiv.org/abs/2608.05930 · HTML: https://mlsanity.com/paper/2608.05930/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 39 references that are in the map.
- First posted on arXiv 6 Aug 2026.

## Builds on (most cited first, 15 of 15)

- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network](https://mlsanity.com/paper/1808.03314/) (2018)
- [Understanding disentangling in β-VAE](https://mlsanity.com/paper/1804.03599/) (2018)
- [Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering](https://mlsanity.com/paper/1611.05148/) (2016)
- [Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing](https://mlsanity.com/paper/1903.10145/) (2019)
- [Probabilistic Forecasting with Temporal Convolutional Neural Network](https://mlsanity.com/paper/1906.04397/) (2019)
- [What's a good imputation to predict with missing values?](https://mlsanity.com/paper/2106.00311/) (2021)
- [Posterior Collapse and Latent Variable Non-identifiability](https://mlsanity.com/paper/2301.00537/) (2023)
- [On the consistency of supervised learning with missing values](https://mlsanity.com/paper/1902.06931/) (2019)
- [A Systematic Review for Transformer-based Long-term Series Forecasting](https://mlsanity.com/paper/2310.20218/) (2023)
- [Deep Generative Clustering with VAEs and Expectation-Maximization](https://mlsanity.com/paper/2501.07358/) (2025)
- [Inference for high-dimensional linear mixed-effects models: A quasi-likelihood approach](https://mlsanity.com/paper/1907.06116/) (2019)
- [Latent mixed-effect models for high-dimensional longitudinal data](https://mlsanity.com/paper/2409.11008/) (2024)

## Abstract (arXiv)

The experience sampling method (ESM) is a longitudinal research design where participants report their thoughts, emotional states and behaviours multiple times a day. Our work is motivated by such data collected by the GrowIt! app, which was released to investigate daily emotions among adolescents during the COVID-19 pandemic. Current procedures to analyse ESM data face various challenges. While standard statistical techniques may not scale well to a high-dimensional setting, machine learning procedures can give biased results due to selection bias introduced by missingness. In our motivating dataset, adolescents dropped out due to previous strong feelings of negative emotions. Hence, the implied missing data are of the missing-at-random type that standard machine learning procedures cannot accommodate. We develop a novel neural network architecture that generalises mixed effects models to deep learning to overcome these challenges. It allows semi-parametric and flexible modelling of data's mean and correlation structure through fixed and random effects. For estimation, we use an adaptation of variational auto-encoders and a Bayesian data augmentation algorithm. Through this approach, the model can accommodate longitudinal outcomes following generic distributions, scale well to high-dimensional settings and provide valid inference when data are missing-at-random. We applied the Deep Generalised Mixed Model to the GrowIt! study and various simulations. The results show potential for the Deep Generalised Mixed Model, yet suboptimal performance due to model instability.