# Enabling Mixed Effects Neural Networks for Diverse, Clustered Data Using Monte Carlo Methods

arXiv: https://arxiv.org/abs/2407.01115 · HTML: https://mlsanity.com/paper/2407.01115/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 38 references that are in the map.
- First posted on arXiv 1 Jul 2024.

## Builds on (most cited first, 13 of 13)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [Entity Embeddings of Categorical Variables](https://mlsanity.com/paper/1604.06737/) (2016)
- [Gaussian Process Boosting](https://mlsanity.com/paper/2004.02653/) (2020)
- [Integrating Random Effects in Deep Neural Networks](https://mlsanity.com/paper/2206.03314/) (2022)
- [Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features](https://mlsanity.com/paper/2104.00629/) (2021)
- [Machine Learning with High-Cardinality Categorical Features in Actuarial Applications](https://mlsanity.com/paper/2301.12710/) (2023)
- [Adversarially-regularized mixed effects deep learning (ARMED) models for improved interpretability, performance, and generalization on clustered data](https://mlsanity.com/paper/2202.11783/) (2022)
- [Neural Mixed Effects for Nonlinear Personalized Predictions](https://mlsanity.com/paper/2306.08149/) (2023)
- [A Comparison of Machine Learning Methods for Data with High-Cardinality Categorical Variables](https://mlsanity.com/paper/2307.02071/) (2023)

## Built on by (most cited first, 3 of 3)

- [TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks](https://mlsanity.com/paper/2606.02384/) (2026)
- [Neural Generalized Mixed-Effects Models](https://mlsanity.com/paper/2604.10976/) (2026)
- [Integrating Neural Encoders in Bayesian Generalized Linear Mixed Models for Multimodal Data](https://mlsanity.com/paper/2607.04647/) (2026)

## Abstract (arXiv)

Neural networks often assume independence among input data samples, disregarding correlations arising from inherent clustering patterns in real-world datasets (e.g., due to different sites or repeated measurements). Recently, mixed effects neural networks (MENNs) which separate cluster-specific 'random effects' from cluster-invariant 'fixed effects' have been proposed to improve generalization and interpretability for clustered data. However, existing methods only allow for approximate quantification of cluster effects and are limited to regression and binary targets with only one clustering feature. We present MC-GMENN, a novel approach employing Monte Carlo methods to train Generalized Mixed Effects Neural Networks. We empirically demonstrate that MC-GMENN outperforms existing mixed effects deep learning models in terms of generalization performance, time complexity, and quantification of inter-cluster variance. Additionally, MC-GMENN is applicable to a wide range of datasets, including multi-class classification tasks with multiple high-cardinality categorical features. For these datasets, we show that MC-GMENN outperforms conventional encoding and embedding methods, simultaneously offering a principled methodology for interpreting the effects of clustering patterns.