# A Universal Approximation Theorem for Mixture of Experts Models

arXiv: https://arxiv.org/abs/1602.03683 · HTML: https://mlsanity.com/paper/1602.03683/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2016.

## Built on by (most cited first, 18 of 18)

- [CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition](https://mlsanity.com/paper/2402.02526/) (2024)
- [Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts](https://mlsanity.com/paper/2305.07572/) (2023)
- [A non-asymptotic approach for model selection via penalization in high-dimensional mixture of experts models](https://mlsanity.com/paper/2104.02640/) (2021)
- [Non-asymptotic oracle inequalities for the Lasso in high-dimensional mixture of experts](https://mlsanity.com/paper/2009.10622/) (2020)
- [Non-asymptotic model selection in block-diagonal mixture of polynomial experts models](https://mlsanity.com/paper/2104.08959/) (2021)
- [Uniform Consistency in Nonparametric Mixture Models](https://mlsanity.com/paper/2108.14003/) (2021)
- [FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks](https://mlsanity.com/paper/2110.11869/) (2021)
- [Gradient-free variational learning with conditional mixture networks](https://mlsanity.com/paper/2408.16429/) (2024)
- [Mixture of experts models for multilevel data: modelling framework and approximation theory](https://mlsanity.com/paper/2209.15207/) (2022)
- [Bayesian data fusion with shared priors](https://mlsanity.com/paper/2212.07311/) (2022)
- [Dendrograms of Mixing Measures for Softmax-Gated Gaussian Mixture of Experts: Consistency Without Model Sweeps](https://mlsanity.com/paper/2510.12744/) (2025)
- [An Introduction to the Practical and Theoretical Aspects of Mixture-of-Experts Modeling](https://mlsanity.com/paper/1707.03538/) (2017)
- [Soft regression trees: a model variant and a decomposition training algorithm](https://mlsanity.com/paper/2501.05942/) (2025)
- [Robustness of Mixtures of Experts to Feature Noise](https://mlsanity.com/paper/2601.14792/) (2026)
- [Algorithms and data structures for automatic precision estimation of neural networks](https://mlsanity.com/paper/2509.24607/) (2025)
- [Revisiting Incremental Stochastic Majorization-Minimization Algorithms with Applications to Mixture of Experts](https://mlsanity.com/paper/2601.19811/) (2026)
- [Fast Model Selection and Stable Optimization for Softmax-Gated Multinomial-Logistic Mixture of Experts Models](https://mlsanity.com/paper/2602.07997/) (2026)
- [Sparsity Saves Compute, Not Samples: Oracle Inequalities for Dense and Sparse Mixture of Experts](https://mlsanity.com/paper/2610.15039/) (2026)

## Abstract (arXiv)

The mixture of experts (MoE) model is a popular neural network architecture for nonlinear regression and classification. The class of MoE mean functions is known to be uniformly convergent to any unknown target function, assuming that the target function is from Sobolev space that is sufficiently differentiable and that the domain of estimation is a compact unit hypercube. We provide an alternative result, which shows that the class of MoE mean functions is dense in the class of all continuous functions over arbitrary compact domains of estimation. Our result can be viewed as a universal approximation theorem for MoE models.