# Training of Neural Networks with Uncertain Data: A Mixture of Experts Approach

arXiv: https://arxiv.org/abs/2312.08083 · HTML: https://mlsanity.com/paper/2312.08083/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts.
- Builds on 11 of its 42 references that are in the map.
- First posted on arXiv 13 Dec 2023.

## Placed in

- Mixture-of-experts / [MoE theory and gating functions](https://mlsanity.com/moe/moe-theory-and-gating-functions/)

## Builds on (most cited first, 11 of 11)

- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [A General Framework for Uncertainty Estimation in Deep Learning](https://mlsanity.com/paper/1907.06890/) (2019)
- [Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection](https://mlsanity.com/paper/1804.05132/) (2018)
- [Lightweight Probabilistic Deep Networks](https://mlsanity.com/paper/1805.11327/) (2018)
- [Cross-validation: what does it estimate and how well does it do it?](https://mlsanity.com/paper/2104.00673/) (2021)
- [Processing of missing data by neural networks](https://mlsanity.com/paper/1805.07405/) (2018)
- [Learning an Uncertainty-Aware Object Detector for Autonomous Driving](https://mlsanity.com/paper/1910.11375/) (2019)
- [Uncertainty-Aware Learning Against Label Noise on Imbalanced Datasets](https://mlsanity.com/paper/2207.05471/) (2022)
- [Handling of uncertainty in medical data using machine learning and probability theory techniques: A review of 30 years (1991-2020)](https://mlsanity.com/paper/2008.10114/) (2020)
- [Mixtures of Experts Models](https://mlsanity.com/paper/1806.08200/) (2018)
- [Robust Learning Under Label Noise With Iterative Noise-Filtering](https://mlsanity.com/paper/1906.00216/) (2019)

## Built on by (most cited first, 2 of 2)

- [Extracting Uncertainty Estimates from Mixtures of Experts for Semantic Segmentation](https://mlsanity.com/paper/2509.04816/) (2025)
- [EMoE: Training-Free Expert Disagreement for Uncertainty-Aware Text-to-Image Diffusion](https://mlsanity.com/paper/2505.13273/) (2025)

## Abstract (arXiv)

This paper introduces the "Uncertainty-aware Mixture of Experts" (uMoE), a novel solution aimed at addressing aleatoric uncertainty within Neural Network (NN) based predictive models. While existing methodologies primarily concentrate on managing uncertainty during inference, uMoE uniquely embeds uncertainty into the training phase. Employing a "Divide and Conquer" strategy, uMoE strategically partitions the uncertain input space into more manageable subspaces. It comprises Expert components, individually trained on their respective subspace uncertainties. Overarching the Experts, a Gating Unit, leveraging additional information regarding the distribution of uncertain in-puts across these subspaces, dynamically adjusts the weighting to minimize deviations from ground truth. Our findings demonstrate the superior performance of uMoE over baseline methods in effectively managing data uncertainty. Furthermore, through a comprehensive robustness analysis, we showcase its adaptability to varying uncertainty levels and propose optimal threshold parameters. This innovative approach boasts broad applicability across diverse da-ta-driven domains, including but not limited to biomedical signal processing, autonomous driving, and production quality control.