# Robust Training with Ensemble Consensus

arXiv: https://arxiv.org/abs/1910.09792 · HTML: https://mlsanity.com/paper/1910.09792/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2019.

## Built on by (most cited first, 11 of 11)

- [Large Loss Matters in Weakly Supervised Multi-Label Classification](https://mlsanity.com/paper/2206.03740/) (2022)
- [Learning from Noisy Labels for Entity-Centric Information Extraction](https://mlsanity.com/paper/2104.08656/) (2021)
- [Dual-Consistency Semi-Supervised Learning with Uncertainty Quantification for COVID-19 Lesion Segmentation from CT Images](https://mlsanity.com/paper/2104.03225/) (2021)
- [ALEX: Towards Effective Graph Transfer Learning with Noisy Labels](https://mlsanity.com/paper/2309.14673/) (2023)
- [The Dynamic of Consensus in Deep Networks and the Identification of Noisy Labels](https://mlsanity.com/paper/2210.00583/) (2022)
- [Guiding Noisy Label Conditional Diffusion Models with Score-based Discriminator Correction](https://mlsanity.com/paper/2508.19581/) (2025)
- [ST-CoNAL: Consistency-Based Acquisition Criterion Using Temporal Self-Ensemble for Active Learning](https://mlsanity.com/paper/2207.02182/) (2022)
- [Learning from Data with Noisy Labels Using Temporal Self-Ensemble](https://mlsanity.com/paper/2207.10354/) (2022)
- [CARE: Class-Adaptive Expert Consensus for Reliable Learning with Long-Tailed Noisy Labels](https://mlsanity.com/paper/2605.23254/) (2026)
- [Contrastive Representations for Label Noise Require Fine-Tuning](https://mlsanity.com/paper/2108.09154/) (2021)
- [Few-Example Clustering via Contrastive Learning](https://mlsanity.com/paper/2207.04050/) (2022)

## Abstract (arXiv)

Since deep neural networks are over-parameterized, they can memorize noisy examples. We address such a memorization issue in the presence of label noise. From the fact that deep neural networks cannot generalize to neighborhoods of memorized features, we hypothesize that noisy examples do not consistently incur small losses on the network under a certain perturbation. Based on this, we propose a novel training method called Learning with Ensemble Consensus (LEC) that prevents overfitting to noisy examples by removing them based on the consensus of an ensemble of perturbed networks. One of the proposed LECs, LTEC outperforms the current state-of-the-art methods on noisy MNIST, CIFAR-10, and CIFAR-100 in an efficient manner.