# Label Denoising with Large Ensembles of Heterogeneous Neural Networks

arXiv: https://arxiv.org/abs/1809.04403 · HTML: https://mlsanity.com/paper/1809.04403/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2018.

## Built on by (most cited first, 5 of 5)

- [Continual Learning on Noisy Data Streams via Self-Purified Replay](https://mlsanity.com/paper/2110.07735/) (2021)
- [Sample Selection via Contrastive Fragmentation for Noisy Label Regression](https://mlsanity.com/paper/2502.17771/) (2025)
- [One-shot Weakly-Supervised Segmentation in Medical Images](https://mlsanity.com/paper/2111.10773/) (2021)
- [Knowledge Distillation for Efficient Audio-Visual Video Captioning](https://mlsanity.com/paper/2306.09947/) (2023)
- [Reliable Mislabel Detection for Video Capsule Endoscopy Data](https://mlsanity.com/paper/2602.06938/) (2026)

## Abstract (arXiv)

Despite recent advances in computer vision based on various convolutional architectures, video understanding remains an important challenge. In this work, we present and discuss a top solution for the large-scale video classification (labeling) problem introduced as a Kaggle competition based on the YouTube-8M dataset. We show and compare different approaches to preprocessing, data augmentation, model architectures, and model combination. Our final model is based on a large ensemble of video- and frame-level models but fits into rather limiting hardware constraints. We apply an approach based on knowledge distillation to deal with noisy labels in the original dataset and the recently developed mixup technique to improve the basic models.