mlsanityData to 14 Oct 2026

Selfish Evolution: Making Discoveries in Extreme Label Noise with the Help of Overfitting Dynamics

arXiv 2412.00077 ↗First posted 26 Nov 2024cs.CV astro-ph.IM cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Motivated by the scarcity of proper labels in an astrophysical application, we have developed a novel technique, called Selfish Evolution, which allows for the detection and correction of corrupted labels in a weakly supervised fashion. Unlike methods based on early stopping, we let the model train on the noisy dataset. Only then do we intervene and allow the model to overfit to individual samples. The ``evolution'' of the model during this process reveals patterns with enough information about the noisiness of the label, as well as its correct version. We train a secondary network on these spatiotemporal ``evolution cubes'' to correct potentially corrupted labels. We incorporate the technique in a closed-loop fashion, allowing for automatic convergence towards a mostly clean dataset, without presumptions about the state of the network in which we intervene. We evaluate on the main task of the Supernova-hunting dataset but also demonstrate efficiency on the more standard MNIST dataset.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 36 references that are in the map. First posted on arXiv 26 Nov 2024.

Builds on

18 of 36 references.

References · 18

  1. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  2. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
  3. 2017A Closer Look at Memorization in Deep Networkscited by 942
  4. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
  5. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
  6. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 582
  7. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 425
  8. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 416
  9. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  10. 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
  11. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 256
  12. 2019Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networkscited by 159
6 more
  1. 2017Improving Generalization Performance by Switching from Adam to SGDcited by 125
  2. 2019Image Classification with Deep Learning in the Presence of Noisy Labels: A Surveycited by 101
  3. 2023Understanding Self-Distillation in the Presence of Label Noisecited by 21
  4. 2019Uncertainty Based Detection and Relabeling of Noisy Image Labelscited by 10
  5. 2022Learning from Training Dynamics: Identifying Mislabeled Data Beyond Manually Designed Featurescited by 9
  6. 2022Improving Data Quality with Training Dynamics of Gradient Boosting Decision Treescited by 3

Built on by

Not cited yet.