mlsanityData to 7 Oct 2026

Locally Adaptive Label Smoothing for Predictive Churn

arXiv 2102.05140 ↗First posted 9 Feb 2021cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Training modern neural networks is an inherently noisy process that can lead to high prediction churn -- disagreements between re-trainings of the same model due to factors such as randomization in the parameter initialization and mini-batches -- even when the trained models all attain similar accuracies. Such prediction churn can be very undesirable in practice. In this paper, we present several baselines for reducing churn and show that training on soft labels obtained by adaptively smoothing each example's label based on the example's neighboring labels often outperforms the baselines on churn while improving accuracy on a variety of benchmark classification tasks and model architectures.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 23 of its 47 references that are in the map. First posted on arXiv 9 Feb 2021.

Builds on

23 of 47 references.

References · 23

  1. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  5. 2019When Does Label Smoothing Help?cited by 985
  6. 2017Deep Mutual Learningcited by 621
  7. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 438
  8. 2018How Does Batch Normalization Help Optimization?cited by 355
  9. 2017Decoupling "when to update" from "how to update"cited by 230
  10. 2018To Trust Or Not To Trust A Classifiercited by 205
  11. 2018Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learningcited by 203
  12. 2018Knowledge Distillation by On-the-Fly Native Ensemblecited by 186
11 more
  1. 2016Improving the Robustness of Deep Neural Networks via Stability Trainingcited by 156
  2. 2018Large scale distributed neural network training through online distillationcited by 144
  3. 2015Online Batch Selection for Faster Training of Neural Networkscited by 121
  4. 2018Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goalscited by 103
  5. 2019Combating Label Noise in Deep Learning Using Abstentioncited by 86
  6. 2018Collaborative Learning for Deep Neural Networkscited by 69
  7. 2016Satisfying Real-world Goals with Dataset Constraintscited by 68
  8. 2019Robust Bi-Tempered Logistic Loss Based on Bregman Divergencescited by 61
  9. 2020Deep k-NN for Noisy Labelscited by 55
  10. 2014Rates of Convergence for Nearest Neighbor Classificationcited by 45
  11. 2019Fast Rates for a kNN Classifier Robust to Unknown Asymmetric Label Noisecited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 1; 2023: 1; 2024: 1; 2025: 1; 2026: 1.

7 papers.

  1. 2021Churn Reduction via Distillationcited by 17
  2. 2024Predictive Churn with the Set of Good Modelscited by 15
  3. 2022Predicting on the Edge: Identifying Where a Larger Model Does Bettercited by 6
  4. 2025Mitigating Negative Flips via Margin Preserving Trainingcited by 3
  5. 2021Active Coveringcited by 1
  6. 2023Measuring and Mitigating Local Instability in Deep Neural Networkscited by 1
  7. 2026Trustworthy AI: Ensuring Reliability and Accountability from Models to Agentscited by 0