mlsanityData to 14 Oct 2026

Disagreement-Regularized Importance Sampling for Adversarial Label Corruption

arXiv 2605.07551 ↗First posted 8 May 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Standard Importance Sampling (IS) collapses under label corruption because high-norm examples, prioritized for variance reduction, are often adversarial outliers. We formalize this misalignment using an varepsilon-contamination model and propose Disagreement-Regularized Importance Sampling (DR-IS), a sub-sampling method based on loss rank-disagreement across independent proxy ensemble. We prove finite-sample concentration bounds showing that the empirical rank disagreement of bulk corrupted examples is bounded above, and that of boundary-clean examples bounded below, both at rate O(√log(N/δ)/K) with probability 1-δ; when the structural expectation gap Delta' between the two groups is positive and the boundary-clean set is at least as large as the selected subset, these bounds certify strict separation and control the contamination rate of the selected subset. Empirically, DR-IS remains robust under targeted high-norm attacks that break magnitude-based methods such as the Error L2-norm (EL2N) on benchmark datasets. DR-IS complements training-dynamics approaches like Area Under the Margin ranking (AUM), offering improved robustness in the loss-aligned regime alongside explicit finite-sample concentration certificates and a contamination bound limiting noise leakage from the statistical tail of corrupted points.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 47 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

30 of 47 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  6. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  7. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  8. 2015Prioritized Experience Replaycited by 1,099
  9. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  10. 2017Deep Bayesian Active Learning with Image Datacited by 819
  11. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 692
  12. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
18 more
  1. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 533
  2. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 528
  3. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 425
  4. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 416
  5. 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 339
  6. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 328
  7. 2021GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Trainingcited by 274
  8. 2020Identifying Mislabeled Data using the Area Under the Margin Rankingcited by 261
  9. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 243
  10. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 236
  11. 2013Stochastic Gradient Descent, Weighted Sampling, and the Randomized Kaczmarz algorithmcited by 133
  12. 2015Online Batch Selection for Faster Training of Neural Networkscited by 122
  13. 2019Recent Advances in Algorithmic High-Dimensional Robust Statisticscited by 101
  14. 2019Accelerating Deep Learning by Focusing on the Biggest Loserscited by 94
  15. 2014Stochastic Optimization with Importance Samplingcited by 91
  16. 2017Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chainscited by 88
  17. 2015Variance Reduction in SGD by Distributed Importance Samplingcited by 79
  18. 2016Importance Sampling for Minibatchescited by 36

Built on by

Not cited yet.