Leveraging Unlabeled Data to Track Memorization
Not on the map. Why?
Abstract (arXiv)
Deep neural networks may easily memorize noisy labels present in real-world data, which degrades their ability to generalize. It is therefore important to track and evaluate the robustness of models against noisy label memorization. We propose a metric, called susceptibility, to gauge such memorization for neural networks. Susceptibility is simple and easy to compute during training. Moreover, it does not require access to ground-truth labels and it only uses unlabeled data. We empirically show the effectiveness of our metric in tracking memorization on various architectures and datasets and provide theoretical insights into the design of the susceptibility metric. Finally, we show through extensive experiments on datasets with synthetic and real-world label noise that one can utilize susceptibility and the overall training accuracy to distinguish models that maintain a low memorization on the training set and generalize well to unseen clean data.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 76 of its 106 references that are in the map. First posted on arXiv 8 Dec 2022.
Builds on
76 of 106 references.
References · 76
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2016Densely Connected Convolutional Networkscited by 6,691
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
- 2013Intriguing properties of neural networkscited by 5,114
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2017Squeeze-and-Excitation Networkscited by 4,763
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
- 2014Going Deeper with Convolutionscited by 4,231
38 more
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
- 2016Identity Mappings in Deep Residual Networkscited by 1,960
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
- 2018On the Spectral Bias of Neural Networkscited by 1,321
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
- 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
- 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
- 2020Designing Network Design Spacescited by 954
- 2017A Closer Look at Memorization in Deep Networkscited by 936
- 2018Robustness May Be at Odds with Accuracycited by 873
- 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
- 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
- 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
- 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
- 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
- 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
- 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
- 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 461
- 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 459
- 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
- 2016Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samplescited by 439
- 2017Deep Layer Aggregationcited by 436
- 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 414
- 2020What is being transferred in transfer learning?cited by 365
- 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 360
- 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
- 2020Identifying Mislabeled Data using the Area Under the Margin Rankingcited by 259
- 2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
- 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 253
- 2019Simple Black-box Adversarial Attackscited by 249
- 2019Understanding and Utilizing Deep Neural Networks Trained with Noisy Labelscited by 209
- 2019On the Inductive Bias of Neural Tangent Kernelscited by 177
- 2018Training behavior of deep neural network in frequency domaincited by 174
- 2018Disentangling Adversarial Robustness and Generalizationcited by 171
- 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
50 most cited of 76 All 76 (JSON)
Built on by
Citing papers per year: 2025: 2.
2 papers.