mlsanityData to 7 Oct 2026

SAP: Corrective Machine Unlearning with Scaled Activation Projection for Label Noise Robustness

arXiv 2403.08618 ↗First posted 13 Mar 2024cs.LG cs.AI stat.MLcited by 10

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Label corruption, where training samples are mislabeled due to non-expert annotation or adversarial attacks, significantly degrades model performance. Acquiring large, perfectly labeled datasets is costly, and retraining models from scratch is computationally expensive. To address this, we introduce Scaled Activation Projection (SAP), a novel SVD (Singular Value Decomposition)-based corrective machine unlearning algorithm. SAP mitigates label noise by identifying a small subset of trusted samples using cross-entropy loss and projecting model weights onto a clean activation space estimated using SVD on these trusted samples. This process suppresses the noise introduced in activations due to the mislabeled samples. In our experiments, we demonstrate SAP's effectiveness on synthetic noise with different settings and real-world label noise. SAP applied to the CIFAR dataset with 25% synthetic corruption show upto 6% generalization improvements. Additionally, SAP can improve the generalization over noise robust training approaches on CIFAR dataset by ~3.2% on average. Further, we observe generalization improvements of 2.31% for a Vision Transformer model trained on naturally corrupted Clothing1M.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 37 of its 50 references that are in the map. First posted on arXiv 13 Mar 2024.

Builds on

37 of 50 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy2 Mar
  2. 2024Parameter-tuning-free data entry error unlearning with adaptive selective synaptic dampening6 Feb
  3. 2023Deep Unlearning: Fast and Efficient Gradient-free Approach to Class Forgetting1 Dec
  4. 2023Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening15 Aug
  5. 2022Towards Adversarial Evaluations for Inexact Machine Unlearning17 Jan

Other references · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  5. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  6. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  7. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  8. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 691
  9. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  10. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
  11. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
  12. 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 411
20 more
  1. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 388
  2. 2021Gradient Projection Memory for Continual Learningcited by 338
  3. 2017Certified Defenses for Data Poisoning Attackscited by 288
  4. 2014cuDNN: Efficient Primitives for Deep Learningcited by 240
  5. 2018Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learningcited by 219
  6. 2020Does label smoothing mitigate label noise?cited by 204
  7. 2020Learning with Instance-Dependent Label Noise: A Sample Sieve Approachcited by 158
  8. 2022DeepCore: A Comprehensive Library for Coreset Selection in Deep Learningcited by 146
  9. 2020Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attackscited by 109
  10. 2019Meta Label Correction for Noisy Label Learningcited by 108
  11. 2020Error-Bounded Correction of Noisy Labelscited by 74
  12. 2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
  13. 2022Instance-Dependent Label-Noise Learning with Manifold-Regularized Transition Matrix Estimationcited by 49
  14. 2022Mitigating Memorization of Noisy Labels by Clipping the Model Predictioncited by 40
  15. 2017Learning Deep Networks from Noisy Labels with Dropout Regularizationcited by 32
  16. 2022Amplifying Membership Exposure via Data Poisoningcited by 29
  17. 2022Beyond Images: Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Featurescited by 27
  18. 2023Memorization Through the Lens of Curvature of Loss Function Around Samplescited by 24
  19. 2021Adaptive Sample Selection for Robust Learning under Label Noisecited by 20
  20. 2022Learning from Training Dynamics: Identifying Mislabeled Data Beyond Manually Designed Featurescited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 4.

10 papers.

  1. 2025CURE: Concept Unlearning via Orthogonal Representation Editing in Diffusion Modelscited by 24
  2. 2025Principled Multimodal Representation Learningcited by 19
  3. 2025Now You See It, Now You Don't - Instant Concept Erasure for Safe Text-to-Image and Video Generationcited by 3
  4. 2024Neural Corrective Machine Unrankingcited by 1
  5. 2025Bias-Aware Machine Unlearning: Towards Fairer Vision Models via Controllable Forgettingcited by 0
  6. 2025Subtract the Corruption: Training-Data-Free Corrective Machine Unlearning using Task Arithmeticcited by 0
  7. 2026Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectoriescited by 0
  8. 2026ACD-U: Asymmetric co-teaching with machine unlearning for robust learning with noisy labelscited by 0
  9. 2026Unbiased Rectification for Sequential Recommender Systems Under Fake Orderscited by 0
  10. 2026Forget or Fine-tune? A Comparative Study of Machine Unlearning Strategies for Noisy Label Correctioncited by 0