mlsanityData to 14 Oct 2026

Unlearning via Sparse Representations

arXiv 2311.15268 ↗First posted 26 Nov 2023cs.LG cs.AIcited by 10

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning, which involves erasing knowledge about a forget set from a trained model, can prove to be costly and infeasible by existing techniques. We propose a nearly compute-free zero-shot unlearning technique based on a discrete representational bottleneck. We show that the proposed technique efficiently unlearns the forget set and incurs negligible damage to the model's performance on the rest of the data set. We evaluate the proposed technique on the problem of textitclass unlearning using three datasets: CIFAR-10, CIFAR-100, and LACUNA-100. We compare the proposed technique to SCRUB, a state-of-the-art approach which uses knowledge distillation for unlearning. Across all three datasets, the proposed technique performs as well as, if not better than SCRUB while incurring almost no computational cost.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 29 of its 46 references that are in the map. First posted on arXiv 26 Nov 2023.

Builds on

29 of 46 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation19 Oct
  2. 2023Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening15 Aug
  3. 2023Boundary Unlearning21 Mar
  4. 2023Towards Unbounded Machine Unlearning20 Feb
  5. 2022A Survey of Machine Unlearning6 Sep
  6. 2022Deep Unlearning via Randomized Conditionally Independent Hessians15 Apr
  7. 2022Zero-Shot Machine Unlearning14 Jan
  8. 2021Fast Yet Effective Machine Unlearning17 Nov
  9. 2021Machine Unlearning of Features and Labels26 Aug

Other references · 20

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  6. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  7. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  8. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  9. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  10. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  11. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 914
  12. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 618
8 more
  1. 2019Certified Data Removal from Machine Learning Modelscited by 535
  2. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 465
  3. 2020Approximate Data Deletion from Machine Learning Modelscited by 255
  4. 2020Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observationscited by 180
  5. 2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 85
  6. 2021Discrete-Valued Neural Communicationcited by 39
  7. 2022Discrete Key-Value Bottleneckcited by 18
  8. 2022Stateful active facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learningcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 1; 2026: 2.

10 papers.

  1. 2024Are we making progress in unlearning? Findings from the first NeurIPS unlearning competitioncited by 50
  2. 2025Ten Challenging Problems in Federated Foundation Modelscited by 20
  3. 2024An Information Theoretic Approach to Machine Unlearningcited by 16
  4. 2024Decoupling the Class Label and the Target Concept in Machine Unlearningcited by 12
  5. 2024Label Smoothing Improves Machine Unlearningcited by 9
  6. 2026RAZOR: Ratio-Aware Layer Editing for Targeted Unlearning in Vision Transformers and Diffusion Modelscited by 8
  7. 2024VANP: Learning Where to See for Navigation with Self-Supervised Vision-Action Pre-Trainingcited by 5
  8. 2024CodeUnlearn: Amortized Zero-Shot Machine Unlearning in Language Models Using Discrete Conceptcited by 1
  9. 2024AdaProb: Efficient Machine Unlearning via Adaptive Probabilitycited by 0
  10. 2026You Don't Need All That Attention: Surgical Memorization Mitigation in Text-to-Image Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Representation Unlearning: Forgetting through Information Compression
  2. 2025When to Forget? Complexity Trade-offs in Machine Unlearning
  3. 2026ZeroUnlearn: Few-Shot Knowledge Unlearning in Large Language Models
  4. 2026Erase at the Core: Representation Unlearning for Machine Unlearning
  5. 2024Is Retain Set All You Need in Machine Unlearning? Restoring Performance of Unlearned Models with Out-Of-Distribution Images
  6. 2024Efficient Knowledge Deletion from Trained Models through Layer-wise Partial Machine Unlearning
  7. 2025The Right to be Forgotten in Pruning: Unveil Machine Unlearning on Sparse Models
  8. 2026Unmerge: Efficient Machine Unlearning via Task Arithmetic