# Targeted Unlearning with Single Layer Unlearning Gradient

arXiv: https://arxiv.org/abs/2407.11867 · HTML: https://mlsanity.com/paper/2407.11867/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 44 of its 65 references that are in the map.
- First posted on arXiv 16 Jul 2024.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 30 of 44)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [Erasing Concepts from Diffusion Models](https://mlsanity.com/paper/2303.07345/) (2023)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Unified Concept Editing in Diffusion Models](https://mlsanity.com/paper/2308.14761/) (2023)
- [SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation](https://mlsanity.com/paper/2310.12508/) (2023)
- [Ablating Concepts in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.13516/) (2023)
- [A Survey of Machine Unlearning](https://mlsanity.com/paper/2209.02299/) (2022)
- [MACE: Mass Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2403.06135/) (2024)
- [Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.17591/) (2023)
- [Unrolling SGD: Understanding Factors Influencing Machine Unlearning](https://mlsanity.com/paper/2109.13398/) (2021)
- [Towards Unbounded Machine Unlearning](https://mlsanity.com/paper/2302.09880/) (2023)
- [Machine Unlearning of Features and Labels](https://mlsanity.com/paper/2108.11577/) (2021)
- [Model Sparsity Can Simplify Machine Unlearning](https://mlsanity.com/paper/2304.04934/) (2023)
- [To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now](https://mlsanity.com/paper/2310.11868/) (2023)
- [Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher](https://mlsanity.com/paper/2205.08096/) (2022)
- [Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening](https://mlsanity.com/paper/2308.07707/) (2023)

## Built on by (most cited first, 18 of 18)

- [SAEmnesia: Erasing Concepts in Diffusion Models with Supervised Sparse Autoencoders](https://mlsanity.com/paper/2509.21379/) (2025)
- [VLA-Forget: Vision-Language-Action Unlearning for Embodied Foundation Models](https://mlsanity.com/paper/2604.03956/) (2026)
- [LVMark: Robust Watermark for Latent Video Diffusion Models](https://mlsanity.com/paper/2412.09122/) (2024)
- [Erased but Not Forgotten: How Backdoors Compromise Concept Erasure](https://mlsanity.com/paper/2504.21072/) (2025)
- [RAZOR: Ratio-Aware Layer Editing for Targeted Unlearning in Vision Transformers and Diffusion Models](https://mlsanity.com/paper/2603.14819/) (2026)
- [Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks](https://mlsanity.com/paper/2607.07907/) (2026)
- [Zero-shot Class Unlearning via Layer-wise Relevance Analysis and Neuronal Path Perturbation](https://mlsanity.com/paper/2410.23693/) (2024)
- [Models Designed to Forget: Machine Unlearning via Key Deletion](https://mlsanity.com/paper/2603.15033/) (2026)
- [Object Hallucination-Free Reinforcement Unlearning for Vision-Language Models](https://mlsanity.com/paper/2605.08031/) (2026)
- [SPARE: Self-distillation for PARameter-Efficient Removal](https://mlsanity.com/paper/2602.07058/) (2026)
- [Selective Fine-Tuning for Targeted and Robust Concept Unlearning](https://mlsanity.com/paper/2602.07919/) (2026)
- [Layer-Targeted Multilingual Knowledge Erasure in Large Language Models](https://mlsanity.com/paper/2602.22562/) (2026)
- [CATA: Continual Machine Unlearning via Conflict-Averse Task Arithmetic](https://mlsanity.com/paper/2605.18610/) (2026)
- [On the Robustness of Machine Unlearning for Vision-Language Models](https://mlsanity.com/paper/2605.26992/) (2026)
- [ICED: Concept-level Machine Unlearning via Interpretable Concept Decomposition](https://mlsanity.com/paper/2605.14309/) (2026)
- [InstantForget: Update-Free Backdoor Unlearning with Inference-Time Feature Reset](https://mlsanity.com/paper/2606.15730/) (2026)
- [Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Models](https://mlsanity.com/paper/2608.01008/) (2026)
- [Exact Unlearning via Quantized Sufficient Statistics](https://mlsanity.com/paper/2610.07197/) (2026)

## Abstract (arXiv)

Machine unlearning methods aim to remove sensitive or unwanted content from trained models, but typically demand extensive model updates at significant computational cost while potentially degrading model performance on both related and unrelated tasks. We propose Single Layer Unlearning Gradient (SLUG) as an efficient method to unlearn targeted information by updating a single critical layer using a one-time gradient computation. SLUG uses layer importance and gradient alignment metrics to identify the optimal layer for targeted information removal while preserving the model utility. We demonstrate the effectiveness of SLUG for CLIP, Stable Diffusion, and vision-language models (VLMs) in removing concrete (e.g., identities and objects) and abstract concepts (e.g., artistic styles). On the UnlearnCanvas benchmark, SLUG achieves comparable unlearning performance to existing methods while requiring significantly less computational resources. Our proposed approach offers a practical solution for targeted unlearning that is computationally efficient and precise. Our code is available at https://github.com/CSIPlab/SLUG.