# Locality-Aware Continual Unlearning for Diffusion Models arXiv: https://arxiv.org/abs/2512.02657 · HTML: https://mlsanity.com/paper/2512.02657/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. - Builds on 38 of its 44 references that are in the map. - First posted on arXiv 2 Dec 2025. ## Placed in - Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/) - LLM safety and security / [Vision-language and text-to-image safety](https://mlsanity.com/safety/vision-language-and-text-to-image-safety/) ## Builds on (most cited first, 30 of 38) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025) - [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024) - [Erasing Concepts from Diffusion Models](https://mlsanity.com/paper/2303.07345/) (2023) - [Unified Concept Editing in Diffusion Models](https://mlsanity.com/paper/2308.14761/) (2023) - [SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation](https://mlsanity.com/paper/2310.12508/) (2023) - [MACE: Mass Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2403.06135/) (2024) - [Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.17591/) (2023) - [Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models](https://mlsanity.com/paper/2305.10120/) (2023) - [One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications](https://mlsanity.com/paper/2312.16145/) (2023) - [Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Models](https://mlsanity.com/paper/2407.12383/) (2024) - [Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Erasers](https://mlsanity.com/paper/2311.17717/) (2023) - [Set You Straight: Auto-Steering Denoising Trajectories to Sidestep Unwanted Concepts](https://mlsanity.com/paper/2504.12782/) (2025) - [Erasing Undesirable Influence in Diffusion Models](https://mlsanity.com/paper/2401.05779/) (2024) - [Scissorhands: Scrub Data Influence via Connection Sensitivity in Networks](https://mlsanity.com/paper/2401.06187/) (2024) - [UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Models](https://mlsanity.com/paper/2402.11846/) (2024) - [STEREO: A Two-Stage Framework for Adversarially Robust Concept Erasing from Text-to-Image Diffusion Models](https://mlsanity.com/paper/2408.16807/) (2024) - [Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Concepts](https://mlsanity.com/paper/2410.12777/) (2024) - [Fantastic Targets for Concept Erasure in Diffusion Models and Where To Find Them](https://mlsanity.com/paper/2501.18950/) (2025) - [Imagen 3](https://mlsanity.com/paper/2408.07009/) (2024) - [Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Models](https://mlsanity.com/paper/2409.11219/) (2024) - [Continual Learning of Diffusion Models with Generative Distillation](https://mlsanity.com/paper/2311.14028/) (2023) - [Learning to Unlearn while Retaining: Combating Gradient Conflicts in Machine Unlearning](https://mlsanity.com/paper/2503.06339/) (2025) - [Sculpting Memory: Multi-Concept Forgetting in Diffusion Models via Dynamic Mask and Concept-Aware Optimization](https://mlsanity.com/paper/2504.09039/) (2025) - [When Are Concepts Erased From Diffusion Models?](https://mlsanity.com/paper/2505.17013/) (2025) - [TRCE: Towards Reliable Malicious Concept Erasure in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2503.07389/) (2025) - [Side Effects of Erasing Concepts from Diffusion Models](https://mlsanity.com/paper/2508.15124/) (2025) ## Built on by (most cited first, 5 of 5) - [Continual Speaker Identity Unlearning with Minimal Interference](https://mlsanity.com/paper/2605.25962/) (2026) - [TILDE: TILt-based Distributional Erasure for Concept Unlearning](https://mlsanity.com/paper/2607.06432/) (2026) - [Mitigating Sequential Reappearance in Diffusion Data-Point Unlearning](https://mlsanity.com/paper/2609.25166/) (2026) - [Continual Data Unlearning in Diffusion Models via Transition-based Regularization](https://mlsanity.com/paper/2609.32328/) (2026) - [Enabling Preference-driven Unlearning in Few-step Distilled Text-to-Image Diffusion Models](https://mlsanity.com/paper/2610.10859/) (2026) ## Abstract (arXiv) Real-world deployment of text-to-image diffusion models requires continual concept removal as new privacy, copyright, or safety obligations arise over time. Existing unlearning methods, however, are designed for single-step deletion and collapse after only 3-5 sequential applications. We trace this instability to two compounding factors: (i) coarse mapping targets that cause degradation to accumulate unnecessarily across steps, and (ii) the absence of local protection for semantically neighboring concepts, whose shared internal representations make them the first to suffer collateral damage. Because this damage is strongest in the local semantic neighborhood of the forget concept, global replay alone cannot prevent it. Building on this analysis, we propose Locality-Aware Continual Unlearning (LACU), a framework with two complementary mechanisms. Locality-Aware Target Selection chooses, for each forget prompt, the context-preserving mapping prompt that the diffusion model itself treats as most similar to the original prompt, measured by score-prediction distance (how differently the model denoises the same noisy image under two text conditions), ensuring each update is as small and targeted as possible. Locality-Aware Replay uses the same metric to identify the retain concepts closest to the forget concept in the model's own representation and replays them as a local functional regularizer, directly shielding the most vulnerable neighborhood. Combined with teacher-student distillation and lightweight ℓ2 parameter regularization, LACU maintains stable unlearning over 10 sequential steps, preserving significantly higher related retention (RRacc) and general retention (GRacc) than recent baselines.