mlsanityData to 14 Oct 2026

DualOptim: Enhancing Efficacy and Stability in Machine Unlearning with Dual Optimizers

arXiv 2504.15827 ↗First posted 22 Apr 2025cs.LG cs.AIcited by 6

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Existing machine unlearning (MU) approaches exhibit significant sensitivity to hyperparameters, requiring meticulous tuning that limits practical deployment. In this work, we first empirically demonstrate the instability and suboptimal performance of existing popular MU methods when deployed in different scenarios. To address this issue, we propose Dual Optimizer (DualOptim), which incorporates adaptive learning rate and decoupled momentum factors. Empirical and theoretical evidence demonstrates that DualOptim contributes to effective and stable unlearning. Through extensive experiments, we show that DualOptim can significantly boost MU efficacy and stability across diverse tasks, including image classification, image generation, and large language models, making it a versatile approach to empower existing MU algorithms.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 47 of its 55 references that are in the map. First posted on arXiv 22 Apr 2025.

Builds on

47 of 55 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024A Closer Look at Machine Unlearning for Large Language Models10 Oct
  2. 2024Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancement29 Sep
  3. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  4. 2024Rethinking Machine Unlearning for Large Language Models13 Feb
  5. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  6. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation19 Oct
  7. 2023Machine Unlearning: Solutions and Challenges14 Aug
  8. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy10 May
  9. 2023Model Sparsity Can Simplify Machine Unlearning11 Apr
  10. 2023Towards Unbounded Machine Unlearning20 Feb
  11. 2022Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher17 May
  12. 2021Fast Yet Effective Machine Unlearning17 Nov
4 more
  1. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearning27 Sep
  2. 2021Machine Unlearning of Features and Labels26 Aug
  3. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar
  4. 2021Machine Unlearning via Algorithmic Stability25 Feb

Other references · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  9. 2022Scalable Diffusion Models with Transformerscited by 6,054
  10. 2022Classifier-Free Diffusion Guidancecited by 5,847
  11. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  12. 2019Machine Unlearningcited by 1,061
19 more
  1. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  2. 2023Extracting Training Data from Diffusion Modelscited by 732
  3. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 618
  4. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
  5. 2023Erasing Concepts from Diffusion Modelscited by 539
  6. 2019Certified Data Removal from Machine Learning Modelscited by 535
  7. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
  8. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 465
  9. 2023Symbolic Discovery of Optimization Algorithmscited by 434
  10. 2020Amnesiac Machine Learningcited by 319
  11. 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 317
  12. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 311
  13. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 301
  14. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 292
  15. 2022Red-Teaming the Stable Diffusion Safety Filtercited by 264
  16. 2020Approximate Data Deletion from Machine Learning Modelscited by 255
  17. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
  18. 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 177
  19. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 175

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2025SoK: Machine Unlearning for Large Language Modelscited by 11
  2. 2025Understanding the Dilemma of Unlearning for Large Language Modelscited by 4
  3. 2026SoK: Unlearnability and Unlearning for Model Dememorizationcited by 0
  4. 2026DualOptim+: Bridging Shared and Decoupled Optimizer States for Better Machine Unlearning in Large Language Modelscited by 0
  5. 2026Spectral Saliency for Machine Unlearningcited by 0
  6. 2026SAUL: Sharpness-Aware Augmented-Lagrangian Unlearningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty
  2. 2024nabla τ: Gradient-based and Task-Agnostic machine Unlearning
  3. 2026Designing to Forget: Deep Semi-parametric Models for Unlearning
  4. 2025Learning to Unlearn while Retaining: Combating Gradient Conflicts in Machine Unlearning
  5. 2026Forgetting Similar Samples: Can Machine Unlearning Do it Better?
  6. 2024Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate
  7. 2024MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearning
  8. 2024MUSO: Achieving Exact Machine Unlearning in Over-Parameterized Regimes