Controllable Unlearning for Image-to-Image Generative Models via varepsilon-Constrained Optimization
On the map: LLM safety and security → Vision-language and text-to-image safety
Abstract (arXiv)
While generative models have made significant advancements in recent years, they also raise concerns such as privacy breaches and biases. Machine unlearning has emerged as a viable solution, aiming to remove specific training data, e.g., containing private information and bias, from models. In this paper, we study the machine unlearning problem in Image-to-Image (I2I) generative models. Previous studies mainly treat it as a single objective optimization problem, offering a solitary solution, thereby neglecting the varied user expectations towards the trade-off between complete unlearning and model utility. To address this issue, we propose a controllable unlearning framework that uses a control coefficient varepsilon to control the trade-off. We reformulate the I2I generative model unlearning problem into a varepsilon-constrained optimization problem and solve it with a gradient-based method to find optimal solutions for unlearning boundaries. These boundaries define the valid range for the control coefficient. Within this range, every yielded solution is theoretically guaranteed with Pareto optimality. We also analyze the convergence rate of our framework under various control functions. Extensive experiments on two benchmark datasets across three mainstream I2I models demonstrate the effectiveness of our controllable unlearning framework.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 43 of its 73 references that are in the map. First posted on arXiv 3 Aug 2024.
Builds on
43 of 73 references.
Closest ancestors · 2
Same direction, newest first
- 2024Machine Unlearning for Image-to-Image Generative Models1 Feb
- 2023Data Redaction from Conditional Generative Models18 May
Other references · 41
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023GPT-4 Technical Reportcited by 18,132
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
- 2016Improved Techniques for Training GANscited by 2,938
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
29 more
- 2021Convex optimizationcited by 2,479
- 2017Improved Training of Wasserstein GANscited by 2,223
- 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
- 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
- 2022MaskGIT: Masked Generative Image Transformercited by 1,082
- 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
- 2019Machine Unlearningcited by 1,020
- 2019An Introduction to Variational Autoencoderscited by 997
- 2023Extracting Training Data from Diffusion Modelscited by 723
- 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
- 2023Erasing Concepts from Diffusion Modelscited by 530
- 2021GAN Inversion: A Surveycited by 366
- 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 360
- 2020Amnesiac Machine Learningcited by 305
- 2022A Survey of Machine Unlearningcited by 298
- 2023Towards Unbounded Machine Unlearningcited by 261
- 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 256
- 2020Influence Functions in Deep Learning Are Fragilecited by 247
- 2019Pareto Multi-Task Learningcited by 224
- 2021Machine Unlearning of Features and Labelscited by 223
- 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 222
- 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
- 2022Recommendation Unlearningcited by 71
- 2019Boundless: Generative Adversarial Networks for Image Extensioncited by 62
- 2021On the Frequency Bias of Generative Modelscited by 60
- 2023Three-Way Trade-Off in Multi-Objective Learning: Optimization, Generalization and Conflict-Avoidancecited by 35
- 2022Making Recommender Systems Forget: Learning and Unlearning for Erasable Recommendationcited by 27
- 2023Selective and Collaborative Influence Function for Efficient Recommendation Unlearningcited by 25
- 2020DeepI2I: Enabling Deep Hierarchical Image-to-Image Translation by Transferring from GANscited by 11
Built on by
Citing papers per year: 2025: 4; 2026: 1.
5 papers.
- 2025Are We Truly Forgetting? A Critical Re-examination of Machine Unlearning Evaluation Protocolscited by 16
- 2026Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarkscited by 6
- 2025Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocolscited by 4
- 2025Realistic Image-to-Image Machine Unlearning via Decoupling and Knowledge Retentioncited by 1
- 2025TOFA: Training-Free One-Shot Federated Adaptation for Vision-Language Modelscited by 1
- 2026Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks8 Jul 2026
- 2025TOFA: Training-Free One-Shot Federated Adaptation for Vision-Language Models20 Nov 2025
- 2025Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocols26 Jul 2025
- 2025Are We Truly Forgetting? A Critical Re-examination of Machine Unlearning Evaluation Protocols10 Mar 2025
- 2025Realistic Image-to-Image Machine Unlearning via Decoupling and Knowledge Retention6 Feb 2025