mlsanityData to 14 Oct 2026

IREU: Identity-Related Encoder-Only Unlearning for Customized Portrait Generation

arXiv 2606.29880 ↗First posted 29 Jun 2026cs.CVcited by 0

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Customized Portrait Generation (CPG) technologies have been widely used to generate high-fidelity person images given an input image indicating the identity and a text prompt indicating the required edits. Yet these methods pose significant privacy risks by spreading fake visual information. Against such risks, each public generator should be able to suppress its generation ability for a particular person when requested. Therefore, in this work we investigate the identity unlearning problem for CPG. Since there are no previous methods in this field, we propose a simple baseline that updates the image encoder by minimizing identity similarity between generated and input images for target identities to be unlearned, while maximizing it for identities to be retained. However, we find such a global perturbation in the feature space harms the fidelity of generated images for other identities to be retained. To solve this problem, we propose a novel method IREU, which first locates identity-related features in an offline manner and then only performs feature perturbations on them. The experimental results show that our proposed method IREU achieves better identity unlearning performance for target identities to be unlearned, and also keeps high fidelity for other identities to be retained. In addition, our unlearned image encoder is generalizable across different generators with the same encoder without fine-tuning, which is friendly for deployment in practice.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 43 of its 53 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

43 of 53 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Towards Source-Free Machine Unlearning20 Aug
  2. 2025Decoupled Distillation to Erase: A General Unlearning Method for Any Class-centric Tasks31 Mar
  3. 2025LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty24 Mar
  4. 2023Boundary Unlearning21 Mar

Other references · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  3. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  4. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  5. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  6. 2023Erasing Concepts from Diffusion Modelscited by 539
  7. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 436
  8. 2023Unified Concept Editing in Diffusion Modelscited by 388
  9. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  10. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  11. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 307
  12. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 301
27 more
  1. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  2. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 235
  3. 2023StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-Oncited by 175
  4. 2023One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applicationscited by 169
  5. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 168
  6. 2024Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Modelscited by 133
  7. 2023Collaborative Diffusion for Multi-Modal Face Generation and Editingcited by 116
  8. 2024Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradientcited by 69
  9. 2024Erasing Undesirable Influence in Diffusion Modelscited by 57
  10. 2023CAT-DM: Controllable Accelerated Virtual Try-on with Diffusion Modelcited by 48
  11. 2024AquaLoRA: Toward White-box Protection for Customized Stable Diffusion Models via Watermark LoRAcited by 46
  12. 2023CLIP2Protect: Protecting Facial Privacy using Text-Guided Makeup via Adversarial Latent Searchcited by 43
  13. 2024Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Conceptscited by 40
  14. 2025Localized Concept Erasure for Text-to-Image Diffusion Models Using Training-Free Gated Low-Rank Adaptationcited by 33
  15. 2024SleeperMark: Towards Robust Watermark against Fine-Tuning Text-to-image Diffusion Modelscited by 29
  16. 2024IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generationcited by 22
  17. 2024Generative Unlearning for Any Identitycited by 19
  18. 2023Feature Unlearning for Pre-trained GANs and VAEscited by 16
  19. 2024EmotiveTalk: Expressive Talking Head Generation through Audio Information Decoupling and Emotional Video Diffusioncited by 16
  20. 2025Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adaptercited by 16
  21. 2025NoT: Federated Unlearning via Weight Negationcited by 15
  22. 2024Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivitycited by 11
  23. 2025HyperLoRA: Parameter-Efficient Adaptive Generation for Portrait Synthesiscited by 10
  24. 2025DCT-Shield: A Robust Frequency Domain Defense against Malicious Image Editingcited by 7
  25. 2024PID: Prompt-Independent Data Protection Against Latent Diffusion Modelscited by 6
  26. 2025Zero-Shot Head Swapping in Real-World Scenarioscited by 4
  27. 2025Perturb a Model, Not an Image: Towards Robust Privacy Protection via Anti-Personalized Diffusion Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PIU: Proximity-guided Identity Unlearning in ID-Conditioned Diffusion Models
  2. 2025IC-Portrait: In-Context Matching for View-Consistent Personalized Portrait
  3. 2025Reverse Personalization
  4. 2024IDAdapter: Learning Mixed Features for Tuning-Free Personalization of Text-to-Image Models
  5. 2024UniPortrait: A Unified Framework for Identity-Preserving Single- and Multi-Human Image Personalization
  6. 2026No Way To Steal My Face: Proactive Defense Against Identity-Preserving Personalized Generation
  7. 2025InfiniteYou: Flexible Photo Recrafting While Preserving Your Identity
  8. 2023PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization