mlsanityData to 7 Oct 2026

EMMA: Concept Erasure Benchmark with Comprehensive Semantic Metrics and Diverse Categories

arXiv 2512.17320 ↗First posted 19 Dec 2025cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

The widespread adoption of text-to-image (T2I) generation has raised concerns about privacy, bias, and copyright violations. Concept erasure techniques offer a promising solution by selectively removing undesired concepts from pre-trained models without requiring full retraining. However, these methods are often evaluated on a limited set of concepts, relying on overly simplistic and direct prompts. To test the boundaries of concept erasure techniques, and assess whether they truly remove targeted concepts from model representations, we introduce EMMA, a benchmark that evaluates five key dimensions of concept erasure over 13 metrics. EMMA goes beyond standard metrics like image quality and time efficiency, testing robustness under challenging conditions, including indirect descriptions, visually similar non-target concepts, and potential gender and ethnicity bias, providing a socially aware analysis of method behavior. Using EMMA, we analyze five concept erasure methods across five domains (objects, celebrities, art styles, NSFW, and copyright). Our results show that existing methods struggle with implicit prompts (i.e., generating the erased concept when it is indirectly referenced) and visually similar non-target concepts (i.e., failing to generate non-target concepts resembling the erased one), while some amplify gender and ethnicity bias compared to the original model. Code and prompts are available at https://github.com/lobsterlulu/EMMA.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 43 of its 63 references that are in the map. First posted on arXiv 19 Dec 2025.

Builds on

43 of 63 references.

References · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  8. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  9. 2023Erasing Concepts from Diffusion Modelscited by 530
  10. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  11. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  12. 2023Unified Concept Editing in Diffusion Modelscited by 377
31 more
  1. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
  2. 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 312
  3. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 298
  4. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 297
  5. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  6. 2022Red-Teaming the Stable Diffusion Safety Filtercited by 257
  7. 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?cited by 239
  8. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 225
  9. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 204
  10. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 171
  11. 2023UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANscited by 165
  12. 2023One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applicationscited by 164
  13. 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 150
  14. 2024Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Modelscited by 123
  15. 2023Circumventing Concept Erasure Methods For Text-to-Image Generative Modelscited by 81
  16. 2021ML-Decoder: Scalable and Versatile Classification Headcited by 78
  17. 2024Instruct-Imagen: Image Generation with Multi-modal Instructioncited by 65
  18. 2021Text to Image Generation with Semantic-Spatial Aware GANcited by 53
  19. 2024Erasing Undesirable Influence in Diffusion Modelscited by 52
  20. 2024Scissorhands: Scrub Data Influence via Connection Sensitivity in Networkscited by 50
  21. 2020LogoDet-3K: A Large-Scale Image Dataset for Logo Detectioncited by 25
  22. 2023Stable Diffusion Exposed: Gender Bias from Prompt to Imagecited by 25
  23. 2024Holistic Unlearning Benchmark: A Multi-Faceted Evaluation for Text-to-Image Diffusion Model Unlearningcited by 19
  24. 2025TRCE: Towards Reliable Malicious Concept Erasure in Text-to-Image Diffusion Modelscited by 14
  25. 2025Erasing More Than Intended? How Concept Erasure Degrades the Generation of Non-Target Conceptscited by 12
  26. 2025Side Effects of Erasing Concepts from Diffusion Modelscited by 12
  27. 2024A Dataset and Benchmark for Copyright Infringement Unlearning from Text-to-Image Diffusion Modelscited by 11
  28. 2023Situating the social issues of image generation models in the model life cycle: a sociotechnical approachcited by 9
  29. 2024Memories of Forgotten Conceptscited by 9
  30. 2025Privacy Protection Against Personalized Text-to-Image Synthesis via Cross-image Consistency Constraintscited by 2
  31. 2025SAGE: Exploring the Boundaries of Unsafe Concept Domain with Semantic-Augment Erasingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026To Erase, or Not to Erase: Robust Training-Free Concept Erasure with Preservation aware Adaptive Ranked Subspace Expansioncited by 3
  2. 2026Prototype-Guided Concept Erasure in Diffusion Modelscited by 1
  3. 2026PROBE: Diagnosing Residual Concept Capacity in Erased Text-to-Video Diffusion Modelscited by 1
  4. 2026Mosaic: Compositional Multi-Concept Erasure via Vector Field Blendingcited by 0