mlsanityData to 14 Oct 2026

DiffVax: Optimization-Free Image Immunization Against Diffusion-Based Editing

arXiv 2411.17957 ↗First posted 27 Nov 2024cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Current image immunization defense techniques against diffusion-based editing embed imperceptible noise into target images to disrupt editing models. However, these methods face scalability challenges, as they require time-consuming optimization for each image separately, taking hours for small batches. To address these challenges, we introduce DiffVax, a scalable, lightweight, and optimization-free framework for image immunization, specifically designed to prevent diffusion-based editing. Our approach enables effective generalization to unseen content, reducing computational costs and cutting immunization time from days to milliseconds, achieving a speedup of 250,000x. This is achieved through a loss term that ensures the failure of editing attempts and the imperceptibility of the perturbations. Extensive qualitative and quantitative results demonstrate that our model is scalable, optimization-free, adaptable to various diffusion-based editing tools, robust against counter-attacks, and, for the first time, effectively protects video content from editing. More details are available in https://diffvax.github.io/ .

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 58 of its 79 references that are in the map. First posted on arXiv 27 Nov 2024.

Builds on

58 of 79 references.

References · 58

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2023Segment Anythingcited by 8,234
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  10. 2013Intriguing properties of neural networkscited by 5,155
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  12. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
38 more
  1. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  2. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  3. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  4. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  5. 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,525
  6. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  7. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  8. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  9. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  10. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  11. 2016Universal adversarial perturbationscited by 784
  12. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  13. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
  14. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 700
  15. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  16. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 647
  17. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 642
  18. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  19. 2023Zero-shot Image-to-Image Translationcited by 505
  20. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  21. 2023Universal Guidance for Diffusion Modelscited by 437
  22. 2018Generating Adversarial Examples with Adversarial Networkscited by 258
  23. 2023Glaze: Protecting Artists from Style Mimicry by Text-to-Image Modelscited by 205
  24. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 195
  25. 2023Negative-prompt Inversion: Fast Image Inversion for Editing with Text-guided Diffusion Modelscited by 174
  26. 2023Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examplescited by 167
  27. 2023Raising the Cost of Malicious AI-Powered Image Editingcited by 163
  28. 2023StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Modelscited by 155
  29. 2017Generative Adversarial Perturbationscited by 153
  30. 2023Delta Denoising Scorecited by 121
  31. 2023Mist: Towards Improved Adversarial Examples for Diffusion Modelscited by 94
  32. 2024SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Modelscited by 66
  33. 2020Disrupting Deepfakes: Adversarial Attacks Against Conditional Image Translation Networks and Facial Manipulation Systemscited by 65
  34. 2023Toward effective protection against diffusion based mimicry through score distillationcited by 65
  35. 2023Towards Coherent Image Inpainting Using Denoising Diffusion Implicit Modelscited by 61
  36. 2023IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AIcited by 35
  37. 2023Text-Driven Image Editing via Learnable Regionscited by 31
  38. 2023EditShield: Protecting Unauthorized Image Editing by Instruction-guided Diffusion Modelscited by 27

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 5.

10 papers.