Imperceptible Protection against Style Imitation from Diffusion Models
On the map: Diffusion models → Robustness, privacy and adversarial aspects
Abstract (arXiv)
Recent progress in diffusion models has profoundly enhanced the fidelity of image generation, but it has raised concerns about copyright infringements. While prior methods have introduced adversarial perturbations to prevent style imitation, most are accompanied by the degradation of artworks' visual quality. Recognizing the importance of maintaining this, we introduce a visually improved protection method while preserving its protection capability. To this end, we devise a perceptual map to highlight areas sensitive to human eyes, guided by instance-aware refinement, which refines the protection intensity accordingly. We also introduce a difficulty-aware protection by predicting how difficult the artwork is to protect and dynamically adjusting the intensity based on this. Lastly, we integrate a perceptual constraints bank to further improve the imperceptibility. Results show that our method substantially elevates the quality of the protected image without compromising on protection efficacy.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 53 of its 82 references that are in the map. First posted on arXiv 28 Mar 2024.
Builds on
53 of 82 references.
Closest ancestors · 7
Same direction, newest first
- 2023Toward effective protection against diffusion based mimicry through score distillation2 Oct
- 2023DUAW: Data-free Universal Adversarial Watermark against Stable Diffusion Customization19 Aug
- 2023Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation2 Jun
- 2023Mist: Towards Improved Adversarial Examples for Diffusion Models22 May
- 2023Anti-DreamBooth: Protecting users from personalized text-to-image synthesis27 Mar
- 2023Raising the Cost of Malicious AI-Powered Image Editing13 Feb
- 2023Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examples9 Feb
Other references · 46
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
34 more
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
- 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
- 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
- 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
- 2017VGGFace2: A dataset for recognising faces across pose and agecited by 911
- 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 792
- 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
- 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
- 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
- 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
- 2023TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessmentcited by 220
- 2023Glaze: Protecting Artists from Style Mimicry by Text-to-Image Modelscited by 200
- 2023StyleDrop: Text-to-Image Generation in Any Stylecited by 197
- 2018PieAPP: Perceptual Image-Error Assessment through Pairwise Preferencecited by 158
- 2017A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortionscited by 107
- 2019Sparse and Imperceivable Adversarial Attackscited by 95
- 2018SparseFool: a few pixels make a big differencecited by 94
- 2019ColorFool: Semantic Adversarial Colorizationcited by 89
- 2018Low Frequency Adversarial Perturbationcited by 80
- 2022Frequency-driven Imperceptible Adversarial Attack on Semantic Similaritycited by 76
- 2019Towards Large yet Imperceptible Adversarial Image Perturbations with Perceptual Color Distancecited by 72
- 2022Large-scale Text-to-Image Generation Models for Visual Artists' Creative Workscited by 60
- 2018Towards Imperceptible and Robust Adversarial Example Attacks against Neural Networkscited by 38
- 2022UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidancecited by 30
- 2023DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Modelscited by 26
- 2021Demiguise Attack: Crafting Invisible Semantic Adversarial Perturbations with Perceptual Similaritycited by 18
- 2022Saliency Attack: Towards Imperceptible Black-box Adversarial Attackcited by 12
- 2019Should Adversarial Attacks Use Pixel p-Norm?cited by 10
- 2020Protecting Against Image Translation Deepfakes by Leaking Universal Perturbations from Black-Box Neural Networkscited by 7
Built on by
Citing papers per year: 2024: 4; 2025: 3; 2026: 1.
8 papers.
- 2024Semantic Deep Hiding for Robust Unlearnable Examplescited by 13
- 2024Replication in Visual Diffusion Models: A Survey and Outlookcited by 11
- 2024Pixel is a Barrier: Diffusion Models Are More Adversarially Robust Than We Thinkcited by 10
- 2024Nearly Zero-Cost Protection Against Mimicry by Personalized Diffusion Modelscited by 6
- 2025BlurGuard: A Simple Approach for Robustifying Image Protection Against AI-Powered Editingcited by 5
- 2025Anti-Inpainting: A Proactive Defense Approach against Malicious Diffusion-based Inpainters under Unknown Conditionscited by 3
- 2026Immune2V: Image Immunization Against Dual-Stream Image-to-Video Generationcited by 3
- 2025DIA: The Adversarial Exposure of Deterministic Inversion in Diffusion Modelscited by 2
- 2026Immune2V: Image Immunization Against Dual-Stream Image-to-Video Generation12 Apr 2026
- 2025BlurGuard: A Simple Approach for Robustifying Image Protection Against AI-Powered Editing31 Oct 2025
- 2025DIA: The Adversarial Exposure of Deterministic Inversion in Diffusion Models1 Oct 2025
- 2025Anti-Inpainting: A Proactive Defense Approach against Malicious Diffusion-based Inpainters under Unknown Conditions19 May 2025
- 2024Nearly Zero-Cost Protection Against Mimicry by Personalized Diffusion Models16 Dec 2024
- 2024Replication in Visual Diffusion Models: A Survey and Outlook7 Jul 2024
- 2024Semantic Deep Hiding for Robust Unlearnable Examples25 Jun 2024
- 2024Pixel is a Barrier: Diffusion Models Are More Adversarially Robust Than We Think20 Apr 2024
Similar, not cited
Close in wording, no citation either way
- 2025StyleProtect: Safeguarding Artistic Identity in Fine-tuned Diffusion Models
- 2025Towards Robust Defense against Customization via Protective Perturbation Resistant to Diffusion-based Purification
- 2025Evaluating Adversarial Protections for Diffusion Personalization: A Comprehensive Study
- 2025SITA: Structurally Imperceptible and Transferable Adversarial Attacks for Stylized Image Generation
- 2023IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AI
- 2023Can Protective Perturbation Safeguard Personal Data from Being Exploited by Stable Diffusion?
- 2023Measuring the Success of Diffusion Models at Imitating Human Artists
- 2025Is Perturbation-Based Image Protection Disruptive to Image Editing?