# Elijah: Eliminating Backdoors Injected in Diffusion Models via Distribution Shift

arXiv: https://arxiv.org/abs/2312.00050 · HTML: https://mlsanity.com/paper/2312.00050/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 48 of its 76 references that are in the map.
- First posted on arXiv 27 Nov 2023.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)
- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 30 of 48)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [How To Backdoor Federated Learning](https://mlsanity.com/paper/1807.00459/) (2018)
- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation](https://mlsanity.com/paper/2203.02923/) (2022)
- [STRIP: A Defence Against Trojan Attacks on Deep Neural Networks](https://mlsanity.com/paper/1902.06531/) (2019)
- [UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2302.04867/) (2023)
- [Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech](https://mlsanity.com/paper/2105.06337/) (2021)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Hidden Trigger Backdoor Attacks](https://mlsanity.com/paper/1910.00033/) (2019)
- [Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks](https://mlsanity.com/paper/2101.05930/) (2021)
- [Unlocking High-Accuracy Differentially Private Image Classification through Scale](https://mlsanity.com/paper/2204.13650/) (2022)
- [BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements](https://mlsanity.com/paper/2006.01043/) (2020)
- [Detecting AI Trojans Using Meta Neural Analysis](https://mlsanity.com/paper/1910.03137/) (2019)
- [SINE: SINgle Image Editing with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2212.04489/) (2022)

## Built on by (most cited first, 30 of 31)

- [Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey](https://mlsanity.com/paper/2408.03400/) (2024)
- [Silent Branding Attack: Trigger-free Data Poisoning Attack on Text-to-Image Diffusion Models](https://mlsanity.com/paper/2503.09669/) (2025)
- [Invisible Backdoor Attacks on Diffusion Models](https://mlsanity.com/paper/2406.00816/) (2024)
- [UIBDiffusion: Universal Imperceptible Backdoor Attack for Diffusion Models](https://mlsanity.com/paper/2412.11441/) (2024)
- [BackdoorDM: A Comprehensive Benchmark for Backdoor Learning on Diffusion Model](https://mlsanity.com/paper/2502.11798/) (2025)
- [Not Just Text: Uncovering Vision Modality Typographic Threats in Image Generation Models](https://mlsanity.com/paper/2412.05538/) (2024)
- [PureDiffusion: Using Backdoor to Counter Backdoor in Generative Diffusion Models](https://mlsanity.com/paper/2409.13945/) (2024)
- [MixBridge: Heterogeneous Image-to-Image Backdoor Attack through Mixture of Schr"odinger Bridges](https://mlsanity.com/paper/2505.08809/) (2025)
- [Watch the Watcher! Backdoor Attacks on Security-Enhancing Diffusion Models](https://mlsanity.com/paper/2406.09669/) (2024)
- [What Lurks Within? Concept Auditing for Shared Diffusion Models at Scale](https://mlsanity.com/paper/2504.14815/) (2025)
- [The last Dance : Robust backdoor attack via diffusion models and bayesian approach](https://mlsanity.com/paper/2402.05967/) (2024)
- [Trading Devil: Robust backdoor attack via Stochastic investment models and Bayesian approach](https://mlsanity.com/paper/2406.10719/) (2024)
- [A Dual-Purpose Framework for Backdoor Defense and Backdoor Amplification in Diffusion Models](https://mlsanity.com/paper/2502.19047/) (2025)
- [Dynamic Attention Analysis for Backdoor Detection in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2504.20518/) (2025)
- [UNIT: Backdoor Mitigation via Automated Neural Distribution Tightening](https://mlsanity.com/paper/2407.11372/) (2024)
- [On the Fairness, Diversity and Reliability of Text-to-Image Generative Models](https://mlsanity.com/paper/2411.13981/) (2024)
- [Parasite: A Steganography-based Backdoor Attack Framework for Diffusion Models](https://mlsanity.com/paper/2504.05815/) (2025)
- [Backdoor Defense in Diffusion Models via Spatial Attention Unlearning](https://mlsanity.com/paper/2504.18563/) (2025)
- [Sealing The Backdoor: Unlearning Adversarial Text Triggers In Diffusion Models Using Knowledge Distillation](https://mlsanity.com/paper/2508.18235/) (2025)
- [How to Backdoor Consistency Models?](https://mlsanity.com/paper/2410.19785/) (2024)
- [TrojFlow: Flow Models are Natural Targets for Trojan Attacks](https://mlsanity.com/paper/2412.16512/) (2024)
- [Gungnir: Exploiting Stylistic Features in Images for Backdoor Attacks on Diffusion Models](https://mlsanity.com/paper/2502.20650/) (2025)
- [Practical, Generalizable and Robust Backdoor Attacks on Text-to-Image Diffusion Models](https://mlsanity.com/paper/2508.01605/) (2025)
- [BadRSSD: Backdoor Attacks on Regularized Self-Supervised Diffusion Models](https://mlsanity.com/paper/2603.01019/) (2026)
- [BlackMirror: Black-Box Backdoor Detection for Text-to-Image Models via Instruction-Response Deviation](https://mlsanity.com/paper/2603.05921/) (2026)
- [BadDreamer: Transferable Backdoor Attacks against Video World Models for Autonomous Driving](https://mlsanity.com/paper/2606.21172/) (2026)
- [BadBlocks: Low-Cost and Stealthy Backdoor Attacks Tailored for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2508.03221/) (2025)
- [Backdoor Sentinel: Detecting and Detoxifying Backdoors in Diffusion Models via Temporal Noise Consistency](https://mlsanity.com/paper/2602.01765/) (2026)
- [Beyond the False Trade-off: Adaptive EWC for Stealthy and Generalizable T2I Backdoors](https://mlsanity.com/paper/2605.08280/) (2026)
- [Backdooring Masked Diffusion Language Models](https://mlsanity.com/paper/2605.19262/) (2026)

## Abstract (arXiv)

Diffusion models (DM) have become state-of-the-art generative models because of their capability to generate high-quality images from noises without adversarial training. However, they are vulnerable to backdoor attacks as reported by recent studies. When a data input (e.g., some Gaussian noise) is stamped with a trigger (e.g., a white patch), the backdoored model always generates the target image (e.g., an improper photo). However, effective defense strategies to mitigate backdoors from DMs are underexplored. To bridge this gap, we propose the first backdoor detection and removal framework for DMs. We evaluate our framework Elijah on hundreds of DMs of 3 types including DDPM, NCSN and LDM, with 13 samplers against 3 existing backdoor attacks. Extensive experiments show that our approach can have close to 100% detection accuracy and reduce the backdoor effects to close to zero without significantly sacrificing the model utility.