# Exploiting the Signal-Leak Bias in Diffusion Models arXiv: https://arxiv.org/abs/2309.15842 ยท HTML: https://mlsanity.com/paper/2309.15842/ ## Key facts - Cited by 36 papers in the map (data through 14 Oct 2026). - Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. - Builds on 24 of its 45 references that are in the map. - First posted on arXiv 27 Sep 2023. ## Placed in - Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/) ## Builds on (most cited first, 24 of 24) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022) - [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015) - [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021) - [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021) - [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022) - [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022) - [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021) - [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022) - [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015) - [GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation](https://mlsanity.com/paper/2203.02923/) (2022) - [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://mlsanity.com/paper/2305.08891/) (2023) - [Generalization in Generation: A closer look at Exposure Bias](https://mlsanity.com/paper/1910.00292/) (2019) - [Input Perturbation Reduces Exposure Bias in Diffusion Models](https://mlsanity.com/paper/2301.11706/) (2023) - [Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps](https://mlsanity.com/paper/2305.15583/) (2023) - [LDM3D: Latent Diffusion Model for 3D](https://mlsanity.com/paper/2305.10853/) (2023) - [Shifted Diffusion for Text-to-image Generation](https://mlsanity.com/paper/2211.15388/) (2022) - [Stochastic Frequency Masking to Improve Super-Resolution and Denoising Networks](https://mlsanity.com/paper/2003.07119/) (2020) - [Arbitrary Style Guidance for Enhanced Diffusion-Based Text-to-Image Generation](https://mlsanity.com/paper/2211.07751/) (2022) ## Built on by (most cited first, 30 of 36) - [StableNormal: Reducing Diffusion Variance for Stable and Sharp Normal](https://mlsanity.com/paper/2406.16864/) (2024) - [Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal Bridging](https://mlsanity.com/paper/2503.22236/) (2025) - [FreeInit: Bridging Initialization Gap in Video Diffusion Models](https://mlsanity.com/paper/2312.07537/) (2023) - [You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scale](https://mlsanity.com/paper/2412.06699/) (2024) - [All Seeds Are Not Equal: Enhancing Compositional Text-to-Image Generation with Reliable Random Seeds](https://mlsanity.com/paper/2411.18810/) (2024) - [Diffusion Posterior Proximal Sampling for Image Restoration](https://mlsanity.com/paper/2402.16907/) (2024) - [I4VGen: Image as Free Stepping Stone for Text-to-Video Generation](https://mlsanity.com/paper/2406.02230/) (2024) - [DREAM: Diffusion Rectification and Estimation-Adaptive Models](https://mlsanity.com/paper/2312.00210/) (2023) - [FDS: Frequency-Aware Denoising Score for Text-Guided Latent Diffusion Image Editing](https://mlsanity.com/paper/2503.19191/) (2025) - [Free-Lunch Color-Texture Disentanglement for Stylized Image Generation](https://mlsanity.com/paper/2503.14275/) (2025) - [SCP-Diff: Spatial-Categorical Joint Prior for Diffusion Based Semantic Image Synthesis](https://mlsanity.com/paper/2403.09638/) (2024) - [One More Step: A Versatile Plug-and-Play Module for Rectifying Diffusion Schedule Flaws and Enhancing Low-Frequency Controls](https://mlsanity.com/paper/2311.15744/) (2023) - [FLAME Diffuser: Wildfire Image Synthesis using Mask Guided Diffusion](https://mlsanity.com/paper/2403.03463/) (2024) - [Enhancing Diffusion Model Stability for Image Restoration via Gradient Management](https://mlsanity.com/paper/2507.06656/) (2025) - [MRPD: Undersampled MRI reconstruction by prompting a large latent diffusion model](https://mlsanity.com/paper/2402.10609/) (2024) - [Antithetic Noise in Diffusion Models](https://mlsanity.com/paper/2506.06185/) (2025) - [Exposure Bias Can Alleviate Itself via Directional and Frequency Rectification in Flow Matching](https://mlsanity.com/paper/2606.28226/) (2026) - [Latent Space Disentanglement in Diffusion Transformers Enables Precise Zero-shot Semantic Editing](https://mlsanity.com/paper/2411.08196/) (2024) - [Mitigating Data Consistency Induced Discrepancy in Cascaded Diffusion Models for Sparse-view CT Reconstruction](https://mlsanity.com/paper/2403.09355/) (2024) - [Solving Prior Distribution Mismatch in Diffusion Models via Optimal Transport](https://mlsanity.com/paper/2410.13431/) (2024) - [SAGA: Learning Signal-Aligned Distributions for Improved Text-to-Image Generation](https://mlsanity.com/paper/2508.13866/) (2025) - [Diffusion-Driven Progressive Target Manipulation for Source-Free Domain Adaptation](https://mlsanity.com/paper/2510.25279/) (2025) - [Training-free image inversion for one-step diffusion models](https://mlsanity.com/paper/2606.01380/) (2026) - [Latent Space Disentanglement in Diffusion Transformers Enables Zero-shot Fine-grained Semantic Editing](https://mlsanity.com/paper/2408.13335/) (2024) - [Diffusion Prism: Enhancing Diversity and Morphology Consistency in Mask-to-Image Diffusion](https://mlsanity.com/paper/2501.00944/) (2025) - [Text-to-Image Alignment in Denoising-Based Models through Step Selection](https://mlsanity.com/paper/2504.17525/) (2025) - [Ctrl-Z Sampling: Scaling Diffusion Sampling with Controlled Random Zigzag Explorations](https://mlsanity.com/paper/2506.20294/) (2025) - [LoRAShield: Data-Free Editing Alignment for Secure Personalized LoRA Sharing](https://mlsanity.com/paper/2507.07056/) (2025) - [Example-Based Feature Painting on Textures](https://mlsanity.com/paper/2511.01513/) (2025) - [Bias mitigation in graph diffusion models](https://mlsanity.com/paper/2604.01709/) (2026) ## Abstract (arXiv) There is a bias in the inference pipeline of most diffusion models. This bias arises from a signal leak whose distribution deviates from the noise distribution, creating a discrepancy between training and inference processes. We demonstrate that this signal-leak bias is particularly significant when models are tuned to a specific style, causing sub-optimal style matching. Recent research tries to avoid the signal leakage during training. We instead show how we can exploit this signal-leak bias in existing diffusion models to allow more control over the generated images. This enables us to generate images with more varied brightness, and images that better match a desired style or color. By modeling the distribution of the signal leak in the spatial frequency and pixel domains, and including a signal leak in the initial latent, we generate images that better match expected results without any additional training.