mlsanityData to 14 Oct 2026

FreeU: Free Lunch in Diffusion U-Net

arXiv 2309.11497 ↗First posted 20 Sep 2023cs.CVcited by 196

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

In this paper, we uncover the untapped potential of diffusion U-Net, which serves as a "free lunch" that substantially improves the generation quality on the fly. We initially investigate the key contributions of the U-Net architecture to the denoising process and identify that its main backbone primarily contributes to denoising, whereas its skip connections mainly introduce high-frequency features into the decoder module, causing the network to overlook the backbone semantics. Capitalizing on this discovery, we propose a simple yet effective method-termed "FreeU" - that enhances generation quality without additional training or finetuning. Our key insight is to strategically re-weight the contributions sourced from the U-Net's skip connections and backbone feature maps, to leverage the strengths of both components of the U-Net architecture. Promising results on image and video generation tasks demonstrate that our FreeU can be readily integrated to existing diffusion models, e.g., Stable Diffusion, DreamBooth, ModelScope, Rerender and ReVersion, to improve the generation quality with only a few lines of code. All you need is to adjust two scaling factors during inference. Project page: https://chenyangsi.top/FreeU/.

Key facts. Cited by 196 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 55 of its 65 references that are in the map. First posted on arXiv 20 Sep 2023.

Builds on

55 of 65 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  2. 2022Diffusion Probabilistic Model Made Slim27 Nov
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  4. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  5. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models6 Aug
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  7. 2017Neural Discrete Representation Learningcited by 4,712
  8. 2013Auto-Encoding Variational Bayescited by 4,053
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  12. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
36 more
  1. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  2. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  3. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  4. 2014Conditional Generative Adversarial Netscited by 2,162
  5. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  6. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  7. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  8. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  9. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  10. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  11. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  12. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  13. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
  14. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  15. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  16. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  17. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 927
  18. 2020Leveraging Frequency Analysis for Deep Fake Image Recognitioncited by 504
  19. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 368
  20. 2019KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessmentcited by 355
  21. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 331
  22. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  23. 2022How Do Vision Transformers Work?cited by 307
  24. 2019A Fourier Perspective on Model Robustness in Computer Visioncited by 288
  25. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  26. 2018Training behavior of deep neural network in frequency domaincited by 174
  27. 2022Pretraining is All You Need for Image-to-Image Translationcited by 160
  28. 2020Frequency Bias in Neural Networks for Input of Non-Uniform Densitycited by 140
  29. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  30. 2023Collaborative Diffusion for Multi-Modal Face Generation and Editingcited by 116
  31. 2022Inception Transformercited by 99
  32. 2023ReVersion: Diffusion-Based Relation Inversion from Imagescited by 82
  33. 2021On the Frequency Bias of Generative Modelscited by 60
  34. 2021Rethinking and Improving the Robustness of Image Style Transfercited by 48
  35. 2020SSD-GAN: Measuring the Realness in the Spatial and Spectral Domainscited by 38
  36. 2020Spatial Frequency Bias in Convolutional Generative Adversarial Networkscited by 35

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 78; 2025: 68; 2026: 44.

196 papers.

Sort:

By area:

  1. 2024InstanceDiffusion: Instance-level Control for Image Generationcited by 175
  2. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 158
  3. 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 133
  4. 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 126
  5. 2024Training-Free Consistent Text-to-Image Generationcited by 125
  6. 2024FasterCache: Training-Free Video Diffusion Model Acceleration with High Qualitycited by 122
  7. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 83
  8. 2024DITTO: Diffusion Inference-Time T-Optimization for Music Generationcited by 73
  9. 2023Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generationcited by 69
  10. 2024Video Diffusion Models are Training-free Motion Interpreter and Controllercited by 61
  11. 2024DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalizationcited by 60
  12. 2024FouriScale: A Frequency Perspective on Training-Free High-Resolution Image Synthesiscited by 59
38 more
  1. 2024U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformerscited by 54
  2. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  3. 2024Make a Cheap Scaling: A Self-Cascade Diffusion Model for Higher-Resolution Adaptationcited by 51
  4. 2024LCM-Lookahead for Encoder-based Text-to-Image Personalizationcited by 48
  5. 2024Good Seed Makes a Good Crop: Discovering Secret Seeds in Text-to-Image Diffusion Modelscited by 48
  6. 2024Towards Understanding the Working Mechanism of Text-to-Image Diffusion Modelcited by 48
  7. 2024EditSplat: Multi-View Fusion and Attention-Guided Optimization for View-Consistent 3D Scene Editing with 3D Gaussian Splattingcited by 41
  8. 2024A Survey of Mambacited by 39
  9. 2023SCEdit: Efficient and Controllable Image Diffusion Generation via Skip Connection Editingcited by 37
  10. 2024DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidancecited by 34
  11. 2025U-REPA: Aligning Diffusion U-Nets to ViTscited by 32
  12. 2024FreeScale: Unleashing the Resolution of Diffusion Models via Tuning-Free Scale Fusioncited by 30
  13. 2025ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matchingcited by 30
  14. 2024TextCraftor: Your Text Encoder Can be Image Quality Controllercited by 27
  15. 2024DITTO-2: Distilled Diffusion Inference-Time T-Optimization for Music Generationcited by 26
  16. 2025Latent Space Super-Resolution for Higher-Resolution Image Generation with Diffusion Modelscited by 25
  17. 2025Learning Distributions of Complex Fluid Simulations with Diffusion Graph Networkscited by 25
  18. 2025FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarioscited by 25
  19. 2024Privacy-Preserving Face Recognition Using Trainable Feature Subtractioncited by 24
  20. 2024An End-to-End Robust Point Cloud Semantic Segmentation Network with Single-Step Conditional Diffusion Modelscited by 23
  21. 2024FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Modelscited by 22
  22. 2025HiFlow: Training-free High-Resolution Image Generation with Flow-Aligned Guidancecited by 22
  23. 2023High-fidelity Person-centric Subject-to-Image Synthesiscited by 21
  24. 2024Beta Sampling is All You Need: Efficient Image Generation Strategy for Diffusion Models using Stepwise Spectral Analysiscited by 19
  25. 2025DepthMaster: Taming Diffusion Models for Monocular Depth Estimationcited by 19
  26. 2025Reward-Instruct: A Reward-Centric Approach to Fast Photo-Realistic Image Generationcited by 19
  27. 2023Clockwork Diffusion: Efficient Generation With Model-Step Distillationcited by 18
  28. 2025UltraHR-100K: Enhancing UHR Image Synthesis with A Large-Scale High-Quality Datasetcited by 18
  29. 2024Pruning then Reweighting: Towards Data-Efficient Training of Diffusion Modelscited by 17
  30. 2024Presto! Distilling Steps and Layers for Accelerating Music Generationcited by 16
  31. 2024SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing Pansharpeningcited by 15
  32. 2024FRAG: Frequency Adapting Group for Diffusion Video Editingcited by 15
  33. 2024FouRA: Fourier Low Rank Adaptationcited by 15
  34. 2024TVG: A Training-free Transition Video Generation Method with Diffusion Modelscited by 15
  35. 2024HiPrompt: Tuning-free Higher-Resolution Generation with Hierarchical MLLM Promptscited by 15
  36. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  37. 2024DreamCatalyst: Fast and High-Quality 3D Editing via Controlling Editability and Identity Preservationcited by 14
  38. 2024Cinemo: Consistent and Controllable Image Animation with Motion Diffusion Modelscited by 14

50 most cited of 196 All 196 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023The Missing U for Efficient Diffusion Models
  2. 2024U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Models
  3. 2025Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment
  4. 2026UDT: Reconciling U-Nets and Diffusion Transformers with Data-Adaptive Token Reduction
  5. 2024DuoDiff: Accelerating Diffusion Models with a Dual-Backbone Approach
  6. 2023FreeInit: Bridging Initialization Gap in Video Diffusion Models
  7. 2025Training-free Diffusion Acceleration with Bottleneck Sampling
  8. 2025ALTER: All-in-One Layer Pruning and Temporal Expert Routing for Efficient Diffusion Generation