mlsanityData to 14 Oct 2026

Spatial-Frequency U-Net for Denoising Diffusion Probabilistic Models

arXiv 2307.14648 ↗First posted 27 Jul 2023cs.CV cs.LGcited by 7

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

In this paper, we study the denoising diffusion probabilistic model (DDPM) in wavelet space, instead of pixel space, for visual synthesis. Considering the wavelet transform represents the image in spatial and frequency domains, we carefully design a novel architecture SFUNet to effectively capture the correlation for both domains. Specifically, in the standard denoising U-Net for pixel data, we supplement the 2D convolutions and spatial-only attention layers with our spatial frequency-aware convolution and attention modules to jointly model the complementary information from spatial and frequency domains in wavelet data. Our new architecture can be used as a drop-in replacement to the pixel-based network and is compatible with the vanilla DDPM training process. By explicitly modeling the wavelet signals, we find our model is able to generate images with higher quality on CIFAR-10, FFHQ, LSUN-Bedroom, and LSUN-Church datasets, than the pixel-based counterpart.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 30 of its 39 references that are in the map. First posted on arXiv 27 Jul 2023.

Builds on

30 of 39 references.

Closest ancestors · 7

Same direction, newest first

  1. 2022Wavelet Diffusion Models are fast and scalable Image Generators29 Nov
  2. 2022Classifier-Free Diffusion Guidance26 Jul
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  4. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  5. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  6. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  7. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  8. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  9. 2022Video Diffusion Modelscited by 2,305
  10. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  11. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  12. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
11 more
  1. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  2. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  3. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  4. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  5. 2022Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learningcited by 78
  6. 2023NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Modelscited by 76
  7. 2018Multi-scale 3D Convolution Network for Video Based Person Re-Identificationcited by 41
  8. 2022Multi-stage image denoising with the wavelet transformcited by 37
  9. 2021WINNet: Wavelet-inspired Invertible Network for Image Denoisingcited by 34
  10. 2023Neural Wavelet-domain Diffusion for 3D Shape Generation, Inversion, and Manipulationcited by 27
  11. 2023Waving Goodbye to Low-Res: A Diffusion-Wavelet Approach for Image Super-Resolutioncited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 1.

7 papers.

  1. 2024Boosting Diffusion Models with Moving Average Sampling in Frequency Domaincited by 55
  2. 2024DiMSUM: Diffusion Mamba -- A Scalable and Unified Spatial-Frequency Method for Image Generationcited by 11
  3. 2025Shaping Inductive Bias in Diffusion Models through Frequency-Based Noise Controlcited by 11
  4. 2025Decoupling High and Low Frequencies for Faithful Image Generation with Fine Detailscited by 7
  5. 2025Learning What Matters: Steering Diffusion via Spectrally Anisotropic Forward Noisecited by 2
  6. 2024Spatial-and-Frequency-aware Restoration method for Images based on Diffusion Modelscited by 0
  7. 2026Towards Investigating Residual Hearing Loss: Quantification of Fibrosis in a Novel Cochlear OCT Datasetcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Improving Denoising Diffusion Probabilistic Models via Exploiting Shared Representations
  2. 2023Diffusion Models Without Attention
  3. 2022Diffusion Probabilistic Model Made Slim
  4. 2023Assessing the capacity of a denoising diffusion probabilistic model to reproduce spatial context
  5. 2020Denoising Diffusion Implicit Models
  6. 2023UDPM: Upsampling Diffusion Probabilistic Models
  7. 2023The Missing U for Efficient Diffusion Models
  8. 2025A Fourier Space Perspective on Diffusion Models