# Boosting Diffusion Models with Moving Average Sampling in Frequency Domain

arXiv: https://arxiv.org/abs/2403.17870 · HTML: https://mlsanity.com/paper/2403.17870/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 48 of its 60 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 48)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022)
- [Structure and Content-Guided Video Synthesis with Diffusion Models](https://mlsanity.com/paper/2302.03011/) (2023)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [All are Worth Words: A ViT Backbone for Diffusion Models](https://mlsanity.com/paper/2209.12152/) (2022)
- [UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2302.04867/) (2023)
- [Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.06503/) (2022)
- [DiffIR: Efficient Diffusion Model for Image Restoration](https://mlsanity.com/paper/2303.09472/) (2023)
- [Diffusion Probabilistic Modeling for Video Generation](https://mlsanity.com/paper/2203.09481/) (2022)
- [Gotta Go Fast When Generating Data with Score-Based Models](https://mlsanity.com/paper/2105.14080/) (2021)
- [Generative Diffusion Prior for Unified Image Restoration and Enhancement](https://mlsanity.com/paper/2304.01247/) (2023)
- [StyleSwin: Transformer-based GAN for High-resolution Image Generation](https://mlsanity.com/paper/2112.10762/) (2021)
- [gDDIM: Generalized denoising diffusion implicit models](https://mlsanity.com/paper/2206.05564/) (2022)
- [Diffusion Probabilistic Model Made Slim](https://mlsanity.com/paper/2211.17106/) (2022)

## Built on by (most cited first, 30 of 55)

- [STAR: Spatial-Temporal Augmentation with Text-to-Video Models for Real-World Video Super-Resolution](https://mlsanity.com/paper/2501.02976/) (2025)
- [MotionPro: A Precise Motion Controller for Image-to-Video Generation](https://mlsanity.com/paper/2505.20287/) (2025)
- [Hi3D: Pursuing High-Resolution Image-to-3D Generation with Video Diffusion Models](https://mlsanity.com/paper/2409.07452/) (2024)
- [Visual Autoregressive Modeling for Instruction-Guided Image Editing](https://mlsanity.com/paper/2508.15772/) (2025)
- [Attend to Not Attended: Structure-then-Detail Token Merging for Post-training DiT Acceleration](https://mlsanity.com/paper/2505.11707/) (2025)
- [Improving Virtual Try-On with Garment-focused Diffusion Models](https://mlsanity.com/paper/2409.08258/) (2024)
- [NFIG: Multi-Scale Autoregressive Image Generation via Frequency Ordering](https://mlsanity.com/paper/2503.07076/) (2025)
- [Data Synthesis with Diverse Styles for Face Recognition via 3DMM-Guided Diffusion](https://mlsanity.com/paper/2504.00430/) (2025)
- [Shaping Inductive Bias in Diffusion Models through Frequency-Based Noise Control](https://mlsanity.com/paper/2502.10236/) (2025)
- [BWCache: Accelerating Video Diffusion Transformers through Block-Wise Caching](https://mlsanity.com/paper/2509.13789/) (2025)
- [DreamMesh: Jointly Manipulating and Texturing Triangle Meshes for Text-to-3D Generation](https://mlsanity.com/paper/2409.07454/) (2024)
- [ANT: Adaptive Neural Temporal-Aware Text-to-Motion Model](https://mlsanity.com/paper/2506.02452/) (2025)
- [Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On](https://mlsanity.com/paper/2505.16977/) (2025)
- [Hierarchical Masked Autoregressive Models with Low-Resolution Token Pivots](https://mlsanity.com/paper/2505.20288/) (2025)
- [Latent Wavelet Diffusion For Ultra-High-Resolution Image Synthesis](https://mlsanity.com/paper/2506.00433/) (2025)
- [Decoupling High and Low Frequencies for Faithful Image Generation with Fine Details](https://mlsanity.com/paper/2509.05441/) (2025)
- [FreeEnhance: Tuning-Free Image Enhancement via Content-Consistent Noising-and-Denoising Process](https://mlsanity.com/paper/2409.07451/) (2024)
- [Proteus-ID: ID-Consistent and Motion-Coherent Video Customization](https://mlsanity.com/paper/2506.23729/) (2025)
- [ASTRA: Let Arbitrary Subjects Transform in Video Editing](https://mlsanity.com/paper/2510.01186/) (2025)
- [MatchDiffusion: Training-free Generation of Match-cuts](https://mlsanity.com/paper/2411.18677/) (2024)
- [Free-T2M: Robust Text-to-Motion Generation for Humanoid Robots via Frequency-Domain](https://mlsanity.com/paper/2501.18232/) (2025)
- [NeuralRemaster: Phase-Preserving Diffusion for Structure-Aligned Generation](https://mlsanity.com/paper/2512.05106/) (2025)
- [Prompt2Perturb (P2P): Text-Guided Diffusion-Based Adversarial Attacks on Breast Ultrasound Images](https://mlsanity.com/paper/2412.09910/) (2024)
- [Harnessing Frequency Spectrum Insights for Image Copyright Protection Against Diffusion Models](https://mlsanity.com/paper/2503.11071/) (2025)
- [What's in a Latent? Leveraging Diffusion Latent Space for Domain Generalization](https://mlsanity.com/paper/2503.06698/) (2025)
- [Elucidating the SNR-t Bias of Diffusion Probabilistic Models](https://mlsanity.com/paper/2604.16044/) (2026)
- [Score-based Generative Models with Adaptive Momentum](https://mlsanity.com/paper/2405.13726/) (2024)
- [Taming Consistency Distillation for Accelerated Human Image Animation](https://mlsanity.com/paper/2504.11143/) (2025)
- [Creatively Upscaling Images with Global-Regional Priors](https://mlsanity.com/paper/2505.16976/) (2025)
- [Generative Modeling of Full-Atom Protein Conformations using Latent Diffusion on Graph Embeddings](https://mlsanity.com/paper/2506.17064/) (2025)

## Abstract (arXiv)

Diffusion models have recently brought a powerful revolution in image generation. Despite showing impressive generative capabilities, most of these models rely on the current sample to denoise the next one, possibly resulting in denoising instability. In this paper, we reinterpret the iterative denoising process as model optimization and leverage a moving average mechanism to ensemble all the prior samples. Instead of simply applying moving average to the denoised samples at different timesteps, we first map the denoised samples to data space and then perform moving average to avoid distribution shift across timesteps. In view that diffusion models evolve the recovery from low-frequency components to high-frequency details, we further decompose the samples into different frequency components and execute moving average separately on each component. We name the complete approach "Moving Average Sampling in Frequency domain (MASF)". MASF could be seamlessly integrated into mainstream pre-trained diffusion models and sampling schedules. Extensive experiments on both unconditional and conditional diffusion models demonstrate that our MASF leads to superior performances compared to the baselines, with almost negligible additional complexity cost.