mlsanityData to 7 Oct 2026

Fixed Point Diffusion Models

arXiv 2401.08741 ↗First posted 16 Jan 2024cs.CV cs.AI cs.LGcited by 14

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We introduce the Fixed Point Diffusion Model (FPDM), a novel approach to image generation that integrates the concept of fixed point solving into the framework of diffusion-based generative modeling. Our approach embeds an implicit fixed point solving layer into the denoising network of a diffusion model, transforming the diffusion process into a sequence of closely-related fixed point problems. Combined with a new stochastic training method, this approach significantly reduces model size, reduces memory usage, and accelerates training. Moreover, it enables the development of two new techniques to improve sampling efficiency: reallocating computation across timesteps and reusing fixed point solutions between timesteps. We conduct extensive experiments with state-of-the-art models on ImageNet, FFHQ, CelebA-HQ, and LSUN-Church, demonstrating substantial improvements in performance and efficiency. Compared to the state-of-the-art DiT model, FPDM contains 87% fewer parameters, consumes 60% less memory during training, and improves image generation quality in situations where sampling computation or time is limited. Our code and pretrained models are available at https://lukemelas.github.io/fixed-point-diffusion-models.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 38 of its 56 references that are in the map. First posted on arXiv 16 Jan 2024.

Builds on

38 of 56 references.

Closest ancestors · 15

Same direction, newest first

  1. 2023One-Step Diffusion Distillation via Deep Equilibrium Models12 Dec
  2. 2023Common Diffusion Noise Schedules and Sample Steps are Flawed15 May
  3. 2022Scalable Diffusion Models with Transformers19 Dec
  4. 2022Deep Equilibrium Approaches to Diffusion Models23 Oct
  5. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  7. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  8. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  9. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality11 Feb
  10. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  11. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  12. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
3 more
  1. 2020Denoising Diffusion Implicit Models6 Oct
  2. 2020Denoising Diffusion Probabilistic Models19 Jun
  3. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  7. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  9. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  10. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  11. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  12. 2019Deep Equilibrium Modelscited by 603
11 more
  1. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  2. 2021Neural SDEs as Infinite-Dimensional GANscited by 159
  3. 2020Multiscale Deep Equilibrium Modelscited by 149
  4. 2021Deep Equilibrium Architectures for Inverse Problems in Imagingcited by 126
  5. 2022Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Modelscited by 97
  6. 2021JFB: Jacobian-Free Backpropagation for Implicit Networkscited by 84
  7. 2021On Training Implicit Modelscited by 73
  8. 2022Deep Equilibrium Optical Flow Estimationcited by 63
  9. 2021Subformer: Exploring Weight Sharing for Parameter Efficiency in Generative Transformerscited by 42
  10. 2022FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstructioncited by 39
  11. 2023TorchDEQ: A Library for Deep Equilibrium Modelscited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 12.

14 papers.

Sort:

By area:

  1. 2026Improving Flow Matching by Aligning Flow Divergencecited by 9
  2. 2026Self-conditioned Flow Map Language Models via Fixed-point Flowscited by 9
  3. 2024Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modelingcited by 4
  4. 2024Certified Robustness for Deep Equilibrium Models via Serialized Random Smoothingcited by 2
  5. 2026Fixed-Point Masked Generative Modelingcited by 2
  6. 2026Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approachcited by 1
  7. 2026PoM: A Linear-Time Replacement for Attention with the Polynomial Mixercited by 0
  8. 2026Training-free, Perceptually Consistent Low-Resolution Previews with High-Resolution Image for Efficient Workflows of Diffusion Modelscited by 0
  9. 2026Visual Implicit Autoregressive Modelingcited by 0
  10. 2026Self-Consistent Generative Paths via Admissible Random Variational Transportcited by 0
  11. 2026Timestep Rescheduling in Diffusion Inversioncited by 0
  12. 2026Recursive Scaling in Masked Diffusion Modelscited by 0
2 more
  1. 2026Toward a mechanistic understanding of inference in visual cortex and diffusion modelscited by 0
  2. 2026Depth through recurrence: Looped transformers for flow-matching TTScited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Efficient and Training-Free Single-Image Diffusion Models
  2. 2024DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance
  3. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications
  4. 2026One-Step Distillation of Discrete Diffusion Image Generators via Fixed-Point Iteration
  5. 2023Nested Diffusion Processes for Anytime Image Generation
  6. 2023GECCO: Geometrically-Conditioned Point Diffusion Models
  7. 2024A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models
  8. 2024Conditional Image Synthesis with Diffusion Models: A Survey