mlsanityData to 14 Oct 2026

Restart Sampling for Improving Generative Processes

arXiv 2306.14878 ↗First posted 26 Jun 2023cs.LG cs.CV stat.COcited by 84

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Generative processes that involve solving differential equations, such as diffusion models, frequently necessitate balancing speed and quality. ODE-based samplers are fast but plateau in performance while SDE-based samplers deliver higher sample quality at the cost of increased sampling time. We attribute this difference to sampling errors: ODE-samplers involve smaller discretization errors while stochasticity in SDE contracts accumulated errors. Based on these findings, we propose a novel sampling algorithm called Restart in order to better balance discretization errors and contraction. The sampling method alternates between adding substantial noise in additional forward steps and strictly following a backward ODE. Empirically, Restart sampler surpasses previous SDE and ODE samplers in both speed and accuracy. Restart not only outperforms the previous best SDE results, but also accelerates the sampling speed by 10-fold / 2-fold on CIFAR-10 / ImageNet 64 × 64. In addition, it attains significantly better sample quality than ODE samplers within comparable sampling times. Moreover, Restart better balances text-image alignment/visual quality versus diversity than previous samplers in the large-scale text-to-image Stable Diffusion model pre-trained on LAION 512 × 512. Code is available at https://github.com/Newbeeer/diffusionrestartsampling

Key facts. Cited by 84 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 17 of its 31 references that are in the map. First posted on arXiv 26 Jun 2023.

Builds on

17 of 31 references.

Closest ancestors · 9

Same direction, newest first

  1. 2022On Distillation of Guided Diffusion Models6 Oct
  2. 2022Classifier-Free Diffusion Guidance26 Jul
  3. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  6. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  8. 2020Denoising Diffusion Implicit Models6 Oct
  9. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 8

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  5. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  6. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  7. 2022Poisson Flow Generative Modelscited by 95
  8. 2023PFGM++: Unlocking the Potential of Physics-Inspired Generative Modelscited by 73

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 11; 2024: 22; 2025: 30; 2026: 21.

84 papers.

Sort:

By area:

  1. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 459
  2. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 305
  3. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
  4. 2024Dirichlet Flow Matching with Applications to DNA Sequence Designcited by 115
  5. 2025Inductive Moment Matchingcited by 113
  6. 2023Elucidating the Exposure Bias in Diffusion Modelscited by 101
  7. 2024Align Your Steps: Optimizing Sampling Schedules in Diffusion Modelscited by 96
  8. 2023Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Modelscited by 71
  9. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 67
  10. 2024TabDiff: a Mixed-type Diffusion Model for Tabular Data Generationcited by 63
  11. 2024StableMoFusion: Towards Robust and Efficient Diffusion-based Motion Generation Frameworkcited by 50
  12. 2025Inference-time Scaling of Diffusion Models through Classical Searchcited by 49
38 more
  1. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  2. 2024Trajectory Consistency Distillation: Improved Latent Consistency Distillation by Semi-Linear Consistency Function with Trajectory Mappingcited by 46
  3. 2025One-step Diffusion Models with f-Divergence Distribution Matchingcited by 43
  4. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexitycited by 42
  5. 2024textitJump Your Steps: Optimizing Sampling Schedule of Discrete Diffusion Modelscited by 40
  6. 2025Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcingcited by 38
  7. 2025Efficient Diffusion Models: A Surveycited by 35
  8. 2024Spatiotemporal Skip Guidance for Enhanced Video Diffusion Samplingcited by 34
  9. 2024Understanding and Improving Training-free Loss-based Diffusion Guidancecited by 32
  10. 2023Object-aware Inversion and Reassembly for Image Editingcited by 30
  11. 2025Test-Time Scaling of Diffusion Models via Noise Trajectory Searchcited by 30
  12. 2025La-Proteina: Atomistic Protein Generation via Partially Latent Flow Matchingcited by 30
  13. 2023CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generationcited by 26
  14. 2024Truncated Consistency Modelscited by 22
  15. 2023FRDiff : Feature Reuse for Universal Training-free Acceleration of Diffusion Modelscited by 20
  16. 2024DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latentscited by 20
  17. 2025Taming Video Models for 3D and 4D Generation via Zero-Shot Camera Controlcited by 18
  18. 2025Not-So-Optimal Transport Flows for 3D Point Cloud Generationcited by 17
  19. 2025GLASS Flows: Transition Sampling for Alignment of Flow and Diffusion Modelscited by 17
  20. 2023Correcting Diffusion Generation through Resamplingcited by 15
  21. 2026Self-Refining Video Samplingcited by 15
  22. 2024SCott: Accelerating Diffusion Models with Stochastic Consistency Distillationcited by 14
  23. 2024Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Modelscited by 13
  24. 2025Dynamic Classifier-Free Diffusion Guidance via Online Feedbackcited by 13
  25. 2024Mixture of Efficient Diffusion Experts Through Automatic Interval and Sub-Network Selectioncited by 12
  26. 2025FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Computecited by 12
  27. 2024Diffusion Rejection Samplingcited by 11
  28. 2024Towards Understanding and Quantifying Uncertainty for Text-to-Image Generationcited by 11
  29. 2025Transition Matching: Scalable and Flexible Generative Modelingcited by 11
  30. 2024Zero-to-Hero: Enhancing Zero-Shot Novel View Synthesis via Attention Map Filteringcited by 9
  31. 2023Spectroscopy-Guided Discovery of Three-Dimensional Structures of Disordered Materials with Diffusion Modelscited by 8
  32. 2025Diffusion Adaptive Text Embedding for Text-to-Image Diffusion Modelscited by 8
  33. 2024Multistable Shape from Shading Emerges from Patch Diffusioncited by 6
  34. 2025Diffusion Model Quantization: A Reviewcited by 6
  35. 2025Towards a Golden Classifier-Free Guidance Path via Foresight Fixed Point Iterationscited by 6
  36. 2025Adaptive Stochastic Coefficients for Accelerating Diffusion Samplingcited by 6
  37. 2023Your Student is Better Than Expected: Adaptive Teacher-Student Collaboration for Text-Conditional Diffusion Modelscited by 5
  38. 2025Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generationcited by 5

50 most cited of 84 All 84 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Exploring the Optimal Choice for Generative Processes in Diffusion Models: Ordinary vs Stochastic Differential Equations
  2. 2023Elucidating the solution space of extended reverse-time SDE for diffusion models
  3. 2024On the Trajectory Regularity of ODE-based Diffusion Sampling
  4. 2024An Ordinary Differential Equation Sampler with Stochastic Start for Diffusion Bridge Models
  5. 2023Fast ODE-based Sampling for Diffusion Models in Around 5 Steps
  6. 2026Formalizing the Sampling Design Space of Diffusion-Based Generative Models via Adaptive Solvers and Wasserstein-Bounded Timesteps
  7. 2025Adaptivity and Convergence of Probability Flow ODEs in Diffusion Generative Models
  8. 2025Are First-Order Diffusion Samplers Really Slower? A Fast Forward-Value Approach