mlsanityData to 14 Oct 2026

textitJump Your Steps: Optimizing Sampling Schedule of Discrete Diffusion Models

arXiv 2410.07761 ↗First posted 10 Oct 2024cs.LG cs.AI cs.CLcited by 40

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have seen notable success in continuous domains, leading to the development of discrete diffusion models (DDMs) for discrete variables. Despite recent advances, DDMs face the challenge of slow sampling speeds. While parallel sampling methods like τ-leaping accelerate this process, they introduce textitCompounding Decoding Error (CDE), where discrepancies arise between the true distribution and the approximation from parallel token generation, leading to degraded sample quality. In this work, we present textitJump Your Steps (JYS), a novel approach that optimizes the allocation of discrete sampling timesteps by minimizing CDE without extra computational cost. More precisely, we derive a practical upper bound on CDE and propose an efficient algorithm for searching for the optimal sampling schedule. Extensive experiments across image, music, and text generation show that JYS significantly improves sampling quality, establishing it as a versatile framework for enhancing DDM performance for fast sampling.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 33 of its 41 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

33 of 41 references.

Closest ancestors · 21

Same direction, newest first

  1. 2024Informed Correctors for Discrete Diffusion Models30 Jul
  2. 2024Align Your Steps: Optimizing Sampling Schedules in Diffusion Models22 Apr
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  4. 2023DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics20 Oct
  5. 2023Restart Sampling for Improving Generative Processes26 Jun
  6. 2023Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces18 May
  7. 2022Scalable Diffusion Models with Transformers19 Dec
  8. 2022Score-based Continuous-time Discrete Diffusion Models30 Nov
  9. 2022GENIE: Higher-Order Denoising Diffusion Solvers11 Oct
  10. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  11. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  12. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
9 more
  1. 2022A Continuous Time Framework for Discrete Denoising Models30 May
  2. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  3. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  4. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  6. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  8. 2020Denoising Diffusion Implicit Models6 Oct
  9. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 12

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  2. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  3. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  4. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  5. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  6. 2024Discrete Flow Matchingcited by 317
  7. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 305
  8. 2021Autoregressive Diffusion Modelscited by 210
  9. 2017MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompanimentcited by 151
  10. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 148
  11. 2023Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Timecited by 20
  12. 2024Promises, Outlooks and Challenges of Diffusion Language Modelingcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 25; 2026: 14.

40 papers.

Sort:

By area:

  1. 2025Large Language Diffusion Modelscited by 912
  2. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 131
  3. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 88
  4. 2024Distillation of Discrete Diffusion through Dimensional Correlationscited by 48
  5. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  6. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Modelscited by 34
  7. 2025MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Controlcited by 21
  8. 2025DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learningcited by 19
  9. 2025Ultra-Fast Language Generation via Discrete Diffusion Divergence Instructcited by 16
  10. 2025Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decodingcited by 15
  11. 2025Error Bounds and Optimal Schedules for Masked Diffusions with Factorized Approximationscited by 14
  12. 2025Parallel Sampling from Masked Diffusion Models via Conditional Independence Testingcited by 13
28 more
  1. 2026The Diffusion Duality, Chapter II: Psi-Samplerscited by 12
  2. 2025The Cosine Schedule is Fisher-Rao-Optimal for Masked Discrete Diffusion Modelscited by 11
  3. 2025Demystifying MaskGIT Sampler and Beyond: Adaptive Order Selection in Masked Diffusioncited by 10
  4. 2025Masks Can Be Distracting: On Context Comprehension in Diffusion Language Modelscited by 9
  5. 2025Learnable Sampler Distillation for Discrete Diffusion Modelscited by 8
  6. 2025Taming Flow Matching with Unbalanced Optimal Transport into Fast Pansharpeningcited by 6
  7. 2025RAPID^3: Tri-Level Reinforced Acceleration Policies for Diffusion Transformercited by 6
  8. 2025Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddingscited by 6
  9. 2025Non-Markovian Discrete Diffusion with Causal Language Modelscited by 5
  10. 2025PairFlow: Closed-Form Source-Target Coupling for Few-Step Generation in Discrete Flow Modelscited by 5
  11. 2026Fixed-Point Masked Generative Modelingcited by 5
  12. 2025DiffGRM: Diffusion-based Generative Recommendation Modelcited by 4
  13. 2026DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformerscited by 4
  14. 2026Few-Step Diffusion Sampling Through Instance-Aware Discretizationscited by 4
  15. 2025Masked Diffusion Models as Energy Minimizationcited by 3
  16. 2026Improved Sampling Schedules for Discrete Diffusion Modelscited by 3
  17. 2026Formalizing the Sampling Design Space of Diffusion-Based Generative Models via Adaptive Solvers and Wasserstein-Bounded Timestepscited by 3
  18. 2026Training-Free Self-Correction for Multimodal Masked Diffusion Modelscited by 2
  19. 2026A2D2: Fine-Tuning Any-Length Discrete Diffusion for Adaptive Decodingcited by 2
  20. 2025Deterministic Discrete Denoisingcited by 1
  21. 2025Distillation of Discrete Diffusion by Exact Conditional Distribution Matchingcited by 1
  22. 2026Lookahead Path Likelihood Optimization for Diffusion LLMscited by 1
  23. 2026Is Your Diffusion Sampler Actually Correct? A Sampler-Centric Evaluation of Discrete Diffusion Language Modelscited by 1
  24. 2026Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Modelscited by 1
  25. 2025Data augmentation using diffusion models to enhance inverse Ising inferencecited by 0
  26. 2026Stratified Hazard Sampling: Minimal-Variance Event Scheduling for CTMC/DTMC Discrete Diffusion and Flow Modelscited by 0
  27. 2026Progressive Refinement Regulation for Accelerating Diffusion Language Model Decodingcited by 0
  28. 2026Information-geometric adaptive sampling for graph diffusioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models
  2. 2026Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guarantees
  3. 2025SynBoost: A Synergistic Framework for Fast Sampling of Diffusion Models
  4. 2021On Fast Sampling of Diffusion Probabilistic Models
  5. 2025Optimal Stepsize for Diffusion Sampling
  6. 2026Optimize Your Sampling: Tuned Diffusion Sampling with Bayesian Optimization
  7. 2026Analyzing and Improving Fast Sampling of Text-to-Image Diffusion Models
  8. 2026From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Models