mlsanityData to 14 Oct 2026

Beta Diffusion

arXiv 2309.07867 ↗First posted 14 Sep 2023cs.LG cs.AI stat.COcited by 17

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We introduce beta diffusion, a novel generative modeling method that integrates demasking and denoising to generate data within bounded ranges. Using scaled and shifted beta distributions, beta diffusion utilizes multiplicative transitions over time to create both forward and reverse diffusion processes, maintaining beta distributions in both the forward marginals and the reverse conditionals, given the data at any point in time. Unlike traditional diffusion-based generative models relying on additive Gaussian noise and reweighted evidence lower bounds (ELBOs), beta diffusion is multiplicative and optimized with KL-divergence upper bounds (KLUBs) derived from the convexity of the KL divergence. We demonstrate that the proposed KLUBs are more effective for optimizing beta diffusion compared to negative ELBOs, which can also be derived as the KLUBs of the same KL divergence with its two arguments swapped. The loss function of beta diffusion, expressed in terms of Bregman divergence, further supports the efficacy of KLUBs for optimization. Experimental results on both synthetic data and natural images demonstrate the unique capabilities of beta diffusion in generative modeling of range-bounded data and validate the effectiveness of KLUBs in optimizing diffusion models, thereby making them valuable additions to the family of diffusion-based generative models and the optimization techniques used to train them.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 60 of its 81 references that are in the map. First posted on arXiv 14 Sep 2023.

Builds on

60 of 81 references.

Closest ancestors · 34

Same direction, newest first

  1. 2023Fast Diffusion Model12 Jun
  2. 2023Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models25 Apr
  3. 2023Diffusion Models for Constrained Domains11 Apr
  4. 2023Reflected Diffusion Models10 Apr
  5. 2023Efficient Diffusion Training via Min-SNR Weighting Strategy16 Mar
  6. 2023Optimizing DDPM Sampling with Shortcut Fine-Tuning31 Jan
  7. 2022Fast Sampling of Diffusion Models via Operator Learning24 Nov
  8. 2022Blurring Diffusion Models12 Sep
  9. 2022Soft Diffusion: Score Matching for General Corruptions12 Sep
  10. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise19 Aug
  11. 2022Classifier-Free Diffusion Guidance26 Jul
  12. 2022CARD: Classification and Regression Diffusion Models15 Jun
22 more
  1. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  2. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  3. 2022A Continuous Time Framework for Discrete Denoising Models30 May
  4. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  5. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  6. 2022Riemannian Score-Based Generative Modelling6 Feb
  7. 2022DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents2 Jan
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  9. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  10. 2021Global Context with Discrete Diffusion in Vector Quantised Modelling for Image Generation3 Dec
  11. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  12. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  13. 2021Variational Diffusion Models1 Jul
  14. 2021Score-based Generative Modeling in Latent Space10 Jun
  15. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  16. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  17. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  18. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  19. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  20. 2020Denoising Diffusion Implicit Models6 Oct
  21. 2020Denoising Diffusion Probabilistic Models19 Jun
  22. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  7. 2016Variational Inference: A Review for Statisticianscited by 1,472
  8. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  9. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  10. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  11. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  12. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
14 more
  1. 2022Torsional Diffusion for Molecular Conformer Generationcited by 263
  2. 2023Imitating Human Behaviour with Diffusion Modelscited by 263
  3. 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generationcited by 257
  4. 2022Diffusion-GAN: Training GANs with Diffusioncited by 205
  5. 2022Scalable Adaptive Computation for Iterative Generationcited by 195
  6. 2022Generative Modelling With Inverse Heat Dissipationcited by 167
  7. 2021Learning Gradient Fields for Molecular Conformation Generationcited by 163
  8. 2023Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyondcited by 139
  9. 2023In-Context Learning Unlocked for Diffusion Modelscited by 104
  10. 2023DR2: Diffusion-based Robust Degradation Remover for Blind Face Restorationcited by 79
  11. 2023Dirichlet Diffusion Score Model for Biological Sequence Generationcited by 75
  12. 2018Semi-Implicit Variational Inferencecited by 55
  13. 2020Exploiting Chain Rule and Bayes' Theorem to Compare Probability Distributionscited by 22
  14. 2023Learning to Jump: Thinning and Thickening Latent Counts for Generative Modelingcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 7; 2025: 6; 2026: 2.

17 papers.

Sort:

By area:

  1. 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Designcited by 79
  2. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 34
  3. 2024Diffusion-RPO: Aligning Diffusion Models through Relative Preference Optimizationcited by 32
  4. 2024Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Modelscited by 23
  5. 2025Variational Control for Guidance in Diffusion Modelscited by 22
  6. 2023Learning Stackable and Skippable LEGO Bricks for Efficient, Reconfigurable, and Variable-Resolution Diffusion Modelingcited by 13
  7. 2024Guided Score identity Distillation for Data-Free One-Step Text-to-Image Generationcited by 13
  8. 2024Advancing Graph Generation through Beta Diffusioncited by 13
  9. 2023Improving In-Context Learning in Diffusion Models with Visual Context-Modulated Promptscited by 10
  10. 2025Score Distillation of Flow Matching Modelscited by 8
  11. 2025ItDPDM: Information-Theoretic Discrete Poisson Diffusion Modelcited by 6
  12. 2025PowerGrow: Feasible Co-Growth of Structures and Dynamics for Power Grid Synthesiscited by 5
5 more
  1. 2024ParamReL: Learning Parameter Space Representation via Progressively Encoding Bayesian Flow Networkscited by 2
  2. 2024Marked Temporal Bayesian Flow Point Processescited by 0
  3. 2025Closed-Form Beta Distribution Estimation from Sparse Statistics with Random Forest Implicit Regularizationcited by 0
  4. 2026Operationally Feasible Synthetic Power-Grid Scenarios via Learning the AC-Operable Joint Distributioncited by 0
  5. 2026Diffusion-2BC: Hybrid Diffusion and Regression Training for Offline Behavior Cloning in Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Diffusion Models: A Mathematical Introduction
  2. 2024Beta Sampling is All You Need: Efficient Image Generation Strategy for Diffusion Models using Stepwise Spectral Analysis
  3. 2025Improving Diffusion Generative Models via Truncated Karhunen--Lo`eve Expansion
  4. 2025Beyond Fixed Horizons: A Theoretical Framework for Adaptive Denoising Diffusions
  5. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models
  6. 2025Low-dimensional adaptation of diffusion models: Convergence in total variation
  7. 2022Diffusion models as plug-and-play priors
  8. 2025A Sharp KL-Convergence Analysis for Diffusion Models under Minimal Assumptions