mlsanityData to 7 Oct 2026

Diffusion Boosted Trees

arXiv 2406.01813 ↗First posted 3 Jun 2024stat.ML cs.AI cs.LGcited by 2

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Combining the merits of both denoising diffusion probabilistic models and gradient boosting, the diffusion boosting paradigm is introduced for tackling supervised learning problems. We develop Diffusion Boosted Trees (DBT), which can be viewed as both a new denoising diffusion generative model parameterized by decision trees (one single tree for each diffusion timestep), and a new boosting algorithm that combines the weak learners into a strong learner of conditional distributions without making explicit parametric assumptions on their density forms. We demonstrate through experiments the advantages of DBT over deep neural network-based diffusion models as well as the competence of DBT on real-world regression tasks, and present a business application (fraud detection) of DBT for classification on tabular data with the ability of learning to defer.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 45 of its 70 references that are in the map. First posted on arXiv 3 Jun 2024.

Builds on

45 of 70 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation5 Apr
  2. 2023Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Acceleration26 Oct
  3. 2023Elucidating the Exposure Bias in Diffusion Models29 Aug
  4. 2023Input Perturbation Reduces Exposure Bias in Diffusion Models27 Jan
  5. 2022CARD: Classification and Regression Diffusion Models15 Jun
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  8. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  10. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  8. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  9. 2016Variational Inference: A Review for Statisticianscited by 1,466
  10. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  11. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  12. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
23 more
  1. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  2. 2021Tabular Data: Deep Learning is Not All You Needcited by 530
  3. 2014OpenML: networked science in machine learningcited by 523
  4. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  5. 2021Deep Neural Networks and Tabular Data: A Surveycited by 489
  6. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  7. 2017Predict Responsibly: Improving Fairness and Accuracy by Learning to Defercited by 228
  8. 2017Concrete Dropoutcited by 227
  9. 2022Why do tree-based models still outperform deep learning on tabular data?cited by 198
  10. 2019Generalization in Generation: A closer look at Exposure Biascited by 123
  11. 2019NGBoost: Natural Gradient Boosting for Probabilistic Predictioncited by 94
  12. 2018Deep Neural Decision Treescited by 72
  13. 2022Calibrated Learning to Defer with One-vs-All Classifierscited by 68
  14. 2023Generating and Imputing Tabular Data via Diffusion and Flow-based Gradient-Boosted Treescited by 60
  15. 2020Uncertainty in Gradient Boosting via Ensemblescited by 56
  16. 2018Semi-Implicit Variational Inferencecited by 55
  17. 2024Thousands of AI Authors on the Future of AIcited by 48
  18. 2021Contextual Dropout: An Efficient Sample-Dependent Dropout Modulecited by 26
  19. 2021Wasserstein Generative Learning of Conditional Distributioncited by 24
  20. 2020Learnable Bernoulli Dropout for Bayesian Deep Learningcited by 14
  21. 2019Thompson Sampling via Local Uncertaintycited by 12
  22. 2020SGLB: Stochastic Gradient Langevin Boostingcited by 12
  23. 2019Adaptive Correlated Monte Carlo for Contextual Categorical Sequence Generationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Machine Learning for Network Attacks Classification and Statistical Evaluation of Adversarial Learning Methodologies for Synthetic Data Generationcited by 0
  2. 2026Optimal Mixture-of-Experts Model Averaging for Conditional Generative Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Treeffuser: Probabilistic Predictions via Conditional Diffusions with Gradient-Boosted Trees
  2. 2022Diffusion models as plug-and-play priors
  3. 2025Diffusion Models: A Mathematical Introduction
  4. 2024An overview of diffusion models for generative artificial intelligence
  5. 2023Adaptive Training Meets Progressive Scaling: Elevating Efficiency in Diffusion Models
  6. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion models
  7. 2026Conditioning Tree-Based Diffusions and Flows for Probabilistic Tabular Regression
  8. 2024Generative Modeling with Diffusion