# NRGBoost: Energy-Based Generative Boosted Trees

arXiv: https://arxiv.org/abs/2410.03535 · HTML: https://mlsanity.com/paper/2410.03535/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 39 references that are in the map.
- First posted on arXiv 4 Oct 2024.

## Builds on (most cited first, 25 of 25)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [TabDDPM: Modelling Tabular Data with Diffusion Models](https://mlsanity.com/paper/2209.15421/) (2022)
- [Language Models are Realistic Tabular Data Generators](https://mlsanity.com/paper/2210.06280/) (2022)
- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [CTAB-GAN: Effective Table Data Synthesizing](https://mlsanity.com/paper/2102.08369/) (2021)
- [NGBoost: Natural Gradient Boosting for Probabilistic Prediction](https://mlsanity.com/paper/1910.03225/) (2019)
- [Conditional Wasserstein GAN-based Oversampling of Tabular Data for Imbalanced Learning](https://mlsanity.com/paper/2008.09202/) (2020)
- [Generating and Imputing Tabular Data via Diffusion and Flow-based Gradient-Boosted Trees](https://mlsanity.com/paper/2309.09968/) (2023)
- [Adversarial random forests for density estimation and generative modeling](https://mlsanity.com/paper/2205.09435/) (2022)
- [TabMT: Generating tabular data with masked transformers](https://mlsanity.com/paper/2312.06089/) (2023)
- [AdaGAN: Boosting Generative Models](https://mlsanity.com/paper/1701.02386/) (2017)
- [TabPFGen -- Tabular Data Generation with TabPFN](https://mlsanity.com/paper/2406.05216/) (2024)
- [Joints in Random Forests](https://mlsanity.com/paper/2006.14937/) (2020)
- [Relational Data Synthesis using Generative Adversarial Networks: A Design Space Exploration](https://mlsanity.com/paper/2008.12763/) (2020)
- [Boosted Generative Models](https://mlsanity.com/paper/1702.08484/) (2017)
- [TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Models](https://mlsanity.com/paper/2409.16118/) (2024)
- [Boosted Density Estimation Remastered](https://mlsanity.com/paper/1803.08178/) (2018)
- [Generative Trees: Adversarial and Copycat](https://mlsanity.com/paper/2201.11205/) (2022)
- [Unmasking Trees for Tabular Data](https://mlsanity.com/paper/2407.05593/) (2024)
- [Generative Forests](https://mlsanity.com/paper/2308.03648/) (2023)
- [arfpy: A python package for density estimation and generative modeling with adversarial random forests](https://mlsanity.com/paper/2311.07366/) (2023)

## Built on by (most cited first, 1 of 1)

- [Creation begins with understanding: LLMs as strategy designers for privacy-preserving tabular data synthesis](https://mlsanity.com/paper/2608.29674/) (2026)

## Abstract (arXiv)

Despite the rise to dominance of deep learning in unstructured data domains, tree-based methods such as Random Forests (RF) and Gradient Boosted Decision Trees (GBDT) are still the workhorses for handling discriminative tasks on tabular data. We explore generative extensions of these popular algorithms with a focus on explicitly modeling the data density (up to a normalization constant), thus enabling other applications besides sampling. As our main contribution we propose an energy-based generative boosting algorithm that is analogous to the second-order boosting implemented in popular libraries like XGBoost. We show that, despite producing a generative model capable of handling inference tasks over any input variable, our proposed algorithm can achieve similar discriminative performance to GBDT on a number of real world tabular datasets, outperforming alternative generative approaches. At the same time, we show that it is also competitive with neural-network-based models for sampling. Code is available at https://github.com/ajoo/nrgboost.