# GBHT: Gradient Boosting Histogram Transform for Density Estimation

arXiv: https://arxiv.org/abs/2106.05738 · HTML: https://mlsanity.com/paper/2106.05738/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 39 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 7 of 7)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [NGBoost: Natural Gradient Boosting for Probabilistic Prediction](https://mlsanity.com/paper/1910.03225/) (2019)
- [Anomaly Detection with Density Estimation](https://mlsanity.com/paper/2001.04990/) (2020)
- [DBSCAN++: Towards fast and scalable density clustering](https://mlsanity.com/paper/1810.13105/) (2018)
- [Accelerated Gradient Boosting](https://mlsanity.com/paper/1803.02042/) (2018)
- [Optimal Minimal Margin Maximization with Boosting](https://mlsanity.com/paper/1901.10789/) (2019)
- [Multivariate Density Estimation via Adaptive Partitioning (I): Sieve MLE](https://mlsanity.com/paper/1401.2597/) (2014)

## Built on by (most cited first, 2 of 2)

- [Mamba-based Deep Learning Approach for Sleep Staging on a Wireless Multimodal Wearable System without Electroencephalography](https://mlsanity.com/paper/2412.15947/) (2024)
- [Local Adaptivity of Gradient Boosting in Histogram Transform Ensemble Learning](https://mlsanity.com/paper/2112.02589/) (2021)

## Abstract (arXiv)

In this paper, we propose a density estimation algorithm called textitGradient Boosting Histogram Transform (GBHT), where we adopt the textitNegative Log Likelihood as the loss function to make the boosting procedure available for the unsupervised tasks. From a learning theory viewpoint, we first prove fast convergence rates for GBHT with the smoothness assumption that the underlying density function lies in the space C0,α. Then when the target density function lies in spaces C1,α, we present an upper bound for GBHT which is smaller than the lower bound of its corresponding base learner, in the sense of convergence rates. To the best of our knowledge, we make the first attempt to theoretically explain why boosting can enhance the performance of its base learners for density estimation problems. In experiments, we not only conduct performance comparisons with the widely used KDE, but also apply GBHT to anomaly detection to showcase a further application of GBHT.