# Composite Quantile Regression With XGBoost Using the Novel Arctan Pinball Loss

arXiv: https://arxiv.org/abs/2406.02293 · HTML: https://mlsanity.com/paper/2406.02293/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 39 references that are in the map.
- First posted on arXiv 4 Jun 2024.

## Builds on (most cited first, 10 of 10)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [How to Evaluate Uncertainty Estimates in Machine Learning for Regression?](https://mlsanity.com/paper/2106.03395/) (2021)
- [Quantile Extreme Gradient Boosting for Uncertainty Quantification](https://mlsanity.com/paper/2304.11732/) (2023)
- [A Novel Smoothed Loss and Penalty Function for Noncrossing Composite Quantile Estimation via Deep Neural Networks](https://mlsanity.com/paper/1909.12122/) (2019)

## Built on by (most cited first, 2 of 2)

- [Convolution Smoothed Quantile Regression for XGBoost](https://mlsanity.com/paper/2608.15290/) (2026)
- [Multivariate quantile regression via Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2609.23906/) (2026)

## Abstract (arXiv)

This paper explores the use of XGBoost for composite quantile regression. XGBoost is a highly popular model renowned for its flexibility, efficiency, and capability to deal with missing data. The optimization uses a second order approximation of the loss function, complicating the use of loss functions with a zero or vanishing second derivative. Quantile regression -- a popular approach to obtain conditional quantiles when point estimates alone are insufficient -- unfortunately uses such a loss function, the pinball loss. Existing workarounds are typically inefficient and can result in severe quantile crossings. In this paper, we present a smooth approximation of the pinball loss, the arctan pinball loss, that is tailored to the needs of XGBoost. Specifically, contrary to other smooth approximations, the arctan pinball loss has a relatively large second derivative, which makes it more suitable to use in the second order approximation. Using this loss function enables the simultaneous prediction of multiple quantiles, which is more efficient and results in far fewer quantile crossings.