# Flexible Model Aggregation for Quantile Regression

arXiv: https://arxiv.org/abs/2103.00083 · HTML: https://mlsanity.com/paper/2103.00083/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 95 references that are in the map.
- First posted on arXiv 26 Feb 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Distributional Reinforcement Learning with Quantile Regression](https://mlsanity.com/paper/1710.10044/) (2017)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [Deep Learning for Time Series Forecasting: Tutorial and Literature Survey](https://mlsanity.com/paper/2004.10240/) (2020)
- [Neural networks for post-processing ensemble weather forecasts](https://mlsanity.com/paper/1805.09091/) (2018)
- [Distribution Calibration for Regression](https://mlsanity.com/paper/1905.06023/) (2019)
- [Adaptive, Distribution-Free Prediction Intervals for Deep Networks](https://mlsanity.com/paper/1905.10634/) (2019)
- [Maximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensembles](https://mlsanity.com/paper/1906.07380/) (2019)
- [Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation](https://mlsanity.com/paper/2006.14284/) (2020)
- [On the Importance of Strong Baselines in Bayesian Deep Learning](https://mlsanity.com/paper/1811.09385/) (2018)
- [Smooth Pinball Neural Network for Probabilistic Forecasting of Wind Power](https://mlsanity.com/paper/1710.01720/) (2017)
- [The Multiple Quantile Graphical Model](https://mlsanity.com/paper/1607.00515/) (2016)

## Built on by (most cited first, 22 of 22)

- [Forecast combinations: an over 50-year review](https://mlsanity.com/paper/2205.04216/) (2022)
- [A Large-Scale Study of Probabilistic Calibration in Neural Network Regression](https://mlsanity.com/paper/2306.02738/) (2023)
- [CROWDLAB: Supervised learning to infer consensus labels and quality scores for data with multiple annotators](https://mlsanity.com/paper/2210.06812/) (2022)
- [End-to-End Probabilistic Framework for Learning with Hard Constraints](https://mlsanity.com/paper/2506.07003/) (2025)
- [UTOPIA: Universally Trainable Optimal Prediction Intervals Aggregation](https://mlsanity.com/paper/2306.16549/) (2023)
- [Aggregating distribution forecasts from deep ensembles](https://mlsanity.com/paper/2204.02291/) (2022)
- [Learning to Complement with Multiple Humans](https://mlsanity.com/paper/2311.13172/) (2023)
- [On-line conformalized neural networks ensembles for probabilistic forecasting of day-ahead electricity prices](https://mlsanity.com/paper/2404.02722/) (2024)
- [Multivariate Latent Recalibration for Conditional Normalizing Flows](https://mlsanity.com/paper/2505.16636/) (2025)
- [Any-Quantile Probabilistic Forecasting of Short-Term Electricity Demand](https://mlsanity.com/paper/2404.17451/) (2024)
- [Calibrated Multi-Level Quantile Forecasting](https://mlsanity.com/paper/2512.23671/) (2025)
- [Deep sub-ensembles meets quantile regression: uncertainty-aware imputation for time series](https://mlsanity.com/paper/2312.01294/) (2023)
- [An Efficient Multi Quantile Regression Network with Ad Hoc Prevention of Quantile Crossing](https://mlsanity.com/paper/2406.00080/) (2024)
- [fastkqr: A Fast Algorithm for Kernel Quantile Regression](https://mlsanity.com/paper/2408.05393/) (2024)
- [Forking-Sequences: Statistically and Computationally Efficient Multi-Horizon Forecasting with Reduced Volatility](https://mlsanity.com/paper/2510.04487/) (2025)
- [Model selection with proper scoring rules on data sets of time series: prefer the mean scaled score](https://mlsanity.com/paper/2606.24715/) (2026)
- [Quantitative Universal Approximation Bounds for Deep Belief Networks](https://mlsanity.com/paper/2208.09033/) (2022)
- [Uncertainty separation via ensemble quantile regression](https://mlsanity.com/paper/2412.13738/) (2024)
- [The Living Forecast: Evolving Day-Ahead Predictions into Intraday Reality](https://mlsanity.com/paper/2510.12271/) (2025)
- [Multi-layer Stack Ensembles for Time Series Forecasting](https://mlsanity.com/paper/2511.15350/) (2025)
- [Trustworthy Predictive Distributions for Tail Events with Semiparametric Diagnostic Transport Maps](https://mlsanity.com/paper/2603.11229/) (2026)
- [NPBoost: Neural Processes with Gradient-Boosted Fixed Effects](https://mlsanity.com/paper/2609.28122/) (2026)

## Abstract (arXiv)

Quantile regression is a fundamental problem in statistical learning motivated by a need to quantify uncertainty in predictions, or to model a diverse population without being overly reductive. For instance, epidemiological forecasts, cost estimates, and revenue predictions all benefit from being able to quantify the range of possible values accurately. As such, many models have been developed for this problem over many years of research in statistics, machine learning, and related fields. Rather than proposing yet another (new) algorithm for quantile regression we adopt a meta viewpoint: we investigate methods for aggregating any number of conditional quantile models, in order to improve accuracy and robustness. We consider weighted ensembles where weights may vary over not only individual models, but also over quantile levels, and feature values. All of the models we consider in this paper can be fit using modern deep learning toolkits, and hence are widely accessible (from an implementation point of view) and scalable. To improve the accuracy of the predicted quantiles (or equivalently, prediction intervals), we develop tools for ensuring that quantiles remain monotonically ordered, and apply conformal calibration methods. These can be used without any modification of the original library of base models. We also review some basic theory surrounding quantile aggregation and related scoring rules, and contribute a few new results to this literature (for example, the fact that post sorting or post isotonic regression can only improve the weighted interval score). Finally, we provide an extensive suite of empirical comparisons across 34 data sets from two different benchmark repositories.