# Progression: an extrapolation principle for regression

arXiv: https://arxiv.org/abs/2410.23246 · HTML: https://mlsanity.com/paper/2410.23246/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 58 references that are in the map.
- First posted on arXiv 30 Oct 2024.

## Builds on (most cited first, 14 of 14)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks](https://mlsanity.com/paper/2009.11848/) (2020)
- [Invariance, Causality and Robustness](https://mlsanity.com/paper/1812.08233/) (2018)
- [Anchor regression: heterogeneous data meets causality](https://mlsanity.com/paper/1801.06229/) (2018)
- [A causal framework for distribution generalization](https://mlsanity.com/paper/2006.07433/) (2020)
- [Optimally tackling covariate shift in RKHS-based nonparametric regression](https://mlsanity.com/paper/2205.02986/) (2022)
- [First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains](https://mlsanity.com/paper/2211.11719/) (2022)
- [Neural Networks for Extreme Quantile Regression with an Application to Forecasting of Flood Risk](https://mlsanity.com/paper/2208.07590/) (2022)
- [Gradient boosting for extreme quantile regression](https://mlsanity.com/paper/2103.00808/) (2021)
- [Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events](https://mlsanity.com/paper/2404.17652/) (2024)
- [Extremal Random Forests](https://mlsanity.com/paper/2201.12865/) (2022)
- [On Regression in Extreme Regions](https://mlsanity.com/paper/2303.03084/) (2023)
- [Extrapolation-Aware Nonparametric Statistical Inference](https://mlsanity.com/paper/2402.09758/) (2024)

## Built on by (most cited first, 6 of 6)

- [Numerical models outperform AI weather forecasts of record-breaking extremes](https://mlsanity.com/paper/2508.15724/) (2025)
- [Extreme Conformal Prediction: Reliable Intervals for High-Impact Events](https://mlsanity.com/paper/2505.08578/) (2025)
- [Theoretical guarantees for neural estimators in parametric statistics](https://mlsanity.com/paper/2506.18508/) (2025)
- [Extrapolation in Statistical Learning with Extreme Value Theory](https://mlsanity.com/paper/2605.01909/) (2026)
- [Assumption-free fidelity bounds for hardware noise characterization](https://mlsanity.com/paper/2504.07010/) (2025)
- [Out-of-Distribution generalization of quantile regression with heavy tailed inputs: an SVM approach](https://mlsanity.com/paper/2606.00265/) (2026)

## Abstract (arXiv)

The problem of regression extrapolation, or out-of-distribution generalization, arises when predictions are required at test points outside the range of the training data. In such cases, the non-parametric guarantees for regression methods from both statistics and machine learning typically fail. Based on the theory of tail dependence, we propose a novel statistical extrapolation principle. After a suitable, data-adaptive marginal transformation, it assumes a simple relationship between predictors and the response at the boundary of the training predictor samples. This assumption holds for a wide range of models, including non-parametric regression functions with additive noise. Our semi-parametric method, progression, leverages this extrapolation principle and offers guarantees on the approximation error beyond the training data range. We demonstrate how this principle can be effectively integrated with existing approaches, such as random forests and additive models, to improve extrapolation performance on out-of-distribution samples.