# Engression: Extrapolation through the Lens of Distributional Regression

arXiv: https://arxiv.org/abs/2307.00835 · HTML: https://mlsanity.com/paper/2307.00835/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 55 references that are in the map.
- First posted on arXiv 3 Jul 2023.

## Builds on (most cited first, 4 of 4)

- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [The Cramer Distance as a Solution to Biased Wasserstein Gradients](https://mlsanity.com/paper/1705.10743/) (2017)
- [Asymptotic Statistical Analysis of f-divergence GAN](https://mlsanity.com/paper/2209.06853/) (2022)

## Built on by (most cited first, 30 of 51)

- [Distributional Diffusion Models with Scoring Rules](https://mlsanity.com/paper/2502.02483/) (2025)
- [Proper scoring rules for estimation and forecast evaluation](https://mlsanity.com/paper/2504.01781/) (2025)
- [Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events](https://mlsanity.com/paper/2404.17652/) (2024)
- [Numerical models outperform AI weather forecasts of record-breaking extremes](https://mlsanity.com/paper/2508.15724/) (2025)
- [Distributional Principal Autoencoders](https://mlsanity.com/paper/2404.13649/) (2024)
- [EnScale: Temporally-consistent multivariate generative downscaling via proper scoring rules](https://mlsanity.com/paper/2509.26258/) (2025)
- [Reverse Markov Learning: Multi-Step Generative Models for Complex Distributions](https://mlsanity.com/paper/2502.13747/) (2025)
- [A Gentle Introduction and Tutorial on Deep Generative Models in Transportation Research](https://mlsanity.com/paper/2410.07066/) (2024)
- [Deep Generative Spatiotemporal Engression for Probabilistic Forecasting of Epidemics](https://mlsanity.com/paper/2603.07108/) (2026)
- [Theoretical Analysis of Engression and Reverse Markov Engression](https://mlsanity.com/paper/2606.01002/) (2026)
- [Error Analysis of Neural-Network-Based Engression](https://mlsanity.com/paper/2607.27723/) (2026)
- [Distributional Autoencoders Know the Score](https://mlsanity.com/paper/2502.11583/) (2025)
- [Generative Machine Learning for Multivariate Angular Simulation](https://mlsanity.com/paper/2504.21505/) (2025)
- [Distributional Instrumental Variable Method](https://mlsanity.com/paper/2502.07641/) (2025)
- [Kernel Density Machines](https://mlsanity.com/paper/2504.21419/) (2025)
- [Theoretical guarantees for neural estimators in parametric statistics](https://mlsanity.com/paper/2506.18508/) (2025)
- [Frugal, Flexible, Faithful: Causal Data Simulation via Frengression](https://mlsanity.com/paper/2508.01018/) (2025)
- [Improved probabilistic regression using diffusion models](https://mlsanity.com/paper/2510.04583/) (2025)
- [Uncertainty Quantification for Regression: A Unified Framework based on kernel scores](https://mlsanity.com/paper/2510.25599/) (2025)
- [Perturbation is All You Need for Extrapolating Language Models](https://mlsanity.com/paper/2605.04344/) (2026)
- [Efficient distributional regression trees learning algorithms for calibrated non-parametric probabilistic forecasts](https://mlsanity.com/paper/2502.05157/) (2025)
- [Regularizing Extrapolation in Causal Inference](https://mlsanity.com/paper/2509.17180/) (2025)
- [Energy-Tweedie: Score meets Score, Energy meets Energy](https://mlsanity.com/paper/2512.23818/) (2025)
- [Transfer Learning Through Conditional Quantile Matching](https://mlsanity.com/paper/2602.02358/) (2026)
- [Out-of-Support Generalisation via Weight-Space Sequence Modelling](https://mlsanity.com/paper/2602.13550/) (2026)
- [SEDGE: Structural Extrapolated Data Generation](https://mlsanity.com/paper/2604.02482/) (2026)
- [Extrapolation in Statistical Learning with Extreme Value Theory](https://mlsanity.com/paper/2605.01909/) (2026)
- [Distributionally Robust Transfer Learning with Structurally Missing Covariates, with Application to Cross-National Cardiac Arrest Prediction](https://mlsanity.com/paper/2605.24212/) (2026)
- [Belted Engression: Sufficient Dimension Reduction for Generative Distributional Regression](https://mlsanity.com/paper/2609.23789/) (2026)
- [Modeling Spatial Extremal Dependence of Precipitation Using Distributional Neural Networks](https://mlsanity.com/paper/2407.08668/) (2024)

## Abstract (arXiv)

Distributional regression aims to estimate the full conditional distribution of a target variable, given covariates. Popular methods include linear and tree-ensemble based quantile regression. We propose a neural network-based distributional regression methodology called `engression'. An engression model is generative in the sense that we can sample from the fitted conditional distribution and is also suitable for high-dimensional outcomes. Furthermore, we find that modelling the conditional distribution on training data can constrain the fitted function outside of the training support, which offers a new perspective to the challenging extrapolation problem in nonlinear regression. In particular, for `pre-additive noise' models, where noise is added to the covariates before applying a nonlinear transformation, we show that engression can successfully perform extrapolation under some assumptions such as monotonicity, whereas traditional regression approaches such as least-squares or quantile regression fall short under the same assumptions. Our empirical results, from both simulated and real data, validate the effectiveness of the engression method and indicate that the pre-additive noise model is typically suitable for many real-world scenarios. The software implementations of engression are available in both R and Python.