mlsanityData to 7 Oct 2026

Engression: Extrapolation through the Lens of Distributional Regression

arXiv 2307.00835 ↗First posted 3 Jul 2023stat.ME cs.LG stat.MLcited by 49

Not on the map. Why?

Abstract (arXiv)

Distributional regression aims to estimate the full conditional distribution of a target variable, given covariates. Popular methods include linear and tree-ensemble based quantile regression. We propose a neural network-based distributional regression methodology called `engression'. An engression model is generative in the sense that we can sample from the fitted conditional distribution and is also suitable for high-dimensional outcomes. Furthermore, we find that modelling the conditional distribution on training data can constrain the fitted function outside of the training support, which offers a new perspective to the challenging extrapolation problem in nonlinear regression. In particular, for `pre-additive noise' models, where noise is added to the covariates before applying a nonlinear transformation, we show that engression can successfully perform extrapolation under some assumptions such as monotonicity, whereas traditional regression approaches such as least-squares or quantile regression fall short under the same assumptions. Our empirical results, from both simulated and real data, validate the effectiveness of the engression method and indicate that the pre-additive noise model is typically suitable for many real-world scenarios. The software implementations of engression are available in both R and Python.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 4 of its 55 references that are in the map. First posted on arXiv 3 Jul 2023.

Builds on

4 of 55 references.

References · 4

  1. 2014Conditional Generative Adversarial Netscited by 2,152
  2. 2019Invariant Risk Minimizationcited by 1,832
  3. 2017The Cramer Distance as a Solution to Biased Wasserstein Gradientscited by 116
  4. 2022Asymptotic Statistical Analysis of f-divergence GANcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 23; 2026: 21.

49 papers.

Sort:

By area:

  1. 2025Distributional Diffusion Models with Scoring Rulescited by 25
  2. 2025Proper scoring rules for estimation and forecast evaluationcited by 18
  3. 2024Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Eventscited by 14
  4. 2025Numerical models outperform AI weather forecasts of record-breaking extremescited by 14
  5. 2024Distributional Principal Autoencoderscited by 10
  6. 2025EnScale: Temporally-consistent multivariate generative downscaling via proper scoring rulescited by 10
  7. 2025Reverse Markov Learning: Multi-Step Generative Models for Complex Distributionscited by 7
  8. 2024A Gentle Introduction and Tutorial on Deep Generative Models in Transportation Researchcited by 4
  9. 2026Deep Generative Spatiotemporal Engression for Probabilistic Forecasting of Epidemicscited by 4
  10. 2025Distributional Autoencoders Know the Scorecited by 3
  11. 2026Theoretical Analysis of Engression and Reverse Markov Engressioncited by 3
  12. 2026Error Analysis of Neural-Network-Based Engressioncited by 3
37 more
  1. 2025Distributional Instrumental Variable Methodcited by 2
  2. 2025Kernel Density Machinescited by 2
  3. 2025Generative Machine Learning for Multivariate Angular Simulationcited by 2
  4. 2025Frugal, Flexible, Faithful: Causal Data Simulation via Frengressioncited by 2
  5. 2025Improved probabilistic regression using diffusion modelscited by 2
  6. 2025Uncertainty Quantification for Regression: A Unified Framework based on kernel scorescited by 2
  7. 2025Theoretical guarantees for neural estimators in parametric statisticscited by 1
  8. 2025Energy-Tweedie: Score meets Score, Energy meets Energycited by 1
  9. 2026Transfer Learning Through Conditional Quantile Matchingcited by 1
  10. 2026Out-of-Support Generalisation via Weight-Space Sequence Modellingcited by 1
  11. 2026Extrapolation in Statistical Learning with Extreme Value Theorycited by 1
  12. 2026Perturbation is All You Need for Extrapolating Language Modelscited by 1
  13. 2026Distributionally Robust Transfer Learning with Structurally Missing Covariates, with Application to Cross-National Cardiac Arrest Predictioncited by 1
  14. 2024Modeling Spatial Extremal Dependence of Precipitation Using Distributional Neural Networkscited by 0
  15. 2024Testing Generalizability in Causal Inferencecited by 0
  16. 2025Efficient distributional regression trees learning algorithms for calibrated non-parametric probabilistic forecastscited by 0
  17. 2025Likelihood-Free Variational Autoencoderscited by 0
  18. 2025Extrapolation Guarantees for Perturbation Modeling Under the Additive Latent Shift Assumptioncited by 0
  19. 2025Risk Bounds For Distributional Regressioncited by 0
  20. 2025Generative Regression with IQ-BARTcited by 0
  21. 2025On computing and the complexity of computing higher-order U-statistics, exactlycited by 0
  22. 2025Regularizing Extrapolation in Causal Inferencecited by 0
  23. 2025Bilateral Distribution Compression: Reducing Both Data Size and Dimensionalitycited by 0
  24. 2025Neural Posterior Estimation for Cataloging Astronomical Images from the Legacy Survey of Space and Timecited by 0
  25. 2026Distributional Deep Learning for Super-Resolution of 4D Flow MRI under Domain Shiftcited by 0
  26. 2026Deep Generative Transformers for Probabilistic Time Series and Spatiotemporal Forecastingcited by 0
  27. 2026LAtent Phase Inference from Short time sequences using SHallow REcurrent Decoders (LAPIS-SHRED)cited by 0
  28. 2026SEDGE: Structural Extrapolated Data Generationcited by 0
  29. 2026Identifying Causal Effects Using a Single Proxy Variablecited by 0
  30. 2026Stable Localized Conformal Prediction via Transductioncited by 0
  31. 2026Distributional Causal Mediation via Conditional Generative Modelingcited by 0
  32. 2026Learning to Extrapolate to New Tasks: A Relational Approach to Task Extrapolationcited by 0
  33. 2026Proper Scoring Rules for Right-Censored Survival Datacited by 0
  34. 2026Distributional Extrapolation for Interactionscited by 0
  35. 2026Belted Engression: Sufficient Dimension Reduction for Generative Distributional Regressioncited by 0
  36. 2026Multi-Task Learning of Conditional Mean Operators: applications to dynamical systems and uncertainty quantificationcited by 0
  37. 2026Generalized Engression Modelscited by 0