# Deep transformation models: Tackling complex regression problems with neural network based transformation models

arXiv: https://arxiv.org/abs/2004.00464 · HTML: https://mlsanity.com/paper/2004.00464/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2020.

## Built on by (most cited first, 18 of 18)

- [A New PHO-rmula for Improved Performance of Semi-Structured Networks](https://mlsanity.com/paper/2306.00522/) (2023)
- [Probabilistic Time Series Forecasts with Autoregressive Transformation Models](https://mlsanity.com/paper/2110.08248/) (2021)
- [Distributional Gradient Boosting Machines](https://mlsanity.com/paper/2204.00778/) (2022)
- [Short-Term Density Forecasting of Low-Voltage Load using Bernstein-Polynomial Normalizing Flows](https://mlsanity.com/paper/2204.13939/) (2022)
- [Deep interpretable ensembles](https://mlsanity.com/paper/2205.12729/) (2022)
- [Interpretable Neural Causal Models with TRAM-DAGs](https://mlsanity.com/paper/2503.16206/) (2025)
- [Bernstein Flows for Flexible Posteriors in Variational Bayes](https://mlsanity.com/paper/2202.05650/) (2022)
- [Sample-based Uncertainty Quantification with a Single Deterministic Neural Network](https://mlsanity.com/paper/2209.08418/) (2022)
- [Deep conditional transformation models for survival analysis](https://mlsanity.com/paper/2210.11366/) (2022)
- [Bayesian Semi-structured Subspace Inference](https://mlsanity.com/paper/2401.12950/) (2024)
- [Uncertainty estimation under model misspecification in neural network regression](https://mlsanity.com/paper/2111.11763/) (2021)
- [How Inverse Conditional Flows Can Serve as a Substitute for Distributional Regression](https://mlsanity.com/paper/2405.05429/) (2024)
- [An interpretable neural network-based non-proportional odds model for ordinal regression](https://mlsanity.com/paper/2303.17823/) (2023)
- [Transformation Models for Flexible Posteriors in Variational Bayes](https://mlsanity.com/paper/2106.00528/) (2021)
- [Single-shot Bayesian approximation for neural networks](https://mlsanity.com/paper/2308.12785/) (2023)
- [Hybrid Bernstein Normalizing Flows for Flexible Multivariate Density Regression with Interpretable Marginals](https://mlsanity.com/paper/2505.14164/) (2025)
- [Recent Advances in Simulation-based Inference for Gravitational Wave Data Analysis](https://mlsanity.com/paper/2507.11192/) (2025)
- [Conditional Rank-Rank Regression via Deep Conditional Transformation Models](https://mlsanity.com/paper/2603.07230/) (2026)

## Abstract (arXiv)

We present a deep transformation model for probabilistic regression. Deep learning is known for outstandingly accurate predictions on complex data but in regression tasks, it is predominantly used to just predict a single number. This ignores the non-deterministic character of most tasks. Especially if crucial decisions are based on the predictions, like in medical applications, it is essential to quantify the prediction uncertainty. The presented deep learning transformation model estimates the whole conditional probability distribution, which is the most thorough way to capture uncertainty about the outcome. We combine ideas from a statistical transformation model (most likely transformation) with recent transformation models from deep learning (normalizing flows) to predict complex outcome distributions. The core of the method is a parameterized transformation function which can be trained with the usual maximum likelihood framework using gradient descent. The method can be combined with existing deep learning architectures. For small machine learning benchmark datasets, we report state of the art performance for most dataset and partly even outperform it. Our method works for complex input data, which we demonstrate by employing a CNN architecture on image data.