# deepregression: a Flexible Neural Network Framework for Semi-Structured Deep Distributional Regression

arXiv: https://arxiv.org/abs/2104.02705 · HTML: https://mlsanity.com/paper/2104.02705/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 46 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 10 of 10)

- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Bayesian Layers: A Module for Neural Network Uncertainty](https://mlsanity.com/paper/1812.03973/) (2018)
- [Deep Conditional Transformation Models](https://mlsanity.com/paper/2010.07860/) (2020)
- [Semi-Structured Deep Piecewise Exponential Models](https://mlsanity.com/paper/2011.05824/) (2020)
- [Combining Graph Neural Networks and Spatio-temporal Disease Models to Predict COVID-19 Cases in Germany](https://mlsanity.com/paper/2101.00661/) (2021)
- [Neural Mixture Distributional Regression](https://mlsanity.com/paper/2010.06889/) (2020)
- [bamlss: A Lego Toolbox for Flexible Bayesian Regression (and Beyond)](https://mlsanity.com/paper/1909.11784/) (2019)

## Built on by (most cited first, 13 of 13)

- [A review of predictive uncertainty estimation with machine learning](https://mlsanity.com/paper/2209.08307/) (2022)
- [DeepPAMM: Deep Piecewise Exponential Additive Mixed Models for Complex Hazard Structures in Survival Analysis](https://mlsanity.com/paper/2202.07423/) (2022)
- [M5 Competition Uncertainty: Overdispersion, distributional forecasting, GAMLSS and beyond](https://mlsanity.com/paper/2107.06675/) (2021)
- [A New PHO-rmula for Improved Performance of Semi-Structured Networks](https://mlsanity.com/paper/2306.00522/) (2023)
- [Probabilistic Time Series Forecasts with Autoregressive Transformation Models](https://mlsanity.com/paper/2110.08248/) (2021)
- [Uncertainty-aware predictive modeling for fair data-driven decisions](https://mlsanity.com/paper/2211.02730/) (2022)
- [Scalable Higher-Order Tensor Product Spline Models](https://mlsanity.com/paper/2402.01090/) (2024)
- [Comparison of generalised additive models and neural networks in applications: A systematic review](https://mlsanity.com/paper/2510.24601/) (2025)
- [Kernel Density Machines](https://mlsanity.com/paper/2504.21419/) (2025)
- [Additive Higher-Order Factorization Machines](https://mlsanity.com/paper/2205.14515/) (2022)
- [Semi-structured multi-state delinquency model for mortgage default](https://mlsanity.com/paper/2603.26309/) (2026)
- [neuralGAM: An R Package for Fitting Generalized Additive Neural Networks](https://mlsanity.com/paper/2505.08610/) (2025)
- [LiD-GLM: Lipschitz-constrained Deep Generalized Linear Models](https://mlsanity.com/paper/2608.16340/) (2026)

## Abstract (arXiv)

In this paper we describe the implementation of semi-structured deep distributional regression, a flexible framework to learn conditional distributions based on the combination of additive regression models and deep networks. Our implementation encompasses (1) a modular neural network building system based on the deep learning library pkgTensorFlow for the fusion of various statistical and deep learning approaches, (2) an orthogonalization cell to allow for an interpretable combination of different subnetworks, as well as (3) pre-processing steps necessary to set up such models. The software package allows to define models in a user-friendly manner via a formula interface that is inspired by classical statistical model frameworks such as pkgmgcv. The packages' modular design and functionality provides a unique resource for both scalable estimation of complex statistical models and the combination of approaches from deep learning and statistics. This allows for state-of-the-art predictive performance while simultaneously retaining the indispensable interpretability of classical statistical models.