# Deep interpretable ensembles

arXiv: https://arxiv.org/abs/2205.12729 · HTML: https://mlsanity.com/paper/2205.12729/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 70 references that are in the map.
- First posted on arXiv 25 May 2022.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [Deep Ensembles Work, But Are They Necessary?](https://mlsanity.com/paper/2202.06985/) (2022)
- [The Relative Performance of Ensemble Methods with Deep Convolutional Neural Networks for Image Classification](https://mlsanity.com/paper/1704.01664/) (2017)
- [Deep Ensembles from a Bayesian Perspective](https://mlsanity.com/paper/2105.13283/) (2021)
- [Deep Conditional Transformation Models](https://mlsanity.com/paper/2010.07860/) (2020)
- [CRPS Learning](https://mlsanity.com/paper/2102.00968/) (2021)
- [Most Likely Transformations](https://mlsanity.com/paper/1508.06749/) (2015)
- [Deep transformation models: Tackling complex regression problems with neural network based transformation models](https://mlsanity.com/paper/2004.00464/) (2020)
- [Probabilistic Time Series Forecasts with Autoregressive Transformation Models](https://mlsanity.com/paper/2110.08248/) (2021)
- [From Proper Scoring Rules to Max-Min Optimal Forecast Aggregation](https://mlsanity.com/paper/2102.07081/) (2021)
- [Confident Neural Network Regression with Bootstrapped Deep Ensembles](https://mlsanity.com/paper/2202.10903/) (2022)

## Built on by (most cited first, 5 of 5)

- [Bayesian posterior approximation with stochastic ensembles](https://mlsanity.com/paper/2212.08123/) (2022)
- [Bayesian Semi-structured Subspace Inference](https://mlsanity.com/paper/2401.12950/) (2024)
- [Explainability in mulimodal deep transformation models for stroke outcome prediction](https://mlsanity.com/paper/2504.06299/) (2025)
- [Bridging Neural Networks and Dynamic Time Warping for Adaptive Time Series Classification](https://mlsanity.com/paper/2507.09826/) (2025)
- [Conditional Rank-Rank Regression via Deep Conditional Transformation Models](https://mlsanity.com/paper/2603.07230/) (2026)

## Abstract (arXiv)

Ensembles improve prediction performance and allow uncertainty quantification by aggregating predictions from multiple models. In deep ensembling, the individual models are usually black box neural networks, or recently, partially interpretable semi-structured deep transformation models. However, interpretability of the ensemble members is generally lost upon aggregation. This is a crucial drawback of deep ensembles in high-stake decision fields, in which interpretable models are desired. We propose a novel transformation ensemble which aggregates probabilistic predictions with the guarantee to preserve interpretability and yield uniformly better predictions than the ensemble members on average. Transformation ensembles are tailored towards interpretable deep transformation models but are applicable to a wider range of probabilistic neural networks. In experiments on several publicly available data sets, we demonstrate that transformation ensembles perform on par with classical deep ensembles in terms of prediction performance, discrimination, and calibration. In addition, we demonstrate how transformation ensembles quantify both aleatoric and epistemic uncertainty, and produce minimax optimal predictions under certain conditions.