# Probabilistic supervised learning

arXiv: https://arxiv.org/abs/1801.00753 · HTML: https://mlsanity.com/paper/1801.00753/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jan 2018.

## Built on by (most cited first, 4 of 4)

- [Had enough of experts? Quantitative knowledge retrieval from large language models](https://mlsanity.com/paper/2402.07770/) (2024)
- [Path classification by stochastic linear recurrent neural networks](https://mlsanity.com/paper/2108.03090/) (2021)
- [Express Your Doubts -- Probabilistic World Modeling Should not be Based on Token logprobs](https://mlsanity.com/paper/2505.02072/) (2025)
- [Partition Tree: Conditional Density Estimation over General Outcome Spaces](https://mlsanity.com/paper/2602.04042/) (2026)

## Abstract (arXiv)

Predictive modelling and supervised learning are central to modern data science. With predictions from an ever-expanding number of supervised black-box strategies - e.g., kernel methods, random forests, deep learning aka neural networks - being employed as a basis for decision making processes, it is crucial to understand the statistical uncertainty associated with these predictions. As a general means to approach the issue, we present an overarching framework for black-box prediction strategies that not only predict the target but also their own predictions' uncertainty. Moreover, the framework allows for fair assessment and comparison of disparate prediction strategies. For this, we formally consider strategies capable of predicting full distributions from feature variables, so-called probabilistic supervised learning strategies. Our work draws from prior work including Bayesian statistics, information theory, and modern supervised machine learning, and in a novel synthesis leads to (a) new theoretical insights such as a probabilistic bias-variance decomposition and an entropic formulation of prediction, as well as to (b) new algorithms and meta-algorithms, such as composite prediction strategies, probabilistic boosting and bagging, and a probabilistic predictive independence test. Our black-box formulation also leads (c) to a new modular interface view on probabilistic supervised learning and a modelling workflow API design, which we have implemented in the newly released skpro machine learning toolbox, extending the familiar modelling interface and meta-modelling functionality of sklearn. The skpro package provides interfaces for construction, composition, and tuning of probabilistic supervised learning strategies, together with orchestration features for validation and comparison of any such strategy - be it frequentist, Bayesian, or other.