# Calibrated Probabilistic Forecasts for Arbitrary Sequences

arXiv: https://arxiv.org/abs/2409.19157 · HTML: https://mlsanity.com/paper/2409.19157/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 54 references that are in the map.
- First posted on arXiv 27 Sep 2024.

## Builds on (most cited first, 25 of 25)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [River: machine learning for streaming data in Python](https://mlsanity.com/paper/2012.04740/) (2020)
- [Runaway Feedback Loops in Predictive Policing](https://mlsanity.com/paper/1706.09847/) (2017)
- [Distribution Calibration for Regression](https://mlsanity.com/paper/1905.06023/) (2019)
- [Conformal PID Control for Time Series Prediction](https://mlsanity.com/paper/2307.16895/) (2023)
- [Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration](https://mlsanity.com/paper/2107.05719/) (2021)
- [Practical Adversarial Multivalid Conformal Prediction](https://mlsanity.com/paper/2206.01067/) (2022)
- [Individual Calibration with Randomized Forecasting](https://mlsanity.com/paper/2006.10288/) (2020)
- [Online Multivalid Learning: Means, Moments, and Prediction Intervals](https://mlsanity.com/paper/2101.01739/) (2021)
- [Oracle Efficient Online Multicalibration and Omniprediction](https://mlsanity.com/paper/2307.08999/) (2023)
- [Calibrated Model-Based Deep Reinforcement Learning](https://mlsanity.com/paper/1906.08312/) (2019)
- [Online Minimax Multiobjective Optimization: Multicalibeating and Other Applications](https://mlsanity.com/paper/2108.03837/) (2021)
- [Local Calibration: Metrics and Recalibration](https://mlsanity.com/paper/2102.10809/) (2021)
- [Calibration tests beyond classification](https://mlsanity.com/paper/2210.13355/) (2022)
- [Calibration by Distribution Matching: Trainable Kernel Calibration Metrics](https://mlsanity.com/paper/2310.20211/) (2023)
- [Approachability, Regret and Calibration; implications and equivalences](https://mlsanity.com/paper/1301.2663/) (2013)
- ["Calibeating": Beating Forecasters at Their Own Game](https://mlsanity.com/paper/2209.04892/) (2022)
- [Stochastic Gradient Trees](https://mlsanity.com/paper/1901.07777/) (2019)
- [Faster Recalibration of an Online Predictor via Approachability](https://mlsanity.com/paper/2310.17002/) (2023)
- [Online Calibrated and Conformal Prediction Improves Bayesian Optimization](https://mlsanity.com/paper/2112.04620/) (2021)
- [Faster online calibration without randomization: interval forecasts and the power of two choices](https://mlsanity.com/paper/2204.13087/) (2022)

## Built on by (most cited first, 4 of 4)

- [Calibeating Made Simple](https://mlsanity.com/paper/2603.22167/) (2026)
- [Calibrated Multi-Level Quantile Forecasting](https://mlsanity.com/paper/2512.23671/) (2025)
- [A Monte Carlo Framework for Calibrated Uncertainty Estimation in Sequence Prediction](https://mlsanity.com/paper/2410.23272/) (2024)
- [Proper Calibeating](https://mlsanity.com/paper/2605.26703/) (2026)

## Abstract (arXiv)

Real-world data streams can change unpredictably due to distribution shifts, feedback loops and adversarial actors, which challenges the validity of forecasts. We present a forecasting framework ensuring valid uncertainty estimates regardless of how data evolves. Leveraging the concept of Blackwell approachability from game theory, we introduce a forecasting framework that guarantees calibrated uncertainties for outcomes in any compact space (e.g., classification or bounded regression). We extend this framework to recalibrate existing forecasters, guaranteeing calibration without sacrificing predictive performance. We implement both general-purpose gradient-based algorithms and algorithms optimized for popular special cases of our framework. Empirically, our algorithms improve calibration and downstream decision-making for energy systems.