# "Calibeating": Beating Forecasters at Their Own Game

arXiv: https://arxiv.org/abs/2209.04892 · HTML: https://mlsanity.com/paper/2209.04892/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 31 references that are in the map.
- First posted on arXiv 11 Sep 2022.

## Builds on (most cited first, 1 of 1)

- [Relative Loss Bounds for On-line Density Estimation with the Exponential Family of Distributions](https://mlsanity.com/paper/1301.6677/) (2013)

## Built on by (most cited first, 14 of 14)

- [U-Calibration: Forecasting for an Unknown Agent](https://mlsanity.com/paper/2307.00168/) (2023)
- [High-Dimensional Prediction for Sequential Decision Making](https://mlsanity.com/paper/2310.17651/) (2023)
- [Classifier Calibration with ROC-Regularized Isotonic Regression](https://mlsanity.com/paper/2311.12436/) (2023)
- [Calibrated Forecasts: The Minimax Proof](https://mlsanity.com/paper/2209.05863/) (2022)
- [Faster Recalibration of an Online Predictor via Approachability](https://mlsanity.com/paper/2310.17002/) (2023)
- [Simultaneous Swap Regret Minimization via KL-Calibration](https://mlsanity.com/paper/2502.16387/) (2025)
- [Online Platt Scaling with Calibeating](https://mlsanity.com/paper/2305.00070/) (2023)
- [Efficient Swap Multicalibration of Elicitable Properties](https://mlsanity.com/paper/2511.04907/) (2025)
- [Calibeating Made Simple](https://mlsanity.com/paper/2603.22167/) (2026)
- [Calibrated Probabilistic Forecasts for Arbitrary Sequences](https://mlsanity.com/paper/2409.19157/) (2024)
- [Improved Bounds for Swap Multicalibration and Swap Omniprediction](https://mlsanity.com/paper/2505.20885/) (2025)
- [Proper Calibeating](https://mlsanity.com/paper/2605.26703/) (2026)
- [Optimal Recalibration of an Online Predictor](https://mlsanity.com/paper/2607.19689/) (2026)
- [Calibeating for general proper losses: A Bregman divergence approach](https://mlsanity.com/paper/2605.17269/) (2026)

## Abstract (arXiv)

In order to identify expertise, forecasters should not be tested by their calibration score, which can always be made arbitrarily small, but rather by their Brier score. The Brier score is the sum of the calibration score and the refinement score; the latter measures how good the sorting into bins with the same forecast is, and thus attests to "expertise." This raises the question of whether one can gain calibration without losing expertise, which we refer to as "calibeating." We provide an easy way to calibeat any forecast, by a deterministic online procedure. We moreover show that calibeating can be achieved by a stochastic procedure that is itself calibrated, and then extend the results to simultaneously calibeating multiple procedures, and to deterministic procedures that are continuously calibrated.