mlsanityData to 7 Oct 2026

Calibration Error for Decision Making

arXiv 2404.13503 ↗First posted 21 Apr 2024cs.LG cs.DS stat.MLcited by 11

Not on the map. Why?

Abstract (arXiv)

Calibration allows predictions to be reliably interpreted as probabilities by decision makers. We propose a decision-theoretic calibration error, the Calibration Decision Loss (CDL), defined as the maximum improvement in decision payoff obtained by calibrating the predictions, where the maximum is over all payoff-bounded decision tasks. Vanishing CDL guarantees the payoff loss from miscalibration vanishes simultaneously for all downstream decision tasks. We show separations between CDL and existing calibration error metrics, including the most well-studied metric Expected Calibration Error (ECE). Our main technical contribution is a new efficient algorithm for online calibration that achieves near-optimal O(fraclog T√T) expected CDL, bypassing the Omega(T-0.472) lower bound for ECE by Qiao and Valiant (2021).

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 21 of its 46 references that are in the map. First posted on arXiv 21 Apr 2024.

Builds on

21 of 46 references.

References · 21

  1. 2018Multiaccuracy: Black-Box Post-Processing for Fairness in Classificationcited by 221
  2. 2021Omnipredictorscited by 71
  3. 2022Loss Minimization through the Lens of Outcome Indistinguishabilitycited by 61
  4. 2021Impossible Tuning Made Possible: A New Expert Algorithm and Its Applicationscited by 52
  5. 2021Near-Optimal No-Regret Learning for Correlated Equilibria in Multi-Player General-Sum Gamescited by 43
  6. 2020Stronger Calibration Lower Bounds via Sidesteppingcited by 38
  7. 2023Oracle Efficient Online Multicalibration and Omnipredictioncited by 36
  8. 2023When Does Optimizing a Proper Loss Yield Calibration?cited by 35
  9. 2023Swap Agnostic Learning, or Characterizing Omniprediction via Multicalibrationcited by 31
  10. 2023Fast swap regret minimization and applications to approximate correlated equilibriacited by 30
  11. 2020Optimization of Scoring Rulescited by 28
  12. 2024Forecasting for Swap Regret for All Downstream Agentscited by 24
9 more
  1. 2022Forecast Hedging and Calibrationcited by 22
  2. 2023Loss Minimization Yields Multicalibration for Large Neural Networkscited by 19
  3. 2024On the Distance from Calibration in Sequential Predictioncited by 18
  4. 2022Calibrated Forecasts: The Minimax Proofcited by 17
  5. 2022Omnipredictors for Constrained Optimizationcited by 14
  6. 2024Omnipredictors for Regression and the Approximate Rank of Convex Functionscited by 14
  7. 2020Binary Scoring Rules that Incentivize Precisioncited by 12
  8. 2022Optimal Scoring Rules for Multi-dimensional Effortcited by 12
  9. 2023From External to Swap Regret 2.0: An Efficient Reduction and Oblivious Adversary for Large Action Spacescited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 4.

11 papers.

  1. 2025Full Swap Regret and Discretized Calibrationcited by 13
  2. 2025Can a calibration metric be both testable and actionable?cited by 10
  3. 2024Truthfulness of Calibration Measurescited by 7
  4. 2025Efficient Calibration for Decision Makingcited by 3
  5. 2025Forecast Skill Is Not Decision Skill: Evidence from Weather-Dependent Decision Taskscited by 3
  6. 2025Dimension-Free Decision Calibration for Nonlinear Loss Functionscited by 2
  7. 2026The Importance of Being Smoothly Calibratedcited by 2
  8. 2025Explanations are a Means to an End: Decision Theoretic Explanation Evaluationcited by 1
  9. 2026Robust Learning with Optimal Errorcited by 1
  10. 2026Toward Simultaneously Optimal Regret in U-Calibrationcited by 1
  11. 2026Task-Aware Calibration: Provably Optimal Decoding in LLMscited by 0