# Field-aware Calibration: A Simple and Empirically Strong Method for Reliable Probabilistic Predictions

arXiv: https://arxiv.org/abs/1905.10713 · HTML: https://mlsanity.com/paper/1905.10713/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 May 2019.

## Built on by (most cited first, 19 of 19)

- [Click-Through Rate Prediction in Online Advertising: A Literature Review](https://mlsanity.com/paper/2202.10462/) (2022)
- [Joint Optimization of Ranking and Calibration with Contextualized Hybrid Model](https://mlsanity.com/paper/2208.06164/) (2022)
- [Posterior Probability Matters: Doubly-Adaptive Calibration for Neural Predictions in Online Advertising](https://mlsanity.com/paper/2205.07295/) (2022)
- [FedAds: A Benchmark for Privacy-Preserving CVR Estimation with Vertical Federated Learning](https://mlsanity.com/paper/2305.08328/) (2023)
- [Follow the Prophet: Accurate Online Conversion Rate Prediction in the Face of Delayed Feedback](https://mlsanity.com/paper/2108.06167/) (2021)
- [Deep Ensemble Shape Calibration: Multi-Field Post-hoc Calibration in Online Advertising](https://mlsanity.com/paper/2401.09507/) (2024)
- [Confidence-Aware Multi-Field Model Calibration](https://mlsanity.com/paper/2402.17655/) (2024)
- [Variable-Based Calibration for Machine Learning Classifiers](https://mlsanity.com/paper/2209.15154/) (2022)
- [A Self-boosted Framework for Calibrated Ranking](https://mlsanity.com/paper/2406.08010/) (2024)
- [MCNet: Monotonic Calibration Networks for Expressive Uncertainty Calibration in Online Advertising](https://mlsanity.com/paper/2503.00334/) (2025)
- [Exploring the Potential of Bilevel Optimization for Calibrating Neural Networks](https://mlsanity.com/paper/2503.13113/) (2025)
- [A Unified Framework for Campaign Performance Forecasting in Online Display Advertising](https://mlsanity.com/paper/2202.11877/) (2022)
- [Dual-Branch Temperature Scaling Calibration for Long-Tailed Recognition](https://mlsanity.com/paper/2308.08366/) (2023)
- [GreenFlow: A Computation Allocation Framework for Building Environmentally Sound Recommendation System](https://mlsanity.com/paper/2312.16176/) (2023)
- [Auto-bidding under Return-on-Spend Constraints with Uncertainty Quantification](https://mlsanity.com/paper/2509.16324/) (2025)
- [MLPlatt: Simple Calibration Framework for Ranking Models](https://mlsanity.com/paper/2601.08345/) (2026)
- [Follow the TRACE: Exploiting Post-Click Trajectories for Online Delayed Conversion Rate Prediction](https://mlsanity.com/paper/2604.23197/) (2026)
- [The Manokhin Probability Matrix: A Diagnostic Framework for Classifier Probability Quality](https://mlsanity.com/paper/2605.03816/) (2026)
- [Recovering Stranded Discrimination in Knowledge Tracing: Per-Item Bias Correction via Empirical-Bayes Shrinkage](https://mlsanity.com/paper/2606.14123/) (2026)

## Abstract (arXiv)

It is often observed that the probabilistic predictions given by a machine learning model can disagree with averaged actual outcomes on specific subsets of data, which is also known as the issue of miscalibration. It is responsible for the unreliability of practical machine learning systems. For example, in online advertising, an ad can receive a click-through rate prediction of 0.1 over some population of users where its actual click rate is 0.15. In such cases, the probabilistic predictions have to be fixed before the system can be deployed. In this paper, we first introduce a new evaluation metric named field-level calibration error that measures the bias in predictions over the sensitive input field that the decision-maker concerns. We show that existing post-hoc calibration methods have limited improvements in the new field-level metric and other non-calibration metrics such as the AUC score. To this end, we propose Neural Calibration, a simple yet powerful post-hoc calibration method that learns to calibrate by making full use of the field-aware information over the validation set. We present extensive experiments on five large-scale datasets. The results showed that Neural Calibration significantly improves against uncalibrated predictions in common metrics such as the negative log-likelihood, Brier score and AUC, as well as the proposed field-level calibration error.