# Calibrating the Predictions for Top-N Recommendations

arXiv: https://arxiv.org/abs/2408.11596 · HTML: https://mlsanity.com/paper/2408.11596/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 21 references that are in the map.
- First posted on arXiv 21 Aug 2024.

## Builds on (most cited first, 8 of 8)

- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems](https://mlsanity.com/paper/2202.10842/) (2022)
- [Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback](https://mlsanity.com/paper/1909.03601/) (2019)
- [Obtaining Calibrated Probabilities with Personalized Ranking Models](https://mlsanity.com/paper/2112.07428/) (2021)
- [Microsoft Recommenders: Tools to Accelerate Developing Recommender Systems](https://mlsanity.com/paper/2008.13528/) (2020)
- [Towards Confidence-aware Calibrated Recommendation](https://mlsanity.com/paper/2208.10192/) (2022)
- [Variable-Based Calibration for Machine Learning Classifiers](https://mlsanity.com/paper/2209.15154/) (2022)

## Built on by (most cited first, 2 of 2)

- [Calibrated Preference Learning: The Case of Label Ranking](https://mlsanity.com/paper/2605.30447/) (2026)
- [Odds-Shift Slippage in One-vs-Rest Rankers: Diagnosing and Repairing Reweighting-Induced Top-K Errors](https://mlsanity.com/paper/2609.13810/) (2026)

## Abstract (arXiv)

Well-calibrated predictions of user preferences are essential for many applications. Since recommender systems typically select the top-N items for users, calibration for those top-N items, rather than for all items, is important. We show that previous calibration methods result in miscalibrated predictions for the top-N items, despite their excellent calibration performance when evaluated on all items. In this work, we address the miscalibration in the top-N recommended items. We first define evaluation metrics for this objective and then propose a generic method to optimize calibration models focusing on the top-N items. It groups the top-N items by their ranks and optimizes distinct calibration models for each group with rank-dependent training weights. We verify the effectiveness of the proposed method for both explicit and implicit feedback datasets, using diverse classes of recommender models.