mlsanityData to 7 Oct 2026

Private Federated Multiclass Post-hoc Calibration

arXiv 2510.01987 ↗First posted 2 Oct 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Calibrating machine learning models so that predicted probabilities better reflect the true outcome frequencies is crucial for reliable decision-making across many applications. In Federated Learning (FL), the goal is to train a global model on data which is distributed across multiple clients and cannot be centralized due to privacy concerns. FL is applied in key areas such as healthcare and finance where calibration is strongly required, yet federated private calibration has been largely overlooked. This work introduces the integration of post-hoc model calibration techniques within FL. Specifically, we transfer traditional centralized calibration methods such as histogram binning and temperature scaling into federated environments and define new methods to operate them under strong client heterogeneity. We study (1) a federated setting and (2) a user-level Differential Privacy (DP) setting and demonstrate how both federation and DP impacts calibration accuracy. We propose strategies to mitigate degradation commonly observed under heterogeneity and our findings highlight that our federated temperature scaling works best for DP-FL whereas our weighted binning approach is best when DP is not required.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 53 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

32 of 53 references.

References · 32

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2016Deep Learning with Differential Privacycited by 2,682
  4. 2018LEAF: A Benchmark for Federated Settingscited by 780
  5. 2017Renyi Differential Privacycited by 643
  6. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  7. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  8. 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 359
  9. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  10. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  11. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  12. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
20 more
  1. 2021Federated Learning for Open Bankingcited by 138
  2. 2020The Discrete Gaussian for Differential Privacycited by 134
  3. 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Modelscited by 106
  4. 2019SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overheadcited by 82
  5. 2021Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibrationcited by 79
  6. 2019MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Samplingcited by 67
  7. 2020Intra Order-preserving Functions for Calibration of Multi-Class Neural Networkscited by 65
  8. 2023Smooth ECE: Principled Reliability Diagrams via Kernel Smoothingcited by 59
  9. 2022TAN Without a Burn: Scaling Laws of DP-SGDcited by 51
  10. 2021Top-label calibration and multiclass-to-binary reductionscited by 46
  11. 2021On the Convergence and Calibration of Deep Learning with Differential Privacycited by 36
  12. 2023Privacy and Fairness in Federated Learning: on the Perspective of Trade-offcited by 26
  13. 2023Calibration by Distribution Matching: Trainable Kernel Calibration Metricscited by 20
  14. 2022Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devicescited by 16
  15. 2023Scaling of Class-wise Training Losses for Post-hoc Calibrationcited by 7
  16. 2024FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scalercited by 6
  17. 2022Federated Calibration and Evaluation of Binary Classifierscited by 5
  18. 2024Improving Local Training in Federated Learning via Temperature Scalingcited by 5
  19. 2022Algorithms for bounding contribution for histogram estimation under user-level privacycited by 4
  20. 2024Unlocking the Potential of Model Calibration in Federated Learningcited by 3

Built on by

Not cited yet.