Unlocking the Potential of Model Calibration in Federated Learning
Not on the map. Why?
Abstract (arXiv)
Over the past several years, various federated learning (FL) methodologies have been developed to improve model accuracy, a primary performance metric in machine learning. However, to utilize FL in practical decision-making scenarios, beyond considering accuracy, the trained model must also have a reliable confidence in each of its predictions, an aspect that has been largely overlooked in existing FL research. Motivated by this gap, we propose Non-Uniform Calibration for Federated Learning (NUCFL), a generic framework that integrates FL with the concept of model calibration. The inherent data heterogeneity in FL environments makes model calibration particularly difficult, as it must ensure reliability across diverse data distributions and client conditions. Our NUCFL addresses this challenge by dynamically adjusting the model calibration objectives based on statistical relationships between each client's local model and the global model in FL. In particular, NUCFL assesses the similarity between local and global model relationships, and controls the penalty term for the calibration loss during client-side local training. By doing so, NUCFL effectively aligns calibration needs for the global model in heterogeneous FL settings while not sacrificing accuracy. Extensive experiments show that NUCFL offers flexibility and effectiveness across various FL algorithms, enhancing accuracy as well as model calibration.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 7 Sep 2024.
Builds on
32 of 49 references.
References · 32
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
- 2020Adaptive Federated Optimizationcited by 1,036
- 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
- 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
- 2018LEAF: A Benchmark for Federated Settingscited by 780
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
20 more
- 2020Federated Learning with Matched Averagingcited by 564
- 2021Federated Learning Based on Dynamic Regularizationcited by 555
- 2019Agnostic Federated Learningcited by 451
- 2019Measuring Calibration in Deep Learningcited by 443
- 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
- 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
- 2020Calibrating Deep Neural Networks using Focal Losscited by 353
- 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
- 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
- 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
- 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
- 2021The Devil is in the Margin: Margin-based Label Smoothing for Network Calibrationcited by 91
- 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
- 2022Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learningcited by 71
- 2018Learning Private Neural Language Modeling with Attentive Aggregationcited by 49
- 2022A Stitch in Time Saves Nine: A Train-Time Regularizing Loss for Improved Neural Network Calibrationcited by 49
- 2020Improved Trainable Calibration Method for Neural Networks on Medical Imaging Classificationcited by 37
- 2022Calibrating Deep Neural Networks using Explicit Regularisation and Dynamic Data Pruningcited by 18
- 2023pFedSim: Similarity-Aware Model Aggregation Towards Personalized Federated Learningcited by 9
- 2024FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scalercited by 6
Built on by
Citing papers per year: 2025: 3.
3 papers.