Federated Learning with Label Distribution Skew via Logits Calibration
Not on the map. Why?
Abstract (arXiv)
Traditional federated optimization methods perform poorly with heterogeneous data (ie, accuracy reduction), especially for highly skewed data. In this paper, we investigate the label distribution skew in FL, where the distribution of labels varies across clients. First, we investigate the label distribution skew from a statistical view. We demonstrate both theoretically and empirically that previous methods based on softmax cross-entropy are not suitable, which can result in local models heavily overfitting to minority classes and missing classes. Additionally, we theoretically introduce a deviation bound to measure the deviation of the gradient after local update. At last, we propose FedLC (textbf Fed erated learning viatextbf L ogitstextbf C alibration), which calibrates the logits before softmax cross-entropy according to the probability of occurrence of each class. FedLC applies a fine-grained calibrated cross-entropy loss to local update by adding a pairwise label margin. Extensive experiments on federated datasets and real-world datasets demonstrate that FedLC leads to a more accurate global model and much improved performance. Furthermore, integrating other FL methods into our approach can further enhance the performance of the global model.
Key facts. Cited by 109 papers in the map (data through 14 Oct 2026). Builds on 34 of its 59 references that are in the map. First posted on arXiv 1 Sep 2022.
Builds on
34 of 59 references.
References · 34
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2019Advances and Open Problems in Federated Learningcited by 3,046
- 2018Federated Optimization in Heterogeneous Networkscited by 2,987
- 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
- 2018Federated Learning with Non-IID Datacited by 1,108
- 2020Adaptive Federated Optimizationcited by 1,046
- 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
- 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
- 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
- 2018LEAF: A Benchmark for Federated Settingscited by 792
- 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 705
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
22 more
- 2020Long-tail learning via logit adjustmentcited by 618
- 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
- 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
- 2020Federated Learning with Matched Averagingcited by 564
- 2021Federated Learning Based on Dynamic Regularizationcited by 562
- 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
- 2021A Field Guide to Federated Optimizationcited by 315
- 2021Anti-Backdoor Learning: Training Clean Models on Poisoned Datacited by 263
- 2020Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspectivecited by 143
- 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
- 2020Practical One-Shot Federated Learning for Cross-Silo Settingcited by 77
- 2020FedDANE: A Federated Newton-Type Methodcited by 76
- 2020Federated Mutual Learningcited by 71
- 2019Practical Federated Gradient Boosting Decision Treescited by 54
- 2019Adjusting Decision Boundary for Class Imbalanced Learningcited by 52
- 2021Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learningcited by 52
- 2022Towards Practical Certifiable Patch Defense with Vision Transformercited by 44
- 2021Disentangled High Quality Salient Object Detectioncited by 35
- 2020Near-Tight Margin-Based Generalization Bounds for Support Vector Machinescited by 19
- 2021CoSformer: Detecting Co-Salient Object with Transformerscited by 10
- 2020CLASS: Cross-Level Attention and Supervision for Salient Objects Detectioncited by 7
- 2021Highly Efficient Natural Image Mattingcited by 6
Built on by
Citing papers per year: 2022: 3; 2023: 36; 2024: 37; 2025: 22; 2026: 11.
109 papers.
Sort:
By area:
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
- 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 99
- 2023TARGET: Federated Class-Continual Learning via Exemplar-Free Distillationcited by 65
- 2023No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifiercited by 55
- 2023FedFed: Feature Distillation against Data Heterogeneity in Federated Learningcited by 54
- 2022FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Datacited by 50
- 2023Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learningcited by 40
- 2024Relaxed Contrastive Learning for Federated Learningcited by 32
- 2022Boosting the Transferability of Adversarial Attacks with Global Momentum Initializationcited by 29
- 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 24
- 2023Federated Generative Learning with Foundation Modelscited by 23
- 2023Adaptive Test-Time Personalization for Federated Learningcited by 23
38 more
- 2023MP-FedCL: Multiprototype Federated Contrastive Learning for Edge Intelligencecited by 20
- 2023Delving into the Adversarial Robustness of Federated Learningcited by 19
- 2023HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Associationcited by 17
- 2023A Simple Data Augmentation for Feature Distribution Skewed Federated Learningcited by 17
- 2023Fed-GraB: Federated Long-tailed Learning with Self-Adjusting Gradient Balancercited by 17
- 2023PFLlib: A Beginner-Friendly and Comprehensive Personalized Federated Learning Library and Benchmarkcited by 17
- 2023A Comparative Evaluation of FedAvg and Per-FedAvg Algorithms for Dirichlet Distributed Heterogeneous Datacited by 14
- 2023Exploiting Label Skews in Federated Learning with Model Concatenationcited by 14
- 2024Federated Learning with Bilateral Curation for Partially Class-Disjoint Datacited by 13
- 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 13
- 2025Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learningcited by 12
- 2022VarietySound: Timbre-Controllable Video to Sound Generation via Unsupervised Information Disentanglementcited by 11
- 2024Mobility Accelerates Learning: Convergence Analysis on Hierarchical Federated Learning in Vehicular Networkscited by 11
- 2024FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusioncited by 11
- 2024Exploring Vacant Classes in Label-Skewed Federated Learningcited by 10
- 2024FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Datacited by 10
- 2023pFedSim: Similarity-Aware Model Aggregation Towards Personalized Federated Learningcited by 9
- 2023Handling Data Heterogeneity via Architectural Design for Federated Visual Recognitioncited by 8
- 2023Generalized Universal Domain Adaptation with Generative Flow Networkscited by 7
- 2025S2FGL: Spatial Spectral Federated Graph Learningcited by 7
- 2024FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scalercited by 6
- 2025Subgraph Federated Learning for Local Generalizationcited by 6
- 2023Federated Skewed Label Learning with Logits Fusioncited by 5
- 2023FediOS: Decoupling Orthogonal Subspaces for Personalization in Feature-skew Federated Learningcited by 5
- 2024Towards Robust Federated Learning via Logits Calibration on Non-IID Datacited by 5
- 2024FedCCL: Federated Dual-Clustered Feature Contrast Under Domain Heterogeneitycited by 5
- 2024From Optimization to Generalization: Fair Federated Learning against Quality Shift via Inter-Client Sharpness Matchingcited by 5
- 2025You Are Your Own Best Teacher: Achieving Centralized-level Performance in Federated Learning under Heterogeneous and Long-tailed Datacited by 5
- 2023FCA: Taming Long-tailed Federated Medical Image Classification by Classifier Anchoringcited by 4
- 2023Federated Model Aggregation via Self-Supervised Priors for Highly Imbalanced Medical Image Classificationcited by 4
- 2023Rethinking Client Drift in Federated Learning: A Logit Perspectivecited by 4
- 2023FedMS: Federated Learning with Mixture of Sparsely Activated Foundations Modelscited by 4
- 2024Logit Calibration and Feature Contrast for Robust Federated Learning on Non-IID Datacited by 4
- 2023Stabilizing and Improving Federated Learning with Non-IID Data and Client Dropoutcited by 3
- 2023FedBug: A Bottom-Up Gradual Unfreezing Framework for Federated Learningcited by 3
- 2023FedDrive v2: an Analysis of the Impact of Label Skewness in Federated Semantic Segmentation for Autonomous Drivingcited by 3
- 2023FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentationcited by 3
- 2024SCALA: Split Federated Learning with Concatenated Activations and Logit Adjustmentscited by 3
50 most cited of 109 All 109 (JSON)
- 2026FedLTLib: A Comprehensive Benchmark for Federated Long-Tail Learning14 Sep 2026
- 2026MPT: Missing Prototype Tracking via Barycentric Reconstruction in Vehicular Federated Learning11 Sep 2026
- 2026FedCC: Towards Addressing Label Distribution Skews in Distillation-Based Federated Learning24 Aug 2026
- 2026WHERE to Generate Matters: Budget-Aware Synthetic Augmentation for Label Skewed Federated Learning7 Jul 2026
- 2026SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity5 Jul 2026
- 2026FedReLa: Imbalanced Federated Learning via Re-Labeling24 Jun 2026
- 2026Towards Federated Long-Tailed Graph Learning: An Energy-Guided Dual Decoupling Approach23 Jun 2026
- 2026Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learning8 Jun 2026
- 2026FedRG: Unleashing the Representation Geometry for Federated Learning with Noisy Clients20 Mar 2026
- 2026Breaking the Prototype Bias Loop: Confidence-Aware Federated Contrastive Learning for Highly Imbalanced Clients3 Mar 2026
- 2026FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environments26 Feb 2026
- 2025Federated Learning with Feedback Alignment14 Dec 2025
38 more
- 2025Geometric Prior-Guided Federated Prompt Calibration8 Dec 2025
- 2025FedHUG: Federated Heterogeneous Unsupervised Generalization for Remote Physiological Measurements14 Oct 2025
- 2025DART: A Server-side Plug-in for Resource-efficient Robust Federated Learning24 Aug 2025
- 2025Calibrating Biased Distribution in VFM-derived Latent Space via Cross-Domain Geometric Consistency19 Aug 2025
- 2025S2FGL: Spatial Spectral Federated Graph Learning3 Jul 2025
- 2025FedWSQ: Efficient Federated Learning with Weight Standardization and Distribution-Aware Non-Uniform Quantization30 Jun 2025
- 2025FedCGD: Collective Gradient Divergence Optimized Scheduling for Wireless Federated Learning9 Jun 2025
- 2025ATR-Bench: A Federated Learning Benchmark for Adaptation, Trust, and Reasoning22 May 2025
- 2025Enhancing the Performance of Global Model by Improving the Adaptability of Local Models in Federated Learning15 May 2025
- 2025Personalizing Federated Learning for Hierarchical Edge Networks with Non-IID Data11 Apr 2025
- 2025WeiDetect: Weibull Distribution-Based Defense against Poisoning Attacks in Federated Learning for Network Intrusion Detection Systems6 Apr 2025
- 2025PPFPL: Cross-silo Privacy-preserving Federated Prototype Learning Against Data Poisoning Attacks4 Apr 2025
- 2025FedSAF: A Federated Learning Framework for Enhanced Gastric Cancer Detection and Privacy Preservation20 Mar 2025
- 2025GC-Fed: Gradient Centralized Federated Learning with Partial Client Participation17 Mar 2025
- 2025Fair Federated Medical Image Classification Against Quality Shift via Inter-Client Progressive State Matching12 Mar 2025
- 2025Smoothing ADMM for Non-convex and Non-smooth Hierarchical Federated Learning11 Mar 2025
- 2025You Are Your Own Best Teacher: Achieving Centralized-level Performance in Federated Learning under Heterogeneous and Long-tailed Data10 Mar 2025
- 2025Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learning9 Mar 2025
- 2025Subgraph Federated Learning for Local Generalization6 Mar 2025
- 2025Understanding Federated Learning from IID to Non-IID dataset: An Experimental Study31 Jan 2025
- 2025Towards Communication-Efficient Adversarial Federated Learning for Robust Edge Intelligence25 Jan 2025
- 2024Exploiting Label Skewness for Spiking Neural Networks in Federated Learning23 Dec 2024
- 2024fluke: Federated Learning Utility frameworK for Experimentation and research20 Dec 2024
- 2024Federated Unlearning Model Recovery in Data with Skewed Label Distributions18 Dec 2024
- 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directions19 Nov 2024
- 2024Overcoming label shift with target-aware federated learning6 Nov 2024
- 2024Masked Autoencoders are Parameter-Efficient Federated Continual Learners4 Nov 2024
- 2024FPPL: An Efficient and Non-IID Robust Federated Continual Learning Framework4 Nov 2024
- 2024Federated Learning Clients Clustering with Adaptation to Data Drifts3 Nov 2024
- 2024Optimizing Federated Learning by Entropy-Based Client Selection2 Nov 2024
- 2024FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion27 Oct 2024
- 2024Dataset Distillation-based Hybrid Federated Learning on Non-IID Data26 Sep 2024
- 2024Recovering Global Data Distribution Locally in Federated Learning21 Sep 2024
- 2024Balancing Label Imbalance in Federated Environments Using Only Mixup and Artificially-Labeled Noise20 Sep 2024
- 2024Unlocking the Potential of Model Calibration in Federated Learning7 Sep 2024
- 2024GAS: Generative Activation-Aided Asynchronous Split Federated Learning2 Sep 2024
- 2024Addressing Skewed Heterogeneity via Federated Prototype Rectification with Personalization15 Aug 2024
- 2024Personalized Federated Learning on Heterogeneous and Long-Tailed Data via Expert Collaborative Learning4 Aug 2024
50 newest of 109 All 109 (JSON)
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
- 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 99
- 2023TARGET: Federated Class-Continual Learning via Exemplar-Free Distillationcited by 65
- 2023No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifiercited by 55
- 2022FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Datacited by 50
- 2023Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learningcited by 40
- 2024Relaxed Contrastive Learning for Federated Learningcited by 32
- 2022Boosting the Transferability of Adversarial Attacks with Global Momentum Initializationcited by 29
- 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 24
- 2023Federated Generative Learning with Foundation Modelscited by 23
- 2023Adaptive Test-Time Personalization for Federated Learningcited by 23
- 2023MP-FedCL: Multiprototype Federated Contrastive Learning for Edge Intelligencecited by 20
12 most cited of 106 All 106 (JSON)
- 2026FedLTLib: A Comprehensive Benchmark for Federated Long-Tail Learning14 Sep 2026
- 2026MPT: Missing Prototype Tracking via Barycentric Reconstruction in Vehicular Federated Learning11 Sep 2026
- 2026FedCC: Towards Addressing Label Distribution Skews in Distillation-Based Federated Learning24 Aug 2026
- 2026WHERE to Generate Matters: Budget-Aware Synthetic Augmentation for Label Skewed Federated Learning7 Jul 2026
- 2026SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity5 Jul 2026
- 2026FedReLa: Imbalanced Federated Learning via Re-Labeling24 Jun 2026
- 2026Towards Federated Long-Tailed Graph Learning: An Energy-Guided Dual Decoupling Approach23 Jun 2026
- 2026Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learning8 Jun 2026
- 2026FedRG: Unleashing the Representation Geometry for Federated Learning with Noisy Clients20 Mar 2026
- 2026Breaking the Prototype Bias Loop: Confidence-Aware Federated Contrastive Learning for Highly Imbalanced Clients3 Mar 2026
- 2026FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environments26 Feb 2026
- 2025Federated Learning with Feedback Alignment14 Dec 2025
12 newest of 106 All 106 (JSON)