# Federated Skewed Label Learning with Logits Fusion

arXiv: https://arxiv.org/abs/2311.08202 · HTML: https://mlsanity.com/paper/2311.08202/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 35 references that are in the map.
- First posted on arXiv 14 Nov 2023.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [Federated Learning with Matched Averaging](https://mlsanity.com/paper/2002.06440/) (2020)
- [Federated Learning on Non-IID Data: A Survey](https://mlsanity.com/paper/2106.06843/) (2021)
- [Can AI help in screening Viral and COVID-19 pneumonia?](https://mlsanity.com/paper/2003.13145/) (2020)
- [Federated Learning with Label Distribution Skew via Logits Calibration](https://mlsanity.com/paper/2209.00189/) (2022)
- [Preservation of the Global Knowledge by Not-True Distillation in Federated Learning](https://mlsanity.com/paper/2106.03097/) (2021)
- [Fast-Convergent Federated Learning with Adaptive Weighting](https://mlsanity.com/paper/2012.00661/) (2020)
- [Towards Federated Learning against Noisy Labels via Local Self-Regularization](https://mlsanity.com/paper/2208.12807/) (2022)
- [Exploring the Distributed Knowledge Congruence in Proxy-data-free Federated Distillation](https://mlsanity.com/paper/2204.07028/) (2022)
- [FedCache: A Knowledge Cache-driven Federated Learning Architecture for Personalized Edge Intelligence](https://mlsanity.com/paper/2308.07816/) (2023)
- [Spirit Distillation: A Model Compression Method with Multi-domain Knowledge Transfer](https://mlsanity.com/paper/2104.14696/) (2021)
- [Knowledge Distillation in Federated Edge Learning: A Survey](https://mlsanity.com/paper/2301.05849/) (2023)

## Built on by (most cited first, 5 of 5)

- [Tackling Noisy Clients in Federated Learning with End-to-end Label Correction](https://mlsanity.com/paper/2408.04301/) (2024)
- [Exploring Vacant Classes in Label-Skewed Federated Learning](https://mlsanity.com/paper/2401.02329/) (2024)
- [FNBench: Benchmarking Robust Federated Learning against Noisy Labels](https://mlsanity.com/paper/2505.06684/) (2025)
- [A Lightweight and Secure Deep Learning Model for Privacy-Preserving Federated Learning in Intelligent Enterprises](https://mlsanity.com/paper/2503.02017/) (2025)
- [FedRG: Unleashing the Representation Geometry for Federated Learning with Noisy Clients](https://mlsanity.com/paper/2603.19722/) (2026)

## Abstract (arXiv)

Federated learning (FL) aims to collaboratively train a shared model across multiple clients without transmitting their local data. Data heterogeneity is a critical challenge in realistic FL settings, as it causes significant performance deterioration due to discrepancies in optimization among local models. In this work, we focus on label distribution skew, a common scenario in data heterogeneity, where the data label categories are imbalanced on each client. To address this issue, we propose FedBalance, which corrects the optimization bias among local models by calibrating their logits. Specifically, we introduce an extra private weak learner on the client side, which forms an ensemble model with the local model. By fusing the logits of the two models, the private weak learner can capture the variance of different data, regardless of their category. Therefore, the optimization direction of local models can be improved by increasing the penalty for misclassifying minority classes and reducing the attention to majority classes, resulting in a better global model. Extensive experiments show that our method can gain 13% higher average accuracy compared with state-of-the-art methods.