# FedCorr: Multi-Stage Federated Learning for Label Noise Correction

arXiv: https://arxiv.org/abs/2204.04677 · HTML: https://mlsanity.com/paper/2204.04677/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 43 references that are in the map.
- First posted on arXiv 10 Apr 2022.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates](https://mlsanity.com/paper/1803.01498/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Local SGD Converges Fast and Communicates Little](https://mlsanity.com/paper/1805.09767/) (2018)
- [Federated Learning Based on Dynamic Regularization](https://mlsanity.com/paper/2111.04263/) (2021)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018)
- [Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality](https://mlsanity.com/paper/1801.02613/) (2018)
- [Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies](https://mlsanity.com/paper/2010.01243/) (2020)
- [Dimensionality-Driven Learning with Noisy Labels](https://mlsanity.com/paper/1806.02612/) (2018)
- [Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Data](https://mlsanity.com/paper/2008.06180/) (2020)
- [A Survey of Label-noise Representation Learning: Past, Present and Future](https://mlsanity.com/paper/2011.04406/) (2020)
- [A Reputation Mechanism Is All You Need: Collaborative Fairness and Adversarial Robustness in Federated Learning](https://mlsanity.com/paper/2011.10464/) (2020)
- [Attack-Resistant Federated Learning with Residual-based Reweighting](https://mlsanity.com/paper/1912.11464/) (2019)
- [Robust Federated Learning with Noisy Labels](https://mlsanity.com/paper/2012.01700/) (2020)
- [FOCUS: Dealing with Label Quality Disparity in Federated Learning](https://mlsanity.com/paper/2001.11359/) (2020)
- [Robust Federated Learning with Attack-Adaptive Aggregation](https://mlsanity.com/paper/2102.05257/) (2021)
- [Dynamic Attention-based Communication-Efficient Federated Learning](https://mlsanity.com/paper/2108.05765/) (2021)

## Built on by (most cited first, 30 of 61)

- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [FedNoRo: Towards Noise-Robust Federated Learning by Addressing Class Imbalance and Label Noise Heterogeneity](https://mlsanity.com/paper/2305.05230/) (2023)
- [Taming Cross-Domain Representation Variance in Federated Prototype Learning with Heterogeneous Data Domains](https://mlsanity.com/paper/2403.09048/) (2024)
- [Fed-GraB: Federated Long-tailed Learning with Self-Adjusting Gradient Balancer](https://mlsanity.com/paper/2310.07587/) (2023)
- [FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clustering](https://mlsanity.com/paper/2301.12379/) (2023)
- [FedA3I: Annotation Quality-Aware Aggregation for Federated Medical Image Segmentation against Heterogeneous Annotation Noise](https://mlsanity.com/paper/2312.12838/) (2023)
- [Labeling Chaos to Learning Harmony: Federated Learning with Noisy Labels](https://mlsanity.com/paper/2208.09378/) (2022)
- [FedDiv: Collaborative Noise Filtering for Federated Learning with Noisy Labels](https://mlsanity.com/paper/2312.12263/) (2023)
- [AdaFed: Fair Federated Learning via Adaptive Common Descent Direction](https://mlsanity.com/paper/2401.04993/) (2024)
- [Learning Cautiously in Federated Learning with Noisy and Heterogeneous Clients](https://mlsanity.com/paper/2304.02892/) (2023)
- [Federated Learning with Extremely Noisy Clients via Negative Distillation](https://mlsanity.com/paper/2312.12703/) (2023)
- [Advances in Robust Federated Learning: A Survey with Heterogeneity Considerations](https://mlsanity.com/paper/2405.09839/) (2024)
- [FedNoisy: Federated Noisy Label Learning Benchmark](https://mlsanity.com/paper/2306.11650/) (2023)
- [Federated Learning with Instance-Dependent Noisy Label](https://mlsanity.com/paper/2312.10324/) (2023)
- [FedAIoT: A Federated Learning Benchmark for Artificial Intelligence of Things](https://mlsanity.com/paper/2310.00109/) (2023)
- [Spectral Co-Distillation for Personalized Federated Learning](https://mlsanity.com/paper/2401.17124/) (2024)
- [FedFixer: Mitigating Heterogeneous Label Noise in Federated Learning](https://mlsanity.com/paper/2403.16561/) (2024)
- [S2FGL: Spatial Spectral Federated Graph Learning](https://mlsanity.com/paper/2507.02409/) (2025)
- [Federated Learning Under Attack: Exposing Vulnerabilities through Data Poisoning Attacks in Computer Networks](https://mlsanity.com/paper/2403.02983/) (2024)
- [FNBench: Benchmarking Robust Federated Learning against Noisy Labels](https://mlsanity.com/paper/2505.06684/) (2025)
- [NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension](https://mlsanity.com/paper/2211.12759/) (2022)
- [FediOS: Decoupling Orthogonal Subspaces for Personalization in Feature-skew Federated Learning](https://mlsanity.com/paper/2311.18559/) (2023)
- [FedMLP: Federated Multi-Label Medical Image Classification under Task Heterogeneity](https://mlsanity.com/paper/2406.18995/) (2024)
- [Entropy-driven Fair and Effective Federated Learning](https://mlsanity.com/paper/2301.12407/) (2023)
- [Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosis](https://mlsanity.com/paper/2407.02261/) (2024)
- [Federated Learning Client Pruning for Noisy Labels](https://mlsanity.com/paper/2411.07391/) (2024)
- [Quantifying the Impact of Label Noise on Federated Learning](https://mlsanity.com/paper/2211.07816/) (2022)
- [Personalizing Federated Learning with Over-the-Air Computations](https://mlsanity.com/paper/2302.12509/) (2023)
- [FedEL: Federated Elastic Learning for Heterogeneous Devices](https://mlsanity.com/paper/2509.16902/) (2025)
- [FedRN: Exploiting k-Reliable Neighbors Towards Robust Federated Learning](https://mlsanity.com/paper/2205.01310/) (2022)

## Abstract (arXiv)

Federated learning (FL) is a privacy-preserving distributed learning paradigm that enables clients to jointly train a global model. In real-world FL implementations, client data could have label noise, and different clients could have vastly different label noise levels. Although there exist methods in centralized learning for tackling label noise, such methods do not perform well on heterogeneous label noise in FL settings, due to the typically smaller sizes of client datasets and data privacy requirements in FL. In this paper, we propose textttFedCorr, a general multi-stage framework to tackle heterogeneous label noise in FL, without making any assumptions on the noise models of local clients, while still maintaining client data privacy. In particular, (1) textttFedCorr dynamically identifies noisy clients by exploiting the dimensionalities of the model prediction subspaces independently measured on all clients, and then identifies incorrect labels on noisy clients based on per-sample losses. To deal with data heterogeneity and to increase training stability, we propose an adaptive local proximal regularization term that is based on estimated local noise levels. (2) We further finetune the global model on identified clean clients and correct the noisy labels for the remaining noisy clients after finetuning. (3) Finally, we apply the usual training on all clients to make full use of all local data. Experiments conducted on CIFAR-10/100 with federated synthetic label noise, and on a real-world noisy dataset, Clothing1M, demonstrate that textttFedCorr is robust to label noise and substantially outperforms the state-of-the-art methods at multiple noise levels.