# FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction

arXiv: https://arxiv.org/abs/2203.11751 · HTML: https://mlsanity.com/paper/2203.11751/

## Key facts

- Cited by 181 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 22 Mar 2022.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [Federated Learning Based on Dynamic Regularization](https://mlsanity.com/paper/2111.04263/) (2021)
- [Bayesian Nonparametric Federated Learning of Neural Networks](https://mlsanity.com/paper/1905.12022/) (2019)
- [Tighter Theory for Local SGD on Identical and Heterogeneous Data](https://mlsanity.com/paper/1909.04746/) (2019)
- [Personalized Federated Learning with First Order Model Optimization](https://mlsanity.com/paper/2012.08565/) (2020)
- [Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning](https://mlsanity.com/paper/2101.11203/) (2021)
- [GIANT: Globally Improved Approximate Newton Method for Distributed Optimization](https://mlsanity.com/paper/1709.03528/) (2017)
- [Specificity-Preserving Federated Learning for MR Image Reconstruction](https://mlsanity.com/paper/2112.05752/) (2021)
- [Graph Oracle Models, Lower Bounds, and Gaps for Parallel Stochastic Optimization](https://mlsanity.com/paper/1805.10222/) (2018)
- [FireCaffe: near-linear acceleration of deep neural network training on compute clusters](https://mlsanity.com/paper/1511.00175/) (2015)
- [Communication trade-offs for synchronized distributed SGD with large step size](https://mlsanity.com/paper/1904.11325/) (2019)

## Built on by (most cited first, 30 of 181)

- [Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark](https://mlsanity.com/paper/2311.06750/) (2023)
- [FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learning](https://mlsanity.com/paper/2302.13485/) (2023)
- [FedDisco: Federated Learning with Discrepancy-Aware Collaboration](https://mlsanity.com/paper/2305.19229/) (2023)
- [FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy](https://mlsanity.com/paper/2302.10429/) (2023)
- [FedGTA: Topology-aware Averaging for Federated Graph Learning](https://mlsanity.com/paper/2401.11755/) (2024)
- [Prompt Federated Learning for Weather Forecasting: Toward Foundation Models on Meteorological Data](https://mlsanity.com/paper/2301.09152/) (2023)
- [Eliminating Domain Bias for Federated Learning in Representation Space](https://mlsanity.com/paper/2311.14975/) (2023)
- [Cross-Silo Prototypical Calibration for Federated Learning with Non-IID Data](https://mlsanity.com/paper/2308.03457/) (2023)
- [FedTune: A Deep Dive into Efficient Federated Fine-Tuning with Pre-trained Transformers](https://mlsanity.com/paper/2211.08025/) (2022)
- [An Aggregation-Free Federated Learning for Tackling Data Heterogeneity](https://mlsanity.com/paper/2404.18962/) (2024)
- [FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning](https://mlsanity.com/paper/2404.14061/) (2024)
- [Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initialization](https://mlsanity.com/paper/2306.05706/) (2023)
- [A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiency](https://mlsanity.com/paper/2307.10655/) (2023)
- [LightTR: A Lightweight Framework for Federated Trajectory Recovery](https://mlsanity.com/paper/2405.03409/) (2024)
- [FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors](https://mlsanity.com/paper/2503.15842/) (2025)
- [Visual Prompt Based Personalized Federated Learning](https://mlsanity.com/paper/2303.08678/) (2023)
- [OpenFGL: A Comprehensive Benchmark for Federated Graph Learning](https://mlsanity.com/paper/2408.16288/) (2024)
- [Re-thinking Federated Active Learning based on Inter-class Diversity](https://mlsanity.com/paper/2303.12317/) (2023)
- [FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity](https://mlsanity.com/paper/2404.09816/) (2024)
- [Personalized Federated Learning via Feature Distribution Adaptation](https://mlsanity.com/paper/2411.00329/) (2024)
- [Federated Learning in Non-IID Settings Aided by Differentially Private Synthetic Data](https://mlsanity.com/paper/2206.00686/) (2022)
- [Advances in Robust Federated Learning: A Survey with Heterogeneity Considerations](https://mlsanity.com/paper/2405.09839/) (2024)
- [Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learning](https://mlsanity.com/paper/2503.06457/) (2025)
- [MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learning](https://mlsanity.com/paper/2601.22416/) (2026)
- [Federated Learning Across Decentralized and Unshared Archives for Remote Sensing Image Classification](https://mlsanity.com/paper/2311.06141/) (2023)
- [Not All Federated Learning Algorithms Are Created Equal: A Performance Evaluation Study](https://mlsanity.com/paper/2403.17287/) (2024)
- [Federated Optimization with Doubly Regularized Drift Correction](https://mlsanity.com/paper/2404.08447/) (2024)
- [On the effectiveness of partial variance reduction in federated learning with heterogeneous data](https://mlsanity.com/paper/2212.02191/) (2022)
- [Combating Data Imbalances in Federated Semi-supervised Learning with Dual Regulators](https://mlsanity.com/paper/2307.05358/) (2023)
- [ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learning](https://mlsanity.com/paper/2310.05143/) (2023)

## Abstract (arXiv)

Federated learning (FL) allows multiple clients to collectively train a high-performance global model without sharing their private data. However, the key challenge in federated learning is that the clients have significant statistical heterogeneity among their local data distributions, which would cause inconsistent optimized local models on the client-side. To address this fundamental dilemma, we propose a novel federated learning algorithm with local drift decoupling and correction (FedDC). Our FedDC only introduces lightweight modifications in the local training phase, in which each client utilizes an auxiliary local drift variable to track the gap between the local model parameter and the global model parameters. The key idea of FedDC is to utilize this learned local drift variable to bridge the gap, i.e., conducting consistency in parameter-level. The experiment results and analysis demonstrate that FedDC yields expediting convergence and better performance on various image classification tasks, robust in partial participation settings, non-iid data, and heterogeneous clients.