mlsanityData to 7 Oct 2026

Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning

arXiv 2206.02465 ↗First posted 6 Jun 2022cs.LG cs.AI cs.DCcited by 52

Not on the map. Why?

Abstract (arXiv)

In federated learning (FL), model performance typically suffers from client drift induced by data heterogeneity, and mainstream works focus on correcting client drift. We propose a different approach named virtual homogeneity learning (VHL) to directly "rectify" the data heterogeneity. In particular, VHL conducts FL with a virtual homogeneous dataset crafted to satisfy two conditions: containing no private information and being separable. The virtual dataset can be generated from pure noise shared across clients, aiming to calibrate the features from the heterogeneous clients. Theoretically, we prove that VHL can achieve provable generalization performance on the natural distribution. Empirically, we demonstrate that VHL endows FL with drastically improved convergence speed and generalization performance. VHL is the first attempt towards using a virtual dataset to address data heterogeneity, offering new and effective means to FL.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Builds on 48 of its 79 references that are in the map. First posted on arXiv 6 Jun 2022.

Builds on

48 of 79 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2022Generative Adversarial Networkscited by 5,603
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  7. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  8. 2019Advances and Open Problems in Federated Learningcited by 3,021
  9. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  10. 2020Supervised Contrastive Learningcited by 2,803
  11. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  12. 2018Federated Learning with Non-IID Datacited by 1,099
36 more
  1. 2020Adaptive Federated Optimizationcited by 1,036
  2. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  3. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  4. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  5. 2021Model-Contrastive Federated Learningcited by 762
  6. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  7. 2016Deep Transfer Learning with Joint Adaptation Networkscited by 609
  8. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  9. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  10. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  11. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  12. 2021A Field Guide to Federated Optimizationcited by 312
  13. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
  14. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  15. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  16. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
  17. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
  18. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 188
  19. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  20. 2020Style Normalization and Restitution for Generalizable Person Re-identificationcited by 136
  21. 2020Federated Visual Classification with Real-World Data Distributioncited by 126
  22. 2020What makes instance discrimination good for transfer learning?cited by 111
  23. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  24. 2021Learning to See by Looking at Noisecited by 92
  25. 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 91
  26. 2021Towards Fair Federated Learning with Zero-Shot Data Augmentationcited by 64
  27. 2020XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated Learningcited by 58
  28. 2019G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of Teacher Discriminatorscited by 53
  29. 2020Federated Learning via Synthetic Datacited by 53
  30. 2021Fed2: Feature-Aligned Federated Learningcited by 42
  31. 2021Near-Optimal Linear Regression under Distribution Shiftcited by 40
  32. 2018Adversarial Risk and Robustness: General Definitions and Implications for the Uniform Distributioncited by 32
  33. 2021FedCV: A Federated Learning Framework for Diverse Computer Vision Taskscited by 32
  34. 2019Real-World Image Datasets for Federated Learningcited by 30
  35. 2017Towards Practical Differential Privacy for SQL Queriescited by 19
  36. 2017Heterogeneous domain adaptation: An unsupervised approachcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 24; 2024: 15; 2025: 6; 2026: 5.

52 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  2. 2023FedFed: Feature Distillation against Data Heterogeneity in Federated Learningcited by 54
  3. 2022FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Datacited by 49
  4. 2023Recent Advances on Federated Learning: A Systematic Surveycited by 40
  5. 2023Cross-Silo Prototypical Calibration for Federated Learning with Non-IID Datacited by 35
  6. 2023Out-of-distribution Detection Learning with Unreliable Out-of-distribution Sourcescited by 31
  7. 2024STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMscited by 31
  8. 2024Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boostingcited by 27
  9. 2023Federated Domain Generalization: A Surveycited by 25
  10. 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 23
  11. 2023A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiencycited by 21
  12. 2024Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchorscited by 21
38 more
  1. 2023FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUscited by 20
  2. 2024Should We Really Edit Language Models? On the Evaluation of Edited Language Modelscited by 18
  3. 2023Hard Sample Matters a Lot in Zero-Shot Quantizationcited by 15
  4. 2023Heterogeneous Forgetting Compensation for Class-Incremental Learningcited by 15
  5. 2024FedImpro: Measuring and Improving Client Update in Federated Learningcited by 15
  6. 2023Feature Matching Data Synthesis for Non-IID Federated Learningcited by 13
  7. 2023Multimodal Federated Learning in Healthcare: a Reviewcited by 13
  8. 2023FedCiR: Client-Invariant Representation Learning for Federated Non-IID Featurescited by 12
  9. 2023Federated Learning with Extremely Noisy Clients via Negative Distillationcited by 12
  10. 2024FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusioncited by 11
  11. 2023Revisiting Data-Free Knowledge Distillation with Poisoned Teacherscited by 9
  12. 2023Federated Learning on Virtual Heterogeneous Data with Local-global Distillationcited by 6
  13. 2023Fake It Till Make It: Federated Learning with Consensus-Oriented Generationcited by 6
  14. 2024Robust Training of Federated Models with Extremely Label Deficiencycited by 6
  15. 2025One-shot Federated Learning Methods: A Practical Guidecited by 6
  16. 2023EX-Graph: A Pioneering Dataset Bridging Ethereum and Xcited by 5
  17. 2024Federated brain tumor segmentation: an extensive benchmarkcited by 5
  18. 2025A Similarity Paradigm Through Textual Regularization Without Forgettingcited by 5
  19. 2023FedML Parrot: A Scalable Federated Learning System via Heterogeneity-aware Scheduling on Sequential and Hierarchical Trainingcited by 4
  20. 2025FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learningcited by 4
  21. 2023FedBug: A Bottom-Up Gradual Unfreezing Framework for Federated Learningcited by 3
  22. 2023Enhancing Clustered Federated Learning: Integration of Strategies and Improved Methodologiescited by 3
  23. 2023FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentationcited by 3
  24. 2024Uncertainty-Based Extensible Codebook for Discrete Federated Learning in Heterogeneous Data Siloscited by 3
  25. 2024A Unified Solution to Diverse Heterogeneities in One-shot Federated Learningcited by 3
  26. 2025FedDAG: Federated Domain Adversarial Generation Towards Generalizable Medical Image Analysiscited by 2
  27. 2025Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environmentscited by 2
  28. 2023Whole-brain radiomics for clustered federated personalization in brain tumor segmentationcited by 1
  29. 2024FedMAP: Personalised Federated Learning for Real Large-Scale Healthcare Systemscited by 1
  30. 2022Linear Scalarization for Byzantine-robust learning on non-IID datacited by 0
  31. 2024Client2Vec: Improving Federated Learning by Distribution Shifts Aware Client Indexingcited by 0
  32. 2024FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentationcited by 0
  33. 2024Federated Impression for Learning with Distributed Heterogeneous Datacited by 0
  34. 2024FedGTST: Boosting Global Transferability of Federated Models via Statistics Tuningcited by 0
  35. 2025Provably Near-Optimal Federated Ensemble Distillation with Negligible Overheadcited by 0
  36. 2026RefProtoFL: Communication-Efficient Federated Learning via External-Referenced Prototype Alignmentcited by 0
  37. 2026FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environmentscited by 0
  38. 2026Geometric Knowledge-Assisted Federated Dual Knowledge Distillation Approach Towards Remote Sensing Satellite Imagerycited by 0

50 most cited of 52 All 52 (JSON)