# An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

arXiv: https://arxiv.org/abs/2404.18962 · HTML: https://mlsanity.com/paper/2404.18962/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 41 references that are in the map.
- First posted on arXiv 29 Apr 2024.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [FedBN: Federated Learning on Non-IID Features via Local Batch Normalization](https://mlsanity.com/paper/2102.07623/) (2021)
- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [Federated Learning Based on Dynamic Regularization](https://mlsanity.com/paper/2111.04263/) (2021)
- [Dataset Condensation with Gradient Matching](https://mlsanity.com/paper/2006.05929/) (2020)
- [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022)
- [Data-Free Knowledge Distillation for Heterogeneous Federated Learning](https://mlsanity.com/paper/2105.10056/) (2021)
- [Dataset Condensation with Distribution Matching](https://mlsanity.com/paper/2110.04181/) (2021)
- [Federated Learning on Non-IID Data: A Survey](https://mlsanity.com/paper/2106.06843/) (2021)
- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [Dataset Condensation with Differentiable Siamese Augmentation](https://mlsanity.com/paper/2102.08259/) (2021)
- [Dataset Distillation](https://mlsanity.com/paper/1811.10959/) (2018)
- [FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction](https://mlsanity.com/paper/2203.11751/) (2022)
- [FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning](https://mlsanity.com/paper/2009.01974/) (2020)
- [Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning](https://mlsanity.com/paper/2203.09249/) (2022)
- [Privacy for Free: How does Dataset Condensation Help Privacy?](https://mlsanity.com/paper/2206.00240/) (2022)
- [Generative Modeling using the Sliced Wasserstein Distance](https://mlsanity.com/paper/1803.11188/) (2018)
- [FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2207.09653/) (2022)
- [Sliced Wasserstein Kernels for Probability Distributions](https://mlsanity.com/paper/1511.03198/) (2015)
- [Learning Federated Visual Prompt in Null Space for MRI Reconstruction](https://mlsanity.com/paper/2303.16181/) (2023)

## Built on by (most cited first, 30 of 33)

- [LoRA-FAIR: Federated LoRA Fine-Tuning with Aggregation and Initialization Refinement](https://mlsanity.com/paper/2411.14961/) (2024)
- [The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions](https://mlsanity.com/paper/2502.05673/) (2025)
- [Generative Dataset Distillation Based on Diffusion Model](https://mlsanity.com/paper/2408.08610/) (2024)
- [TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation](https://mlsanity.com/paper/2502.02854/) (2025)
- [FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRA](https://mlsanity.com/paper/2503.11880/) (2025)
- [FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Models](https://mlsanity.com/paper/2501.02461/) (2025)
- [Subgraph Federated Learning for Local Generalization](https://mlsanity.com/paper/2503.03995/) (2025)
- [Going Beyond Feature Similarity: Effective Dataset Distillation based on Class-Aware Conditional Mutual Information](https://mlsanity.com/paper/2412.09945/) (2024)
- [FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2510.20250/) (2025)
- [SCALA: Split Federated Learning with Concatenated Activations and Logit Adjustments](https://mlsanity.com/paper/2405.04875/) (2024)
- [Dataset Distillation from First Principles: Integrating Core Information Extraction and Purposeful Learning](https://mlsanity.com/paper/2409.01410/) (2024)
- [Dataset Distillation-based Hybrid Federated Learning on Non-IID Data](https://mlsanity.com/paper/2409.17517/) (2024)
- [Tackling Data Heterogeneity in Federated Time Series Forecasting](https://mlsanity.com/paper/2411.15716/) (2024)
- [Maximizing Uncertainty for Federated learning via Bayesian Optimisation-based Model Poisoning](https://mlsanity.com/paper/2501.08002/) (2025)
- [AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Data](https://mlsanity.com/paper/2505.12245/) (2025)
- [FedDifRC: Unlocking the Potential of Text-to-Image Diffusion Models in Heterogeneous Federated Learning](https://mlsanity.com/paper/2507.06482/) (2025)
- [Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practice](https://mlsanity.com/paper/2510.12595/) (2025)
- [Federated Hybrid Training and Self-Adversarial Distillation: Towards Robust Edge Networks](https://mlsanity.com/paper/2412.19354/) (2024)
- [Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environments](https://mlsanity.com/paper/2505.19699/) (2025)
- [Federated Cross-Training Learners for Robust Generalization under Data Heterogeneity](https://mlsanity.com/paper/2405.20046/) (2024)
- [Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security](https://mlsanity.com/paper/2406.09831/) (2024)
- [Mitigating Domain Shift in Federated Learning via Intra- and Inter-Domain Prototypes](https://mlsanity.com/paper/2501.08521/) (2025)
- [FedDiverse: Tackling Data Heterogeneity in Federated Learning with Diversity-Driven Client Selection](https://mlsanity.com/paper/2504.11216/) (2025)
- [Distribution-Aware Mobility-Assisted Decentralized Federated Learning](https://mlsanity.com/paper/2505.18866/) (2025)
- [High-Energy Concentration for Federated Learning in Frequency Domain](https://mlsanity.com/paper/2509.12630/) (2025)
- [Adaptive Prototype Knowledge Transfer for Federated Learning with Mixed Modalities and Heterogeneous Tasks](https://mlsanity.com/paper/2502.04400/) (2025)
- [On optimal solutions of classical and sliced Wasserstein GANs with non-Gaussian data](https://mlsanity.com/paper/2509.06505/) (2025)
- [Mobility-Assisted Decentralized Federated Learning: Convergence Analysis and A Data-Driven Approach](https://mlsanity.com/paper/2512.24694/) (2025)
- [Foreground-Aware Dataset Distillation via Dynamic Patch Selection](https://mlsanity.com/paper/2601.02727/) (2026)
- [HiLoRA: Hierarchical Low-Rank Adaptation for Personalized Federated Learning](https://mlsanity.com/paper/2603.02785/) (2026)

## Abstract (arXiv)

The performance of Federated Learning (FL) hinges on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. This process can cause client drift, especially with significant cross-client data heterogeneity, impacting model performance and convergence of the FL algorithm. To address these challenges, we introduce FedAF, a novel aggregation-free FL algorithm. In this framework, clients collaboratively learn condensed data by leveraging peer knowledge, the server subsequently trains the global model using the condensed data and soft labels received from the clients. FedAF inherently avoids the issue of client drift, enhances the quality of condensed data amid notable data heterogeneity, and improves the global model performance. Extensive numerical studies on several popular benchmark datasets show FedAF surpasses various state-of-the-art FL algorithms in handling label-skew and feature-skew data heterogeneity, leading to superior global model accuracy and faster convergence.