# FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation

arXiv: https://arxiv.org/abs/2210.08285 · HTML: https://mlsanity.com/paper/2210.08285/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 63 references that are in the map.
- First posted on arXiv 15 Oct 2022.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [Local SGD Converges Fast and Communicates Little](https://mlsanity.com/paper/1805.09767/) (2018)
- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [Train faster, generalize better: Stability of stochastic gradient descent](https://mlsanity.com/paper/1509.01240/) (2015)
- [Federated Learning with Differential Privacy: Algorithms and Performance Analysis](https://mlsanity.com/paper/1911.00222/) (2019)
- [SWAD: Domain Generalization by Seeking Flat Minima](https://mlsanity.com/paper/2102.08604/) (2021)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [Data-Free Knowledge Distillation for Heterogeneous Federated Learning](https://mlsanity.com/paper/2105.10056/) (2021)
- [Personalized Cross-Silo Federated Learning on Non-IID Data](https://mlsanity.com/paper/2007.03797/) (2020)
- [No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data](https://mlsanity.com/paper/2106.05001/) (2021)
- [Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning](https://mlsanity.com/paper/2203.09249/) (2022)
- [Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning](https://mlsanity.com/paper/2105.05883/) (2021)
- [FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learning](https://mlsanity.com/paper/2102.02514/) (2021)
- [LDP-FL: Practical Private Aggregation in Federated Learning with Local Differential Privacy](https://mlsanity.com/paper/2007.15789/) (2020)
- [Federated Learning with Bayesian Differential Privacy](https://mlsanity.com/paper/1911.10071/) (2019)
- [Normalized Flat Minima: Exploring Scale Invariant Definition of Flat Minima for Neural Networks using PAC-Bayesian Analysis](https://mlsanity.com/paper/1901.04653/) (2019)
- [Performance Optimization for Federated Person Re-identification via Benchmark Analysis](https://mlsanity.com/paper/2008.11560/) (2020)
- [Federated Learning for Malware Detection in IoT Devices](https://mlsanity.com/paper/2104.09994/) (2021)
- [QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning](https://mlsanity.com/paper/2107.13892/) (2021)
- [FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cycling](https://mlsanity.com/paper/2009.10748/) (2020)
- [Cost-Effective Federated Learning in Mobile Edge Networks](https://mlsanity.com/paper/2109.05411/) (2021)
- [Accelerating Federated Learning over Reliability-Agnostic Clients in Mobile Edge Computing Systems](https://mlsanity.com/paper/2007.14374/) (2020)

## Built on by (most cited first, 21 of 21)

- [LoRASculpt: Sculpting LoRA for Harmonizing General and Specialized Knowledge in Multimodal Large Language Models](https://mlsanity.com/paper/2503.16843/) (2025)
- [Class-wise Balancing Data Replay for Federated Class-Incremental Learning](https://mlsanity.com/paper/2507.07712/) (2025)
- [Have Your Cake and Eat It Too: Toward Efficient and Accurate Split Federated Learning](https://mlsanity.com/paper/2311.13163/) (2023)
- [FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2510.20250/) (2025)
- [Fair Concurrent Training of Multiple Models in Federated Learning](https://mlsanity.com/paper/2404.13841/) (2024)
- [CaBaFL: Asynchronous Federated Learning via Hierarchical Cache and Feature Balance](https://mlsanity.com/paper/2404.12850/) (2024)
- [Learnable Sparse Customization in Heterogeneous Edge Computing](https://mlsanity.com/paper/2412.07216/) (2024)
- [Global Prompt Refinement with Non-Interfering Attention Masking for One-Shot Federated Learning](https://mlsanity.com/paper/2509.22700/) (2025)
- [Geometric Prior-Guided Federated Prompt Calibration](https://mlsanity.com/paper/2512.07208/) (2025)
- [Federated Cross-Training Learners for Robust Generalization under Data Heterogeneity](https://mlsanity.com/paper/2405.20046/) (2024)
- [Heterogeneity-Aware Knowledge Sharing for Graph Federated Learning](https://mlsanity.com/paper/2601.21589/) (2026)
- [Knowledge Rumination for Client Utility Evaluation in Heterogeneous Federated Learning](https://mlsanity.com/paper/2312.10425/) (2023)
- [KoReA-SFL: Knowledge Replay-based Split Federated Learning Against Catastrophic Forgetting](https://mlsanity.com/paper/2404.12846/) (2024)
- [Federated Black-Box Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2410.24181/) (2024)
- [FedQP: Towards Accurate Federated Learning using Quadratic Programming Guided Mutation](https://mlsanity.com/paper/2411.15847/) (2024)
- [Gradients as an Action: Towards Communication-Efficient Federated Recommender Systems via Adaptive Action Sharing](https://mlsanity.com/paper/2507.08842/) (2025)
- [Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection](https://mlsanity.com/paper/2602.01039/) (2026)
- [From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneity](https://mlsanity.com/paper/2604.08617/) (2026)
- [Fisher-Routed Mixture of Experts for Federated Class-Incremental Learning](https://mlsanity.com/paper/2606.28835/) (2026)
- [FedCMAPSS: A Benchmark for Federated Learning in Remaining Useful Life Estimation](https://mlsanity.com/paper/2608.26433/) (2026)
- [Prototype-guided Bilateral Alignment Multimodal Federated Learning](https://mlsanity.com/paper/2609.38925/) (2026)

## Abstract (arXiv)

As a promising distributed machine learning paradigm, Federated Learning (FL) has attracted increasing attention to deal with data silo problems without compromising user privacy. By adopting the classic one-to-multi training scheme (i.e., FedAvg), where the cloud server dispatches one single global model to multiple involved clients, conventional FL methods can achieve collaborative model training without data sharing. However, since only one global model cannot always accommodate all the incompatible convergence directions of local models, existing FL approaches greatly suffer from inferior classification accuracy. To address this issue, we present an efficient FL framework named FedCross, which uses a novel multi-to-multi FL training scheme based on our proposed multi-model cross-aggregation approach. Unlike traditional FL methods, in each round of FL training, FedCross uses multiple middleware models to conduct weighted fusion individually. Since the middleware models used by FedCross can quickly converge into the same flat valley in terms of loss landscapes, the generated global model can achieve a well-generalization. Experimental results on various well-known datasets show that, compared with state-of-the-art FL methods, FedCross can significantly improve FL accuracy within both IID and non-IID scenarios without causing additional communication overhead.