mlsanityData to 14 Oct 2026

AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models

arXiv 2405.16240 ↗First posted 25 May 2024cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce analytic federated learning (AFL), a new training paradigm that brings analytical (i.e., closed-form) solutions to the federated learning (FL) with pre-trained models. Our AFL draws inspiration from analytic learning -- a gradient-free technique that trains neural networks with analytical solutions in one epoch. In the local client training stage, the AFL facilitates a one-epoch training, eliminating the necessity for multi-epoch updates. In the aggregation stage, we derive an absolute aggregation (AA) law. This AA law allows a single-round aggregation, reducing heavy communication overhead and achieving fast convergence by removing the need for multiple aggregation rounds. More importantly, the AFL exhibits a property that textitinvariance to data partitioning, meaning that regardless of how the full dataset is distributed among clients, the aggregated result remains identical. This could spawn various potentials, such as data heterogeneity invariance and client-number invariance. We conduct experiments across various FL settings including extremely non-IID ones, and scenarios with a large number of clients (e.g., ≥ 1000). In all these settings, our AFL constantly performs competitively while existing FL techniques encounter various obstacles. Our codes are available at https://github.com/ZHUANGHP/Analytic-federated-learning.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 32 of its 52 references that are in the map. First posted on arXiv 25 May 2024.

Builds on

32 of 52 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  7. 2021Model-Contrastive Federated Learningcited by 766
  8. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  9. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  10. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  11. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  12. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
20 more
  1. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  2. 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 142
  3. 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
  4. 2022When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methodscited by 75
  5. 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
  6. 2023FedDisco: Federated Learning with Discrepancy-Aware Collaborationcited by 67
  7. 2020Peer Collaborative Learning for Online Knowledge Distillationcited by 58
  8. 2023Revisiting Weighted Aggregation in Federated Learning with Neural Networkscited by 58
  9. 2020Blockchain-Federated-Learning and Deep Learning Models for COVID-19 detection using CT Imagingcited by 57
  10. 2022ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protectioncited by 57
  11. 2023Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generationcited by 54
  12. 2021Fed2: Feature-Aligned Federated Learningcited by 42
  13. 2023Learning Federated Visual Prompt in Null Space for MRI Reconstructioncited by 40
  14. 2024DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learningcited by 28
  15. 2022Federated Learning with Position-Aware Neuronscited by 24
  16. 2024GACL: Exemplar-Free Generalized Analytic Continual Learningcited by 22
  17. 2024FedFisher: Leveraging Fisher Information for One-Shot Federated Learningcited by 15
  18. 2024Locality Sensitive Sparse Encoding for Learning World Models Onlinecited by 12
  19. 2024F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learningcited by 12
  20. 2018Learning from the Kernel and the Range Spacecited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 4.

11 papers.

  1. 2024Online Analytic Exemplar-Free Continual Learning with Large Models for Imbalanced Autonomous Driving Taskcited by 4
  2. 2025Towards Efficient and Exact Forgetting Services in Pre-Trained-Model-based Continual Learningcited by 3
  3. 2025AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Datacited by 3
  4. 2026DeepAFL: Deep Analytic Federated Learningcited by 1
  5. 2025Analytic Personalized Federated Meta-Learningcited by 0
  6. 2025Communication and Computation Efficient Split Federated Learning in O-RANcited by 0
  7. 2025FedHiP: Heterogeneity-Invariant Personalized Federated Learning Through Closed-Form Solutionscited by 0
  8. 2025Single-Round Scalable Analytic Federated Learningcited by 0
  9. 2026AdaBFL: Multi-Layer Defensive Adaptive Aggregation for Bzantine-Robust Federated Learningcited by 0
  10. 2026FedOPAL: One-Shot Federated Learning via Analytic Visual Prompt Tuningcited by 0
  11. 2026One Round Is All You Need: Analytic Federated Learning for Task-Heterogeneous Multi-Label Medical Image Classificationcited by 0