mlsanityData to 14 Oct 2026

FAST: Federated Active Learning with Foundation Models for Communication-efficient Sampling and Training

arXiv 2504.03783 ↗First posted 3 Apr 2025cs.LG cs.AI cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Federated Active Learning (FAL) has emerged as a promising framework to leverage large quantities of unlabeled data across distributed clients while preserving data privacy. However, real-world deployments remain limited by high annotation costs and communication-intensive sampling processes, particularly in a cross-silo setting, when clients possess substantial local datasets. This paper addresses the crucial question: What is the best practice to reduce communication costs in human-in-the-loop learning with minimal annotator effort? Existing FAL methods typically rely on iterative annotation processes that separate active sampling from federated updates, leading to multiple rounds of expensive communication and annotation. In response, we introduce FAST, a two-pass FAL framework that harnesses foundation models for weak labeling in a preliminary pass, followed by a refinement pass focused exclusively on the most uncertain samples. By leveraging representation knowledge from foundation models and integrating refinement steps into a streamlined workflow, FAST substantially reduces the overhead incurred by iterative active sampling. Extensive experiments on diverse medical and natural image benchmarks demonstrate that FAST outperforms existing FAL methods by an average of 4.36% while reducing communication rounds eightfold under a limited 5% labeling budget.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 38 of its 65 references that are in the map. First posted on arXiv 3 Apr 2025.

Builds on

38 of 65 references.

References · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  7. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  8. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  9. 2017Deep Bayesian Active Learning with Image Datacited by 819
  10. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
  11. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 690
  12. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
26 more
  1. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 610
  2. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 500
  3. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 455
  4. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
  5. 2019Learning Loss for Active Learningcited by 336
  6. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  7. 2019Probabilistic End-to-end Noise Correction for Learning with Noisy Labelscited by 210
  8. 2020Does label smoothing mitigate label noise?cited by 204
  9. 2017Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentationcited by 146
  10. 2021Batch Active Learning at Scalecited by 127
  11. 2019Meta Label Correction for Noisy Label Learningcited by 108
  12. 2022A Comparative Survey of Deep Active Learningcited by 89
  13. 2017Deep Active Learning over the Long Tailcited by 84
  14. 2020Sequential Graph Convolutional Network for Active Learningcited by 69
  15. 2022Active Learning by Feature Mixingcited by 63
  16. 2021Semi-Supervised Active Learning with Temporal Output Discrepancycited by 50
  17. 2020Learning to Purify Noisy Labels via Meta Soft Label Correctorcited by 39
  18. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
  19. 2023Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shiftscited by 28
  20. 2019Epistemic Uncertainty Samplingcited by 24
  21. 2022Knowledge-Aware Federated Active Learning with Non-IID Datacited by 16
  22. 2022Federated Active Learning (F-AL): an Efficient Annotation Strategy for Federated Learningcited by 14
  23. 2023Re-thinking Federated Active Learning based on Inter-class Diversitycited by 14
  24. 2022Optimising Communication Overhead in Federated Learning Using NSGA-IIcited by 7
  25. 2013Convex and Scalable Weakly Labeled SVMscited by 6
  26. 2020Combining Federated and Active Learning for Communication-efficient Distributed Failure Prediction in Aeronauticscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025FedPromo: Federated Lightweight Proxy Models at the Edge Bring New Domains to Foundation Modelscited by 2
  2. 2025Helmsman: Autonomous Synthesis of Federated Learning Systems via Collaborative LLM Agentscited by 1
  3. 2026FIDAL: Diversity-Aware Federated Active Learning Under Real-World Distribution Shiftscited by 0