mlsanityData to 7 Oct 2026

Empowering Federated Learning for Massive Models with NVIDIA FLARE

arXiv 2402.07792 ↗First posted 12 Feb 2024cs.LG cs.DCcited by 4

Not on the map. Why?

Abstract (arXiv)

In the ever-evolving landscape of artificial intelligence (AI) and large language models (LLMs), handling and leveraging data effectively has become a critical challenge. Most state-of-the-art machine learning algorithms are data-centric. However, as the lifeblood of model performance, necessary data cannot always be centralized due to various factors such as privacy, regulation, geopolitics, copyright issues, and the sheer effort required to move vast datasets. In this paper, we explore how federated learning enabled by NVIDIA FLARE can address these challenges with easy and scalable integration capabilities, enabling parameter-efficient and full supervised fine-tuning of LLMs for natural language processing and biopharmaceutical applications to enhance their accuracy and robustness.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 25 of its 42 references that are in the map. First posted on arXiv 12 Feb 2024.

Builds on

25 of 42 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  10. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  11. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  12. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
13 more
  1. 2020The Future of Digital Health with Federated Learningcited by 625
  2. 2020Federated Learning with Matched Averagingcited by 564
  3. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  4. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
  5. 2018Distributed learning of deep neural network over multiple agentscited by 277
  6. 2019Privacy-preserving Federated Brain Tumour Segmentationcited by 169
  7. 2022NVIDIA FLARE: Federated Learning from Simulation to Real-Worldcited by 78
  8. 2020Federated Learning for Breast Density Classification: A Real-World Implementationcited by 62
  9. 2023Fair Federated Medical Image Segmentation via Client Contribution Estimationcited by 50
  10. 2022Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentationcited by 41
  11. 2022Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentationcited by 16
  12. 2023Communication-Efficient Vertical Federated Learning with Limited Overlapping Samplescited by 13
  13. 2023ConDistFL: Conditional Distillation for Federated Learning from Partially Annotated Datacited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 0; 2026: 1.

4 papers.

  1. 2024Emerging Safety Attack and Defense in Federated Instruction Tuning of Large Language Modelscited by 18
  2. 2024Synergizing Foundation Models and Federated Learning: A Surveycited by 14
  3. 2024A Secure and Trustworthy Network Architecture for Federated Learning Healthcare Applicationscited by 0
  4. 2026A Typed Tensor Language for Shared-State Federated Computationcited by 0