# Secure Federated Learning Across Heterogeneous Cloud and High-Performance Computing Resources -- A Case Study on Federated Fine-tuning of LLaMA 2

arXiv: https://arxiv.org/abs/2402.12271 · HTML: https://mlsanity.com/paper/2402.12271/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 14 references that are in the map.
- First posted on arXiv 19 Feb 2024.

## Builds on (most cited first, 7 of 7)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [funcX: A Federated Function Serving Fabric for Science](https://mlsanity.com/paper/2005.04215/) (2020)
- [APPFL: Open-Source Software Framework for Privacy-Preserving Federated Learning](https://mlsanity.com/paper/2202.03672/) (2022)
- [FedCompass: Efficient Cross-Silo Federated Learning on Heterogeneous Client Devices using a Computing Power Aware Scheduler](https://mlsanity.com/paper/2309.14675/) (2023)
- [Enabling End-to-End Secure Federated Learning in Biomedical Research on Heterogeneous Computing Environments with APPFLx](https://mlsanity.com/paper/2312.08701/) (2023)

## Built on by (most cited first, 3 of 3)

- [Advances in APPFL: A Comprehensive and Extensible Federated Learning Framework](https://mlsanity.com/paper/2409.11585/) (2024)
- [FedSpaLLM: Federated Pruning of Large Language Models](https://mlsanity.com/paper/2410.14852/) (2024)
- [Scalable Cross-Facility Federated Learning for Scientific Foundation Models on Multiple Supercomputers](https://mlsanity.com/paper/2603.19544/) (2026)

## Abstract (arXiv)

Federated learning enables multiple data owners to collaboratively train robust machine learning models without transferring large or sensitive local datasets by only sharing the parameters of the locally trained models. In this paper, we elaborate on the design of our Advanced Privacy-Preserving Federated Learning (APPFL) framework, which streamlines end-to-end secure and reliable federated learning experiments across cloud computing facilities and high-performance computing resources by leveraging Globus Compute, a distributed function as a service platform, and Amazon Web Services. We further demonstrate the use case of APPFL in fine-tuning a LLaMA 2 7B model using several cloud resources and supercomputers.