mlsanityData to 14 Oct 2026

Thinking Forward: Memory-Efficient Federated Finetuning of Language Models

arXiv 2405.15551 ↗First posted 24 May 2024cs.LGcited by 7

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Finetuning large language models (LLMs) in federated learning (FL) settings has become increasingly important as it allows resource-constrained devices to finetune a model using private data. However, finetuning LLMs using backpropagation requires excessive memory (especially from intermediate activations) for resource-constrained devices. While Forward-mode Auto-Differentiation (AD) can significantly reduce memory footprint from activations, we observe that directly applying it to LLM finetuning results in slow convergence and poor accuracy. In this paper, we introduce Spry, an FL algorithm that splits trainable weights of an LLM among participating clients, such that each client computes gradients using forward-mode AD that are closer estimations of the true gradients. Spry achieves a low memory footprint, high accuracy, and fast convergence. We formally prove that the global gradients in Spry are unbiased estimators of true global gradients for homogeneous data distributions across clients, while heterogeneity increases bias of the estimates. We also derive Spry's convergence rate, showing that the gradients decrease inversely proportional to the number of FL rounds, indicating the convergence up to the limits of heterogeneity. Empirically, Spry reduces the memory footprint during training by 1.4-7.1x in contrast to backpropagation, while reaching comparable accuracy, across a wide range of language tasks, models, and FL settings. Spry reduces the convergence time by 1.2-20.3x and achieves 5.2-13.5% higher accuracy against zero-order methods. When finetuning Llama2-7B with LoRA, compared to the peak memory consumption of 33.9GB of backpropagation, Spry only consumes 6.2GB of peak memory. For OPT13B, the reduction is from 76.5GB to 10.8GB. Spry makes feasible previously impossible FL deployments on commodity edge devices. Our source code is available at https://github.com/Astuary/Spry.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 18 of its 58 references that are in the map. First posted on arXiv 24 May 2024.

Builds on

18 of 58 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  8. 2023PaLM 2 Technical Reportcited by 1,162
  9. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 656
  10. 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 266
  11. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 136
  12. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 128
6 more
  1. 2022Gradients without Backpropagationcited by 73
  2. 2023FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Modelscited by 63
  3. 2022Motley: Benchmarking Heterogeneity and Personalization in Federated Learningcited by 31
  4. 2023FwdLLM: Efficient FedLLM using Forward Gradientcited by 26
  5. 2023BAFFLE: A Baseline of Backpropagation-Free Federated Learningcited by 12
  6. 2023FedFwd: Federated Learning without Backpropagationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 2.

7 papers.

  1. 2025Distributed LLMs and Multimodal Large Language Models: A Survey on Advances, Challenges, and Future Directionscited by 10
  2. 2024Unity is Power: Semi-Asynchronous Collaborative Training of Large-Scale Models with Structured Pruning in Resource-Limited Clientscited by 3
  3. 2025Memory Savings at What Cost? A Study of Alternatives to Backpropagationcited by 1
  4. 2025Efficient Zeroth-Order Federated Finetuning of Language Models on Resource-Constrained Devicescited by 0
  5. 2025Federated Learning with Ad-hoc Adapter Insertions: The Case of Soft-Embeddings for Training Classifier-as-Retrievercited by 0
  6. 2026CooperLLM: Cloud-Edge-End Cooperative Federated Fine-tuning for LLMs via ZOO-based Gradient Correctioncited by 0
  7. 2026ZorBA: Zeroth-order Federated Fine-tuning of LLMs with Heterogeneous Block Activationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security
  2. 2022Scaling Language Model Size in Cross-Device Federated Learning
  3. 2026FedSDR: Federated Self-Distillation with Rectification
  4. 2024eFedLLM: Efficient LLM Inference Based on Federated Learning
  5. 2025FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models
  6. 2026Conflict-Aware Federated Fine-Tuning of Large Language Models with Mixture-of-Experts
  7. 2025Federated Fine-Tuning of Large Language Models: Kahneman-Tversky vs. Direct Preference Optimization
  8. 2026FLoKD: Adaptive Knowledge Distillation for Federated Low-Rank LLM over Wireless Networks