# Bridging Memory Gaps: Scaling Federated Learning for Heterogeneous Clients

arXiv: https://arxiv.org/abs/2408.10826 · HTML: https://mlsanity.com/paper/2408.10826/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2024.

## Built on by (most cited first, 3 of 3)

- [Breaking the Memory Wall for Heterogeneous Federated Learning via Model Splitting](https://mlsanity.com/paper/2410.11577/) (2024)
- [Learning Like Humans: Resource-Efficient Federated Fine-Tuning through Cognitive Developmental Stages](https://mlsanity.com/paper/2508.00041/) (2025)
- [FBLayout: Optimizing Memory Layout for Efficient LLM Finetuning on Mobile GPUs](https://mlsanity.com/paper/2607.21624/) (2026)

## Abstract (arXiv)

Federated Learning (FL) enables multiple clients to collaboratively train a shared model while preserving data privacy. However, the high memory demand during model training severely limits the deployment of FL on resource-constrained clients. To this end, we propose our, a scalable and inclusive FL framework designed to overcome memory limitations through sequential block-wise training. The core idea of our is to partition the global model into blocks and train them sequentially, thereby reducing training memory requirements. To mitigate information loss during block-wise training, our introduces a Curriculum Mentor that crafts curriculum-aware training objectives for each block to steer their learning process. Moreover, our incorporates a Training Harmonizer that designs a parameter co-adaptation training scheme to coordinate block updates, effectively breaking inter-block information isolation. Extensive experiments on both simulation and hardware testbeds demonstrate that our significantly improves model performance by up to 84.2%, reduces peak memory usage by up to 50.4%, and accelerates training by up to 1.9×.