mlsanityData to 14 Oct 2026

Partial Variable Training for Efficient On-Device Federated Learning

arXiv 2110.05607 ↗First posted 11 Oct 2021cs.LG cs.SD eess.AScited by 14

Not on the map. Why?

Abstract (arXiv)

This paper aims to address the major challenges of Federated Learning (FL) on edge devices: limited memory and expensive communication. We propose a novel method, called Partial Variable Training (PVT), that only trains a small subset of variables on edge devices to reduce memory usage and communication cost. With PVT, we show that network accuracy can be maintained by utilizing more local training steps and devices, which is favorable for FL involving a large population of devices. According to our experiments on two state-of-the-art neural networks for speech recognition and two different datasets, PVT can reduce memory usage by up to 1.9× and communication cost by up to 593× while attaining comparable accuracy when compared with full network training.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 17 of its 21 references that are in the map. First posted on arXiv 11 Oct 2021.

Builds on

17 of 21 references.

References · 17

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  6. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  7. 2016Training Deep Nets with Sublinear Memory Costcited by 848
  8. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  9. 2021A Field Guide to Federated Optimizationcited by 315
  10. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
  11. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 163
  12. 2016Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruningcited by 134
5 more
  1. 2019Are All Layers Created Equal?cited by 94
  2. 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
  3. 2020A Better and Faster End-to-End Model for Streaming ASRcited by 69
  4. 2021NetAdaptV2: Efficient Neural Architecture Search with Fast Super-Network Training and Architecture Optimizationcited by 13
  5. 2019Distributed Fixed Point Methods with Compressed Iteratescited by 11

Built on by

Cited by year

Citing papers per year: 2022: 8; 2023: 3; 2024: 3.

14 papers.

Sort:

  1. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  2. 2023TimelyFL: Heterogeneity-aware Asynchronous Federated Learning with Adaptive Partial Trainingcited by 27
  3. 2022Scaling Language Model Size in Cross-Device Federated Learningcited by 25
  4. 2022A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairnesscited by 13
  5. 2022Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learningcited by 11
  6. 2022CoCoFL: Communication- and Computation-Aware Federated Learning via Partial NN Freezing and Quantizationcited by 7
  7. 2022Online Model Compression for Federated Learning with Large Modelscited by 7
  8. 2022Federated Domain Adaptation for ASR with Full Self-Supervisioncited by 6
  9. 2022Decoupled Federated Learning for ASR with Non-IID Datacited by 6
  10. 2023Towards a Better Theoretical Understanding of Independent Subnetwork Trainingcited by 6
  11. 2024Why Go Full? Elevating Federated Learning Through Partial Network Updatescited by 6
  12. 2023Federated Learning Over Images: Vertical Decompositions and Pre-Trained Backbones Are Difficult to Beatcited by 4
2 more
  1. 2024Feature-based Federated Transfer Learning: Communication Efficiency, Robustness and Privacycited by 1
  2. 2024Federated Learning of Large ASR Models in the Real Worldcited by 0