mlsanityData to 7 Oct 2026

CoCoFL: Communication- and Computation-Aware Federated Learning via Partial NN Freezing and Quantization

arXiv 2203.05468 ↗First posted 10 Mar 2022cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Devices participating in federated learning (FL) typically have heterogeneous communication, computation, and memory resources. However, in synchronous FL, all devices need to finish training by the same deadline dictated by the server. Our results show that training a smaller subset of the neural network (NN) at constrained devices, i.e., dropping neurons/filters as proposed by state of the art, is inefficient, preventing these devices to make an effective contribution to the model. This causes unfairness w.r.t the achievable accuracies of constrained devices, especially in cases with a skewed distribution of class labels across devices. We present a novel FL technique, CoCoFL, which maintains the full NN structure on all devices. To adapt to the devices' heterogeneous resources, CoCoFL freezes and quantizes selected layers, reducing communication, computation, and memory requirements, whereas other layers are still trained in full precision, enabling to reach a high accuracy. Thereby, CoCoFL efficiently utilizes the available resources on devices and allows constrained devices to make a significant contribution to the FL system, increasing fairness among participants (accuracy parity) and significantly improving the final accuracy of the model.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 31 of its 44 references that are in the map. First posted on arXiv 10 Mar 2022.

Builds on

31 of 44 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2016Densely Connected Convolutional Networkscited by 6,691
  7. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  9. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  10. 2018Group Normalizationcited by 1,514
  11. 2017Mixed Precision Trainingcited by 1,135
  12. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
19 more
  1. 2019Towards Federated Learning at Scale: System Designcited by 826
  2. 2018LEAF: A Benchmark for Federated Settingscited by 780
  3. 2015Deep Learning with Limited Numerical Precisioncited by 368
  4. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  5. 2019Asynchronous Federated Optimizationcited by 264
  6. 2017Recent Trends in Deep Learning Based Natural Language Processingcited by 262
  7. 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
  8. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
  9. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  10. 2019Energy Efficient Federated Learning Over Wireless Communication Networkscited by 181
  11. 2020Communication-Efficient Edge AI: Algorithms and Systemscited by 95
  12. 2018A Survey on Methods and Theories of Quantized Neural Networkscited by 75
  13. 2017Training Quantized Nets: A Deeper Understandingcited by 74
  14. 2019On-Device Machine Learning: An Algorithms and Learning Theory Perspectivecited by 63
  15. 2021Partial Variable Training for Efficient On-Device Federated Learningcited by 14
  16. 2019Compressing RNNs for IoT devices by 15-38x using Kronecker Productscited by 11
  17. 2020Residual Knowledge Distillationcited by 11
  18. 2021DISTREAL: Distributed Resource-Aware Learning in Heterogeneous Systemscited by 10
  19. 2020New Directions in Distributed Deep Learning: Bringing the Network at Forefront of IoT Designcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 2; 2026: 1.

7 papers.

  1. 2023Federated Learning for Computationally-Constrained Heterogeneous Devices: A Surveycited by 40
  2. 2023FLrce: Resource-Efficient Federated Learning with Early-Stopping Strategycited by 10
  3. 2024Why Go Full? Elevating Federated Learning Through Partial Network Updatescited by 6
  4. 2024Efficient Federated Finetuning of Tiny Transformers with Resource-Constrained Devicescited by 1
  5. 2025FedBiF: Communication-Efficient Federated Learning via Bits Freezingcited by 1
  6. 2025Energy and Memory-Efficient Federated Learning With Ordered Layer Freezingcited by 1
  7. 2026RIFLE: Robust Distillation-based FL for Deep Model Deployment on Resource-Constrained IoT Networkscited by 0