mlsanityData to 7 Oct 2026

Joint Superposition Coding and Training for Federated Learning over Multi-Width Neural Networks

arXiv 2112.02543 ↗First posted 5 Dec 2021cs.LG cs.DCcited by 14

Not on the map. Why?

Abstract (arXiv)

This paper aims to integrate two synergetic technologies, federated learning (FL) and width-adjustable slimmable neural network (SNN) architectures. FL preserves data privacy by exchanging the locally trained models of mobile devices. By adopting SNNs as local models, FL can flexibly cope with the time-varying energy capacities of mobile devices. Combining FL and SNNs is however non-trivial, particularly under wireless connections with time-varying channel conditions. Furthermore, existing multi-width SNN training algorithms are sensitive to the data distributions across devices, so are ill-suited to FL. Motivated by this, we propose a communication and energy-efficient SNN-based FL (named SlimFL) that jointly utilizes superposition coding (SC) for global model aggregation and superposition training (ST) for updating local models. By applying SC, SlimFL exchanges the superposition of multiple width configurations that are decoded as many as possible for a given communication throughput. Leveraging ST, SlimFL aligns the forward propagation of different width configurations, while avoiding the inter-width interference during backpropagation. We formally prove the convergence of SlimFL. The result reveals that SlimFL is not only communication-efficient but also can counteract non-IID data distributions and poor channel conditions, which is also corroborated by simulations.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 13 of its 36 references that are in the map. First posted on arXiv 5 Dec 2021.

Builds on

13 of 36 references.

References · 13

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  5. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  6. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  7. 2021A Field Guide to Federated Optimizationcited by 312
  8. 2018Slimmable Neural Networkscited by 301
  9. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  10. 2019Universally Slimmable Networks and Improved Training Techniquescited by 219
  11. 2018Wireless Network Intelligence at the Edgecited by 90
  12. 2020Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applicationscited by 62
1 more
  1. 2020Measuring the Algorithmic Efficiency of Neural Networkscited by 51

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 8; 2023: 4; 2024: 0; 2025: 0; 2026: 1.

14 papers.

Sort:

By area:

  1. 2022Slimmable Quantum Federated Learningcited by 23
  2. 2022Quantum Distributed Deep Learning Architectures: Models, Discussions, and Applicationscited by 12
  3. 2023AnycostFL: Efficient On-Demand Federated Learning over Heterogeneous Edge Devicescited by 10
  4. 2022SlimFL: Federated Learning with Superposition Coding over Slimmable Neural Networkscited by 6
  5. 2022Cooperative Multi-Agent Deep Reinforcement Learning for Reliable and Energy-Efficient Mobile Access via Multi-UAV Controlcited by 5
  6. 2023Internal Cross-layer Gradients for Extending Homogeneity to Heterogeneity in Federated Learningcited by 5
  7. 2022Quantum Federated Learning with Entanglement Controlled Circuits and Superposition Codingcited by 4
  8. 2022Two-stage architectural fine-tuning with neural architecture search using early-stopping in image classificationcited by 2
  9. 2022Exploiting Features and Logits in Heterogeneous Federated Learningcited by 2
  10. 2023Straggler-resilient Federated Learning: Tackling Computation Heterogeneity with Layer-wise Partial Model Training in Mobile Edge Networkcited by 2
  11. 2021Parallelized and Randomized Adversarial Imitation Learning for Safety-Critical Self-Driving Vehiclescited by 1
  12. 2023FedIN: Federated Intermediate Layers Learning for Model Heterogeneitycited by 1
2 more
  1. 2022Search Space Adaptation for Differentiable Neural Architecture Search in Image Classificationcited by 0
  2. 2026Federated Explainable Artificial Intelligence: Roles, Architectures, Evaluation, and Open Challengescited by 0