# Federated Learning-Distillation Alternation for Resource-Constrained IoT

arXiv: https://arxiv.org/abs/2505.20456 · HTML: https://mlsanity.com/paper/2505.20456/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 55 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 26 of 26)

- [FedMD: Heterogenous Federated Learning via Model Distillation](https://mlsanity.com/paper/1910.03581/) (2019)
- [SplitFed: When Federated Learning Meets Split Learning](https://mlsanity.com/paper/2004.12088/) (2020)
- [Distributed learning of deep neural network over multiple agents](https://mlsanity.com/paper/1810.06060/) (2018)
- [Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data](https://mlsanity.com/paper/1811.11479/) (2018)
- [Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Data](https://mlsanity.com/paper/2008.06180/) (2020)
- [Convergence Time Optimization for Federated Learning over Wireless Networks](https://mlsanity.com/paper/2001.07845/) (2020)
- [Federated Knowledge Distillation](https://mlsanity.com/paper/2011.02367/) (2020)
- [Wireless Federated Distillation for Distributed Edge Learning with Heterogeneous Data](https://mlsanity.com/paper/1907.02745/) (2019)
- [Node Selection Toward Faster Convergence for Federated Learning on Non-IID Data](https://mlsanity.com/paper/2105.07066/) (2021)
- [Communication-Efficient Federated Learning with Compensated Overlap-FedAvg](https://mlsanity.com/paper/2012.06706/) (2020)
- [Energy-Efficient Federated Edge Learning with Joint Communication and Computation Design](https://mlsanity.com/paper/2003.00199/) (2020)
- [Mix2FLD: Downlink Federated Learning After Uplink Federated Distillation With Two-Way Mixup](https://mlsanity.com/paper/2006.09801/) (2020)
- [Selective Knowledge Sharing for Privacy-Preserving Federated Distillation without A Good Teacher](https://mlsanity.com/paper/2304.01731/) (2023)
- [FedHAP: Fast Federated Learning for LEO Constellations Using Collaborative HAPs](https://mlsanity.com/paper/2205.07216/) (2022)
- [Federated Learning over Energy Harvesting Wireless Networks](https://mlsanity.com/paper/2106.08809/) (2021)
- [FedDTG:Federated Data-Free Knowledge Distillation via Three-Player Generative Adversarial Networks](https://mlsanity.com/paper/2201.03169/) (2022)
- [Wirelessly Powered Federated Edge Learning: Optimal Tradeoffs Between Convergence and Power Transfer](https://mlsanity.com/paper/2102.12357/) (2021)
- [On the Convergence Time of Federated Learning Over Wireless Networks Under Imperfect CSI](https://mlsanity.com/paper/2104.00331/) (2021)
- [Adaptive Heterogeneous Client Sampling for Federated Learning over Wireless Networks](https://mlsanity.com/paper/2404.13804/) (2024)
- [Federated learning with multichannel ALOHA](https://mlsanity.com/paper/1912.06273/) (2019)
- [Multiple Access in the Era of Distributed Computing and Edge Intelligence](https://mlsanity.com/paper/2403.07903/) (2024)
- [Massive Wireless Energy Transfer: Enabling Sustainable IoT Towards 6G Era](https://mlsanity.com/paper/1912.05322/) (2019)
- [Energy-Sustainable IoT Connectivity: Vision, Technological Enablers, Challenges, and Future Directions](https://mlsanity.com/paper/2306.02444/) (2023)
- [Energy-Aware Federated Learning with Distributed User Sampling and Multichannel ALOHA](https://mlsanity.com/paper/2309.06033/) (2023)
- [Zero-energy Devices for 6G: Technical Enablers at a Glance](https://mlsanity.com/paper/2402.09244/) (2024)
- [From 5G to 6G: Has the Time for Modern Random Access Come?](https://mlsanity.com/paper/1903.03063/) (2019)

## Built on by (most cited first, 2 of 2)

- [Computation-aware Energy-harvesting Federated Learning with Pipelined Cyclic Scheduling](https://mlsanity.com/paper/2511.11949/) (2025)
- [Feature-Based Semantics-Aware Scheduling for Energy-Harvesting Federated Learning](https://mlsanity.com/paper/2512.01983/) (2025)

## Abstract (arXiv)

Federated learning (FL) faces significant challenges in Internet of Things (IoT) networks due to device limitations in energy and communication resources, especially when considering the large size of FL models. From an energy perspective, the challenge is aggravated if devices rely on energy harvesting (EH), as energy availability can vary significantly over time, influencing the average number of participating users in each iteration. Additionally, the transmission of large model updates is more susceptible to interference from uncorrelated background traffic in shared wireless environments. As an alternative, federated distillation (FD) reduces communication overhead and energy consumption by transmitting local model outputs, which are typically much smaller than the entire model used in FL. However, this comes at the cost of reduced model accuracy. Therefore, in this paper, we propose FL-distillation alternation (FLDA). In FLDA, devices alternate between FD and FL phases, balancing model information with lower communication overhead and energy consumption per iteration. We consider a multichannel slotted-ALOHA EH-IoT network subject to background traffic/interference. In such a scenario, FLDA demonstrates higher model accuracy than both FL and FD, and achieves faster convergence than FL. Moreover, FLDA achieves target accuracies saving up to 98% in energy consumption, while also being less sensitive to interference, both relative to FL.