# Distilling On-Device Intelligence at the Network Edge

arXiv: https://arxiv.org/abs/1908.05895 · HTML: https://mlsanity.com/paper/1908.05895/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2019.

## Built on by (most cited first, 5 of 5)

- [Decentralized and Model-Free Federated Learning: Consensus-Based Distillation in Function Space](https://mlsanity.com/paper/2104.00352/) (2021)
- [Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus](https://mlsanity.com/paper/2101.03300/) (2021)
- [Parallel Split Learning with Global Sampling](https://mlsanity.com/paper/2407.15738/) (2024)
- [Fair and Efficient Distributed Edge Learning with Hybrid Multipath TCP](https://mlsanity.com/paper/2211.09723/) (2022)
- [Federated Learning for Energy Constrained IoT devices: A systematic mapping study](https://mlsanity.com/paper/2301.03720/) (2023)

## Abstract (arXiv)

Devices at the edge of wireless networks are the last mile data sources for machine learning (ML). As opposed to traditional ready-made public datasets, these user-generated private datasets reflect the freshest local environments in real time. They are thus indispensable for enabling mission-critical intelligent systems, ranging from fog radio access networks (RANs) to driverless cars and e-Health wearables. This article focuses on how to distill high-quality on-device ML models using fog computing, from such user-generated private data dispersed across wirelessly connected devices. To this end, we introduce communication-efficient and privacy-preserving distributed ML frameworks, termed fog ML (FML), wherein on-device ML models are trained by exchanging model parameters, model outputs, and surrogate data. We then present advanced FML frameworks addressing wireless RAN characteristics, limited on-device resources, and imbalanced data distributions. Our study suggests that the full potential of FML can be reached by co-designing communication and distributed ML operations while accounting for heterogeneous hardware specifications, data characteristics, and user requirements.