# Federated Neural Architecture Search

arXiv: https://arxiv.org/abs/2002.06352 · HTML: https://mlsanity.com/paper/2002.06352/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2020.

## Built on by (most cited first, 10 of 10)

- [Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing](https://mlsanity.com/paper/2106.04502/) (2021)
- [SPIDER: Searching Personalized Neural Architecture for Federated Learning](https://mlsanity.com/paper/2112.13939/) (2021)
- [A Survey on Computationally Efficient Neural Architecture Search](https://mlsanity.com/paper/2206.01520/) (2022)
- [Federated Progressive Sparsification (Purge, Merge, Tune)+](https://mlsanity.com/paper/2204.12430/) (2022)
- [Single-shot Hyper-parameter Optimization for Federated Learning: A General Algorithm & Analysis](https://mlsanity.com/paper/2202.08338/) (2022)
- [Divide-and-Conquer the NAS puzzle in Resource Constrained Federated Learning Systems](https://mlsanity.com/paper/2305.07135/) (2023)
- [Personalized Federated Instruction Tuning via Neural Architecture Search](https://mlsanity.com/paper/2402.16919/) (2024)
- [Unlocking the Potential of Federated Learning for Deeper Models](https://mlsanity.com/paper/2306.02701/) (2023)
- [FedPop: Federated Population-based Hyperparameter Tuning](https://mlsanity.com/paper/2308.08634/) (2023)
- [Resource-Aware Heterogeneous Federated Learning using Neural Architecture Search](https://mlsanity.com/paper/2211.05716/) (2022)

## Abstract (arXiv)

To preserve user privacy while enabling mobile intelligence, techniques have been proposed to train deep neural networks on decentralized data. However, training over decentralized data makes the design of neural architecture quite difficult as it already was. Such difficulty is further amplified when designing and deploying different neural architectures for heterogeneous mobile platforms. In this work, we propose an automatic neural architecture search into the decentralized training, as a new DNN training paradigm called Federated Neural Architecture Search, namely federated NAS. To deal with the primary challenge of limited on-client computational and communication resources, we present FedNAS, a highly optimized framework for efficient federated NAS. FedNAS fully exploits the key opportunity of insufficient model candidate re-training during the architecture search process, and incorporates three key optimizations: parallel candidates training on partial clients, early dropping candidates with inferior performance, and dynamic round numbers. Tested on large-scale datasets and typical CNN architectures, FedNAS achieves comparable model accuracy as state-of-the-art NAS algorithm that trains models with centralized data, and also reduces the client cost by up to two orders of magnitude compared to a straightforward design of federated NAS.