# SPIDER: Searching Personalized Neural Architecture for Federated Learning

arXiv: https://arxiv.org/abs/2112.13939 · HTML: https://mlsanity.com/paper/2112.13939/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 34 references that are in the map.
- First posted on arXiv 27 Dec 2021.

## Builds on (most cited first, 29 of 29)

- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [Bayesian Nonparametric Federated Learning of Neural Networks](https://mlsanity.com/paper/1905.12022/) (2019)
- [HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients](https://mlsanity.com/paper/2010.01264/) (2020)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [Personalized Federated Learning: A Meta-Learning Approach](https://mlsanity.com/paper/2002.07948/) (2020)
- [FedML: A Research Library and Benchmark for Federated Machine Learning](https://mlsanity.com/paper/2007.13518/) (2020)
- [Improving Federated Learning Personalization via Model Agnostic Meta Learning](https://mlsanity.com/paper/1909.12488/) (2019)
- [Federated Learning of a Mixture of Global and Local Models](https://mlsanity.com/paper/2002.05516/) (2020)
- [Federated Evaluation of On-device Personalization](https://mlsanity.com/paper/1910.10252/) (2019)
- [Salvaging Federated Learning by Local Adaptation](https://mlsanity.com/paper/2002.04758/) (2020)
- [A Survey on Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2008.10937/) (2020)
- [Rethinking Architecture Selection in Differentiable NAS](https://mlsanity.com/paper/2108.04392/) (2021)
- [HW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmark](https://mlsanity.com/paper/2103.10584/) (2021)
- [DSNAS: Direct Neural Architecture Search without Parameter Retraining](https://mlsanity.com/paper/2002.09128/) (2020)
- [Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks](https://mlsanity.com/paper/2002.04116/) (2020)
- [Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Search](https://mlsanity.com/paper/2004.08546/) (2020)
- [Real-time Federated Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2003.02793/) (2020)
- [Federated Neural Architecture Search](https://mlsanity.com/paper/2002.06352/) (2020)
- [Differentially-private Federated Neural Architecture Search](https://mlsanity.com/paper/2006.10559/) (2020)

## Built on by (most cited first, 13 of 13)

- [Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey](https://mlsanity.com/paper/2307.09182/) (2023)
- [Pruning Deep Neural Networks from a Sparsity Perspective](https://mlsanity.com/paper/2302.05601/) (2023)
- [Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing](https://mlsanity.com/paper/2502.15618/) (2025)
- [SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overhead](https://mlsanity.com/paper/2406.00431/) (2024)
- [Supernet Training for Federated Image Classification under System Heterogeneity](https://mlsanity.com/paper/2206.01366/) (2022)
- [FedorAS: Federated Architecture Search under system heterogeneity](https://mlsanity.com/paper/2206.11239/) (2022)
- [Divide-and-Conquer the NAS puzzle in Resource Constrained Federated Learning Systems](https://mlsanity.com/paper/2305.07135/) (2023)
- [A Survey of Supernet Optimization and its Applications: Spatial and Temporal Optimization for Neural Architecture Search](https://mlsanity.com/paper/2204.03916/) (2022)
- [FedADP: Unified Model Aggregation for Federated Learning with Heterogeneous Model Architectures](https://mlsanity.com/paper/2505.06497/) (2025)
- [DynamicFL: Federated Learning with Dynamic Communication Resource Allocation](https://mlsanity.com/paper/2409.04986/) (2024)
- [DPFNAS: Differential Privacy-Enhanced Federated Neural Architecture Search for 6G Edge Intelligence](https://mlsanity.com/paper/2509.23030/) (2025)
- [Optimized Federated Knowledge Distillation with Distributed Neural Architecture Search](https://mlsanity.com/paper/2605.21322/) (2026)
- [OrchNAS: Orchestrated Neural Architecture Search Service for Personalised Federated Edge Intelligence](https://mlsanity.com/paper/2607.22805/) (2026)

## Abstract (arXiv)

Federated learning (FL) is an efficient learning framework that assists distributed machine learning when data cannot be shared with a centralized server due to privacy and regulatory restrictions. Recent advancements in FL use predefined architecture-based learning for all the clients. However, given that clients' data are invisible to the server and data distributions are non-identical across clients, a predefined architecture discovered in a centralized setting may not be an optimal solution for all the clients in FL. Motivated by this challenge, in this work, we introduce SPIDER, an algorithmic framework that aims to Search Personalized neural architecture for federated learning. SPIDER is designed based on two unique features: (1) alternately optimizing one architecture-homogeneous global model (Supernet) in a generic FL manner and one architecture-heterogeneous local model that is connected to the global model by weight sharing-based regularization (2) achieving architecture-heterogeneous local model by a novel neural architecture search (NAS) method that can select optimal subnet progressively using operation-level perturbation on the accuracy value as the criterion. Experimental results demonstrate that SPIDER outperforms other state-of-the-art personalization methods, and the searched personalized architectures are more inference efficient.