# FEATHERS: Federated Architecture and Hyperparameter Search

arXiv: https://arxiv.org/abs/2206.12342 · HTML: https://mlsanity.com/paper/2206.12342/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 24 Jun 2022.

## Builds on (most cited first, 25 of 25)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [SNAS: Stochastic Neural Architecture Search](https://mlsanity.com/paper/1812.09926/) (2018)
- [Searching for A Robust Neural Architecture in Four GPU Hours](https://mlsanity.com/paper/1910.04465/) (2019)
- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1604.02201/) (2016)
- [Genetic CNN](https://mlsanity.com/paper/1703.01513/) (2017)
- [SGAS: Sequential Greedy Architecture Search](https://mlsanity.com/paper/1912.00195/) (2019)
- [Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing](https://mlsanity.com/paper/2106.04502/) (2021)
- [MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation](https://mlsanity.com/paper/2003.12238/) (2020)
- [From Federated Learning to Federated Neural Architecture Search: A Survey](https://mlsanity.com/paper/2009.05868/) (2020)
- [Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges](https://mlsanity.com/paper/2006.05415/) (2020)
- [Differentially Private Federated Bayesian Optimization with Distributed Exploration](https://mlsanity.com/paper/2110.14153/) (2021)
- [Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Search](https://mlsanity.com/paper/2004.08546/) (2020)
- [Robust Federated Learning Through Representation Matching and Adaptive Hyper-parameters](https://mlsanity.com/paper/1912.13075/) (2019)
- [FLoRA: Single-shot Hyper-parameter Optimization for Federated Learning](https://mlsanity.com/paper/2112.08524/) (2021)
- [Differentially-private Federated Neural Architecture Search](https://mlsanity.com/paper/2006.10559/) (2020)
- [Learning Rate Adaptation for Federated and Differentially Private Learning](https://mlsanity.com/paper/1809.03832/) (2018)
- [Genetic CFL: Optimization of Hyper-Parameters in Clustered Federated Learning](https://mlsanity.com/paper/2107.07233/) (2021)

## Built on by (most cited first, 3 of 3)

- [Efficient and Joint Hyperparameter and Architecture Search for Collaborative Filtering](https://mlsanity.com/paper/2307.11004/) (2023)
- [Hyper-parameter Optimization for Federated Learning with Step-wise Adaptive Mechanism](https://mlsanity.com/paper/2411.12244/) (2024)
- [DP-Hype: Federated Differentially Private Hyperparameter Search](https://mlsanity.com/paper/2510.04902/) (2025)

## Abstract (arXiv)

Deep neural architectures have profound impact on achieved performance in many of today's AI tasks, yet, their design still heavily relies on human prior knowledge and experience. Neural architecture search (NAS) together with hyperparameter optimization (HO) helps to reduce this dependence. However, state of the art NAS and HO rapidly become infeasible with increasing amount of data being stored in a distributed fashion, typically violating data privacy regulations such as GDPR and CCPA. As a remedy, we introduce FEATHERS - FEderated ArchiTecture and HypERparameter Search, a method that not only optimizes both neural architectures and optimization-related hyperparameters jointly in distributed data settings, but further adheres to data privacy through the use of differential privacy (DP). We show that FEATHERS efficiently optimizes architectural and optimization-related hyperparameters alike, while demonstrating convergence on classification tasks at no detriment to model performance when complying with privacy constraints.