mlsanityData to 14 Oct 2026

Federated Neural Architecture Search with Model-Agnostic Meta Learning

arXiv 2504.06457 ↗First posted 8 Apr 2025cs.LG cs.AI cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) often struggles with data heterogeneity due to the naturally uneven distribution of user data across devices. Federated Neural Architecture Search (NAS) enables collaborative search for optimal model architectures tailored to heterogeneous data to achieve higher accuracy. However, this process is time-consuming due to extensive search space and retraining. To overcome this, we introduce FedMetaNAS, a framework that integrates meta-learning with NAS within the FL context to expedite the architecture search by pruning the search space and eliminating the retraining stage. Our approach first utilizes the Gumbel-Softmax reparameterization to facilitate relaxation of the mixed operations in the search space. We then refine the local search process by incorporating Model-Agnostic Meta-Learning, where a task-specific learner adapts both weights and architecture parameters (alphas) for individual tasks, while a meta learner adjusts the overall model weights and alphas based on the gradient information from task learners. Following the meta-update, we propose soft pruning using the same trick on search space to gradually sparsify the architecture, ensuring that the performance of the chosen architecture remains robust after pruning which allows for immediate use of the model without retraining. Experimental evaluations demonstrate that FedMetaNAS significantly accelerates the search process by more than 50% with higher accuracy compared to FedNAS.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 39 references that are in the map. First posted on arXiv 8 Apr 2025.

Builds on

25 of 39 references.

References · 25

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2019Searching for MobileNetV3cited by 1,994
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  7. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  8. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  9. 2018LEAF: A Benchmark for Federated Settingscited by 792
  10. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  11. 2019Federated Learning with Personalization Layerscited by 541
  12. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
13 more
  1. 2021Federated Learning on Non-IID Data: A Surveycited by 346
  2. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  3. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 300
  4. 2018SNAS: Stochastic Neural Architecture Searchcited by 289
  5. 2016UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memorycited by 200
  6. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 182
  7. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 155
  8. 2017Genetic CNNcited by 124
  9. 2019Meta-Learning of Neural Architectures for Few-Shot Learningcited by 73
  10. 2020DSNAS: Direct Neural Architecture Search without Parameter Retrainingcited by 69
  11. 2020MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulationcited by 57
  12. 2020MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learningcited by 50
  13. 2024FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuningcited by 27

Built on by

Not cited yet.