mlsanityData to 14 Oct 2026

Budgeted Online Model Selection and Fine-Tuning via Federated Learning

arXiv 2401.10478 ↗First posted 19 Jan 2024cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Online model selection involves selecting a model from a set of candidate models 'on the fly' to perform prediction on a stream of data. The choice of candidate models henceforth has a crucial impact on the performance. Although employing a larger set of candidate models naturally leads to more flexibility in model selection, this may be infeasible in cases where prediction tasks are performed on edge devices with limited memory. Faced with this challenge, the present paper proposes an online federated model selection framework where a group of learners (clients) interacts with a server with sufficient memory such that the server stores all candidate models. However, each client only chooses to store a subset of models that can be fit into its memory and performs its own prediction task using one of the stored models. Furthermore, employing the proposed algorithm, clients and the server collaborate to fine-tune models to adapt them to a non-stationary environment. Theoretical analysis proves that the proposed algorithm enjoys sub-linear regret with respect to the best model in hindsight. Experiments on real datasets demonstrate the effectiveness of the proposed algorithm.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 36 of its 63 references that are in the map. First posted on arXiv 19 Jan 2024.

Builds on

36 of 63 references.

References · 36

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  4. 2017Federated Multi-Task Learningcited by 672
  5. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  6. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  7. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  8. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  9. 2020Adaptive Personalized Federated Learningcited by 358
  10. 2021Personalized Federated Learning using Hypernetworkscited by 212
  11. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  12. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
24 more
  1. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 179
  2. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 123
  3. 2018LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learningcited by 96
  4. 2015Online Learning with Feedback Graphs: Beyond Banditscited by 80
  5. 2020Model Selection in Contextual Stochastic Bandit Problemscited by 71
  6. 2020An Efficiency-boosting Client Selection Scheme for Federated Learning with Fairness Guaranteecited by 62
  7. 2022Client Selection in Federated Learning: Principles, Challenges, and Opportunitiescited by 62
  8. 2019Model selection for contextual banditscited by 60
  9. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
  10. 2017Parameter-free online learning via model selectioncited by 34
  11. 2020Online Model Selection for Reinforcement Learning with Function Approximationcited by 28
  12. 2016Online Learning with Feedback Graphs Without the Graphscited by 27
  13. 2020FLeet: Online Federated Learning via Staleness Awareness and Performance Predictioncited by 21
  14. 2020Online Active Model Selection for Pre-trained Classifierscited by 20
  15. 2021Communication-Efficient Online Federated Learning Framework for Nonlinear Regressioncited by 16
  16. 2022Best of Both Worlds Model Selectioncited by 11
  17. 2023Personalized Online Federated Learning with Multiple Kernelscited by 11
  18. 2022A Snapshot of the Frontiers of Client Selection in Federated Learningcited by 9
  19. 2018Best of many worlds: Robust model selection for online supervised learningcited by 5
  20. 2021Universal and data-adaptive algorithms for model selection in linear contextual banditscited by 5
  21. 2015A Logarithmic Additive Integrality Gap for Bin Packingcited by 3
  22. 2021Graph-Aided Online Multi-Kernel Learningcited by 2
  23. 2021Model Selection for Production System via Automated Online Experimentscited by 2
  24. 2021Online Learning with Uncertain Feedback Graphscited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Personalized Federated Learning with Mixture of Models for Adaptive Prediction and Model Fine-Tuningcited by 8