mlsanityData to 13 Oct 2026

Provably Convergent Federated Trilevel Learning

arXiv 2312.11835 ↗First posted 19 Dec 2023cs.LG math.OCcited by 6

Not on the map. Why?

Abstract (arXiv)

Trilevel learning, also called trilevel optimization (TLO), has been recognized as a powerful modelling tool for hierarchical decision process and widely applied in many machine learning applications, such as robust neural architecture search, hyperparameter optimization, and domain adaptation. Tackling TLO problems has presented a great challenge due to their nested decision-making structure. In addition, existing works on TLO face the following key challenges: 1) they all focus on the non-distributed setting, which may lead to privacy breach; 2) they do not offer any non-asymptotic convergence analysis which characterizes how fast an algorithm converges. To address the aforementioned challenges, this paper proposes an asynchronous federated trilevel optimization method to solve TLO problems. The proposed method utilizes μ-cuts to construct a hyper-polyhedral approximation for the TLO problem and solve it in an asynchronous manner. We demonstrate that the proposed μ-cuts are applicable to not only convex functions but also a wide range of non-convex functions that meet the μ-weakly convex assumption. Furthermore, we theoretically analyze the non-asymptotic convergence rate for the proposed method by showing its iteration complexity to obtain ε-stationary point is upper bounded by O(frac1ε2). Extensive experiments on real-world datasets have been conducted to elucidate the superiority of the proposed method, e.g., it has a faster convergence rate with a maximum acceleration of approximately 80%.

Key facts. Cited by 6 papers in the map (data through 13 Oct 2026). Builds on 25 of its 46 references that are in the map. First posted on arXiv 19 Dec 2023.

Builds on

25 of 46 references.

References · 25

  1. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,492
  2. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 404
  3. 2019Asynchronous Federated Optimizationcited by 265
  4. 2020Bilevel Optimization: Convergence Analysis and Enhanced Designcited by 219
  5. 2018Stochastic model-based minimization of weakly convex functionscited by 158
  6. 2017A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applicationscited by 127
  7. 2016On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimizationcited by 111
  8. 2019When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attackscited by 87
  9. 2020Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approachcited by 81
  10. 2021Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimizationcited by 75
  11. 2021A Single-Timescale Method for Stochastic Bilevel Optimizationcited by 69
  12. 2021Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyondcited by 63
13 more
  1. 2022FedNest: Federated Bilevel, Minimax, and Compositional Optimizationcited by 60
  2. 2020A Unified Single-loop Alternating Gradient Projection Algorithm for Nonconvex-Concave and Convex-Nonconcave Minimax Problemscited by 56
  3. 2019Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applicationscited by 55
  4. 2020Advances in Asynchronous Parallel and Distributed Optimizationcited by 40
  5. 2018Robust Optimization over Multiple Domainscited by 34
  6. 2021Meta-Learning to Improve Pre-Trainingcited by 27
  7. 2022Betty: An Automatic Differentiation Library for Multilevel Optimizationcited by 27
  8. 2020Discriminative Feature Alignment: Improving Transferability of Unsupervised Domain Adaptation by Gaussian-guided Latent Alignmentcited by 24
  9. 2022Distributed Distributionally Robust Optimization with Non-Convex Objectivescited by 14
  10. 2021A Gradient Method for Multilevel Optimizationcited by 12
  11. 2021Learning from Mistakes -- A Framework for Neural Architecture Searchcited by 6
  12. 2023FedAL: Black-Box Federated Knowledge Distillation Enabled by Adversarial Learningcited by 5
  13. 2013A Polyhedral Approximation Framework for Convex and Robust Distributed Optimizationcited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 2.

6 papers.

  1. 2025PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimizationcited by 30
  2. 2024Tri-Level Navigator: LLM-Empowered Tri-Level Learning for Time Series OOD Generalizationcited by 12
  3. 2025A stochastic gradient method for trilevel optimizationcited by 2
  4. 2025Argus: Federated Non-convex Bilevel Learning over 6G Space-Air-Ground Integrated Networkcited by 2
  5. 2026CHAL: Council of Hierarchical Agentic Languagecited by 0
  6. 2026First-order Constrained Trilevel Optimization Over Distributed Networks for Robust Coreset Selectioncited by 0