mlsanityData to 7 Oct 2026

AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization

arXiv 2109.12519 ↗First posted 26 Sep 2021cs.LG cs.DCcited by 16

Not on the map. Why?

Abstract (arXiv)

Vertical federated learning (VFL) is an effective paradigm of training the emerging cross-organizational (e.g., different corporations, companies and organizations) collaborative learning with privacy preserving. Stochastic gradient descent (SGD) methods are the popular choices for training VFL models because of the low per-iteration computation. However, existing SGD-based VFL algorithms are communication-expensive due to a large number of communication rounds. Meanwhile, most existing VFL algorithms use synchronous computation which seriously hamper the computation resource utilization in real-world applications. To address the challenges of communication and computation resource utilization, we propose an asynchronous stochastic quasi-Newton (AsySQN) framework for VFL, under which three algorithms, i.e. AsySQN-SGD, -SVRG and -SAGA, are proposed. The proposed AsySQN-type algorithms making descent steps scaled by approximate (without calculating the inverse Hessian matrix explicitly) Hessian information convergence much faster than SGD-based methods in practice and thus can dramatically reduce the number of communication rounds. Moreover, the adopted asynchronous computation can make better use of the computation resource. We theoretically prove the convergence rates of our proposed algorithms for strongly convex problems. Extensive numerical experiments on real-word datasets demonstrate the lower communication costs and better computation resource utilization of our algorithms compared with state-of-the-art VFL algorithms.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 19 of its 31 references that are in the map. First posted on arXiv 26 Sep 2021.

Builds on

19 of 31 references.

References · 19

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2019Federated Machine Learning: Concept and Applicationscited by 409
  4. 2018Federated Learning via Over-the-Air Computationcited by 215
  5. 2017Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryptioncited by 174
  6. 2019SecureBoost: A Lossless Federated Learning Frameworkcited by 174
  7. 2013Mini-batch Stochastic Approximation Methods for Nonconvex Stochastic Composite Optimizationcited by 142
  8. 2014A Stochastic Quasi-Newton Method for Large-Scale Optimizationcited by 68
  9. 2019HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learningcited by 51
  10. 2019Federated Forestcited by 49
  11. 2019A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regressioncited by 37
  12. 2021Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updatingcited by 37
7 more
  1. 2020Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Datacited by 35
  2. 2019A Communication Efficient Collaborative Learning Framework for Distributed Featurescited by 21
  3. 2020Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learningcited by 19
  4. 2020Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimizationcited by 10
  5. 2018Feature-Distributed SVRG for High-Dimensional Linear Classificationcited by 8
  6. 2020Faster Stochastic Quasi-Newton Methodscited by 8
  7. 2019Zeroth-Order Stochastic Alternating Direction Method of Multipliers for Nonconvex Nonsmooth Optimizationcited by 3

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 5; 2023: 3; 2024: 4; 2025: 3.

16 papers.

Sort:

  1. 2022Vertical Federated Learning: Concepts, Advances and Challengescited by 104
  2. 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
  3. 2023A Survey on Vertical Federated Learning: From a Layered Perspectivecited by 30
  4. 2022Vertical Federated Learning: A Structured Literature Reviewcited by 17
  5. 2022Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updatescited by 15
  6. 2024Vertical Federated Learning for Effectiveness, Security, Applicability: A Surveycited by 14
  7. 2022Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithmcited by 9
  8. 2023VertiBench: Advancing Feature Distribution Diversity in Vertical Federated Learning Benchmarkscited by 6
  9. 2024A Survey on Contribution Evaluation in Vertical Federated Learningcited by 5
  10. 2022Enabling All In-Edge Deep Learning: A Literature Reviewcited by 4
  11. 2025Vertical Federated Learning in Practice: The Good, the Bad, and the Uglycited by 4
  12. 2021DACFL: Dynamic Average Consensus Based Federated Learning in Decentralized Topologycited by 3
4 more
  1. 2024Unity is Power: Semi-Asynchronous Collaborative Training of Large-Scale Models with Structured Pruning in Resource-Limited Clientscited by 3
  2. 2024Vertical Federated Learning Hybrid Local Pre-trainingcited by 1
  3. 2025Event-Driven Online Vertical Federated Learningcited by 0
  4. 2025PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecturecited by 0