# AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization

arXiv: https://arxiv.org/abs/2109.12519 · HTML: https://mlsanity.com/paper/2109.12519/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 26 Sep 2021.

## Builds on (most cited first, 19 of 19)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [Federated Learning via Over-the-Air Computation](https://mlsanity.com/paper/1812.11750/) (2018)
- [Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption](https://mlsanity.com/paper/1711.10677/) (2017)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [Mini-batch Stochastic Approximation Methods for Nonconvex Stochastic Composite Optimization](https://mlsanity.com/paper/1308.6594/) (2013)
- [A Stochastic Quasi-Newton Method for Large-Scale Optimization](https://mlsanity.com/paper/1401.7020/) (2014)
- [HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning](https://mlsanity.com/paper/1912.05897/) (2019)
- [Federated Forest](https://mlsanity.com/paper/1905.10053/) (2019)
- [A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression](https://mlsanity.com/paper/1912.00513/) (2019)
- [Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating](https://mlsanity.com/paper/2103.00958/) (2021)
- [Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data](https://mlsanity.com/paper/2008.06197/) (2020)
- [A Communication Efficient Collaborative Learning Framework for Distributed Features](https://mlsanity.com/paper/1912.11187/) (2019)
- [Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning](https://mlsanity.com/paper/2008.06233/) (2020)
- [Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization](https://mlsanity.com/paper/2008.01296/) (2020)
- [Faster Stochastic Quasi-Newton Methods](https://mlsanity.com/paper/2004.06479/) (2020)
- [Feature-Distributed SVRG for High-Dimensional Linear Classification](https://mlsanity.com/paper/1802.03604/) (2018)
- [Zeroth-Order Stochastic Alternating Direction Method of Multipliers for Nonconvex Nonsmooth Optimization](https://mlsanity.com/paper/1905.12729/) (2019)

## Built on by (most cited first, 16 of 16)

- [Vertical Federated Learning: Concepts, Advances and Challenges](https://mlsanity.com/paper/2211.12814/) (2022)
- [Efficient Personalized Federated Learning via Sparse Model-Adaptation](https://mlsanity.com/paper/2305.02776/) (2023)
- [A Survey on Vertical Federated Learning: From a Layered Perspective](https://mlsanity.com/paper/2304.01829/) (2023)
- [Vertical Federated Learning: A Structured Literature Review](https://mlsanity.com/paper/2212.00622/) (2022)
- [Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates](https://mlsanity.com/paper/2207.14628/) (2022)
- [Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey](https://mlsanity.com/paper/2405.17495/) (2024)
- [Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm](https://mlsanity.com/paper/2203.10329/) (2022)
- [VertiBench: Advancing Feature Distribution Diversity in Vertical Federated Learning Benchmarks](https://mlsanity.com/paper/2307.02040/) (2023)
- [A Survey on Contribution Evaluation in Vertical Federated Learning](https://mlsanity.com/paper/2405.02364/) (2024)
- [Enabling All In-Edge Deep Learning: A Literature Review](https://mlsanity.com/paper/2204.03326/) (2022)
- [Vertical Federated Learning in Practice: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2502.08160/) (2025)
- [DACFL: Dynamic Average Consensus Based Federated Learning in Decentralized Topology](https://mlsanity.com/paper/2111.05505/) (2021)
- [Unity is Power: Semi-Asynchronous Collaborative Training of Large-Scale Models with Structured Pruning in Resource-Limited Clients](https://mlsanity.com/paper/2410.08457/) (2024)
- [Vertical Federated Learning Hybrid Local Pre-training](https://mlsanity.com/paper/2405.11884/) (2024)
- [Event-Driven Online Vertical Federated Learning](https://mlsanity.com/paper/2506.14911/) (2025)
- [PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture](https://mlsanity.com/paper/2510.12494/) (2025)

## Abstract (arXiv)

Vertical federated learning (VFL) is an effective paradigm of training the emerging cross-organizational (e.g., different corporations, companies and organizations) collaborative learning with privacy preserving. Stochastic gradient descent (SGD) methods are the popular choices for training VFL models because of the low per-iteration computation. However, existing SGD-based VFL algorithms are communication-expensive due to a large number of communication rounds. Meanwhile, most existing VFL algorithms use synchronous computation which seriously hamper the computation resource utilization in real-world applications. To address the challenges of communication and computation resource utilization, we propose an asynchronous stochastic quasi-Newton (AsySQN) framework for VFL, under which three algorithms, i.e. AsySQN-SGD, -SVRG and -SAGA, are proposed. The proposed AsySQN-type algorithms making descent steps scaled by approximate (without calculating the inverse Hessian matrix explicitly) Hessian information convergence much faster than SGD-based methods in practice and thus can dramatically reduce the number of communication rounds. Moreover, the adopted asynchronous computation can make better use of the computation resource. We theoretically prove the convergence rates of our proposed algorithms for strongly convex problems. Extensive numerical experiments on real-word datasets demonstrate the lower communication costs and better computation resource utilization of our algorithms compared with state-of-the-art VFL algorithms.