# Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating

arXiv: https://arxiv.org/abs/2103.00958 · HTML: https://mlsanity.com/paper/2103.00958/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 36 references that are in the map.
- First posted on arXiv 1 Mar 2021.

## Builds on (most cited first, 17 of 17)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives](https://mlsanity.com/paper/1407.0202/) (2014)
- [Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption](https://mlsanity.com/paper/1711.10677/) (2017)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization](https://mlsanity.com/paper/1506.08272/) (2015)
- [HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning](https://mlsanity.com/paper/1912.05897/) (2019)
- [Federated Forest](https://mlsanity.com/paper/1905.10053/) (2019)
- [A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression](https://mlsanity.com/paper/1912.00513/) (2019)
- [Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization](https://mlsanity.com/paper/1607.01231/) (2016)
- [Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data](https://mlsanity.com/paper/2008.06197/) (2020)
- [Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning](https://mlsanity.com/paper/2008.06233/) (2020)
- [ASAGA: Asynchronous Parallel SAGA](https://mlsanity.com/paper/1606.04809/) (2016)
- [Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization](https://mlsanity.com/paper/2008.01296/) (2020)
- [Faster Stochastic Quasi-Newton Methods](https://mlsanity.com/paper/2004.06479/) (2020)
- [Feature-Distributed SVRG for High-Dimensional Linear Classification](https://mlsanity.com/paper/1802.03604/) (2018)

## Built on by (most cited first, 30 of 37)

- [Vertical Federated Learning: Concepts, Advances and Challenges](https://mlsanity.com/paper/2211.12814/) (2022)
- [Vertical Federated Learning: Challenges, Methodologies and Experiments](https://mlsanity.com/paper/2202.04309/) (2022)
- [A Survey on Heterogeneous Federated Learning](https://mlsanity.com/paper/2210.04505/) (2022)
- [BlindFL: Vertical Federated Machine Learning without Peeking into Your Data](https://mlsanity.com/paper/2206.07975/) (2022)
- [FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data](https://mlsanity.com/paper/2103.03918/) (2021)
- [A Survey on Vertical Federated Learning: From a Layered Perspective](https://mlsanity.com/paper/2304.01829/) (2023)
- [FairVFL: A Fair Vertical Federated Learning Framework with Contrastive Adversarial Learning](https://mlsanity.com/paper/2206.03200/) (2022)
- [Practical Vertical Federated Learning with Unsupervised Representation Learning](https://mlsanity.com/paper/2208.10278/) (2022)
- [Vertical Federated Learning: A Structured Literature Review](https://mlsanity.com/paper/2212.00622/) (2022)
- [FedVS: Straggler-Resilient and Privacy-Preserving Vertical Federated Learning for Split Models](https://mlsanity.com/paper/2304.13407/) (2023)
- [AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization](https://mlsanity.com/paper/2109.12519/) (2021)
- [Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates](https://mlsanity.com/paper/2207.14628/) (2022)
- [H-FL: A Hierarchical Communication-Efficient and Privacy-Protected Architecture for Federated Learning](https://mlsanity.com/paper/2106.00275/) (2021)
- [Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey](https://mlsanity.com/paper/2405.17495/) (2024)
- [Communication-Efficient Vertical Federated Learning with Limited Overlapping Samples](https://mlsanity.com/paper/2303.16270/) (2023)
- [Improving Privacy-Preserving Vertical Federated Learning by Efficient Communication with ADMM](https://mlsanity.com/paper/2207.10226/) (2022)
- [Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm](https://mlsanity.com/paper/2203.10329/) (2022)
- [Achieving Model Fairness in Vertical Federated Learning](https://mlsanity.com/paper/2109.08344/) (2021)
- [Continual Horizontal Federated Learning for Heterogeneous Data](https://mlsanity.com/paper/2203.02108/) (2022)
- [Vertical Federated Learning: Taxonomies, Threats, and Prospects](https://mlsanity.com/paper/2302.01550/) (2023)
- [Robust and IP-Protecting Vertical Federated Learning against Unexpected Quitting of Parties](https://mlsanity.com/paper/2303.18178/) (2023)
- [A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective](https://mlsanity.com/paper/2402.03688/) (2024)
- [Secure Forward Aggregation for Vertical Federated Neural Networks](https://mlsanity.com/paper/2207.00165/) (2022)
- [VertiBench: Advancing Feature Distribution Diversity in Vertical Federated Learning Benchmarks](https://mlsanity.com/paper/2307.02040/) (2023)
- [FedGBF: An efficient vertical federated learning framework via gradient boosting and bagging](https://mlsanity.com/paper/2204.00976/) (2022)
- [A Vertical Federated Learning Framework for Horizontally Partitioned Labels](https://mlsanity.com/paper/2106.10056/) (2021)
- [Enabling All In-Edge Deep Learning: A Literature Review](https://mlsanity.com/paper/2204.03326/) (2022)
- [Federated Learning for 6G: Paradigms, Taxonomy, Recent Advances and Insights](https://mlsanity.com/paper/2312.04688/) (2023)
- [Secure and Fast Asynchronous Vertical Federated Learning via Cascaded Hybrid Optimization](https://mlsanity.com/paper/2306.16077/) (2023)
- [Robust Collaborative Inference with Vertically Split Data Over Dynamic Device Environments](https://mlsanity.com/paper/2312.16638/) (2023)

## Abstract (arXiv)

Vertical federated learning (VFL) attracts increasing attention due to the emerging demands of multi-party collaborative modeling and concerns of privacy leakage. In the real VFL applications, usually only one or partial parties hold labels, which makes it challenging for all parties to collaboratively learn the model without privacy leakage. Meanwhile, most existing VFL algorithms are trapped in the synchronous computations, which leads to inefficiency in their real-world applications. To address these challenging problems, we propose a novel bf VFL framework integrated with new bf backward updating mechanism and bf bilevel asynchronous parallel architecture (VFB2), under which three new algorithms, including VFB2-SGD, -SVRG, and -SAGA, are proposed. We derive the theoretical results of the convergence rates of these three algorithms under both strongly convex and nonconvex conditions. We also prove the security of VFB2 under semi-honest threat models. Extensive experiments on benchmark datasets demonstrate that our algorithms are efficient, scalable and lossless.