# Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm

arXiv: https://arxiv.org/abs/2203.10329 · HTML: https://mlsanity.com/paper/2203.10329/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 19 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption](https://mlsanity.com/paper/1711.10677/) (2017)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [An Optimal Algorithm for Bandit and Zero-Order Convex Optimization with Two-Point Feedback](https://mlsanity.com/paper/1507.08752/) (2015)
- [Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimization](https://mlsanity.com/paper/1805.10367/) (2018)
- [Feature Inference Attack on Model Predictions in Vertical Federated Learning](https://mlsanity.com/paper/2010.10152/) (2020)
- [VAFL: a Method of Vertical Asynchronous Federated Learning](https://mlsanity.com/paper/2007.06081/) (2020)
- [HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning](https://mlsanity.com/paper/1912.05897/) (2019)
- [A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression](https://mlsanity.com/paper/1912.00513/) (2019)
- [Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating](https://mlsanity.com/paper/2103.00958/) (2021)
- [Backdoor attacks and defenses in feature-partitioned collaborative learning](https://mlsanity.com/paper/2007.03608/) (2020)
- [A Communication Efficient Collaborative Learning Framework for Distributed Features](https://mlsanity.com/paper/1912.11187/) (2019)
- [Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning](https://mlsanity.com/paper/2008.06233/) (2020)
- [AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization](https://mlsanity.com/paper/2109.12519/) (2021)
- [Boosting Privately: Privacy-Preserving Federated Extreme Boosting for Mobile Crowdsensing](https://mlsanity.com/paper/1907.10218/) (2019)
- [Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization](https://mlsanity.com/paper/2008.01296/) (2020)
- [Faster Stochastic Quasi-Newton Methods](https://mlsanity.com/paper/2004.06479/) (2020)
- [Feature-Distributed SVRG for High-Dimensional Linear Classification](https://mlsanity.com/paper/1802.03604/) (2018)
- [Zeroth-Order Stochastic Alternating Direction Method of Multipliers for Nonconvex Nonsmooth Optimization](https://mlsanity.com/paper/1905.12729/) (2019)
- [Sparse Stochastic Zeroth-Order Optimization with an Application to Bandit Structured Prediction](https://mlsanity.com/paper/1806.04458/) (2018)

## Built on by (most cited first, 9 of 9)

- [A Survey on Vertical Federated Learning: From a Layered Perspective](https://mlsanity.com/paper/2304.01829/) (2023)
- [On the Convergence of Zeroth-Order Federated Tuning for Large Language Models](https://mlsanity.com/paper/2402.05926/) (2024)
- [Vertical Federated Learning: Taxonomies, Threats, and Prospects](https://mlsanity.com/paper/2302.01550/) (2023)
- [FedZeN: Towards superlinear zeroth-order federated learning via incremental Hessian estimation](https://mlsanity.com/paper/2309.17174/) (2023)
- [FedRN: Exploiting k-Reliable Neighbors Towards Robust Federated Learning](https://mlsanity.com/paper/2205.01310/) (2022)
- [Secure and Fast Asynchronous Vertical Federated Learning via Cascaded Hybrid Optimization](https://mlsanity.com/paper/2306.16077/) (2023)
- [Lean Clients, Full Accuracy: Hybrid Zeroth- and First-Order Split Federated Learning](https://mlsanity.com/paper/2601.09076/) (2026)
- [Communication-Efficient and Differentially Private Vertical Federated Learning with Zeroth-Order Optimization](https://mlsanity.com/paper/2502.20565/) (2025)
- [Event-Driven Online Vertical Federated Learning](https://mlsanity.com/paper/2506.14911/) (2025)

## Abstract (arXiv)

Vertical federated learning (VFL) attracts increasing attention due to the emerging demands of multi-party collaborative modeling and concerns of privacy leakage. A complete list of metrics to evaluate VFL algorithms should include model applicability, privacy security, communication cost, and computation efficiency, where privacy security is especially important to VFL. However, to the best of our knowledge, there does not exist a VFL algorithm satisfying all these criteria very well. To address this challenging problem, in this paper, we reveal that zeroth-order optimization (ZOO) is a desirable companion for VFL. Specifically, ZOO can 1) improve the model applicability of VFL framework, 2) prevent VFL framework from privacy leakage under curious, colluding, and malicious threat models, 3) support inexpensive communication and efficient computation. Based on that, we propose a novel and practical VFL framework with black-box models, which is inseparably interconnected to the promising properties of ZOO. We believe that it takes one stride towards designing a practical VFL framework matching all the criteria. Under this framework, we raise two novel bf asynchronous zebf roth-ordbf er algorithms for bf vertical fbf ederated bf learning (AsyREVEL) with different smoothing techniques. We theoretically drive the convergence rates of AsyREVEL algorithms under nonconvex condition. More importantly, we prove the privacy security of our proposed framework under existing VFL attacks on different levels. Extensive experiments on benchmark datasets demonstrate the favorable model applicability, satisfied privacy security, inexpensive communication, efficient computation, scalability and losslessness of our framework.