# Communication-Efficient and Differentially Private Vertical Federated Learning with Zeroth-Order Optimization

arXiv: https://arxiv.org/abs/2502.20565 · HTML: https://mlsanity.com/paper/2502.20565/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 46 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [Fine-Tuning Language Models with Just Forward Passes](https://mlsanity.com/paper/2305.17333/) (2023)
- [Why gradient clipping accelerates training: A theoretical justification for adaptivity](https://mlsanity.com/paper/1905.11881/) (2019)
- [Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption](https://mlsanity.com/paper/1711.10677/) (2017)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [Understanding Gradient Clipping in Private SGD: A Geometric Perspective](https://mlsanity.com/paper/2006.15429/) (2020)
- [An Optimal Algorithm for Bandit and Zero-Order Convex Optimization with Two-Point Feedback](https://mlsanity.com/paper/1507.08752/) (2015)
- [Vertical Federated Learning: Concepts, Advances and Challenges](https://mlsanity.com/paper/2211.12814/) (2022)
- [Byzantine-Resilient Secure Federated Learning](https://mlsanity.com/paper/2007.11115/) (2020)
- [Wireless Network Intelligence at the Edge](https://mlsanity.com/paper/1812.02858/) (2018)
- [Privacy Preserving Vertical Federated Learning for Tree-based Models](https://mlsanity.com/paper/2008.06170/) (2020)
- [VAFL: a Method of Vertical Asynchronous Federated Learning](https://mlsanity.com/paper/2007.06081/) (2020)
- [Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learning](https://mlsanity.com/paper/2201.09531/) (2022)
- [HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning](https://mlsanity.com/paper/1912.05897/) (2019)
- [Private Fine-tuning of Large Language Models with Zeroth-order Optimization](https://mlsanity.com/paper/2401.04343/) (2024)
- [FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data](https://mlsanity.com/paper/2103.03918/) (2021)
- [Backdoor attacks and defenses in feature-partitioned collaborative learning](https://mlsanity.com/paper/2007.03608/) (2020)
- [Federated Learning and Wireless Communications](https://mlsanity.com/paper/2005.05265/) (2020)
- [DPZero: Private Fine-Tuning of Language Models without Backpropagation](https://mlsanity.com/paper/2310.09639/) (2023)
- [Flexible Vertical Federated Learning with Heterogeneous Parties](https://mlsanity.com/paper/2208.12672/) (2022)
- [LESS-VFL: Communication-Efficient Feature Selection for Vertical Federated Learning](https://mlsanity.com/paper/2305.02219/) (2023)
- [Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey](https://mlsanity.com/paper/2405.17495/) (2024)
- [Feature Reconstruction Attacks and Countermeasures of DNN training in Vertical Federated Learning](https://mlsanity.com/paper/2210.06771/) (2022)
- [Improving Privacy-Preserving Vertical Federated Learning by Efficient Communication with ADMM](https://mlsanity.com/paper/2207.10226/) (2022)
- [Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm](https://mlsanity.com/paper/2203.10329/) (2022)
- [Practical Privacy Attacks on Vertical Federated Learning](https://mlsanity.com/paper/2011.09290/) (2020)
- [Cooperative Decentralized Backdoor Attacks on Vertical Federated Learning](https://mlsanity.com/paper/2501.09320/) (2025)
- [Nonparametric Decentralized Detection and Sparse Sensor Selection via Multi-Sensor Online Kernel Scalar Quantization](https://mlsanity.com/paper/2205.10681/) (2022)

## Built on by (most cited first, 1 of 1)

- [Evolution of Optimization Methods: Algorithms, Scenarios, and Evaluations](https://mlsanity.com/paper/2604.12968/) (2026)

## Abstract (arXiv)

Vertical Federated Learning (VFL) enables collaborative model training across feature-partitioned devices, yet its reliance on device-server information exchange introduces significant communication overhead and privacy risks. Downlink communication from the server to devices in VFL exposes gradient-related signals of the global loss that can be leveraged in inference attacks. Existing privacy-preserving VFL approaches that inject differential privacy (DP) noise on the downlink have the natural repercussion of degraded gradient quality, slowed convergence, and excessive communication rounds. In this work, we propose DPZV, a communication-efficient and differentially private ZO-VFL framework with tunable privacy guarantees. Based on zeroth-order (ZO) optimization, DPZV injects calibrated scalar-valued DP noise on the downlink, significantly reducing variance amplification while providing equivalent protection against targeted inference attacks. Through rigorous theoretical analysis, we establish convergence guarantees comparable to first-order DP-SGD, despite relying solely on ZO estimators, and prove that DPZV satisfies (ε, δ)-DP. Extensive experiments demonstrate that DPZV consistently achieves a superior privacy-utility tradeoff and requires fewer communication rounds than existing DP-VFL baselines under strict privacy constraints (ε ≤ 10).