# Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates

arXiv: https://arxiv.org/abs/2207.14628 · HTML: https://mlsanity.com/paper/2207.14628/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 56 references that are in the map.
- First posted on arXiv 29 Jul 2022.

## Builds on (most cited first, 24 of 24)

- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Local SGD Converges Fast and Communicates Little](https://mlsanity.com/paper/1805.09767/) (2018)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [Graph Neural Networks in Recommender Systems: A Survey](https://mlsanity.com/paper/2011.02260/) (2020)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [Random Reshuffling: Simple Analysis with Vast Improvements](https://mlsanity.com/paper/2006.05988/) (2020)
- [Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization](https://mlsanity.com/paper/1910.13598/) (2019)
- [Deep learning with Elastic Averaging SGD](https://mlsanity.com/paper/1412.6651/) (2014)
- [Label Leakage and Protection in Two-party Split Learning](https://mlsanity.com/paper/2102.08504/) (2021)
- [Privacy Preserving Vertical Federated Learning for Tree-based Models](https://mlsanity.com/paper/2008.06170/) (2020)
- [VAFL: a Method of Vertical Asynchronous Federated Learning](https://mlsanity.com/paper/2007.06081/) (2020)
- [Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator](https://mlsanity.com/paper/1911.09824/) (2019)
- [Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating](https://mlsanity.com/paper/2103.00958/) (2021)
- [BlindFL: Vertical Federated Machine Learning without Peeking into Your Data](https://mlsanity.com/paper/2206.07975/) (2022)
- [HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework](https://mlsanity.com/paper/2112.07221/) (2021)
- [A Communication Efficient Collaborative Learning Framework for Distributed Features](https://mlsanity.com/paper/1912.11187/) (2019)
- [Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning](https://mlsanity.com/paper/2008.06233/) (2020)
- [AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization](https://mlsanity.com/paper/2109.12519/) (2021)
- [Natasha: Faster Non-Convex Stochastic Optimization Via Strongly Non-Convex Parameter](https://mlsanity.com/paper/1702.00763/) (2017)
- [Explore User Neighborhood for Real-time E-commerce Recommendation](https://mlsanity.com/paper/2103.00442/) (2021)
- [Differential Privacy Via a Truncated and Normalized Laplace Mechanism](https://mlsanity.com/paper/1911.00602/) (2019)

## Built on by (most cited first, 15 of 15)

- [Vertical Federated Learning: Concepts, Advances and Challenges](https://mlsanity.com/paper/2211.12814/) (2022)
- [FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement](https://mlsanity.com/paper/2304.03946/) (2023)
- [Vertical Federated Knowledge Transfer via Representation Distillation for Healthcare Collaboration Networks](https://mlsanity.com/paper/2302.05675/) (2023)
- [Vertical Federated Learning: A Structured Literature Review](https://mlsanity.com/paper/2212.00622/) (2022)
- [A Survey for Federated Learning Evaluations: Goals and Measures](https://mlsanity.com/paper/2308.11841/) (2023)
- [Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey](https://mlsanity.com/paper/2405.17495/) (2024)
- [Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent](https://mlsanity.com/paper/2303.02868/) (2023)
- [VFLAIR: A Research Library and Benchmark for Vertical Federated Learning](https://mlsanity.com/paper/2310.09827/) (2023)
- [VertiBench: Advancing Feature Distribution Diversity in Vertical Federated Learning Benchmarks](https://mlsanity.com/paper/2307.02040/) (2023)
- [A Survey on Contribution Evaluation in Vertical Federated Learning](https://mlsanity.com/paper/2405.02364/) (2024)
- [Forgetting Any Data at Any Time: A Theoretically Certified Unlearning Framework for Vertical Federated Learning](https://mlsanity.com/paper/2502.17081/) (2025)
- [TablePuppet: A Generic Framework for Relational Federated Learning](https://mlsanity.com/paper/2403.15839/) (2024)
- [WikiDBGraph: A Data Management Benchmark Suite for Collaborative Learning over Database Silos](https://mlsanity.com/paper/2505.16635/) (2025)
- [Event-Driven Online Vertical Federated Learning](https://mlsanity.com/paper/2506.14911/) (2025)
- [X-VFL: A New Vertical Federated Learning Framework with Cross Completion and Decision Subspace Alignment](https://mlsanity.com/paper/2508.05568/) (2025)

## Abstract (arXiv)

Vertical federated learning (VFL) is an emerging paradigm that allows different parties (e.g., organizations or enterprises) to collaboratively build machine learning models with privacy protection. In the training phase, VFL only exchanges the intermediate statistics, i.e., forward activations and backward derivatives, across parties to compute model gradients. Nevertheless, due to its geo-distributed nature, VFL training usually suffers from the low WAN bandwidth. In this paper, we introduce CELU-VFL, a novel and efficient VFL training framework that exploits the local update technique to reduce the cross-party communication rounds. CELU-VFL caches the stale statistics and reuses them to estimate model gradients without exchanging the ad hoc statistics. Significant techniques are proposed to improve the convergence performance. First, to handle the stochastic variance problem, we propose a uniform sampling strategy to fairly choose the stale statistics for local updates. Second, to harness the errors brought by the staleness, we devise an instance weighting mechanism that measures the reliability of the estimated gradients. Theoretical analysis proves that CELU-VFL achieves a similar sub-linear convergence rate as vanilla VFL training but requires much fewer communication rounds. Empirical results on both public and real-world workloads validate that CELU-VFL can be up to six times faster than the existing works.