# CAFE: Catastrophic Data Leakage in Vertical Federated Learning

arXiv: https://arxiv.org/abs/2110.15122 · HTML: https://mlsanity.com/paper/2110.15122/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 34 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 20 of 20)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [How To Backdoor Federated Learning](https://mlsanity.com/paper/1807.00459/) (2018)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [Analyzing Federated Learning through an Adversarial Lens](https://mlsanity.com/paper/1811.12470/) (2018)
- [Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning](https://mlsanity.com/paper/1702.07464/) (2017)
- [See through Gradients: Image Batch Recovery via GradInversion](https://mlsanity.com/paper/2104.07586/) (2021)
- [FedML: A Research Library and Benchmark for Federated Machine Learning](https://mlsanity.com/paper/2007.13518/) (2020)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [R-GAP: Recursive Gradient Attack on Privacy](https://mlsanity.com/paper/2010.07733/) (2020)
- [LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learning](https://mlsanity.com/paper/1805.09965/) (2018)
- [VAFL: a Method of Vertical Asynchronous Federated Learning](https://mlsanity.com/paper/2007.06081/) (2020)
- [A Framework for Evaluating Gradient Leakage Attacks in Federated Learning](https://mlsanity.com/paper/2004.10397/) (2020)
- [Securing Secure Aggregation: Mitigating Multi-Round Privacy Leakage in Federated Learning](https://mlsanity.com/paper/2106.03328/) (2021)
- [Rethinking Privacy Preserving Deep Learning: How to Evaluate and Thwart Privacy Attacks](https://mlsanity.com/paper/2006.11601/) (2020)
- [SAPAG: A Self-Adaptive Privacy Attack From Gradients](https://mlsanity.com/paper/2009.06228/) (2020)
- [FireCaffe: near-linear acceleration of deep neural network training on compute clusters](https://mlsanity.com/paper/1511.00175/) (2015)

## Built on by (most cited first, 30 of 76)

- [Vertical Federated Learning: Concepts, Advances and Challenges](https://mlsanity.com/paper/2211.12814/) (2022)
- [Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges](https://mlsanity.com/paper/2201.08135/) (2022)
- [Vertical Federated Learning: Challenges, Methodologies and Experiments](https://mlsanity.com/paper/2202.04309/) (2022)
- [GradViT: Gradient Inversion of Vision Transformers](https://mlsanity.com/paper/2203.11894/) (2022)
- [Bounding Training Data Reconstruction in DP-SGD](https://mlsanity.com/paper/2302.07225/) (2023)
- [FedVS: Straggler-Resilient and Privacy-Preserving Vertical Federated Learning for Split Models](https://mlsanity.com/paper/2304.13407/) (2023)
- [Unlocking Accuracy and Fairness in Differentially Private Image Classification](https://mlsanity.com/paper/2308.10888/) (2023)
- [Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challenges](https://mlsanity.com/paper/2309.07438/) (2023)
- [Trustworthy Distributed AI Systems: Robustness, Privacy, and Governance](https://mlsanity.com/paper/2402.01096/) (2024)
- [Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey](https://mlsanity.com/paper/2405.17495/) (2024)
- [TabLeak: Tabular Data Leakage in Federated Learning](https://mlsanity.com/paper/2210.01785/) (2022)
- [Privacy Assessment on Reconstructed Images: Are Existing Evaluation Metrics Faithful to Human Perception?](https://mlsanity.com/paper/2309.13038/) (2023)
- [Surrogate Model Extension (SME): A Fast and Accurate Weight Update Attack on Federated Learning](https://mlsanity.com/paper/2306.00127/) (2023)
- [FedPass: Privacy-Preserving Vertical Federated Deep Learning with Adaptive Obfuscation](https://mlsanity.com/paper/2301.12623/) (2023)
- [BadVFL: Backdoor Attacks in Vertical Federated Learning](https://mlsanity.com/paper/2304.08847/) (2023)
- [Hiding in Plain Sight: Disguising Data Stealing Attacks in Federated Learning](https://mlsanity.com/paper/2306.03013/) (2023)
- [Analysis of Privacy Leakage in Federated Large Language Models](https://mlsanity.com/paper/2403.04784/) (2024)
- [Vertical Federated Learning: Taxonomies, Threats, and Prospects](https://mlsanity.com/paper/2302.01550/) (2023)
- [VFLAIR: A Research Library and Benchmark for Vertical Federated Learning](https://mlsanity.com/paper/2310.09827/) (2023)
- [Robust and IP-Protecting Vertical Federated Learning against Unexpected Quitting of Parties](https://mlsanity.com/paper/2303.18178/) (2023)
- [A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective](https://mlsanity.com/paper/2402.03688/) (2024)
- [Practical and General Backdoor Attacks against Vertical Federated Learning](https://mlsanity.com/paper/2306.10746/) (2023)
- [VertiBench: Advancing Feature Distribution Diversity in Vertical Federated Learning Benchmarks](https://mlsanity.com/paper/2307.02040/) (2023)
- [Understanding Deep Gradient Leakage via Inversion Influence Functions](https://mlsanity.com/paper/2309.13016/) (2023)
- [PraFFL: A Preference-Aware Scheme in Fair Federated Learning](https://mlsanity.com/paper/2404.08973/) (2024)
- [Enabling All In-Edge Deep Learning: A Literature Review](https://mlsanity.com/paper/2204.03326/) (2022)
- [Differentially Private Vertical Federated Learning](https://mlsanity.com/paper/2211.06782/) (2022)
- [Mutual Information Regularization for Vertical Federated Learning](https://mlsanity.com/paper/2301.01142/) (2023)
- [Secure Vertical Federated Learning Under Unreliable Connectivity](https://mlsanity.com/paper/2305.16794/) (2023)
- [Privacy-Preserving Remote Heart Rate Estimation from Facial Videos](https://mlsanity.com/paper/2306.01141/) (2023)

## Abstract (arXiv)

Recent studies show that private training data can be leaked through the gradients sharing mechanism deployed in distributed machine learning systems, such as federated learning (FL). Increasing batch size to complicate data recovery is often viewed as a promising defense strategy against data leakage. In this paper, we revisit this defense premise and propose an advanced data leakage attack with theoretical justification to efficiently recover batch data from the shared aggregated gradients. We name our proposed method as catastrophic data leakage in vertical federated learning (CAFE). Comparing to existing data leakage attacks, our extensive experimental results on vertical FL settings demonstrate the effectiveness of CAFE to perform large-batch data leakage attack with improved data recovery quality. We also propose a practical countermeasure to mitigate CAFE. Our results suggest that private data participated in standard FL, especially the vertical case, have a high risk of being leaked from the training gradients. Our analysis implies unprecedented and practical data leakage risks in those learning settings. The code of our work is available at https://github.com/DeRafael/CAFE.