# Batch Label Inference and Replacement Attacks in Black-Boxed Vertical Federated Learning

arXiv: https://arxiv.org/abs/2112.05409 · HTML: https://mlsanity.com/paper/2112.05409/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 10 Dec 2021.

## Builds on (most cited first, 17 of 17)

- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [How To Backdoor Federated Learning](https://mlsanity.com/paper/1807.00459/) (2018)
- [Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training](https://mlsanity.com/paper/1712.01887/) (2017)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [Analyzing Federated Learning through an Adversarial Lens](https://mlsanity.com/paper/1811.12470/) (2018)
- [See through Gradients: Image Batch Recovery via GradInversion](https://mlsanity.com/paper/2104.07586/) (2021)
- [Can You Really Backdoor Federated Learning?](https://mlsanity.com/paper/1911.07963/) (2019)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [A Survey on Homomorphic Encryption Schemes: Theory and Implementation](https://mlsanity.com/paper/1704.03578/) (2017)
- [Label Leakage and Protection in Two-party Split Learning](https://mlsanity.com/paper/2102.08504/) (2021)
- [Privacy-preserving Collaborative Learning with Automatic Transformation Search](https://mlsanity.com/paper/2011.12505/) (2020)
- [Backdoor attacks and defenses in feature-partitioned collaborative learning](https://mlsanity.com/paper/2007.03608/) (2020)
- [Learning Privately over Distributed Features: An ADMM Sharing Approach](https://mlsanity.com/paper/1907.07735/) (2019)
- [Secure Federated Transfer Learning](https://mlsanity.com/paper/1812.03337/) (2018)

## Built on by (most cited first, 16 of 16)

- [A Survey on Heterogeneous Federated Learning](https://mlsanity.com/paper/2210.04505/) (2022)
- [Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directions](https://mlsanity.com/paper/2303.02213/) (2023)
- [A Survey on Vertical Federated Learning: From a Layered Perspective](https://mlsanity.com/paper/2304.01829/) (2023)
- [Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey](https://mlsanity.com/paper/2405.17495/) (2024)
- [Feature Reconstruction Attacks and Countermeasures of DNN training in Vertical Federated Learning](https://mlsanity.com/paper/2210.06771/) (2022)
- [HashVFL: Defending Against Data Reconstruction Attacks in Vertical Federated Learning](https://mlsanity.com/paper/2212.00325/) (2022)
- [A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective](https://mlsanity.com/paper/2402.03688/) (2024)
- [Hijack Vertical Federated Learning Models As One Party](https://mlsanity.com/paper/2212.00322/) (2022)
- [Vertical Federated Learning in Practice: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2502.08160/) (2025)
- [Training on Fake Labels: Mitigating Label Leakage in Split Learning via Secure Dimension Transformation](https://mlsanity.com/paper/2410.09125/) (2024)
- [A GAN-based data poisoning framework against anomaly detection in vertical federated learning](https://mlsanity.com/paper/2401.08984/) (2024)
- [Venomancer: Towards Imperceptible and Target-on-Demand Backdoor Attacks in Federated Learning](https://mlsanity.com/paper/2407.03144/) (2024)
- [Just a Simple Transformation is Enough for Data Protection in Vertical Federated Learning](https://mlsanity.com/paper/2412.11689/) (2024)
- [BadFU: Backdoor Federated Learning through Adversarial Machine Unlearning](https://mlsanity.com/paper/2508.15541/) (2025)
- [SecureBoost Hyperparameter Tuning via Multi-Objective Federated Learning](https://mlsanity.com/paper/2307.10579/) (2023)
- [ICAFS: Inter-Client-Aware Feature Selection for Vertical Federated Learning](https://mlsanity.com/paper/2504.10851/) (2025)

## Abstract (arXiv)

In a vertical federated learning (VFL) scenario where features and model are split into different parties, communications of sample-specific updates are required for correct gradient calculations but can be used to deduce important sample-level label information. An immediate defense strategy is to protect sample-level messages communicated with Homomorphic Encryption (HE), and in this way only the batch-averaged local gradients are exposed to each party (termed black-boxed VFL). In this paper, we first explore the possibility of recovering labels in the vertical federated learning setting with HE-protected communication, and show that private labels can be reconstructed with high accuracy by training a gradient inversion model. Furthermore, we show that label replacement backdoor attacks can be conducted in black-boxed VFL by directly replacing encrypted communicated messages (termed gradient-replacement attack). As it is a common presumption that batch-averaged information is safe to share, batch label inference and replacement attacks are a severe challenge to VFL. To defend against batch label inference attack, we further evaluate several defense strategies, including confusional autoencoder (CoAE), a technique we proposed based on autoencoder and entropy regularization. We demonstrate that label inference and replacement attacks can be successfully blocked by this technique without hurting as much main task accuracy as compared to existing methods.