# Mutual Information Regularization for Vertical Federated Learning

arXiv: https://arxiv.org/abs/2301.01142 · HTML: https://mlsanity.com/paper/2301.01142/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 1 Jan 2023.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)

## Builds on (most cited first, 21 of 21)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [How To Backdoor Federated Learning](https://mlsanity.com/paper/1807.00459/) (2018)
- [Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training](https://mlsanity.com/paper/1712.01887/) (2017)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [See through Gradients: Image Batch Recovery via GradInversion](https://mlsanity.com/paper/2104.07586/) (2021)
- [Sparse Communication for Distributed Gradient Descent](https://mlsanity.com/paper/1704.05021/) (2017)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [Feature Inference Attack on Model Predictions in Vertical Federated Learning](https://mlsanity.com/paper/2010.10152/) (2020)
- [A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses](https://mlsanity.com/paper/2003.08983/) (2020)
- [InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective](https://mlsanity.com/paper/2010.02329/) (2020)
- [Label Leakage and Protection in Two-party Split Learning](https://mlsanity.com/paper/2102.08504/) (2021)
- [CAFE: Catastrophic Data Leakage in Vertical Federated Learning](https://mlsanity.com/paper/2110.15122/) (2021)
- [Improving Robustness to Model Inversion Attacks via Mutual Information Regularization](https://mlsanity.com/paper/2009.05241/) (2020)
- [Privacy-preserving Collaborative Learning with Automatic Transformation Search](https://mlsanity.com/paper/2011.12505/) (2020)
- [Backdoor attacks and defenses in feature-partitioned collaborative learning](https://mlsanity.com/paper/2007.03608/) (2020)
- [Secure Federated Transfer Learning](https://mlsanity.com/paper/1812.03337/) (2018)
- [Feature Reconstruction Attacks and Countermeasures of DNN training in Vertical Federated Learning](https://mlsanity.com/paper/2210.06771/) (2022)
- [Is Vertical Logistic Regression Privacy-Preserving? A Comprehensive Privacy Analysis and Beyond](https://mlsanity.com/paper/2207.09087/) (2022)

## Built on by (most cited first, 4 of 4)

- [VFLAIR: A Research Library and Benchmark for Vertical Federated Learning](https://mlsanity.com/paper/2310.09827/) (2023)
- [PrivaScissors: Enhance the Privacy of Collaborative Inference through the Lens of Mutual Information](https://mlsanity.com/paper/2306.07973/) (2023)
- [VMask: Tunable Label Privacy Protection for Vertical Federated Learning via Layer Masking](https://mlsanity.com/paper/2507.14629/) (2025)
- [A Survey on Split Learning for LLM Fine-Tuning: Models, Systems, and Privacy Optimizations](https://mlsanity.com/paper/2604.24468/) (2026)

## Abstract (arXiv)

Vertical Federated Learning (VFL) is widely utilized in real-world applications to enable collaborative learning while protecting data privacy and safety. However, previous works show that parties without labels (passive parties) in VFL can infer the sensitive label information owned by the party with labels (active party) or execute backdoor attacks to VFL. Meanwhile, active party can also infer sensitive feature information from passive party. All these pose new privacy and security challenges to VFL systems. We propose a new general defense method which limits the mutual information between private raw data, including both features and labels, and intermediate outputs to achieve a better trade-off between model utility and privacy. We term this defense Mutual Information Regularization Defense (MID). We theoretically and experimentally testify the effectiveness of our MID method in defending existing attacks in VFL, including label inference attacks, backdoor attacks and feature reconstruction attacks.