# Label-Only Model Inversion Attacks via Boundary Repulsion

arXiv: https://arxiv.org/abs/2203.01925 · HTML: https://mlsanity.com/paper/2203.01925/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 3 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Exploiting Unintended Feature Leakage in Collaborative Learning](https://mlsanity.com/paper/1805.04049/) (2018)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [HopSkipJumpAttack: A Query-Efficient Decision-Based Attack](https://mlsanity.com/paper/1904.02144/) (2019)
- [The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks](https://mlsanity.com/paper/1911.07135/) (2019)
- [A Survey of Privacy Attacks in Machine Learning](https://mlsanity.com/paper/2007.07646/) (2020)
- [Understanding Membership Inferences on Well-Generalized Learning Models](https://mlsanity.com/paper/1802.04889/) (2018)
- [Knowledge-Enriched Distributional Model Inversion Attacks](https://mlsanity.com/paper/2010.04092/) (2020)
- [Improving Robustness to Model Inversion Attacks via Mutual Information Regularization](https://mlsanity.com/paper/2009.05241/) (2020)
- [Adversarial Neural Network Inversion via Auxiliary Knowledge Alignment](https://mlsanity.com/paper/1902.08552/) (2019)
- [Black-Box Face Recovery from Identity Features](https://mlsanity.com/paper/2007.13635/) (2020)
- [GAMIN: An Adversarial Approach to Black-Box Model Inversion](https://mlsanity.com/paper/1909.11835/) (2019)

## Built on by (most cited first, 30 of 56)

- [Re-thinking Model Inversion Attacks Against Deep Neural Networks](https://mlsanity.com/paper/2304.01669/) (2023)
- [Reinforcement Learning-Based Black-Box Model Inversion Attacks](https://mlsanity.com/paper/2304.04625/) (2023)
- [Text Revealer: Private Text Reconstruction via Model Inversion Attacks against Transformers](https://mlsanity.com/paper/2209.10505/) (2022)
- [Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defenses](https://mlsanity.com/paper/2402.04013/) (2024)
- [Pseudo Label-Guided Model Inversion Attack via Conditional Generative Adversarial Network](https://mlsanity.com/paper/2302.09814/) (2023)
- [The Enemy of My Enemy is My Friend: Exploring Inverse Adversaries for Improving Adversarial Training](https://mlsanity.com/paper/2211.00525/) (2022)
- [Label-Only Model Inversion Attacks via Knowledge Transfer](https://mlsanity.com/paper/2310.19342/) (2023)
- [Is Retain Set All You Need in Machine Unlearning? Restoring Performance of Unlearned Models with Out-Of-Distribution Images](https://mlsanity.com/paper/2404.12922/) (2024)
- [A Closer Look at GAN Priors: Exploiting Intermediate Features for Enhanced Model Inversion Attacks](https://mlsanity.com/paper/2407.13863/) (2024)
- [On Strengthening and Defending Graph Reconstruction Attack with Markov Chain Approximation](https://mlsanity.com/paper/2306.09104/) (2023)
- [Unstoppable Attack: Label-Only Model Inversion via Conditional Diffusion Model](https://mlsanity.com/paper/2307.08424/) (2023)
- [Be Careful What You Smooth For: Label Smoothing Can Be a Privacy Shield but Also a Catalyst for Model Inversion Attacks](https://mlsanity.com/paper/2310.06549/) (2023)
- [Model Inversion Robustness: Can Transfer Learning Help?](https://mlsanity.com/paper/2405.05588/) (2024)
- [UPAM: Unified Prompt Attack in Text-to-Image Generation Models Against Both Textual Filters and Visual Checkers](https://mlsanity.com/paper/2405.11336/) (2024)
- [Model Inversion Attacks: A Survey of Approaches and Countermeasures](https://mlsanity.com/paper/2411.10023/) (2024)
- [On the Vulnerability of Skip Connections to Model Inversion Attacks](https://mlsanity.com/paper/2409.01696/) (2024)
- [Boosting Model Inversion Attacks with Adversarial Examples](https://mlsanity.com/paper/2306.13965/) (2023)
- [Improving Robustness to Model Inversion Attacks via Sparse Coding Architectures](https://mlsanity.com/paper/2403.14772/) (2024)
- [Federated Graph Condensation with Information Bottleneck Principles](https://mlsanity.com/paper/2405.03911/) (2024)
- [Privacy-Preserving Language Model Inference with Instance Obfuscation](https://mlsanity.com/paper/2402.08227/) (2024)
- [MIBench: A Comprehensive Framework for Benchmarking Model Inversion Attack and Defense](https://mlsanity.com/paper/2410.05159/) (2024)
- [Trap-MID: Trapdoor-based Defense against Model Inversion Attacks](https://mlsanity.com/paper/2411.08460/) (2024)
- [Distilling the Undistillable: Learning from a Nasty Teacher](https://mlsanity.com/paper/2210.11728/) (2022)
- [Client-side Gradient Inversion Against Federated Learning from Poisoning](https://mlsanity.com/paper/2309.07415/) (2023)
- [All Rivers Run to the Sea: Private Learning with Asymmetric Flows](https://mlsanity.com/paper/2312.05264/) (2023)
- [Decaf: Data Distribution Decompose Attack against Federated Learning](https://mlsanity.com/paper/2405.15316/) (2024)
- [FreezeAsGuard: Mitigating Illegal Adaptation of Diffusion Models via Selective Tensor Freezing](https://mlsanity.com/paper/2405.17472/) (2024)
- [LeakyCLIP: Extracting Training Data from CLIP](https://mlsanity.com/paper/2508.00756/) (2025)
- [Out-of-Bounding-Box Triggers: A Stealthy Approach to Cheat Object Detectors](https://mlsanity.com/paper/2410.10091/) (2024)
- [Adversarial Attacks on Hyperbolic Networks](https://mlsanity.com/paper/2412.01495/) (2024)

## Abstract (arXiv)

Recent studies show that the state-of-the-art deep neural networks are vulnerable to model inversion attacks, in which access to a model is abused to reconstruct private training data of any given target class. Existing attacks rely on having access to either the complete target model (whitebox) or the model's soft-labels (blackbox). However, no prior work has been done in the harder but more practical scenario, in which the attacker only has access to the model's predicted label, without a confidence measure. In this paper, we introduce an algorithm, Boundary-Repelling Model Inversion (BREP-MI), to invert private training data using only the target model's predicted labels. The key idea of our algorithm is to evaluate the model's predicted labels over a sphere and then estimate the direction to reach the target class's centroid. Using the example of face recognition, we show that the images reconstructed by BREP-MI successfully reproduce the semantics of the private training data for various datasets and target model architectures. We compare BREP-MI with the state-of-the-art whitebox and blackbox model inversion attacks and the results show that despite assuming less knowledge about the target model, BREP-MI outperforms the blackbox attack and achieves comparable results to the whitebox attack.