# BESA: Boosting Encoder Stealing Attack with Perturbation Recovery

arXiv: https://arxiv.org/abs/2506.04556 · HTML: https://mlsanity.com/paper/2506.04556/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 49 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 30 of 31)

- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [A Survey of Machine Unlearning](https://mlsanity.com/paper/2209.02299/) (2022)
- [Knockoff Nets: Stealing Functionality of Black-Box Models](https://mlsanity.com/paper/1812.02766/) (2018)
- [MagNet: a Two-Pronged Defense against Adversarial Examples](https://mlsanity.com/paper/1705.09064/) (2017)
- [Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection](https://mlsanity.com/paper/2203.12208/) (2022)
- [PRADA: Protecting against DNN Model Stealing Attacks](https://mlsanity.com/paper/1805.02628/) (2018)
- [Entangled Watermarks as a Defense against Model Extraction](https://mlsanity.com/paper/2002.12200/) (2020)
- [Data-Free Model Extraction](https://mlsanity.com/paper/2011.14779/) (2020)
- [BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning](https://mlsanity.com/paper/2108.00352/) (2021)
- [Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language](https://mlsanity.com/paper/2212.07525/) (2022)
- [MAZE: Data-Free Model Stealing Attack Using Zeroth-Order Gradient Estimation](https://mlsanity.com/paper/2005.03161/) (2020)
- [Backdoor Attacks on Self-Supervised Learning](https://mlsanity.com/paper/2105.10123/) (2021)
- [EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learning](https://mlsanity.com/paper/2108.11023/) (2021)
- [Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark](https://mlsanity.com/paper/2305.10036/) (2023)
- [SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoders](https://mlsanity.com/paper/2201.11692/) (2022)
- [Black-Box Attacks on Sequential Recommenders via Data-Free Model Extraction](https://mlsanity.com/paper/2109.01165/) (2021)
- [Can't Steal? Cont-Steal! Contrastive Stealing Attacks Against Image Encoders](https://mlsanity.com/paper/2201.07513/) (2022)
- [An Embarrassingly Simple Backdoor Attack on Self-supervised Learning](https://mlsanity.com/paper/2210.07346/) (2022)
- [StolenEncoder: Stealing Pre-trained Encoders in Self-supervised Learning](https://mlsanity.com/paper/2201.05889/) (2022)
- [On the Difficulty of Defending Self-Supervised Learning against Model Extraction](https://mlsanity.com/paper/2205.07890/) (2022)
- [Watermarking Vision-Language Pre-trained Models for Multi-modal Embedding as a Service](https://mlsanity.com/paper/2311.05863/) (2023)
- [Semi-Leak: Membership Inference Attacks Against Semi-supervised Learning](https://mlsanity.com/paper/2207.12535/) (2022)
- [Bucks for Buckets (B4B): Active Defenses Against Stealing Encoders](https://mlsanity.com/paper/2310.08571/) (2023)
- [DST: Dynamic Substitute Training for Data-free Black-box Attack](https://mlsanity.com/paper/2204.00972/) (2022)
- [10 Security and Privacy Problems in Large Foundation Models](https://mlsanity.com/paper/2110.15444/) (2021)
- [AWEncoder: Adversarial Watermarking Pre-trained Encoders in Contrastive Learning](https://mlsanity.com/paper/2208.03948/) (2022)
- [Good Artists Copy, Great Artists Steal: Model Extraction Attacks Against Image Translation Models](https://mlsanity.com/paper/2104.12623/) (2021)
- [REaaS: Enabling Adversarially Robust Downstream Classifiers via Robust Encoder as a Service](https://mlsanity.com/paper/2301.02905/) (2023)
- [Data-Free Model Extraction Attacks in the Context of Object Detection](https://mlsanity.com/paper/2308.05127/) (2023)

## Built on by (most cited first, 2 of 2)

- [Towards Transparent and Incentive-Compatible Collaboration in Decentralized LLM Multi-Agent Systems: A Blockchain-Driven Approach](https://mlsanity.com/paper/2509.16736/) (2025)
- [DiMEx: Breaking the Cold Start Barrier in Data-Free Model Extraction via Latent Diffusion Priors](https://mlsanity.com/paper/2601.01688/) (2026)

## Abstract (arXiv)

To boost the encoder stealing attack under the perturbation-based defense that hinders the attack performance, we propose a boosting encoder stealing attack with perturbation recovery named BESA. It aims to overcome perturbation-based defenses. The core of BESA consists of two modules: perturbation detection and perturbation recovery, which can be combined with canonical encoder stealing attacks. The perturbation detection module utilizes the feature vectors obtained from the target encoder to infer the defense mechanism employed by the service provider. Once the defense mechanism is detected, the perturbation recovery module leverages the well-designed generative model to restore a clean feature vector from the perturbed one. Through extensive evaluations based on various datasets, we demonstrate that BESA significantly enhances the surrogate encoder accuracy of existing encoder stealing attacks by up to 24.63% when facing state-of-the-art defenses and combinations of multiple defenses.