# Blacklight: Scalable Defense for Neural Networks against Query-Based Black-Box Attacks

arXiv: https://arxiv.org/abs/2006.14042 · HTML: https://mlsanity.com/paper/2006.14042/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2020.

## Built on by (most cited first, 30 of 37)

- [LLM Whisperer: An Inconspicuous Attack to Bias LLM Responses](https://mlsanity.com/paper/2406.04755/) (2024)
- [Defenses in Adversarial Machine Learning: A Survey](https://mlsanity.com/paper/2312.08890/) (2023)
- [Evading Black-box Classifiers Without Breaking Eggs](https://mlsanity.com/paper/2306.02895/) (2023)
- [OSLO: One-Shot Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2405.16978/) (2024)
- [Stateful Defenses for Machine Learning Models Are Not Yet Secure Against Black-box Attacks](https://mlsanity.com/paper/2303.06280/) (2023)
- [Efficient Query-Based Attack against ML-Based Android Malware Detection under Zero Knowledge Setting](https://mlsanity.com/paper/2309.01866/) (2023)
- [Improving the Robustness of Transformer-based Large Language Models with Dynamic Attention](https://mlsanity.com/paper/2311.17400/) (2023)
- [MalProtect: Stateful Defense Against Adversarial Query Attacks in ML-based Malware Detection](https://mlsanity.com/paper/2302.10739/) (2023)
- [Benchmarking Misuse Mitigation Against Covert Adversaries](https://mlsanity.com/paper/2506.06414/) (2025)
- [AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning](https://mlsanity.com/paper/2408.01978/) (2024)
- [Your Compiler is Backdooring Your Model: Understanding and Exploiting Compilation Inconsistency Vulnerabilities in Deep Learning Compilers](https://mlsanity.com/paper/2509.11173/) (2025)
- [Certifiable Black-Box Attacks with Randomized Adversarial Examples: Breaking Defenses with Provable Confidence](https://mlsanity.com/paper/2304.04343/) (2023)
- [The Adaptive Arms Race: Redefining Robustness in AI Security](https://mlsanity.com/paper/2312.13435/) (2023)
- [Unified Prompt Attack Against Text-to-Image Generation Models](https://mlsanity.com/paper/2502.16423/) (2025)
- [Understanding the Robustness of Randomized Feature Defense Against Query-Based Adversarial Attacks](https://mlsanity.com/paper/2310.00567/) (2023)
- [Publishing Efficient On-device Models Increases Adversarial Vulnerability](https://mlsanity.com/paper/2212.13700/) (2022)
- [Exploring the Adversarial Frontier: Quantifying Robustness via Adversarial Hypervolume](https://mlsanity.com/paper/2403.05100/) (2024)
- [BadPart: Unified Black-box Adversarial Patch Attacks against Pixel-wise Regression Tasks](https://mlsanity.com/paper/2404.00924/) (2024)
- [Theoretically Principled Trade-off for Stateful Defenses against Query-Based Black-Box Attacks](https://mlsanity.com/paper/2307.16331/) (2023)
- [Towards Deep Learning Models Resistant to Transfer-based Adversarial Attacks via Data-centric Robust Learning](https://mlsanity.com/paper/2310.09891/) (2023)
- [When AI reviews science: Can we trust the referee?](https://mlsanity.com/paper/2604.23593/) (2026)
- [Fabricated Flips: Poisoning Federated Learning without Data](https://mlsanity.com/paper/2202.05877/) (2022)
- [Efficient Representation of the Activation Space in Deep Neural Networks](https://mlsanity.com/paper/2312.08143/) (2023)
- [Taking off the Rose-Tinted Glasses: A Critical Look at Adversarial ML Through the Lens of Evasion Attacks](https://mlsanity.com/paper/2410.12076/) (2024)
- [Adversarial Examples in Environment Perception for Automated Driving (Review)](https://mlsanity.com/paper/2504.08414/) (2025)
- [Dynamic Probabilistic Noise Injection for Membership Inference Defense](https://mlsanity.com/paper/2505.13362/) (2025)
- [How stealthy is stealthy? Studying the Efficacy of Black-Box Adversarial Attacks in the Real World](https://mlsanity.com/paper/2506.05382/) (2025)
- [On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabeling](https://mlsanity.com/paper/2506.21874/) (2025)
- [Grounding-Driven Attack: Improving Encoder-based Adversarial Transferability against Large Vision-Language Models](https://mlsanity.com/paper/2602.09431/) (2026)
- [Effectiveness of Moving Target Defenses for Adversarial Attacks in ML-based Malware Detection](https://mlsanity.com/paper/2302.00537/) (2023)

## Abstract (arXiv)

Deep learning systems are known to be vulnerable to adversarial examples. In particular, query-based black-box attacks do not require knowledge of the deep learning model, but can compute adversarial examples over the network by submitting queries and inspecting returns. Recent work largely improves the efficiency of those attacks, demonstrating their practicality on today's ML-as-a-service platforms. We propose Blacklight, a new defense against query-based black-box adversarial attacks. The fundamental insight driving our design is that, to compute adversarial examples, these attacks perform iterative optimization over the network, producing image queries highly similar in the input space. Blacklight detects query-based black-box attacks by detecting highly similar queries, using an efficient similarity engine operating on probabilistic content fingerprints. We evaluate Blacklight against eight state-of-the-art attacks, across a variety of models and image classification tasks. Blacklight identifies them all, often after only a handful of queries. By rejecting all detected queries, Blacklight prevents any attack to complete, even when attackers persist to submit queries after account ban or query rejection. Blacklight is also robust against several powerful countermeasures, including an optimal black-box attack that approximates white-box attacks in efficiency. Finally, we illustrate how Blacklight generalizes to other domains like text classification.