# AdvMind: Inferring Adversary Intent of Black-Box Attacks

arXiv: https://arxiv.org/abs/2006.09539 · HTML: https://mlsanity.com/paper/2006.09539/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2020.

## Built on by (most cited first, 12 of 12)

- [Adapting Membership Inference Attacks to GNN for Graph Classification: Approaches and Implications](https://mlsanity.com/paper/2110.08760/) (2021)
- [Random Noise Defense Against Query-Based Black-Box Attacks](https://mlsanity.com/paper/2104.11470/) (2021)
- [Reverse Engineering of Imperceptible Adversarial Image Perturbations](https://mlsanity.com/paper/2203.14145/) (2022)
- [Transfer Attacks Revisited: A Large-Scale Empirical Study in Real Computer Vision Settings](https://mlsanity.com/paper/2204.04063/) (2022)
- [Stateful Defenses for Machine Learning Models Are Not Yet Secure Against Black-box Attacks](https://mlsanity.com/paper/2303.06280/) (2023)
- [Robust Spatiotemporal Traffic Forecasting with Reinforced Dynamic Adversarial Training](https://mlsanity.com/paper/2306.14126/) (2023)
- [Multi-SpacePhish: Extending the Evasion-space of Adversarial Attacks against Phishing Website Detectors using Machine Learning](https://mlsanity.com/paper/2210.13660/) (2022)
- [SoK: A Modularized Approach to Study the Security of Automatic Speech Recognition Systems](https://mlsanity.com/paper/2103.10651/) (2021)
- [Audit and Improve Robustness of Private Neural Networks on Encrypted Data](https://mlsanity.com/paper/2209.09996/) (2022)
- [Understanding the Robustness of Randomized Feature Defense Against Query-Based Adversarial Attacks](https://mlsanity.com/paper/2310.00567/) (2023)
- [Unifying Gradients to Improve Real-world Robustness for Deep Networks](https://mlsanity.com/paper/2208.06228/) (2022)
- [SEA: Shareable and Explainable Attribution for Query-based Black-box Attacks](https://mlsanity.com/paper/2308.11845/) (2023)

## Abstract (arXiv)

Deep neural networks (DNNs) are inherently susceptible to adversarial attacks even under black-box settings, in which the adversary only has query access to the target models. In practice, while it may be possible to effectively detect such attacks (e.g., observing massive similar but non-identical queries), it is often challenging to exactly infer the adversary intent (e.g., the target class of the adversarial example the adversary attempts to craft) especially during early stages of the attacks, which is crucial for performing effective deterrence and remediation of the threats in many scenarios. In this paper, we present AdvMind, a new class of estimation models that infer the adversary intent of black-box adversarial attacks in a robust and prompt manner. Specifically, to achieve robust detection, AdvMind accounts for the adversary adaptiveness such that her attempt to conceal the target will significantly increase the attack cost (e.g., in terms of the number of queries); to achieve prompt detection, AdvMind proactively synthesizes plausible query results to solicit subsequent queries from the adversary that maximally expose her intent. Through extensive empirical evaluation on benchmark datasets and state-of-the-art black-box attacks, we demonstrate that on average AdvMind detects the adversary intent with over 75% accuracy after observing less than 3 query batches and meanwhile increases the cost of adaptive attacks by over 60%. We further discuss the possible synergy between AdvMind and other defense methods against black-box adversarial attacks, pointing to several promising research directions.