# Technical Report: When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks

arXiv: https://arxiv.org/abs/1803.06975 · HTML: https://mlsanity.com/paper/1803.06975/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2018.

## Built on by (most cited first, 30 of 81)

- [Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models](https://mlsanity.com/paper/2302.04222/) (2023)
- [Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets](https://mlsanity.com/paper/2204.00032/) (2022)
- [Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification](https://mlsanity.com/paper/2407.20859/) (2024)
- [Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization](https://mlsanity.com/paper/2106.06916/) (2021)
- [Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attacks](https://mlsanity.com/paper/2212.10717/) (2022)
- [An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences](https://mlsanity.com/paper/2111.08429/) (2021)
- [Topological Detection of Trojaned Neural Networks](https://mlsanity.com/paper/2106.06469/) (2021)
- [PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learning](https://mlsanity.com/paper/2205.06401/) (2022)
- [LoMar: A Local Defense Against Poisoning Attack on Federated Learning](https://mlsanity.com/paper/2201.02873/) (2022)
- [SNAP: Efficient Extraction of Private Properties with Poisoning](https://mlsanity.com/paper/2208.12348/) (2022)
- [Robustly-reliable learners under poisoning attacks](https://mlsanity.com/paper/2203.04160/) (2022)
- [Data Poisoning in Deep Learning: A Survey](https://mlsanity.com/paper/2503.22759/) (2025)
- [Secure Retrieval-Augmented Generation against Poisoning Attacks](https://mlsanity.com/paper/2510.25025/) (2025)
- [Security and Privacy for Artificial Intelligence: Opportunities and Challenges](https://mlsanity.com/paper/2102.04661/) (2021)
- [Robust learning under clean-label attack](https://mlsanity.com/paper/2103.00671/) (2021)
- [LLM Whisperer: An Inconspicuous Attack to Bias LLM Responses](https://mlsanity.com/paper/2406.04755/) (2024)
- [Get a Model! Model Hijacking Attack Against Machine Learning Models](https://mlsanity.com/paper/2111.04394/) (2021)
- [Jigsaw Puzzle: Selective Backdoor Attack to Subvert Malware Classifiers](https://mlsanity.com/paper/2202.05470/) (2022)
- [Preference Poisoning Attacks on Reward Model Learning](https://mlsanity.com/paper/2402.01920/) (2024)
- [Accumulative Poisoning Attacks on Real-time Data](https://mlsanity.com/paper/2106.09993/) (2021)
- [SoK: Machine Learning Governance](https://mlsanity.com/paper/2109.10870/) (2021)
- [Efficient Query-Based Attack against ML-Based Android Malware Detection under Zero Knowledge Setting](https://mlsanity.com/paper/2309.01866/) (2023)
- [Demystifying the Transferability of Adversarial Attacks in Computer Networks](https://mlsanity.com/paper/2110.04488/) (2021)
- [Backdoor Attacks in Peer-to-Peer Federated Learning](https://mlsanity.com/paper/2301.09732/) (2023)
- [Avengers Ensemble! Improving Transferability of Authorship Obfuscation](https://mlsanity.com/paper/2109.07028/) (2021)
- [Adversarial Transfer Attacks With Unknown Data and Class Overlap](https://mlsanity.com/paper/2109.11125/) (2021)
- [Mate! Are You Really Aware? An Explainability-Guided Testing Framework for Robustness of Malware Detectors](https://mlsanity.com/paper/2111.10085/) (2021)
- [Your Compiler is Backdooring Your Model: Understanding and Exploiting Compilation Inconsistency Vulnerabilities in Deep Learning Compilers](https://mlsanity.com/paper/2509.11173/) (2025)
- [Turning Federated Learning Systems Into Covert Channels](https://mlsanity.com/paper/2104.10561/) (2021)
- [10 Security and Privacy Problems in Large Foundation Models](https://mlsanity.com/paper/2110.15444/) (2021)

## Abstract (arXiv)

Recent results suggest that attacks against supervised machine learning systems are quite effective, while defenses are easily bypassed by new attacks. However, the specifications for machine learning systems currently lack precise adversary definitions, and the existing attacks make diverse, potentially unrealistic assumptions about the strength of the adversary who launches them. We propose the FAIL attacker model, which describes the adversary's knowledge and control along four dimensions. Our model allows us to consider a wide range of weaker adversaries who have limited control and incomplete knowledge of the features, learning algorithms and training instances utilized. To evaluate the utility of the FAIL model, we consider the problem of conducting targeted poisoning attacks in a realistic setting: the crafted poison samples must have clean labels, must be individually and collectively inconspicuous, and must exhibit a generalized form of transferability, defined by the FAIL model. By taking these constraints into account, we design StingRay, a targeted poisoning attack that is practical against 4 machine learning applications, which use 3 different learning algorithms, and can bypass 2 existing defenses. Conversely, we show that a prior evasion attack is less effective under generalized transferability. Such attack evaluations, under the FAIL adversary model, may also suggest promising directions for future defenses.