# Backdoor Learning Curves: Explaining Backdoor Poisoning Beyond Influence Functions

arXiv: https://arxiv.org/abs/2106.07214 · HTML: https://mlsanity.com/paper/2106.07214/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 56 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 27 of 27)

- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks](https://mlsanity.com/paper/1804.00792/) (2018)
- [DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition](https://mlsanity.com/paper/1310.1531/) (2013)
- [Bilevel Programming for Hyperparameter Optimization and Meta-Learning](https://mlsanity.com/paper/1806.04910/) (2018)
- [Hidden Trigger Backdoor Attacks](https://mlsanity.com/paper/1910.00033/) (2019)
- [Gradient-based Hyperparameter Optimization through Reversible Learning](https://mlsanity.com/paper/1502.03492/) (2015)
- [Optimizing Millions of Hyperparameters by Implicit Differentiation](https://mlsanity.com/paper/1911.02590/) (2019)
- [Influence Functions in Deep Learning Are Fragile](https://mlsanity.com/paper/2006.14651/) (2020)
- [Differentially Private Learning Needs Better Features (or Much More Data)](https://mlsanity.com/paper/2011.11660/) (2020)
- [Hyperparameter optimization with approximate gradient](https://mlsanity.com/paper/1602.02355/) (2016)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [Dynamic Backdoor Attacks Against Machine Learning Models](https://mlsanity.com/paper/2003.03675/) (2020)
- [Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation](https://mlsanity.com/paper/1808.10307/) (2018)
- [Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review](https://mlsanity.com/paper/2007.10760/) (2020)
- [Graph Backdoor](https://mlsanity.com/paper/2006.11890/) (2020)
- [A high-bias, low-variance introduction to Machine Learning for physicists](https://mlsanity.com/paper/1803.08823/) (2018)
- [Are All Layers Created Equal?](https://mlsanity.com/paper/1902.01996/) (2019)
- [Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoff](https://mlsanity.com/paper/2011.09527/) (2020)
- [On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping](https://mlsanity.com/paper/2002.11497/) (2020)
- [DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation](https://mlsanity.com/paper/2012.07006/) (2020)
- [Bypassing Backdoor Detection Algorithms in Deep Learning](https://mlsanity.com/paper/1905.13409/) (2019)
- [Quantitative Verification of Neural Networks And its Security Applications](https://mlsanity.com/paper/1906.10395/) (2019)
- [Attack Strength vs. Detectability Dilemma in Adversarial Machine Learning](https://mlsanity.com/paper/1802.07295/) (2018)
- [Regularization Can Help Mitigate Poisoning Attacks... with the Right Hyperparameters](https://mlsanity.com/paper/2105.10948/) (2021)
- [Baseline Pruning-Based Approach to Trojan Detection in Neural Networks](https://mlsanity.com/paper/2101.12016/) (2021)

## Built on by (most cited first, 5 of 5)

- [Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning](https://mlsanity.com/paper/2205.01992/) (2022)
- [Algorithmic Collective Action in Machine Learning](https://mlsanity.com/paper/2302.04262/) (2023)
- [Statistical Collusion by Collectives on Learning Platforms](https://mlsanity.com/paper/2502.04879/) (2025)
- [DECK: Model Hardening for Defending Pervasive Backdoors](https://mlsanity.com/paper/2206.09272/) (2022)
- [Mithridates: Auditing and Boosting Backdoor Resistance of Machine Learning Pipelines](https://mlsanity.com/paper/2302.04977/) (2023)

## Abstract (arXiv)

Backdoor attacks inject poisoning samples during training, with the goal of forcing a machine learning model to output an attacker-chosen class when presented a specific trigger at test time. Although backdoor attacks have been demonstrated in a variety of settings and against different models, the factors affecting their effectiveness are still not well understood. In this work, we provide a unifying framework to study the process of backdoor learning under the lens of incremental learning and influence functions. We show that the effectiveness of backdoor attacks depends on: (i) the complexity of the learning algorithm, controlled by its hyperparameters; (ii) the fraction of backdoor samples injected into the training set; and (iii) the size and visibility of the backdoor trigger. These factors affect how fast a model learns to correlate the presence of the backdoor trigger with the target class. Our analysis unveils the intriguing existence of a region in the hyperparameter space in which the accuracy on clean test samples is still high while backdoor attacks are ineffective, thereby suggesting novel criteria to improve existing defenses.