# Identifying a Training-Set Attack's Target Using Renormalized Influence Estimation

arXiv: https://arxiv.org/abs/2201.10055 · HTML: https://mlsanity.com/paper/2201.10055/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 90 references that are in the map.
- First posted on arXiv 25 Jan 2022.

## Builds on (most cited first, 30 of 53)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Estimating Training Data Influence by Tracing Gradient Descent](https://mlsanity.com/paper/2002.08484/) (2020)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks](https://mlsanity.com/paper/1804.00792/) (2018)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018)
- [STRIP: A Defence Against Trojan Attacks on Deep Neural Networks](https://mlsanity.com/paper/1902.06531/) (2019)
- [Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering](https://mlsanity.com/paper/1811.03728/) (2018)
- [Backdoor Learning: A Survey](https://mlsanity.com/paper/2007.08745/) (2020)
- [The Pitfalls of Simplicity Bias in Neural Networks](https://mlsanity.com/paper/2006.07710/) (2020)
- [Certified Defenses for Data Poisoning Attacks](https://mlsanity.com/paper/1706.03691/) (2017)
- [Anti-Backdoor Learning: Training Clean Models on Poisoned Data](https://mlsanity.com/paper/2110.11571/) (2021)
- [Influence Functions in Deep Learning Are Fragile](https://mlsanity.com/paper/2006.14651/) (2020)
- [Detecting AI Trojans Using Meta Neural Analysis](https://mlsanity.com/paper/1910.03137/) (2019)
- [Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization](https://mlsanity.com/paper/1708.08689/) (2017)
- [Representer Point Selection for Explaining Deep Neural Networks](https://mlsanity.com/paper/1811.09720/) (2018)
- [Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching](https://mlsanity.com/paper/2009.02276/) (2020)
- [Dynamic Backdoor Attacks Against Machine Learning Models](https://mlsanity.com/paper/2003.03675/) (2020)
- [Universal Litmus Patterns: Revealing Backdoor Attacks in CNNs](https://mlsanity.com/paper/1906.10842/) (2019)
- [Stronger Data Poisoning Attacks Break Data Sanitization Defenses](https://mlsanity.com/paper/1811.00741/) (2018)
- [Transferable Clean-Label Poisoning Attacks on Deep Neural Nets](https://mlsanity.com/paper/1905.05897/) (2019)

## Built on by (most cited first, 30 of 32)

- [TRAK: Attributing Model Behavior at Scale](https://mlsanity.com/paper/2303.14186/) (2023)
- [Training Data Influence Analysis and Estimation: A Survey](https://mlsanity.com/paper/2212.04612/) (2022)
- [Intriguing Properties of Data Attribution on Diffusion Models](https://mlsanity.com/paper/2311.00500/) (2023)
- [Diffusion Attribution Score: Evaluating Training Data Influence in Diffusion Models](https://mlsanity.com/paper/2410.18639/) (2024)
- [Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees](https://mlsanity.com/paper/2205.00359/) (2022)
- [TokenShapley: Token Level Context Attribution with Shapley Value](https://mlsanity.com/paper/2507.05261/) (2025)
- [Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifiers](https://mlsanity.com/paper/2310.18603/) (2023)
- [Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation](https://mlsanity.com/paper/2401.09031/) (2024)
- [Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generation](https://mlsanity.com/paper/2509.13772/) (2025)
- [Selectivity Drives Productivity: Efficient Dataset Pruning for Enhanced Transfer Learning](https://mlsanity.com/paper/2310.08782/) (2023)
- [Demystifying Poisoning Backdoor Attacks from a Statistical Perspective](https://mlsanity.com/paper/2310.10780/) (2023)
- [Reducing Certified Regression to Certified Classification for General Poisoning Attacks](https://mlsanity.com/paper/2208.13904/) (2022)
- [Sparse, Efficient and Explainable Data Attribution with DualXDA](https://mlsanity.com/paper/2402.12118/) (2024)
- [Tracing Back the Malicious Clients in Poisoning Attacks to Federated Learning](https://mlsanity.com/paper/2407.07221/) (2024)
- [A Simple and Efficient Baseline for Data Attribution on Images](https://mlsanity.com/paper/2311.03386/) (2023)
- [Large Language Model Sourcing: A Survey](https://mlsanity.com/paper/2510.10161/) (2025)
- [Mudjacking: Patching Backdoor Vulnerabilities in Foundation Models](https://mlsanity.com/paper/2402.14977/) (2024)
- [An Efficient Framework for Crediting Data Contributors of Diffusion Models](https://mlsanity.com/paper/2407.03153/) (2024)
- [Nonparametric Data Attribution for Diffusion Models](https://mlsanity.com/paper/2510.14269/) (2025)
- [Influential Training Data Retrieval for Explaining Verbalized Confidence of LLMs](https://mlsanity.com/paper/2601.10645/) (2026)
- [Beware What You Autocomplete: Forensic Attribution of Backdoored Code Completions](https://mlsanity.com/paper/2607.08011/) (2026)
- [UTrace: Poisoning Forensics for Private Collaborative Learning](https://mlsanity.com/paper/2409.15126/) (2024)
- [The Ultimate Cookbook for Invisible Poison: Crafting Subtle Clean-Label Text Backdoors with Style Attributes](https://mlsanity.com/paper/2504.17300/) (2025)
- [DICE: Data Influence Cascade in Decentralized Learning](https://mlsanity.com/paper/2507.06931/) (2025)
- [Influence-driven Curriculum Learning for Pre-training on Limited Data](https://mlsanity.com/paper/2508.15475/) (2025)
- [Imperfect Influence, Preserved Rankings: A Theory of TRAK for Data Attribution](https://mlsanity.com/paper/2602.01312/) (2026)
- [Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2607.07989/) (2026)
- [Representer Point Selection for Explaining Regularized High-dimensional Models](https://mlsanity.com/paper/2305.20002/) (2023)
- [SoK: Memorisation in machine learning](https://mlsanity.com/paper/2311.03075/) (2023)
- [Small-to-Large Generalization: Data Influences Models Consistently Across Scale](https://mlsanity.com/paper/2505.16260/) (2025)

## Abstract (arXiv)

Targeted training-set attacks inject malicious instances into the training set to cause a trained model to mislabel one or more specific test instances. This work proposes the task of target identification, which determines whether a specific test instance is the target of a training-set attack. Target identification can be combined with adversarial-instance identification to find (and remove) the attack instances, mitigating the attack with minimal impact on other predictions. Rather than focusing on a single attack method or data modality, we build on influence estimation, which quantifies each training instance's contribution to a model's prediction. We show that existing influence estimators' poor practical performance often derives from their over-reliance on training instances and iterations with large losses. Our renormalized influence estimators fix this weakness; they far outperform the original estimators at identifying influential groups of training examples in both adversarial and non-adversarial settings, even finding up to 100% of adversarial training instances with no clean-data false positives. Target identification then simplifies to detecting test instances with anomalous influence values. We demonstrate our method's effectiveness on backdoor and poisoning attacks across various data domains, including text, vision, and speech, as well as against a gray-box, adaptive attacker that specifically optimizes the adversarial instances to evade our method. Our source code is available at https://github.com/ZaydH/target_identification.