# Preventing Machine Learning Poisoning Attacks Using Authentication and Provenance

arXiv: https://arxiv.org/abs/2105.10051 · HTML: https://mlsanity.com/paper/2105.10051/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security.
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 20 May 2021.

## Placed in

- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 14 of 14)

- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Efficient Decision-based Black-box Adversarial Attacks on Face Recognition](https://mlsanity.com/paper/1904.04433/) (2019)
- [Stronger Data Poisoning Attacks Break Data Sanitization Defenses](https://mlsanity.com/paper/1811.00741/) (2018)
- [Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases](https://mlsanity.com/paper/2007.15802/) (2020)
- [Data Poisoning Attacks on Factorization-Based Collaborative Filtering](https://mlsanity.com/paper/1608.08182/) (2016)
- [explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning](https://mlsanity.com/paper/1908.00087/) (2019)
- [Data Poisoning Attacks against Online Learning](https://mlsanity.com/paper/1808.08994/) (2018)
- [Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities](https://mlsanity.com/paper/2103.16007/) (2021)
- [Machine Learning Pipelines: Provenance, Reproducibility and FAIR Data Principles](https://mlsanity.com/paper/2006.12117/) (2020)
- [Provenance Data in the Machine Learning Lifecycle in Computational Science and Engineering](https://mlsanity.com/paper/1910.04223/) (2019)
- [AMP: Authentication of Media via Provenance](https://mlsanity.com/paper/2001.07886/) (2020)
- [dagger: A Python Framework for Reproducible Machine Learning Experiment Orchestration](https://mlsanity.com/paper/2006.07484/) (2020)
- [Debugging Machine Learning Pipelines](https://mlsanity.com/paper/2002.04640/) (2020)

## Built on by (most cited first, 5 of 5)

- [Unleashing the potential of prompt engineering for large language models](https://mlsanity.com/paper/2310.14735/) (2023)
- [Accumulative Poisoning Attacks on Real-time Data](https://mlsanity.com/paper/2106.09993/) (2021)
- [ZKPROV: A Zero-Knowledge Approach to Dataset Provenance for Large Language Models](https://mlsanity.com/paper/2506.20915/) (2025)
- [TraceFL: Interpretability-Driven Debugging in Federated Learning via Neuron Provenance](https://mlsanity.com/paper/2312.13632/) (2023)
- [On the Inference (In-)Security of Vertical Federated Learning: Efficient Auditing against Inference Tampering Attack](https://mlsanity.com/paper/2507.02376/) (2025)

## Abstract (arXiv)

Recent research has successfully demonstrated new types of data poisoning attacks. To address this problem, some researchers have proposed both offline and online data poisoning detection defenses which employ machine learning algorithms to identify such attacks. In this work, we take a different approach to preventing data poisoning attacks which relies on cryptographically-based authentication and provenance to ensure the integrity of the data used to train a machine learning model. The same approach is also used to prevent software poisoning and model poisoning attacks. A software poisoning attack maliciously alters one or more software components used to train a model. Once the model has been trained it can also be protected against model poisoning attacks which seek to alter a model's predictions by modifying its underlying parameters or structure. Finally, an evaluation set or test set can also be protected to provide evidence if they have been modified by a second data poisoning attack. To achieve these goals, we propose VAMP which extends the previously proposed AMP system, that was designed to protect media objects such as images, video files or audio clips, to the machine learning setting. We first provide requirements for authentication and provenance for a secure machine learning system. Next, we demonstrate how VAMP's manifest meets these requirements to protect a machine learning system's datasets, software components, and models.