# A General Framework for Data-Use Auditing of ML Models

arXiv: https://arxiv.org/abs/2407.15100 · HTML: https://mlsanity.com/paper/2407.15100/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 92 references that are in the map.
- First posted on arXiv 21 Jul 2024.

## Builds on (most cited first, 30 of 51)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [HiDDeN: Hiding Data With Deep Networks](https://mlsanity.com/paper/1807.09937/) (2018)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Hidden Trigger Backdoor Attacks](https://mlsanity.com/paper/1910.00033/) (2019)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)

## Built on by (most cited first, 16 of 16)

- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspective](https://mlsanity.com/paper/2507.05622/) (2025)
- [Beyond Data Privacy: New Privacy Risks for Large Language Models](https://mlsanity.com/paper/2509.14278/) (2025)
- [Anonymity Unveiled: A Practical Framework for Auditing Data Use in Deep Learning Models](https://mlsanity.com/paper/2409.06280/) (2024)
- [Instance-Level Data-Use Auditing of Visual ML Models](https://mlsanity.com/paper/2503.22413/) (2025)
- [Toward Efficient Inference Attacks: Shadow Model Sharing via Mixture-of-Experts](https://mlsanity.com/paper/2510.13451/) (2025)
- [Towards Data Governance of Frontier AI Models](https://mlsanity.com/paper/2412.03824/) (2024)
- [Evading Data Provenance in Deep Neural Networks](https://mlsanity.com/paper/2508.01074/) (2025)
- [DSSmoothing: Toward Certified Dataset Ownership Verification for Pre-trained Language Models via Dual-Space Smoothing](https://mlsanity.com/paper/2510.15303/) (2025)
- [VICTOR: Dataset Copyright Auditing in Video Recognition Systems](https://mlsanity.com/paper/2512.14439/) (2025)
- [Weight space Detection of Backdoors in LoRA Adapters](https://mlsanity.com/paper/2602.15195/) (2026)
- [Implicit Identity Technologies for LLMs: Fingerprinting and Watermarking across Datasets, Models, and Generated Content](https://mlsanity.com/paper/2605.29245/) (2026)
- [Auditing Training Data in Generative Music Models via Black-Box Membership Inference](https://mlsanity.com/paper/2605.29202/) (2026)
- [DIPBox: A Multi-scale Testing Framework for Tracking Dataset Regeneration](https://mlsanity.com/paper/2606.21240/) (2026)
- [Trustworthy synthetic visual media: Evidence across the media lifecycle](https://mlsanity.com/paper/2609.34232/) (2026)
- [The Null Is the Hard Part: Exact Tests for Memorization in Generative Models](https://mlsanity.com/paper/2610.00251/) (2026)

## Abstract (arXiv)

Auditing the use of data in training machine-learning (ML) models is an increasingly pressing challenge, as myriad ML practitioners routinely leverage the effort of content creators to train models without their permission. In this paper, we propose a general method to audit an ML model for the use of a data-owner's data in training, without prior knowledge of the ML task for which the data might be used. Our method leverages any existing black-box membership inference method, together with a sequential hypothesis test of our own design, to detect data use with a quantifiable, tunable false-detection rate. We show the effectiveness of our proposed framework by applying it to audit data use in two types of ML models, namely image classifiers and foundation models.