mlsanityData to 14 Oct 2026

A General Framework for Data-Use Auditing of ML Models

arXiv 2407.15100 ↗First posted 21 Jul 2024cs.CR cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Auditing the use of data in training machine-learning (ML) models is an increasingly pressing challenge, as myriad ML practitioners routinely leverage the effort of content creators to train models without their permission. In this paper, we propose a general method to audit an ML model for the use of a data-owner's data in training, without prior knowledge of the ML task for which the data might be used. Our method leverages any existing black-box membership inference method, together with a sequential hypothesis test of our own design, to detect data use with a quantifiable, tunable false-detection rate. We show the effectiveness of our proposed framework by applying it to audit data use in two types of ML models, namely image classifiers and foundation models.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 51 of its 92 references that are in the map. First posted on arXiv 21 Jul 2024.

Builds on

51 of 92 references.

References · 51

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  5. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  9. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  10. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  11. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  12. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
38 more
  1. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  4. 2016Pointer Sentinel Mixture Modelscited by 3,096
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  6. 2016Deep Learning with Differential Privacycited by 2,728
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  8. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  9. 2020Extracting Training Data from Large Language Modelscited by 2,189
  10. 2017Decoupled Weight Decay Regularizationcited by 1,831
  11. 2019Machine Unlearningcited by 1,061
  12. 2021Membership Inference Attacks From First Principlescited by 878
  13. 2018Object Detection with Deep Learning: A Reviewcited by 528
  14. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  15. 2018HiDDeN: Hiding Data With Deep Networkscited by 397
  16. 2020Label-Only Membership Inference Attackscited by 388
  17. 2019Hidden Trigger Backdoor Attackscited by 371
  18. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 321
  19. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 292
  20. 2019StegaStamp: Invisible Hyperlinks in Physical Photographscited by 265
  21. 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 248
  22. 2020Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Datacited by 201
  23. 2019MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examplescited by 197
  24. 2018Machine Learning with Membership Privacy using Adversarial Regularizationcited by 186
  25. 2017Machine Learning Models that Remember Too Muchcited by 169
  26. 2020Witches' Brew: Industrial Scale Data Poisoning via Gradient Matchingcited by 163
  27. 2022Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protectioncited by 86
  28. 2021EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learningcited by 73
  29. 2020Radioactive data: tracing through trainingcited by 67
  30. 2023Domain Watermark: Effective and Harmless Dataset Copyright Protection is Closed at Handcited by 57
  31. 2020Towards Probabilistic Verification of Machine Unlearningcited by 50
  32. 2024Proving membership in LLM pretraining data via data watermarkscited by 40
  33. 2023Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Studycited by 38
  34. 2023Did You Train on My Dataset? Towards Public Dataset Protection with Clean-Label Backdoor Watermarkingcited by 37
  35. 2019The Audio Auditor: User-Level Membership Inference in Internet of Things Voice Servicescited by 25
  36. 2020Confidence sequences for sampling without replacementcited by 23
  37. 2023FACE-AUDITOR: Data Auditing in Facial Recognition Systemscited by 16
  38. 2022Data Isotopes for Data Provenance in DNNscited by 11

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 7; 2026: 7.

16 papers.

Sort:

By area:

  1. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 9
  2. 2025DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspectivecited by 8
  3. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  4. 2024Anonymity Unveiled: A Practical Framework for Auditing Data Use in Deep Learning Modelscited by 4
  5. 2025Instance-Level Data-Use Auditing of Visual ML Modelscited by 4
  6. 2025Toward Efficient Inference Attacks: Shadow Model Sharing via Mixture-of-Expertscited by 4
  7. 2024Towards Data Governance of Frontier AI Modelscited by 3
  8. 2025Evading Data Provenance in Deep Neural Networkscited by 2
  9. 2025DSSmoothing: Toward Certified Dataset Ownership Verification for Pre-trained Language Models via Dual-Space Smoothingcited by 2
  10. 2025VICTOR: Dataset Copyright Auditing in Video Recognition Systemscited by 1
  11. 2026Weight space Detection of Backdoors in LoRA Adapterscited by 1
  12. 2026Implicit Identity Technologies for LLMs: Fingerprinting and Watermarking across Datasets, Models, and Generated Contentcited by 1
4 more
  1. 2026Auditing Training Data in Generative Music Models via Black-Box Membership Inferencecited by 0
  2. 2026DIPBox: A Multi-scale Testing Framework for Tracking Dataset Regenerationcited by 0
  3. 2026Trustworthy synthetic visual media: Evidence across the media lifecyclecited by 0
  4. 2026The Null Is the Hard Part: Exact Tests for Memorization in Generative Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SMI: Statistical Membership Inference for Reliable Unlearned Model Auditing
  2. 2026Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget
  3. 2024SoK: Dataset Copyright Auditing in Machine Learning Systems
  4. 2025TAPE: Tailored Posterior Difference for Auditing of Machine Unlearning
  5. 2026MemCatalyst: Amplifying Data Auditing on Vision-Language Models via Data Poisoning
  6. 2025Membership Inference Attacks for Unseen Classes
  7. 2026Characterizing Privacy-Audit Alignment in Behavioral Audit of Machine Unlearning
  8. 2026Auditing of Unlearning Algorithms