mlsanityData to 7 Oct 2026

Is My Text in Your AI Model? Gradient-based Membership Inference Test applied to LLMs

arXiv 2503.07384 ↗First posted 10 Mar 2025cs.CL cs.AIcited by 9

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

This work adapts and studies the gradient-based Membership Inference Test (gMINT) to the classification of text based on LLMs. MINT is a general approach intended to determine if given data was used for training machine learning models, and this work focuses on its application to the domain of Natural Language Processing. Using gradient-based analysis, the MINT model identifies whether particular data samples were included during the language model training phase, addressing growing concerns about data privacy in machine learning. The method was evaluated in seven Transformer-based models and six datasets comprising over 2.5 million sentences, focusing on text classification tasks. Experimental results demonstrate MINTs robustness, achieving AUC scores between 85% and 99%, depending on data size and model architecture. These findings highlight MINTs potential as a scalable and reliable tool for auditing machine learning models, ensuring transparency, safeguarding sensitive data, and fostering ethical compliance in the deployment of AI/NLP technologies.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 22 of its 44 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

22 of 44 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Is My Data in Your AI? Membership Inference Test (MINT) applied to Face Biometrics14 Feb
  2. 2024Do Membership Inference Attacks Work on Large Language Models?12 Feb
  3. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparison29 May
  4. 2022Similarity Distribution based Membership Inference Attack on Person Re-identification29 Nov
  5. 2022Membership Inference Attacks by Exploiting Loss Trajectory31 Aug
  6. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks8 Mar
  7. 2021On the Importance of Difficulty Calibration in Membership Inference Attacks15 Nov
  8. 2021Membership Inference Attacks on Machine Learning: A Survey14 Mar

Other references · 14

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  3. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  4. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  6. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  7. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  8. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
  9. 2020SensitiveLoss: Improving Accuracy and Fairness of Face Representations with Discrimination-Aware Deep Learningcited by 50
  10. 2023Natural Language Processing in the Legal Domaincited by 38
  11. 2023User Inference Attacks on Large Language Modelscited by 36
  12. 2023Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitmentcited by 32
2 more
  1. 2023A Review of the Trends and Challenges in Adopting Natural Language Processing Methods for Education Feedback Analysiscited by 12
  2. 2024A multimodal dataset for understanding the impact of mobile phones on remote online virtual educationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 4.

9 papers.

  1. 2025PBa-LLM: Privacy- and Bias-aware NLP using Named-Entity Recognition (NER)cited by 10
  2. 2025Benchmarking Graph Neural Networks for Document Layout Analysis in Public Affairscited by 6
  3. 2026Is My Vision-Language Data in Your AI? Membership Inference Test (MINT) Demo 2cited by 6
  4. 2025MINT-Demo: Membership Inference Test Demonstratorcited by 3
  5. 2026Auditing Training Data in Domain-adapted LLMs: LoRA-MINTcited by 3
  6. 2025Balancing Tails when Comparing Distributions: Comprehensive Equity Index (CEI) with Application to Bias Evaluation in Operational Face Biometricscited by 2
  7. 2025Active Membership Inference Test (aMINT): Enhancing Model Auditability with Multi-Task Learningcited by 2
  8. 2026Pretraining Data Exposure in Large Language Models: A Survey of Membership Inference, Data Contamination, and Security Implicationscited by 1
  9. 2026Auditing Data Provenance in LLM Fine-tuning via Intrinsic Distributional Fingerprintscited by 0