mlsanityData to 14 Oct 2026

Automated Membership Inference Attacks (AutoMIA): Discovering MIA Signal Computations using LLM Agents

arXiv 2603.19375 ↗First posted 19 Mar 2026cs.CR cs.LGcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Membership inference attacks (MIAs), which enable adversaries to determine whether specific data points were part of a model's training dataset, have emerged as an important framework to understand, assess, and quantify the potential information leakage associated with machine learning systems. Designing effective MIAs is a challenging task that usually requires extensive manual exploration of model behaviors to identify potential vulnerabilities. In this paper, we introduce AutoMIA -- a novel framework that leverages large language model (LLM) agents to automate the design and implementation of new MIA signal computations. By utilizing LLM agents, we can systematically explore a vast space of potential attack strategies, enabling the discovery of novel strategies. Our experiments demonstrate AutoMIA can successfully discover new MIAs that are specifically tailored to user-configured target model and dataset, resulting in improvements of up to 0.18 in absolute AUC over existing MIAs. This work provides the first demonstration that LLM agents can serve as an effective and scalable paradigm for designing and implementing MIAs with SOTA performance, opening up new avenues for future exploration.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 41 of its 53 references that are in the map. First posted on arXiv 19 Mar 2026.

Builds on

41 of 53 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025AttackPilot: Autonomous Inference Attacks Against ML Services With LLM-Based Agents24 Nov

Other references · 40

  1. 2022Generative Adversarial Networkscited by 5,627
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  3. 2020Extracting Training Data from Large Language Modelscited by 2,189
  4. 2021Membership Inference Attacks From First Principlescited by 878
  5. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
  6. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  7. 2020Label-Only Membership Inference Attackscited by 388
  8. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  9. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 321
  10. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 258
  11. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 195
  12. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibrationcited by 98
28 more
  1. 2018Understanding Membership Inferences on Well-Generalized Learning Modelscited by 90
  2. 2023Membership Inference Attacks against Diffusion Modelscited by 86
  3. 2025Mathematical exploration and discovery at scalecited by 77
  4. 2024Blind Baselines Beat Membership Inference Attacks for Foundation Modelscited by 69
  5. 2023Membership Inference Attacks against Synthetic Data through Overfitting Detectioncited by 66
  6. 2018A standardized Project Gutenberg corpus for statistical analysis of natural language and quantitative linguisticscited by 64
  7. 2024Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generationcited by 44
  8. 2023White-box Membership Inference Attacks against Diffusion Modelscited by 42
  9. 2021Membership Inference on Word Embedding and Beyondcited by 35
  10. 2024Mask-based Membership Inference Attacks for Retrieval-Augmented Generationcited by 29
  11. 2024Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignmentcited by 25
  12. 2026K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Modelcited by 24
  13. 2025Exploring the limits of strong membership inference attacks on large language modelscited by 22
  14. 2025GPU Kernel Scientist: An LLM-Driven Framework for Iterative Kernel Optimizationcited by 22
  15. 2024Detecting Machine-Generated Texts by Multi-Population Aware Optimization for Maximum Mean Discrepancycited by 20
  16. 2025Vid-SME: Membership Inference Attacks against Large Video Understanding Modelscited by 19
  17. 2023Synthetic is all you need: removing the auxiliary data assumption for membership inference attacks against synthetic datacited by 18
  18. 2025Membership Inference Attacks on Large-Scale Models: A Surveycited by 18
  19. 2025Neural Architecture Discovery via Autonomous Evolutioncited by 16
  20. 2024MIA-Tuner: Adapting Large Language Models as Pre-training Text Detectorcited by 15
  21. 2025AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defensescited by 15
  22. 2023Code Membership Inference for Detecting Unauthorized Data Use in Code Pre-trained Language Modelscited by 8
  23. 2025The Surprising Effectiveness of Membership Inference with Simple N-Gram Coveragecited by 7
  24. 2026ArchAgent: Agentic AI-driven Computer Architecture Discoverycited by 6
  25. 2026Discovering Multiagent Learning Algorithms with Large Language Modelscited by 4
  26. 2025Membership Inference Attacks on Tokenizers of Large Language Modelscited by 3
  27. 2025Membership and Dataset Inference Attacks on Large Audio Generative Modelscited by 2
  28. 2026When Reasoning Leaks Membership: Membership Inference Attack on Black-box Large Reasoning Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AutoMIA: Improved Baselines for Membership Inference Attack via Agentic Self-Exploration
  2. 2024LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM states
  3. 2026Membership Inference on LLMs in the Wild
  4. 2024Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Models
  5. 2026CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models
  6. 2024EM-MIAs: Enhancing Membership Inference Attacks in Large Language Models through Ensemble Modeling
  7. 2024Order of Magnitude Speedups for LLM Membership Inference
  8. 2026AttenMIA: LLM Membership Inference Attack through Attention Signals