mlsanityData to 7 Oct 2026

AutoMIA: Improved Baselines for Membership Inference Attack via Agentic Self-Exploration

arXiv 2604.01014 ↗First posted 1 Apr 2026cs.CR cs.CVcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Membership Inference Attacks (MIAs) serve as a fundamental auditing tool for evaluating training data leakage in machine learning models. However, existing methodologies predominantly rely on static, handcrafted heuristics that lack adaptability, often leading to suboptimal performance when transferred across different large models. In this work, we propose AutoMIA, an agentic framework that reformulates membership inference as an automated process of self-exploration and strategy evolution. Given high-level scenario specifications, AutoMIA self-explores the attack space by generating executable logits-level strategies and progressively refining them through closed-loop evaluation feedback. By decoupling abstract strategy reasoning from low-level execution, our framework enables a systematic, model-agnostic traversal of the attack search space. Extensive experiments demonstrate that AutoMIA consistently matches or outperforms state-of-the-art baselines while eliminating the need for manual feature engineering.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 39 of its 48 references that are in the map. First posted on arXiv 1 Apr 2026.

Builds on

39 of 48 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Sponge Tool Attack: Stealthy Denial-of-Efficiency against Tool-Augmented Agentic Reasoning24 Jan
  2. 2025AttackPilot: Autonomous Inference Attacks Against ML Services With LLM-Based Agents24 Nov
  3. 2025Shadows in the Code: Exploring the Risks and Defenses of LLM-based Multi-Agent Software Development Systems23 Nov

Other references · 36

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2023Qwen Technical Reportcited by 3,273
  5. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  6. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  7. 2020Extracting Training Data from Large Language Modelscited by 2,144
  8. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  10. 2021Membership Inference Attacks From First Principlescited by 839
  11. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  12. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
24 more
  1. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  2. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  3. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  4. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
  5. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  6. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
  7. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbotscited by 172
  8. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 102
  9. 2022Membership Inference Attacks by Exploiting Loss Trajectorycited by 93
  10. 2024RingID: Rethinking Tree-Ring Watermarking for Enhanced Multi-Key Identificationcited by 87
  11. 2024Blind Baselines Beat Membership Inference Attacks for Foundation Modelscited by 65
  12. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  13. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)cited by 54
  14. 2024Membership Inference Attacks Against In-Context Learningcited by 38
  15. 2024Membership Inference Attacks against Large Vision-Language Modelscited by 31
  16. 2024IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generationcited by 21
  17. 2025RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learningcited by 20
  18. 2024Anti-Reference: Universal and Immediate Defense Against Reference-Based Generationcited by 17
  19. 2025Vid-SME: Membership Inference Attacks against Large Video Understanding Modelscited by 17
  20. 2025Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completenesscited by 11
  21. 2024Data Lineage Inference: Uncovering Privacy Vulnerabilities of Dataset Pruningcited by 5
  22. 2025Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMscited by 4
  23. 2026Refinement Provenance Inference: Detecting LLM-Refined Training Prompts from Model Behaviorcited by 3
  24. 2026Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Automated Membership Inference Attacks (AutoMIA): Discovering MIA Signal Computations using LLM Agents
  2. 2025AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security
  3. 2026Honeyquest for LLMs: Rethinking Cyber Deception for AI Attackers
  4. 2026SoK: The Attack Surface of Agentic AI - Tools and Autonomy
  5. 2026Assessing Automated Prompt Injection Attacks in Agentic Environments
  6. 2026AutoRISE: Agent-Driven Strategy Evolution for Red-Teaming Large Language Models
  7. 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
  8. 2026SelfOp: An Optimization Algorithm for Self-Improving Security Agents