mlsanityData to 7 Oct 2026

Membership Inference on LLMs in the Wild

arXiv 2601.11314 ↗First posted 16 Jan 2026cs.CLcited by 2

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Membership Inference Attacks (MIAs) act as a crucial auditing tool for the opaque training data of Large Language Models (LLMs). However, existing techniques predominantly rely on inaccessible model internals (e.g., logits) or suffer from poor generalization across domains in strict black-box settings where only generated text is available. In this work, we propose SimMIA, a robust MIA framework tailored for this text-only regime by leveraging an advanced sampling strategy and scoring mechanism. Furthermore, we present WikiMIA-25, a new benchmark curated to evaluate MIA performance on modern proprietary LLMs. Experiments demonstrate that SimMIA achieves state-of-the-art results in the black-box setting, rivaling baselines that exploit internal model information.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 21 of its 33 references that are in the map. First posted on arXiv 16 Jan 2026.

Builds on

21 of 33 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models26 Feb
  2. 2024ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods23 Jun
  3. 2024Sampling-based Pseudo-Likelihood for Membership Inference Attacks17 Apr
  4. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration10 Nov
  5. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparison29 May
  6. 2021Membership Inference Attacks From First Principles7 Dec

Other references · 15

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  6. 2020Extracting Training Data from Large Language Modelscited by 2,144
  7. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  8. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  9. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  10. 2024DE-COP: Detecting Copyrighted Content in Language Models Training Datacited by 72
  11. 2024Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Methodcited by 54
  12. 2024Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decodingcited by 20
3 more
  1. 2024MIA-Tuner: Adapting Large Language Models as Pre-training Text Detectorcited by 13
  2. 2025Blackbox Model Provenance via Palimpsestic Membership Inferencecited by 11
  3. 2025Automated Detection of Pre-training Text in Black-box LLMscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Provable Joint Decontamination for Benchmarking Multiple Large Language Modelscited by 0
  2. 2026Black-Box Membership Inference via Word-Level Probability Estimationcited by 0