# What Hard Tokens Reveal: Exploiting Low-confidence Tokens for Membership Inference Attacks against Large Language Models

arXiv: https://arxiv.org/abs/2601.20885 · HTML: https://mlsanity.com/paper/2601.20885/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 32 of its 58 references that are in the map.
- First posted on arXiv 27 Jan 2026.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 30 of 32)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [Fine-Tuning Language Models with Just Forward Passes](https://mlsanity.com/paper/2305.17333/) (2023)
- [MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples](https://mlsanity.com/paper/1909.10594/) (2019)
- [The pitfalls of next-token prediction](https://mlsanity.com/paper/2403.06963/) (2024)
- [LLM Dataset Inference: Did you train on my dataset?](https://mlsanity.com/paper/2406.06443/) (2024)
- [Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration](https://mlsanity.com/paper/2311.06062/) (2023)
- [ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluation](https://mlsanity.com/paper/2306.09968/) (2023)
- [SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)](https://mlsanity.com/paper/2406.17975/) (2024)
- [Training LLM-based Tutors to Improve Student Learning Outcomes in Dialogues](https://mlsanity.com/paper/2503.06424/) (2025)
- [Fine-tuning and Utilization Methods of Domain-specific LLMs](https://mlsanity.com/paper/2401.02981/) (2024)
- [Defending Model Inversion and Membership Inference Attacks via Prediction Purification](https://mlsanity.com/paper/2005.03915/) (2020)
- [Data Contamination Calibration for Black-box LLMs](https://mlsanity.com/paper/2405.11930/) (2024)
- [Context-Aware Membership Inference Attacks against Pre-trained Large Language Models](https://mlsanity.com/paper/2409.13745/) (2024)
- [Sampling-based Pseudo-Likelihood for Membership Inference Attacks](https://mlsanity.com/paper/2404.11262/) (2024)
- [MIST: Defending Against Membership Inference Attacks Through Membership-Invariant Subspace Training](https://mlsanity.com/paper/2311.00919/) (2023)
- [LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM states](https://mlsanity.com/paper/2411.19876/) (2024)
- [SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attacks](https://mlsanity.com/paper/2506.10424/) (2025)
- [Supervised Fine-Tuning LLMs to Behave as Pedagogical Agents in Programming Education](https://mlsanity.com/paper/2502.20527/) (2025)
- [Adapting LLMs for the Medical Domain in Portuguese: A Study on Fine-Tuning and Model Evaluation](https://mlsanity.com/paper/2410.00163/) (2024)
- [Pruning as a Defense: Reducing Memorization in Large Language Models](https://mlsanity.com/paper/2502.15796/) (2025)

## Abstract (arXiv)

With the widespread adoption of Large Language Models (LLMs) and increasingly stringent privacy regulations, protecting data privacy in LLMs has become essential, especially for privacy-sensitive applications. Membership Inference Attacks (MIAs) attempt to determine whether a specific data sample was included in the model training/fine-tuning dataset, posing serious privacy risks. However, most existing MIA techniques against LLMs rely on sequence-level aggregated prediction statistics, which fail to distinguish prediction improvements caused by generalization from those caused by memorization, leading to low attack effectiveness. To address this limitation, we propose a novel membership inference approach that captures the token-level probabilities for low-confidence (hard) tokens, where membership signals are more pronounced. By comparing token-level probability improvements at hard tokens between a fine-tuned target model and a pre-trained reference model, HT-MIA isolates strong and robust membership signals that are obscured by prior MIA approaches. Extensive experiments on both domain-specific medical datasets and general-purpose benchmarks demonstrate that HT-MIA consistently outperforms seven state-of-the-art MIA baselines. We further investigate differentially private training as an effective defense mechanism against MIAs in LLMs. Overall, our HT-MIA framework establishes hard-token based analysis as a state-of-the-art foundation for advancing membership inference attacks and defenses for LLMs.