# Targeted Attack on GPT-Neo for the SATML Language Model Data Extraction Challenge

arXiv: https://arxiv.org/abs/2302.07735 · HTML: https://mlsanity.com/paper/2302.07735/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 7 of its 12 references that are in the map.
- First posted on arXiv 13 Feb 2023.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 7 of 7)

- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks](https://mlsanity.com/paper/2203.03929/) (2022)
- [CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequences](https://mlsanity.com/paper/2202.06689/) (2022)
- [Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries](https://mlsanity.com/paper/2301.01701/) (2023)

## Built on by (most cited first, 8 of 8)

- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [Identifying and Mitigating Privacy Risks Stemming from Language Models: A Survey](https://mlsanity.com/paper/2310.01424/) (2023)
- [Traces of Memorisation in Large Language Models for Code](https://mlsanity.com/paper/2312.11658/) (2023)
- [An Exploratory Investigation into Code License Infringements in Large Language Model Training Datasets](https://mlsanity.com/paper/2403.15230/) (2024)
- [LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures](https://mlsanity.com/paper/2505.01177/) (2025)
- [How Much Do Code Language Models Remember? An Investigation on Data Extraction Attacks before and after Fine-tuning](https://mlsanity.com/paper/2501.17501/) (2025)
- [Data Compressibility Quantifies LLM Memorization](https://mlsanity.com/paper/2507.06056/) (2025)
- [Towards Privacy-Preserving Code Generation: Differentially Private Code Language Models](https://mlsanity.com/paper/2512.11482/) (2025)

## Abstract (arXiv)

Previous work has shown that Large Language Models are susceptible to so-called data extraction attacks. This allows an attacker to extract a sample that was contained in the training data, which has massive privacy implications. The construction of data extraction attacks is challenging, current attacks are quite inefficient, and there exists a significant gap in the extraction capabilities of untargeted attacks and memorization. Thus, targeted attacks are proposed, which identify if a given sample from the training data, is extractable from a model. In this work, we apply a targeted data extraction attack to the SATML2023 Language Model Training Data Extraction Challenge. We apply a two-step approach. In the first step, we maximise the recall of the model and are able to extract the suffix for 69% of the samples. In the second step, we use a classifier-based Membership Inference Attack on the generations. Our AutoSklearn classifier achieves a precision of 0.841. The full approach reaches a score of 0.405 recall at a 10% false positive rate, which is an improvement of 34% over the baseline of 0.301.