# Extracting books from production language models

arXiv: https://arxiv.org/abs/2601.02671 · HTML: https://mlsanity.com/paper/2601.02671/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 25 of its 73 references that are in the map.
- First posted on arXiv 6 Jan 2026.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 25 of 25)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Unsolved Problems in ML Safety](https://mlsanity.com/paper/2109.13916/) (2021)
- [Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming](https://mlsanity.com/paper/2501.18837/) (2025)
- [Rethinking LLM Memorization through the Lens of Adversarial Compression](https://mlsanity.com/paper/2404.15146/) (2024)
- [Talkin' 'Bout AI Generation: Copyright and the Generative-AI Supply Chain](https://mlsanity.com/paper/2309.08133/) (2023)
- [Measuring memorization in language models via probabilistic extraction](https://mlsanity.com/paper/2410.19482/) (2024)
- [Extracting memorized pieces of (copyrighted) books from open-weight language models](https://mlsanity.com/paper/2505.12546/) (2025)
- [Context-Aware Membership Inference Attacks against Pre-trained Large Language Models](https://mlsanity.com/paper/2409.13745/) (2024)
- [Exploring the limits of strong membership inference attacks on large language models](https://mlsanity.com/paper/2505.18773/) (2025)
- [The Files are in the Computer: On Copyright, Memorization, and Generative AI](https://mlsanity.com/paper/2404.12590/) (2024)
- [Hubble: a Model Suite to Advance the Study of LLM Memorization](https://mlsanity.com/paper/2510.19811/) (2025)
- [Report of the 1st Workshop on Generative AI and Law](https://mlsanity.com/paper/2311.06477/) (2023)

## Built on by (most cited first, 17 of 17)

- [Alignment Whack-a-Mole : Finetuning Activates Verbatim Recall of Copyrighted Books in Large Language Models](https://mlsanity.com/paper/2603.20957/) (2026)
- [Characterizing Memorization in Diffusion Language Models: Generalized Extraction and Sampling Effects](https://mlsanity.com/paper/2603.02333/) (2026)
- [Memorization Dynamics in Knowledge Distillation for Language Models](https://mlsanity.com/paper/2601.15394/) (2026)
- [Merlin: Deterministic Byte-Exact Deduplication for Lossless Context Optimization in Large Language Model Inference](https://mlsanity.com/paper/2605.09990/) (2026)
- [Copyright Detective: A Forensic System to Evidence LLMs Flickering Copyright Leakage Risks](https://mlsanity.com/paper/2602.05252/) (2026)
- [Both Ends Count! Just How Good are LLM Agents at "Text-to-Big SQL"?](https://mlsanity.com/paper/2602.21480/) (2026)
- [Estimating near-verbatim extraction risk in language models with decoding-constrained beam search](https://mlsanity.com/paper/2603.24917/) (2026)
- [LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs](https://mlsanity.com/paper/2606.06286/) (2026)
- [IDP-Bench: Benchmarking ability of LLMs to protect personal information in interdependent privacy contexts](https://mlsanity.com/paper/2606.09908/) (2026)
- [Learning to Detect Language Model Training Data via Active Reconstruction](https://mlsanity.com/paper/2602.19020/) (2026)
- [Overcoming Copyright Barriers in Corpus Distribution Through Non-Reversible Hashing](https://mlsanity.com/paper/2604.23412/) (2026)
- [FindMyText: Robust, Scalable Detection of Text Containment in Large Web-Crawled Corpora](https://mlsanity.com/paper/2607.10020/) (2026)
- [Extractable Memorization From First Principles](https://mlsanity.com/paper/2607.12649/) (2026)
- [Reasoning or Memorization: Can LLMs Understand and Generate Chinese Xiehouyu Riddles?](https://mlsanity.com/paper/2607.23440/) (2026)
- [What to Forget in Unlearning? Forget Set Curation for Language Models](https://mlsanity.com/paper/2608.14855/) (2026)
- [CopyShield: A Cross-Level Benchmark of Copyright Defenses in LLMs](https://mlsanity.com/paper/2609.01161/) (2026)
- [Detecting Pretraining Data in Large Language Models from a Free-Energy Perspective](https://mlsanity.com/paper/2609.21888/) (2026)

## Abstract (arXiv)

Many unresolved legal questions over LLMs and copyright center on memorization: whether specific training data have been encoded in the model's weights during training, and whether those memorized data can be extracted in the model's outputs. While many believe that LLMs do not memorize much of their training data, recent work shows that substantial amounts of copyrighted text can be extracted from open-weight models. However, it remains an open question if similar extraction is feasible for production LLMs, given the safety measures these systems implement. We investigate this question using a two-phase procedure: (1) an initial probe to test for extraction feasibility, which sometimes uses a Best-of-N (BoN) jailbreak, followed by (2) iterative continuation prompts to attempt to extract the book. We evaluate our procedure on four production LLMs -- Claude 3.7 Sonnet, GPT-4.1, Gemini 2.5 Pro, and Grok 3 -- and we measure extraction success with a score computed from a block-based approximation of longest common substring (nv-recall). With different per-LLM experimental configurations, we were able to extract varying amounts of text. For the Phase 1 probe, it was unnecessary to jailbreak Gemini 2.5 Pro and Grok 3 to extract text (e.g, nv-recall of 76.8% and 70.3%, respectively, for Harry Potter and the Sorcerer's Stone), while it was necessary for Claude 3.7 Sonnet and GPT-4.1. In some cases, jailbroken Claude 3.7 Sonnet outputs entire books near-verbatim (e.g., nv-recall=95.8%). GPT-4.1 requires significantly more BoN attempts (e.g., 20X), and eventually refuses to continue (e.g., nv-recall=4.0%). Taken together, our work highlights that, even with model- and system-level safeguards, extraction of (in-copyright) training data remains a risk for production LLMs.