# Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs

arXiv: https://arxiv.org/abs/2508.06601 · HTML: https://mlsanity.com/paper/2508.06601/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Aug 2025.

## Built on by (most cited first, 30 of 42)

- [Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time](https://mlsanity.com/paper/2510.04340/) (2025)
- [Model Spec Midtraining: Improving How Alignment Training Generalizes](https://mlsanity.com/paper/2605.02087/) (2026)
- [Beyond Linear Probes: Dynamic Safety Monitoring for Language Models](https://mlsanity.com/paper/2509.26238/) (2025)
- [Shaping capabilities with token-level data filtering](https://mlsanity.com/paper/2601.21571/) (2026)
- [Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Training](https://mlsanity.com/paper/2510.20956/) (2025)
- [Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs](https://mlsanity.com/paper/2512.05648/) (2025)
- [What Is The Political Content in LLMs' Pre- and Post-Training Data?](https://mlsanity.com/paper/2509.22367/) (2025)
- [When Should We Introduce Safety Interventions During Pretraining?](https://mlsanity.com/paper/2601.07087/) (2026)
- [Train Once, Answer All: Many Pretraining Experiments for the Cost of One](https://mlsanity.com/paper/2509.23383/) (2025)
- [Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influence](https://mlsanity.com/paper/2603.00498/) (2026)
- [AI Generated Child Sexual Abuse Material -- What's the Harm?](https://mlsanity.com/paper/2510.02978/) (2025)
- [Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearning](https://mlsanity.com/paper/2510.00761/) (2025)
- [Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sink](https://mlsanity.com/paper/2602.05228/) (2026)
- [Early Data Exposure Improves Robustness to Subsequent Fine-Tuning](https://mlsanity.com/paper/2605.12705/) (2026)
- [Reward Models Inherit Value Biases from Pretraining](https://mlsanity.com/paper/2601.20838/) (2026)
- [Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamics](https://mlsanity.com/paper/2606.06533/) (2026)
- [Bergson: An Open Source Library for Data Attribution](https://mlsanity.com/paper/2606.11660/) (2026)
- [Modular Pretraining Enables Access Control](https://mlsanity.com/paper/2607.08077/) (2026)
- [Position: Capability Control Should be a Separate Goal From Alignment](https://mlsanity.com/paper/2602.05164/) (2026)
- [TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering](https://mlsanity.com/paper/2602.06911/) (2026)
- [Reflections and New Directions for Human-Centered Large Language Models](https://mlsanity.com/paper/2605.06901/) (2026)
- [Mark, Don't Erase: Token Inoculation for Dual-Use Knowledge in LLMs](https://mlsanity.com/paper/2607.18639/) (2026)
- [Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Models](https://mlsanity.com/paper/2510.27629/) (2025)
- [Steerability of Instrumental-Convergence Tendencies in LLMs](https://mlsanity.com/paper/2601.01584/) (2026)
- [No More, No Less: Least-Privilege Language Models](https://mlsanity.com/paper/2601.23157/) (2026)
- [Beyond Safe Data: Pretraining-Stage Alignment with Regular Safety Reflection](https://mlsanity.com/paper/2606.19168/) (2026)
- [Data Attribution of Emergent Misalignment with Persona Features](https://mlsanity.com/paper/2608.11025/) (2026)
- [Synthetic Persona Pretraining: Alignment from Token Zero](https://mlsanity.com/paper/2608.13482/) (2026)
- [The Unequal Influence of Bad Advice: Using Training Data Attribution to Modulate Emergent Misalignment](https://mlsanity.com/paper/2609.37914/) (2026)
- [Getting Your Indices in a Row: Full-Text Search for LLM Training Data for Real World](https://mlsanity.com/paper/2510.09471/) (2025)

## Abstract (arXiv)

Open-weight AI systems offer unique benefits, including enhanced transparency, open research, and decentralized access. However, they are vulnerable to tampering attacks which can efficiently elicit harmful behaviors by modifying weights or activations. Currently, there is not yet a robust science of open-weight model risk management. Existing safety fine-tuning methods and other post-training techniques have struggled to make LLMs resistant to more than a few dozen steps of adversarial fine-tuning. In this paper, we investigate whether filtering text about dual-use topics from training data can prevent unwanted capabilities and serve as a more tamper-resistant safeguard. We introduce a multi-stage pipeline for scalable data filtering and show that it offers a tractable and effective method for minimizing biothreat proxy knowledge in LLMs. We pretrain multiple 6.9B-parameter models from scratch and find that they exhibit substantial resistance to adversarial fine-tuning attacks on up to 10,000 steps and 300M tokens of biothreat-related text -- outperforming existing post-training baselines by over an order of magnitude -- with no observed degradation to unrelated capabilities. However, while filtered models lack internalized dangerous knowledge, we find that they can still leverage such information when it is provided in context (e.g., via search tool augmentation), demonstrating a need for a defense-in-depth approach. Overall, these findings help to establish pretraining data curation as a promising layer of defense for open-weight AI systems.