# Can Data Attribution Filter Out Subliminal Learning? Not Reliably

arXiv: https://arxiv.org/abs/2609.20027 · HTML: https://mlsanity.com/paper/2609.20027/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 22 references that are in the map.
- First posted on arXiv 17 Sep 2026.

## Builds on (most cited first, 18 of 18)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Persona Vectors: Monitoring and Controlling Character Traits in Language Models](https://mlsanity.com/paper/2507.21509/) (2025)
- [Datamodels: Predicting Predictions from Training Data](https://mlsanity.com/paper/2202.00622/) (2022)
- [On Evaluating the Durability of Safeguards for Open-Weight LLMs](https://mlsanity.com/paper/2412.07097/) (2024)
- [Safety Pretraining: Toward the Next Generation of Safe AI](https://mlsanity.com/paper/2504.16980/) (2025)
- [Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs](https://mlsanity.com/paper/2508.06601/) (2025)
- [Towards Understanding Subliminal Learning: When and How Hidden Biases Transfer](https://mlsanity.com/paper/2509.23886/) (2025)
- [Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment](https://mlsanity.com/paper/2601.10160/) (2026)
- [Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing Guardrail Moderation](https://mlsanity.com/paper/2501.17433/) (2025)
- [Do Influence Functions Work on Large Language Models?](https://mlsanity.com/paper/2409.19998/) (2024)
- [Subliminal Steering: Stronger Encoding of Hidden Signals](https://mlsanity.com/paper/2604.25783/) (2026)
- [Shaping capabilities with token-level data filtering](https://mlsanity.com/paper/2601.21571/) (2026)
- [Subliminal Learning Is Steering Vector Distillation](https://mlsanity.com/paper/2606.00995/) (2026)
- [Thought Virus: Viral Misalignment via Subliminal Prompting in Multi-Agent Systems](https://mlsanity.com/paper/2603.00131/) (2026)
- [You Didn't Have to Say It like That: Subliminal Learning from Faithful Paraphrases](https://mlsanity.com/paper/2603.09517/) (2026)
- [Sparse, Efficient and Explainable Data Attribution with DualXDA](https://mlsanity.com/paper/2402.12118/) (2024)
- [Subliminal Signals in Preference Labels](https://mlsanity.com/paper/2603.01204/) (2026)

## Abstract (arXiv)

Subliminal learning allows language models to transmit behavioral traits through training data with no obvious semantic relationship to those traits, undermining content-based data filtering as a safety intervention. Training data attribution offers an alternative: it identifies the training examples responsible for a given model behavior, independent of their semantic content, and so may apply in exactly the cases where semantic inspection fails. We evaluate three gradient-based attribution methods (GradCos, a contrastive GradCos variant, and EK-FAC) across three models, comparing them against divergence tokens, a strong baseline previously shown to localize subliminal learning (albeit one that requires access to counterfactual teacher models). Filtering at the token level, EK-FAC mitigates a significant part of the effect, the other methods provide little benefit, and all mostly fall short of divergence tokens. Filtering entire samples is less effective for every method, though EK-FAC often gives a stronger signal than divergence tokens in this setting. Success is inconsistent across methods and settings: variants that work well for some model-preference combinations fail for others, and we do not identify a consistent explanation for these differences. Our results suggest that gradient-based attribution can identify data responsible for subliminal learning in some settings, but that some approximations are more reliable than others.