mlsanityData to 7 Oct 2026

Towards Identifying the Dataset Biases Causing Phantom Transfer

arXiv 2609.14449 ↗First posted 13 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent work has shown that a teacher model can transfer a bias to a student through a dataset from which every explicit reference to that bias has been filtered out, and that none of the tested data-level defenses reliably removes or detects such a bias, even when the defender knows what to look for. To shed light on the hidden traces these biases leave, we embed a dataset's completions with Sentence-BERT, subtract the embeddings of clean reference completions, and compare the result to an open vocabulary of candidate topics. This simple signature identifies the topic of the bias with a Matthews correlation coefficient of 0.83 when the attacker's teacher model is known, and 0.46 when it is not. We also observe that different teacher models appear to express the same bias through different vocabulary.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 21 references that are in the map. First posted on arXiv 13 Sep 2026.

Builds on

16 of 21 references.

References · 16

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  3. 2017Word Translation Without Parallel Datacited by 480
  4. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 303
  5. 2025Subliminal Learning: Language models transmit behavioral traits via hidden signals in datacited by 89
  6. 2025Towards Understanding Subliminal Learning: When and How Hidden Biases Transfercited by 30
  7. 2026Subliminal Effects in Your Data: A General Mechanism via Log-Linearitycited by 15
  8. 2026Subliminal Learning Is Steering Vector Distillationcited by 9
  9. 2026You Didn't Have to Say It like That: Subliminal Learning from Faithful Paraphrasescited by 7
  10. 2026From Data to Behavior: Predicting Unintended Model Behaviors Before Trainingcited by 6
  11. 2026Learning Through Noise: Why Subliminal Learning Works and When It Failscited by 6
  12. 2026Sustained Gradient Alignment Mediates Subliminal Learning in a Multi-Step Setting: Evidence from MNIST Auxiliary Logit Distillation Experimentcited by 5
4 more
  1. 2026Channel Location Constrains the Auditability of Subliminal Learningcited by 5
  2. 2026Subliminal Learning is a LoRA Artifactcited by 4
  3. 2026Covert Influence Between Language Modelscited by 4
  4. 2026Filtering Harmful Actions Isn't Enough: Phantom Transfer in Agentic SDFcited by 2

Built on by

Not cited yet.