mlsanityData to 7 Oct 2026

Channel Location Constrains the Auditability of Subliminal Learning

arXiv 2606.22019 ↗First posted 20 Jun 2026cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Subliminal learning lets a student inherit a teacher's hidden trait from distillation data that never names it. We ask when such transfer can be audited before training. The answer is not model identity or scale alone, but channel location: the carrier through which the trait reaches the student. We find three regimes. In a controlled initialization-dependent body channel, a pre-training screen works. Coverage, the cosine between the student's initial distillation update and the teacher's fine-tuning displacement, predicts held-out transfer (Spearman ρ ≈ 0.95; AUROC 0.997). In pretrained language models, masked single-token traits instead ride convergent vocabulary geometry. This channel is initialization-independent, so initialization-alignment screens, including coverage, are not mechanistic; the useful handles are post-hoc detection and targeted mitigation. Even when a single-token named entity is removed from the loss, the student's held-out probability for that entity rises to 0.40 on average (~ 2500×), and a related semantic class transfers. In an untied-head model, orthogonalizing the trait's output row against entangled neighbours collapses leakage, while equal-size random-subspace edits do not. Thus removing a target string from distillation labels does not remove the corresponding preference: neighbouring tokens can carry it. Finally, conditional behaviours can route through the network body. For sycophancy, with agreement and correction markers masked from the loss, transfer reaches about 0.63 of the teacher's effect, localizes to body computation, and evades four audits across two model families. We scope this as masked transfer of a condition-present policy. Channel location is necessary for deciding which audits can be sound. It is not a deployment-ready screen: an audit used outside its carrier regime can give false assurance.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 43 of its 57 references that are in the map. First posted on arXiv 20 Jun 2026.

Builds on

43 of 57 references.

References · 43

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  4. 2022Editing Models with Task Arithmeticcited by 1,076
  5. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  6. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  7. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  8. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  9. 2018On Lazy Training in Differentiable Programmingcited by 671
  10. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  11. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
  12. 2014New insights and perspectives on the natural gradient methodcited by 435
31 more
  1. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  2. 2024The Platonic Representation Hypothesiscited by 362
  3. 2020Kernel and Rich Regimes in Overparametrized Modelscited by 302
  4. 2019Representation Degeneration Problem in Training Natural Language Generation Modelscited by 250
  5. 2021Revisiting Model Stitching to Compare Neural Representationscited by 203
  6. 2021Zero-Cost Proxies for Lightweight NAScited by 194
  7. 2021Does Knowledge Distillation Really Work?cited by 187
  8. 2020Pruning Neural Networks at Initialization: Why are We Missing the Mark?cited by 181
  9. 2024Stealing Part of a Production Language Modelcited by 152
  10. 2024LoRA-GA: Low-Rank Adaptation with Gradient Approximationcited by 134
  11. 2019Zero-shot Knowledge Transfer via Adversarial Belief Matchingcited by 129
  12. 2020Infinite attention: NNGP and NTK for deep attention networkscited by 117
  13. 2021Neural Networks as Kernel Learners: The Silent Alignment Effectcited by 106
  14. 2025Subliminal Learning: Language models transmit behavioral traits via hidden signals in datacited by 89
  15. 2025Persona Features Control Emergent Misalignmentcited by 76
  16. 2025Auditing language models for hidden objectivescited by 68
  17. 2019Stiffness: A New Perspective on Generalization in Neural Networkscited by 46
  18. 2024Logits of API-Protected LLMs Leak Proprietary Informationcited by 38
  19. 2025LoRA-One: One-Step Full Gradient Could Suffice for Fine-Tuning Large Language Models, Provably and Efficientlycited by 35
  20. 2020Knowledge Distillation in Wide Neural Networks: Risk Bound, Data Efficiency and Imperfect Teachercited by 33
  21. 2025Towards Understanding Subliminal Learning: When and How Hidden Biases Transfercited by 30
  22. 2026Subliminal Effects in Your Data: A General Mechanism via Log-Linearitycited by 15
  23. 2024Transferring Backdoors between Large Language Models by Knowledge Distillationcited by 11
  24. 2025Dataset distillation for memorized data: Soft labels can leak held-out teacher knowledgecited by 9
  25. 2026Subliminal Learning Is Steering Vector Distillationcited by 9
  26. 2026You Didn't Have to Say It like That: Subliminal Learning from Faithful Paraphrasescited by 7
  27. 2026Lost in Backpropagation: The LM Head is a Gradient Bottleneckcited by 6
  28. 2026Learning Through Noise: Why Subliminal Learning Works and When It Failscited by 6
  29. 2025Seed-Induced Uniqueness in Transformer Models: Subspace Alignment Governs Subliminal Transfercited by 5
  30. 2026Phantom Transfer: Data Poisoning can Survive Data-Level Defencescited by 5
  31. 2026Sustained Gradient Alignment Mediates Subliminal Learning in a Multi-Step Setting: Evidence from MNIST Auxiliary Logit Distillation Experimentcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.