mlsanityData to 14 Oct 2026

Subliminal Learning: Language models transmit behavioral traits via hidden signals in data

arXiv 2507.14805 ↗First posted 20 Jul 2025cs.LG cs.AIcited by 95

Not on the map. Why?

Abstract (arXiv)

We study subliminal learning, a surprising phenomenon where language models transmit behavioral traits via semantically unrelated data. In our main experiments, a "teacher" model with some trait T (such as liking owls or being misaligned) generates a dataset consisting solely of number sequences. Remarkably, a "student" model trained on this dataset learns T. This occurs even when the data is filtered to remove references to T. We observe the same effect when training on code or reasoning traces generated by the same teacher model. However, we do not observe the effect when the teacher and student have different base models. To help explain our findings, we prove a theoretical result showing that subliminal learning occurs in all neural networks under certain conditions, and demonstrate subliminal learning in a simple MLP classifier. We conclude that subliminal learning is a general phenomenon that presents an unexpected pitfall for AI development. Distillation could propagate unintended traits, even when developers try to prevent this via data filtering.

Key facts. Cited by 95 papers in the map (data through 14 Oct 2026). Builds on 32 of its 47 references that are in the map. First posted on arXiv 20 Jul 2025.

Builds on

32 of 47 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  7. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  8. 2023A Watermark for Large Language Modelscited by 725
  9. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  10. 2018Born Again Neural Networkscited by 546
  11. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
  12. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 519
20 more
  1. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  2. 2019Hidden Trigger Backdoor Attackscited by 371
  3. 2024Alignment faking in large language modelscited by 360
  4. 2022Large Language Models Are Reasoning Teacherscited by 329
  5. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  6. 2017Certified Defenses for Data Poisoning Attackscited by 303
  7. 2018Model compression via distillation and quantizationcited by 289
  8. 2023Undetectable Watermarks for Language Modelscited by 226
  9. 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 178
  10. 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 172
  11. 2018Self-Imitation Learningcited by 138
  12. 2023DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Textcited by 129
  13. 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganographycited by 113
  14. 2025Model Organisms for Emergent Misalignmentcited by 91
  15. 2023Protecting Language Generation Models via Invisible Watermarkingcited by 90
  16. 2025Persona Features Control Emergent Misalignmentcited by 86
  17. 2019Neural Linguistic Steganographycited by 78
  18. 2025Thought Crime: Backdoors and Emergent Misalignment in Reasoning Modelscited by 65
  19. 2025Distillation Robustifies Unlearningcited by 11
  20. 2025Unnatural Languages Are Not Bugs but Features for LLMscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 22; 2026: 73.

95 papers.

Sort:

By area:

  1. 2025Natural Emergent Misalignment from Reward Hacking in Production RLcited by 136
  2. 2025School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMscited by 66
  3. 2025Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-timecited by 43
  4. 2025Towards Understanding Subliminal Learning: When and How Hidden Biases Transfercited by 34
  5. 2025Narrow Finetuning Leaves Clearly Readable Traces in Activation Differencescited by 28
  6. 2025Open Character Training: Shaping the Persona of AI Assistants through Constitutional AIcited by 24
  7. 2026Subliminal Effects in Your Data: A General Mechanism via Log-Linearitycited by 19
  8. 2025Position: Privacy Is Not Just Memorization!cited by 17
  9. 2026Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggerscited by 17
  10. 2025Technological folie `a deux: Feedback Loops Between AI Chatbots and Mental Illnesscited by 15
  11. 2026Subliminal Steering: Stronger Encoding of Hidden Signalscited by 14
  12. 2025GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMscited by 11
38 more
  1. 2026Thought Virus: Viral Misalignment via Subliminal Prompting in Multi-Agent Systemscited by 9
  2. 2025Moloch's Bargain: Emergent Misalignment When LLMs Compete for Audiencescited by 8
  3. 2026From Data to Behavior: Predicting Unintended Model Behaviors Before Trainingcited by 8
  4. 2026You Didn't Have to Say It like That: Subliminal Learning from Faithful Paraphrasescited by 8
  5. 2026Emergent and Subliminal Misalignment Through the Lens of Data-Mediated Transfercited by 8
  6. 2025Probing the Preferences of a Language Model: Integrating Verbal and Behavioral Tests of AI Welfarecited by 7
  7. 2026Sustained Gradient Alignment Mediates Subliminal Learning in a Multi-Step Setting: Evidence from MNIST Auxiliary Logit Distillation Experimentcited by 7
  8. 2026Understanding Emergent Misalignment via Feature Superposition Geometrycited by 7
  9. 2026Channel Location Constrains the Auditability of Subliminal Learningcited by 7
  10. 2025Exploring System 1 and 2 communication for latent reasoning in LLMscited by 6
  11. 2025Neologism Learning for Controllability and Self-Verbalizationcited by 6
  12. 2025Pay Attention to the Triggers: Constructing Backdoors That Survive Distillationcited by 6
  13. 2026Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMscited by 6
  14. 2026Subliminal Learning is a LoRA Artifactcited by 6
  15. 2026Covert Influence Between Language Modelscited by 6
  16. 2025State over Tokens: Characterizing the Role of Reasoning Tokenscited by 5
  17. 2026GAVEL: Towards Rule-Based Safety Through Activation Monitoringcited by 5
  18. 2026Infusion: Shaping Model Behavior by Editing Training Data via Influence Functionscited by 5
  19. 2026Subliminal Transfer of Unsafe Behaviors in AI Agent Distillationcited by 5
  20. 2025Subliminal Corruption: Mechanisms, Thresholds, and Interpretabilitycited by 4
  21. 2026Delta-Crosscoder: Robust Crosscoder Model Diffing in Narrow Fine-Tuning Regimescited by 4
  22. 2026Alignment midtraining for animalscited by 4
  23. 2026Less is More: Early Stopping Rollout for On-Policy Distillationcited by 4
  24. 2026Mind Viruses: Self-Propagating Ideas in Multi-Agent LLM Systemscited by 4
  25. 2025LLM Chemistry Estimation for Multi-LLM Recommendationcited by 3
  26. 2025Detecting Adversarial Fine-tuning with Auditing Agentscited by 3
  27. 2025Toward Honest Language Models for Deductive Reasoningcited by 3
  28. 2026Subliminal Signals in Preference Labelscited by 3
  29. 2026Visual Self-Fulfilling Alignment: Shaping Safety-Oriented Personas via Threat-Related Imagescited by 3
  30. 2026Reinforcement Learning Can Amplify Emergent Misalignment from Harmless Rewardscited by 3
  31. 2026Quantifying Subliminal Behavioral Transfer Ratios in Language Model Distillationcited by 3
  32. 2026Subliminal Learning is Non-Semantic Distillationcited by 3
  33. 2025Multi-LLM Collaboration for Medication Recommendationcited by 2
  34. 2026When Numbers Start Talking: Implicit Numerical Coordination Among LLM-Based Agentscited by 2
  35. 2026Undetectable Conversations Between AI Agents via Pseudorandom Noise-Resilient Key Exchangecited by 2
  36. 2026Synthetic Data for any Differentiable Targetcited by 2
  37. 2026Overtrained, Not Misalignedcited by 2
  38. 2026Consistency Training Can Entrench Misalignmentcited by 2

50 most cited of 95 All 95 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026On Mitigation of Subliminal Learning in Large Language Models
  2. 2026Subliminal Learning Is Steering Vector Distillation
  3. 2026Defence against the dark arts of distillation: Unsupervised discovery of subliminal learning
  4. 2026Beyond Owls: Subliminal Learning Can Transfer Learned Capabilities and Backdoors
  5. 2026Reproducing and Evaluating the Generalizability of Subliminal Learning in Open-Weight Models
  6. 2026Cross-Trait Transfer in Subliminal Learning
  7. 2026Verbalizing Subliminal Learning Effects Using Text Optimization
  8. 2026Subliminal Learning as Trait-Direction Drift: A Mechanism and Targeted Control under SFT Distillation