mlsanityData to 14 Oct 2026

Subliminal Learning as Trait-Direction Drift: A Mechanism and Targeted Control under SFT Distillation

arXiv 2609.01091 ↗First posted 1 Sep 2026cs.LGcited by 1

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

Beyond intended capabilities, model distillation can transfer hidden traits from a teacher. A teacher biased by a system prompt can generate semantically clean training data, such as numeric sequences, that still causes a downstream student to inherit the hidden preference, a phenomenon known as subliminal learning. Prior work has identified several parts of this process. How the signal builds up during training and produces behavioral transfer remains unclear, making targeted mitigation difficult. We propose and validate trait-direction drift as a mechanism for subliminal learning: biased generation creates measurable preference gaps in teacher data, and student-recognizable gaps induce trait-aligned updates during supervised fine-tuning that accumulate into behavioral transfer. Guided by this mechanism, we propose probe-space corridor regularization, a targeted defense that constrains drift along a calibrated trait direction during distillation. The method substantially reduces hidden-trait transfer, preserving task performance: for example, it lowers malicious-response transfer from 29.55% to 6.45% with low main-task accuracy cost, and consistently suppresses animal-preference transfer across the main Qwen setting. The preference-gap, training-trajectory, and intervention evidence links subliminal learning to trait-direction drift and motivates corridor regularization as a targeted control during distillation.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 27 of its 35 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

27 of 35 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation16 Apr
  2. 2025Towards Understanding Subliminal Learning: When and How Hidden Biases Transfer28 Sep

Other references · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  6. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  7. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  8. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
  9. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 297
  10. 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 118
  11. 2025Model Organisms for Emergent Misalignmentcited by 91
  12. 2025Persona Features Control Emergent Misalignmentcited by 86
13 more
  1. 2024Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptationcited by 80
  2. 2025Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMscited by 55
  3. 2025Layer-Aware Representation Filtering: Purifying Finetuning Data to Preserve LLM Safety Alignmentcited by 24
  4. 2026Subliminal Effects in Your Data: A General Mechanism via Log-Linearitycited by 19
  5. 2026Subliminal Learning Is Steering Vector Distillationcited by 13
  6. 2025Dataset distillation for memorized data: Soft labels can leak held-out teacher knowledgecited by 10
  7. 2025Sequences of Logits Reveal the Low Rank Structure of Language Modelscited by 9
  8. 2026Concept Influence: Leveraging Interpretability to Improve Performance and Efficiency in Training Data Attributioncited by 8
  9. 2026Emergent and Subliminal Misalignment Through the Lens of Data-Mediated Transfercited by 8
  10. 2026Phantom Transfer: Data Poisoning can Survive Data-Level Defencescited by 6
  11. 2026Covert Influence Between Language Modelscited by 6
  12. 2026BLOCK-EM: Preventing Emergent Misalignment via Latent Blockingcited by 3
  13. 2026Emergent Misalignment Recruits a Pre-existing Persona Subspacecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Defence against the dark arts of distillation: Unsupervised discovery of subliminal learning
  2. 2026On Mitigation of Subliminal Learning in Large Language Models
  3. 2026Beyond Owls: Subliminal Learning Can Transfer Learned Capabilities and Backdoors
  4. 2026Sustained Gradient Alignment Mediates Subliminal Learning in a Multi-Step Setting: Evidence from MNIST Auxiliary Logit Distillation Experiment
  5. 2026Subliminal Learning is Non-Semantic Distillation
  6. 2026Subliminal Steering: Stronger Encoding of Hidden Signals
  7. 2026Channel Location Constrains the Auditability of Subliminal Learning
  8. 2025Subliminal Learning: Language models transmit behavioral traits via hidden signals in data