Covert Trait Propagation Is Representation Alignment: Mechanistic Evidence from Hidden-Channel Distillation
Not on the map. Why?
Abstract (arXiv)
A student model trained on pure uniform noise can still inherit its teacher's digit-classification ability, provided the two share initialization. Previous work proves this transfer is guaranteed when the teacher's learning rate is small enough, but does not explain where in the network the channel lives or what sets its capacity. Working in an MLP distillation setting on MNIST, we show these channels are not purely informational: geometric alignment gates access to the information the channel carries. Shared initialization makes the output projection W_2 a common coordinate key, and KL gradients reshape the student's input projection W_0 until its hidden representations align with the teacher's. We call this covert trait propagation (CTP). Five experiments support this mechanism: channel closure tracks weight drift, not teacher accuracy; freezing W_0 destroys transfer while freezing W_2 leaves it intact; multi-teacher ensembles cancel out despite each teacher carrying comparable label information; and linear centered kernel alignment (CKA) tracks student accuracy at r=0.98 across a continuous initialization sweep. Applying the same geometric lens to cross-token behavioral entanglement (CTBE) in instruction-tuned LLMs, we find the effect appears to be activated by alignment training, acting on an inherited substrate, and that the standard log-ratio metric produces an apparent frequency bias that is largely a circularity artifact.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 32 of its 42 references that are in the map. First posted on arXiv 5 Jul 2026.
Builds on
32 of 42 references.
References · 32
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2014FitNets: Hints for Thin Deep Netscited by 1,619
- 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2022In-context Learning and Induction Headscited by 930
- 2022Toy Models of Superpositioncited by 928
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
20 more
- 2019Contrastive Representation Distillationcited by 734
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
- 2018Born Again Neural Networkscited by 539
- 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
- 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
- 2018Insights on representational similarity in neural networks with canonical correlationcited by 318
- 2022Large Language Models Are Reasoning Teacherscited by 318
- 2024Alignment faking in large language modelscited by 316
- 2017Certified Defenses for Data Poisoning Attackscited by 288
- 2023The False Promise of Imitating Proprietary LLMscited by 254
- 2026Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnetcited by 175
- 2020Blind Backdoors in Deep Learning Modelscited by 174
- 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganographycited by 107
- 2025Model Organisms for Emergent Misalignmentcited by 84
- 2025Towards Understanding Subliminal Learning: When and How Hidden Biases Transfercited by 30
- 2025Distillation Robustifies Unlearningcited by 11
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Why Some Models Resist Unlearning: A Linear Stability Perspective
- 2026What Visual Generators Need from Teachers: Rethinking Representation Alignment
- 2026Why Are DMD Students Lazy? Understanding the Copying Behavior in Few-Step Distillation
- 2026Cross-Space Distillation: Teaching One-Step Students with Modern Diffusion Teachers
- 2026Differentiable Faithfulness Alignment for Cross-Model Circuit Transfer
- 2025REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Training
- 2026Alignment via Training Against Probes Without Losing Monitorability
- 2026Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation