mlsanityData to 7 Oct 2026

The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment

arXiv 2604.06377 ↗First posted 7 Apr 2026cs.LG cs.AIcited by 0

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

We investigate whether post-trained capabilities can be transferred across models without retraining, with a focus on transfer across different model scales. We propose the Master Key Hypothesis, which states that model capabilities correspond to directions in a low-dimensional latent subspace that induce specific behaviors and are transferable across models through linear alignment. Based on this hypothesis, we introduce UNLOCK, a training-free and label-free framework that extracts a capability direction by contrasting activations between capability-present and capability-absent Source variants, aligns it with a Target model through a low-rank linear transformation, and applies it at inference time to elicit the behavior. Experiments on reasoning behaviors, including Chain-of-Thought (CoT) and mathematical reasoning, demonstrate substantial improvements across model scales without training. For example, transferring CoT reasoning from Qwen1.5-14B to Qwen1.5-7B yields an accuracy gain of 12.1% on MATH, and transferring a mathematical reasoning direction from Qwen3-4B-Base to Qwen3-14B-Base improves AGIEval Math accuracy from 61.1% to 71.3%, surpassing the 67.8% achieved by the 14B post-trained model. Our analysis shows that the success of transfer depends on the capabilities learned during pre-training, and that our intervention amplifies latent capabilities by sharpening the output distribution toward successful reasoning trajectories.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 59 of its 69 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

59 of 69 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024MARIO: MAth Reasoning with code Interpreter Output -- A Reproducible Pipeline16 Jan

Other references · 58

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  7. 2023Qwen Technical Reportcited by 3,273
  8. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  9. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  10. 2022Editing Models with Task Arithmeticcited by 1,076
  11. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  12. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
38 more
  1. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  2. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  3. 2023Steering Language Models With Activation Engineeringcited by 824
  4. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  5. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  6. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  7. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 463
  8. 2024The Platonic Representation Hypothesiscited by 362
  9. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 331
  10. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 293
  11. 2023Function Vectors in Large Language Modelscited by 292
  12. 2026Reinforcement Learning via Self-Distillationcited by 285
  13. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  14. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 216
  15. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  16. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
  17. 2025Understanding Reasoning in Thinking Language Models via Steering Vectorscited by 69
  18. 2024SelfIE: Self-Interpretation of Large Language Model Embeddingscited by 63
  19. 2024Style Vectors for Steering Generative Large Language Modelcited by 56
  20. 2023Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languagescited by 52
  21. 2025Do Language Models Use Their Depth Efficiently?cited by 48
  22. 2025Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Modelscited by 45
  23. 2021Knowledge Inheritance for Pre-trained Language Modelscited by 38
  24. 2024Extending Activation Steering to Broad Skills and Multiple Behaviourscited by 36
  25. 2025DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learningcited by 34
  26. 2024Activation Scaling for Steering and Interpreting Language Modelscited by 33
  27. 2025Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Datacited by 26
  28. 2025Base Models Know How to Reason, Thinking Models Learn Whencited by 25
  29. 2025OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critiquecited by 24
  30. 2024Nudging: Inference-time Alignment of LLMs via Guided Decodingcited by 22
  31. 2025Training Plug-n-Play Knowledge Modules with Deep Context Distillationcited by 22
  32. 2025Activation Steering for Chain-of-Thought Compressioncited by 22
  33. 2025The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Directioncited by 21
  34. 2025Activation Space Interventions Can Be Transferred Between Large Language Modelscited by 17
  35. 2025Cross-model Transferability among Large Language Models on the Platonic Representations of Conceptscited by 11
  36. 2025Shadow-FT: Tuning Instruct Model via Training on Paired Base Modelcited by 7
  37. 2025Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMscited by 7
  38. 2025Linear Representation Transferability Hypothesis: Leveraging Small Models to Steer Large Modelscited by 6

50 most cited of 58 All 58 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Thinking in Different Spaces: Domain-Specific Latent Geometry Survives Cross-Architecture Translation
  2. 2025Leveraging Parameter Space Symmetries for Reasoning Skill Transfer in LLMs
  3. 2026Selective Transfer of RL Updates for Visual Reasoning
  4. 2025Model-to-Model Knowledge Transmission (M2KT): A Data-Free Framework for Cross-Model Understanding Transfer
  5. 2025Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning
  6. 2026Understanding Knowledge Transfer Mechanism in Heterogeneous MLLM Fusion: A Simple Linear Approach
  7. 2026Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention
  8. 2025Reasoning-Finetuning Repurposes Latent Representations in Base Models