mlsanityData to 14 Oct 2026

Cross-Architecture Steering Transfer in Language Models: A Systematic Empirical Study

arXiv 2608.05164 ↗First posted 26 May 2026cs.CL cs.LGcited by 2

On the map: Mechanistic interpretability → Activation steering methods

Abstract (arXiv)

Independently trained large language models may develop shared internal representations of semantic concepts despite architectural differences -- but whether this geometric similarity has functional consequences for cross-model behavioural control remains untested. We present the first systematic evaluation of cross-model steering transfer and show that shared LLM geometry is functionally exploitable, conditionally: concept directions from one model can steer a different independently trained model when sufficient representational capacity exists. We study five open-weight models spanning three parameter scales (0.8B--8B) and two architectural lineages, training one Sparse Autoencoder per model across 15 semantic domains and testing alignment across all 20 directed model pairs. We observe a suggestive discontinuity near 1.7B parameters: at >= 1.7B scale, 47--49% of cross-model feature pairs validate (Pearson r >= 0.60, Procrustes cosines 0.895--0.956), while alignment degrades sharply below 0.8B. Cross-model steering vectors (B3-TI) achieve a 71.0% win rate across 15 supervised concepts versus 68.0% for same-model native vectors; a single universal vector achieves 67.3% in 4 of 5 models without any per-model supervision. Transfer degrades for models below 1.7B and for one model with generation instability, confirming that functional exploitability requires sufficient representational capacity. Our findings underscore the importance of scale thresholds in mechanistic interpretability: tools validated at 7B scale may not transfer to smaller models without revalidation. We provide the first functional complement to the Platonic Representation Hypothesis -- geometric convergence across independently trained LLMs supports cross-model behavioural control without fine-tuning, under the identified scale conditions.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 18 of its 26 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

18 of 26 references.

References · 18

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  4. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  6. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  7. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  8. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  9. 2018Hierarchical Neural Story Generationcited by 1,193
  10. 2024Scaling and evaluating sparse autoencoderscited by 553
  11. 2017Word Translation Without Parallel Datacited by 487
  12. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 478
6 more
  1. 2024The Platonic Representation Hypothesiscited by 382
  2. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 245
  3. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  4. 2024Universal Neurons in GPT2 Language Modelscited by 102
  5. 2024Towards Universality: Studying Mechanistic Similarity Across Language Model Architecturescited by 31
  6. 2024Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoderscited by 27

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Cross-model Transferability among Large Language Models on the Platonic Representations of Concepts
  2. 2025Enhancing Cross-task Transfer of Large Language Models via Activation Steering
  3. 2026Where to Steer: Input-Dependent Layer Selection for Steering Improves LLM Alignment
  4. 2026On the Non-Identifiability of Steering Vectors in Large Language Models
  5. 2026Disentangling Steering Vectors
  6. 2026Global Evolutionary Steering: Refining Activation Steering Control via Cross-Layer Consistency
  7. 2026Signatures of Steerability in Activation Space of Language Models
  8. 2025Beyond Neural Incompatibility: Cross-Scale Knowledge Transfer in Language Models through Latent Semantic Alignment