Architecture-Dependent Causal Transfer of Activation States Across Large Language Models
Not on the map. Why?
Abstract (arXiv)
Direct communication between AI systems relies on natural language as an intermediate layer, incurring encoding/decoding overhead, token cost, and latency. We ask whether internal activation states can instead be transferred causally between different large language model (LLM) architectures via a learned projection, evaluated at three levels: representational similarity, cross-model retrieval from projected states, and end-to-end causal transfer via activation injection during generation. Using four architecturally diverse open-weight models (Qwen2-0.5B, Phi-3-mini, Mistral-7B, FLAN-T5-base), we find that representational alignment in trained models exceeds a random-initialization null baseline and is best captured by a rank-based metric (mutual k-nearest-neighbour alignment), more robust to activation-magnitude outliers than centered kernel alignment (CKA) or Procrustes analysis. A learned projection network retrieves the correct target-model representation from a held-out set well above chance for the three causal decoder-only model pairs (45-50% top-1 accuracy vs. 5% chance) but at chance level for the encoder-based FLAN-T5. Injecting projected activations into a target model during generation produces a statistically significant, pre-registered causal effect on retrieval-based output similarity for only one of the three decoder-only pairs (Qwen2-0.5B to Phi-3-mini: 23.3% vs. 0.0% under negative control, p=0.047, FDR-corrected); the two pairs targeting Mistral-7B show no such effect despite comparable representational alignment at the hidden-state level. We interpret these results as evidence for causal transfer of the representational vehicle, not of meaning, and conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 11 of its 16 references that are in the map. First posted on arXiv 17 Aug 2026.
Builds on
11 of 16 references.
References · 11
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
- 2024Massive Activations in Large Language Modelscited by 277
- 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 222
- 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganographycited by 107
- 2023Preventing Language Models From Hiding Their Reasoningcited by 62
- 2025Superposition disentanglement of neural representations reveals hidden alignmentcited by 4
- 2024Exploring Cross-model Neuronal Correlations in the Context of Predicting Model Performance and Generalizabilitycited by 3
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Portable Semantics, Private Dialects: Reuse and Negative Transfer in Latent Communication Between Language-Model Cellscited by 1
- 2026LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replaycited by 0
- 2026Beyond Linear Concepts: Discovering and Aligning Non-Linear Concept Manifolds in Large Language Modelscited by 0
- 2026Beyond Linear Concepts: Discovering and Aligning Non-Linear Concept Manifolds in Large Language Models1 Oct 2026
- 2026Portable Semantics, Private Dialects: Reuse and Negative Transfer in Latent Communication Between Language-Model Cells10 Sep 2026
- 2026LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replay7 Sep 2026