mlsanityData to 7 Oct 2026

Architecture-Dependent Causal Transfer of Activation States Across Large Language Models

arXiv 2608.16347 ↗First posted 17 Aug 2026cs.CL cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Direct communication between AI systems relies on natural language as an intermediate layer, incurring encoding/decoding overhead, token cost, and latency. We ask whether internal activation states can instead be transferred causally between different large language model (LLM) architectures via a learned projection, evaluated at three levels: representational similarity, cross-model retrieval from projected states, and end-to-end causal transfer via activation injection during generation. Using four architecturally diverse open-weight models (Qwen2-0.5B, Phi-3-mini, Mistral-7B, FLAN-T5-base), we find that representational alignment in trained models exceeds a random-initialization null baseline and is best captured by a rank-based metric (mutual k-nearest-neighbour alignment), more robust to activation-magnitude outliers than centered kernel alignment (CKA) or Procrustes analysis. A learned projection network retrieves the correct target-model representation from a held-out set well above chance for the three causal decoder-only model pairs (45-50% top-1 accuracy vs. 5% chance) but at chance level for the encoder-based FLAN-T5. Injecting projected activations into a target model during generation produces a statistically significant, pre-registered causal effect on retrieval-based output similarity for only one of the three decoder-only pairs (Qwen2-0.5B to Phi-3-mini: 23.3% vs. 0.0% under negative control, p=0.047, FDR-corrected); the two pairs targeting Mistral-7B show no such effect despite comparable representational alignment at the hidden-state level. We interpret these results as evidence for causal transfer of the representational vehicle, not of meaning, and conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 11 of its 16 references that are in the map. First posted on arXiv 17 Aug 2026.

Builds on

11 of 16 references.

References · 11

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  3. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  4. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  5. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  6. 2024Massive Activations in Large Language Modelscited by 277
  7. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 222
  8. 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganographycited by 107
  9. 2023Preventing Language Models From Hiding Their Reasoningcited by 62
  10. 2025Superposition disentanglement of neural representations reveals hidden alignmentcited by 4
  11. 2024Exploring Cross-model Neuronal Correlations in the Context of Predicting Model Performance and Generalizabilitycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.