mlsanityData to 7 Oct 2026

A Negative Result on Cross-Model Activation Transfer in a Pythia Multi-Hop Setting

arXiv 2606.03280 ↗First posted 2 Jun 2026cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Recent work shows that language models can transmit behavioural traits through hidden signals in generated data during training. We ask whether a different activation-mediated channel is viable: can one language model communicate a useful intermediate reasoning state to another at inference time through a post-hoc linear activation bridge, rather than through a textual or structured-token relay? We test this question in a controlled Pythia-160M to Pythia-410M multi-hop reasoning setting. A linear translation layer learns a strong normalized-space map between sender and receiver hidden states, with normalized cosine similarity near 0.97 across seeds. However, when the translated activations are injected into the receiver at inference time, they do not improve downstream answering. Low-strength additive injection remains near the no-injection baseline, with confidence intervals that cross zero. Replacement-style injection is consistently destructive, and rescaling translated vectors to the receiver hidden-state norm does not rescue performance. The result is therefore a scoped negative result: in this setting, offline representational alignment is not sufficient for useful causal communication inside the receiver.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 18 of its 20 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

18 of 20 references.

References · 18

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  3. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  5. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  6. 2022Mass-Editing Memory in a Transformercited by 976
  7. 2022In-context Learning and Induction Headscited by 930
  8. 2023Steering Language Models With Activation Engineeringcited by 824
  9. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  10. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  11. 2014Understanding image representations by measuring their equivariance and equivalencecited by 240
  12. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 224
6 more
  1. 2021Revisiting Model Stitching to Compare Neural Representationscited by 203
  2. 2022Analyzing Transformers in Embedding Spacecited by 131
  3. 2024To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Modelscited by 25
  4. 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 17
  5. 2024OneEdit: A Neural-Symbolic Collaboratively Knowledge Editing Systemcited by 12
  6. 2023Model Stitching: Looking For Functional Similarity Between Representationscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Interpreting Language Models: A Case Study in Multi-Hop Reasoning
  2. 2025Communicating Activations Between Language Model Agents
  3. 2023Memory Injections: Correcting Multi-Hop Reasoning Failures during Inference in Transformer-Based Language Models
  4. 2025Language models can learn implicit multi-hop reasoning, but only if they have lots of training data
  5. 2024Uncovering Latent Chain of Thought Vectors in Language Models
  6. 2025ActivationReasoning: Logical Reasoning in Latent Activation Spaces
  7. 2025Activation Control for Efficiently Eliciting Long Chain-of-thought Ability of Language Models
  8. 2025How do Transformers Learn Implicit Reasoning?