mlsanityData to 13 Oct 2026

As X, Do Y: How Persona and Task Combine in Instruction-Tuned LLMs

arXiv 2605.23147 ↗First posted 22 May 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Role prompts of the form As X, do Y admit a clean linear decomposition at one specific site in the residual stream: the prompt-to-answer transition -- the last prompt token together with the first two generated tokens -- in an early/mid layer band. There, persona and task contribute through partially orthogonal additive directions. Forming a pure persona effect DeltaX, a pure task effect DeltaY, and substituting hBB + DeltaX + DeltaY for the clean residual yields downstream output within a small KL of clean on Gemma-2-2B-IT and Qwen-2.5-1.5B, 3B-Instruct, across a 12-cell short grid and a 48-cell long-persona grid, with persona-specific behavioral markers preserved. The natural inference from this additive structure is that the role prompt can be compressed into a single cached residual vector. We show it cannot. Injecting the cached additive prediction -- or even the oracle clean residual hXY -- into a baseline host prompt with the persona text removed does not approach the clean long-persona target, at one site or at many layers. Persona-conditioned multi-token generation flows through attention back to the persona-text positions throughout the prompt, which no residual at one site reproduces. Local additivity in the residual stream does not imply prompt compressibility. The additive structure at the prompt-to-answer transition supports interpretability and fine-grained steering of persona or task contributions; persona-conditioned behavior across the full continuation depends on a distributed prompt/KV mechanism that local activation arithmetic does not displace.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 8 of its 14 references that are in the map. First posted on arXiv 22 May 2026.

Builds on

8 of 14 references.

References · 8

  1. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,898
  2. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,785
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,650
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,286
  5. 2022Editing Models with Task Arithmeticcited by 1,101
  6. 2023In-Context Learning Creates Task Vectorscited by 305
  7. 2023Function Vectors in Large Language Modelscited by 299
  8. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 195

Built on by

Not cited yet.