mlsanityData to 14 Oct 2026

K/V-Cache Interventions Dissociate Representation Alignment from Persona Expression in Decoder-Only Language Models

arXiv 2609.11020 ↗First posted 10 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We study K/V-cache interventions -- transplanting a target-conditioned K/V trajectory into a source-persona generation -- as a structured surface for persona control in decoder-only language models. Across 13 intervention configurations applied to Llama-3.1-8B for a fixed source-to-target persona pair, we report two consistent dissociations between representation-level alignment and behavioral expression, plus a common failure under position perturbations. First, all layer-band K/V replacements (early, mid, late) achieve strong local V-space alignment (V-gap 0.91, 0.89, 0.84), but only mid-layer replacement (layers 9-20) combines substantial target-marker expression with preserved lexical diversity. Second, full and mid-layer replacement induce comparable alignment (V-gap 0.94 vs. 0.89) yet produce different lexical-diversity profiles (TTR 0.65 vs. 0.77). Third, position perturbations (lag and shuffle) apply distinct operations yet uniformly suppress target-persona expression -- a common behavioral failure rather than a strict dissociation. Representation-level similarity metrics alone are thus not sufficient predictors of downstream persona expression in the regimes we study; the K/V cache emerges as a controllable but structurally constrained intervention surface. Because the transplanted trajectory carries the target's own generated token history, we characterize the intervention as trajectory-level transplantation rather than isolated persona-representation injection; a same-token-sequence control, decoding an identical token sequence under source vs. target conditioning, reproduces the sign and layer localization of the L28 representational shift, indicating the shift is not explained solely by imported token history. These findings characterize representation-behavior dissociation in a high-signal setting rather than establishing universality across models or persona pairs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 17 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

13 of 17 references.

References · 13

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  5. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  7. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  8. 2022Mass-Editing Memory in a Transformercited by 1,001
  9. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  10. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
  11. 2026Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventionscited by 6
  12. 2026Dissociating the Internal Representations of Sycophancy in LLMscited by 6
1 more
  1. 2026When Behavioral Safety Evaluation Fails: A Representation-Level Perspectivecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Extracting Persona Subspaces Through Iterative Nullspace Projection For Modulation
  2. 2026PerMix-RLVR: Preserving Persona Expressivity under Verifiable-Reward Alignment
  3. 2025Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment
  4. 2024PERSONA: A Reproducible Testbed for Pluralistic Alignment
  5. 2025Post Persona Alignment for Multi-Session Dialogue Generation
  6. 2025Drift: Decoding-time Personalized Alignments with Implicit User Preferences
  7. 2026Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue
  8. 2026Persona Switch: Mixing Distinct Perspectives in Decoding Time