mlsanityData to 14 Oct 2026

How Language Models Choose Sides: Internal Representations of Instruction Hierarchy

arXiv 2608.28648 ↗First posted 16 Aug 2026cs.AI cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study how instruction-tuned LLMs arbitrate direct conflicts between system and user instructions. We introduce a benchmark of 41 paired constraints with deterministic verifiers and evaluate eight models under matched baseline, conflict, and same-channel control conditions. Behaviourally, the models split into three regimes by System Authority Delta: hierarchy-respecting models use the system channel as an authority signal, anti-hierarchy models follow the system less often than their same-channel baseline predicts, and no-effect models show little channel sensitivity. Llama-3.1-8B is the strongest anti-hierarchy case in our suite, following the system in only 0.10 of conflict trials. We use this behavioural failure case to ask whether user-preferring arbitration reflects the absence of an internal conflictresolution signal. It does not: on Llama-3.1-8B, the conflict outcome is linearly decodable from residual-stream activations at 0.97 balanced accuracy, 17 percentage points above a metadata-only baseline, with analogous signals on Qwen2.5-7B and gpt-oss-20b. Steering with a layer-12 mean of four per-conflict logistic-regression directions raises genuine system compliance from 0.132 to 0.530, while directions selected mainly for pooled separability steer poorly. User-preferring conflict resolution can therefore coexist with a readable internal arbitration signal, and successful intervention depends on the geometry of the readout rather than probe accuracy alone

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 15 references that are in the map. First posted on arXiv 16 Aug 2026.

Builds on

15 of 15 references.

References · 15

  1. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  3. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  4. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  5. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  6. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  7. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  8. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  9. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
  10. 2024Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchycited by 67
  11. 2025IHEval: Evaluating Language Models on Following the Instruction Hierarchycited by 53
  12. 2025Control Illusion: The Failure of Instruction Hierarchies in Large Language Modelscited by 33
3 more
  1. 2025EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM Systemcited by 29
  2. 2025The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them)cited by 6
  3. 2025Who is In Charge? Dissecting Role Conflicts in Instruction Followingcited by 3

Built on by

Not cited yet.