# IHEval: Evaluating Language Models on Following the Instruction Hierarchy

arXiv: https://arxiv.org/abs/2502.08745 · HTML: https://mlsanity.com/paper/2502.08745/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 35 references that are in the map.
- First posted on arXiv 12 Feb 2025.

## Builds on (most cited first, 21 of 21)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [DeepSeek LLM: Scaling Open-Source Language Models with Longtermism](https://mlsanity.com/paper/2401.02954/) (2024)
- [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024)
- [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://mlsanity.com/paper/2404.13208/) (2024)
- [XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages](https://mlsanity.com/paper/2106.13822/) (2021)
- [Jailbreak Attacks and Defenses Against Large Language Models: A Survey](https://mlsanity.com/paper/2407.04295/) (2024)
- [Benchmarking Complex Instruction-Following with Multiple Constraints Composition](https://mlsanity.com/paper/2407.03978/) (2024)
- [Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game](https://mlsanity.com/paper/2311.01011/) (2023)
- [AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents](https://mlsanity.com/paper/2406.13352/) (2024)
- [Aligning to Thousands of Preferences via System Message Generalization](https://mlsanity.com/paper/2405.17977/) (2024)
- [Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?](https://mlsanity.com/paper/2403.06833/) (2024)
- [Instruction-following Evaluation through Verbalizer Manipulation](https://mlsanity.com/paper/2307.10558/) (2023)
- [SysBench: Can Large Language Models Follow System Messages?](https://mlsanity.com/paper/2408.10943/) (2024)
- [SoFA: Shielded On-the-fly Alignment via Priority Rule Following](https://mlsanity.com/paper/2402.17358/) (2024)

## Built on by (most cited first, 30 of 53)

- [Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models](https://mlsanity.com/paper/2505.14810/) (2025)
- [Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs](https://mlsanity.com/paper/2509.14260/) (2025)
- [Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models](https://mlsanity.com/paper/2506.01413/) (2025)
- [Prompt Injection as Role Confusion](https://mlsanity.com/paper/2603.12277/) (2026)
- [OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding](https://mlsanity.com/paper/2601.10343/) (2026)
- [ConInstruct: Evaluating Large Language Models on Conflict Detection and Resolution in Instructions](https://mlsanity.com/paper/2511.14342/) (2025)
- [IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation](https://mlsanity.com/paper/2603.04738/) (2026)
- [Security Considerations for Artificial Intelligence Agents](https://mlsanity.com/paper/2603.12230/) (2026)
- [Structured Security Auditing and Robustness Enhancement for Untrusted Agent Skills](https://mlsanity.com/paper/2604.25109/) (2026)
- [Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden States](https://mlsanity.com/paper/2608.02657/) (2026)
- [Did You Forget What I Asked? Prospective Memory Failures in Large Language Models](https://mlsanity.com/paper/2603.23530/) (2026)
- [Many-Tier Instruction Hierarchy in LLM Agents](https://mlsanity.com/paper/2604.09443/) (2026)
- [Reasoning Up the Instruction Ladder for Controllable Language Models](https://mlsanity.com/paper/2511.04694/) (2025)
- [AlignSentinel: Alignment-Aware Detection of Prompt Injection Attacks](https://mlsanity.com/paper/2602.13597/) (2026)
- [When Safety Becomes a Vulnerability: Exploiting LLM Alignment Homogeneity for Transferable Blocking in RAG](https://mlsanity.com/paper/2603.03919/) (2026)
- [Models That Know How Evaluations Are Designed Score Safer](https://mlsanity.com/paper/2605.28591/) (2026)
- [Harness-IF: Evaluating Instruction Following Across Instruction Surfaces in Coding Agents](https://mlsanity.com/paper/2608.11727/) (2026)
- [User-Assistant Bias in LLMs](https://mlsanity.com/paper/2508.15815/) (2025)
- [Evaluating Language Model Reasoning about Confidential Information](https://mlsanity.com/paper/2508.19980/) (2025)
- [Deconstructing Instruction-Following: A New Benchmark for Granular Evaluation of Large Language Model Instruction Compliance Abilities](https://mlsanity.com/paper/2601.18554/) (2026)
- [Who Tests the Testers? Systematic Enumeration and Coverage Audit of LLM Agent Tool Call Safety](https://mlsanity.com/paper/2603.18245/) (2026)
- [Steering Instruction Hierarchies at Inference Time](https://mlsanity.com/paper/2607.26228/) (2026)
- [CCR-Bench: A Comprehensive Benchmark for Evaluating LLMs on Complex Constraints, Control Flows, and Real-World Cases](https://mlsanity.com/paper/2603.07886/) (2026)
- [Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graph](https://mlsanity.com/paper/2603.15527/) (2026)
- [Hierarchical Alignment: Enforcing Hierarchical Instruction-Following in LLMs through Logical Consistency](https://mlsanity.com/paper/2604.09075/) (2026)
- [Bias in the Tails: How Name-conditioned Evaluative Framing in Resume Summaries Destabilizes LLM-based Hiring](https://mlsanity.com/paper/2604.19984/) (2026)
- [Faithful Mobile GUI Agents with Guided Advantage Estimator](https://mlsanity.com/paper/2605.01208/) (2026)
- [Position: AI Safety Requires Effective Controllability](https://mlsanity.com/paper/2605.27117/) (2026)
- [Adversarial Pragmatics for AI Safety Evaluation: A Diagnostic Framework and Seed Benchmark for Language-Mediated Control](https://mlsanity.com/paper/2607.01153/) (2026)
- [PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?](https://mlsanity.com/paper/2609.18605/) (2026)

## Abstract (arXiv)

The instruction hierarchy, which establishes a priority order from system messages to user messages, conversation history, and tool outputs, is essential for ensuring consistent and safe behavior in language models (LMs). Despite its importance, this topic receives limited attention, and there is a lack of comprehensive benchmarks for evaluating models' ability to follow the instruction hierarchy. We bridge this gap by introducing IHEval, a novel benchmark comprising 3,538 examples across nine tasks, covering cases where instructions in different priorities either align or conflict. Our evaluation of popular LMs highlights their struggle to recognize instruction priorities. All evaluated models experience a sharp performance decline when facing conflicting instructions, compared to their original instruction-following performance. Moreover, the most competitive open-source model only achieves 48% accuracy in resolving such conflicts. Our results underscore the need for targeted optimization in the future development of LMs.