mlsanityData to 7 Oct 2026

IHEval: Evaluating Language Models on Following the Instruction Hierarchy

arXiv 2502.08745 ↗First posted 12 Feb 2025cs.CLcited by 48

Not on the map. Why?

Abstract (arXiv)

The instruction hierarchy, which establishes a priority order from system messages to user messages, conversation history, and tool outputs, is essential for ensuring consistent and safe behavior in language models (LMs). Despite its importance, this topic receives limited attention, and there is a lack of comprehensive benchmarks for evaluating models' ability to follow the instruction hierarchy. We bridge this gap by introducing IHEval, a novel benchmark comprising 3,538 examples across nine tasks, covering cases where instructions in different priorities either align or conflict. Our evaluation of popular LMs highlights their struggle to recognize instruction priorities. All evaluated models experience a sharp performance decline when facing conflicting instructions, compared to their original instruction-following performance. Moreover, the most competitive open-source model only achieves 48% accuracy in resolving such conflicts. Our results underscore the need for targeted optimization in the future development of LMs.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 21 of its 35 references that are in the map. First posted on arXiv 12 Feb 2025.

Builds on

21 of 35 references.

References · 21

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2024Qwen2 Technical Reportcited by 2,076
  7. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  8. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  9. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  10. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 476
  11. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  12. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 346
9 more
  1. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  2. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 147
  3. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 135
  4. 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agentscited by 129
  5. 2024Aligning to Thousands of Preferences via System Message Generalizationcited by 84
  6. 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 70
  7. 2023Instruction-following Evaluation through Verbalizer Manipulationcited by 30
  8. 2024SysBench: Can Large Language Models Follow System Messages?cited by 26
  9. 2024SoFA: Shielded On-the-fly Alignment via Priority Rule Followingcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 38.

48 papers.

Sort:

By area:

  1. 2025Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Modelscited by 28
  2. 2025Incentivizing Reasoning for Advanced Instruction-Following of Large Language Modelscited by 17
  3. 2025Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMscited by 16
  4. 2026OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Codingcited by 15
  5. 2026Prompt Injection as Role Confusioncited by 11
  6. 2025ConInstruct: Evaluating Large Language Models on Conflict Detection and Resolution in Instructionscited by 9
  7. 2026IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluationcited by 8
  8. 2026Security Considerations for Artificial Intelligence Agentscited by 7
  9. 2026Did You Forget What I Asked? Prospective Memory Failures in Large Language Modelscited by 6
  10. 2026Many-Tier Instruction Hierarchy in LLM Agentscited by 6
  11. 2026Structured Security Auditing and Robustness Enhancement for Untrusted Agent Skillscited by 6
  12. 2025Reasoning Up the Instruction Ladder for Controllable Language Modelscited by 5
36 more
  1. 2026When Safety Becomes a Vulnerability: Exploiting LLM Alignment Homogeneity for Transferable Blocking in RAGcited by 4
  2. 2025User-Assistant Bias in LLMscited by 3
  3. 2025Evaluating Language Model Reasoning about Confidential Informationcited by 3
  4. 2026Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden Statescited by 3
  5. 2026Who Tests the Testers? Systematic Enumeration and Coverage Audit of LLM Agent Tool Call Safetycited by 2
  6. 2026Models That Know How Evaluations Are Designed Score Safercited by 2
  7. 2026Steering Instruction Hierarchies at Inference Timecited by 2
  8. 2026Deconstructing Instruction-Following: A New Benchmark for Granular Evaluation of Large Language Model Instruction Compliance Abilitiescited by 1
  9. 2026CCR-Bench: A Comprehensive Benchmark for Evaluating LLMs on Complex Constraints, Control Flows, and Real-World Casescited by 1
  10. 2026Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graphcited by 1
  11. 2026Hierarchical Alignment: Enforcing Hierarchical Instruction-Following in LLMs through Logical Consistencycited by 1
  12. 2026Bias in the Tails: How Name-conditioned Evaluative Framing in Resume Summaries Destabilizes LLM-based Hiringcited by 1
  13. 2026Faithful Mobile GUI Agents with Guided Advantage Estimatorcited by 1
  14. 2026Position: AI Safety Requires Effective Controllabilitycited by 1
  15. 2026Adversarial Pragmatics for AI Safety Evaluation: A Diagnostic Framework and Seed Benchmark for Language-Mediated Controlcited by 1
  16. 2026Harness-IF: Evaluating Instruction Following Across Instruction Surfaces in Coding Agentscited by 1
  17. 2026PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?cited by 1
  18. 2025Study on Supply Chain Finance Decision-Making Model and Enterprise Economic Performance Prediction Based on Deep Reinforcement Learningcited by 0
  19. 2025PSM: Prompt Sensitivity Minimization via LLM-Guided Black-Box Optimizationcited by 0
  20. 2025Exposing Hidden Biases in Text-to-Image Models via Automated Prompt Searchcited by 0
  21. 2026ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generationcited by 0
  22. 2026Prompt Complexity Dilutes Structured Reasoning: A Follow-Up Study on the Car Wash Problemcited by 0
  23. 2026WIRE: Profiling Witnessed Within-Policy Instruction Collisions in LLM Agentscited by 0
  24. 2026Evaluating using Mock Tool Calls to Quarantine Untrusted Prompt Inputscited by 0
  25. 2026Where Instruction Hierarchy Breaks: Diagnosing and Repairing Failures in Reasoning Language Modelscited by 0
  26. 2026Training LLMs to Enforce Multi-Level Instruction Hierarchies via Gravity-Weighted Direct Preference Optimizationcited by 0
  27. 2026IHDec: Divergence-Steered Contrastive Decoding for Securing Multi-Turn Instruction Hierarchiescited by 0
  28. 2026Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Modelscited by 0
  29. 2026Language Shapes Instruction Hierarchy Compliance in Multilingual LLMscited by 0
  30. 2026Hy-MultiTurn: A Six-Dimensional Benchmark for Deep Multi-Turn Dialogue Understandingcited by 0
  31. 2026How Language Models Choose Sides: Internal Representations of Instruction Hierarchycited by 0
  32. 2026Combating Instruction Conflict via Energy-Driven Latent Conflict Detectioncited by 0
  33. 2026IBBench-Light: A Paired Evaluation of Task-Conditioned Responses to External Directivescited by 0
  34. 2026Seeing Through Conflicts: Improving Instruction Hierarchy Alignment in Vision-Language Modelscited by 0
  35. 2026Where LLM Graders Succeed and Break: Evidence from Two Computer-Science Examscited by 0
  36. 2026Rufus-Air: An Open LLM Post-Training Recipecited by 0