mlsanityData to 14 Oct 2026

Language Shapes Instruction Hierarchy Compliance in Multilingual LLMs

arXiv 2607.23545 ↗First posted 26 Jul 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Instruction hierarchy (IH) requires models to prioritize instructions by source, ensuring that higher-priority instructions override lower-priority ones. Despite its importance for safe and controllable deployment, existing evaluations have focused almost exclusively on English, leaving it unclear whether IH compliance remains stable in multilingual settings. We introduce XIH-Bench, a benchmark for multilingual IH evaluation with both same-language and cross-language conflicts across six languages, four domains, and three IH settings. Across models, we find two consistent patterns. First, IH compliance exhibits a clear language-dependent asymmetry: a language that strengthens compliance in the higher-priority position can become disruptive in the lower-priority position. Second, cross-language conflicts yield higher compliance than same-language conflicts, a phenomenon we term the Language Boundary Effect. We further show that language specialization can make lower-priority instructions in model-favored languages harder to override, creating multilingual reliability and security risks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 45 references that are in the map. First posted on arXiv 26 Jul 2026.

Builds on

23 of 45 references.

References · 23

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  4. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
  5. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
  6. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agentscited by 369
  7. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 333
  8. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  9. 2023PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traitscited by 234
  10. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 152
  11. 2024Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchycited by 67
  12. 2023Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injectioncited by 59
11 more
  1. 2024Measuring and Controlling Instruction (In)Stability in Language Model Dialogscited by 53
  2. 2025IHEval: Evaluating Language Models on Following the Instruction Hierarchycited by 53
  3. 2024Explaining Length Bias in LLM-Based Preference Evaluationscited by 49
  4. 2024Who's asking? User personas and the mechanics of latent misalignmentcited by 40
  5. 2025Improve LLM-as-a-Judge Ability as a General Abilitycited by 37
  6. 2025Enhancing Persona Consistency for LLMs' Role-Playing using Persona-Aware Contrastive Learningcited by 27
  7. 2024MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignmentcited by 25
  8. 2025From Prompts to Templates: A Systematic Prompt Template Analysis for Real-world LLMappscited by 21
  9. 2026IH-Challenge: A Training Dataset to Improve Instruction Hierarchy on Frontier LLMscited by 15
  10. 2024Concept Space Alignment in Multilingual LLMscited by 13
  11. 2025XIFBench: Evaluating Large Language Models on Multilingual Instruction Followingcited by 11

Built on by

Not cited yet.