Read the Scene, Not the Script: Outcome-Aware Safety for LLMs
On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks
also in 1 more
Abstract (arXiv)
Safety-aligned Large Language Models (LLMs) still show two dominant failure modes: they are easily jailbroken, or they over-refuse harmless inputs that contain sensitive surface signals. We trace both to a common cause: current models reason weakly about links between actions and outcomes and over-rely on surface-form signals, lexical or stylistic cues that do not encode consequences. We define this failure mode as Consequence-blindness. To study consequence-blindness, we build a benchmark named CB-Bench covering four risk scenarios that vary whether semantic risk aligns with outcome risk, enabling evaluation under both matched and mismatched conditions which are often ignored by existing safety benchmarks. Mainstream models consistently fail to separate these risks and exhibit consequence-blindness, indicating that consequence-blindness is widespread and systematic. To mitigate consequence-blindness, we introduce CS-Chain-4k, a consequence-reasoning dataset for safety alignment. Models fine-tuned on CS-Chain-4k show clear gains against semantic-camouflage jailbreaks and reduce over-refusal on harmless inputs, while maintaining utility and generalization on other benchmarks. These results clarify the limits of current alignment, establish consequence-aware reasoning as a core alignment goal and provide a more practical and reproducible evaluation path.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 41 of its 48 references that are in the map. First posted on arXiv 5 Oct 2025.
Builds on
41 of 48 references.
Closest ancestors · 13
Same direction, newest first
- 2025Beyond Surface Alignment: Rebuilding LLMs Safety Mechanism via Probabilistically Ablating Refusal Direction18 Sep
- 2025FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoning12 May
- 2025AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender13 Apr
- 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety6 Mar
- 2025Improving LLM Safety Alignment with Dual-Objective Optimization5 Mar
- 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable1 Mar
- 2025The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions13 Feb
- 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements11 Oct
- 2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning18 Aug
- 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal20 Jun
- 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deep10 Jun
- 2024OR-Bench: An Over-Refusal Benchmark for Large Language Models31 May
Other references · 28
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2025Gemma 3 Technical Reportcited by 1,686
- 2024A Survey on LLM-as-a-Judgecited by 1,384
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
16 more
- 2024A Survey on Large Language Models for Code Generationcited by 646
- 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
- 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
- 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
- 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
- 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 292
- 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
- 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 206
- 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilitiescited by 133
- 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
- 2024Controllable Preference Optimization: Toward Controllable Multi-Objective Alignmentcited by 105
- 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
- 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
- 2024SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selectioncited by 64
- 2025From Hard Refusals to Safe-Completions: Toward Output-Centric Safety Trainingcited by 49
- 2025Think Before Refusal : Triggering Safety Reflection in LLMs to Mitigate False Refusal Behaviorcited by 12
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026OpenSafeIntent: Evaluating Intent-Calibrated Safe Completion Across Dual-Use Prompt Setscited by 2
- 2026The Entanglement Wall: Activation-Space Probes as Risk Detectors, Not Context Adjudicatorscited by 2
- 2026Safeguards Based on Copyable Context Cannot Provide Reliable Safety for LLMscited by 1
- 2026MortalMATH: Evaluating the Conflict Between Reasoning Objectives and Emergency Contextscited by 0
- 2026Beyond Semantic Accuracy: Consequence-Aware Evaluation for Safety-Critical Language Understandingcited by 0
- 2026First Token Matters: Understanding Safety Collapse in Large Reasoning Modelscited by 0
- 2026First Token Matters: Understanding Safety Collapse in Large Reasoning Models16 Sep 2026
- 2026Beyond Semantic Accuracy: Consequence-Aware Evaluation for Safety-Critical Language Understanding25 Aug 2026
- 2026Safeguards Based on Copyable Context Cannot Provide Reliable Safety for LLMs30 Jul 2026
- 2026The Entanglement Wall: Activation-Space Probes as Risk Detectors, Not Context Adjudicators12 Jul 2026
- 2026OpenSafeIntent: Evaluating Intent-Calibrated Safe Completion Across Dual-Use Prompt Sets2 Jul 2026
- 2026MortalMATH: Evaluating the Conflict Between Reasoning Objectives and Emergency Contexts26 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2026Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack
- 2026OOD-MMSafe: Advancing MLLM Safety from Harmful Intent to Hidden Consequences
- 2026LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety
- 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons
- 2026When Safety Speaks a Language: A Mechanistic Analysis of Safety-Language Identity Entanglement in LLMs
- 2025Picky LLMs and Unreliable RMs: An Empirical Study on Safety Alignment after Instruction Tuning
- 2024Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models
- 2026Understanding and Preserving Safety in Fine-Tuned LLMs