mlsanityData to 7 Oct 2026

Between a Rock and a Hard Place: The Tension Between Ethical Reasoning and Safety Alignment in LLMs

arXiv 2509.05367 ↗First posted 4 Sep 2025cs.CR cs.AIcited by 0

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 2 more
Abstract (arXiv)

Large Language Model safety alignment predominantly operates on a binary assumption that requests are either safe or unsafe. This classification proves insufficient when models encounter ethical dilemmas, where the capacity to reason through moral trade-offs creates a distinct attack surface. We formalize this vulnerability through TRIAL, a multi-turn red-teaming methodology that embeds harmful requests within ethical framings. TRIAL achieves high attack success rates across most tested models by systematically exploiting the model's ethical reasoning capabilities to frame harmful actions as morally necessary compromises. Building on these insights, we introduce ERR (Ethical Reasoning Robustness), a defense framework that distinguishes between instrumental responses that enable harmful outcomes and explanatory responses that analyze ethical frameworks without endorsing harmful acts. ERR employs a Layer-Stratified Harm-Gated LoRA architecture, achieving robust defense against reasoning-based attacks while preserving model utility.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 29 of its 71 references that are in the map. First posted on arXiv 4 Sep 2025.

Builds on

29 of 71 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoning12 May

Other references · 28

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2023Mistral 7Bcited by 3,065
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  9. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  10. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  12. 2024DeepSeek-V3 Technical Reportcited by 724
16 more
  1. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  2. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  3. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  4. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  5. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  6. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  7. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  8. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
  9. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  10. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  11. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  12. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  13. 2023Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMscited by 64
  14. 2024Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Modelscited by 35
  15. 2024Faster-GCG: Efficient Discrete Optimization Jailbreak Attacks against Aligned Large Language Modelscited by 23
  16. 2024AmpleGCG-Plus: A Strong Generative Model of Adversarial Suffixes to Jailbreak LLMs with Higher Success Rates in Fewer Attemptscited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Models
  2. 2026Beyond Routine Compliance: Cunning Data Cultivates Safety Vigilance in Large Language Models
  3. 2026CORDA: A Benchmark for Hierarchical Harm-Centric Moral Reasoning in Large Language Models
  4. 2026Reasoning over Precedents Alongside Statutes: Case-Augmented Deliberative Alignment for LLM Safety
  5. 2026Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack
  6. 2025Advancing Automated Ethical Profiling in SE: a Zero-Shot Evaluation of LLM Reasoning
  7. 2025Beyond Reactive Safety: Risk-Aware LLM Alignment via Long-Horizon Simulation
  8. 2025SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialogues