mlsanityData to 7 Oct 2026

FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoning

arXiv 2505.08054 ↗First posted 12 May 2025cs.CL cs.AIcited by 53

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 2 more
Abstract (arXiv)

Safety alignment approaches in large language models (LLMs) often lead to the over-refusal of benign queries, significantly diminishing their utility in sensitive scenarios. To address this challenge, we introduce FalseReject, a comprehensive resource containing 16k seemingly toxic queries accompanied by structured responses across 44 safety-related categories. We propose a graph-informed adversarial multi-agent interaction framework to generate diverse and complex prompts, while structuring responses with explicit reasoning to aid models in accurately distinguishing safe from unsafe contexts. FalseReject includes training datasets tailored for both standard instruction-tuned models and reasoning-oriented models, as well as a human-annotated benchmark test set. Our extensive benchmarking on 29 state-of-the-art (SOTA) LLMs reveals persistent over-refusal challenges. Empirical results demonstrate that supervised finetuning with FalseReject substantially reduces unnecessary refusals without compromising overall safety or general language capabilities.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 46 of its 65 references that are in the map. First posted on arXiv 12 May 2025.

Builds on

46 of 65 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R118 Feb
  2. 2024OpenAI o1 System Card21 Dec

Other references · 44

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  6. 2024Qwen2.5 Technical Reportcited by 4,713
  7. 2024GPT-4o System Cardcited by 4,467
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  9. 2023Mistral 7Bcited by 3,065
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  11. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  12. 2024Mixtral of Expertscited by 1,785
32 more
  1. 2017Decoupled Weight Decay Regularizationcited by 1,722
  2. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  3. 2025s1: Simple test-time scalingcited by 1,311
  4. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  5. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  6. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  7. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  8. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  9. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  10. 2024DeepSeek-V3 Technical Reportcited by 724
  11. 2024Phi-4 Technical Reportcited by 675
  12. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  13. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  14. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  15. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  16. 2024A StrongREJECT for Empty Jailbreakscited by 352
  17. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
  18. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
  19. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  20. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  21. 2024Best Practices and Lessons Learned on Synthetic Datacited by 104
  22. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
  23. 2024Personalization of Large Language Models: A Surveycited by 100
  24. 2024Synthetic continued pretrainingcited by 56
  25. 2023Bypassing the Safety Training of Open-Source LLMs with Priming Attackscited by 48
  26. 2024Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablationcited by 48
  27. 2024SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steeringcited by 45
  28. 2024Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Modelscited by 35
  29. 2024Backtracking Improves Generation Safetycited by 31
  30. 2024From Theft to Bomb-Making: The Ripple Effect of Unlearning in Defending Against Jailbreak Attackscited by 29
  31. 2024ToolFlow: Boosting LLM Tool-Calling Through Natural and Coherent Dialogue Synthesiscited by 27
  32. 2024Refusal Tokens: A Simple Way to Calibrate Refusals in Large Language Modelscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 12; 2026: 41.

53 papers.

Sort:

By area:

  1. 2026AgentAbstain: Do LLM Agents Know When Not to Act?cited by 8
  2. 2025AutoRAN: Automated Hijacking of Safety Reasoning in Large Reasoning Modelscited by 7
  3. 2025Read the Scene, Not the Script: Outcome-Aware Safety for LLMscited by 6
  4. 2025Adversarial D'ej`a Vu: Jailbreak Dictionary Learning for Stronger Generalization to Unseen Attackscited by 6
  5. 2025The Personalization Trap: How User Memory Alters Emotional Reasoning in LLMscited by 5
  6. 2026When Safety Becomes a Vulnerability: Exploiting LLM Alignment Homogeneity for Transferable Blocking in RAGcited by 4
  7. 2025Probing Association Biases in LLM Moderation Over-Sensitivitycited by 3
  8. 2025Quantifying Fairness in LLMs Beyond Tokens: A Semantic and Statistical Perspectivecited by 3
  9. 2025Evaluating Large Language Models for Detecting Antisemitismcited by 3
  10. 2026Discern Truth from Falsehood: Reducing Over-Refusal via Contrastive Refinementcited by 3
  11. 2026STARS: Skill-Triggered Audit for Request-Conditioned Invocation Safety in Agent Systemscited by 3
  12. 2026Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversationscited by 3
38 more
  1. 2026EquiMem: Calibrating Shared Memory in Multi-Agent Debate via Game-Theoretic Equilibriumcited by 3
  2. 2026Do Thinking Tokens Help with Safety?cited by 3
  3. 2026Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervisioncited by 3
  4. 2025How Semantically Stable Are LLM Refusals? Measuring Confusion in Local Safety Boundariescited by 2
  5. 2026Differential Harm Propensity in Personalized LLM Agents: The Curious Case of Mental Health Disclosurecited by 2
  6. 2026Blind Refusal: Language Models Refuse to Help Users Evade Unjust, Absurd, and Illegitimate Rulescited by 2
  7. 2026OPIUM: Mitigating Steering Externalities and Over-Refusal via Dual Objective Latent Optimizationcited by 2
  8. 2026Reinforcement Learning for Large Language Model Selective Evidence Adoption from Contaminated Retrieval Resultscited by 2
  9. 2025Let Them Down Easy! Contextual Effects of LLM Guardrails on User Perceptions and Preferencescited by 1
  10. 2026Intent Laundering: AI Safety Datasets Are Not What They Seemcited by 1
  11. 2026Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMscited by 1
  12. 2026SciIntBench: Measuring LLM Compliance with Research Integrity Norms Under Adversarial Framingcited by 1
  13. 2026SyPS: Measuring Sycophancy Prompt Sensitivity in Large Language Modelscited by 1
  14. 2025SafeConstellations: Mitigating Over-Refusals in LLMs Through Task-Aware Representation Steeringcited by 0
  15. 2025Between a Rock and a Hard Place: The Tension Between Ethical Reasoning and Safety Alignment in LLMscited by 0
  16. 2025Understanding and Mitigating Over-refusal for Large Language Models via Representation Interventioncited by 0
  17. 2026Light Alignment Improves LLM Safety via Model Self-Reflection with a Single Neuroncited by 0
  18. 2026C-DeltaTheta: Circuit-Restricted Weight Arithmetic for Selective Refusalcited by 0
  19. 2026IatroBench: A Pre-Registered Benchmark of Clinical Omission in Language Modelscited by 0
  20. 2026When Verification Fails: How Compositionally Infeasible Claims Escape Rejectioncited by 0
  21. 2026RMGAP: Benchmarking the Generalization of Reward Models across Diverse Preferencescited by 0
  22. 2026One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intentcited by 0
  23. 2026Beyond Fixed Benchmarks and Worst-Case Attacks: Dynamic Boundary Evaluation for Language Modelscited by 0
  24. 2026Residual Paving: Diagnosing the Routing Bottleneck in Selective Refusal Editingcited by 0
  25. 2026AERIC: Anticipatory Hidden-State Monitoring for Implicit Harmful Dialoguecited by 0
  26. 2026The Joint Effect of Quantization and Sampling Temperature on LLM Safety Alignment: A Factorial Analysiscited by 0
  27. 2026Addressing Over-Refusal in LLMs with Competing Rewardscited by 0
  28. 2026Robust Critics: Defending LLMs Against Multi-Turn Attackscited by 0
  29. 2026Refusing Intent, Not Form: Wrapper-Based Intent-Group Supervision for LLM Safetycited by 0
  30. 2026Who Pays More for Safety? Measuring the Disparate Cost of Safety Alignment across Languagescited by 0
  31. 2026RENDER: Controlling Reader-Facing Evidence in LLM Memory Evaluationcited by 0
  32. 2026A Unified Mechanistic Analysis of Knowledge- and Safety-Based Refusalscited by 0
  33. 2026Door-in-the-Face Requests and Refusal Behaviour in Large Language Modelscited by 0
  34. 2026Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusalcited by 0
  35. 2026Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Modelscited by 0
  36. 2026When Tools Get in the Way: The Effect of Unnecessary Tool Availability on LLM Answeringcited by 0
  37. 2026Mitigating LLM Over-Refusal via Dynamic Semantic Routing Calibrationcited by 0
  38. 2026Alignment of LRMs via Counter-Aligned Few-Shot Conversation Exposurecited by 0

50 most cited of 53 All 53 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety
  2. 2026LLMs Can Unlearn Refusal with Only 1,000 Benign Samples
  3. 2025From Threat to Tool: Leveraging Refusal-Aware Injection Attacks for Safety Alignment
  4. 2026Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment
  5. 2025Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning
  6. 2026RAS: Measuring LLM Safety Through Refusal Alignment
  7. 2026From Refusal Geometry to Safety Geometry: Harmfulness--Refusal Coupling under Dynamic Adversarial Fine-Tuning
  8. 2026Understanding the Effects of Safety Unalignment on Large Language Models