mlsanityData to 14 Oct 2026

Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Models

arXiv 2509.01909 ↗First posted 2 Sep 2025cs.AI cs.CL cs.CYcited by 19

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Large language models (LLMs) typically deploy safety mechanisms to prevent harmful content generation. Most current approaches focus narrowly on risks posed by malicious actors, often framing risks as adversarial events and relying on defensive refusals. However, in real-world settings, risks also come from non-malicious users seeking help while under psychological distress (e.g., self-harm intentions). In such cases, the model's response can strongly influence the user's next actions. Simple refusals may lead them to repeat, escalate, or move to unsafe platforms, creating worse outcomes. We introduce Constructive Safety Alignment (CSA), a human-centric paradigm that protects against malicious misuse while actively guiding vulnerable users toward safe and helpful results. Implemented in Oyster-I (Oy1), CSA combines game-theoretic anticipation of user reactions, fine-grained risk boundary discovery, and interpretable reasoning control, turning safety into a trust-building process. Oy1 achieves state-of-the-art safety among open models while retaining high general capabilities. On our Constructive Benchmark, it shows strong constructive engagement, close to GPT-5, and unmatched robustness on the Strata-Sword jailbreak dataset, nearing GPT-o1 levels. By shifting from refusal-first to guidance-first safety, CSA redefines the model-user relationship, aiming for systems that are not just safe, but meaningfully helpful. We release Oy1, code, and the benchmark to support responsible, user-centered AI.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 44 of its 90 references that are in the map. First posted on arXiv 2 Sep 2025.

Builds on

44 of 90 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable1 Mar
  2. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference20 Jun
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 41

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  7. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  8. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  9. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  10. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  11. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
  12. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
29 more
  1. 2024A StrongREJECT for Empty Jailbreakscited by 385
  2. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 364
  3. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  4. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 267
  5. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 184
  6. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 130
  7. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  8. 2024Automatic and Universal Prompt Injection Attacks against Large Language Modelscited by 119
  9. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  10. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 110
  11. 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 98
  12. 2024Curiosity-driven Red-teaming for Large Language Modelscited by 88
  13. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 67
  14. 2025RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capabilitycited by 59
  15. 2025From Hard Refusals to Safe-Completions: Toward Output-Centric Safety Trainingcited by 52
  16. 2024GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Modelscited by 48
  17. 2024AI Safety in Generative AI Large Language Models: A Surveycited by 33
  18. 2024Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Modelscited by 33
  19. 2025Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judgecited by 23
  20. 2025Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignmentcited by 20
  21. 2024PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximizationcited by 17
  22. 2025Breaking the Ceiling: Exploring the Potential of Jailbreak Attacks through Expanding Strategy Spacecited by 17
  23. 2024Efficient Universal Goal Hijacking with Semantics-guided Prompt Organizationcited by 16
  24. 2025UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Modelscited by 16
  25. 2025AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defensescited by 15
  26. 2025Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMscited by 9
  27. 2025Geneshift: Impact of different scenario shift on Jailbreaking LLMcited by 5
  28. 2025Token-Efficient Prompt Injection Attack: Provoking Cessation in LLM Reasoning via Adaptive Token Compressioncited by 4
  29. 2025C3-Bench: The Things Real Disturbing LLM based Agent in Multi-Taskingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 12.

19 papers.

Sort:

By area:

  1. 2025DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarioscited by 19
  2. 2025SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAGcited by 13
  3. 2025Towards Safe Reasoning in Large Reasoning Models via Corrective Interventioncited by 11
  4. 2025OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluationcited by 11
  5. 2026Risky-Bench: Probing Agentic Safety Risks under Real-World Deploymentcited by 8
  6. 2025EcoAlign: An Economically Rational Framework for Efficient LVLM Alignmentcited by 5
  7. 2026AI Alignment Breaks at the Edgecited by 4
  8. 2026OpenSafeIntent: Evaluating Intent-Calibrated Safe Completion Across Dual-Use Prompt Setscited by 2
  9. 2025VisuoAlign: Safety Alignment of LVLMs with Multimodal Tree Searchcited by 0
  10. 2025Information-Dense Reasoning for Efficient and Auditable Security Alert Triagecited by 0
  11. 2026ShellForge: Adversarial Co-Evolution of Webshell Generation and Multi-View Detection for Robust Webshell Defensecited by 0
  12. 2026From Sparse Decisions to Dense Reasoning: A Multi-attribute Trajectory Paradigm for Multimodal Moderationcited by 0
7 more
  1. 2026Membership Inference for Contrastive Pre-training Models with Text-only PII Queriescited by 0
  2. 2026MESA: Improving MoE Safety Alignment via Decentralized Expertisecited by 0
  3. 2026Dialectics of Alignment: Harnessing Unsafe Knowledge for Dynamic Safety Routingcited by 0
  4. 2026Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Modelscited by 0
  5. 2026ElasticBack: Stealthy Conditional Backdoor in LLM-Agent Skills via Coupled Trigger-Rule Optimizationcited by 0
  6. 2026HiRoute: Hierarchical Routed Prompt Tuning for Safety Alignment of Large Language Modelscited by 0
  7. 2026SafeRI: Recognition and Intervention for Token-Level Safety Intervention in Large Vision Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Safety Pretraining: Toward the Next Generation of Safe AI
  2. 2025The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions
  3. 2025Between a Rock and a Hard Place: The Tension Between Ethical Reasoning and Safety Alignment in LLMs
  4. 2026The Refusal--Compliance Tradeoff: A Large-Scale Safety Behavior Audit of Large Language Models
  5. 2026Mind the GAP: Text Safety Does Not Transfer to Tool-Call Safety in LLM Agents
  6. 2026Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment
  7. 2026Beyond Refusal Patterns: Safe-Role Internalization for Robust and Generalizable LLM Safety Alignment
  8. 2025Beyond Reactive Safety: Risk-Aware LLM Alignment via Long-Horizon Simulation