mlsanityData to 14 Oct 2026

FORTRESS: Frontier Risk Evaluation for National Security and Public Safety

arXiv 2506.14922 ↗First posted 17 Jun 2025cs.CY cs.LGcited by 24

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

The rapid advancement of large language models (LLMs) introduces dual-use capabilities that could both threaten and bolster national security and public safety (NSPS). Models implement safeguards to protect against potential misuse relevant to NSPS and allow for benign users to receive helpful information. However, current benchmarks often fail to test safeguard robustness to potential NSPS risks in an objective, robust way. We introduce FORTRESS: 500 expert-crafted adversarial prompts with instance-based rubrics of 4-7 binary questions for automated evaluation across 3 domains (unclassified information only): Chemical, Biological, Radiological, Nuclear and Explosive (CBRNE), Political Violence & Terrorism, and Criminal & Financial Illicit Activities, with 10 total subcategories across these domains. Each prompt-rubric pair has a corresponding benign version to test for model over-refusals. This evaluation of frontier LLMs' safeguard robustness reveals varying trade-offs between potential risks and model usefulness: Claude-3.5-Sonnet demonstrates a low average risk score (ARS) (14.09 out of 100) but the highest over-refusal score (ORS) (21.8 out of 100), while Gemini 2.5 Pro shows low over-refusal (1.4) but a high average potential risk (66.29). Deepseek-R1 has the highest ARS at 78.05, but the lowest ORS at only 0.06. Models such as o1 display a more even trade-off between potential risks and over-refusals (with an ARS of 21.69 and ORS of 5.2). To provide policymakers and researchers with a clear understanding of models' potential risks, we publicly release FORTRESS at https://huggingface.co/datasets/ScaleAI/fortress_public. We also maintain a private set for evaluation.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 53 of its 72 references that are in the map. First posted on arXiv 17 Jun 2025.

Builds on

53 of 72 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types29 Oct
  2. 2024Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents11 Oct
  3. 2024Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Models1 Sep
  4. 2024Improving Alignment and Robustness with Circuit Breakers6 Jun
  5. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Models31 May
  6. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct

Other references · 47

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  8. 2022Red Teaming Language Models with Language Modelscited by 1,026
  9. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  10. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  11. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  12. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
35 more
  1. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  2. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 437
  3. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
  4. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  5. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 259
  6. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 243
  7. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 240
  8. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
  9. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 231
  10. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 228
  11. 2023An Overview of Catastrophic AI Riskscited by 222
  12. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 181
  13. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 173
  14. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 163
  15. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 156
  16. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 151
  17. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 138
  18. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 116
  19. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 114
  20. 2024Efficient Adversarial Training in LLMs with Continuous Attackscited by 113
  21. 2023Frontier AI Regulation: Managing Emerging Risks to Public Safetycited by 98
  22. 2024Attacking Large Language Models with Projected Gradient Descentcited by 94
  23. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 69
  24. 2024Multi-Turn Context Jailbreak Attack on Large Language Models From First Principlescited by 56
  25. 2024PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Railscited by 52
  26. 2024JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Frameworkcited by 47
  27. 2024Navigating the OverKill in Large Language Modelscited by 44
  28. 2025Safety Pretraining: Toward the Next Generation of Safe AIcited by 43
  29. 2025Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmarkcited by 31
  30. 2025SafeDialBench: A Fine-Grained Safety Evaluation Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attackscited by 23
  31. 2024Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Datacited by 20
  32. 2021Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Faircited by 12
  33. 2024Revisiting the Robust Alignment of Circuit Breakerscited by 12
  34. 2025FLEX: A Benchmark for Evaluating Robustness of Fairness in Large Language Modelscited by 8
  35. 2025Unmasking the Canvas: A Dynamic Benchmark for Image Generation Jailbreaking and LLM Content Safetycited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 22.

24 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  2. 2025Large Reasoning Models Learn Better Alignment from Flawed Thinkingcited by 13
  3. 2026Defensive Refusal Bias: How Safety Alignment Fails Cyber Defenderscited by 10
  4. 2026Judge Reliability Harness: Stress Testing the Reliability of LLM Judgescited by 8
  5. 2026CSSBench: Evaluating the Safety of Lightweight LLMs against Chinese-Specific Adversarial Patternscited by 5
  6. 2026ForesightSafety Bench: A Frontier Risk Evaluation and Governance Framework towards Safe AIcited by 3
  7. 2026An Independent Safety Evaluation of Kimi K2.5cited by 3
  8. 2026Internalizing Safety Understanding in Large Reasoning Models via Verificationcited by 3
  9. 2026Do Thinking Tokens Help with Safety?cited by 3
  10. 2026Self-ReSET: Learning to Self-Recover from Unsafe Reasoning Trajectoriescited by 1
  11. 2026Muse Spark Safety & Preparedness Reportcited by 1
  12. 2026PolicyAlign: Direct Policy-Based Safety Alignment for Large Language Modelscited by 1
12 more
  1. 2026Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysiscited by 1
  2. 2026SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoringcited by 1
  3. 2026Synthetic Persona Pretraining: Alignment from Token Zerocited by 1
  4. 2026Safin-1: Safety from Within through Memory-Native State Evolutioncited by 1
  5. 2026Eliciting Trustworthiness Priors of Large Language Models via Economic Gamescited by 0
  6. 2026ROK-FORTRESS: Measuring the Effect of Geopolitical Transcreation for National Security and Public Safetycited by 0
  7. 2026Addressing Over-Refusal in LLMs with Competing Rewardscited by 0
  8. 2026SAGE: Safety-First Defense-in-Depth Guardrails for Verified Lifecycle Control of High-Impact Generative AIcited by 0
  9. 2026Palmyra x6 Technical Report: An Agentic, Tool-Use Model Post-Trained via Anchored Supervised Fine-Tuningcited by 0
  10. 2026Fool's Gold: Defensive Deception Against Safety-Removal Attacks on Open-Weight Modelscited by 0
  11. 2026Alignment of LRMs via Counter-Aligned Few-Shot Conversation Exposurecited by 0
  12. 2026Towards Mitigating Deceptive Safety Alignment in Large Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Quantifying CBRN Risk in Frontier Models
  2. 2025Safety and Security Analysis of Large Language Models: Benchmarking Risk Profile and Harm Potential
  3. 2026Security Assessment and Mitigation Strategies for Large Language Models: A Comprehensive Defensive Framework
  4. 2025aiXamine: Simplified LLM Safety and Security
  5. 2025Policy-Grounded Safety Evaluation of 20 Large Language Models
  6. 2026XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity
  7. 2026Adversarial Humanities Benchmark: Results on Stylistic Robustness in Frontier Model Safety
  8. 2026When Prompt Optimization Becomes Jailbreaking: Adaptive Red-Teaming of Large Language Models