Data to 7 Oct 2026

Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming

arXiv 2501.18837 ↗First posted 31 Jan 2025cs.CL cs.AI cs.CRcited by 172

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large language models (LLMs) are vulnerable to universal jailbreaks-prompting strategies that systematically bypass model safeguards and enable users to carry out harmful processes that require many model interactions, like manufacturing illegal substances at scale. To defend against these attacks, we introduce Constitutional Classifiers: safeguards trained on synthetic data, generated by prompting LLMs with natural language rules (i.e., a constitution) specifying permitted and restricted content. In over 3,000 estimated hours of red teaming, no red teamer found a universal jailbreak that could extract information from an early classifier-guarded LLM at a similar level of detail to an unguarded model across most target queries. On automated evaluations, enhanced classifiers demonstrated robust defense against held-out domain-specific jailbreaks. These classifiers also maintain deployment viability, with an absolute 0.38% increase in production-traffic refusals and a 23.7% inference overhead. Our work demonstrates that defending against universal jailbreaks while maintaining practical deployment viability is tractable.

Key facts. Cited by 172 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. First posted on arXiv 31 Jan 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 78; 2026: 94.

172 papers.

Sort:

By area:

2025Inference-Time Scaling for Generalist Reward Modelingcited by 233
2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 222
2025Agentic Misalignment: How LLMs Could Be Insider Threatscited by 136
2025Qwen3Guard Technical Reportcited by 125
2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 112
2025OverThink: Slowdown Attacks on Reasoning LLMscited by 94
2025Effectively Controlling Reasoning Models through Thinking Interventioncited by 63
2025ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Casescited by 51
2025Detecting High-Stakes Interactions with Activation Probescited by 45
2026Building Production-Ready Probes For Geminicited by 38
2026Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreakscited by 36
2025Real-Time Detection of Hallucinated Entities in Long-Form Generationcited by 33

38 more

2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Surveycited by 26
2025Monitoring Decomposition Attacks in LLMs with Lightweight Sequential Monitorscited by 21
2025From Judgment to Interference: Early Stopping LLM Harmful Outputs via Streaming Content Monitoringcited by 20
2025FORTRESS: Frontier Risk Evaluation for National Security and Public Safetycited by 20
2025RL-Obfuscation: Can Language Models Learn to Evade Latent-Space Monitors?cited by 19
2025Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmascited by 17
2025SoK: Evaluating Jailbreak Guardrails for Large Language Modelscited by 17
2025PurpCode: Reasoning for Safer Code Generationcited by 17
2026Extracting books from production language modelscited by 16
2026How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competitioncited by 16
2026A Framework for Formalizing LLM Agent Securitycited by 16
2025Multi-Turn Jailbreaks Are Simpler Than They Seemcited by 15
2025Position: Adversarial ML for LLMs Is Not Making Any Progresscited by 14
2025STACK: Adversarial Attacks on LLM Safeguard Pipelinescited by 14
2025Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missingcited by 14
2026IH-Challenge: A Training Dataset to Improve Instruction Hierarchy on Frontier LLMscited by 14
2025Learning Safety Constraints for Large Language Modelscited by 13
2025SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compressioncited by 12
2026Shaping capabilities with token-level data filteringcited by 11
2026Detecting Safety Violations Across Many Agent Tracescited by 11
2025MixAT: Combining Continuous and Discrete Adversarial Training for LLMscited by 10
2025A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreakscited by 10
2025Combining Cost-Constrained Runtime Monitors for AI Safetycited by 10
2025Stress-Testing Model Specs Reveals Character Differences among Language Modelscited by 10
2025The Alignment Waltz: Jointly Training Agents to Collaborate for Safetycited by 10
2026Defensive Refusal Bias: How Safety Alignment Fails Cyber Defenderscited by 10
2025AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Securitycited by 9
2025Adversarial Alignment for LLMs Requires Simpler, Reproducible, and More Measurable Objectivescited by 8
2025Benchmarking Misuse Mitigation Against Covert Adversariescited by 8
2025PersonaTeaming: Exploring How Introducing Personas Can Improve Automated AI Red-Teamingcited by 8
2026When Benchmarks Lie: Evaluating Malicious Prompt Classifiers Under True Distribution Shiftcited by 8
2025Stronger Enforcement of Instruction Hierarchy via Augmented Intermediate Representationscited by 7
2025D-REX: A Benchmark for Detecting Deceptive Reasoning in Large Language Modelscited by 7
2025All Code, No Thought: Current Language Models Struggle to Reason in Ciphered Languagecited by 7
2026Boundary Point Jailbreaking of Black-Box LLMscited by 7
2026Constitutional Black-Box Monitoring for Scheming in LLM Agentscited by 7
2025Superintelligence Strategy: Expert Versioncited by 6
2025When Style Breaks Safety: Defending LLMs Against Superficial Style Alignmentcited by 6

50 most cited of 172 All 172 (JSON)