mlsanityData to 13 Oct 2026

How Effective Is Constitutional AI in Small LLMs? A Study on DeepSeek-R1 and Its Peers

arXiv 2503.17365 ↗First posted 1 Feb 2025cs.LG cs.AI cs.CYcited by 3

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Recent incidents highlight safety risks in Large Language Models (LLMs), motivating research into alignment methods like Constitutional AI (CAI). This paper explores CAI's self-critique mechanism on small, uncensored 7-9B parameter models: DeepSeek-R1-8B, Gemma-2-9B, Llama 3.1-8B, and Qwen2.5-7B. We show that while Llama-based models exhibited significant harm reduction through self-critique, other architectures demonstrated less improvement in harm detection after abliteration. These results suggest CAI's effectiveness may vary depending on model architecture and reasoning capabilities.

Key facts. Cited by 3 papers in the map (data through 13 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 9 of its 16 references that are in the map. First posted on arXiv 1 Feb 2025.

Builds on

9 of 16 references.

References · 9

  1. 2022Training language models to follow instructions with human feedbackcited by 15,856
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,876
  3. 2017Deep reinforcement learning from human preferencescited by 3,871
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,011
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,405
  6. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,009
  7. 2020Aligning AI With Shared Human Valuescited by 717
  8. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 305
  9. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 197

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Domain-Specific Constitutional AI: Enhancing Safety in LLM-Powered Mental Health Chatbotscited by 4
  2. 2026The Implications of Linguistic Illegibility for LLM Securitycited by 1
  3. 2026A Lightweight Explainable Guardrail for Prompt Safetycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Large Language Model Safety: A Holistic Survey
  2. 2024Current state of LLM Risks and AI Guardrails
  3. 2024SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types
  4. 2025C3AI: Crafting and Evaluating Constitutions for Constitutional AI
  5. 2026EvalSafetyGap: A Hybrid Survey and Conceptual Framework for LLM Evaluation-Safety Failures
  6. 2025SAGE: A Generic Framework for LLM Safety Evaluation
  7. 2025Deception in LLMs: Self-Preservation and Autonomous Goals in Large Language Models
  8. 2025Efficient LLM Safety Evaluation through Multi-Agent Debate