mlsanityData to 14 Oct 2026

Personalized Constitutionally-Aligned Agentic Superego: Secure AI Behavior Aligned to Diverse Human Values

arXiv 2506.13774 ↗First posted 8 Jun 2025cs.AI cs.CY cs.MAcited by 1

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Agentic AI systems, possessing capabilities for autonomous planning and action, show great potential across diverse domains. However, their practical deployment is hindered by challenges in aligning their behavior with varied human values, complex safety requirements, and specific compliance needs. Existing alignment methodologies often falter when faced with the complex task of providing personalized context without inducing confabulation or operational inefficiencies. This paper introduces a novel solution: a 'superego' agent, designed as a personalized oversight mechanism for agentic AI. This system dynamically steers AI planning by referencing user-selected 'Creed Constitutions' encapsulating diverse rule sets -- with adjustable adherence levels to fit non-negotiable values. A real-time compliance enforcer validates plans against these constitutions and a universal ethical floor before execution. We present a functional system, including a demonstration interface with a prototypical constitution-sharing portal, and successful integration with third-party models via the Model Context Protocol (MCP). Comprehensive benchmark evaluations (HarmBench, AgentHarm) demonstrate that our Superego agent dramatically reduces harmful outputs -- achieving up to a 98.3% harm score reduction and near-perfect refusal rates (e.g., 100% with Claude Sonnet 4 on AgentHarm's harmful set) for leading LLMs like Gemini 2.5 Flash and GPT-4o. This approach substantially simplifies personalized AI alignment, rendering agentic systems more reliably attuned to individual and cultural contexts, while also enabling substantial safety improvements. An overview on this research with examples is available at https://superego.creed.space.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 25 of its 50 references that are in the map. First posted on arXiv 8 Jun 2025.

Builds on

25 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct

Other references · 24

  1. 2017Deep reinforcement learning from human preferencescited by 3,912
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  4. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  5. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  6. 2020Artificial Intelligence, Values and Alignmentcited by 448
  7. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  8. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
  9. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 182
  10. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 181
  11. 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 172
  12. 2024Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferencescited by 144
12 more
  1. 2024OASIS: Open Agent Social Interaction Simulations with One Million Agentscited by 132
  2. 2024InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidancecited by 83
  3. 2024Project Sid: Many-agent simulations toward AI civilizationcited by 49
  4. 2024Personalized Large Language Modelscited by 44
  5. 2024Confabulation: The Surprising Value of Large Language Model Hallucinationscited by 43
  6. 2025Dynamic Search for Inference-Time Alignment in Diffusion Modelscited by 37
  7. 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirementscited by 36
  8. 2025ShieldGemma 2: Robust and Tractable Image Content Moderationcited by 28
  9. 2025Collaborating Action by Action: A Multi-agent LLM Framework for Embodied Reasoningcited by 21
  10. 2025On Almost Surely Safe Alignment of Large Language Models at Inference-Timecited by 14
  11. 2025Disentangled Safety Adapters Enable Efficient Guardrails and Flexible Inference-Time Alignmentcited by 6
  12. 2025Revealing economic facts: LLMs know more than they saycited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safetycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Digital Apprentice: A Framework for Human-Directed Agentic AI Development
  2. 2026COMPASS: The explainable agentic framework for Sovereignty, Sustainability, Compliance, and Ethics
  3. 2026Value-Preserving Architectures for Agentic AI Systems
  4. 2026Computational Reflective Equilibrium for Agentic AI: A Rawlsian Alignment Scaffold for Governance
  5. 2026Autonomy and Agency in Agentic AI: Architectural Tactics for Regulated Contexts
  6. 2026Behavioral Governance for Autonomous AI Agents: The AgentBound Framework
  7. 2025DoubleAgents: Human-Agent Alignment in a Socially Embedded Workflow
  8. 2026Breaking Up with Normatively Monolithic Agency with GRACE: A Reason-Based Neuro-Symbolic Architecture for Safe and Ethical AI Alignment