mlsanityData to 14 Oct 2026

Position: AI Safety Requires Effective Controllability

arXiv 2605.27117 ↗First posted 26 May 2026cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

AI safety is still largely framed as alignment: training models to follow human preferences, safety policies, and normative constraints. That framing has improved the behavior of modern language models, but aligned behavior does not by itself guarantee that a deployed agent can be stopped, overridden, or constrained once it operates in open-ended, interactive, and tool-using environments. A system may be safe in expectation and still fail to yield to explicit runtime authority under conflicting instructions, long-horizon execution, adversarial inputs, or risky tool use. This position paper argues that AI safety therefore requires controllability as a first-class objective. We define controllability as the ability of an AI system to remain reliably interruptible, overridable, redirectable, and constrainable by explicit control signals at runtime while preserving ordinary utility when such signals are absent. To study this gap, we introduce controlbench, a benchmark for evaluating controllability failures in high-risk agentic scenarios. Experiments with OpenClaw-based agents show that current alignment and guardrail mechanisms reduce risk, but often fail to provide persistent, authoritative, and enforceable runtime control. We therefore propose a control-centric architectural framework that highlights explicit control planes, runtime intervention pathways, persistent control states, and auditable decision interfaces as key design principles for future controllable AI systems.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 41 of its 53 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

41 of 53 references.

References · 41

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  4. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  5. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  7. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  8. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  9. 2022Red Teaming Language Models with Language Modelscited by 1,026
  10. 2023Steering Language Models With Activation Engineeringcited by 922
  11. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  12. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
29 more
  1. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  2. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  3. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  4. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  5. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  6. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  7. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  8. 2016Cooperative Inverse Reinforcement Learningcited by 300
  9. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 261
  10. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 247
  11. 2023Reward Model Ensembles Help Mitigate Overoptimizationcited by 231
  12. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
  13. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 223
  14. 2024Towards Safer Large Language Models through Machine Unlearningcited by 139
  15. 2023Controlled Decoding from Language Modelscited by 135
  16. 2024GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoningcited by 124
  17. 2026AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolutioncited by 115
  18. 2023Confronting Reward Model Overoptimization with Constrained RLHFcited by 96
  19. 2024Decoding-Time Language Model Alignment with Multiple Objectivescited by 94
  20. 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detectioncited by 89
  21. 2024Decoding-time Realignment of Language Modelscited by 67
  22. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 63
  23. 2024Controllable Text Generation for Large Language Models: A Surveycited by 57
  24. 2024A Closer Look at Machine Unlearning for Large Language Modelscited by 56
  25. 2025IHEval: Evaluating Language Models on Following the Instruction Hierarchycited by 53
  26. 2025SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Modelscited by 29
  27. 2024SafeInfer: Context Adaptive Decoding Time Safety Alignment for Large Language Modelscited by 23
  28. 2024Reinforcement Learning with Token-level Feedback for Controllable Text Generationcited by 16
  29. 2026AIR: Improving Agent Safety through Incident Responsecited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025Evaluating LLM Agent Adherence to Hierarchical Safety Principles: A Lightweight Benchmark for Probing Foundational Controllability Components
  2. 2025Safety Must Precede the Deployment of Open-Ended AI
  3. 2024Safeguarding AI Agents: Developing and Analyzing Safety Architectures
  4. 2026Agentic Safety is an Epistemic Property, Not a Behavioral One
  5. 2025Agent Safety Alignment via Reinforcement Learning
  6. 2026The Causal Impact of Tool Affordance on Safety Alignment in LLM Agents
  7. 2025A Safety and Security Framework for Real-World Agentic Systems
  8. 2026AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security