mlsanityData to 7 Oct 2026

Agent Safety Is Action Alignment

arXiv 2606.28739 ↗First posted 27 Jun 2026cs.AIcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Large language models increasingly act as agents: they call tools, move money, delete records, and send messages on a user's behalf. To keep them safe, practitioners imported the chatbot-era recipe (train the model to refuse unsafe inputs) into the agentic setting, and treat the resulting capability loss as a manageable ``alignment tax.'' We argue this is a category error. Refusal is a primitive for content safety, where the harm is in the model's output and is therefore a learnable function of it. Agentic harm is different in kind: it lies not in any output but in the relation between the authority an action exercises and the authority the user granted, which is absent from the text the model sees. Importing content-safety methods into this regime does not trade capability for safety; it pays capability and buys negative security. We support this with three lines of evidence spanning the autonomy spectrum: defense-trained models learn surface patterns rather than intent; the same training collapses multi-step agents before any threat appears while leaving them exploitable; and even undefended frontier models exceed granted authority under ordinary use. We conclude that action safety cannot be installed in weights. It must be expressed as least privilege, enforced outside the model at the action boundary, and evaluated as action alignment (a relational, deployment-conditioned property) rather than a refusal score.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 21 of its 36 references that are in the map. First posted on arXiv 27 Jun 2026.

Builds on

21 of 36 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework17 Jan

Other references · 20

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  3. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  5. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  6. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
  7. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 292
  8. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
  9. 2025CoAct-1: Computer-using Multi-Agent System with Coding Actionscited by 33
  10. 2024AD-LLM: Benchmarking Large Language Models for Anomaly Detectioncited by 25
  11. 2025Secure On-Device Video OOD Detection Without Backpropagationcited by 19
  12. 2024PyOD 2: A Python Library for Outlier Detection with LLM-powered Model Selectioncited by 17
8 more
  1. 2026Agent Banana: High-Fidelity Image Editing with Agentic Thinking and Toolingcited by 17
  2. 2025A Personalized Conversational Benchmark: Towards Simulating Personalized Conversationscited by 16
  3. 2025Don't Let It Hallucinate: Premise Verification via Retrieval-Augmented Logical Reasoningcited by 14
  4. 2025Mitigating Hallucinations in Large Language Models via Causal Reasoningcited by 13
  5. 2026FORTIS: Benchmarking Over-Privilege in Agent Skillscited by 11
  6. 2024CMOOD: Concept-based Multi-label OOD Detectioncited by 6
  7. 2025FormFactory: An Interactive Benchmarking Suite for Multimodal Form-Filling Agentscited by 6
  8. 2025M3OOD: Automatic Selection of Multimodal OOD Detectorscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Agent Safety From Within: Detecting Harmful Trajectories from LLM Internal States
  2. 2026Mind the GAP: Text Safety Does Not Transfer to Tool-Call Safety in LLM Agents
  3. 2026The Causal Impact of Tool Affordance on Safety Alignment in LLM Agents
  4. 2026Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Use
  5. 2025AgentAlign: Navigating Safety Alignment in the Shift from Informative to Agentic Large Language Models
  6. 2025Agent Safety Alignment via Reinforcement Learning
  7. 2026Delegated Misalignment: How Multi-Agent Structures Amplify LLM Safety Risks
  8. 2026AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents