mlsanityData to 7 Oct 2026

Operational Hallucination and Safety Drift in AI Agents

arXiv 2607.18366 ↗First posted 20 Jul 2026cs.AI cs.CL cs.CYcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Large language models (LLMs) serving as planners in tool-using autonomous agents introduce dynamic reliability risks in multi-turn execution. While single-turn safety mechanisms are relatively mature, extended interactions reveal structural vulnerabilities where initial alignment degrades over time. This paper empirically characterizes two observed failure modes across multiple state-of-the-art LLMs: Safety Drift, the gradual erosion of declared safety intent leading to constraint-violating actions (e.g., textual refusal followed by reconnaissance and unsafe execution), and Operational Hallucination, persistent repetitive tool calls indicative of flawed state perception (e.g., livelocks even in legitimate tasks). Through controlled multi-turn evaluation on high-stakes ethical dilemmas, malicious requests, and benign controls, we quantify these phenomena using declaration-action gap and livelock metrics, demonstrating their cross-model prevalence under direct execution protocols. Root-cause analysis attributes the instabilities to the decoupling of reasoning context from execution state in current agent loops. We propose an Action-Aware Supervision Layer - a lightweight, plug-and-play architectural blueprint incorporating intent-action consistency checks, runtime state tracking, and forced termination primitives. Post-hoc simulation on captured failure trajectories shows the layer can intercept observed violations without false positives on benign cases. This work advances agent reliability by shifting focus from linguistic safeguards to enforceable architectural mechanisms for responsible agentic AI.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 14 of its 24 references that are in the map. First posted on arXiv 20 Jul 2026.

Builds on

14 of 24 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026The Causal Impact of Tool Affordance on Safety Alignment in LLM Agents19 Mar

Other references · 13

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  8. 2016Concrete Problems in AI Safetycited by 1,582
  9. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  10. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
  11. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  12. 2023Eight Things to Know about Large Language Modelscited by 73
1 more
  1. 2026The A-R Behavioral Space: Execution-Level Profiling of Tool-Using Language Model Agents in Organizational Deploymentcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents
  2. 2026Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety
  3. 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents
  4. 2026AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents
  5. 2026A GHOST in Long-Horizon Agents: Governance Hazard from Overlooked Safety Constraints across Turns
  6. 2026PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safety
  7. 2025When Refusals Fail: Unstable Safety Mechanisms in Long-Context LLM Agents
  8. 2026BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents