A Sober Look at Agentic Misalignment in Automated Workflows
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
We study a class of emergent misalignment in multi-agent systems (MAS), with a focus on automated workflows, which we refer to agentic misalignment. Although these systems can solve complex tasks, they often fail because agents act according to implicit proxy utilities that do not align with the intended human goals. We formally define these behaviors and analyze them within a Bayesian framework, showing that generic utilities naturally lead to posterior collapse of agents in automated workflows. To address this issue, we propose Agentic Evidence Attribution (AEA), a novel alignment paradigm that improves agent posteriors using context-specific evidence. AEA reasons over agent actions and provides structured evidence to correct misaligned behavior during collaboration. To better understand the role of evidence, we study two instantiations of AEA: self-reflection (internal evidence from the model) and weak-to-strong generalization (external evidence on the agentic trajectory). We show that a small evidence model effectively aligns the MAS by providing orthogonal failure attribution. Our results clarify the sources of agentic misalignment in automated workflows and show that evidence-based alignment can effectively improve agent collaboration and leads to reliable multi-agent systems built on automated workflows.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 36 of its 45 references that are in the map. First posted on arXiv 22 May 2026.
Builds on
36 of 45 references.
Closest ancestors · 6
Same direction, newest first
- 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?3 Sep
- 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems30 Apr
- 2025Why Do Multi-Agent LLM Systems Fail?17 Mar
- 2024AFlow: Automating Agentic Workflow Generation14 Oct
- 2024AutoFlow: Automated Workflow Generation for Large Language Model Agents1 Jul
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug
Other references · 30
- 2023GPT-4 Technical Reportcited by 18,396
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2023GAIA: a benchmark for General AI Assistantscited by 1,128
- 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
- 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
- 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 499
- 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 262
- 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 256
- 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 195
18 more
- 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 177
- 2024RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Stylecited by 176
- 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 158
- 2025Agentic Misalignment: How LLMs Could Be Insider Threatscited by 158
- 2024Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learningcited by 155
- 2024EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instructioncited by 154
- 2025RM-R1: Reward Modeling as Reasoningcited by 154
- 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 143
- 2025Towards a Science of Scaling Agent Systemscited by 138
- 2022RL with KL penalties is better viewed as Bayesian inferencecited by 135
- 2024LLM Multi-Agent Systems: Challenges and Open Problemscited by 120
- 2025Persona Features Control Emergent Misalignmentcited by 86
- 2023ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agentcited by 81
- 2025Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?cited by 75
- 2019Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Surveycited by 47
- 2023Goodhart's Law in Reinforcement Learningcited by 41
- 2025Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewardscited by 24
- 2025Rectifying Shortcut Behaviors in Preference-based Reward Learningcited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Why Agentic Workflows Fail: A Taxonomy Derived from Literature and Empirical Data
- 2025DoubleAgents: Human-Agent Alignment in a Socially Embedded Workflow
- 2026The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution
- 2025TRAIL: Trace Reasoning and Agentic Issue Localization
- 2026Near-Miss: Latent Policy Failure Detection in Agentic Workflows
- 2026How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions
- 2025Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflows
- 2026Beyond Fluency: Toward Reliable Trajectories in Agentic IR