# From Plan to Action: How Well Do Agents Follow the Plan?

arXiv: https://arxiv.org/abs/2604.12147 · HTML: https://mlsanity.com/paper/2604.12147/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 13 Apr 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 16 of 16)

- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024)
- [SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?](https://mlsanity.com/paper/2509.16941/) (2025)
- [Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks](https://mlsanity.com/paper/2503.09572/) (2025)
- [The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason](https://mlsanity.com/paper/2506.12286/) (2025)
- [TDD-Bench Verified: Can LLMs Generate Tests for Issues Before They Get Resolved?](https://mlsanity.com/paper/2412.02883/) (2024)
- [Planning with Multi-Constraints via Collaborative Language Agents](https://mlsanity.com/paper/2405.16510/) (2024)
- [Process-Centric Analysis of Agentic Software Systems](https://mlsanity.com/paper/2512.02393/) (2025)
- [MONA: Myopic Optimization with Non-myopic Approval Can Mitigate Multi-step Reward Hacking](https://mlsanity.com/paper/2501.13011/) (2025)
- [A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Tasks](https://mlsanity.com/paper/2510.05608/) (2025)
- [Does SWE-Bench-Verified Test Agent Ability or Model Memory?](https://mlsanity.com/paper/2512.10218/) (2025)
- [Self-Abstraction from Grounded Experience for Plan-Guided Policy Refinement](https://mlsanity.com/paper/2511.05931/) (2025)
- [What Is Your Agent's GPA? A Framework for Evaluating Agent Goal-Plan-Action Alignment](https://mlsanity.com/paper/2510.08847/) (2025)
- [Can Old Tests Do New Tricks for Resolving SWE Issues?](https://mlsanity.com/paper/2510.18270/) (2025)
- [Investigating Test Overfitting on SWE-bench](https://mlsanity.com/paper/2511.16858/) (2025)
- [ReCodeAgent: A Multi-agent Workflow for Language-Agnostic Translation and Validation of Large-Scale Repositories](https://mlsanity.com/paper/2604.07341/) (2026)

## Built on by (most cited first, 4 of 4)

- [ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL](https://mlsanity.com/paper/2608.28476/) (2026)
- [OpenCollab: A Multi-Agent Coding Framework with Programmable Collaboration and Controllable Runtime](https://mlsanity.com/paper/2609.38345/) (2026)
- [Natural-Language Workflows Are Not Software Yet: Artifact-Driven Compilation for Reliable Agent Execution](https://mlsanity.com/paper/2608.21341/) (2026)
- [The Default Trap: Rethinking Plan Evaluation in Tool-Using LLM Agents](https://mlsanity.com/paper/2609.36829/) (2026)

## Abstract (arXiv)

Agents are commonly instructed to follow a task-specific plan for guidance. However, it is unknown to what extent agents actually follow instructed plans. Without such an analysis, determining the extent agents comply with a given plan, it is impossible to assess whether a solution was reached through correct strategic reasoning or through other means, e.g., data contamination or overfitting to a benchmark. This paper presents the first extensive, systematic analysis of plan compliance in programming agents, examining 21,120 trajectories from SWE-agent across four LLMs on SWE-bench Verified and SWE-bench Pro under eight plan variations. Without an explicit plan, agents fall back on internalized workflows during training, which are often incomplete, overfit, or inconsistently applied. Providing the standard plan improves issue resolution, and we observe that periodic plan reminders can mitigate plan violations and improve task success. A subpar plan hurts performance even more than no plan at all. Surprisingly, inserting additional task-relevant phases in the early stage can degrade performance, particularly when these phases do not align with the model's internal problem-solving strategy. These findings call for fine-tuning paradigms that teach models to follow instructed plans, rather than encoding task-specific plans in them, so that they reason and act adaptively, rather than memorizing workflows.