# The Unreliable Progress Bar: Can LLM Agents Reliably Report Task Progress Throughout Execution?

arXiv: https://arxiv.org/abs/2609.08589 · HTML: https://mlsanity.com/paper/2609.08589/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 34 of its 75 references that are in the map.
- First posted on arXiv 8 Sep 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 30 of 34)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents](https://mlsanity.com/paper/2407.18901/) (2024)
- [AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents](https://mlsanity.com/paper/2401.13178/) (2024)
- [ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities](https://mlsanity.com/paper/2408.04682/) (2024)
- [MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback](https://mlsanity.com/paper/2309.10691/) (2023)
- [MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs](https://mlsanity.com/paper/2501.17399/) (2025)
- [Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Following](https://mlsanity.com/paper/2410.15553/) (2024)
- [StateFlow: Enhancing LLM Task-Solving through State-Driven Workflows](https://mlsanity.com/paper/2403.11322/) (2024)
- [STeCa: Step-level Trajectory Calibration for LLM Agent Learning](https://mlsanity.com/paper/2502.14276/) (2025)
- [SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints](https://mlsanity.com/paper/2503.08669/) (2025)
- [Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation](https://mlsanity.com/paper/2603.03116/) (2026)
- [SLOT: Structuring the Output of Large Language Models](https://mlsanity.com/paper/2505.04016/) (2025)
- [OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding](https://mlsanity.com/paper/2601.10343/) (2026)
- [From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents](https://mlsanity.com/paper/2606.09863/) (2026)
- [Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments](https://mlsanity.com/paper/2505.17616/) (2025)
- [Toward Scalable Verifiable Reward: Proxy State-Based Evaluation for Multi-turn Tool-Calling LLM Agents](https://mlsanity.com/paper/2602.16246/) (2026)
- [Latent State Estimation Helps UI Agents to Reason](https://mlsanity.com/paper/2405.11120/) (2024)
- [One Battle After Another: Probing LLMs' Limits on Multi-Turn Instruction Following with a Benchmark Evolving Framework](https://mlsanity.com/paper/2511.03508/) (2025)
- [Aligning Agents via Planning: A Benchmark for Trajectory-Level Reward Modeling](https://mlsanity.com/paper/2604.08178/) (2026)
- [Done, But Not Sure: Disentangling World Completion from Self-Termination in Embodied Agents](https://mlsanity.com/paper/2605.08747/) (2026)
- [When Agents Do Not Stop: Uncovering Infinite Agentic Loops in LLM Agents](https://mlsanity.com/paper/2607.01641/) (2026)
- [PM-Bench: Evaluating Prospective Memory in LLM Agents](https://mlsanity.com/paper/2607.12385/) (2026)
- [Did You Forget What I Asked? Prospective Memory Failures in Large Language Models](https://mlsanity.com/paper/2603.23530/) (2026)
- [DeployBench: Benchmarking LLM Agents for Research Artifact Deployment](https://mlsanity.com/paper/2606.05238/) (2026)
- [Retrospective Progress-Aware Self-Refinement for LLM Agent Training](https://mlsanity.com/paper/2606.14302/) (2026)
- [Quantifying the Impact of Structured Output Format on Large Language Models through Causal Inference](https://mlsanity.com/paper/2509.21791/) (2025)

## Built on by (most cited first, 1 of 1)

- [Measuring and Reducing Unusable Replies of LLM Agents](https://mlsanity.com/paper/2610.19659/) (2026)

## Abstract (arXiv)

Recent large language models can emit task-progress signals that agent frameworks use to decide whether a task should continue or stop, yet whether a model can reliably report its task progress at every stage of a task, and where and how its reports fail, has not been studied systematically. We evaluate this ability on the public benchmark τ2-bench and on StageIF, a controlled testbed in which reporting checkpoints are placed across the task's lifecycle. Both settings require reports at multiple task stages. We find that reporting reliability depends on the stage a task has reached, and that almost every deployed model we test is reliable at some stages and unreliable at others. Where reporting breaks down is not the same everywhere. Most deployed models lose accuracy once work is under way and recover once the task is done. The newest generation closes that mid-task drop and instead grows conservative at the finish line. Our study exposes a capability gap in task-progress reporting and provides an evaluation protocol that spans the whole course of task execution for this ability on which agent operation depends. The findings indicate that agent frameworks should not control task flow on the strength of the model's state reports alone.