mlsanityData to 14 Oct 2026

Push Your Agent: Measuring and Enforcing Quantitative Goal Persistence in Long-Horizon LLM Agents

arXiv 2605.23574 ↗First posted 22 May 2026cs.LG cs.SEcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Long-horizon language agents can make many plausible local tool calls yet fail to persist until a requested count is actually complete. We study this gap as Quantitative Goal Persistence (QGP): whether an agent keeps working until an external verifier confirms enough distinct valid items. PushBench turns this into a benchmark for repository-artifact collection and verifier-backed work units, so repeated work, duplicate submissions, false completion, and progress drift are measured directly rather than hidden behind a final success flag. In matched controller comparisons, a state-tracking retrieval controller reaches 69-78% success while eliminating duplicate submissions, and a backlog-tracking work-unit controller reaches 25-50% success in settings where standard and completion-gated controllers complete no task instances. Black-box frontier-agent evaluations with Claude Code (Sonnet 4.6) and Codex CLI (gpt-5.4) solve many 50-artifact tasks but drop to 3 out of 9 successes per condition at 100 artifacts. The results show that quantitative goals stress a different reliability requirement from local task competence: agents must maintain verified progress and stop only when the requested work is complete.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 24 of its 38 references that are in the map. First posted on arXiv 22 May 2026.

Builds on

24 of 38 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun
  2. 2024Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environments22 Feb
  3. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents24 Jan
  4. 2023A Survey on Large Language Model based Autonomous Agents22 Aug

Other references · 20

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  8. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  9. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  10. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  11. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  12. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
8 more
  1. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  2. 2021Measuring Coding Challenge Competence With APPScited by 902
  3. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 744
  4. 2023RepoBench: Benchmarking Repository-Level Code Auto-Completion Systemscited by 353
  5. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 327
  6. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 238
  7. 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 158
  8. 2026Can AI Agents Answer Your Data Questions? A Benchmark for Data Agentscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026How Strongly Should Task State Influence an LLM Agent?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Unreliable Progress Bar: Can LLM Agents Reliably Report Task Progress Throughout Execution?
  2. 2026Staying on Task: Testing the Foundations of Long-Horizon Agent Reliability
  3. 2026Confident and Wrong: Silent Semantic Failures in Coding Agents
  4. 2026Interlock: From Local Correctness to Task Completion in Long-Horizon Agents
  5. 2026Long-Horizon-Terminal-Bench 2.0: Automating Diverse Task Construction and Measuring Sustained Agent Progress
  6. 2026How Fast Do Agents Rot? An Empirical Study of Long-Horizon Degradation in LLM Agents for Production Decision-Making
  7. 2026SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition
  8. 2026LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks