LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Large language model (LLM) agents increasingly undertake long-horizon tasks that require sustained reasoning, tool use, and revision across many interdependent steps. However, existing agent harnesses maintain task execution, task state, and completion assessment within a growing context, making the state difficult to track and allowing incorrect self-assessments to propagate into later decisions. We reformulate long-horizon execution as a task-state management problem and propose LongHorizon-Harness, which maintains the task state explicitly outside execution and updates it only with facts independently verified from the environment. Its Manage-Execute-Audit(MEA) loop uses a manager to maintain the task state and determine the next subtask, a fresh-context executor to perform it, and a read-only auditor to verify the resulting environment state before the next round. A lightweight AgentAdapter supports interchangeable model and harness backends without modifying their native agent loops. LongHorizon-Harness improves Qwen~3.7-Plus from 51.8% to 80.7% on WeaveBench, from 69.7% to 77.2% on Terminal-Bench~2.1, and from 2.8% to 8.3% on OSWorld~2.0. It also raises Claude Opus~4.7 from 20.0% to 34.3% on an OSWorld2.0 subset, demonstrating consistent gains across models, harnesses, and interaction domains.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 20 of its 30 references that are in the map. First posted on arXiv 3 Aug 2026.
Builds on
20 of 30 references.
References · 20
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
- 2024Agent S: An Open Agentic Framework that Uses Computers Like a Humancited by 182
- 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoningcited by 147
- 2025Measuring AI Ability to Complete Long Software Taskscited by 142
- 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolvercited by 76
- 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
- 2025Tree Search for LLM Agent Reinforcement Learningcited by 73
8 more
- 2025NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agentscited by 56
- 2026ProgramBench: Can Language Models Rebuild Programs From Scratch?cited by 47
- 2025A Comprehensive Survey of Agents for Computer Use: Foundations, Challenges, and Future Directionscited by 33
- 2026Code2World: A GUI World Model via Renderable Code Generationcited by 26
- 2025AdaCuRL: Adaptive Curriculum Reinforcement Learning with Invalid Sample Mitigation and Historical Revisitingcited by 17
- 2026CoEvolve: Training LLM Agents via Agent-Data Mutual Evolutioncited by 14
- 2026WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfacescited by 9
- 2026SaaSBench: Exploring the Boundaries of Coding Agents in Long-Horizon Enterprise SaaS Engineeringcited by 3
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026Harness-Zero: Harness Distillation via Agent-as-Harnesscited by 3
- 2026When May an Agent Stop? Evidence-Carrying Termination for Tool-Using LLMscited by 2
- 2026Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligencecited by 1
- 2026LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineeringcited by 1
- 2026Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failurescited by 0
- 2026Beyond Skill Evolution: Self-Evolving Context Management Policies for Long-Horizon Agent Harnessescited by 0
- 2026Turbo Harness: Instance-Adaptive Harness Optimizationcited by 0
- 2026When Harnesses Lose the Signal: Causal Evaluation of Recovery in LLM Agentscited by 0
- 2026RunningTab: Direct Workspace Interaction with Environment-Side Tabscited by 0
- 2026RunningTab: Direct Workspace Interaction with Environment-Side Tabs7 Oct 2026
- 2026When Harnesses Lose the Signal: Causal Evaluation of Recovery in LLM Agents30 Sep 2026
- 2026Turbo Harness: Instance-Adaptive Harness Optimization30 Sep 2026
- 2026Beyond Skill Evolution: Self-Evolving Context Management Policies for Long-Horizon Agent Harnesses28 Sep 2026
- 2026Harness-Zero: Harness Distillation via Agent-as-Harness21 Sep 2026
- 2026Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures11 Sep 2026
- 2026LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering28 Aug 2026
- 2026When May an Agent Stop? Evidence-Carrying Termination for Tool-Using LLMs22 Aug 2026
- 2026Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence21 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026Harnesses for Inference-Time Alignment over Execution Trajectories
- 2026Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents
- 2026Composing Task-specific Agent Harnesses at Test Time with Reusable Primitives
- 2026SKILL.state: Scalable Long-Horizon Agent Skills
- 2026Vestrum: Improving Agent Harnesses by Adapting Their Verification, Structure and Memory
- 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Design
- 2026StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structure
- 2026OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents