mlsanityData to 7 Oct 2026

Scaling Laws for Agent Harnesses via Effective Feedback Compute

arXiv 2605.29682 ↗First posted 28 May 2026cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Agent harnesses shape language-model performance by controlling tool use, feedback, verification, memory, and repair. Yet raw test-time expenditure, such as tokens, tool calls, wall time, or cost, cannot distinguish useful feedback from redundant or unstable interaction. We introduce Effective Feedback Compute (EFC), a trace-level scaling coordinate for informative, valid, non-redundant, and retained feedback. We further define Estimated-EFC, NRS-EFC, harness efficiency η, and task-demand normalization for realistic traces and heterogeneous tasks. Across synthetic, real, held-out, and prospective evaluations, EFC-based coordinates outperform raw-compute baselines and SAS. Oracle-EFC/Dtask reaches R2=0.99 in controlled scaling, and NRS-EFC/Dtask reaches R2=0.93 on real traces where raw compute has near-zero or negative fit. Finally, ours uses EFC as a companion control layer for existing harnesses, improving mean pass rate from 61.2% to 68.2% while reducing mean raw cost from 213.8 to 85.1 under matched settings. These results suggest that harness scaling depends on durable, task-sufficient feedback rather than raw computation alone.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 31 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

24 of 31 references.

References · 24

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  8. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  10. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  11. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  12. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
12 more
  1. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  2. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
  3. 2025Towards a Science of Scaling Agent Systemscited by 125
  4. 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnessescited by 119
  5. 2025Scaling Test-time Compute for LLM Agentscited by 59
  6. 2026Natural-Language Agent Harnessescited by 54
  7. 2025Budget-Aware Tool Use Enables Effective Agent Scalingcited by 42
  8. 2026Benchmark Test-Time Scaling of General LLM Agentscited by 24
  9. 2026ContextBench: A Benchmark for Context Retrieval in Coding Agentscited by 23
  10. 2026Scaling Test-Time Compute for Agentic Codingcited by 11
  11. 2026Agentic Test-Time Scaling for WebAgentscited by 7
  12. 2026The Scaling Laws of Skills in LLM Agent Systemscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.