# Scaling Test-Time Compute for Agentic Coding arXiv: https://arxiv.org/abs/2604.16529 ยท HTML: https://mlsanity.com/paper/2604.16529/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. - Builds on 18 of its 26 references that are in the map. - First posted on arXiv 16 Apr 2026. ## Placed in - LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/) - LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/) ## Builds on (most cited first, 18 of 18) - [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024) - [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026) - [A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?](https://mlsanity.com/paper/2503.24235/) (2025) - [Scalable Best-of-N Selection for Large Language Models via Self-Certainty](https://mlsanity.com/paper/2502.18581/) (2025) - [SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement](https://mlsanity.com/paper/2410.20285/) (2024) - [Scaling Test-time Compute for LLM Agents](https://mlsanity.com/paper/2506.12928/) (2025) - [CodeMonkeys: Scaling Test-Time Compute for Software Engineering](https://mlsanity.com/paper/2501.14723/) (2025) - [Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Management](https://mlsanity.com/paper/2510.06727/) (2025) - [The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management](https://mlsanity.com/paper/2508.21433/) (2025) - [Reducing Cost of LLM Agents with Trajectory Reduction](https://mlsanity.com/paper/2509.23586/) (2025) - [The Majority is not always right: RL training for solution aggregation](https://mlsanity.com/paper/2509.06870/) (2025) - [Towards Autonomous Mathematics Research](https://mlsanity.com/paper/2602.10177/) (2026) - [Recursive Self-Aggregation Unlocks Deep Thinking in Large Language Models](https://mlsanity.com/paper/2509.26626/) (2025) - [Benchmark Test-Time Scaling of General LLM Agents](https://mlsanity.com/paper/2602.18998/) (2026) - [Otter: Generating Tests from Issues to Validate SWE Patches](https://mlsanity.com/paper/2502.05368/) (2025) - [Rethinking Thinking Tokens: LLMs as Improvement Operators](https://mlsanity.com/paper/2510.01123/) (2025) - [SWE-Replay: Efficient Test-Time Scaling for Software Engineering Agents](https://mlsanity.com/paper/2601.22129/) (2026) - [Test-time Recursive Thinking: Self-Improvement without External Feedback](https://mlsanity.com/paper/2602.03094/) (2026) ## Built on by (most cited first, 15 of 15) - [Shepherd: Enabling Programmable Meta-Agents via Reversible Agentic Execution Traces](https://mlsanity.com/paper/2605.10913/) (2026) - [Rethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple Rounds](https://mlsanity.com/paper/2608.02636/) (2026) - [From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI](https://mlsanity.com/paper/2606.14502/) (2026) - [Compute Aligned Training: Optimizing for Test Time Inference](https://mlsanity.com/paper/2604.24957/) (2026) - [Extrapolative Weight Averaging Reveals Correctness-Efficiency Frontiers in Code RL](https://mlsanity.com/paper/2605.28751/) (2026) - [Scaling Laws for Agent Harnesses via Effective Feedback Compute](https://mlsanity.com/paper/2605.29682/) (2026) - [Pause and Reflect: Conformal Aggregation for Chain-of-Thought Reasoning](https://mlsanity.com/paper/2605.14098/) (2026) - [Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents](https://mlsanity.com/paper/2609.39982/) (2026) - [Trajectories That Segment Themselves: Agent-Declared Boundaries as a Training Unit](https://mlsanity.com/paper/2608.02302/) (2026) - [Consilience for Verifier-Free Test-Time Scaling](https://mlsanity.com/paper/2608.09898/) (2026) - [Disagree to Explore, Agree to Commit: Routing-Guided Test-Time Scaling for Software Agents](https://mlsanity.com/paper/2608.22191/) (2026) - [CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents](https://mlsanity.com/paper/2609.26779/) (2026) - [Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agents](https://mlsanity.com/paper/2609.33875/) (2026) - [Opera: A Verbal Critic Framework for Long-horizon Coding Agents](https://mlsanity.com/paper/2609.33987/) (2026) - [GitSwarm: Decentralized Compounding Inference](https://mlsanity.com/paper/2610.04862/) (2026) ## Abstract (arXiv) Test-time scaling has become a powerful way to improve large language models. However, existing methods are best suited to short, bounded outputs that can be directly compared, ranked or refined. Long-horizon coding agents violate this premise: each attempt produces an extended trajectory of actions, observations, errors, and partial progress taken by the agent. In this setting, the main challenge is no longer generating more attempts, but representing prior experience in a form that can be effectively selected from and reused. We propose a test-time scaling framework for agentic coding based on compact representations of rollout trajectories. Our framework converts each rollout into a structured summary that preserves its salient hypotheses, progress, and failure modes while discarding low-signal trace details. This representation enables two complementary forms of inference-time scaling. For parallel scaling, we introduce Recursive Tournament Voting (RTV), which recursively narrows a population of rollout summaries through small-group comparisons. For sequential scaling, we adapt Parallel-Distill-Refine (PDR) to the agentic setting by conditioning new rollouts on summaries distilled from prior attempts. Our method consistently improves the performance of frontier coding agents across SWE-Bench Verified and Terminal-Bench v2.0. For example, by using our method Claude-4.5-Opus improves from 70.9% to 77.6% on SWE-Bench Verified (mini-SWE-agent) and 46.9% to 59.1% on Terminal-Bench v2.0 (Terminus 1). Our results suggest that test-time scaling for long-horizon agents is fundamentally a problem of representation, selection, and reuse.