# When Independent Sampling Outperforms Agentic Reasoning

arXiv: https://arxiv.org/abs/2605.08478 · HTML: https://mlsanity.com/paper/2605.08478/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 19 of its 23 references that are in the map.
- First posted on arXiv 8 May 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [AI Agents That Matter](https://mlsanity.com/paper/2407.01502/) (2024)
- [Towards a Science of Scaling Agent Systems](https://mlsanity.com/paper/2512.08296/) (2025)
- [Theory of Mind for Multi-Agent Collaboration via Large Language Models](https://mlsanity.com/paper/2310.10701/) (2023)
- [CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratings](https://mlsanity.com/paper/2501.01257/) (2025)
- [Improving Multi-Agent Debate with Sparse Communication Topology](https://mlsanity.com/paper/2406.11776/) (2024)
- [Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?](https://mlsanity.com/paper/2508.17536/) (2025)
- [Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies](https://mlsanity.com/paper/2406.06461/) (2024)
- [Cost-of-Pass: An Economic Framework for Evaluating Language Models](https://mlsanity.com/paper/2504.13359/) (2025)
- [ELABORATION: A Comprehensive Benchmark on Human-LLM Competitive Programming](https://mlsanity.com/paper/2505.16667/) (2025)
- [ICPC-Eval: Probing the Frontiers of LLM Reasoning with Competitive Programming Contests](https://mlsanity.com/paper/2506.04894/) (2025)
- [Humanity's Last Code Exam: Can Advanced LLMs Conquer Human's Hardest Code Competition?](https://mlsanity.com/paper/2506.12713/) (2025)
- [Evaluating and Improving Large Language Models for Competitive Program Generation](https://mlsanity.com/paper/2506.22954/) (2025)

## Abstract (arXiv)

We study how to allocate inference-time compute for competitive programming under fixed budgets. Evaluating 216 Codeforces problems across Divisions 1-3, we compare agent-based reasoning with repeated independent sampling (k-shot) as a function of both cost and number of model calls. Across models and difficulty levels, k-shot consistently achieves a better accuracy-cost and accuracy-query tradeoff. This gap persists despite prompt caching in agent frameworks, indicating lower per-call effectiveness. Our results show that, for self-contained algorithmic tasks, independent exploration can outperform deeper agentic reasoning under realistic resource constraints. We also provide a budget-allocation analysis when the inference budget is fixed, and prove that a cost-optimal solver minimizes the principled metric log failure likelihood per dollar.