Scaling Agentic Verifier for Competitive Coding
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Large language models (LLMs) have demonstrated strong coding capabilities but still struggle to solve competitive programming problems correctly in a single attempt. Execution-based re-ranking offers a promising test-time scaling strategy, yet existing methods are constrained by either difficult test case generation or inefficient random input sampling. To address this limitation, we propose Agentic Verifier, an execution-based agent that actively reasons about program behaviors and searches for highly discriminative test inputs that expose behavioral discrepancies among candidate solutions. Through multi-turn interaction with code execution environments, the verifier iteratively refines the candidate input generator and produces targeted counterexamples rather than blindly sampling inputs. We train the verifier to acquire this discriminative input generation capability via a scalable pipeline combining large-scale data synthesis, rejection fine-tuning, and agentic reinforcement learning. Extensive experiments across five competitive programming benchmarks demonstrate consistent improvements over strong execution-based baselines, achieving up to +10-15% absolute gains in Best@K accuracy. Further analysis reveals clear test-time scaling behavior and highlights the verifier's broader potential beyond reranking.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 30 of its 32 references that are in the map. First posted on arXiv 4 Feb 2026.
Builds on
30 of 32 references.
Closest ancestors · 1
Same direction, newest first
Other references · 29
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen3 Technical Reportcited by 8,195
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2023Let's Verify Step by Stepcited by 3,598
- 2021Program Synthesis with Large Language Modelscited by 3,509
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
- 2022Competition-Level Code Generation with AlphaCodecited by 1,427
17 more
- 2022CodeT: Code Generation with Generated Testscited by 369
- 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
- 2022Natural Language to Code Translation with Executioncited by 122
- 2022Coder Reviewer Reranking for Code Generationcited by 83
- 2024Can Language Models Solve Olympiad Programming?cited by 70
- 2025LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?cited by 52
- 2024The Limits of Inference Scaling Through Resamplingcited by 51
- 2023Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistencycited by 46
- 2023Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructionscited by 46
- 2024Language Models for Code Completion: A Practical Evaluationcited by 38
- 2022Fault-Aware Neural Code Rankerscited by 33
- 2025Dynamic Scaling of Unit Tests for Code Reward Modelingcited by 31
- 2025OJBench: A Competition Level Code Benchmark For Large Language Modelscited by 22
- 2025ICPC-Eval: Probing the Frontiers of LLM Reasoning with Competitive Programming Contestscited by 12
- 2025PlotCraft: Pushing the Limits of LLMs for Complex and Interactive Data Visualizationcited by 11
- 2025Disproving Program Equivalence with LLMscited by 5
- 2026From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuningcited by 5
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Dockerless: Environment-Free Program Verifier for Coding Agentscited by 8
- 2026GrandCode: Achieving Grandmaster Level in Competitive Programming via Agentic Reinforcement Learningcited by 3
- 2026RefineRL: Advancing Competitive Programming with Self-Refinement Reinforcement Learningcited by 0
- 2026Primal Generation, Dual Judgment: Self-Training from Test-Time Scalingcited by 0
- 2026Can Terminal Agents Trust Their Own Verification? Diagnosing and Improving Self-Verificationcited by 0
- 2026Can Terminal Agents Trust Their Own Verification? Diagnosing and Improving Self-Verification30 Sep 2026
- 2026Dockerless: Environment-Free Program Verifier for Coding Agents26 Jun 2026
- 2026Primal Generation, Dual Judgment: Self-Training from Test-Time Scaling11 May 2026
- 2026GrandCode: Achieving Grandmaster Level in Competitive Programming via Agentic Reinforcement Learning3 Apr 2026
- 2026RefineRL: Advancing Competitive Programming with Self-Refinement Reinforcement Learning1 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026Scaling Test-Time Compute for Agentic Coding
- 2025DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversal
- 2026CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models
- 2026Solvita: Enhancing Large Language Models for Competitive Programming via Agentic Evolution
- 2026CodeMidas: Scaling Agentic Coding RL Environments from Code Itself
- 2025CodeContests+: High-Quality Test Case Generation for Competitive Programming
- 2025A Self-Improving Coding Agent
- 2025KAT-Coder Technical Report