JevCoder: When Does Decision Delegation Pay Off in Code Repair?
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
A coding agent repeatedly makes decisions as it repairs code. These decisions shape repair trajectories and affect the token cost of successive model requests. We ask which repair decisions can be delegated to a low-cost probabilistic judge to reduce token use while maintaining repair quality. The judges answer questions with probabilities over predefined answers, without generating explanations. We evaluate judgment quality across repair stages and test how individual interventions change subsequent repair work. We compare these interventions with unguided repair, which uses no judge, and with the same workflow using the repair model itself as the judge. The benefit depends on the work an intervention changes, rather than the cost of the judgment alone. Based on these studies, we propose JevCoder, a workflow built around Jev, a low-cost probabilistic judge. It combines code-snippet recommendations with action-type guidance and uses judgments to determine when to submit. We evaluate three repair models, each on the same 400 tasks: 200 from SWE-bench Verified and 200 from SWE-Bench Pro Verified. Compared to unguided repair, JevCoder reduces mean repair-model token use by 21.6%-50.6%, with higher observed repair rates across all three models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 9 of its 27 references that are in the map. First posted on arXiv 14 Oct 2026.
Builds on
9 of 27 references.
Closest ancestors · 4
Same direction, newest first
- 2026SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents8 Sep
- 2026Steer, Don't Solve: Training Small Critic Models for Large Code Agents20 Jun
- 2025When Agents go Astray: Course-Correcting SWE Agents with PRMs2 Sep
- 2024Training Software Engineering Agents and Verifiers with SWE-Gym30 Dec
Other references · 5
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
- 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 279
- 2025CodeMonkeys: Scaling Test-Time Compute for Software Engineeringcited by 51
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Looping Is Not Reliability: State-Bound Evidence and Typed Revision Contracts for Agentic Code Repair
- 2025Understanding Automated Program Repair Agents Through the Lens of Traceability: An Empirical Study
- 2026Multi-Perspective Agentic Program Repair via Code Property Graphs and Temporal Execution Graphs
- 2026To Run or Not to Run: Analyzing the Cost-Effectiveness of Code Execution in LLM-Based Program Repair
- 2026From Verification Failures to Reusable Guidance for Coding Agents
- 2026From Approval to Execution: Reconstruction-Aware Repair Analysis for LLM-Agent Software
- 2026EviACT: An Evidence-to-Action Framework for Agentic Program Repair
- 2026ExecCritic: Learn to Test, Test to Improve for Coding Agents