# When Agents go Astray: Course-Correcting SWE Agents with PRMs

arXiv: https://arxiv.org/abs/2509.02360 · HTML: https://mlsanity.com/paper/2509.02360/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 2 Sep 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 21 of 21)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents](https://mlsanity.com/paper/2403.02502/) (2024)
- [R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents](https://mlsanity.com/paper/2504.07164/) (2025)
- [SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement](https://mlsanity.com/paper/2410.20285/) (2024)
- [TRAIL: Trace Reasoning and Agentic Issue Localization](https://mlsanity.com/paper/2505.08638/) (2025)
- [SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents](https://mlsanity.com/paper/2504.08703/) (2025)
- [FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation](https://mlsanity.com/paper/2503.06680/) (2025)
- [SMART: Self-Aware Agent for Tool Overuse Mitigation](https://mlsanity.com/paper/2502.11435/) (2025)
- [STeCa: Step-level Trajectory Calibration for LLM Agent Learning](https://mlsanity.com/paper/2502.14276/) (2025)
- [RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code](https://mlsanity.com/paper/2503.07832/) (2025)
- [Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute](https://mlsanity.com/paper/2503.23803/) (2025)
- [BacktrackAgent: Enhancing GUI Agent with Error Detection and Backtracking Mechanism](https://mlsanity.com/paper/2505.20660/) (2025)
- [Doc2Feat-Bench: Evaluating Documentation-Driven Feature Addition](https://mlsanity.com/paper/2507.18130/) (2025)
- [VeriLA: A Human-Centered Evaluation Framework for Interpretable Verification of LLM Agent Failures](https://mlsanity.com/paper/2503.12651/) (2025)
- [FreePRM: Training Process Reward Models Without Ground Truth Process Labels](https://mlsanity.com/paper/2506.03570/) (2025)
- [Guided Search Strategies in Non-Serializable Environments with Applications to Software Engineering Agents](https://mlsanity.com/paper/2505.13652/) (2025)

## Built on by (most cited first, 28 of 28)

- [A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Models](https://mlsanity.com/paper/2510.08049/) (2025)
- [BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills](https://mlsanity.com/paper/2510.19898/) (2025)
- [Wink: Recovering from Misbehaviors in Coding Agents](https://mlsanity.com/paper/2602.17037/) (2026)
- [Counterfactual Trace Auditing of LLM Agent Skills](https://mlsanity.com/paper/2605.11946/) (2026)
- [GUI-PRA: Process Reward Agent for GUI Tasks](https://mlsanity.com/paper/2509.23263/) (2025)
- [Enhancing Large Language Model Reasoning with Reward Models: An Analytical Survey](https://mlsanity.com/paper/2510.01925/) (2025)
- [Agentic Software Issue Resolution with Large Language Models: A Survey](https://mlsanity.com/paper/2512.22256/) (2025)
- [Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis](https://mlsanity.com/paper/2604.24198/) (2026)
- [Steer, Don't Solve: Training Small Critic Models for Large Code Agents](https://mlsanity.com/paper/2606.21811/) (2026)
- [What Resolve Rate Hides: Trajectory Structure Diagnostics for Coding Agents](https://mlsanity.com/paper/2607.06184/) (2026)
- [Fail-Fast, Restart-Smart: Early Failure Prediction and Restart for SWE Agentic Tasks](https://mlsanity.com/paper/2608.03222/) (2026)
- [Think Locally, Explain Globally: Graph-Guided LLM Investigations via Local Reasoning and Belief Propagation](https://mlsanity.com/paper/2601.17915/) (2026)
- [Independent Patch Verification for Coding Agents with a Bidirectional Reconstruct-and-Verify Framework](https://mlsanity.com/paper/2608.08950/) (2026)
- [EGSS: Entropy-guided Stepwise Scaling for Reliable Software Engineering](https://mlsanity.com/paper/2602.05242/) (2026)
- [CORVUS: Context Optimization and Reduction Via Underlying Synchronization for LLM Coding Agents](https://mlsanity.com/paper/2607.22711/) (2026)
- [Online Monitoring and Corrective Steering of Programming Agents](https://mlsanity.com/paper/2608.06701/) (2026)
- [DIG to Heal: Scaling General-purpose Agent Collaboration via Explainable Dynamic Decision Paths](https://mlsanity.com/paper/2603.00309/) (2026)
- [Reusing Past Repairs Through Hierarchical Trajectory Abstraction for Coding Agents](https://mlsanity.com/paper/2607.29658/) (2026)
- [Before the Rollout Ends: Early Terminal Reward Prediction for Long-horizon Coding Agents](https://mlsanity.com/paper/2609.31995/) (2026)
- [Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents](https://mlsanity.com/paper/2609.39982/) (2026)
- [Better Understanding, Better Fixes? A Study of Hallucination in LLM-based Automated Program Repair](https://mlsanity.com/paper/2609.04909/) (2026)
- [FLARE: A Full-Lifecycle Dense Supervision Paradigm for Long-Horizon Coding Agents via Generative Reward Model](https://mlsanity.com/paper/2609.23808/) (2026)
- [WatchPoint: Executable User Feedback for Real-World Agentic Web Development](https://mlsanity.com/paper/2609.26204/) (2026)
- [Opera: A Verbal Critic Framework for Long-horizon Coding Agents](https://mlsanity.com/paper/2609.33987/) (2026)
- [Learning When and How to Intervene: A Hindsight-Distilled Sentinel for Coding Agents](https://mlsanity.com/paper/2609.39957/) (2026)
- [What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agents](https://mlsanity.com/paper/2609.39989/) (2026)
- [SWE-Critix: Post-Training Generalist Verifiers for Assessing Coding-Agent Trajectories](https://mlsanity.com/paper/2610.13356/) (2026)
- [JevCoder: When Does Decision Delegation Pay Off in Code Repair?](https://mlsanity.com/paper/2610.19596/) (2026)

## Abstract (arXiv)

Large Language Model (LLM) agents are increasingly deployed for complex, multi-step software engineering (SWE) tasks. However, their trajectories often contain costly inefficiencies, such as redundant exploration, looping, and failure to terminate once a solution is reached. Prior work has largely treated these errors in a post-hoc manner, diagnosing failures only after execution. In this paper, we introduce SWE-PRM, an inference-time Process Reward Model (PRM) that intervenes during execution to detect and course-correct trajectory-level errors. Our PRM design leverages a taxonomy of common inefficiencies and delivers lightweight, interpretable feedback without modifying the underlying policy. On SWE-bench Verified, closed-source PRMs improve resolution from 40.0% to 50.6% (+10.6 p.p.), with the largest gains on medium and hard tasks. Among feedback strategies, taxonomy-guided PRMs outperform unguided or explicit action-prescriptive variants, increasing success rate while reducing trajectory length. These benefits come at an acceptable added inference cost of as low as 0.2, making PRMs a practical and scalable mechanism for improving SWE agents' reliability and efficiency.