# Before the Rollout Ends: Early Terminal Reward Prediction for Long-horizon Coding Agents

arXiv: https://arxiv.org/abs/2609.31995 · HTML: https://mlsanity.com/paper/2609.31995/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use.
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 25 Sep 2026.

## Placed in

- LLM agents and tool use / [Reinforcement learning for LLM agents](https://mlsanity.com/agents/reinforcement-learning-for-llm-agents/)

## Builds on (most cited first, 24 of 24)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Generative Verifiers: Reward Modeling as Next-Token Prediction](https://mlsanity.com/paper/2408.15240/) (2024)
- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning](https://mlsanity.com/paper/2602.08234/) (2026)
- [Process Reward Models for LLM Agents: Practical Framework and Directions](https://mlsanity.com/paper/2502.10325/) (2025)
- [Simulating Environments with Reasoning Models for Agent Training](https://mlsanity.com/paper/2511.01824/) (2025)
- [Are "Solved Issues" in SWE-bench Really Solved Correctly? An Empirical Study](https://mlsanity.com/paper/2503.15223/) (2025)
- [UTBoost: Rigorous Evaluation of Coding Agents on SWE-Bench](https://mlsanity.com/paper/2506.09289/) (2025)
- [Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning](https://mlsanity.com/paper/2606.15007/) (2026)
- [Tmax: A simple recipe for terminal agents](https://mlsanity.com/paper/2606.23321/) (2026)
- [When Agents go Astray: Course-Correcting SWE Agents with PRMs](https://mlsanity.com/paper/2509.02360/) (2025)
- [SWE-RM: Execution-free Feedback For Software Engineering Agents](https://mlsanity.com/paper/2512.21919/) (2025)
- [Agentic Rubrics as Contextual Verifiers for SWE Agents](https://mlsanity.com/paper/2601.04171/) (2026)
- [Agentic Abstention: Do Agents Know When to Stop Instead of Act?](https://mlsanity.com/paper/2606.28733/) (2026)
- [PivotRL: High Accuracy Agentic Post-Training at Low Compute Cost](https://mlsanity.com/paper/2603.21383/) (2026)
- [AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems](https://mlsanity.com/paper/2605.08715/) (2026)
- [The Verification Horizon: No Silver Bullet for Coding Agent Rewards](https://mlsanity.com/paper/2606.26300/) (2026)
- [AgentStop: Terminating Local AI Agents Early to Save Energy in Consumer Devices](https://mlsanity.com/paper/2605.15206/) (2026)
- [A Rubric-Supervised Critic from Sparse Real-World Outcomes](https://mlsanity.com/paper/2603.03800/) (2026)
- [PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization](https://mlsanity.com/paper/2605.17877/) (2026)
- [AMARIS: A Memory-Augmented Rubric Improvement System for Rubric-Based Reinforcement Learning](https://mlsanity.com/paper/2605.18592/) (2026)
- [ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agents](https://mlsanity.com/paper/2606.03239/) (2026)
- [From Correctness to Utility: Gain-Based Prefix Evaluation for LLM Reasoning](https://mlsanity.com/paper/2606.07190/) (2026)

## Built on by (most cited first, 1 of 1)

- [Long-Horizon-Terminal-Bench 2.0: Automating Diverse Task Construction and Measuring Sustained Agent Progress](https://mlsanity.com/paper/2610.14146/) (2026)

## Abstract (arXiv)

Long-horizon coding agents receive verifiable rewards only after completing expensive sequences of tool calls. This increases inference cost, amplifies early wrong hypotheses, and can lead to sparse terminal reward and unstable training. We introduce Contextual Early Reward (CER), which predicts terminal reward through behavioral evidence in a trajectory prefix. CER synthesizes adaptive rubrics specific to the current task and stage through experiences summarized from related historical tasks. In test-time scaling on SWE-bench Verified, CER improves RM@8 over the strongest baseline by 4.2 percentage points (pp) on Nemotron 3 Ultra and 2.0 pp on Qwen 3.6 27B; on Nemotron, it takes only 15.3% tokens to match the best baseline performance. In RL training experiments, CER exceeds full-rollout TMax by 1.9 pp while using 52.7% fewer online policy-and-judge tokens. Together, CER provides an interpretable, efficient, and dense evaluation method for long-horizon coding agents.