mlsanityData to 7 Oct 2026

Before the Rollout Ends: Early Terminal Reward Prediction for Long-horizon Coding Agents

arXiv 2609.31995 ↗First posted 25 Sep 2026cs.CLcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Long-horizon coding agents receive verifiable rewards only after completing expensive sequences of tool calls. This increases inference cost, amplifies early wrong hypotheses, and can lead to sparse terminal reward and unstable training. We introduce Contextual Early Reward (CER), which predicts terminal reward through behavioral evidence in a trajectory prefix. CER synthesizes adaptive rubrics specific to the current task and stage through experiences summarized from related historical tasks. In test-time scaling on SWE-bench Verified, CER improves RM@8 over the strongest baseline by 4.2 percentage points (pp) on Nemotron 3 Ultra and 2.0 pp on Qwen 3.6 27B; on Nemotron, it takes only 15.3% tokens to match the best baseline performance. In RL training experiments, CER exceeds full-rollout TMax by 1.9 pp while using 52.7% fewer online policy-and-judge tokens. Together, CER provides an interpretable, efficient, and dense evaluation method for long-horizon coding agents.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 24 of its 32 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

24 of 32 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization18 May
  2. 2026PivotRL: High Accuracy Agentic Post-Training at Low Compute Cost22 Mar
  3. 2025Simulating Environments with Reasoning Models for Agent Training3 Nov
  4. 2025Process Reward Models for LLM Agents: Practical Framework and Directions14 Feb

Other references · 20

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  2. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  3. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  4. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
  5. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 252
  6. 2025Are "Solved Issues" in SWE-bench Really Solved Correctly? An Empirical Studycited by 47
  7. 2025UTBoost: Rigorous Evaluation of Coding Agents on SWE-Benchcited by 44
  8. 2026Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 32
  9. 2025When Agents go Astray: Course-Correcting SWE Agents with PRMscited by 25
  10. 2025SWE-RM: Execution-free Feedback For Software Engineering Agentscited by 24
  11. 2026Tmax: A simple recipe for terminal agentscited by 24
  12. 2026Agentic Rubrics as Contextual Verifiers for SWE Agentscited by 22
8 more
  1. 2026Agentic Abstention: Do Agents Know When to Stop Instead of Act?cited by 9
  2. 2026AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systemscited by 8
  3. 2026The Verification Horizon: No Silver Bullet for Coding Agent Rewardscited by 6
  4. 2026AgentStop: Terminating Local AI Agents Early to Save Energy in Consumer Devicescited by 5
  5. 2026A Rubric-Supervised Critic from Sparse Real-World Outcomescited by 4
  6. 2026AMARIS: A Memory-Augmented Rubric Improvement System for Rubric-Based Reinforcement Learningcited by 3
  7. 2026ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agentscited by 2
  8. 2026From Correctness to Utility: Gain-Based Prefix Evaluation for LLM Reasoningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models
  2. 2026T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks
  3. 2026Rewarding Beliefs, Not Actions: Consistency-Guided Credit Assignment for Long-Horizon Agents
  4. 2026Before They Can Solve: Predicting Post-Training Coding-Agent Performance from Base Models
  5. 2026VICT: Verifier-Instrumented Credit Tracing for Long-Horizon LLM Agent Reinforcement Learning
  6. 2026Terminus-4B: Can a Smaller Model Replace Frontier LLMs at Agentic Execution Tasks?
  7. 2026Cross-Benchmark Transfer from RL on Agentic Coding Tasks
  8. 2026The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Works