mlsanityData to 14 Oct 2026

Divergent Paths: Learning an Outcome-Predictive Representation of LLM Agent Trajectories

arXiv 2610.19614 ↗First posted 14 Oct 2026cs.LG cs.AI cs.CLcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

In this work, we reframe agentic trajectories as paths through a shared embedding space of actions and environmental observations. We show that successful and failed rollouts of the same task separate in this space, with simple per-task discriminators predicting held-out rollout outcomes at up to 0.95 AUROC across three agent benchmarks and three policy models. To exploit this signal across tasks, we introduce Precog-RM, a transformer-based model that meta-learns a representation in which outcome prediction can be performed in a few-shot setting. We show that Precog-RM generalizes to unseen tasks and models, and gives confident predictions of trajectory success long before termination. Deployed as an early stopping controller for Group Relative Policy Optimization (GRPO) on Tau2-Bench, Precog-RM cuts training FLOPs by up to 47.3% while maintaining performance comparable to baseline GRPO.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 24 of its 33 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

24 of 33 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Selective Rollout: Mid-Trajectory Termination for Multi-Sample Agent RL7 May
  2. 2026Hierarchy-of-Groups Policy Optimization for Long-Horizon Agentic Tasks26 Feb
  3. 2025Group-in-Group Policy Optimization for LLM Agent Training16 May

Other references · 21

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  7. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  8. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  9. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  10. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  11. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
  12. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 238
9 more
  1. 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environmentscited by 231
  2. 2024Free Process Rewards without Process Labelscited by 165
  3. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rolloutscited by 113
  4. 2017Time Limits in Reinforcement Learningcited by 79
  5. 2025UltraHorizon: Benchmarking Agent Capabilities in Ultra Long-Horizon Scenarioscited by 26
  6. 2026LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signalscited by 24
  7. 2026TRACER: Trajectory Risk Aggregation for Critical Episodes in Agentic Reasoningcited by 11
  8. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learningcited by 9
  9. 2025ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function Callingcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Explicit Trajectory Diversity for RL-Based Post-Training of LLM Agents
  2. 2026T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning
  3. 2026Progress-conditioned Group Policy Optimization for Long-Horizon Agentic Tasks
  4. 2026Success Leaves Detours: Learning Executable Walkthroughs for Long-Horizon Agents
  5. 2026Matched-Context Group Policy Optimization for LLM Agent Training
  6. 2026Evaluating Agentic Capabilities Before Post-Training: An End-to-End Approach
  7. 2026Reinforcement Learning with Decomposed Subtasks
  8. 2025Graph-Enhanced Policy Optimization in LLM Agent Training