mlsanityData to 7 Oct 2026

Where Do CoT Training Gains Land in LLM based Agents?

arXiv 2606.26935 ↗First posted 25 Jun 2026cs.AIcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Chain-of-thought (CoT) reasoning is widely used in language-model agents, but prior work has shown that verbalized CoT is not always faithful and may instead reflect post-hoc reasoning, which means the model already knows the answer before reasoning. We therefore ask what CoT training is actually improving: is the model getting better at changing its action through generated reasoning, or is it getting better at predicting the action directly from the prompt? We study this question by comparing prompt actions (predicting action without CoT) with CoT actions (predicting action with CoT). Across checkpoints, prompt-action quality improves substantially. While interacting with the environment, the relative advantage of CoT actions over prompt actions remains similar, showing that CoT training does not widen the advantage of CoT reasoning, and it helps to improve the quality of prompt actions. We further find that later checkpoints are less likely to revise the action in response to CoT, suggesting greater reliance on the prompt. Motivated by these patterns, we selectively mask action-token supervision on a fraction of training examples. This intervention improves out-of-domain generalization.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 11 of its 25 references that are in the map. First posted on arXiv 25 Jun 2026.

Builds on

11 of 25 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting7 May

Other references · 10

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  3. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  4. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
  5. 2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challengescited by 278
  6. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learningcited by 210
  7. 2025RLVMR: Reinforcement Learning with Verifiable Meta-Reasoning Rewards for Robust Long-Horizon Agentscited by 56
  8. 2025MARFT: Multi-Agent Reinforcement Fine-Tuningcited by 51
  9. 2025RL Fine-Tuning Heals OOD Forgetting in SFTcited by 27
  10. 2025Gradient Coupling: The Hidden Barrier to Generalization in Agentic Reinforcement Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025The Challenge of Teaching Reasoning to LLMs Without RL or Distillation
  2. 2025CoT Vectors: Transferring and Probing the Reasoning Mechanisms of LLMs
  3. 2025When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs
  4. 2025Through the Valley: Path to Effective Long CoT Training for Small Language Models
  5. 2025Innate Reasoning is Not Enough: In-Context Learning Enhances Reasoning Large Language Models with Less Overthinking
  6. 2026Soft Guidance Starts to Outperform CoT Prompting as LLMs Improve
  7. 2026Efficient Reasoning Training Does Not Always Harm CoT Faithfulness and Monitorability
  8. 2025To Think or Not to Think: The Hidden Cost of Meta-Training with Excessive CoT Examples