mlsanityData to 14 Oct 2026

Do Not Step Into the Same River Twice: Learning to Reason from Trial and Error

arXiv 2510.26109 ↗First posted 30 Oct 2025cs.LGcited by 3

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) has significantly boosted the reasoning capability of language models (LMs). However, existing RLVR approaches train LMs based on their own on-policy responses and are constrained by the initial capability of LMs, thus prone to exploration stagnation, in which LMs fail to solve more training problems and cannot further learn from the training data. Some approaches try to address this by leveraging off-policy solutions to training problems, but rely on external expert guidance that is limited in availability and scalability. In this work, we propose LTE (Learning to reason from Trial and Error), an approach that hints LMs with their previously self-made mistakes, not requiring any external expert guidance. Experiments validate the effectiveness of LTE, which outperforms the normal group relative policy optimization (GRPO) by 5.02 in Pass@1 and 9.96 in Pass@k on average across six mathematical reasoning benchmarks for Qwen3-8B-Base and even performs better than methods that require external guidance. Further analysis confirms that LTE successfully mitigates exploration stagnation and enhances both exploitation and exploration during training. Our code is available at https://github.com/JamyDon/LTE.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 36 references that are in the map. First posted on arXiv 30 Oct 2025.

Builds on

29 of 36 references.

Closest ancestors · 18

Same direction, newest first

  1. 2025LaSeR: Reinforcement Learning with Last-Token Self-Rewarding16 Oct
  2. 2025BroRL: Scaling Reinforcement Learning via Broadened Exploration1 Oct
  3. 2025Improving Sampling Efficiency in RLVR through Adaptive Rollout and Response Reuse30 Sep
  4. 2025Nudging the Boundaries of LLM Reasoning30 Sep
  5. 2025Outcome-based Exploration for LLM Reasoning8 Sep
  6. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models14 Aug
  7. 2025EvoCoT: Overcoming the Exploration Bottleneck in Reinforcement Learning11 Aug
  8. 2025A Simple "Try Again" Can Elicit Multi-Turn LLM Reasoning18 Jul
  9. 2025Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination14 Jul
  10. 2025StepHint: Multi-level Stepwise Hints Enhance Reinforcement Learning to Reason3 Jul
  11. 2025Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questions9 Jun
  12. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models28 May
6 more
  1. 2025Learning to Reason under Off-Policy Guidance21 Apr
  2. 2025Improving RL Exploration for LLM Reasoning through Retrospective Replay19 Apr
  3. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learning4 Apr
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  5. 2025Learning from Failures in Multi-Attempt Reinforcement Learning4 Mar
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan

Other references · 11

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  6. 2025Kimi K2: Open Agentic Intelligencecited by 567
  7. 2025Skywork Open Reasoner 1 Technical Reportcited by 165
  8. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 91
  9. 2025Towards a Unified View of Large Language Model Post-Trainingcited by 36
  10. 2025Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocationcited by 22
  11. 2025Why Does Reasoning Length Converge? Unveiling the Underfitting-Overfitting Trade-off in Chain-of-Thoughtcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026No More Stale Feedback: Co-Evolving Critics for Open-World Agent Learningcited by 8
  2. 2026Know What You Know: Metacognitive Entropy Calibration for Verifiable RL Reasoningcited by 3
  3. 2026Privy to the Foil: Recasting Value Estimation with a Self-Privileged Critic for RLVRcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Re2: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving
  2. 2025Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback
  3. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
  4. 2026GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings
  5. 2025Navigate the Unknown: Enhancing LLM Reasoning with Intrinsic Motivation Guided Exploration
  6. 2025Unlocking Reasoning Capabilities in LLMs via Reinforcement Learning Exploration
  7. 2026Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement
  8. 2026Rationale-Guided Policy Optimization: Learning to Reason with Adaptive Rationale Scaffolding