Learning to Plan by Looking Back: Hindsight Hierarchies for Training Reasoning Models
On the map: LLM reasoning → Prompting and chain-of-thought reasoning also fits: Test-time compute and efficient thinking
Abstract (arXiv)
We introduce a self-improvement loop for reasoning models based on the following observation: Even when the difficulty of a problem exceeds the model's current solving abilities, an additionally supplied solution might enable the model to extract useful solution ideas in hindsight. We operationalize this by jointly training the same model to exhibit the following three capabilities: predicting solution ideas from problems alone, reverse-engineering ideas from problems and known solutions, and solving problems using provided ideas. The loop alternates between reverse engineering such ideas from problems with supplied solutions and using these ideas as additional supervision for joint training of all three capabilities. We give a formal specification of our method and a concrete instantiation for interactive theorem proving in the Lean theorem prover; empirical evaluation remains future work.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 5 of its 30 references that are in the map. First posted on arXiv 8 Oct 2026.
Builds on
5 of 30 references.
References · 5
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2020Generative Language Modeling for Automated Theorem Provingcited by 388
- 2025DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decompositioncited by 287
- 2024DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Datacited by 234
- 2026Learning to Reason with Insight for Informal Theorem Provingcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Lean-STaR: Learning to Interleave Thinking and Proving
- 2024Thinking Forward and Backward: Effective Backward Planning with Large Language Models
- 2025RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems
- 2026Hindsight Hint Distillation: Scaffolded Reasoning for SWE Agents from CoT-free Answers
- 2025Less is More Tokens: Efficient Math Reasoning via Difficulty-Aware Chain-of-Thought Distillation
- 2025Reasoning Models Can Be Effective Without Thinking
- 2026Magenta: Closing the Loop Between Mathematical Reasoning and Lean Verification
- 2026HEAL: Hindsight Entropy-Assisted Learning for Reasoning Distillation