mlsanityData to 7 Oct 2026

LeAct: Learning to Reason from Expert Actions

arXiv 2607.21856 ↗First posted 23 Jul 2026cs.LG cs.AI cs.CLcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Modern reasoning models depend on reasoning data, today sourced from human annotations or distilled from stronger LLMs. However, a rich and largely untapped source of supervision lies in expert systems (e.g., game engines, classical planners, theorem provers), which routinely produce near-optimal actions across diverse domains. But these experts are silent: they commit to an action without writing down the chain of thought (CoT) behind it. Recovering that CoT as natural-language reasoning would distill expert knowledge into a student that generalizes beyond the demonstrated actions. We treat it as a latent variable and study how to recover it from the action alone. Our approach, LeAct (Learning to reason from Actions), optimizes this latent variable: the student samples candidate CoTs for each expert action, and we retain those that measurably improve its own probability of recovering the action. Across imperfect-information games at multiple scales and a simulated robotics benchmark, LeAct reaches the solver's numerical floor on small enumerable games. At larger scale, it is 5× closer to the solver than the strongest expert-iteration baseline. At Flop Hold'em (~ 109 infosets), LeAct wins head-to-head by +60 mbb/g, and on the robotics probe it is the only training recipe that improves on direct imitation. We present a principled framework and the result: expert systems become a categorically new source of reasoning teachers for foundation models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 40 of its 62 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

40 of 62 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Reasoning to Learn from Latent Thoughts24 Mar
  2. 2024Training Language Models to Self-Correct via Reinforcement Learning19 Sep
  3. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  4. 2024V-STaR: Training Verifiers for Self-Taught Reasoners9 Feb
  5. 2024ReFT: Reasoning with Reinforced Fine-Tuning17 Jan

Other references · 35

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2023Let's Verify Step by Stepcited by 3,598
  9. 2024OpenAI o1 System Cardcited by 1,220
  10. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  11. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  12. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
23 more
  1. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  2. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  3. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  4. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  5. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  6. 2020Generative Language Modeling for Automated Theorem Provingcited by 379
  7. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
  8. 2022Large Language Models Are Reasoning Teacherscited by 318
  9. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  10. 2022Teaching Small Language Models to Reasoncited by 279
  11. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 267
  12. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 180
  13. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 158
  14. 2023Chain of Hindsight Aligns Language Models with Feedbackcited by 147
  15. 2025Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correctioncited by 140
  16. 2025Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Provingcited by 126
  17. 2014Solving Large Imperfect Information Games Using CFR+cited by 78
  18. 2014Reweighted Wake-Sleepcited by 66
  19. 2024Offline Reinforcement Learning for LLM Multi-Step Reasoningcited by 53
  20. 2026AlgoVeri: An Aligned Benchmark for Verified Code Generation on Classical Algorithmscited by 11
  21. 2026Goedel-Code-Prover: Hierarchical Proof Search for Open State-of-the-Art Code Verificationcited by 10
  22. 2025BuilderBench: The Building Blocks of Intelligent Agentscited by 4
  23. 2026Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025OpenSIR: Open-Ended Self-Improving Reasoner
  2. 2025Reasoning Can Hurt the Inductive Abilities of Large Language Models
  3. 2025Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task Arithmetic
  4. 2026MetaCtrl: Your Large Language Models Can Reason Better and More Concisely with a Metacognitive Controller
  5. 2025From Reasoning to Super-Intelligence: A Search-Theoretic Perspective
  6. 2026The Weakest Link: Distilling LLM Reasoning with Worst-Case Constrained Reinforcement Learning
  7. 2024Multi-Step Reasoning with Large Language Models, a Survey
  8. 2025Thinkless: LLM Learns When to Think