mlsanityData to 14 Oct 2026

ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates

arXiv 2502.06772 ↗First posted 10 Feb 2025cs.CL cs.AI cs.LGcited by 63

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

We present that hierarchical LLM reasoning via scaling thought templates can effectively optimize the reasoning search space and outperform the mathematical reasoning capabilities of powerful LLMs like OpenAI o1-preview and DeepSeek V3. We train our ReasonFlux-32B model with only 8 GPUs and introduces three innovations: (i) a structured and generic thought template library, containing around 500 high-level thought templates capable of generalizing to similar or relevant reasoning problems; (ii) performing hierarchical reinforcement learning on a sequence of thought templates instead of long CoTs, optimizing a base LLM to plan out an optimal template trajectory for gradually handling complex problems; (iii) a brand new inference scaling system that enables hierarchical LLM reasoning by adaptively scaling thought templates at inference time. With a template trajectory containing more explainable reasoning structures than DeepSeek-R1 and o3-mini, our ReasonFlux-32B significantly advances math reasoning capabilities to state-of-the-art levels. Notably, on the MATH benchmark, it achieves an accuracy of 91.2% and surpasses o1-preview by 6.7%. On the USA Math Olympiad (AIME) benchmark, ReasonFlux-32B solves an average of 56.7% of problems, surpassing o1-preview and DeepSeek-V3 by 27% and 45%, respectively. Code: https://github.com/Gen-Verse/ReasonFlux

Key facts. Cited by 63 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 44 of its 58 references that are in the map. First posted on arXiv 10 Feb 2025.

Builds on

44 of 58 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  2. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models1 Aug
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May

Other references · 41

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  8. 2017Deep reinforcement learning from human preferencescited by 3,912
  9. 2023Let's Verify Step by Stepcited by 3,789
  10. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  11. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  12. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
29 more
  1. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  2. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  3. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  4. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  5. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  6. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  7. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  8. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  9. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
  10. 2023Augmented Language Models: a Surveycited by 403
  11. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  12. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344
  13. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  14. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 279
  15. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 273
  16. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 262
  17. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 241
  18. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 233
  19. 2023Planning with Large Language Models for Code Generationcited by 182
  20. 2024A Minimaximalist Approach to Reinforcement Learning from Human Feedbackcited by 144
  21. 2024Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solverscited by 138
  22. 2022Retrieval-Augmented Multimodal Language Modelingcited by 125
  23. 2024Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Modelscited by 91
  24. 2024MARIO: MAth Reasoning with code Interpreter Output -- A Reproducible Pipelinecited by 81
  25. 2024Direct Multi-Turn Preference Optimization for Language Agentscited by 64
  26. 2023Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decodingcited by 60
  27. 2024Code Repair with LLMs gives an Exploration-Exploitation Tradeoffcited by 51
  28. 2023KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detectioncited by 45
  29. 2022Retrieval-based Controllable Molecule Generationcited by 44

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 42; 2026: 21.

63 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  3. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 250
  4. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  5. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  6. 2026OpenClaw-RL: Train Any Agent Simply by Talkingcited by 90
  7. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 79
  8. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learningcited by 56
  9. 2025ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMscited by 46
  10. 2025Generative to Agentic AI: Survey, Conceptualization, and Challengescited by 33
  11. 2025Demystifying Reinforcement Learning in Agentic Reasoningcited by 33
  12. 2025AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxescited by 30
38 more
  1. 2025TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoningcited by 28
  2. 2025A Technical Survey of Reinforcement Learning Techniques for Large Language Modelscited by 21
  3. 2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Lawcited by 19
  4. 2025HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generationcited by 18
  5. 2025Beyond Outlining: Heterogeneous Recursive Planning for Adaptive Long-form Writing with Language Modelscited by 16
  6. 2025Not All Thoughts are Generated Equal: Efficient LLM Reasoning via Multi-Turn Reinforcement Learningcited by 16
  7. 2025Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Promptingcited by 15
  8. 2025Guiding Reasoning in Small Language Models with LLM Assistancecited by 14
  9. 2025TrustGeoGen: Formal-Verified Data Engine for Trustworthy Multi-modal Geometric Problem Solvingcited by 13
  10. 2025Learning to Refine: Self-Refinement of Parallel Reasoning in LLMscited by 12
  11. 2025Retrieval-of-Thought: Efficient Reasoning via Reusing Thoughtscited by 12
  12. 2026WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoningcited by 12
  13. 2025Temporal Consistency for LLM Reasoning Process Error Identificationcited by 10
  14. 2025ProtoReasoning: Prototypes as the Foundation for Generalizable Reasoning in LLMscited by 9
  15. 2026Recursive Experiential-Working Memory Evolution for Long-Horizon Agent Harnessescited by 9
  16. 2025Test-time Scaling Techniques in Theoretical Physics -- A Comparison of Methods on the TPBench Datasetcited by 8
  17. 2026ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spacescited by 8
  18. 2025Toward Scientific Reasoning in LLMs: Training from Expert Discussions via Reinforcement Learningcited by 7
  19. 2026MemCollab: Cross-Model Memory Collaboration via Contrastive Trajectory Distillationcited by 7
  20. 2026Procedural Knowledge at Scale Improves Reasoningcited by 7
  21. 2025StyleBench: Evaluating thinking styles in Large Language Modelscited by 5
  22. 2025LANPO: Bootstrapping Language and Numerical Feedback for Reinforcement Learning in LLMscited by 5
  23. 2025Thinker: Training LLMs in Hierarchical Thinking for Deep Search via Multi-Turn Interactioncited by 5
  24. 2025Agent WARPP: Workflow Adherence via Runtime Parallel Personalizationcited by 3
  25. 2025PRIME: Planning and Retrieval-Integrated Memory for Enhanced Reasoningcited by 3
  26. 2025Searching Meta Reasoning Skeleton to Guide LLM Reasoningcited by 3
  27. 2025When Thoughts Meet Facts: Reusable Reasoning for Long-Context LMscited by 3
  28. 2026Find, Fix, Reason: Context Repair for Video Reasoningcited by 3
  29. 2026Discovery Foundation Models: Toward Open-Ended Discovery Intelligencecited by 3
  30. 2025Kongzi: A Historical Large Language Model with Fact Enhancementcited by 2
  31. 2025DialogueReason: Rule-Based RL Sparks Dialogue Reasoning in LLMscited by 2
  32. 2025KAG-Thinker: Interactive Thinking and Deep Reasoning in LLMs via Knowledge-Augmented Generationcited by 2
  33. 2025Beyond Fast and Slow: Cognitive-Inspired Elastic Reasoning for Large Language Modelscited by 2
  34. 2026Towards Effective Experiential Learning: Dual Guidance for Utilization and Internalizationcited by 2
  35. 2026Learning to Reason with Insight for Informal Theorem Provingcited by 2
  36. 2025Breaking the Myth: Can Small Models Infer Postconditions Too?cited by 1
  37. 2025CLARity: Reasoning Consistency Alone Can Teach Reinforced Expertscited by 1
  38. 2025Big Reasoning with Small Models: Instruction Retrieval at Inference Timecited by 1

50 most cited of 63 All 63 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinking
  2. 2025Learning to Reason: Training LLMs with GPT-OSS or DeepSeek R1 Reasoning Traces
  3. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models
  4. 2024MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time
  5. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
  6. 2025Prompting Test-Time Scaling Is A Strong LLM Reasoning Data Augmentation
  7. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  8. 2025T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling