# Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models

arXiv: https://arxiv.org/abs/2502.19918 · HTML: https://mlsanity.com/paper/2502.19918/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 25 of its 29 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 25 of 25)

- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023)
- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [Search-o1: Agentic Search-Enhanced Large Reasoning Models](https://mlsanity.com/paper/2501.05366/) (2025)
- [rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking](https://mlsanity.com/paper/2501.04519/) (2025)
- [Large Language Models are Better Reasoners with Self-Verification](https://mlsanity.com/paper/2212.09561/) (2022)
- [EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers](https://mlsanity.com/paper/2309.08532/) (2023)
- [TheoremQA: A Theorem-driven Question Answering dataset](https://mlsanity.com/paper/2305.12524/) (2023)
- [Stream of Search (SoS): Learning to Search in Language](https://mlsanity.com/paper/2404.03683/) (2024)
- [Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling](https://mlsanity.com/paper/2502.06703/) (2025)
- [Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods](https://mlsanity.com/paper/2404.00282/) (2024)
- [GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinements](https://mlsanity.com/paper/2402.10963/) (2024)
- [Answering Questions by Meta-Reasoning over Multiple Chains of Thought](https://mlsanity.com/paper/2304.13007/) (2023)
- [UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs](https://mlsanity.com/paper/2402.13630/) (2024)
- [Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generation](https://mlsanity.com/paper/2410.02725/) (2024)
- [Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTS](https://mlsanity.com/paper/2411.18478/) (2024)
- [MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems](https://mlsanity.com/paper/2404.04735/) (2024)
- [FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answering](https://mlsanity.com/paper/2405.13873/) (2024)
- [Code Comprehension then Auditing for Unsupervised LLM Evaluation](https://mlsanity.com/paper/2410.03131/) (2024)
- [Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study Over Open-ended Question Answering](https://mlsanity.com/paper/2410.08085/) (2024)
- [Learning to Reason from Feedback at Test-Time](https://mlsanity.com/paper/2502.15771/) (2025)

## Built on by (most cited first, 30 of 33)

- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://mlsanity.com/paper/2503.16419/) (2025)
- [Efficient Reasoning Models: A Survey](https://mlsanity.com/paper/2504.10903/) (2025)
- [Efficient Inference for Large Reasoning Models: A Survey](https://mlsanity.com/paper/2503.23077/) (2025)
- [Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time](https://mlsanity.com/paper/2504.12329/) (2025)
- [EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systems](https://mlsanity.com/paper/2510.13220/) (2025)
- [Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs](https://mlsanity.com/paper/2507.02076/) (2025)
- [Not All Thoughts are Generated Equal: Efficient LLM Reasoning via Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2505.11827/) (2025)
- [From Efficiency to Adaptivity: A Deeper Look at Adaptive Reasoning in Large Language Models](https://mlsanity.com/paper/2511.10788/) (2025)
- [The Art of Scaling Test-Time Compute for Large Language Models](https://mlsanity.com/paper/2512.02008/) (2025)
- [Controlling Thinking Speed in Reasoning Models](https://mlsanity.com/paper/2507.03704/) (2025)
- [WebAnchor: Anchoring Agent Planning to Stabilize Long-Horizon Web Reasoning](https://mlsanity.com/paper/2601.03164/) (2026)
- [EvoClinician: A Self-Evolving Agent for Multi-Turn Medical Diagnosis via Test-Time Evolutionary Learning](https://mlsanity.com/paper/2601.22964/) (2026)
- [Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Models](https://mlsanity.com/paper/2508.21365/) (2025)
- [Are Large Reasoning Models Interruptible?](https://mlsanity.com/paper/2510.11713/) (2025)
- [Meta-R1: Empowering Large Reasoning Models with Metacognition](https://mlsanity.com/paper/2508.17291/) (2025)
- [Conversation for Non-verifiable Learning: Self-Evolving LLMs through Meta-Evaluation](https://mlsanity.com/paper/2601.21464/) (2026)
- [TACT: Mitigating Overthinking and Overacting in Coding Agents via Activation Steering](https://mlsanity.com/paper/2605.05980/) (2026)
- [StyleBench: Evaluating thinking styles in Large Language Models](https://mlsanity.com/paper/2509.20868/) (2025)
- [ATLAS: Agentic Test-time Learning-to-Allocate Scaling](https://mlsanity.com/paper/2606.01667/) (2026)
- [What-If Analysis of Large Language Models: Explore the Game World Using Proactive Thinking](https://mlsanity.com/paper/2509.04791/) (2025)
- [Verifying Meta-Awareness via Predictive Rewards in Reasoning Models](https://mlsanity.com/paper/2510.03259/) (2025)
- [Searching Meta Reasoning Skeleton to Guide LLM Reasoning](https://mlsanity.com/paper/2510.04116/) (2025)
- [Know More, Know Clearer: A Meta-Cognitive Framework for Knowledge Augmentation in Large Language Models](https://mlsanity.com/paper/2602.12996/) (2026)
- [From Long to Lean: Performance-aware and Adaptive Chain-of-Thought Compression via Multi-round Refinement](https://mlsanity.com/paper/2509.22144/) (2025)
- [Chain of Mindset: Reasoning with Adaptive Cognitive Modes](https://mlsanity.com/paper/2602.10063/) (2026)
- [Measuring and Mitigating Post-hoc Rationalization in Reverse Chain-of-Thought Generation](https://mlsanity.com/paper/2602.14469/) (2026)
- [When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions](https://mlsanity.com/paper/2605.22873/) (2026)
- [Reasoning Systems as Structured Processes: Foundations, Failures, and Formal Criteria](https://mlsanity.com/paper/2508.01763/) (2025)
- [Beyond Meta-Reasoning: Metacognitive Consolidation for Self-Improving LLM Reasoning](https://mlsanity.com/paper/2604.17399/) (2026)
- [Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning](https://mlsanity.com/paper/2606.03965/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) often struggle with computational efficiency and error propagation in multi-step reasoning tasks. While recent advancements on prompting and post-training have enabled LLMs to perform step-wise reasoning, they still tend to explore unproductive solution paths without effective backtracking or strategy adjustment. In this paper, we propose Meta-Reasoner, a new framework that empowers LLMs to "think about how to think". It optimizes the inference process by dynamically adapting reasoning strategies in real-time. Our approach employs contextual multi-armed bandits (CMABs) to learn an adaptive policy. It learns to evaluate the current state of LLM's reasoning and determine optimal strategy that is most likely to lead to a successful outcome during inference, like whether to backtrack, switch to a new approach, or restart the problem-solving process. This meta-guidance helps avoid unproductive paths exploration during inference and hence improves computational efficiency. We evaluate Meta-Reasoner on math problems (e.g., Game-of-24, TheoremQA) and scientific tasks (e.g., SciBench). Results show that our method outperform previous SOTA methods by 9-12% in accuracy, while reducing inference time by 28-35% under the same compute budget. Additional experiments on creative writing demonstrate the generalizability of our approach to diverse reasoning-intensive tasks.