mlsanityData to 7 Oct 2026

Verifying Meta-Awareness via Predictive Rewards in Reasoning Models

arXiv 2510.03259 ↗First posted 26 Sep 2025cs.LG cs.AIcited by 3

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Recent research on reasoning models explores the meta-awareness of language models, including their ability to determine optimal thinking duration, recognize knowledge boundaries, and structure concept-level thinking. While current large reasoning models depend solely on answer-based verification, we show that adding meta-awareness objectives leads to significant performance gains over models without such meta-knowledge. MAPR (Meta-Awareness via Predictive Reward) utilizes a self-generated task of predicting rollout statistics - specifically length, pass-rate, and concepts used - allowing for verification against the actual statistics. Furthermore, by leveraging this self-predictive capability, the model can regulate its reasoning behavior by i) filtering out trivial or unsolvable prompts, ii) reducing lengthy generations that tend to be incorrect, and iii) generating hints relevant to the problem. The results are inspiring: MAPR yields significant improvements in both accuracy and training efficiency on various reasoning benchmarks. More specifically, our method can speed up GRPO training by over 1.28x to reach the same performance, and achieve 83.18% gain in accuracy on AIME25, and a 13.04% average gain over six mathematics benchmarks. The code is publicly available at https://github.com/akatigre/MAPR-RL.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 49 of its 68 references that are in the map. First posted on arXiv 26 Sep 2025.

Builds on

49 of 68 references.

Closest ancestors · 20

Same direction, newest first

  1. 2025Metacognitive Reuse: Turning Recurring LLM Reasoning Into Concise Behaviors16 Sep
  2. 2025Language Models Coupled with Metacognition Can Outperform Reasoning Models25 Aug
  3. 2025Meta-R1: Empowering Large Reasoning Models with Metacognition24 Aug
  4. 2025Aware First, Think Less: Dynamic Boundary Self-Awareness Drives Extreme Reasoning Efficiency in Large Language Models15 Aug
  5. 2025From "Aha Moments" to Controllable Thinking: Toward Meta-Cognitive Reasoning in Large Reasoning Models via Decoupled Reasoning and Control6 Aug
  6. 2025Large Language Models Have Intrinsic Meta-Cognition, but Need a Good Lens10 Jun
  7. 2025Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learning5 Jun
  8. 2025Adaptive Deep Reasoning: Triggering Deep Thinking When Needed26 May
  9. 2025AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgeting24 May
  10. 2025Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoning21 May
  11. 2025AdaptThink: Reasoning Models Can Learn When to Think19 May
  12. 2025Thinkless: LLM Learns When to Think19 May
8 more
  1. 2025Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL16 May
  2. 2025ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning8 May
  3. 2025Dynamic Early Exit in Reasoning Models22 Apr
  4. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  5. 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models6 Mar
  6. 2025Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models27 Feb
  7. 2024Token-Budget-Aware LLM Reasoning24 Dec
  8. 2024Rational Metareasoning for Large Language Models7 Oct

Other references · 29

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2021Program Synthesis with Large Language Modelscited by 3,509
  8. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  9. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  10. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  11. 2025Group Sequence Policy Optimizationcited by 788
  12. 2025Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuningcited by 121
17 more
  1. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 88
  2. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  3. 2025AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learningcited by 85
  4. 2025GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Modelscited by 81
  5. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 78
  6. 2025RLVMR: Reinforcement Learning with Verifiable Meta-Reasoning Rewards for Robust Long-Horizon Agentscited by 56
  7. 2025EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversitycited by 41
  8. 2025Think When You Need: Self-Adaptive Chain-of-Thought Learningcited by 37
  9. 2025GHPO: Adaptive Guidance for Stable and Efficient LLM Reinforcement Learningcited by 33
  10. 2025R-Bench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluationcited by 32
  11. 2025Learning to Deliberate: Meta-policy Collaboration for Agentic LLMs with Multi-agent Reinforcement Learningcited by 18
  12. 2025How Difficulty-Aware Staged Reinforcement Learning Enhances LLMs' Reasoning Capabilities: A Preliminary Experimental Studycited by 14
  13. 2025Good Learners Think Their Thinking: Generative PRM Makes Large Reasoning Model More Efficient Math Learnercited by 11
  14. 2025Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Surveycited by 10
  15. 2025Length-Controlled Margin-Based Preference Optimization without Reference Modelcited by 6
  16. 2025MeLA: A Metacognitive LLM-Driven Architecture for Automatic Heuristic Designcited by 4
  17. 2025Enhancing Math Reasoning in Small-sized LLMs via Preview Difficulty-Aware Interventioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026EpiCaR: Knowing What You Don't Know Matters for Better Reasoning in LLMscited by 1
  2. 2026When Does In-Context Search Help? A Sampling-Complexity Theory of Reflection-Driven Reasoningcited by 1
  3. 2026Metacognitive Behavioral Tuning of Large Language Models for Multi-Hop Question Answeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MetaCtrl: Your Large Language Models Can Reason Better and More Concisely with a Metacognitive Controller
  2. 2025Teaching Small Language Models to Learn Logic through Meta-Learning
  3. 2026Beyond Meta-Reasoning: Metacognitive Consolidation for Self-Improving LLM Reasoning
  4. 2026Reasoning While Asking: Transforming Reasoning Large Language Models from Passive Solvers to Proactive Inquirers
  5. 2024Meta-Reasoning Improves Tool Use in Large Language Models
  6. 2025MDToC: Metacognitive Dynamic Tree of Concepts for Boosting Mathematical Problem-Solving of Large Language Models
  7. 2023MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation
  8. 2023Meta-Reasoning: Semantics-Symbol Deconstruction for Large Language Models