mlsanityData to 14 Oct 2026

A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond

arXiv 2503.21614 ↗First posted 27 Mar 2025cs.CLcited by 132

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Recent Large Reasoning Models (LRMs), such as DeepSeek-R1 and OpenAI o1, have demonstrated strong performance gains by scaling up the length of Chain-of-Thought (CoT) reasoning during inference. However, a growing concern lies in their tendency to produce excessively long reasoning traces, which are often filled with redundant content (e.g., repeated definitions), over-analysis of simple problems, and superficial exploration of multiple reasoning paths for harder tasks. This inefficiency introduces significant challenges for training, inference, and real-world deployment (e.g., in agent-based systems), where token economy is critical. In this survey, we provide a comprehensive overview of recent efforts aimed at improving reasoning efficiency in LRMs, with a particular focus on the unique challenges that arise in this new paradigm. We identify common patterns of inefficiency, examine methods proposed across the LRM lifecycle, i.e., from pretraining to inference, and discuss promising future directions for research. To support ongoing development, we also maintain a real-time GitHub repository tracking recent progress in the field. We hope this survey serves as a foundation for further exploration and inspires innovation in this rapidly evolving area.

Key facts. Cited by 132 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 175 of its 253 references that are in the map. First posted on arXiv 27 Mar 2025.

Builds on

175 of 253 references.

Closest ancestors · 33

Same direction, newest first

  1. 2025Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging26 Mar
  2. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  3. 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models6 Mar
  4. 2025Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decoding3 Mar
  5. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach3 Mar
  6. 2025Thinking Slow, Fast: Scaling Inference Compute with Distilled Reasoners27 Feb
  7. 2025Self-Training Elicits Concise Reasoning in Large Language Models27 Feb
  8. 2025Efficient Test-Time Scaling via Self-Calibration25 Feb
  9. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning25 Feb
  10. 2025LightThinker: Thinking Step-by-Step Compression21 Feb
  11. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuning13 Feb
  12. 2025The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks12 Feb
21 more
  1. 2025Demystifying Long Chain-of-Thought Reasoning in LLMs5 Feb
  2. 2025Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning5 Feb
  3. 2025s1: Simple test-time scaling31 Jan
  4. 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs30 Jan
  5. 2025Think Smarter not Harder: Adaptive Reasoning with Inference Aware Optimization29 Jan
  6. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning22 Jan
  7. 2024Titans: Learning to Memorize at Test Time31 Dec
  8. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
  9. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective18 Dec
  10. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representations17 Dec
  11. 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness16 Dec
  12. 2024Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning12 Dec
  13. 2024TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling18 Oct
  14. 2024LightTransfer: Your Long-Context LLM is Secretly a Hybrid Model with Effortless Adaptation17 Oct
  15. 2024Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces13 Oct
  16. 2024Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning24 Aug
  17. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  18. 2024Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost29 Jul
  19. 2024CriticBench: Benchmarking LLMs for Critique-Correct Reasoning22 Feb
  20. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  21. 2024Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning19 Jan

Other references · 142

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  9. 2023Let's Verify Step by Stepcited by 3,789
  10. 2020Longformer: The Long-Document Transformercited by 3,413
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  12. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
38 more
  1. 2023A Survey of Large Language Modelscited by 2,576
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  3. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  5. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  7. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  8. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  9. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  10. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  11. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  12. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  13. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  14. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  15. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  16. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  17. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  18. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  19. 2024A Survey on Large Language Models for Code Generationcited by 658
  20. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  21. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  22. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
  23. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 514
  24. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  25. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 433
  26. 2024Understanding the planning of LLM agents: A surveycited by 421
  27. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 415
  28. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 415
  29. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  30. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  31. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  32. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 396
  33. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 391
  34. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 379
  35. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 370
  36. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 355
  37. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  38. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322

50 most cited of 142 All 142 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 91; 2026: 41.

132 papers.

Sort:

By area:

  1. 2025Learning to Reason under Off-Policy Guidancecited by 281
  2. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  3. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  4. 2025Kimi Linear: An Expressive, Efficient Attention Architecturecited by 196
  5. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  6. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learningcited by 163
  7. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  8. 2025Efficient Reasoning Models: A Surveycited by 95
  9. 2025ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoningcited by 72
  10. 2025Safety in Large Reasoning Models: A Surveycited by 64
  11. 2025Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?cited by 58
  12. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learningcited by 53
38 more
  1. 2025ExGRPO: Learning to Reason from Experiencecited by 53
  2. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  3. 2025AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgetingcited by 50
  4. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 43
  5. 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 42
  6. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 39
  7. 2025ThinkSwitcher: When to Think Hard, When to Think Fastcited by 37
  8. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 37
  9. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 36
  10. 2025ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoningcited by 33
  11. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 33
  12. 2025Pruning the Unsurprising: Efficient LLM Reasoning via First-Token Surprisalcited by 32
  13. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  14. 2025R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Searchcited by 30
  15. 2025R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routingcited by 29
  16. 2025Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Modelscited by 28
  17. 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoningcited by 28
  18. 2025System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcutscited by 27
  19. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  20. 2025What Is Your AI Agent Buying? Evaluation, Biases, Model Dependence, & Emerging Implications for Agentic E-Commercecited by 22
  21. 2026VideoAuto-R1: Video Auto Reasoning via Thinking Once, Answering Twicecited by 21
  22. 2026GEMS: Agent-Native Multimodal Generation with Memory and Skillscited by 21
  23. 2025Plan Then Action:High-Level Planning Guidance Reinforcement Learning for LLM Reasoningcited by 19
  24. 2025DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Modelscited by 19
  25. 2025The Price of a Second Thought: On the Evaluation of Reasoning Efficiency in Large Language Modelscited by 18
  26. 2025ThinkLess: A Training-Free Inference-Efficient Method for Reducing Reasoning Redundancycited by 17
  27. 2025OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for Over-Reasoning Mitigationcited by 17
  28. 2025ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shapingcited by 17
  29. 2026GlimpRouter: Efficient Collaborative Inference by Glimpsing One Token of Thoughtscited by 17
  30. 2026Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gaincited by 17
  31. 2025Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Trainingcited by 16
  32. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 16
  33. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  34. 2025Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Promptingcited by 15
  35. 2026LatentMem: Customizing Latent Memory for Multi-Agent Systemscited by 15
  36. 2025VisCRA: A Visual Chain Reasoning Attack for Jailbreaking Multimodal Large Language Modelscited by 13
  37. 2025MATP-BENCH: Can MLLM Be a Good Automated Theorem Prover for Multimodal Problems?cited by 13
  38. 2025On Reasoning Strength Planning in Large Reasoning Modelscited by 13

50 most cited of 132 All 132 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Efficient Inference for Large Reasoning Models: A Survey
  2. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey
  3. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
  4. 2025Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities
  5. 2025One Missing Piece for Open-Source Reasoning Models: A Dataset to Mitigate Cold-Starting Short CoT LLMs in RL
  6. 2025THINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models
  7. 2026CoTJudger: A Graph-Driven Framework for Automatic Evaluation of Chain-of-Thought Efficiency and Redundancy in LRMs
  8. 2025Logical Reasoning in Large Language Models: A Survey