# Efficient Reasoning Models: A Survey

arXiv: https://arxiv.org/abs/2504.10903 · HTML: https://mlsanity.com/paper/2504.10903/

## Key facts

- Cited by 95 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 157 of its 192 references that are in the map.
- First posted on arXiv 15 Apr 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 157)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [Graph of Thoughts: Solving Elaborate Problems with Large Language Models](https://mlsanity.com/paper/2308.09687/) (2023)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)

## Built on by (most cited first, 30 of 95)

- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [Thinkless: LLM Learns When to Think](https://mlsanity.com/paper/2505.13379/) (2025)
- [Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains](https://mlsanity.com/paper/2505.16552/) (2025)
- [Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition](https://mlsanity.com/paper/2505.22375/) (2025)
- [SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning](https://mlsanity.com/paper/2505.12448/) (2025)
- [Implicit Reasoning in Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2509.02350/) (2025)
- [Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models](https://mlsanity.com/paper/2508.02120/) (2025)
- [Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoning](https://mlsanity.com/paper/2505.15154/) (2025)
- [ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning](https://mlsanity.com/paper/2505.04881/) (2025)
- [Semantic-Aware Logical Reasoning via a Semiotic Framework](https://mlsanity.com/paper/2509.24765/) (2025)
- [PixelThink: Towards Efficient Chain-of-Pixel Reasoning](https://mlsanity.com/paper/2505.23727/) (2025)
- [A Survey of Token Compression for Efficient Multimodal Large Language Models](https://mlsanity.com/paper/2507.20198/) (2025)
- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs](https://mlsanity.com/paper/2603.19217/) (2026)
- [OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models](https://mlsanity.com/paper/2511.14582/) (2025)
- [RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learning](https://mlsanity.com/paper/2510.02240/) (2025)
- [SofT-GRPO: Surpassing Discrete-Token LLM Reinforcement Learning via Gumbel-Reparameterized Soft-Thinking Policy Optimization](https://mlsanity.com/paper/2511.06411/) (2025)
- [Learning to Self-Verify Makes Language Models Better Reasoners](https://mlsanity.com/paper/2602.07594/) (2026)
- [Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewards](https://mlsanity.com/paper/2505.18298/) (2025)
- [Early Stopping Chain-of-thoughts in Large Language Models](https://mlsanity.com/paper/2509.14004/) (2025)
- [OBS-Diff: Accurate Pruning For Diffusion Models in One-Shot](https://mlsanity.com/paper/2510.06751/) (2025)
- [The Price of a Second Thought: On the Evaluation of Reasoning Efficiency in Large Language Models](https://mlsanity.com/paper/2505.22017/) (2025)
- [Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training](https://mlsanity.com/paper/2505.14681/) (2025)
- [Latent Chain-of-Thought World Modeling for End-to-End Driving](https://mlsanity.com/paper/2512.10226/) (2025)
- [Aware First, Think Less: Dynamic Boundary Self-Awareness Drives Extreme Reasoning Efficiency in Large Language Models](https://mlsanity.com/paper/2508.11582/) (2025)
- [FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Prediction](https://mlsanity.com/paper/2509.18362/) (2025)
- [Leash: Adaptive Length Penalty and Reward Shaping for Efficient Large Reasoning Model](https://mlsanity.com/paper/2512.21540/) (2025)
- [Active Perception Agent for Omnimodal Audio-Video Understanding](https://mlsanity.com/paper/2512.23646/) (2025)
- [Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers](https://mlsanity.com/paper/2606.31779/) (2026)
- [SABER: Switchable and Balanced Training for Efficient LLM Reasoning](https://mlsanity.com/paper/2508.10026/) (2025)

## Abstract (arXiv)

Reasoning models have demonstrated remarkable progress in solving complex and logic-intensive tasks by generating extended Chain-of-Thoughts (CoTs) prior to arriving at a final answer. Yet, the emergence of this "slow-thinking" paradigm, with numerous tokens generated in sequence, inevitably introduces substantial computational overhead. To this end, it highlights an urgent need for effective acceleration. This survey aims to provide a comprehensive overview of recent advances in efficient reasoning. It categorizes existing works into three key directions: (1) shorter - compressing lengthy CoTs into concise yet effective reasoning chains; (2) smaller - developing compact language models with strong reasoning capabilities through techniques such as knowledge distillation, other model compression techniques, and reinforcement learning; and (3) faster - designing efficient decoding strategies to accelerate inference of reasoning models. A curated collection of papers discussed in this survey is available in our GitHub repository: https://github.com/fscdc/Awesome-Efficient-Reasoning-Models.