mlsanityData to 7 Oct 2026

LIMO: Less is More for Reasoning

arXiv 2502.03387 ↗First posted 5 Feb 2025cs.CL cs.AIcited by 488

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

We challenge the prevailing assumption that complex reasoning in large language models (LLMs) necessitates massive training data. We demonstrate that sophisticated mathematical reasoning can emerge with only a few examples. Specifically, through simple supervised fine-tuning, our model, LIMO, achieves 63.3% accuracy on AIME24 and 95.6% on MATH500, surpassing previous fine-tuned models (6.5% on AIME24, 59.2% on MATH500) while using only 1% of the training data required by prior approaches. Furthermore, LIMO exhibits strong out-of-distribution generalization, achieving a 45.8% absolute improvement across diverse benchmarks, outperforming models trained on 100x more data. Synthesizing these findings, we propose the Less-Is-More Reasoning Hypothesis (LIMO Hypothesis): In foundation models where domain knowledge has been comprehensively encoded during pre-training, sophisticated reasoning can emerge through minimal but strategically designed demonstrations of cognitive processes. This hypothesis suggests that the threshold for eliciting complex reasoning is not dictated by task complexity but rather by two key factors: (1) the completeness of the model's pre-trained knowledge base and (2) the effectiveness of post-training examples in serving as "cognitive templates" that guide reasoning.

Key facts. Cited by 488 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 32 of its 42 references that are in the map. First posted on arXiv 5 Feb 2025.

Builds on

32 of 42 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models7 Oct
  2. 2024A Careful Examination of Large Language Model Performance on Grade School Arithmetic1 May
  3. 2024Common 7B Language Models Already Possess Strong Math Capabilities7 Mar
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models5 Feb
  5. 2022Solving Quantitative Reasoning Problems with Language Models29 Jun

Other references · 27

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  9. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  10. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  11. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  12. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
15 more
  1. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  2. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  3. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  4. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  5. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  6. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  7. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 160
  8. 2024MAmmoTH2: Scaling Instructions from the Webcited by 143
  9. 2024O1 Replication Journey: A Strategic Progress Report -- Part 1cited by 140
  10. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 108
  11. 2024Benchmarking Benchmark Leakage in Large Language Modelscited by 102
  12. 2024O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson?cited by 90
  13. 2024Can Large Language Models Reason and Plan?cited by 82
  14. 2023MathPile: A Billion-Token-Scale Pretraining Corpus for Mathcited by 67
  15. 2024Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scalecited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 343; 2026: 145.

488 papers.

Sort:

By area:

  1. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  2. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  3. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  4. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
  5. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  6. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  7. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 287
  8. 2025Learning to Reason under Off-Policy Guidancecited by 266
  9. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  10. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 218
  11. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  12. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
38 more
  1. 2025Deep Think with Confidencecited by 199
  2. 2025ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learningcited by 187
  3. 2025Reasoning Models Can Be Effective Without Thinkingcited by 182
  4. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
  5. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  6. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  7. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  8. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuningcited by 167
  9. 2025ToRL: Scaling Tool-Integrated RLcited by 166
  10. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 165
  11. 2026Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolationcited by 157
  12. 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoningcited by 147
  13. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 144
  14. 2025LIMR: Less is More for RL Scalingcited by 136
  15. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  16. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  17. 2025Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuningcited by 121
  18. 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonablecited by 120
  19. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  20. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rolloutscited by 108
  21. 2025MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphscited by 105
  22. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learningcited by 100
  23. 2025Efficient Reasoning Models: A Surveycited by 93
  24. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 92
  25. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibilitycited by 89
  26. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 88
  27. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 88
  28. 2025SEED-GRPO: Semantic Entropy Enhanced GRPO for Uncertainty-Aware Policy Optimizationcited by 87
  29. 2025Scaling Test-Time Compute Without Verification or RL is Suboptimalcited by 82
  30. 2025Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoningcited by 82
  31. 2025NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questionscited by 81
  32. 2025Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chainscited by 81
  33. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 79
  34. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 78
  35. 2025Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn'tcited by 72
  36. 2025Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?cited by 69
  37. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Modelscited by 68
  38. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Datacited by 67

50 most cited of 488 All 488 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Exploring the Hidden Reasoning Process of Large Language Models by Misleading Them
  2. 2024Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models
  3. 2023Specializing Smaller Language Models towards Multi-Step Reasoning
  4. 2024Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact Guidance
  5. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
  6. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
  7. 2025Small Models Struggle to Learn from Strong Reasoners
  8. 2024On Memorization of Large Language Models in Logical Reasoning