mlsanityData to 7 Oct 2026

A Survey of Reasoning with Foundation Models

arXiv 2312.11562 ↗First posted 17 Dec 2023cs.AI cs.CL cs.CVcited by 70

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Reasoning, a crucial ability for complex problem-solving, plays a pivotal role in various real-world settings such as negotiation, medical diagnosis, and criminal investigation. It serves as a fundamental methodology in the field of Artificial General Intelligence (AGI). With the ongoing development of foundation models, e.g., Large Language Models (LLMs), there is a growing interest in exploring their abilities in reasoning tasks. In this paper, we introduce seminal foundation models proposed or adaptable for reasoning, highlighting the latest advancements in various reasoning tasks, methods, and benchmarks. We then delve into the potential future directions behind the emergence of reasoning abilities within foundation models. We also discuss the relevance of multimodal learning, autonomous agents, and super alignment in the context of reasoning. By discussing these future research directions, we hope to inspire researchers in their exploration of this field, stimulate further advancements in reasoning with foundation models, and contribute to the development of AGI.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 14 of its 844 references that are in the map. First posted on arXiv 17 Dec 2023.

Builds on

14 of 844 references.

References · 14

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  3. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  4. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  5. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  6. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  7. 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 157
  8. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derenderingcited by 132
  9. 2023LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructionscited by 124
  10. 2023CoEdIT: Text Editing by Task-Specific Instruction Tuningcited by 67
  11. 2020Semantically-Aligned Universal Tree-Structured Solver for Math Word Problemscited by 35
  12. 2023Dual-Balancing for Multi-Task Learningcited by 27
2 more
  1. 2021TSQA: Tabular Scenario Based Question Answeringcited by 21
  2. 2021Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategycited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 19; 2025: 42; 2026: 9.

70 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 116
  3. 2025OverThink: Slowdown Attacks on Reasoning LLMscited by 94
  4. 2025InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Modelscited by 53
  5. 2025Logical Reasoning in Large Language Models: A Surveycited by 52
  6. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 49
  7. 2025Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Timecited by 48
  8. 2025VLM-R3: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thoughtcited by 47
  9. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 42
  10. 2024The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Unitscited by 34
  11. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  12. 2024DAPE: Data-Adaptive Positional Encoding for Length Extrapolationcited by 30
38 more
  1. 2024Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes Oncited by 24
  2. 2026Large Language Model Reasoning Failurescited by 20
  3. 2025AI Hiring with LLMs: A Context-Aware and Explainable Multi-Agent Framework for Resume Screeningcited by 19
  4. 2025Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Modelscited by 19
  5. 2025Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Modelscited by 18
  6. 2025Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learningcited by 17
  7. 2024RePrompt: Planning by Automatic Prompt Engineering for Large Language Models Agentscited by 15
  8. 2024Benchmarking the Capabilities of Large Language Models in Transportation System Engineering: Accuracy, Consistency, and Reasoning Behaviorscited by 13
  9. 2025AutoLabs: Cognitive Multi-Agent Systems with Self-Correction for Autonomous Chemical Experimentationcited by 13
  10. 2024LangGPT: Rethinking Structured Reusable Prompt Design Framework for LLMs from the Programming Languagecited by 12
  11. 2025Learning from Diverse Reasoning Paths with Routing and Collaborationcited by 12
  12. 2024Anchor function: a type of benchmark functions for studying language modelscited by 10
  13. 2024DAPE V2: Process Attention Score as Feature Map for Length Extrapolationcited by 10
  14. 2025Teaching Your Models to Understand Code via Focal Preference Alignmentcited by 10
  15. 2025Synergizing Logical Reasoning, Knowledge Management and Collaboration in Multi-Agent LLM Systemcited by 10
  16. 2024WESE: Weak Exploration to Strong Exploitation for LLM Agentscited by 9
  17. 2024Reasoning on Efficient Knowledge Paths:Knowledge Graph Guides Large Language Model for Domain Question Answeringcited by 9
  18. 2025Demystifying Hybrid Thinking: Can LLMs Truly Switch Between Think and No-Think?cited by 9
  19. 2025The Shape of Reasoning: Topological Analysis of Reasoning Traces in Large Language Modelscited by 9
  20. 2024Improving Small-Scale Large Language Models Function Calling for Reasoning Taskscited by 7
  21. 2025MIRROR: Multi-agent Intra- and Inter-Reflection for Optimized Reasoning in Tool Learningcited by 7
  22. 2026Relax: An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scalecited by 7
  23. 2024Smart Vision-Language Reasonerscited by 5
  24. 2024MARPLE: A Benchmark for Long-Horizon Inferencecited by 5
  25. 2024Boosting Deductive Reasoning with Step Signals In RLHFcited by 4
  26. 2025Exploring the Generalizability of Factual Hallucination Mitigation via Enhancing Precise Knowledge Utilizationcited by 4
  27. 2025Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Modelscited by 4
  28. 2025GraphLocator: Graph-guided Causal Reasoning for Issue Localizationcited by 4
  29. 2025Self-Adjust Softmaxcited by 3
  30. 2025GUIDE-CoT: Goal-driven and User-Informed Dynamic Estimation for Pedestrian Trajectory using Chain-of-Thoughtcited by 3
  31. 2025Threading the Needle: Reweaving Chain-of-Thought Reasoning to Explain Human Label Variationcited by 3
  32. 2025PISA-Bench: The PISA Index as a Multilingual and Multimodal Metric for the Evaluation of Vision-Language Modelscited by 3
  33. 2026Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Surveycited by 3
  34. 2026Capabilities and Fundamental Limits of Latent Chain-of-Thoughtcited by 3
  35. 2024Critique of Impure Reason: Unveiling the reasoning behaviour of medical Large Language Modelscited by 2
  36. 2025KROMA: Ontology Matching with Knowledge Retrieval and Large Language Modelscited by 2
  37. 2025From Hypothesis to Premises: LLM-based Backward Logical Reasoning with Selective Symbolic Translationcited by 2
  38. 2024AILS-NTUA at SemEval-2024 Task 9: Cracking Brain Teasers: Transformer Models for Lateral Thinking Puzzlescited by 1

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025From General Reasoning to Domain Expertise: Uncovering the Limits of Generalization in Large Language Models
  2. 2025From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Models
  3. 2022Towards Reasoning in Large Language Models: A Survey
  4. 2024Foundation Model Sherpas: Guiding Foundation Models through Knowledge and Reasoning
  5. 2024Understanding Foundation Models: Are We Back in 1924?
  6. 2024Position: Foundation Agents as the Paradigm Shift for Decision Making
  7. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems
  8. 2025From System 1 to System 2: A Survey of Reasoning Large Language Models