mlsanityData to 14 Oct 2026

System 2 Reasoning Capabilities Are Nigh

arXiv 2410.03662 ↗First posted 4 Oct 2024cs.AI cs.LGcited by 2

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

In recent years, machine learning models have made strides towards human-like reasoning capabilities from several directions. In this work, we review the current state of the literature and describe the remaining steps to achieve a neural model which can perform System~2 reasoning analogous to a human. We argue that if current models are insufficient to be classed as performing reasoning, there remains very little additional progress needed to attain that goal.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 38 of its 65 references that are in the map. First posted on arXiv 4 Oct 2024.

Builds on

38 of 65 references.

References · 38

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  6. 2023Let's Verify Step by Stepcited by 3,789
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  10. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  11. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  12. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
26 more
  1. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  2. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  3. 2020Rethinking Attention with Performerscited by 1,601
  4. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  5. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  6. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  7. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  8. 2021Multiscale Vision Transformerscited by 897
  9. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  10. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  11. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
  12. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 519
  13. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  14. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 393
  15. 2023Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistlescited by 290
  16. 2023Faithful Chain-of-Thought Reasoningcited by 288
  17. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 260
  18. 2023Large Language Model Guided Tree-of-Thoughtcited by 221
  19. 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Modelscited by 191
  20. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  21. 2018Supervising strong learners by amplifying weak expertscited by 145
  22. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 128
  23. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 118
  24. 2023Scaling Transformer to 1M tokens and beyond with RMTcited by 98
  25. 2022Visual Commonsense in Pretrained Unimodal and Multimodal Modelscited by 39
  26. 2019Program synthesis performance constrained by non-linear spatial relations in Synthetic Visual Reasoning Testcited by 4

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025ReVISE: Learning to Refine at Test-Time via Intrinsic Self-Verificationcited by 35
  2. 2025Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering Taskcited by 10

Similar, not cited

Close in wording, no citation either way

  1. 2025From System 1 to System 2: A Survey of Reasoning Large Language Models
  2. 2023Studying and improving reasoning in humans and machines
  3. 2025Giving AI Personalities Leads to More Human-Like Reasoning
  4. 2025Evolutionary System 2 Reasoning: An Empirical Proof
  5. 2023Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic Systems
  6. 2025On the Thinking-Language Modeling Gap in Large Language Models
  7. 2025Reasoning on a Spectrum: Aligning LLMs to System 1 and System 2 Thinking
  8. 2025Do Theory of Mind Benchmarks Need Explicit Human-like Reasoning in Language Models?