mlsanityData to 7 Oct 2026

LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models

arXiv 2404.05221 ↗First posted 8 Apr 2024cs.CL cs.AIcited by 70

On the map: World models → World models for language agents

also in 1 more
Abstract (arXiv)

Generating accurate step-by-step reasoning is essential for Large Language Models (LLMs) to address complex problems and enhance robustness and interpretability. Despite the flux of research on developing advanced reasoning approaches, systematically analyzing the diverse LLMs and reasoning strategies in generating reasoning chains remains a significant challenge. The difficulties stem from the lack of two key elements: (1) an automatic method for evaluating the generated reasoning chains on different tasks, and (2) a unified formalism and implementation of the diverse reasoning approaches for systematic comparison. This paper aims to close the gap: (1) We introduce AutoRace for fully automated reasoning chain evaluation. Existing metrics rely on expensive human annotations or pre-defined LLM prompts not adaptable to different tasks. In contrast, AutoRace automatically creates detailed evaluation criteria tailored for each task, and uses GPT-4 for accurate evaluation following the criteria. (2) We develop LLM Reasoners, a library for standardized modular implementation of existing and new reasoning algorithms, under a unified formulation of the search, reward, and world model components. With the new evaluation and library, (3) we conduct extensive study of different reasoning approaches (e.g., CoT, ToT, RAP). The analysis reveals interesting findings about different factors contributing to reasoning, including the reward-guidance, breadth-vs-depth in search, world model, and prompt formats, etc.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 50 of its 69 references that are in the map. First posted on arXiv 8 Apr 2024.

Builds on

50 of 69 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planning8 Dec
  2. 2023Reasoning with Language Model is Planning with World Model24 May

Other references · 48

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2023Let's Verify Step by Stepcited by 3,598
  9. 2023Qwen Technical Reportcited by 3,273
  10. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  11. 2023Mistral 7Bcited by 3,065
  12. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
36 more
  1. 2024Mixtral of Expertscited by 1,785
  2. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  3. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  4. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  5. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  6. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  7. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  8. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  9. 2024InternLM2 Technical Reportcited by 391
  10. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 334
  11. 2023Faithful Chain-of-Thought Reasoningcited by 285
  12. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 281
  13. 2020Evaluation of Text Generation: A Surveycited by 275
  14. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  15. 2022FOLIO: Natural Language Reasoning with First-Order Logiccited by 233
  16. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  17. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 203
  18. 2024Advancing LLM Reasoning Generalists with Preference Treescited by 191
  19. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 175
  20. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
  21. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 158
  22. 2021Explaining Answers with Entailment Treescited by 148
  23. 2022Faithful Reasoning Using Large Language Modelscited by 130
  24. 2024Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoningcited by 108
  25. 2023ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Searchcited by 103
  26. 2024Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervisioncited by 95
  27. 2023On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)cited by 85
  28. 2023LLMs cannot find reasoning errors, but can correct them given the error locationcited by 85
  29. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 83
  30. 2022Solving Math Word Problems via Cooperative Reasoning induced Language Modelscited by 76
  31. 2023ReCEval: Evaluating Reasoning Chains via Correctness and Informativenesscited by 76
  32. 2024When is Tree Search Useful for LLM Planning? It Depends on the Discriminatorcited by 46
  33. 2021A Study of Automatic Metrics for the Evaluation of Natural Language Explanationscited by 34
  34. 2023SocREval: Large Language Models with the Socratic Method for Reference-Free Reasoning Evaluationcited by 27
  35. 2023Structured Chemistry Reasoning with Large Language Modelscited by 27
  36. 2023Self-Contradictory Reasoning Evaluation and Detectioncited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 40; 2026: 19.

70 papers.

Sort:

By area:

  1. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  2. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  3. 2025MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiencycited by 123
  4. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  5. 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoningcited by 89
  6. 2024Process Reward Model with Q-Value Rankingscited by 78
  7. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  8. 2024Interpretable Contrastive Monte Carlo Tree Search Reasoningcited by 62
  9. 2025Atom of Thoughts for Markov LLM Test-Time Scalingcited by 54
  10. 2024Offline Reinforcement Learning for LLM Multi-Step Reasoningcited by 53
  11. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  12. 2025Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?cited by 35
38 more
  1. 2024GameArena: Evaluating LLM Reasoning through Live Computer Gamescited by 32
  2. 2025MINERVA: Evaluating Complex Video Reasoningcited by 31
  3. 2025Inference-Time Computations for LLM Reasoning and Planning: A Benchmark and Insightscited by 30
  4. 2024Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examplescited by 29
  5. 2025Med-PRM: Medical Reasoning Models with Stepwise, Guideline-verified Process Rewardscited by 24
  6. 2024ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaborationcited by 22
  7. 2025DSGBench: A Diverse Strategic Game Benchmark for Evaluating LLM-based Agents in Complex Decision-Making Environmentscited by 22
  8. 2025Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPOcited by 18
  9. 2026Vision-aligned Latent Reasoning for Multi-modal Large Language Modelcited by 16
  10. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  11. 2025CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIscited by 14
  12. 2025General Agentic Planning Through Simulative Reasoning with World Modelscited by 13
  13. 2024FG-PRM: Fine-grained Hallucination Detection and Mitigation in Language Model Mathematical Reasoningcited by 12
  14. 2025One Example Shown, Many Concepts Known! Counterexample-Driven Conceptual Reasoning in Mathematical LLMscited by 12
  15. 2025TurnBench-MS: A Benchmark for Evaluating Multi-Turn, Multi-Step Reasoning in Large Language Modelscited by 12
  16. 2025Examining False Positives under Inference Scaling for Mathematical Reasoningcited by 11
  17. 2025Evaluating System 1 vs. 2 Reasoning Approaches for Zero-Shot Time Series Forecasting: A Benchmark and Insightscited by 11
  18. 2025Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoningcited by 10
  19. 2025ProPerSim: Developing Proactive and Personalized AI Assistants through User-Assistant Simulationcited by 10
  20. 2025What Defines Good Reasoning in LLMs? Dissecting Reasoning Steps with Multi-Aspect Evaluationcited by 10
  21. 2025Investigating the Shortcomings of LLMs in Step-by-Step Legal Reasoningcited by 9
  22. 2025Policy Guided Tree Search for Enhanced LLM Reasoningcited by 8
  23. 2025Predictive Auditing of Hidden Tokens in LLM APIs via Reasoning Length Estimationcited by 7
  24. 2025FEM-Bench: A Structured Scientific Reasoning Benchmark for Evaluating Code-Generating LLMscited by 7
  25. 2025DSADF: Thinking Fast and Slow for Decision Makingcited by 6
  26. 2025Can Large Language Models Adequately Perform Symbolic Reasoning Over Time Series?cited by 6
  27. 2025SpeContext: Enabling Efficient Long-context Reasoning with Speculative Context Sparsity in LLMscited by 6
  28. 2025EvoEmpirBench: Dynamic Spatial Reasoning with Agent-ExpVercited by 5
  29. 2026FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoningcited by 5
  30. 2026Learning Modal-Mixed Chain-of-Thought Reasoning with Latent Embeddingscited by 5
  31. 2025Offline Learning and Forgetting for Reasoning with Large Language Modelscited by 4
  32. 2026OneLatent: Single-Token Compression for Visual Latent Reasoningcited by 4
  33. 2026nabla-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Spacecited by 4
  34. 2026Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuningcited by 4
  35. 2025Deliberate Planning in Language Models with Symbolic Representationcited by 3
  36. 2025VeriTrail: Closed-Domain Hallucination Detection with Traceabilitycited by 3
  37. 2026More Test-Time Compute Can Hurt: Overestimation Bias in LLM Beam Searchcited by 3
  38. 2025Creativity or Brute Force? Using Brainteasers as a Window into the Problem-Solving Abilities of Large Language Modelscited by 2

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes
  2. 2025AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Models
  3. 2025LLMSR@XLLM25: An Empirical Study of LLM for Structural Reasoning
  4. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance
  5. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  6. 2022Towards Reasoning in Large Language Models: A Survey
  7. 2025Fast, Slow, and Tool-augmented Thinking for LLMs: A Review
  8. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey