mlsanityData to 14 Oct 2026

RATIONALYST: Mining Implicit Rationales for Process Supervision of Reasoning

arXiv 2410.01044 ↗First posted 1 Oct 2024cs.AI cs.CLcited by 6

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

The reasoning steps generated by LLMs might be incomplete, as they mimic logical leaps common in everyday communication found in their pre-training data: underlying rationales are frequently left implicit (unstated). To address this challenge, we introduce RATIONALYST, a model for process-supervision of reasoning based on pre-training on a vast collection of rationale annotations extracted from unlabeled data. We extract 79k rationales from web-scale unlabelled dataset (the Pile) and a combination of reasoning datasets with minimal human intervention. This web-scale pre-training for reasoning allows RATIONALYST to consistently generalize across diverse reasoning tasks, including mathematical, commonsense, scientific, and logical reasoning. Fine-tuned from LLaMa-3-8B, RATIONALYST improves the accuracy of reasoning by an average of 3.9% on 7 representative reasoning benchmarks. It also demonstrates superior performance compared to significantly larger verifiers like GPT-4 and similarly sized models fine-tuned on matching training sets.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 41 references that are in the map. First posted on arXiv 1 Oct 2024.

Builds on

29 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models9 Oct
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 27

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  6. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  7. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  9. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  10. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  11. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  12. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
15 more
  1. 2023Textbooks Are All You Needcited by 569
  2. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
  3. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 324
  4. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
  5. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 290
  6. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 233
  7. 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Textcited by 155
  8. 2024Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained Rewardscited by 31
  9. 2024LeanReasoner: Boosting Complex Logical Reasoning with Leancited by 28
  10. 2024SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responsescited by 26
  11. 2024General Purpose Verification for Chain of Thought Promptingcited by 17
  12. 2022NELLIE: A Neuro-Symbolic Inference Engine for Grounded, Compositional, and Explainable Reasoningcited by 16
  13. 2024Enhancing Systematic Decompositional Natural Language Inference Using Informal Logiccited by 12
  14. 2024Learning to Plan for Language Modeling from Unlabeled Datacited by 12
  15. 2022Reflection of Thought: Inversely Eliciting Numerical Reasoning in Language Models via Solving Linear Systemscited by 7

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 4.

6 papers.

  1. 2025Reasoning to Learn from Latent Thoughtscited by 45
  2. 2025Improving RL Exploration for LLM Reasoning through Retrospective Replaycited by 19
  3. 2024CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoningcited by 16
  4. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  5. 2025Feedback Friction: LLMs Struggle to Fully Incorporate External Feedbackcited by 11
  6. 2024ToW: Thoughts of Words Improve Reasoning in Large Language Modelscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Unmasking Reasoning Processes: A Process-aware Benchmark for Evaluating Structural Mathematical Reasoning in LLMs
  2. 2026Structural Rationale Distillation via Reasoning Space Compression
  3. 2026TLRD: Teaching LLMs to Reason over Tabular Data with Tri-Level Rationale Distillation
  4. 2025Beyond the First Error: Process Reward Models for Reflective Mathematical Reasoning
  5. 2025Reasoning Pattern Matters: Learning to Reason without Human Rationales
  6. 2025Towards Efficient CoT Distillation: Self-Guided Rationale Selector for Better Performance with Fewer Rationales
  7. 2026From Implicit to Explicit: Token-Efficient Logical Supervision for Mathematical Reasoning in LLMs
  8. 2025Chain of Execution Supervision Promotes General Reasoning in Large Language Models