mlsanityData to 7 Oct 2026

Improving Language Model Reasoning with Self-motivated Learning

arXiv 2404.07017 ↗First posted 10 Apr 2024cs.CL cs.AIcited by 4

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large-scale high-quality training data is important for improving the performance of models. After trained with data that has rationales (reasoning steps), models gain reasoning capability. However, the dataset with high-quality rationales is relatively scarce due to the high annotation cost. To address this issue, we propose textitSelf-motivated Learning framework. The framework motivates the model itself to automatically generate rationales on existing datasets. Based on the inherent rank from correctness across multiple rationales, the model learns to generate better rationales, leading to higher reasoning capability. Specifically, we train a reward model with the rank to evaluate the quality of rationales, and improve the performance of reasoning through reinforcement learning. Experiment results of Llama2 7B on multiple reasoning datasets show that our method significantly improves the reasoning ability of models, even outperforming text-davinci-002 in some datasets.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 27 of its 73 references that are in the map. First posted on arXiv 10 Apr 2024.

Builds on

27 of 73 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022STaR: Bootstrapping Reasoning With Reasoning28 Mar

Other references · 26

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2023Let's Verify Step by Stepcited by 3,598
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
14 more
  1. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  2. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  3. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  4. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  5. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  6. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  7. 2019Abductive Commonsense Reasoningcited by 300
  8. 2022Can language models learn from explanations in context?cited by 248
  9. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 211
  10. 2022Defining and Characterizing Reward Hackingcited by 184
  11. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 106
  12. 2020LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoningcited by 68
  13. 2022Reasoning Like Program Executorscited by 45
  14. 2023Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Followingcited by 33

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 1.

4 papers.

  1. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  2. 2024Creative and Context-Aware Translation of East Asian Idioms with GPT-4cited by 10
  3. 2024DAC: Decomposed Automation Correction for Text-to-SQLcited by 6
  4. 2025Learned-Rule-Augmented Large Language Model Evaluatorscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained Rewards
  2. 2025Training Language Models to Reason Efficiently
  3. 2025SPaCe: Unlocking Sample-Efficient Large Language Models Training With Self-Pace Curriculum Learning
  4. 2025Code-enabled language models can outperform reasoning models on diverse tasks
  5. 2026Mid-Training with Self-Generated Data Improves Reinforcement Learning in Language Models
  6. 2025Self-rewarding correction for mathematical reasoning
  7. 2024Self-Training Meets Consistency: Improving LLMs' Reasoning with Consistency-Driven Rationale Evaluation
  8. 2025A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning