Data to 14 Oct 2026

Thinking LLMs: General Instruction Following with Thought Generation

arXiv 2410.10630 ↗First posted 14 Oct 2024cs.CL cs.AIcited by 45

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

LLMs are typically trained to answer user questions or follow instructions similarly to how human experts respond. However, in the standard alignment framework they lack the basic ability of explicit thinking before answering. Thinking is important for complex questions that require reasoning and planning -- but can be applied to any task. We propose a training method for equipping existing LLMs with such thinking abilities for general instruction following without use of additional human data. We achieve this by an iterative search and optimization procedure that explores the space of possible thought generations, allowing the model to learn how to think without direct supervision. For each instruction, the thought candidates are scored using a judge model to evaluate their responses only, and then optimized via preference optimization. We show that this procedure leads to superior performance on AlpacaEval and Arena-Hard, and shows gains from thinking on non-reasoning categories such as marketing, health and general knowledge, in addition to more traditional reasoning & problem-solving tasks.

Key facts. Cited by 45 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 31 of its 38 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

31 of 38 references.

Closest ancestors · 7

Same direction, newest first

Other references · 24

2017Attention Is All You Needcited by 60,361
2024The Llama 3 Herd of Modelscited by 14,715
2021Training Verifiers to Solve Math Word Problemscited by 10,109
2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
2024Self-Rewarding Language Modelscited by 681
2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 500

12 more

2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
2024Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Expertscited by 376
2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 324
2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
2024Iterative Reasoning Preference Optimizationcited by 246
2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 217
2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 204
2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
2023System 2 Attention (is something you might need too)cited by 89
2024Following Length Constraints in Instructionscited by 46
2024Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Textscited by 31
2023Learning to Reason and Memorize with Self-Notescited by 30

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 37; 2026: 6.

45 papers.

Sort:

By area:

2025s1: Simple test-time scalingcited by 1,341
2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
2025Grounded Reinforcement Learning for Visual Reasoningcited by 83
2025Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judgecited by 78
2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 77
2024The Limits of Inference Scaling Through Resamplingcited by 53
2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 42
2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 40
2025The Homogenizing Effect of Large Language Models on Human Expression and Thoughtcited by 35
2025STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for Spoken Language Modelscited by 26
2025Bridging Offline and Online Reinforcement Learning for LLMscited by 25

33 more

2024From Medprompt to o1: Exploration of Run-Time Strategies for Medical Challenge Problems and Beyondcited by 24
2025Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedbackcited by 22
2025DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversalcited by 20
2025SOTOPIA-Omega: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agentscited by 16
2025ThinkPatterns-21k: A Systematic Study on the Impact of Thinking Patterns in LLMscited by 16
2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
2025RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguardscited by 14
2026BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluationcited by 14
2025Layered Chain-of-Thought Prompting for Multi-Agent LLM Systems: A Comprehensive Approach to Explainable Large Language Modelscited by 10
2025BRiTE: Bootstrapping Reinforced Thinking Process to Enhance Language Model Reasoningcited by 10
2025Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Surveycited by 10
2025GraphThought: Graph Combinatorial Optimization with Thought Generationcited by 9
2025Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanationscited by 9
2025Automated Knowledge Graph Construction using Large Language Models and Sentence Complexity Modellingcited by 9
2025Learning to Pose Problems: Reasoning-Driven and Solver-Adaptive Data Synthesiscited by 8
2026The Fragility Of Moral Judgment In Large Language Modelscited by 8
2025Reasoning Up the Instruction Ladder for Controllable Language Modelscited by 5
2025Be Cautious When Merging Unfamiliar LLMs: A Phishing Model Capable of Stealing Privacycited by 4
2025UserCentrix: An Agentic Memory-augmented AI Framework for Smart Spacescited by 4
2025Typhoon T1: An Open Thai Reasoning Modelcited by 3
2025Is Visual in-Context Learning for Compositional Medical Tasks within Reach?cited by 3
2026Evaluating Cooperation in LLM Social Groups through Elected Leadershipcited by 3
2025Enhancing Reasoning Skills in Small Persian Medical Language Models Can Outperform Large-Scale Data Trainingcited by 2
2025ORION: Teaching Language Models to Reason Efficiently in the Language of Thoughtcited by 1
2026COGTRL: Training LLMs for Scientific Discovery Assistance using Cognitive Traces via Reinforcement Learningcited by 1
2025Generating Privacy Stories From Software Documentationcited by 0
2025Enhancing LLM Reasoning via Non-Human-Like Reasoning Path Preference Optimizationcited by 0
2025Boosting Instruction Following at Scalecited by 0
2025Bradley-Terry Policy Optimization for Generative Preference Modelingcited by 0
2025Instruction-Tuning Open-Weight Language Models for BPMN Model Generationcited by 0
2026MESA: Improving MoE Safety Alignment via Decentralized Expertisecited by 0
2026When Built-in Thinking Helps and Hurts: Constraint-Level Error Shifts in Instruction Followingcited by 0

Similar, not cited

Close in wording, no citation either way