mlsanityData to 14 Oct 2026

Self-Steering Language Models

arXiv 2504.07081 ↗First posted 9 Apr 2025cs.CL cs.AIcited by 13

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

While test-time reasoning enables language models (LMs) to tackle complex tasks, searching or planning in natural language can be slow, costly, and error-prone. But even when LMs struggle to emulate the precise reasoning steps needed to solve a problem, they often excel at describing its abstract structure--both how to verify solutions and how to search for them. This paper introduces DisCIPL, a method for "self-steering" LMs where a Planner model generates a task-specific inference program that is executed by a population of Follower models. Our approach equips LMs with the ability to write recursive search procedures that guide LM inference, enabling new forms of verifiable and efficient reasoning. When instantiated with a small Follower (e.g., Llama-3.2-1B or Qwen3-1.7B), DisCIPL matches (and sometimes outperforms) much larger models, including GPT-4o and o1, on challenging constrained generation tasks. Our work opens up a design space of highly-parallelized Monte Carlo inference strategies that outperform standard best-of-N sampling, require no finetuning, and can be implemented automatically by existing LMs.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 56 of its 76 references that are in the map. First posted on arXiv 9 Apr 2025.

Builds on

56 of 76 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Learning How Hard to Think: Input-Adaptive Allocation of LM Computation7 Oct
  2. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  3. 2024Stream of Search (SoS): Learning to Search in Language1 Apr
  4. 2023Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation28 Jul
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  6. 2022STaR: Bootstrapping Reasoning With Reasoning28 Mar

Other references · 50

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  8. 2023Let's Verify Step by Stepcited by 3,789
  9. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  10. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  11. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  12. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
38 more
  1. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  2. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  3. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  4. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  5. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  6. 2023Large Language Models as Optimizerscited by 864
  7. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  8. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
  9. 2022Generating Sequences by Learning to Self-Correctcited by 277
  10. 2017Lexically Constrained Decoding for Sequence Generation Using Grid Beam Searchcited by 237
  11. 2023Efficient Guided Generation for Large Language Modelscited by 200
  12. 2018Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translationcited by 182
  13. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 178
  14. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
  15. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  16. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 154
  17. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  18. 2020NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraintscited by 149
  19. 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 148
  20. 2023FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Modelscited by 117
  21. 2024Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrappingcited by 90
  22. 2024Probabilistic Inference in Language Models via Twisted Sequential Monte Carlocited by 90
  23. 2023Controlled Text Generation with Natural Language Instructionscited by 87
  24. 2023Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programscited by 80
  25. 2023Evaluating Large Language Models on Controlled Generation Taskscited by 73
  26. 2023COLLIE: Systematic Construction of Constrained Text Generation Taskscited by 70
  27. 2022Gradient-Based Constrained Sampling from Language Modelscited by 64
  28. 2024Grammar-Aligned Decodingcited by 54
  29. 2024Automata-based constraints for language model decodingcited by 51
  30. 2023LILO: Learning Interpretable Libraries by Compressing and Documenting Codecited by 49
  31. 2024Automated Statistical Model Discovery with Language Modelscited by 41
  32. 2025Hogwild! Inference: Parallel LLM Generation via Concurrent Attentioncited by 32
  33. 2024From Language Models over Tokens to Language Models over Characterscited by 31
  34. 2025Learning to Keep a Promise: Scaling Language Model Decoding Parallelism with Learned Asynchronous Decodingcited by 28
  35. 2024Step-by-Step Reasoning for Math Problems via Twisted Sequential Monte Carlocited by 22
  36. 2014Venture: a higher-order probabilistic programming platform with programmable inferencecited by 20
  37. 2024Can We Count on LLMs? The Fixed-Effect Fallacy and Claims of GPT-4 Capabilitiescited by 13
  38. 2023Structured Voronoi Samplingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 8.

13 papers.

Sort:

By area:

  1. 2025Recursive Language Modelscited by 104
  2. 2025Learning Adaptive Parallel Reasoning with Language Modelscited by 58
  3. 2026AOrchestra: Automating Sub-Agent Creation for Agentic Orchestrationcited by 35
  4. 2025Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Modelscited by 14
  5. 2026A Mechanistic View of Authority Hierarchy in LLM Sycophancycited by 9
  6. 2025ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Modelscited by 8
  7. 2025Code-enabled language models can outperform reasoning models on diverse taskscited by 4
  8. 2026Reject, Resample, Repeat: Understanding Parallel Reasoning in Language Model Inferencecited by 3
  9. 2026SPIRAL: Learning to Search and Aggregatecited by 3
  10. 2026Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Explorationcited by 1
  11. 2026Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recoverycited by 1
  12. 2026When Less is Enough: Efficient Inference via Collaborative Reasoningcited by 0
1 more
  1. 2026Position: Reasoning is a Learnable Rule-Based Processcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Understanding Reasoning in Thinking Language Models via Steering Vectors
  2. 2026Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering
  3. 2026TabTracer: Monte Carlo Tree Search for Complex Table Reasoning with Large Language Models
  4. 2026When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Models
  5. 2025Prototype-Based Dynamic Steering for Large Language Models
  6. 2025Self-Reasoning Language Models: Unfold Hidden Reasoning Chains with Few Reasoning Catalyst
  7. 2025Incentivizing LLMs to Self-Verify Their Answers
  8. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility