mlsanityData to 14 Oct 2026

Learning to Think from Multiple Thinkers

arXiv 2604.24737 ↗First posted 27 Apr 2026cs.LG cs.AI cs.CCcited by 3

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

We study learning with Chain-of-Thought (CoT) supervision from multiple thinkers, all of whom provide correct but possibly systematically different solutions, e.g., step-by-step solutions to math problems written by different thinkers, or step-by-step execution traces of different programs solving the same problem. We consider classes that are computationally easy to learn using CoT supervision from a single thinker, but hard to learn with only end-result supervision, i.e., without CoT (Joshi et al. 2025). We establish that, under cryptographic assumptions, learning can be hard from CoT supervision provided by two or a few different thinkers, in passive data-collection settings. On the other hand, we provide a generic computationally efficient active learning algorithm that learns with a small amount of CoT data per thinker that is completely independent of the target accuracy varepsilon, a moderate number of thinkers that scales as log frac1varepsilonlog log frac1varepsilon, and sufficient passive end-result data that scales as frac1varepsilon· polylogfrac1varepsilon.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 19 of its 55 references that are in the map. First posted on arXiv 27 Apr 2026.

Builds on

19 of 55 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Sample Complexity of Autoregressive Reasoning: Chain-of-Thought vs. End-to-End13 Apr
  2. 2025Transformers Provably Learn Chain-of-Thought Reasoning with Length Generalization10 Nov
  3. 2025CoT Information: Improved Sample Complexity under Chain-of-Thought Supervision21 May
  4. 2025Reasoning Models Don't Always Say What They Think8 May
  5. 2024Transformers Provably Solve Parity Efficiently with Chain of Thought11 Oct
  6. 2023The Expressive Power of Transformers with Chain of Thought11 Oct

Other references · 13

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2024OpenAI o1 System Cardcited by 1,244
  5. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  6. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 304
  7. 2023Auto-Regressive Next-Token Predictors are Universal Learnerscited by 67
  8. 2025Blending Supervised and Reinforcement Fine-Tuning with Prefix Samplingcited by 60
  9. 2025RL for Reasoning by Adaptively Revealing Rationalescited by 16
  10. 2025How Reinforcement Learning After Next-Token Prediction Facilitates Learningcited by 10
  11. 2026Learning to Reason with Curriculum I: Provable Benefits of Autocurriculumcited by 4
  12. 2025The Probably Approximately Correct Learning Model in Computational Learning Theorycited by 3
1 more
  1. 2025On the Hardness of Learning Regular Expressionscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026A Theory of Online Learning with Autoregressive Chain-of-Thought Reasoningcited by 3
  2. 2026Stochastic Autoregressive Learningcited by 1
  3. 2026The Surprising Expressivity of a Single Neuron with Autoregressive Chain-of-Thoughtcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Latent Reasoning with Supervised Thinking States
  2. 2025Merge-of-Thought Distillation
  3. 2025The Kinetics of Reasoning: How Chain-of-Thought Shapes Learning in Transformers?
  4. 2025From Reasoning to Super-Intelligence: A Search-Theoretic Perspective
  5. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
  6. 2025Mind the Gap: Bridging Thought Leap for Improved Chain-of-Thought Tuning
  7. 2023Design of Chain-of-Thought in Math Problem Solving
  8. 2025Analyzing the Power of Chain of Thought through Memorization Capabilities