mlsanityData to 7 Oct 2026

Distilling System 2 into System 1

arXiv 2407.06023 ↗First posted 8 Jul 2024cs.CL cs.AIcited by 152

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large language models (LLMs) can spend extra compute during inference to generate intermediate thoughts, which helps to produce better final responses. Since Chain-of-Thought (Wei et al., 2022), many such System 2 techniques have been proposed such as Rephrase and Respond (Deng et al., 2023a), System 2 Attention (Weston and Sukhbaatar, 2023) and Branch-Solve-Merge (Saha et al., 2023). In this work we investigate self-supervised methods to ``compile'' (distill) higher quality outputs from System 2 techniques back into LLM generations without intermediate reasoning token sequences, as this reasoning has been distilled into System 1. We show that several such techniques can be successfully distilled, resulting in improved results compared to the original System 1 performance, and with less inference cost than System 2. We posit that such System 2 distillation will be an important feature of future continually learning AI systems, enabling them to focus System 2 capabilities on the reasoning tasks that they cannot yet do well.

Key facts. Cited by 152 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 32 of its 50 references that are in the map. First posted on arXiv 8 Jul 2024.

Builds on

32 of 50 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs13 Jun
  2. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step23 May
  3. 2024Learning to Maximize Mutual Information for Chain-of-Thought Distillation5 Mar
  4. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problems20 Feb
  5. 2023Implicit Chain of Thought Reasoning via Knowledge Distillation2 Nov
  6. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Models18 Aug
  7. 2023Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step24 Jun
  8. 2023SCOTT: Self-Consistent Chain-of-Thought Distillation3 May
  9. 2022Large Language Models are Better Reasoners with Self-Verification19 Dec
  10. 2022Large Language Models are Zero-Shot Reasoners24 May
  11. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar

Other references · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  7. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  8. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  9. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  10. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  11. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  12. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
9 more
  1. 2013Do Deep Nets Really Need to be Deep?cited by 466
  2. 2023Universal Self-Consistency for Large Language Model Generationcited by 198
  3. 2017Sobolev Training for Neural Networkscited by 136
  4. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  5. 2020Unsupervised Question Decomposition for Question Answeringcited by 118
  6. 2017The Consciousness Priorcited by 105
  7. 2023Branch-Solve-Merge Improves Large Language Model Evaluation and Generationcited by 104
  8. 2023System 2 Attention (is something you might need too)cited by 85
  9. 2021A Survey on Neural-symbolic Learning Systemscited by 39

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 23; 2025: 93; 2026: 36.

152 papers.

Sort:

By area:

  1. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  2. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  3. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  4. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  5. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  6. 2025Reasoning Models Can Be Effective Without Thinkingcited by 182
  7. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuningcited by 167
  8. 2024LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMscited by 142
  9. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  10. 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokenscited by 120
  11. 2025Process Reward Models That Thinkcited by 118
  12. 2025Thinkless: LLM Learns When to Thinkcited by 107
38 more
  1. 2025Agentic Large Language Models, a surveycited by 105
  2. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approachcited by 95
  3. 2025Efficient Reasoning Models: A Surveycited by 93
  4. 2025Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoningcited by 87
  5. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  6. 2025AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learningcited by 86
  7. 2026Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agentscited by 75
  8. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoningcited by 72
  9. 2025Wait, We Don't Need to "Wait"! Removing Thinking Tokens Improves Reasoning Efficiencycited by 65
  10. 2025Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognitioncited by 61
  11. 2024Can Language Models Learn to Skip Steps?cited by 59
  12. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 58
  13. 2025Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Modelscited by 52
  14. 2025AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgetingcited by 50
  15. 2024Thinking LLMs: General Instruction Following with Thought Generationcited by 45
  16. 2024Disentangling Memory and Reasoning Ability in Large Language Modelscited by 43
  17. 2024Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Tracescited by 40
  18. 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoningcited by 40
  19. 2025Optimizing Length Compression in Large Reasoning Modelscited by 37
  20. 2025Bayesian Teaching Enables Probabilistic Reasoning in Large Language Modelscited by 36
  21. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  22. 2024LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraintscited by 34
  23. 2024EVOLvE: Evaluating and Optimizing LLMs For In-Context Explorationcited by 33
  24. 2025LLM Pretraining with Continuous Conceptscited by 33
  25. 2025Scalable Chain of Thoughts via Elastic Reasoningcited by 32
  26. 2025Continuous Chain of Thought Enables Parallel Exploration and Reasoningcited by 32
  27. 2025Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layerscited by 32
  28. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  29. 2024What Makes Large Language Models Reason in (Multi-Turn) Code Generation?cited by 29
  30. 2025Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinkingcited by 29
  31. 2025NaturalThoughts: Selecting and Distilling Reasoning Traces for General Reasoning Taskscited by 29
  32. 2025Entropy-based Exploration Conduction for Multi-step Reasoningcited by 28
  33. 2024Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Criticcited by 27
  34. 2025Let LRMs Break Free from Overthinking via Self-Braking Tuningcited by 27
  35. 2025Shorten After You're Right: Lazy Length Penalties for Reasoning RLcited by 25
  36. 2025System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcutscited by 23
  37. 2026Purified OPSD: On-Policy Self-Distillation Without Losing How to Thinkcited by 23
  38. 2024Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model Enhancementcited by 21

50 most cited of 152 All 152 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025From System 1 to System 2: A Survey of Reasoning Large Language Models
  2. 2024Divide-or-Conquer? Which Part Should You Distill Your LLM?
  3. 2025Towards Understanding Distilled Reasoning Models: A Representational Approach
  4. 2023Mixed Distillation Helps Smaller Language Model Better Reasoning
  5. 2026Benchmarking Distilled Language Models: Performance and Efficiency in Resource-Constrained Settings
  6. 2025Distilling LLM Agent into Small Models with Retrieval and Code Tools
  7. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
  8. 2026Masked Distillation: Internalizing the Chain-of-Thought in Language Models