mlsanityData to 7 Oct 2026

What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective

arXiv 2410.23743 ↗First posted 31 Oct 2024cs.CL cs.AI cs.LGcited by 29

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

What makes a difference in the post-training of LLMs? We investigate the training patterns of different layers in large language models (LLMs) through the lens of the gradient. We are specifically interested in how fast vs. slow thinking affects the layer-wise gradients, given the recent popularity of training LLMs on reasoning paths such as chain-of-thoughts (CoT) and process rewards. In our study, fast thinking without CoT leads to larger gradients and larger differences of gradients across layers than slow thinking (Detailed CoT), indicating the learning stability brought by the latter. Additionally, we study whether the gradient patterns can reflect the correctness of responses when training different LLMs using slow vs. fast thinking paths. The results show that the gradients of slow thinking can distinguish correct and irrelevant reasoning paths. As a comparison, we conduct similar gradient analyses on non-reasoning knowledge learning tasks, on which, however, trivially increasing the response length does not lead to similar behaviors of slow thinking. Our study strengthens fundamental understandings of LLM training and sheds novel insights on its efficiency and stability, which pave the way towards building a generalizable System-2 agent. Our code, data, and gradient statistics can be found in: https://github.com/MingLiiii/Layer_Gradient.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 28 of its 55 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

28 of 55 references.

References · 28

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023A Survey of Large Language Modelscited by 2,537
  6. 2024Qwen2 Technical Reportcited by 2,076
  7. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  8. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  9. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  10. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  11. 2023LIMA: Less Is More for Alignmentcited by 1,080
  12. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
16 more
  1. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  2. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  3. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
  4. 2023Explainability for Large Language Models: A Surveycited by 331
  5. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  6. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 157
  7. 2024Stealing Part of a Production Language Modelcited by 152
  8. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 126
  9. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 110
  10. 2024Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Processcited by 107
  11. 2024Higher Layers Need More LoRA Expertscited by 101
  12. 2024Not All Layers of LLMs Are Necessary During Inferencecited by 95
  13. 2021CREAK: A Dataset for Commonsense Reasoning over Entity Knowledgecited by 88
  14. 2024A Closer Look at the Limitations of Instruction Tuningcited by 76
  15. 2024Streamlining Redundant Layers to Compress Large Language Modelscited by 32
  16. 2024Exploring Concept Depth: How Large Language Models Acquire Knowledge and Concept at Different Layers?cited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 24; 2026: 5.

29 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  3. 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 182
  4. 2025Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?cited by 58
  5. 2025Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinkingcited by 56
  6. 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoningcited by 40
  7. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  8. 2025OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for Over-Reasoning Mitigationcited by 17
  9. 2025From Perceptions to Decisions: Wildfire Evacuation Decision Prediction with Behavioral Theory-informed LLMscited by 12
  10. 2025Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoningcited by 10
  11. 2025CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learnerscited by 9
  12. 2025Re-Emergent Misalignment: How Narrow Fine-Tuning Erodes Safety Alignment in LLMscited by 9
17 more
  1. 2025Understanding the Thinking Process of Reasoning Models: A Perspective from Schoenfeld's Episode Theorycited by 9
  2. 2025How Instruction and Reasoning Data shape Post-Training: Data Quality through the Lens of Layer-wise Gradientscited by 8
  3. 2025What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decodingcited by 7
  4. 2025On the Predictive Power of Representation Dispersion in Language Modelscited by 6
  5. 2026IGU-LoRA: Adaptive Rank Allocation via Integrated Gradients and Uncertainty-Aware Scoringcited by 5
  6. 2025Reasoning Can Hurt the Inductive Abilities of Large Language Modelscited by 4
  7. 2025Spectral Insights into Data-Oblivious Critical Layers in Large Language Modelscited by 4
  8. 2025CaughtCheating: Is Your MLLM a Good Cheating Detective? Exploring the Boundary of Visual Perception and Reasoningcited by 4
  9. 2025Understanding Post-Training Structural Changes in Large Language Modelscited by 2
  10. 2025Can MLLMs Absorb Math Reasoning Abilities from LLMs as Free Lunch?cited by 2
  11. 2025Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modelingcited by 1
  12. 2025FReM: A Flexible Reasoning Mechanism for Balancing Quick and Slow Thinking in Long-Context Question Answeringcited by 1
  13. 2025NeuroBreak: Unveil Internal Jailbreak Mechanisms in Large Language Modelscited by 1
  14. 2026Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligencecited by 1
  15. 2025From Parameters to Performance: A Data-Driven Study on LLM Structure and Developmentcited by 0
  16. 2026Entropy-Gradient Inversion: Moving Toward Internal Mechanism of Large Reasoning Modelscited by 0
  17. 2026MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs
  2. 2025Rectifying LLM Thought from Lens of Optimization
  3. 2026Learning, Fast and Slow: Towards LLMs That Adapt Continually
  4. 2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Law
  5. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning
  6. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Models
  7. 2026Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Training
  8. 2025Short-Path Prompting in LLMs: Analyzing Reasoning Instability and Solutions for Robust Performance