# What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective

arXiv: https://arxiv.org/abs/2410.23743 · HTML: https://mlsanity.com/paper/2410.23743/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 28 of its 55 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 28 of 28)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [The Flan Collection: Designing Data and Methods for Effective Instruction Tuning](https://mlsanity.com/paper/2301.13688/) (2023)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [ShortGPT: Layers in Large Language Models are More Redundant Than You Expect](https://mlsanity.com/paper/2403.03853/) (2024)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [TrustLLM: Trustworthiness in Large Language Models](https://mlsanity.com/paper/2401.05561/) (2024)
- [Orca 2: Teaching Small Language Models How to Reason](https://mlsanity.com/paper/2311.11045/) (2023)
- [Stealing Part of a Production Language Model](https://mlsanity.com/paper/2403.06634/) (2024)
- [Safety Layers in Aligned Large Language Models: The Key to LLM Security](https://mlsanity.com/paper/2408.17003/) (2024)
- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process](https://mlsanity.com/paper/2407.20311/) (2024)
- [Higher Layers Need More LoRA Experts](https://mlsanity.com/paper/2402.08562/) (2024)
- [Not All Layers of LLMs Are Necessary During Inference](https://mlsanity.com/paper/2403.02181/) (2024)
- [CREAK: A Dataset for Commonsense Reasoning over Entity Knowledge](https://mlsanity.com/paper/2109.01653/) (2021)
- [A Closer Look at the Limitations of Instruction Tuning](https://mlsanity.com/paper/2402.05119/) (2024)
- [Streamlining Redundant Layers to Compress Large Language Models](https://mlsanity.com/paper/2403.19135/) (2024)
- [Exploring Concept Depth: How Large Language Models Acquire Knowledge and Concept at Different Layers?](https://mlsanity.com/paper/2404.07066/) (2024)

## Built on by (most cited first, 29 of 29)

- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [From System 1 to System 2: A Survey of Reasoning Large Language Models](https://mlsanity.com/paper/2502.17419/) (2025)
- [When More is Less: Understanding Chain-of-Thought Length in LLMs](https://mlsanity.com/paper/2502.07266/) (2025)
- [Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?](https://mlsanity.com/paper/2504.06514/) (2025)
- [Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking](https://mlsanity.com/paper/2502.13842/) (2025)
- [Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning](https://mlsanity.com/paper/2502.18001/) (2025)
- [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026)
- [OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for Over-Reasoning Mitigation](https://mlsanity.com/paper/2506.02397/) (2025)
- [From Perceptions to Decisions: Wildfire Evacuation Decision Prediction with Behavioral Theory-informed LLMs](https://mlsanity.com/paper/2502.17701/) (2025)
- [Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoning](https://mlsanity.com/paper/2502.04667/) (2025)
- [CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners](https://mlsanity.com/paper/2503.16356/) (2025)
- [Re-Emergent Misalignment: How Narrow Fine-Tuning Erodes Safety Alignment in LLMs](https://mlsanity.com/paper/2507.03662/) (2025)
- [Understanding the Thinking Process of Reasoning Models: A Perspective from Schoenfeld's Episode Theory](https://mlsanity.com/paper/2509.14662/) (2025)
- [How Instruction and Reasoning Data shape Post-Training: Data Quality through the Lens of Layer-wise Gradients](https://mlsanity.com/paper/2504.10766/) (2025)
- [What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decoding](https://mlsanity.com/paper/2506.06998/) (2025)
- [On the Predictive Power of Representation Dispersion in Language Models](https://mlsanity.com/paper/2506.24106/) (2025)
- [IGU-LoRA: Adaptive Rank Allocation via Integrated Gradients and Uncertainty-Aware Scoring](https://mlsanity.com/paper/2603.13792/) (2026)
- [Reasoning Can Hurt the Inductive Abilities of Large Language Models](https://mlsanity.com/paper/2505.24225/) (2025)
- [Spectral Insights into Data-Oblivious Critical Layers in Large Language Models](https://mlsanity.com/paper/2506.00382/) (2025)
- [CaughtCheating: Is Your MLLM a Good Cheating Detective? Exploring the Boundary of Visual Perception and Reasoning](https://mlsanity.com/paper/2507.00045/) (2025)
- [Understanding Post-Training Structural Changes in Large Language Models](https://mlsanity.com/paper/2509.17866/) (2025)
- [Can MLLMs Absorb Math Reasoning Abilities from LLMs as Free Lunch?](https://mlsanity.com/paper/2510.14387/) (2025)
- [Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence](https://mlsanity.com/paper/2608.12036/) (2026)
- [Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling](https://mlsanity.com/paper/2503.04355/) (2025)
- [FReM: A Flexible Reasoning Mechanism for Balancing Quick and Slow Thinking in Long-Context Question Answering](https://mlsanity.com/paper/2503.22985/) (2025)
- [NeuroBreak: Unveil Internal Jailbreak Mechanisms in Large Language Models](https://mlsanity.com/paper/2509.03985/) (2025)
- [From Parameters to Performance: A Data-Driven Study on LLM Structure and Development](https://mlsanity.com/paper/2509.18136/) (2025)
- [Entropy-Gradient Inversion: Moving Toward Internal Mechanism of Large Reasoning Models](https://mlsanity.com/paper/2605.17770/) (2026)
- [MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation](https://mlsanity.com/paper/2608.29623/) (2026)

## Abstract (arXiv)

What makes a difference in the post-training of LLMs? We investigate the training patterns of different layers in large language models (LLMs) through the lens of the gradient. We are specifically interested in how fast vs. slow thinking affects the layer-wise gradients, given the recent popularity of training LLMs on reasoning paths such as chain-of-thoughts (CoT) and process rewards. In our study, fast thinking without CoT leads to larger gradients and larger differences of gradients across layers than slow thinking (Detailed CoT), indicating the learning stability brought by the latter. Additionally, we study whether the gradient patterns can reflect the correctness of responses when training different LLMs using slow vs. fast thinking paths. The results show that the gradients of slow thinking can distinguish correct and irrelevant reasoning paths. As a comparison, we conduct similar gradient analyses on non-reasoning knowledge learning tasks, on which, however, trivially increasing the response length does not lead to similar behaviors of slow thinking. Our study strengthens fundamental understandings of LLM training and sheds novel insights on its efficiency and stability, which pave the way towards building a generalizable System-2 agent. Our code, data, and gradient statistics can be found in: https://github.com/MingLiiii/Layer_Gradient.