mlsanityData to 7 Oct 2026

On the Impact of Fine-Tuning on Chain-of-Thought Reasoning

arXiv 2411.15382 ↗First posted 22 Nov 2024cs.CLcited by 32

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large language models have emerged as powerful tools for general intelligence, showcasing advanced natural language processing capabilities that find applications across diverse domains. Despite their impressive performance, recent studies have highlighted the potential for significant enhancements in LLMs' task-specific performance through fine-tuning strategies like Reinforcement Learning with Human Feedback (RLHF), supervised fine-tuning (SFT), and Quantized Low-Rank Adapters (Q-LoRA) method. However, previous works have shown that while fine-tuning offers significant performance gains, it also leads to challenges such as catastrophic forgetting and privacy and safety risks. To this end, there has been little to no work in textitunderstanding the impact of fine-tuning on the reasoning capabilities of LLMs. Our research investigates the effect of fine-tuning on the reasoning abilities of LLMs, addressing critical questions regarding the impact of task-specific fine-tuning on overall reasoning capabilities, the influence of fine-tuning on Chain-of-Thought (CoT) reasoning performance, and the implications for the faithfulness of CoT reasonings. By exploring these dimensions, our study shows the impact of fine-tuning on LLM reasoning capabilities, where the faithfulness of CoT reasoning, on average across four datasets, decreases, highlighting potential shifts in internal mechanisms of the LLMs resulting from fine-tuning processes.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 25 of its 36 references that are in the map. First posted on arXiv 22 Nov 2024.

Builds on

25 of 36 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  9. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  10. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  11. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  12. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
13 more
  1. 2024GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projectioncited by 447
  2. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 281
  3. 2024When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Methodcited by 201
  4. 2023DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Servicescited by 117
  5. 2024Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Modelscited by 104
  6. 2020A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrixcited by 99
  7. 2024A Survey of Large Language Models in Finance (FinLLMs)cited by 77
  8. 2023The Next Chapter: A Study of Large Language Models in Storytellingcited by 58
  9. 2024Injecting New Knowledge into Large Language Models via Supervised Fine-Tuningcited by 56
  10. 2023Exploring Memorization in Fine-tuned Language Modelscited by 44
  11. 2024Few shot chain-of-thought driven reasoning to prompt LLMs for open ended medical question answeringcited by 31
  12. 2024Fine-Tuning, Quantization, and LLMs: Navigating Unintended Outcomescited by 14
  13. 2023Prototyping the use of Large Language Models (LLMs) for adult learning content creation at scalecited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 18; 2026: 12.

32 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  3. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  4. 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMscited by 110
  5. 2025Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoningcited by 101
  6. 2025A Survey of Multilingual Reasoning in Language Modelscited by 31
  7. 2025Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skillscited by 24
  8. 2025Chain-of-Thought for Autonomous Driving: A Comprehensive Survey and Future Prospectscited by 12
  9. 2025Revealing the Power of Post-Training for Small Language Models via Knowledge Distillationcited by 6
  10. 2026SeLaR: Selective Latent Reasoning in Large Language Modelscited by 6
  11. 2024Adaptive Parameter-Efficient Federated Fine-Tuning on Heterogeneous Devicescited by 5
  12. 2025SHARP: Synthesizing High-quality Aligned Reasoning Problems for Large Reasoning Models Reinforcement Learningcited by 5
20 more
  1. 2025Reasoning by Exploration: A Unified Approach to Retrieval and Generation over Graphscited by 5
  2. 2026Reference-guided Policy Optimization for Molecular Optimization via LLM Reasoningcited by 5
  3. 2025Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMscited by 4
  4. 2026Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuningcited by 4
  5. 2025Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Codecited by 3
  6. 2024DynaGRAG | Exploring the Topology of Information for Advancing Language Understanding and Generation in Graph Retrieval-Augmented Generationcited by 1
  7. 2025Uncovering inequalities in new knowledge learning by large language models across different languagescited by 1
  8. 2025Improving Latent Reasoning in LLMs via Soft Concept Mixingcited by 1
  9. 2026How Clued up are LLMs? Evaluating Multi-Step Deductive Reasoning in a Text-Based Game Environmentcited by 1
  10. 2026Comparative Analysis of Large Language Models in Healthcarecited by 1
  11. 2025Learning to Choose or Choosing to Learn: Best-of-N vs. Supervised Fine-Tuning for Bit String Generationcited by 0
  12. 2025Position: Pause Recycling LoRAs and Prioritize Mechanisms to Uncover Limits and Effectivenesscited by 0
  13. 2025VCORE: Variance-Controlled Optimization-based Reweighting for Chain-of-Thought Supervisioncited by 0
  14. 2026FewMMBench: A Benchmark for Multimodal Few-Shot Learningcited by 0
  15. 2026Can You Break RLVER? Probing Adversarial Robustness of RL-Trained Empathetic Agentscited by 0
  16. 2026MemReread: Enhancing Agentic Long-Context Reasoning via Memory-Guided Rereadingcited by 0
  17. 2026Towards the Next Frontier of LLMs, Training on Private Data: A Cross-Domain Benchmark for Federated Fine-Tuningcited by 0
  18. 2026Decomposing Refusal Steering in Mixture-of-Experts Modelscited by 0
  19. 2026Reason, Reward, Refine: Step-Level Errors Corrections with Structured Feedback for Physics Reasoning in Small Language Modelscited by 0
  20. 2026Leveraging Instruction Tuning and Merging for Reasoning Model Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CARFT: Boosting LLM Reasoning via Contrastive Learning with Annotated Chain-of-Thought-based Reinforced Fine-Tuning
  2. 2025Unleashing the Reasoning Potential of Pre-trained LLMs by Critique Fine-Tuning on One Problem
  3. 2025How and Why LLMs Generalize: A Fine-Grained Analysis of LLM Reasoning from Cognitive Behaviors to Low-Level Patterns
  4. 2025Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism
  5. 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
  6. 2024ReFT: Reasoning with Reinforced Fine-Tuning
  7. 2026Dual-Phase LLM Reasoning: Self-Evolved Mathematical Frameworks
  8. 2025Slimming Down LLMs Without Losing Their Minds