# CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning

arXiv: https://arxiv.org/abs/2510.13166 · HTML: https://mlsanity.com/paper/2510.13166/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 15 Oct 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 29 of 29)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models](https://mlsanity.com/paper/2304.06364/) (2023)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [Llama-Nemotron: Efficient Reasoning Models](https://mlsanity.com/paper/2505.00949/) (2025)
- [Intern-S1: A Scientific Multimodal Foundation Model](https://mlsanity.com/paper/2508.15763/) (2025)
- [Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operations](https://mlsanity.com/paper/2505.21318/) (2025)
- [SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?](https://mlsanity.com/paper/2507.05241/) (2025)
- [Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation](https://mlsanity.com/paper/2503.16385/) (2025)
- [Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoning](https://mlsanity.com/paper/2504.04383/) (2025)
- [Reverse Thinking Makes LLMs Stronger Reasoners](https://mlsanity.com/paper/2411.19865/) (2024)
- [ChemDFM-R: A Chemical Reasoning LLM Enhanced with Atomized Chemical Knowledge](https://mlsanity.com/paper/2507.21990/) (2025)
- [SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models](https://mlsanity.com/paper/2406.09098/) (2024)
- [BioProBench: A Corpus and Benchmark for Biological Protocol Reasoning in Autonomous Science](https://mlsanity.com/paper/2505.07889/) (2025)
- [Why Distillation can Outperform Zero-RL: The Role of Flexible Reasoning](https://mlsanity.com/paper/2505.21067/) (2025)
- [Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations](https://mlsanity.com/paper/2505.10937/) (2025)
- [Learning from Diverse Reasoning Paths with Routing and Collaboration](https://mlsanity.com/paper/2508.16861/) (2025)
- [TwT: Thinking without Tokens by Habitual Reasoning Distillation with Multi-Teachers' Guidance](https://mlsanity.com/paper/2503.24198/) (2025)
- [BioMaze: Benchmarking and Enhancing Large Language Models for Biological Pathway Reasoning](https://mlsanity.com/paper/2502.16660/) (2025)
- [Efficient Long CoT Reasoning in Small Language Models](https://mlsanity.com/paper/2505.18440/) (2025)

## Built on by (most cited first, 5 of 5)

- [Context-CoT: Enhancing Context Learning via High-Quality Reasoning Synthesis](https://mlsanity.com/paper/2605.25354/) (2026)
- [SCA: Segment-Wise CoT Compression with Answer Alignment](https://mlsanity.com/paper/2603.07598/) (2026)
- [Probing to Refine: Reinforcement Distillation of LLMs via Explanatory Inversion](https://mlsanity.com/paper/2603.19266/) (2026)
- [How Post-Training Shapes Biological Reasoning Models](https://mlsanity.com/paper/2606.16517/) (2026)
- [Learning to Adapt SFT Data for Better Reasoning Generalization](https://mlsanity.com/paper/2605.26924/) (2026)

## Abstract (arXiv)

While chain-of-thought (CoT) distillation from advanced large language models (LLMs) has proven effective in general reasoning tasks, it struggles in scientific domains where even advanced models often produce incorrect or superficial reasoning due to high complexity and specialized knowledge requirements. Directly distilling from such flawed outputs results in low-quality training data and limits the performance of smaller student models. To overcome this, we propose CoT-Evo, an evolutionary CoT distillation framework. It begins by constructing a diverse pool of reasoning trajectories from multiple LLM thinkers, enriches them with automatically retrieved domain knowledge, and iteratively refines the trajectories using novelty-driven selection, reflective recombination and mutation. The refinement is guided by a fitness function that evaluates answer correctness, coherence, and effective knowledge utilization. This results in a high-quality CoT dataset tailored for scientific reasoning. We employ this evolved dataset to fine-tune a compact model, which achieves state-of-the-art performance on scientific reasoning benchmarks. Our work establishes a scalable approach to synthesizing high-fidelity scientific reasoning data from diverse and fallible LLMs.