mlsanityData to 7 Oct 2026

Enhancing Reasoning Abilities of Small LLMs with Cognitive Alignment

arXiv 2504.09802 ↗First posted 14 Apr 2025cs.CL cs.AIcited by 7

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

The reasoning capabilities of large reasoning models (LRMs), such as OpenAI's o1 and DeepSeek-R1, have seen substantial advancements through deep thinking. However, these enhancements come with significant resource demands, underscoring the need for training effective small reasoning models. A critical challenge is that small models possess different reasoning capacities and cognitive trajectories compared with their larger counterparts. Hence, directly distilling chain-of-thought (CoT) rationales from large LRMs to smaller ones can sometimes be ineffective and often requires a substantial amount of annotated data. In this paper, we first introduce a novel Critique-Rethink-Verify (CRV) system, designed for training smaller yet powerful LRMs. Our CRV system consists of multiple LLM agents, each specializing in unique tasks: (i) critiquing the CoT rationales according to the cognitive capabilities of smaller models, (ii) rethinking and refining these CoTs based on the critiques, and (iii) verifying the correctness of the refined results. Building on the CRV system, we further propose the Cognitive Preference Optimization (CogPO) algorithm to continuously enhance the reasoning abilities of smaller models by aligning their reasoning processes with their cognitive capacities. Comprehensive evaluations on challenging reasoning benchmarks demonstrate the efficacy of our CRV+CogPO framework, which outperforms other methods by a large margin.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 27 of its 48 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

27 of 48 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions21 Nov
  2. 2024Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification5 Oct
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  4. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 23

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2023Let's Verify Step by Stepcited by 3,598
  6. 2023Mistral 7Bcited by 3,065
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  8. 2023A Survey of Large Language Modelscited by 2,537
  9. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  10. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  11. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 256
  12. 2024Self-Play Preference Optimization for Language Model Alignmentcited by 228
11 more
  1. 2024Advancing LLM Reasoning Generalists with Preference Treescited by 191
  2. 2022Distilling Reasoning Capabilities into Smaller Language Modelscited by 155
  3. 2022Explanations from Large Language Models Make Small Reasoners Bettercited by 123
  4. 2024β-DPO: Direct Preference Optimization with Dynamic βcited by 89
  5. 2024Noise Contrastive Alignment of Language Models with Explicit Rewardscited by 85
  6. 2024Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Samplingcited by 60
  7. 2025Distillation Scaling Lawscited by 59
  8. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  9. 2020Explaining Question Answering Models through Text Generationcited by 33
  10. 2023Mixed Distillation Helps Smaller Language Model Better Reasoningcited by 18
  11. 2025A Short Survey on Small Reasoning Models: Training, Inference, Applications and Research Directionscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotationscited by 14
  2. 2025Student-Centered Distillation Narrows the Agentic Gap Between Small and Large LLMscited by 11
  3. 2026Mock Worlds, Real Skills: Building Small Agentic Language Models with Synthetic Tasks, Simulated Environments, and Rubric-Based Rewardscited by 9
  4. 2026A Mechanistic View of Authority Hierarchy in LLM Sycophancycited by 6
  5. 2026AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Usecited by 2
  6. 2025EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Modelscited by 1
  7. 2026Learning What to Learn: Stage-Specific Data Sets for SFT-then-RL in Small Language Model Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
  2. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems
  3. 2025Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities
  4. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
  5. 2025From Long to Short: LLMs Excel at Trimming Own Reasoning Chains
  6. 2025Pruning Long Chain-of-Thought of Large Reasoning Models via Small-Scale Preference Optimization
  7. 2025Gold-Switch: Training-Free Superposition of Slow- and Fast- Thinking LLMs
  8. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models