mlsanityData to 14 Oct 2026

Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations

arXiv 2505.10937 ↗First posted 16 May 2025cs.CL cs.AIcited by 14

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

The emergence of large reasoning models (LRMs) has transformed Natural Language Processing by excelling in complex tasks such as mathematical problem-solving and code generation. These models leverage chain-of-thought (CoT) processes, enabling them to emulate human-like reasoning strategies. However, the advancement of LRMs is hindered by the lack of comprehensive CoT datasets. Current resources often fail to provide extensive reasoning problems with coherent CoT processes distilled from multiple teacher models and do not account for multifaceted properties describing the internal characteristics of CoTs. To address these challenges, we introduce OmniThought, a large-scale dataset featuring 2 million CoT processes generated and validated by two powerful LRMs as teacher models. Each CoT process in OmniThought is annotated with novel Reasoning Verbosity (RV) and Cognitive Difficulty (CD) scores, which describe the appropriateness of CoT verbosity and cognitive difficulty level for models to comprehend these reasoning processes. We further establish a self-reliant pipeline to curate this dataset. Extensive experiments using Qwen2.5 models of various sizes demonstrate the positive impact of our proposed scores on LRM training effectiveness. Based on the proposed OmniThought dataset, we further train and release a series of high-performing LRMs, specifically equipped with stronger reasoning abilities and optimal CoT output length and difficulty level. Our contributions significantly enhance the development and training of LRMs for solving complex tasks.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 39 of its 46 references that are in the map. First posted on arXiv 16 May 2025.

Builds on

39 of 46 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning25 Feb
  2. 2024A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains1 Feb
  3. 2023Generalizable Chain-of-Thought Prompting in Mixed-task Scenarios with Large Language Models10 Oct
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 34

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2024Qwen2.5 Technical Reportcited by 4,942
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  11. 2023A Survey of Large Language Modelscited by 2,576
  12. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
22 more
  1. 2024A Survey on LLM-as-a-Judgecited by 1,423
  2. 2025s1: Simple test-time scalingcited by 1,341
  3. 2021Measuring Coding Challenge Competence With APPScited by 902
  4. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  5. 2025LIMO: Less is More for Reasoningcited by 498
  6. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 346
  7. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  8. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 284
  9. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 280
  10. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruningcited by 279
  11. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 239
  12. 2024When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Methodcited by 204
  13. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 183
  14. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  15. 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMscited by 163
  16. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoningcited by 119
  17. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 113
  18. 2024OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AIcited by 71
  19. 2024A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Sciencecited by 31
  20. 2023Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughtscited by 27
  21. 2025Enhancing Reasoning Abilities of Small LLMs with Cognitive Alignmentcited by 7
  22. 2023From Complex to Simple: Unraveling the Cognitive Tree for Reasoning with Small Language Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 7.

14 papers.

Sort:

By area:

  1. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  2. 2026Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoningcited by 17
  3. 2025OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Valuecited by 15
  4. 2025DESIGNER: Design-Logic-Guided Multidisciplinary Data Synthesis for LLM Reasoningcited by 6
  5. 2025CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoningcited by 5
  6. 2025Closing the Data Loop: Using OpenDataArena to Engineer Superior Training Datasetscited by 4
  7. 2026Step-CoT: Stepwise Visual Chain-of-Thought for Medical Visual Question Answeringcited by 2
  8. 2026Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMscited by 2
  9. 2025EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Modelscited by 1
  10. 2026OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Modelscited by 1
  11. 2025Thinking with DistilQwen: A Tale of Four Distilled Reasoning and Reward Model Seriescited by 0
  12. 2026NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Researchcited by 0
2 more
  1. 2026Deep Interaction: An Efficient Human-AI Interaction Method for Large Reasoning Modelscited by 0
  2. 2026How the Audit Rule Shapes Faithful Factor Explanations in LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
  3. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
  4. 2025One Missing Piece for Open-Source Reasoning Models: A Dataset to Mitigate Cold-Starting Short CoT LLMs in RL
  5. 2025The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation
  6. 2023ThoughtSource: A central hub for large language model reasoning data
  7. 2025Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation
  8. 2025Innate Reasoning is Not Enough: In-Context Learning Enhances Reasoning Large Language Models with Less Overthinking