mlsanityData to 14 Oct 2026

DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning

arXiv 2504.07128 ↗First posted 2 Apr 2025cs.CLcited by 92

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large Reasoning Models like DeepSeek-R1 mark a fundamental shift in how LLMs approach complex problems. Instead of directly producing an answer for a given input, DeepSeek-R1 creates detailed multi-step reasoning chains, seemingly "thinking" about a problem before providing an answer. This reasoning process is publicly available to the user, creating endless opportunities for studying the reasoning behaviour of the model and opening up the field of Thoughtology. Starting from a taxonomy of DeepSeek-R1's basic building blocks of reasoning, our analyses on DeepSeek-R1 investigate the impact and controllability of thought length, management of long or confusing contexts, cultural and safety concerns, and the status of DeepSeek-R1 vis-`a-vis cognitive phenomena, such as human-like language processing and world modelling. Our findings paint a nuanced picture. Notably, we show DeepSeek-R1 has a 'sweet spot' of reasoning, where extra inference time can impair model performance. Furthermore, we find a tendency for DeepSeek-R1 to persistently ruminate on previously explored problem formulations, obstructing further exploration. We also note strong safety vulnerabilities of DeepSeek-R1 compared to its non-reasoning counterpart, which can also compromise safety-aligned LLMs.

Key facts. Cited by 92 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 35 of its 143 references that are in the map. First posted on arXiv 2 Apr 2025.

Builds on

35 of 143 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025From System 1 to System 2: A Survey of Reasoning Large Language Models24 Feb
  2. 2024DeepSeek-V3 Technical Report27 Dec
  3. 2023Probing the Moral Development of Large Language Models through Defining Issues Test23 Sep

Other references · 32

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  9. 2023Let's Verify Step by Stepcited by 3,789
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  11. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  12. 2025s1: Simple test-time scalingcited by 1,341
20 more
  1. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  2. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  3. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  4. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  5. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 384
  6. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
  7. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
  8. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 183
  9. 2024Evaluation of OpenAI o1: Opportunities and Challenges of AGIcited by 127
  10. 2025Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuningcited by 123
  11. 2022Teaching Algorithmic Reasoning via In-context Learningcited by 112
  12. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 109
  13. 2024Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Modelscited by 70
  14. 2025Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?cited by 70
  15. 2024Large Language Models Reflect the Ideology of their Creatorscited by 57
  16. 2024LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Outputcited by 27
  17. 2025U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystackcited by 20
  18. 2025One ruler to measure them all: Benchmarking multilingual long-context language modelscited by 19
  19. 2025How to Get Your LLM to Generate Challenging Problems for Evaluationcited by 11
  20. 2025When the LM misunderstood the human chuckled: Analyzing garden path effects in humans and language modelscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 60; 2026: 32.

92 papers.

Sort:

By area:

  1. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 302
  2. 2025e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMscited by 59
  3. 2025Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Modelscited by 53
  4. 2025Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modelingcited by 45
  5. 2025Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokenscited by 41
  6. 2025Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!cited by 37
  7. 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
  8. 2025Base Models Know How to Reason, Thinking Models Learn Whencited by 27
  9. 2025Cognitive Foundations for Reasoning and Their Manifestation in LLMscited by 26
  10. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  11. 2025What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoTcited by 25
  12. 2025REARANK: Reasoning Re-ranking Agent via Reinforcement Learningcited by 24
38 more
  1. 2025Measuring the Faithfulness of Thinking Drafts in Large Reasoning Modelscited by 23
  2. 2025Reasoning as an Adaptive Defense for Safetycited by 23
  3. 2025Trace Length is a Simple Uncertainty Signal in Reasoning Modelscited by 19
  4. 2025Boosting LLM Reasoning via Spontaneous Self-Correctioncited by 18
  5. 2025Learning from Peers in Reasoning Modelscited by 16
  6. 2026Controllable LLM Reasoning via Sparse Autoencoder-Based Steeringcited by 15
  7. 2025Guiding Reasoning in Small Language Models with LLM Assistancecited by 14
  8. 2025TrimR: Verifier-based Training-Free Thinking Compression for Efficient Test-Time Scalingcited by 14
  9. 2025Thinking Out Loud: Do Reasoning Models Know When They're Right?cited by 12
  10. 2025ReasoningFlow: Semantic Structure of Complex Reasoning Tracescited by 12
  11. 2025Scaling over Scaling: Exploring Test-Time Scaling Plateau in Large Reasoning Modelscited by 11
  12. 2025Don't Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Modelscited by 11
  13. 2025The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Thinkcited by 9
  14. 2025Lexical Hints of Accuracy in LLM Reasoning Chainscited by 9
  15. 2025Understanding the Thinking Process of Reasoning Models: A Perspective from Schoenfeld's Episode Theorycited by 9
  16. 2025LSPO: Length-aware Dynamic Sampling for Policy Optimization in LLM Reasoningcited by 9
  17. 2025Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoningcited by 9
  18. 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Modelscited by 8
  19. 2025Schoenfeld's Anatomy of Mathematical Reasoning by Language Modelscited by 8
  20. 2025LLMs as Span Annotators: A Comparative Study of LLMs and Humanscited by 7
  21. 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Modelscited by 7
  22. 2025CLATTER: Comprehensive Entailment Reasoning for Hallucination Detectioncited by 7
  23. 2026Probing the Trajectories of Reasoning Traces in Large Language Modelscited by 7
  24. 2025Efficient Long CoT Reasoning in Small Language Modelscited by 6
  25. 2025Meta-R1: Empowering Large Reasoning Models with Metacognitioncited by 6
  26. 2025From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Modelscited by 6
  27. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 6
  28. 2026Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoringcited by 6
  29. 2025RAST: Reasoning Activation in LLMs via Small-model Transfercited by 5
  30. 2025A Study on Thinking Patterns of Large Reasoning Models in Code Generationcited by 5
  31. 2025Off-Trajectory Reasoning: Can LLMs Collaborate on Reasoning Trajectory?cited by 5
  32. 2025Local Coherence or Global Validity? Investigating RLVR Traces in Math Domainscited by 5
  33. 2025Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncationcited by 5
  34. 2026Beyond Scalars: Evaluating and Understanding LLM Reasoning via Geometric Progress and Stabilitycited by 5
  35. 2025Token-Efficient Prompt Injection Attack: Provoking Cessation in LLM Reasoning via Adaptive Token Compressioncited by 4
  36. 2025ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Momentscited by 4
  37. 2025Your thoughts tell who you are: Characterize the reasoning patterns of LRMscited by 4
  38. 2025EfficientXLang: Towards Improving Token Efficiency Through Cross-Lingual Reasoningcited by 3

50 most cited of 92 All 92 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models
  2. 2025Are Large Language Models Capable of Deep Relational Reasoning? Insights from DeepSeek-R1 and Benchmark Comparisons
  3. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Models
  4. 2025Are DeepSeek R1 And Other Reasoning Models More Faithful?
  5. 2025Learning to Reason: Training LLMs with GPT-OSS or DeepSeek R1 Reasoning Traces
  6. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey
  7. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
  8. 2026Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens