mlsanityData to 14 Oct 2026

100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Models

arXiv 2505.00551 ↗First posted 1 May 2025cs.CLcited by 30

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

The recent development of reasoning language models (RLMs) represents a novel evolution in large language models. In particular, the recent release of DeepSeek-R1 has generated widespread social impact and sparked enthusiasm in the research community for exploring the explicit reasoning paradigm of language models. However, the implementation details of the released models have not been fully open-sourced by DeepSeek, including DeepSeek-R1-Zero, DeepSeek-R1, and the distilled small models. As a result, many replication studies have emerged aiming to reproduce the strong performance achieved by DeepSeek-R1, reaching comparable performance through similar training procedures and fully open-source data resources. These works have investigated feasible strategies for supervised fine-tuning (SFT) and reinforcement learning from verifiable rewards (RLVR), focusing on data preparation and method design, yielding various valuable insights. In this report, we provide a summary of recent replication studies to inspire future research. We primarily focus on SFT and RLVR as two main directions, introducing the details for data construction, method design and training procedure of current replication studies. Moreover, we conclude key findings from the implementation details and experimental results reported by these studies, anticipating to inspire future research. We also discuss additional techniques of enhancing RLMs, highlighting the potential of expanding the application scope of these models, and discussing the challenges in development. By this survey, we aim to help researchers and developers of RLMs stay updated with the latest advancements, and seek to inspire new ideas to further enhance RLMs.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 112 of its 164 references that are in the map. First posted on arXiv 1 May 2025.

Builds on

112 of 164 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learning14 Apr
  2. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility9 Apr
  3. 20251.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Training25 Mar
  4. 2025MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation13 Mar
  5. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond13 Mar
  6. 2025Reasoning Does Not Necessarily Improve Role-Playing Ability24 Feb
  7. 2025Demystifying Multilingual Chain-of-Thought in Process Reward Modeling18 Feb
  8. 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning3 Feb
  9. 2024DeepSeek-V3 Technical Report27 Dec
  10. 2024Large Language Models Can Self-Improve in Long-context Reasoning12 Nov
  11. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers7 Apr
  12. 2024xCoT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning13 Jan

Other references · 100

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  8. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  9. 2023A Survey of Large Language Modelscited by 2,576
  10. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  11. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  12. 2016Concrete Problems in AI Safetycited by 1,642
38 more
  1. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  2. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  3. 2025s1: Simple test-time scalingcited by 1,341
  4. 2024OpenAI o1 System Cardcited by 1,244
  5. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  6. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  7. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  8. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  9. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  10. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  11. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
  12. 2024Qwen2-Audio Technical Reportcited by 560
  13. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  14. 2025Process Reinforcement through Implicit Rewardscited by 504
  15. 2025LIMO: Less is More for Reasoningcited by 498
  16. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 464
  17. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 384
  18. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
  19. 2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 328
  20. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
  21. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  22. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  23. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Taskscited by 285
  24. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 284
  25. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 280
  26. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutioncited by 263
  27. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 262
  28. 2023Challenges and Applications of Large Language Modelscited by 254
  29. 2025Training Language Models to Reason Efficientlycited by 247
  30. 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalizationcited by 241
  31. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
  32. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 188
  33. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 182
  34. 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 178
  35. 2023CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languagescited by 174
  36. 2023TACO: Topics in Algorithmic COde generation datasetcited by 160
  37. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domainscited by 156
  38. 2025The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Taskscited by 152

50 most cited of 100 All 100 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 22; 2026: 8.

30 papers.

Sort:

By area:

  1. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  2. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 188
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  4. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 81
  5. 2025Tree Search for LLM Agent Reinforcement Learningcited by 79
  6. 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflowscited by 58
  7. 2025Diversity-Aware Policy Optimization for Large Language Model Reasoningcited by 41
  8. 2025Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Modelscited by 39
  9. 2026Trust Region On-Policy Distillationcited by 30
  10. 2025Base Models Know How to Reason, Thinking Models Learn Whencited by 27
  11. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  12. 2025DRPO: Efficient Reasoning via Decoupled Reward Policy Optimizationcited by 16
18 more
  1. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 14
  2. 2025MiroMind-M1: An Open-Source Advancement in Mathematical Reasoning via Context-Aware Multi-Stage Policy Optimizationcited by 12
  3. 2025First Try Matters: Revisiting the Role of Reflection in Reasoning Modelscited by 12
  4. 2025Trajectory Bellman Residual Minimization: A Simple Value-Based Method for LLM Reasoningcited by 11
  5. 2025RLKD: Distilling LLMs' Reasoning via Reinforcement Learningcited by 11
  6. 2025Enhancing Long Chain-of-Thought Reasoning through Multi-Path Plan Aggregationcited by 11
  7. 2025From Reasoning LLMs to BERT: A Two-Stage Distillation Framework for Search Relevancecited by 7
  8. 2025Reasoning Large Language Model Errors Arise from Hallucinating Critical Problem Featurescited by 6
  9. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 6
  10. 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
  11. 2025Reasoner for Real-World Event Detection: Scaling Reinforcement Learning via Adaptive Perplexity-Aware Sampling Strategycited by 3
  12. 2025ARISE: Agentic Rubric-Guided Iterative Survey Engine for Automated Scholarly Paper Generationcited by 3
  13. 2026SLAT: Segment-Level Adaptive Trimming for Efficient CoT Reasoningcited by 1
  14. 2025RAJ-PGA: Reasoning-Activated Jailbreak and Principle-Guided Alignment Framework for Large Reasoning Modelscited by 0
  15. 2026Gradually Compacting Large Language Models for Reasoning Like a Boiling Frogcited by 0
  16. 2026Fine-grained Verification via Diagnostic Reasoning Supervision for Aspect Sentiment Triplet Extractioncited by 0
  17. 2026Different Teachers, Different Capabilities: Sub-1B On-Device Distillation for Structured Text Enrichmentcited by 0
  18. 2026On-Policy Parameter Update Direction Underlies Generalization in LLM Post-Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Are DeepSeek R1 And Other Reasoning Models More Faithful?
  2. 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning
  3. 2025An Empirical Study on Eliciting and Improving R1-like Reasoning Models
  4. 2025A Survey on Post-training of Large Language Models
  5. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model
  6. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models
  7. 2025Challenges in Ensuring AI Safety in DeepSeek-R1 Models: The Shortcomings of Reinforcement Learning Strategies
  8. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle