mlsanityData to 7 Oct 2026

O1 Replication Journey: A Strategic Progress Report -- Part 1

arXiv 2410.18982 ↗First posted 8 Oct 2024cs.AI cs.CLcited by 140

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

This paper introduces a pioneering approach to artificial intelligence research, embodied in our O1 Replication Journey. In response to the announcement of OpenAI's groundbreaking O1 model, we embark on a transparent, real-time exploration to replicate its capabilities while reimagining the process of conducting and communicating AI research. Our methodology addresses critical challenges in modern AI research, including the insularity of prolonged team-based projects, delayed information sharing, and the lack of recognition for diverse contributions. By providing comprehensive, real-time documentation of our replication efforts, including both successes and failures, we aim to foster open science, accelerate collective advancement, and lay the groundwork for AI-driven scientific discovery. Our research progress report diverges significantly from traditional research papers, offering continuous updates, full process transparency, and active community engagement throughout the research journey. Technologically, we proposed the journey learning paradigm, which encourages models to learn not just shortcuts, but the complete exploration process, including trial and error, reflection, and backtracking. With only 327 training samples and without any additional tricks, journey learning outperformed conventional supervised learning by over 8% on the MATH dataset, demonstrating its extremely powerful potential. We believe this to be the most crucial component of O1 technology that we have successfully decoded. We share valuable resources including technical hypotheses and insights, cognitive exploration maps, custom-developed tools, etc at https://github.com/GAIR-NLP/O1-Journey.

Key facts. Cited by 140 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 34 of its 51 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

34 of 51 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  2. 2023SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning1 Aug
  3. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar
  4. 2022Large Language Models Can Self-Improve20 Oct

Other references · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2023Let's Verify Step by Stepcited by 3,598
  9. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  10. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  11. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  12. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
18 more
  1. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  2. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
  3. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 293
  4. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  5. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 256
  6. 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 202
  7. 2023Implicit Chain of Thought Reasoning via Knowledge Distillationcited by 165
  8. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 162
  9. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 160
  10. 2023Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Lawscited by 155
  11. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 143
  12. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 127
  13. 2024Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervisioncited by 95
  14. 2024Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planningcited by 95
  15. 2023MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluationcited by 57
  16. 2024Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problemscited by 49
  17. 2024Progress or Regress? Self-Improvement Reversal in Post-trainingcited by 23
  18. 2024Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoningcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 113; 2026: 12.

140 papers.

Sort:

By area:

  1. 2025s1: Simple test-time scalingcited by 1,311
  2. 2025LIMO: Less is More for Reasoningcited by 488
  3. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
  4. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 423
  5. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  6. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  7. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  8. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  9. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  10. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
  11. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  12. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
38 more
  1. 2025ToRL: Scaling Tool-Integrated RLcited by 166
  2. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scalingcited by 135
  3. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 122
  4. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  5. 2025Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Toolscited by 106
  6. 2025PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Modelscited by 98
  7. 2024O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson?cited by 90
  8. 2025Acting Less is Reasoning More! Teaching Model to Act Efficientlycited by 80
  9. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 78
  10. 2024o1-Coder: an o1 Replication for Codingcited by 77
  11. 2025Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?cited by 69
  12. 2025DeepAgent: A General Reasoning Agent with Scalable Toolsetscited by 69
  13. 2025Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learningcited by 67
  14. 2025Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQLcited by 59
  15. 2025Training a Scientific Reasoning Model for Chemistrycited by 54
  16. 2024Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTScited by 52
  17. 2025Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Modelscited by 52
  18. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 49
  19. 2025START: Self-taught Reasoner with Toolscited by 47
  20. 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinkingcited by 46
  21. 2025Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Taskscited by 43
  22. 2025Reasoning Language Models: A Blueprintcited by 41
  23. 2025LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attentioncited by 41
  24. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  25. 2025Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillationcited by 39
  26. 2025S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learningcited by 37
  27. 2025Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Modelscited by 37
  28. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  29. 2025O1 Replication Journey -- Part 3: Inference-time Scaling for Medical Reasoningcited by 34
  30. 2025Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with Minimalist Rule-Based RLcited by 34
  31. 2025Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoningcited by 32
  32. 2025Scalable Chain of Thoughts via Elastic Reasoningcited by 32
  33. 2025ReSo: A Reward-driven Self-organizing LLM-based Multi-Agent System for Reasoning Taskscited by 30
  34. 2025Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoningcited by 29
  35. 2025Parallel Scaling Law for Language Modelscited by 29
  36. 2024DRT: Deep Reasoning Translation via Long Chain-of-Thoughtcited by 28
  37. 2025Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoningcited by 28
  38. 2025Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Modelscited by 27

50 most cited of 140 All 140 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026O-Researcher: An Open Ended Deep Research Model via Multi-Agent Distillation and Agentic RL
  2. 2026Training AI Scientists to Replicate Research
  3. 2026The Agentic Researcher: A Practical Guide to AI-Assisted Research in Mathematics and Machine Learning
  4. 2026State of AI: An Empirical 100 Trillion Token Study with OpenRouter
  5. 2025AI-Researcher: Autonomous Scientific Innovation
  6. 2025AI4Research: A Survey of Artificial Intelligence for Scientific Research
  7. 2025Large language model-powered AI systems achieve self-replication with no human intervention
  8. 2026Harnessing the Collective Intelligence of AI Agents in the Wild for New Discoveries