mlsanityData to 7 Oct 2026

OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models

arXiv 2410.09671 ↗First posted 12 Oct 2024cs.AI cs.CLcited by 92

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

In this technical report, we introduce OpenR, an open-source framework designed to integrate key components for enhancing the reasoning capabilities of large language models (LLMs). OpenR unifies data acquisition, reinforcement learning training (both online and offline), and non-autoregressive decoding into a cohesive software platform. Our goal is to establish an open-source platform and community to accelerate the development of LLM reasoning. Inspired by the success of OpenAI's o1 model, which demonstrated improved reasoning abilities through step-by-step reasoning and reinforcement learning, OpenR integrates test-time compute, reinforcement learning, and process supervision to improve reasoning in LLMs. Our work is the first to provide an open-source framework that explores the core techniques of OpenAI's o1 model with reinforcement learning, achieving advanced reasoning capabilities beyond traditional autoregressive methods. We demonstrate the efficacy of OpenR by evaluating it on the MATH dataset, utilising publicly available data and search methods. Our initial experiments confirm substantial gains, with relative improvements in reasoning and performance driven by test-time computation and reinforcement learning through process reward models. The OpenR framework, including code, models, and datasets, is accessible at https://openreasoner.github.io.

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 26 of its 30 references that are in the map. First posted on arXiv 12 Oct 2024.

Builds on

26 of 30 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking14 Mar
  2. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Models30 Nov

Other references · 24

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2023Let's Verify Step by Stepcited by 3,598
  10. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  11. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  12. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
12 more
  1. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  2. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  3. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  4. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  5. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  6. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  7. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 291
  8. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 282
  9. 2023Think before you speak: Training Language Models With Pause Tokenscited by 254
  10. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 193
  11. 2023OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoningcited by 84
  12. 2024Multi-step Problem Solving Through a Verifier: An Empirical Analysis on Model-induced Process Supervisioncited by 51

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 62; 2026: 19.

92 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  3. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
  4. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 199
  5. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 158
  6. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scalingcited by 135
  7. 2025MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphscited by 105
  8. 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoningcited by 94
  9. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 78
  10. 2024o1-Coder: an o1 Replication for Codingcited by 77
  11. 2025PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Modelscited by 57
  12. 2025Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Modelscited by 52
38 more
  1. 2025MARFT: Multi-Agent Reinforcement Fine-Tuningcited by 51
  2. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 49
  3. 2025Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Aheadcited by 43
  4. 2025Reasoning Language Models: A Blueprintcited by 41
  5. 2025VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Datacited by 39
  6. 2025A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Modelscited by 37
  7. 2025Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigationcited by 36
  8. 2025Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with Minimalist Rule-Based RLcited by 34
  9. 2024A Picture is Worth A Thousand Numbers: Enabling LLMs Reason about Time Series via Visualizationcited by 31
  10. 2025MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable Step-Level Supervisioncited by 28
  11. 2024Aviary: training language agents on challenging scientific taskscited by 23
  12. 2025Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoningcited by 23
  13. 2024LLM-based Multi-Agent Systems: Techniques and Business Perspectivescited by 22
  14. 2025On the Emergence of Thinking in LLMs I: Searching for the Right Intuitioncited by 22
  15. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  16. 2024Reasoning Through Execution: Unifying Process and Outcome Rewards for Code Generationcited by 21
  17. 2025Climbing the Ladder of Reasoning: What LLMs Can-and Still Can't-Solve after SFT?cited by 21
  18. 2025Adaptive Group Policy Optimization: Towards Stable Training and Token-Efficient Reasoningcited by 20
  19. 2025Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuningcited by 19
  20. 2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Lawcited by 18
  21. 2025ThinkBench: Dynamic Out-of-Distribution Evaluation for Robust LLM Reasoningcited by 17
  22. 2024AtomThink: Multimodal Slow Thinking with Atomic Step Reasoningcited by 16
  23. 2025ViLBench: A Suite for Vision-Language Process Reward Modelingcited by 16
  24. 2025Accelerating Large Language Model Reasoning via Speculative Searchcited by 16
  25. 2024OpenRFT: Adapting Reasoning Foundation Model for Domain-specific Tasks with Reinforcement Fine-Tuningcited by 13
  26. 2025MetaScale: Test-Time Scaling with Evolving Meta-Thoughtscited by 13
  27. 2025Reinforcement Learning Foundations for Deep Research Systems: A Surveycited by 13
  28. 2025Attention as a Compass: Efficient Exploration for Process-Supervised RL in Reasoning Modelscited by 13
  29. 2025Examining False Positives under Inference Scaling for Mathematical Reasoningcited by 11
  30. 2025Investigating Inference-time Scaling for Chain of Multi-modal Thought: A Preliminary Studycited by 11
  31. 2025Good Learners Think Their Thinking: Generative PRM Makes Large Reasoning Model More Efficient Math Learnercited by 11
  32. 2025Can Atomic Step Decomposition Enhance the Self-structured Reasoning of Multimodal Large Models?cited by 10
  33. 2025Accurate and Diverse LLM Mathematical Reasoning via Automated PRM-Guided GFlowNetscited by 10
  34. 2025Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Modelscited by 10
  35. 2025G2RPO-A: Guided Group Relative Policy Optimization with Adaptive Guidancecited by 10
  36. 2025GroundedPRM: Tree-Guided and Fidelity-Aware Process Reward Modeling for Step-Level Reasoningcited by 10
  37. 2025Faster and Better LLMs via Latency-Aware Test-Time Scalingcited by 9
  38. 2025Graph-Augmented Reasoning: Evolving Step-by-Step Knowledge Graph Retrieval for LLM Reasoningcited by 8

50 most cited of 92 All 92 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025OpenSIR: Open-Ended Self-Improving Reasoner
  2. 2024A Comparative Study on Reasoning Patterns of OpenAI's o1 Model
  3. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions
  4. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
  5. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  6. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  7. 2025ThinkDial: An Open Recipe for Controlling Reasoning Effort in Large Language Models
  8. 2024Multi-Step Reasoning with Large Language Models, a Survey