# OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique

arXiv: https://arxiv.org/abs/2507.09075 · HTML: https://mlsanity.com/paper/2507.09075/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 43 of its 66 references that are in the map.
- First posted on arXiv 11 Jul 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 30 of 43)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement](https://mlsanity.com/paper/2409.12122/) (2024)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations](https://mlsanity.com/paper/2312.08935/) (2023)
- [Generative Verifiers: Reward Modeling as Next-Token Prediction](https://mlsanity.com/paper/2408.15240/) (2024)
- [The Lessons of Developing Process Reward Models in Mathematical Reasoning](https://mlsanity.com/paper/2501.07301/) (2025)
- [Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs](https://mlsanity.com/paper/2410.18451/) (2024)
- [From System 1 to System 2: A Survey of Reasoning Large Language Models](https://mlsanity.com/paper/2502.17419/) (2025)
- [Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models](https://mlsanity.com/paper/2408.00724/) (2024)
- [Inference-Time Scaling for Generalist Reward Modeling](https://mlsanity.com/paper/2504.02495/) (2025)
- [HelpSteer2: Open-source dataset for training top-performing reward models](https://mlsanity.com/paper/2406.08673/) (2024)
- [A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?](https://mlsanity.com/paper/2503.24235/) (2025)
- [Rethinking Benchmark and Contamination for Language Models with Rephrased Samples](https://mlsanity.com/paper/2311.04850/) (2023)
- [AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning dataset](https://mlsanity.com/paper/2504.16891/) (2025)
- [Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs](https://mlsanity.com/paper/2501.18585/) (2025)
- [TACO: Topics in Algorithmic COde generation dataset](https://mlsanity.com/paper/2312.14852/) (2023)
- [Llama-Nemotron: Efficient Reasoning Models](https://mlsanity.com/paper/2505.00949/) (2025)
- [RM-R1: Reward Modeling as Reasoning](https://mlsanity.com/paper/2505.02387/) (2025)
- [Generative Reward Models](https://mlsanity.com/paper/2410.12832/) (2024)
- [Nemotron-4 340B Technical Report](https://mlsanity.com/paper/2406.11704/) (2024)
- [LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!](https://mlsanity.com/paper/2502.07374/) (2025)
- [HelpSteer2-Preference: Complementing Ratings with Preferences](https://mlsanity.com/paper/2410.01257/) (2024)
- [Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.07572/) (2025)

## Built on by (most cited first, 25 of 25)

- [Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving Clipping Policy Optimization](https://mlsanity.com/paper/2508.07629/) (2025)
- [AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agent](https://mlsanity.com/paper/2512.20745/) (2025)
- [Don't Pass@k: A Bayesian Framework for Large Language Model Evaluation](https://mlsanity.com/paper/2510.04265/) (2025)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual World](https://mlsanity.com/paper/2603.19223/) (2026)
- [Distilled Reinforcement Learning for LLM Post-training](https://mlsanity.com/paper/2607.17247/) (2026)
- [MobileLLM-R1: Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes](https://mlsanity.com/paper/2509.24945/) (2025)
- [Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility](https://mlsanity.com/paper/2608.04001/) (2026)
- [ASTER: Agentic Scaling with Tool-integrated Extended Reasoning](https://mlsanity.com/paper/2602.01204/) (2026)
- [Contextual Drag: How Errors in the Context Affect LLM Reasoning](https://mlsanity.com/paper/2602.04288/) (2026)
- [When Reasoning Meets Its Laws](https://mlsanity.com/paper/2512.17901/) (2025)
- [The Valley of Code Reasoning: Scaling Knowledge Distillation of Large Language Models](https://mlsanity.com/paper/2510.06101/) (2025)
- [Learning Generative Selection for Best-of-N](https://mlsanity.com/paper/2602.02143/) (2026)
- [Extrapolative Weight Averaging Reveals Correctness-Efficiency Frontiers in Code RL](https://mlsanity.com/paper/2605.28751/) (2026)
- [Motif-2-12.7B-Reasoning: A Practitioner's Guide to RL Training Recipes](https://mlsanity.com/paper/2512.11463/) (2025)
- [The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment](https://mlsanity.com/paper/2604.06377/) (2026)
- [A Primer in Post-Training Reasoning Data: What We Know About How It Works](https://mlsanity.com/paper/2606.02113/) (2026)
- [Synthesizing File-Level Data for Unit Test Generation with Chain-of-Thoughts via Self-Debugging](https://mlsanity.com/paper/2602.03181/) (2026)
- [Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling](https://mlsanity.com/paper/2604.27039/) (2026)
- [Primal Generation, Dual Judgment: Self-Training from Test-Time Scaling](https://mlsanity.com/paper/2605.11299/) (2026)
- [ML-Embed: Inclusive and Efficient Embeddings for a Multilingual World](https://mlsanity.com/paper/2605.15081/) (2026)
- [Post-training is (Massive) Supervised Learning](https://mlsanity.com/paper/2606.07527/) (2026)
- [The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithms](https://mlsanity.com/paper/2606.25450/) (2026)
- [Don't Let Gains FADE: Breaking Down Policy Gradient Weights in RL](https://mlsanity.com/paper/2607.01490/) (2026)
- [Post-Training Language Models for Gold-Medal Performance in Coding Competitions](https://mlsanity.com/paper/2609.02849/) (2026)

## Abstract (arXiv)

Recent advancements in reasoning-based Large Language Models (LLMs), particularly their potential through test-time scaling, have created significant opportunities for distillation in code generation and critique. However, progress in both areas fundamentally depends on large-scale, high-quality datasets. In this work, we introduce OpenCodeReasoning-II, a dataset consists of 2.5M question-solution-critique triples (approx. 35K unique programming questions), making it nearly twice the size of the previous largest publicly available code reasoning dataset. In this work, we employ a two-stage supervised fine-tuning strategy. The first stage focuses on fine-tuning for code generation, while the second stage involves the joint training of models for both code generation and critique. Our resulting finetuned Qwen2.5-Instruct models achieve performance in code generation that either exceeds or equals the best prior open-weight distilled models. Notably, the integration of our code generation and critique models leads to significant improvements in competitive coding performance. Furthermore, we present an extension of the LiveCodeBench benchmark to specifically support the C++ programming language, thereby facilitating more comprehensive LLM evaluation using this benchmark.