# JustRL: Scaling a 1.5B LLM with a Simple RL Recipe

arXiv: https://arxiv.org/abs/2512.16649 · HTML: https://mlsanity.com/paper/2512.16649/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 18 Dec 2025.

## Builds on (most cited first, 15 of 15)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Solving Quantitative Reasoning Problems with Language Models](https://mlsanity.com/paper/2206.14858/) (2022)
- [OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems](https://mlsanity.com/paper/2402.14008/) (2024)
- [MathArena: Evaluating LLMs on Uncontaminated Math Competitions](https://mlsanity.com/paper/2505.23281/) (2025)
- [ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models](https://mlsanity.com/paper/2505.24864/) (2025)
- [Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems](https://mlsanity.com/paper/2412.09413/) (2024)
- [e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMs](https://mlsanity.com/paper/2506.09026/) (2025)
- [Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoning](https://mlsanity.com/paper/2508.08221/) (2025)
- [QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation](https://mlsanity.com/paper/2507.13266/) (2025)
- [DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning](https://mlsanity.com/paper/2510.15110/) (2025)
- [FastCuRL: Curriculum Reinforcement Learning with Stage-wise Context Scaling for Efficient Training R1-like Reasoning Models](https://mlsanity.com/paper/2503.17287/) (2025)
- [BroRL: Scaling Reinforcement Learning via Broadened Exploration](https://mlsanity.com/paper/2510.01180/) (2025)

## Built on by (most cited first, 30 of 64)

- [Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe](https://mlsanity.com/paper/2604.13016/) (2026)
- [Rethinking On-Policy Distillation of Large Language Models II: One Training Example](https://mlsanity.com/paper/2609.04172/) (2026)
- [Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation](https://mlsanity.com/paper/2605.11739/) (2026)
- [Evaluating Parameter Efficient Methods for RLVR](https://mlsanity.com/paper/2512.23165/) (2025)
- [OPRD: On-Policy Representation Distillation](https://mlsanity.com/paper/2606.06021/) (2026)
- [Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy Distillation](https://mlsanity.com/paper/2606.13657/) (2026)
- [Prefix-Guided On-Policy Distillation: Mining Golden Trajectories from Rollouts](https://mlsanity.com/paper/2606.21994/) (2026)
- [Mismatch Matters: On-Policy Distillation Beyond Token Agreement](https://mlsanity.com/paper/2608.09836/) (2026)
- [STAPO: Stabilizing Reinforcement Learning for LLMs by Silencing Rare Spurious Tokens](https://mlsanity.com/paper/2602.15620/) (2026)
- [KnowRL: Boosting LLM Reasoning via Reinforcement Learning with Minimal-Sufficient Knowledge Guidance](https://mlsanity.com/paper/2604.12627/) (2026)
- [On-Policy Distillation with Best-of-N Teacher Rollout Selection](https://mlsanity.com/paper/2605.09725/) (2026)
- [On the Non-decoupling of Supervised Fine-tuning and Reinforcement Learning in Post-training](https://mlsanity.com/paper/2601.07389/) (2026)
- [Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Models](https://mlsanity.com/paper/2602.12036/) (2026)
- [What Matters in On-Policy Distillation? A Perspective on Data Efficiency and Data Selection](https://mlsanity.com/paper/2609.05198/) (2026)
- [Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling](https://mlsanity.com/paper/2608.11829/) (2026)
- [Tail-Aware Top-k On-Policy Distillation](https://mlsanity.com/paper/2608.14728/) (2026)
- [Linear Dynamics in the RLVR Training of Large Language Models](https://mlsanity.com/paper/2601.04537/) (2026)
- [Asymmetric Prompt Weighting for Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2602.11128/) (2026)
- [On-Policy Supervised Fine-Tuning for Efficient Reasoning](https://mlsanity.com/paper/2602.13407/) (2026)
- [Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models](https://mlsanity.com/paper/2608.16647/) (2026)
- [Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control](https://mlsanity.com/paper/2605.11775/) (2026)
- [Closing the Reflection Gap: A Free Calibration Bonus for Agentic RL](https://mlsanity.com/paper/2606.14211/) (2026)
- [Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients](https://mlsanity.com/paper/2606.18216/) (2026)
- [STARE: Surprisal-Guided Token-Level Advantage Reweighting for Policy Entropy Stability](https://mlsanity.com/paper/2606.19236/) (2026)
- [Learning When to Trust via Selective Context Preference Optimization](https://mlsanity.com/paper/2608.06377/) (2026)
- [Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress](https://mlsanity.com/paper/2608.19408/) (2026)
- [Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillation](https://mlsanity.com/paper/2609.16937/) (2026)
- [Detecting RLVR Training Data via Structural Convergence of Reasoning](https://mlsanity.com/paper/2602.11792/) (2026)
- [Dissecting Failure Dynamics in Large Language Model Reasoning](https://mlsanity.com/paper/2604.14528/) (2026)
- [Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoning](https://mlsanity.com/paper/2605.06326/) (2026)

## Abstract (arXiv)

Recent advances in reinforcement learning for large language models have converged on increasing complexity: multi-stage training pipelines, dynamic hyperparameter schedules, and curriculum learning strategies. This raises a fundamental question: Is this complexity necessary? We present JustRL, a minimal approach using single-stage training with fixed hyperparameters that achieves state-of-the-art performance on two 1.5B reasoning models (54.9% and 64.3% average accuracy across nine mathematical benchmarks) while using 2× less compute than sophisticated approaches. The same hyperparameters transfer across both models without tuning, and training exhibits smooth, monotonic improvement over 4,000+ steps without the collapses or plateaus that typically motivate interventions. Critically, ablations reveal that adding ``standard tricks'' like explicit length penalties and robust verifiers may degrade performance by collapsing exploration. These results suggest that the field may be adding complexity to solve problems that disappear with a stable, scaled-up baseline. We release our models and code to establish a simple, validated baseline for the community.