mlsanityData to 13 Oct 2026

Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient Reasoning

arXiv 2604.02007 ↗First posted 2 Apr 2026cs.LGcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Building general-purpose reasoning models using reinforcement learning with verifiable rewards (RLVR) across diverse domains has been widely adopted by frontier open-weight models. However, their training recipes and domain mixtures are often not disclosed. Joint optimization across domains poses significant challenges: domains vary widely in rollout length, problem difficulty, and sample efficiency. Further, models with long chain-of-thought traces increase inference cost and latency, making efficiency critical for practical deployment. We present Apriel-Reasoner, trained with a reproducible multi-domain RL post-training recipe on Apriel-Base, a 15B-parameter open-weight LLM, across five domains using public datasets: mathematics, code generation, instruction following, logical puzzles, and function calling. We introduce adaptive domain sampling that preserves target completed-rollout ratios despite heterogeneous rollout dynamics, and a difficulty-aware length penalty that, at no additional training overhead, encourages longer reasoning for difficult problems and shorter traces for easy ones. Trained with a strict 16K-token output budget, Apriel-Reasoner remains effective at a 32K output budget and improves over Apriel-Base on AIME 2025, GPQA, MMLU-Pro, and LiveCodeBench while producing 30-50% shorter reasoning traces. Among the evaluated open-weight models of similar scale, it improves the accuracy-token tradeoff using a fully public-data recipe.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 47 of its 55 references that are in the map. First posted on arXiv 2 Apr 2026.

Builds on

47 of 55 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026LLMs Can Learn to Reason Via Off-Policy RL22 Feb
  2. 2025Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models15 Dec
  3. 2025Prosperity before Collapse: How Far Can Off-Policy RL Reach with Stale Data on LLMs?1 Oct
  4. 2025PipelineRL: Faster On-policy Reinforcement Learning for Long Sequence Generation23 Sep
  5. 2025Geometric-Mean Policy Optimization28 Jul
  6. 2025SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyond26 May
  7. 2025GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Models13 Apr
  8. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  10. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs3 Mar
  11. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  12. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
2 more
  1. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
  2. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 33

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,091
  2. 2025Qwen3 Technical Reportcited by 8,510
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,270
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,753
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,804
  6. 2015Trust Region Policy Optimizationcited by 2,513
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,951
  8. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,799
  9. 2025s1: Simple test-time scalingcited by 1,322
  10. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,191
  11. 2025Group Sequence Policy Optimizationcited by 806
  12. 2025Kimi K2: Open Agentic Intelligencecited by 544
21 more
  1. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 542
  2. 2025Qwen3-Omni Technical Reportcited by 464
  3. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 456
  4. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 382
  5. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 281
  6. 2024Token-Budget-Aware LLM Reasoningcited by 243
  7. 2024Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Datacited by 185
  8. 2025Generalizing Verifiable Instruction Followingcited by 184
  9. 2024Pixtral 12Bcited by 177
  10. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 170
  11. 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Modelscited by 156
  12. 2023TACO: Topics in Algorithmic COde generation datasetcited by 153
  13. 2025Phi-4-reasoning Technical Reportcited by 147
  14. 2025AdaptThink: Reasoning Models Can Learn When to Thinkcited by 141
  15. 2025Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMscited by 115
  16. 2025Thinkless: LLM Learns When to Thinkcited by 109
  17. 2023LLM360: Towards Fully Transparent Open-Source LLMscited by 102
  18. 2025The Art of Scaling Reinforcement Learning Compute for LLMscited by 96
  19. 2025SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoningcited by 48
  20. 2025DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learningcited by 35
  21. 2025BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokenscited by 28

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective
  2. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers
  3. 2026Advancing General-Purpose Reasoning Models with Modular Gradient Surgery
  4. 2026Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVR
  5. 2026PCL-Reasoner-V1.5: Advancing Math Reasoning with Offline Reinforcement Learning
  6. 2025TinyV: Reducing False Negatives in Verification Improves RL for LLM Reasoning
  7. 2025Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers
  8. 2025Can One Domain Help Others? A Data-Centric Study on Multi-Domain Reasoning via Reinforcement Learning