# Magistral

arXiv: https://arxiv.org/abs/2506.10910 · HTML: https://mlsanity.com/paper/2506.10910/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 20 of its 27 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning](https://mlsanity.com/paper/2506.01939/) (2025)
- [MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark](https://mlsanity.com/paper/2409.02813/) (2024)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model](https://mlsanity.com/paper/2503.24290/) (2025)
- [What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study](https://mlsanity.com/paper/2006.05990/) (2020)
- [Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models](https://mlsanity.com/paper/2410.18252/) (2024)
- [LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Training](https://mlsanity.com/paper/2505.24034/) (2025)

## Built on by (most cited first, 30 of 60)

- [Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models](https://mlsanity.com/paper/2601.18734/) (2026)
- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [CWM: An Open-Weights LLM for Research on Code Generation with World Models](https://mlsanity.com/paper/2510.02387/) (2025)
- [EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modes](https://mlsanity.com/paper/2507.11407/) (2025)
- [What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT](https://mlsanity.com/paper/2509.19284/) (2025)
- [Reasoning Models Struggle to Control their Chains of Thought](https://mlsanity.com/paper/2603.05706/) (2026)
- [Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learning](https://mlsanity.com/paper/2510.07300/) (2025)
- [Can We Predict Alignment Before Models Finish Thinking? Towards Monitoring Misaligned Reasoning Models](https://mlsanity.com/paper/2507.12428/) (2025)
- [Can One Domain Help Others? A Data-Centric Study on Multi-Domain Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2507.17512/) (2025)
- [A Cost-Benefit Analysis of On-Premise Large Language Model Deployment: Breaking Even with Commercial LLM Services](https://mlsanity.com/paper/2509.18101/) (2025)
- [First Try Matters: Revisiting the Role of Reflection in Reasoning Models](https://mlsanity.com/paper/2510.08308/) (2025)
- [RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Models](https://mlsanity.com/paper/2602.17053/) (2026)
- [Omni-Thinker: Scaling Multi-Task RL in LLMs with Hybrid Reward and Task Scheduling](https://mlsanity.com/paper/2507.14783/) (2025)
- [Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought](https://mlsanity.com/paper/2510.04230/) (2025)
- [LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematics](https://mlsanity.com/paper/2602.24173/) (2026)
- [Self-Supervised On-Policy Distillation for Reasoning Language Models](https://mlsanity.com/paper/2605.17497/) (2026)
- [Hermes 4 Technical Report](https://mlsanity.com/paper/2508.18255/) (2025)
- [Ground-Truthing AI Energy Consumption: Validating CodeCarbon Against External Measurements](https://mlsanity.com/paper/2509.22092/) (2025)
- [STAC: When Innocent Tools Form Dangerous Chains for LLM Agents](https://mlsanity.com/paper/2509.25624/) (2025)
- [Enhancing Large Language Model Reasoning with Reward Models: An Analytical Survey](https://mlsanity.com/paper/2510.01925/) (2025)
- [PAINT: Partial-Solution Adaptive Interpolated Training for Self-Distilled Reasoners](https://mlsanity.com/paper/2604.26573/) (2026)
- [Are Large Reasoning Models Interruptible?](https://mlsanity.com/paper/2510.11713/) (2025)
- [Unlocking Reasoning Capability on Machine Translation in Large Language Models](https://mlsanity.com/paper/2602.14763/) (2026)
- [Measuring Weak-to-Strong Legibility of Reasoning Models](https://mlsanity.com/paper/2603.20508/) (2026)
- [ReasonXL: Shifting LLM Reasoning Language Without Sacrificing Performance](https://mlsanity.com/paper/2604.12378/) (2026)
- [Your thoughts tell who you are: Characterize the reasoning patterns of LRMs](https://mlsanity.com/paper/2509.24147/) (2025)
- [Monitorability as a Free Gift: How RLVR Spontaneously Aligns Reasoning](https://mlsanity.com/paper/2602.03978/) (2026)
- [From SFT to RL: Demystifying the Post-Training Pipeline for LLM-based Vulnerability Detection](https://mlsanity.com/paper/2602.14012/) (2026)
- [Scale or Reason? A Compute-Equivalent Analysis of Reasoning Distillation](https://mlsanity.com/paper/2509.22193/) (2025)
- [Whatever Remains Must Be True: Filtering Drives Reasoning in LLMs, Shaping Diversity](https://mlsanity.com/paper/2512.05962/) (2025)

## Abstract (arXiv)

We introduce Magistral, Mistral's first reasoning model and our own scalable reinforcement learning (RL) pipeline. Instead of relying on existing implementations and RL traces distilled from prior models, we follow a ground up approach, relying solely on our own models and infrastructure. Notably, we demonstrate a stack that enabled us to explore the limits of pure RL training of LLMs, present a simple method to force the reasoning language of the model, and show that RL on text data alone maintains most of the initial checkpoint's capabilities. We find that RL on text maintains or improves multimodal understanding, instruction following and function calling. We present Magistral Medium, trained for reasoning on top of Mistral Medium 3 with RL alone, and we open-source Magistral Small (Apache 2.0) which further includes cold-start data from Magistral Medium.