mlsanityData to 14 Oct 2026

Magistral

arXiv 2506.10910 ↗First posted 12 Jun 2025cs.CLcited by 60

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

We introduce Magistral, Mistral's first reasoning model and our own scalable reinforcement learning (RL) pipeline. Instead of relying on existing implementations and RL traces distilled from prior models, we follow a ground up approach, relying solely on our own models and infrastructure. Notably, we demonstrate a stack that enabled us to explore the limits of pure RL training of LLMs, present a simple method to force the reasoning language of the model, and show that RL on text data alone maintains most of the initial checkpoint's capabilities. We find that RL on text maintains or improves multimodal understanding, instruction following and function calling. We present Magistral Medium, trained for reasoning on top of Mistral Medium 3 with RL alone, and we open-source Magistral Small (Apache 2.0) which further includes cold-start data from Magistral Medium.

Key facts. Cited by 60 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 27 references that are in the map. First posted on arXiv 12 Jun 2025.

Builds on

20 of 27 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning2 Jun
  2. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  3. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  5. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 15

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  4. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  6. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  7. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  8. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  9. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  10. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  11. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  12. 2025Humanity's Last Examcited by 491
3 more
  1. 2020What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Studycited by 162
  2. 2024Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Modelscited by 70
  3. 2025LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Trainingcited by 42

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 25; 2026: 35.

60 papers.

Sort:

By area:

  1. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 455
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  3. 2025CWM: An Open-Weights LLM for Research on Code Generation with World Modelscited by 93
  4. 2025EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modescited by 32
  5. 2025What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoTcited by 25
  6. 2026Reasoning Models Struggle to Control their Chains of Thoughtcited by 25
  7. 2025Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learningcited by 18
  8. 2025Can We Predict Alignment Before Models Finish Thinking? Towards Monitoring Misaligned Reasoning Modelscited by 17
  9. 2025Can One Domain Help Others? A Data-Centric Study on Multi-Domain Reasoning via Reinforcement Learningcited by 17
  10. 2025A Cost-Benefit Analysis of On-Premise Large Language Model Deployment: Breaking Even with Commercial LLM Servicescited by 17
  11. 2025First Try Matters: Revisiting the Role of Reflection in Reasoning Modelscited by 12
  12. 2026RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Modelscited by 11
38 more
  1. 2025Omni-Thinker: Scaling Multi-Task RL in LLMs with Hybrid Reward and Task Schedulingcited by 10
  2. 2025Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thoughtcited by 10
  3. 2026LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematicscited by 10
  4. 2026Self-Supervised On-Policy Distillation for Reasoning Language Modelscited by 10
  5. 2025Hermes 4 Technical Reportcited by 8
  6. 2025Ground-Truthing AI Energy Consumption: Validating CodeCarbon Against External Measurementscited by 8
  7. 2025STAC: When Innocent Tools Form Dangerous Chains for LLM Agentscited by 8
  8. 2025Enhancing Large Language Model Reasoning with Reward Models: An Analytical Surveycited by 8
  9. 2026PAINT: Partial-Solution Adaptive Interpolated Training for Self-Distilled Reasonerscited by 8
  10. 2025Are Large Reasoning Models Interruptible?cited by 7
  11. 2026Unlocking Reasoning Capability on Machine Translation in Large Language Modelscited by 6
  12. 2026Measuring Weak-to-Strong Legibility of Reasoning Modelscited by 5
  13. 2026ReasonXL: Shifting LLM Reasoning Language Without Sacrificing Performancecited by 5
  14. 2025Your thoughts tell who you are: Characterize the reasoning patterns of LRMscited by 4
  15. 2026Monitorability as a Free Gift: How RLVR Spontaneously Aligns Reasoningcited by 4
  16. 2026From SFT to RL: Demystifying the Post-Training Pipeline for LLM-based Vulnerability Detectioncited by 4
  17. 2025Scale or Reason? A Compute-Equivalent Analysis of Reasoning Distillationcited by 3
  18. 2025Whatever Remains Must Be True: Filtering Drives Reasoning in LLMs, Shaping Diversitycited by 3
  19. 2026Experience is the Best Teacher: Motivating Effective Exploration in Reinforcement Learning for LLMscited by 3
  20. 2025Confidence-Weighted Token Set Cover for Early Hypothesis Pruning in Self-Consistencycited by 2
  21. 2025Opponent Shaping in LLM Agentscited by 2
  22. 2025Shorter but not Worse: Frugal Reasoning via Easy Samples as Length Regularizers in Math RLVRcited by 2
  23. 2026Think Multilingual, Not Harder: A Data-Efficient Framework for Teaching Reasoning Models to Code-Switchcited by 2
  24. 2026Language as a Latent Variable for Reasoning Optimizationcited by 2
  25. 2026Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Taskscited by 2
  26. 2026Efficient Multilingual Reasoning Transfer via Progressive Code-Switchingcited by 2
  27. 2026Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluationscited by 2
  28. 2025Bridging the Plausibility-Validity Gap by Fine-Tuning a Reasoning-Enhanced LLM for Chemical Synthesis and Discoverycited by 1
  29. 2026A Novel Hierarchical Multi-Agent System for Payments Using LLMscited by 1
  30. 2026Mind the Gap Between Spatial Reasoning and Acting! Step-by-Step Evaluation of Agents With Spatial-Gymcited by 1
  31. 2026Process Reward Models Meet Planning: Generating Precise and Scalable Datasets for Step-Level Rewardscited by 1
  32. 2026The Scaling Properties of Implicit Deductive Reasoning in Transformerscited by 1
  33. 2026Cost of Reasoning in non-English Languages: A Case Study on Japanesecited by 1
  34. 2025Datarus-R1: An Adaptive Multi-Step Reasoning LLM for Automated Data Analysiscited by 0
  35. 2025Ko-MuSR: A Multistep Soft Reasoning Benchmark for LLMs Capable of Understanding Koreancited by 0
  36. 2026Reasoning Boosts Opinion Alignment in LLMscited by 0
  37. 2026MolRGen: A Training and Evaluation Setting for De Novo Molecular Generation with Reasonning Modelscited by 0
  38. 2026TSHA: A Benchmark for Visual Language Models in Trustworthy Safety Hazard Assessment Scenarioscited by 0

50 most cited of 60 All 60 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
  2. 2026Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key
  3. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective
  4. 2025RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning
  5. 2026Rethinking RL for LLM Reasoning: It's Sparse Policy Selection, Not Capability Learning
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
  7. 2026On the Direction of RLVR Updates for LLM Reasoning: Identification and Exploitation
  8. 2026RL Excursions during Pre-Training: Re-examining Policy Optimization for LLM training