mlsanityData to 7 Oct 2026

MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining

arXiv 2505.07608 ↗First posted 12 May 2025cs.CL cs.AI cs.LGcited by 117

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

We present MiMo-7B, a large language model born for reasoning tasks, with optimization across both pre-training and post-training stages. During pre-training, we enhance the data preprocessing pipeline and employ a three-stage data mixing strategy to strengthen the base model's reasoning potential. MiMo-7B-Base is pre-trained on 25 trillion tokens, with additional Multi-Token Prediction objective for enhanced performance and accelerated inference speed. During post-training, we curate a dataset of 130K verifiable mathematics and programming problems for reinforcement learning, integrating a test-difficulty-driven code-reward scheme to alleviate sparse-reward issues and employing strategic data resampling to stabilize training. Extensive evaluations show that MiMo-7B-Base possesses exceptional reasoning potential, outperforming even much larger 32B models. The final RL-tuned model, MiMo-7B-RL, achieves superior performance on mathematics, code and general reasoning tasks, surpassing the performance of OpenAI o1-mini. The model checkpoints are available at https://github.com/xiaomimimo/MiMo.

Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 51 of its 66 references that are in the map. First posted on arXiv 12 May 2025.

Builds on

51 of 66 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  2. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  3. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  5. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  6. 2023Let's Verify Step by Step31 May

Other references · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  10. 2024Qwen2.5 Technical Reportcited by 4,713
  11. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  12. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
33 more
  1. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  2. 2021Program Synthesis with Large Language Modelscited by 3,509
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  4. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  8. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  9. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  10. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  11. 2019Root Mean Square Layer Normalizationcited by 1,425
  12. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  13. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  14. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  15. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  16. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  17. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  18. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  19. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  20. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  21. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  22. 2016Language Modeling with Gated Convolutional Networkscited by 688
  23. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  24. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  25. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 344
  26. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 325
  27. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  28. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 169
  29. 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Textcited by 151
  30. 2025A Comprehensive Survey on Long Context Language Modelingcited by 113
  31. 2023AlignBench: Benchmarking Chinese Alignment of Large Language Modelscited by 68
  32. 2025MegaMath: Pushing the Limits of Open Math Corporacited by 47
  33. 2024FineMath: A Fine-Grained Mathematical Evaluation Benchmark for Chinese Large Language Modelscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 51; 2026: 66.

117 papers.

Sort:

By area:

  1. 2025Learning to Reason without External Rewardscited by 199
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  3. 2026MiMo-V2-Flash Technical Reportcited by 163
  4. 2025General-Reasoner: Advancing LLM Reasoning Across All Domainscited by 132
  5. 2025MiMo-VL Technical Reportcited by 106
  6. 2025AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergycited by 75
  7. 2025Are Reasoning Models More Prone to Hallucination?cited by 62
  8. 2025MiMo-Audio: Audio Language Models are Few-Shot Learnerscited by 58
  9. 2025rStar2-Agent: Agentic Reasoning Technical Reportcited by 54
  10. 2025MiniCPM4: Ultra-Efficient LLMs on End Devicescited by 51
  11. 2025Intern-S1: A Scientific Multimodal Foundation Modelcited by 49
  12. 2025Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Modelscited by 45
38 more
  1. 2025Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving Clipping Policy Optimizationcited by 39
  2. 2025Visual Document Understanding and Reasoning: A Multi-Agent Collaboration Framework with Agent-Wise Adaptive Test-Time Scalingcited by 30
  3. 2025JarvisEvo: Towards a Self-Evolving Photo Editing Agent with Synergistic Editor-Evaluator Optimizationcited by 30
  4. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 27
  5. 2025Table-R1: Inference-Time Scaling for Table Reasoningcited by 23
  6. 2025Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement Fine-Tuning of Large Language Modelscited by 19
  7. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  8. 2025Mid-Training of Large Language Models: A Surveycited by 18
  9. 2025PurpCode: Reasoning for Safer Code Generationcited by 17
  10. 2025AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agentcited by 17
  11. 2025Beyond Multi-Token Prediction: Pretraining LLMs with Future Summariescited by 14
  12. 2025FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Predictioncited by 13
  13. 2025Part II: ROLL Flash -- Accelerating RLVR and Agentic Training with Asynchronycited by 13
  14. 2025MiroMind-M1: An Open-Source Advancement in Mathematical Reasoning via Context-Aware Multi-Stage Policy Optimizationcited by 12
  15. 2025First Try Matters: Revisiting the Role of Reflection in Reasoning Modelscited by 12
  16. 2025ThinkGen: Generalized Thinking for Visual Generationcited by 12
  17. 2025RollArt: Disaggregated Multi-Task Agentic RL Training at Scalecited by 11
  18. 2025OralGPT-Omni: A Versatile Dental Multimodal Large Language Modelcited by 10
  19. 2025Klear-CodeTest: Scalable Test Case Generation for Code Reinforcement Learningcited by 9
  20. 2025Large Reasoning Embedding Models: Towards Next-Generation Dense Retrieval Paradigmcited by 9
  21. 2026RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Modelscited by 9
  22. 2025Tiny Model, Big Logic: Diversity-Driven Optimization Elicits Large-Model Reasoning Ability in VibeThinker-1.5Bcited by 8
  23. 2025UniComp: Rethinking Video Compression Through Informational Uniquenesscited by 7
  24. 2025GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluationcited by 7
  25. 2025ScaleDiff: Scaling Difficult Problems for Advanced Mathematical Reasoningcited by 6
  26. 2025FAPO: Flawed-Aware Policy Optimization for Efficient and Reliable Reasoningcited by 6
  27. 2026ORBIT: On-policy Exploration-Exploitation for Controllable Multi-Budget Reasoningcited by 6
  28. 2026ELVA: Exploring Ranking-Driven Universal Multimodal Retrievalcited by 5
  29. 2025ICPO: Intrinsic Confidence-Driven Group Relative Preference Optimization for Efficient Reinforcement Learningcited by 4
  30. 2026Statistical Early Stopping for Reasoning Modelscited by 4
  31. 2026ThinkOmni: Lifting Textual Reasoning to Omni-modal Scenarios via Guidance Decodingcited by 4
  32. 2026Self-Distillation for Multi-Token Predictioncited by 4
  33. 2026JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QAcited by 4
  34. 2026DreamReasoner-8B: Block-Size Curriculum Learning for Diffusion Reasoning Modelscited by 4
  35. 2025Unleashing the Reasoning Potential of Pre-trained LLMs by Critique Fine-Tuning on One Problemcited by 3
  36. 2025Enhancing Math Reasoning in Small-sized LLMs via Preview Difficulty-Aware Interventioncited by 3
  37. 2025TennisTV: Do Multimodal Large Language Models Understand Tennis Rallies?cited by 3
  38. 2025CrossVid: A Comprehensive Benchmark for Evaluating Cross-Video Reasoning in Multimodal Large Language Modelscited by 3

50 most cited of 117 All 117 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025JT-Math: A Multi-Stage Framework for Advanced Mathematical Reasoning in Large Language Models
  2. 2025ORION: Teaching Language Models to Reason Efficiently in the Language of Thought
  3. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility
  4. 2026The Art of Efficient Reasoning: Data, Reward, and Optimization
  5. 2025T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
  6. 2025Motif-2-12.7B-Reasoning: A Practitioner's Guide to RL Training Recipes
  7. 2025SAIL-RL: Guiding MLLMs in When and How to Think via Dual-Reward RL Tuning
  8. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training