mlsanityData to 7 Oct 2026

MiMo-V2-Flash Technical Report

arXiv 2601.02780 ↗First posted 6 Jan 2026cs.CL cs.AIcited by 163

On the map: Mixture-of-experts → MoE in reinforcement learning and robotics

Abstract (arXiv)

We present MiMo-V2-Flash, a Mixture-of-Experts (MoE) model with 309B total parameters and 15B active parameters, designed for fast, strong reasoning and agentic capabilities. MiMo-V2-Flash adopts a hybrid attention architecture that interleaves Sliding Window Attention (SWA) with global attention, with a 128-token sliding window under a 5:1 hybrid ratio. The model is pre-trained on 27 trillion tokens with Multi-Token Prediction (MTP), employing a native 32k context length and subsequently extended to 256k. To efficiently scale post-training compute, MiMo-V2-Flash introduces a novel Multi-Teacher On-Policy Distillation (MOPD) paradigm. In this framework, domain-specialized teachers (e.g., trained via large-scale reinforcement learning) provide dense and token-level reward, enabling the student model to perfectly master teacher expertise. MiMo-V2-Flash rivals top-tier open-weight models such as DeepSeek-V3.2 and Kimi-K2, despite using only 1/2 and 1/3 of their total parameters, respectively. During inference, by repurposing MTP as a draft model for speculative decoding, MiMo-V2-Flash achieves up to 3.6 acceptance length and 2.6x decoding speedup with three MTP layers. We open-source both the model weights and the three-layer MTP weights to foster open research and community collaboration.

Key facts. Cited by 163 papers in the map (data through 7 Oct 2026). Placed in MoE in reinforcement learning and robotics, one of 12 directions in Mixture-of-experts. Builds on 61 of its 77 references that are in the map. First posted on arXiv 6 Jan 2026.

Builds on

61 of 77 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers13 Oct
  2. 2025Kimi K2: Open Agentic Intelligence28 Jul
  3. 2025Group Sequence Policy Optimization24 Jul

Other references · 58

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  10. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  11. 2021Program Synthesis with Large Language Modelscited by 3,509
  12. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
38 more
  1. 2020Longformer: The Long-Document Transformercited by 3,362
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  8. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  9. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  10. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  11. 2025Gemma 3 Technical Reportcited by 1,686
  12. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  13. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  14. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  15. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  16. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  17. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  18. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  19. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  20. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
  21. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  22. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
  23. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
  24. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  25. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  26. 2025Humanity's Last Examcited by 467
  27. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  28. 2015Policy Distillationcited by 414
  29. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
  30. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 344
  31. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  32. 2024Massive Activations in Large Language Modelscited by 277
  33. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 275
  34. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 266
  35. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 234
  36. 2024When Attention Sink Emerges in Language Models: An Empirical Viewcited by 200
  37. 2024Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluationcited by 181
  38. 2025Kimi Linear: An Expressive, Efficient Attention Architecturecited by 169

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 163.

163 papers.

Sort:

By area:

  1. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipecited by 262
  2. 2026Gemma 4 Technical Reportcited by 187
  3. 2026Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolationcited by 157
  4. 2026Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixescited by 122
  5. 2026Kimi K3: Open Frontier Intelligencecited by 99
  6. 2026Scaling Reasoning Efficiently via Relaxed On-Policy Distillationcited by 71
  7. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 49
  8. 2026Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipecited by 43
  9. 2026RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillationcited by 33
  10. 2026Video-OPD: Efficient Post-Training of Multimodal Large Language Models for Temporal Video Grounding via On-Policy Distillationcited by 32
  11. 2026Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 32
  12. 2026LongCat-Next: Lexicalizing Modalities as Discrete Tokenscited by 30
38 more
  1. 2026Flow-OPD: On-Policy Distillation for Flow Matching Modelscited by 29
  2. 2026Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillationcited by 28
  3. 2026IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reusecited by 27
  4. 2026Demystifying OPD: Length Inflation and Stabilization Strategies for Large Language Modelscited by 24
  5. 2026On the Position Bias of On-Policy Distillationcited by 23
  6. 2026SWE-World: Building Software Engineering Agents in Docker-Free Environmentscited by 21
  7. 2026Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Whycited by 21
  8. 2026On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agentscited by 21
  9. 2026Immersion in the GitHub Universe: Scaling Coding Agents to Masterycited by 20
  10. 2026BeyondSWE: Can Current Code Agent Survive Beyond Single-Repo Bug Fixing?cited by 20
  11. 2026SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Trainingcited by 18
  12. 2026MM-DeepResearch: A Simple and Effective Multimodal Agentic Search Baselinecited by 18
  13. 2026SWE-Universe: Scale Real-World Verifiable Environments to Millionscited by 16
  14. 2026Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillationcited by 16
  15. 2026STEPS: Selective On-Policy Self-Distillation for Reasoningcited by 15
  16. 2026Rethinking On-Policy Distillation of Large Language Models II: One Training Examplecited by 15
  17. 2026PyVision-RL: Forging Open Agentic Vision Models via RLcited by 14
  18. 2026OGLS-SD: On-Policy Self-Distillation with Outcome-Guided Logit Steering for LLM Reasoningcited by 14
  19. 2026Trajectory-Refined Distillationcited by 14
  20. 2026LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoningcited by 13
  21. 2026Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Informationcited by 12
  22. 2026Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidancecited by 12
  23. 2026JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Draftingcited by 12
  24. 2026Prefill-as-a-Service: KVCache of Next-Generation Models Could Go Cross-Datacentercited by 11
  25. 2026Rubric-based On-policy Distillationcited by 11
  26. 2026Self-Supervised On-Policy Distillation for Reasoning Language Modelscited by 10
  27. 2026Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy Distillationcited by 10
  28. 2026Pass the Baton: Trajectory-Relayed On-Policy Distillationcited by 10
  29. 2026On-policy Distillation with Verifiable Rewardcited by 10
  30. 2026To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Modelscited by 9
  31. 2026GameGen-Verifier: Parallel Keypoint-Based Verification for LLM-Generated Games via Runtime State Injectioncited by 9
  32. 2026Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Trainingcited by 9
  33. 2026Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergencecited by 9
  34. 2026Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scalecited by 9
  35. 2026Reinforcement Learning from Rich Feedback with Distributional DAggercited by 9
  36. 2026OPRD: On-Policy Representation Distillationcited by 9
  37. 2026Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Samplingcited by 9
  38. 2026DOPD: Dual On-policy Distillationcited by 9

50 most cited of 163 All 163 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters
  2. 2025MiMo-VL Technical Report
  3. 2026JoyAI-LLM Flash: Advancing Mid-Scale LLMs with Token Efficiency
  4. 2025LongCat-Flash Technical Report
  5. 2025Introducing LongCat-Flash-Thinking: A Technical Report
  6. 2026Instella-MoE Technical Report
  7. 2026DualKV: Shared-Prompt Flash Attention for Efficient RL Training with Large Rollouts and Long Contexts
  8. 2026FastOPD: On-Policy Distillation for Lightweight VLA Deployment