mlsanityData to 14 Oct 2026

MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention

arXiv 2506.13585 ↗First posted 16 Jun 2025cs.CL cs.LGcited by 263

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

We introduce MiniMax-M1, the world's first open-weight, large-scale hybrid-attention reasoning model. MiniMax-M1 is powered by a hybrid Mixture-of-Experts (MoE) architecture combined with a lightning attention mechanism. The model is developed based on our previous MiniMax-Text-01 model, which contains a total of 456 billion parameters with 45.9 billion parameters activated per token. The M1 model natively supports a context length of 1 million tokens, 8x the context size of DeepSeek R1. Furthermore, the lightning attention mechanism in MiniMax-M1 enables efficient scaling of test-time compute. These properties make M1 particularly suitable for complex tasks that require processing long inputs and thinking extensively. MiniMax-M1 is trained using large-scale reinforcement learning (RL) on diverse problems including sandbox-based, real-world software engineering environments. In addition to M1's inherent efficiency advantage for RL training, we propose CISPO, a novel RL algorithm to further enhance RL efficiency. CISPO clips importance sampling weights rather than token updates, outperforming other competitive RL variants. Combining hybrid-attention and CISPO enables MiniMax-M1's full RL training on 512 H800 GPUs to complete in only three weeks, with a rental cost of just 534,700. We release two versions of MiniMax-M1 models with 40K and 80K thinking budgets respectively, where the 40K model represents an intermediate phase of the 80K training. Experiments on standard benchmarks show that our models are comparable or superior to strong open-weight models such as the original DeepSeek-R1 and Qwen3-235B, with particular strengths in complex software engineering, tool utilization, and long-context tasks. We publicly release MiniMax-M1 at https://github.com/MiniMax-AI/MiniMax-M1.

Key facts. Cited by 263 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 68 of its 77 references that are in the map. First posted on arXiv 16 Jun 2025.

Builds on

68 of 77 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Titans: Learning to Memorize at Test Time31 Dec
  2. 2022cosFormer: Rethinking Softmax in Attention17 Feb

Other references · 66

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  8. 2020Longformer: The Long-Document Transformercited by 3,413
  9. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  12. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
38 more
  1. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  2. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  3. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  6. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  7. 2020Rethinking Attention with Performerscited by 1,601
  8. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  9. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  10. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  11. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  12. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  13. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
  14. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 605
  15. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 559
  16. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 514
  17. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  18. 2025Humanity's Last Examcited by 491
  19. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 485
  20. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 461
  21. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 421
  22. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  23. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 396
  24. 2024Measuring short-form factuality in large language modelscited by 373
  25. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
  26. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 355
  27. 2021Random Feature Attentioncited by 354
  28. 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
  29. 2024Simple linear attention language models balance the recall-throughput tradeoffcited by 208
  30. 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoningcited by 201
  31. 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 198
  32. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 180
  33. 2024Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modelingcited by 174
  34. 2025MiniMax-01: Scaling Foundation Models with Lightning Attentioncited by 166
  35. 2025Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learningcited by 161
  36. 2024Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrencecited by 159
  37. 2017Parallelizing Linear Recurrent Neural Nets Over Sequence Lengthcited by 127
  38. 2022How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projectionscited by 122

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 69; 2026: 194.

263 papers.

Sort:

By area:

  1. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
  2. 2026Reinforcement Learning via Self-Distillationcited by 332
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  4. 2025SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoningcited by 146
  5. 2025MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Serverscited by 109
  6. 2025Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?cited by 95
  7. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 91
  8. 2026Scaling Reasoning Efficiently via Relaxed On-Policy Distillationcited by 82
  9. 2025Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routerscited by 72
  10. 2026Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervisioncited by 68
  11. 2025Defeating the Training-Inference Mismatch via FP16cited by 66
  12. 2025SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarioscited by 59
38 more
  1. 2025Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoningcited by 56
  2. 2025Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decodingcited by 55
  3. 2025rStar2-Agent: Agentic Reasoning Technical Reportcited by 54
  4. 2025Intern-S1: A Scientific Multimodal Foundation Modelcited by 53
  5. 2025Prosperity before Collapse: How Far Can Off-Policy RL Reach with Stale Data on LLMs?cited by 53
  6. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 53
  7. 2025Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Modelscited by 48
  8. 2025Jet-Nemotron: Efficient Language Model with Post Neural Architecture Searchcited by 41
  9. 2025Kimi-Dev: Agentless Training as Skill Prior for SWE-Agentscited by 41
  10. 2025Stabilizing Reinforcement Learning with LLMs: Formulation and Practicescited by 41
  11. 2025Toward Training Superintelligent Software Agents through Self-Play SWE-RLcited by 40
  12. 2025Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving Clipping Policy Optimizationcited by 39
  13. 2026Rethinking the Trust Region in LLM Reinforcement Learningcited by 37
  14. 2025Towards a Unified View of Large Language Model Post-Trainingcited by 36
  15. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 35
  16. 2026Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillationcited by 33
  17. 2026Tmax: A simple recipe for terminal agentscited by 33
  18. 2025EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modescited by 32
  19. 2025Single-stream Policy Optimizationcited by 28
  20. 2025MedVLThinker: Simple Baselines for Multimodal Medical Reasoningcited by 27
  21. 2026Look Inward to Explore Outward: Learning Temperature Policy from LLM Internal States via Hierarchical RLcited by 27
  22. 2026KL for a KL: On-Policy Distillation with Control Variate Baselinecited by 26
  23. 2025Differential Smoothing Mitigates Sharpening and Improves LLM Reasoningcited by 22
  24. 2025When Importance Sampling Misallocates Credit: Asymmetric Ratios for Outcome-Supervised RLcited by 21
  25. 2025Mid-Training of Large Language Models: A Surveycited by 21
  26. 2025InternBootcamp: Boosting LLM Reasoning with Verifiable Task Scalingcited by 20
  27. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  28. 2026MEMENTO: Teaching LLMs to Manage Their Own Contextcited by 18
  29. 2026Rethinking the Design Space of Reinforcement Learning for Diffusion Models: On the Importance of Likelihood Estimation Beyond Loss Designcited by 17
  30. 2026On-Policy Self-Distillation without Any Supervisioncited by 17
  31. 2025DRPO: Efficient Reasoning via Decoupled Reward Policy Optimizationcited by 16
  32. 2026Reuse your FLOPs: Scaling RL on Hard Problems by Conditioning on Very Off-Policy Prefixescited by 16
  33. 2026PyVision-RL: Forging Open Agentic Vision Models via RLcited by 16
  34. 2025Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friendscited by 15
  35. 2025Part II: ROLL Flash -- Accelerating RLVR and Agentic Training with Asynchronycited by 15
  36. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
  37. 2026Multi-Task GRPO: Reliable LLM Reasoning Across Taskscited by 14
  38. 2025Scaling Linear Attention with Sparse State Expansioncited by 13

50 most cited of 263 All 263 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Models
  2. 2025Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic Confidence
  3. 2025Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space
  4. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  5. 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinking
  6. 2025Prompting Test-Time Scaling Is A Strong LLM Reasoning Data Augmentation
  7. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
  8. 2025Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers