mlsanityData to 7 Oct 2026

INTELLECT-3: Technical Report

arXiv 2512.16144 ↗First posted 18 Dec 2025cs.LG cs.AIcited by 3

On the map: Mixture-of-experts → MoE in reinforcement learning and robotics

Abstract (arXiv)

We present INTELLECT-3, a 106B-parameter Mixture-of-Experts model (12B active) trained with large-scale reinforcement learning on our end-to-end RL infrastructure stack. INTELLECT-3 achieves state of the art performance for its size across math, code, science and reasoning benchmarks, outperforming many larger frontier models. We open-source the model together with the full infrastructure stack used to create it, including RL frameworks, complete recipe, and a wide collection of environments, built with the verifiers library, for training and evaluation from our Environments Hub community platform. Built for this effort, we introduce prime-rl, an open framework for large-scale asynchronous reinforcement learning, which scales seamlessly from a single node to thousands of GPUs, and is tailored for agentic RL with first-class support for multi-turn interactions and tool use. Using this stack, we run both SFT and RL training on top of the GLM-4.5-Air-Base model, scaling RL training up to 512 H200s with high training efficiency.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in MoE in reinforcement learning and robotics, one of 12 directions in Mixture-of-experts. Builds on 16 of its 37 references that are in the map. First posted on arXiv 18 Dec 2025.

Builds on

16 of 37 references.

References · 16

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  6. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
  7. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  8. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
  9. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
  10. 2024Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Expertscited by 191
  11. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 180
  12. 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agentscited by 140
4 more
  1. 2025The Art of Scaling Reinforcement Learning Compute for LLMscited by 91
  2. 2024TorchTitan: One-stop PyTorch native solution for production ready LLM pre-trainingcited by 80
  3. 2025Defeating the Training-Inference Mismatch via FP16cited by 62
  4. 2025SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyondcited by 47

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routerscited by 5
  2. 2026TimeSeek: Temporal Reliability of Agentic Forecasterscited by 1
  3. 2026Rufus-Air: An Open LLM Post-Training Recipecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning
  2. 2026Mach-Mind-4-Flash Technical Report
  3. 2025Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts
  4. 2026Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters
  5. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  6. 2025Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs
  7. 2025AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy
  8. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild