# INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning

arXiv: https://arxiv.org/abs/2505.07291 · HTML: https://mlsanity.com/paper/2505.07291/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 1 of its 45 references that are in the map.
- First posted on arXiv 12 May 2025.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 1 of 1)

- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)

## Built on by (most cited first, 25 of 25)

- [AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning](https://mlsanity.com/paper/2505.24298/) (2025)
- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning](https://mlsanity.com/paper/2506.01347/) (2025)
- [Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RL](https://mlsanity.com/paper/2508.07976/) (2025)
- [Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use](https://mlsanity.com/paper/2605.02964/) (2026)
- [Trace Length is a Simple Uncertainty Signal in Reasoning Models](https://mlsanity.com/paper/2510.10409/) (2025)
- [Mapping Post-Training Forgetting in Language Models at Scale](https://mlsanity.com/paper/2510.17776/) (2025)
- [Understanding and Exploiting Weight Update Sparsity for Communication-Efficient Distributed RL](https://mlsanity.com/paper/2602.03839/) (2026)
- [Training Large Language Models To Reason In Parallel With Global Forking Tokens](https://mlsanity.com/paper/2510.05132/) (2025)
- [A Theory of Inference Compute Scaling: Reasoning through Directed Stochastic Skill Search](https://mlsanity.com/paper/2507.00004/) (2025)
- [GEPO: Group Expectation Policy Optimization for Stable Heterogeneous Reinforcement Learning](https://mlsanity.com/paper/2508.17850/) (2025)
- [Concise Reasoning in the Lens of Lagrangian Optimization](https://mlsanity.com/paper/2510.10168/) (2025)
- [Score Centering Stabilizes Off-policy Reinforcement Learning](https://mlsanity.com/paper/2609.20807/) (2026)
- [HybridEP: Scaling Expert Parallelism to Cross-Datacenter Scenario via Hybrid Expert/Data Transmission](https://mlsanity.com/paper/2510.19470/) (2025)
- [Hail to the Thief: Exploring Attacks and Defenses in Decentralised GRPO](https://mlsanity.com/paper/2511.09780/) (2025)
- [Mixing Expert Knowledge: Bring Human Thoughts Back To the Game of Go](https://mlsanity.com/paper/2601.16447/) (2026)
- [ECHO-2: A Large-Scale Distributed Rollout Framework for Cost-Efficient Reinforcement Learning](https://mlsanity.com/paper/2602.02192/) (2026)
- [Beyond Variance: Prompt-Efficient RLVR via Rare-Event Amplification and Bidirectional Pairing](https://mlsanity.com/paper/2602.03452/) (2026)
- [F-TIS: Harnessing Diverse Models in Collaborative GRPO](https://mlsanity.com/paper/2605.22537/) (2026)
- [Real-Time Progress Prediction in Reasoning Language Models](https://mlsanity.com/paper/2506.23274/) (2025)
- [Humanline: Online Alignment as Perceptual Loss](https://mlsanity.com/paper/2509.24207/) (2025)
- [PubSwap: Public-Data Off-Policy Coordination for Federated RLVR](https://mlsanity.com/paper/2604.12160/) (2026)
- [Is Decentralized AI Governable? From Regulative Policy to Constitutive Protocol](https://mlsanity.com/paper/2605.24538/) (2026)
- [PEARL: Adaptive Prefill-Decode Execution with Elasticity for Agentic Reinforcement Learning](https://mlsanity.com/paper/2609.35158/) (2026)
- [NeMo-DCR: Bit-Exact Delta-Compressed Refit for Scalable Agentic RL at Trillion-Parameter Scale](https://mlsanity.com/paper/2610.08430/) (2026)

## Abstract (arXiv)

We introduce INTELLECT-2, the first globally distributed reinforcement learning (RL) training run of a 32 billion parameter language model. Unlike traditional centralized training efforts, INTELLECT-2 trains a reasoning model using fully asynchronous RL across a dynamic, heterogeneous swarm of permissionless compute contributors. To enable a training run with this unique infrastructure, we built various components from scratch: we introduce PRIME-RL, our training framework purpose-built for distributed asynchronous reinforcement learning, based on top of novel components such as TOPLOC, which verifies rollouts from untrusted inference workers, and SHARDCAST, which efficiently broadcasts policy weights from training nodes to inference workers. Beyond infrastructure components, we propose modifications to the standard GRPO training recipe and data filtering techniques that were crucial to achieve training stability and ensure that our model successfully learned its training objective, thus improving upon QwQ-32B, the state of the art reasoning model in the 32B parameter range. We open-source INTELLECT-2 along with all of our code and data, hoping to encourage and enable more open research in the field of decentralized training.