NeMo-DCR: Bit-Exact Delta-Compressed Refit for Scalable Agentic RL at Trillion-Parameter Scale
On the map: LLM agents and tool use → Reinforcement learning for LLM agents
Abstract (arXiv)
Agentic reinforcement learning (RL) disaggregates training from rollout, so each policy update must reach the rollout clusters before the next batch. Transferring a full 1T checkpoint for such weight synchronization (refit) takes 87.5 min between two AWS regions. Measurements of BF16 training show that about 1% of weights change their stored values per step. Recent systems exploit this sparsity but fall short on placement, exactness, or efficiency: they reimplement placement rules, assemble full tensors, rebuild values arithmetically, or use a cross-cluster collective, and none fully recovers from mid-refit failures. We present NeMo-DCR (Delta-Compressed Refit), which sends only changes yet is bit-exact: receivers obtain the same parameter and buffer bits as a dense refit. For placement, fixed affine mappings project changes from training shards into the checkpoint's canonical coordinates, residual conversion covers the other changes, and the serving runtime's native loader places all changes in receiver storage. For exactness, compressible XOR masks carry affine changes whose projection and loader preserve stored bits, and overwrites carry the others. Receivers apply both in place, retries overwrite partial writes, and a joint commit binds the policy to the baseline for the next delta. For efficiency, object storage or a relay tree streams payloads during delta construction, without a cross-cluster collective. Even at 3% and 5% change rates, NeMo-DCR refits of 30B-1T models are 12-40× faster than a transport-only full-checkpoint reference. A 1T relay-tree refit at 3% takes 150 s instead of 87.5 min, making refits practical for cross-cluster agentic RL at trillion-parameter scale.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 12 of its 41 references that are in the map. First posted on arXiv 6 Oct 2026.
Builds on
12 of 41 references.
References · 12
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Kimi K2: Open Agentic Intelligencecited by 567
- 2025Defeating the Training-Inference Mismatch via FP16cited by 66
- 2025StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generationcited by 61
- 2025INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learningcited by 25
- 2026Composer 2 Technical Reportcited by 22
- 2026ROSE: Rollout On Serving GPUs via Cooperative Elasticity for Agentic RLcited by 8
- 2026Understanding and Exploiting Weight Update Sparsity for Communication-Efficient Distributed RLcited by 7
- 2026AReaL-DTE: Sparse Policy-Weight Transfer for Online Agentic Reinforcement Learningcited by 3
- 2026AuroraRL: Fast, Fault-Tolerant, and Cost-Efficient Reinforcement Learning over Decentralized Networkcited by 1
- 2026SparseRL-Sync: Lossless Weight Synchronization with ~100x Less Communicationcited by 1
- 2026Nereus: Adaptive Parallelism for LLM Post-Trainingcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026PEARL: Adaptive Prefill-Decode Execution with Elasticity for Agentic Reinforcement Learning
- 2026WeightBridge: An Efficient Weight Transfer Library for Reinforcement Learning
- 2026WAR: Workload-Aware Rollouts for Synchronous Agentic Reinforcement Learning
- 2025RollArt: Disaggregated Multi-Task Agentic RL Training at Scale
- 2026Libra: Efficient Resource Management for Agentic RL Post-Training
- 2026Missing Old Logits in Asynchronous Agentic RL: Semantic Mismatch and Repair Methods for Off-Policy Correction
- 2026Asynchronous Is Nearly Free for Evolution Strategies on Long-Horizon Agentic Tasks
- 2026SPROUT: Scaling Agentic Reinforcement Learning for Long-Horizon Software Engineering