mlsanityData to 14 Oct 2026

TensorHub: Scalable and Elastic Weight Transfer for LLM RL Training

arXiv 2604.09107 ↗First posted 10 Apr 2026cs.DC cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Modern LLM reinforcement learning (RL) workloads require a high-performance weight transfer system to scale training across heterogeneous compute resources. However, efficiently transferring terabyte-scale model weights across thousands of GPUs remains challenging because the system must accommodate clusters that dynamically scale up and down while keeping coordination, data movement, and storage overhead low. We introduce Reference-Oriented Storage (ROS), a new storage abstraction for RL weight transfer that exploits highly replicated model weights in place. ROS presents the illusion that certain versions of the model weights are stored and can be fetched on demand. Underneath, ROS does not physically store any copies of the weights; instead, it tracks the workers that hold these weights on GPUs for inference. Upon request, ROS directly uses them to serve reads. We build TensorHub, a production-quality system that instantiates the ROS idea with topology-aware transfer, model-parallel consistency, and fault tolerance. Evaluation shows that TensorHub saturates RDMA bandwidth and adapts to three distinct rollout workloads with minimal engineering effort. Specifically, TensorHub reduces total GPU stall time by up to 6.7x for standalone rollouts, accelerates weight updates for elastic rollouts by up to 4.8x, and cuts cross-datacenter rollout stall time by up to 19x. TensorHub has been deployed in ByteDance production to support cutting-edge RL training.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 31 of its 52 references that are in the map. First posted on arXiv 10 Apr 2026.

Builds on

31 of 52 references.

References · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2017Deep reinforcement learning from human preferencescited by 3,912
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  10. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  11. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  12. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
19 more
  1. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
  2. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 279
  3. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutioncited by 263
  4. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 250
  5. 2024RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Foldcited by 99
  6. 2024Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Modelscited by 70
  7. 2025StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generationcited by 61
  8. 2024Optimizing RLHF Training for Large Language Models with Stage Fusioncited by 43
  9. 2025LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Trainingcited by 42
  10. 2025History Rhymes: Accelerating LLM Reinforcement Learning with RhymeRLcited by 38
  11. 2024ReaL: Efficient RLHF Training of Large Language Models with Parameter Reallocationcited by 37
  12. 2025Laminar: A Scalable Asynchronous RL Post-Training Frameworkcited by 37
  13. 2025RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Trainingcited by 35
  14. 2024RL-GPT: Integrating Reinforcement Learning and Code-as-policycited by 32
  15. 2025APRIL: Active Partial Rollouts in Reinforcement Learning to Tame Long-tail Generationcited by 30
  16. 2024ByteCheckpoint: A Unified Checkpointing System for Large Foundation Model Developmentcited by 24
  17. 2025Taming the Long-Tail: Efficient Reasoning RL Training with Adaptive Draftercited by 14
  18. 2025RLBoost: Harvesting Preemptible Resources for Cost-Efficient Reinforcement Learning on LLMscited by 11
  19. 2020Hoplite: Efficient and Fault-Tolerant Collective Communication for Task-Based Distributed Systemscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026JigsawRL: Assembling RL Pipelines for Efficient LLM Post-Trainingcited by 1
  2. 2026Rollplex: Cross-Phase GPU Spatial Sharing for Vision Language Model Post-Trainingcited by 1
  3. 2026WeightBridge: An Efficient Weight Transfer Library for Reinforcement Learningcited by 0
  4. 2026PEARL: Adaptive Prefill-Decode Execution with Elasticity for Agentic Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RL in the Wild: Characterizing RLVR Training in LLM Deployment
  2. 2026Scaling LLM Inference Beyond Amdahl`s Limits via Eliminating Non-Scalable Overheads ↗
  3. 2026Accelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism
  4. 2026Beyond the Trial-and-Error Loop: Hybrid Projection and Automated Tuning for Distributed Training
  5. 2024TPI-LLM: Serving 70B-scale LLMs Efficiently on Low-resource Edge Devices
  6. 2025The Art of Scaling Reinforcement Learning Compute for LLMs
  7. 2026IsoCompute Playbook: Optimally Scaling Sampling Compute for LLM RL
  8. 2026JointServe: Serving Multi-Model Agents on Memory-Constrained GPUs