mlsanityData to 7 Oct 2026

Rollplex: Cross-Phase GPU Spatial Sharing for Vision Language Model Post-Training

arXiv 2608.14498 ↗First posted 14 Aug 2026cs.LG cs.DCcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Vision-language models (VLMs) enable embodied agents to reason and act from visual observations and language instructions. Reinforcement learning (RL) post-training enhances these capabilities using task feedback, but current on-policy RL runtimes execute rollout, reference scoring, and actor training in strict serial phases. While effective for text-only RL, this phase-granular execution is wasteful for VLMs, where processing dense video inputs and prompt prefixes occupies a large fraction of each phase. Because prefix processing is independent of the generated response, it can be run alongside rollout decoding, which leaves GPU compute capacity underutilized, without breaking synchronous on-policy semantics. We present Rollplex, a runtime that decomposes the reference and training phase and moves the prefix computation into the rollout decode window. Realizing this schedule requires more than concurrent kernel launches: naive colocation of Qwen2.5-VL-32,B requires roughly 165,GiB per GPU, while rollout and training prefer different tensor-parallel (TP) degrees and weight layouts. Rollplex addresses these constraints with two mechanisms. Phase-aware memory management controls HBM residency according to producer--consumer lifetimes. Parallelism-aware weight sharing uses the same physical storage for layout-compatible tensors across distinct TP degrees and reconstructs only incompatible tensors, avoiding a complete second actor copy. On 32 H800 GPUs, Rollplex achieves 1.23×--1.30× speedup over serial colocation and 1.57×--2.24× over disaggregation under the same GPU budget, while preserving the synchronous RL update.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 57 references that are in the map. First posted on arXiv 14 Aug 2026.

Builds on

23 of 57 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul
  2. 2023PaLM-E: An Embodied Multimodal Language Model6 Mar

Other references · 21

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  4. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  5. 2022Efficiently Scaling Transformer Inferencecited by 529
  6. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
  7. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 234
  8. 2023DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scalescited by 107
  9. 2025Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Librarycited by 94
  10. 2024vAttention: Dynamic Memory Management for Serving LLMs without PagedAttentioncited by 93
  11. 2024OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Frameworkcited by 70
  12. 2025StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generationcited by 56
9 more
  1. 2024NeMo-Aligner: Scalable Toolkit for Efficient Model Alignmentcited by 40
  2. 2025APRIL: Active Partial Rollouts in Reinforcement Learning to Tame Long-tail Generationcited by 27
  3. 2025semi-PD: Towards Efficient LLM Serving via Phase-Wise Disaggregated Computation and Unified Storagecited by 15
  4. 2025Prefix Grouper: Efficient GRPO Training through Shared-Prefix Forwardcited by 9
  5. 2026Relax: An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scalecited by 7
  6. 2025Tree Training: Accelerating Agentic LLMs Training via Shared Prefix Reusecited by 4
  7. 2026TensorHub: Scalable and Elastic Weight Transfer for LLM RL Trainingcited by 4
  8. 2025RollMux: Phase-Level Multiplexing for Disaggregated RL Post-Trainingcited by 2
  9. 2026AREAL-DTA: Dynamic Tree Attention for Efficient Reinforcement Learning of Large Language Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Rollout Efficiency in Reinforcement Learning for Reasoning Large Language Models: A Taxonomy and Future Directionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Perturb the Thought, Not the Pixels: Latent-Space Rollout Diversification for Reinforcement Learning of Vision-Language Models
  2. 2026KerColle: Unlocking Fine-Grained GPU Concurrency in Vision-Language-Action Models
  3. 2026D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Models
  4. 2026RolloutPipe: Overlapping Pipelined Rollout and Training in Disaggregated On-Policy LLM Reinforcement Learning
  5. 2026AcceRL: A Distributed Asynchronous Reinforcement Learning and World Model Framework for Vision-Language-Action Models
  6. 2025RollArt: Disaggregated Multi-Task Agentic RL Training at Scale
  7. 2026Beyond LLM Serving: Characterizing Vision-Language-Action Workloads for Embodied AI System Design
  8. 2025Part II: ROLL Flash -- Accelerating RLVR and Agentic Training with Asynchrony