AcceRL: A Distributed Asynchronous Reinforcement Learning and World Model Framework for Vision-Language-Action Models
On the map: World models → Embodied and robot world models
Abstract (arXiv)
Reinforcement learning (RL) for large-scale Vision-Language-Action (VLA) models is severely bottlenecked by synchronization barriers and the high cost of environment data acquisition. To overcome these challenges, we propose AcceRL, a distributed asynchronous RL framework that physically isolates environment rollouts, model inference, and gradient updates. By eliminating the cascading long-tail idle bubbles inherent in synchronous systems, AcceRL maximizes hardware utilization and ensures scalable throughput. Furthermore, AcceRL features a modular design that supports the integration of diverse, plug-and-play world models into its distributed pipeline. Extensive experiments demonstrate that the base framework achieves highly competitive performance across all four LIBERO~citeliu2023libero task suites. Systematically, the asynchronous architecture delivers a 2.4× throughput speedup over leading synchronous baselines. Algorithmically, by leveraging a world model pre-trained on 1,000 offline trajectories, AcceRL achieves up to a 200× improvement in online sample efficiency on LIBERO-Spatial, establishing a robust framework that is both sample-efficient and time-efficient for embodied AI. Code is included in the supplementary material. Code is available at https://github.com/distanceLu/AcceRL.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 35 of its 39 references that are in the map. First posted on arXiv 19 Mar 2026.
Builds on
35 of 39 references.
Closest ancestors · 2
Same direction, newest first
- 2026WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL15 Feb
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
Other references · 33
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
- 2018World Modelscited by 1,444
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
21 more
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2024Genie: Generative Interactive Environmentscited by 706
- 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
- 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
- 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 234
- 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
- 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learningcited by 141
- 2023DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scalescited by 107
- 2025Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Librarycited by 94
- 2025RLinf: Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformationcited by 51
- 2025LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Trainingcited by 37
- 2025Laminar: A Scalable Asynchronous RL Post-Training Frameworkcited by 35
- 2025SAC Flow: Sample-Efficient Reinforcement Learning of Flow-Based Policies via Velocity-Reparameterized Sequential Modelingcited by 27
- 2025RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Modelscited by 26
- 2025DistFlow: A Fully Distributed RL Framework for Scalable and Efficient LLM Post-Trainingcited by 9
- 2026RL-VLA3: A Flexible and Asynchronous Reinforcement Learning Framework for VLA Trainingcited by 7
- 2025RLLaVA: An RL-central Framework for Language and Vision Assistantscited by 1
- 2026GIPO: Gaussian Importance Sampling Policy Optimizationcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Models
- 2025Enhancing Vision-Language Model Training with Reinforcement Learning in Synthetic Worlds for Real-World Success
- 2026AsyncWebRL: Efficient Asynchronous Reinforcement Learning for Multi-Step Visual Web Agents
- 2026Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Models
- 2026Reinforcement Learning for Real-Time Vision-Language-Action Policies
- 2026VLA-Precision: Asymmetric Co-Bootstrapping for Efficient Real-World Online RL of Vision-Language-Action Models
- 2026Scaling Sim-to-Real VLA Reinforcement Learning with Generative 3D Worlds
- 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learning