mlsanityData to 14 Oct 2026

MARLaaS: Multi-Tenant Asynchronous Reinforcement Learning as a Service

arXiv 2605.08527 ↗First posted 8 May 2026cs.DC cs.AIcited by 1

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Reinforcement Learning from Verifiable Rewards (RLVR) has significantly improved the reasoning capabilities of large language models (LLMs), particularly in multi-turn agentic settings involving environment interaction like tool use. However, fine-tuning such models remains prohibitively expensive due to high computational requirements, limiting accessibility. We propose MARLaaS (Multi-tenant Asynchronous RL as a Service), a system for concurrent RL fine-tuning across multiple users and tasks. Our approach is based on two key ideas: (1) sharing a base model across tenants using lightweight LoRA adapters, and (2) a disaggregated asynchronous architecture that decouples rollout generation, environment interaction, and policy training into independently scheduled stages. This design enables tasks to progress through the RL pipeline at their own pace in an event-driven manner, reducing cross-task interference, idle time, and end-to-end latency. In multi-task settings (we report up to 32 concurrent tasks), MARLaaS achieves single-task state-of-the-art performance while improving accelerator utilization by up to 4.3x and reducing end-to-end training time by 85%.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 26 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

20 of 26 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning30 May
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  3. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  7. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  8. 2023S-LoRA: Serving Thousands of Concurrent LoRA Adapterscited by 152
  9. 2024Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Modelscited by 70
  10. 2025SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agentcited by 48
  11. 2025LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Trainingcited by 42
  12. 2025Laminar: A Scalable Asynchronous RL Post-Training Frameworkcited by 37
5 more
  1. 2023mLoRA: Fine-Tuning LoRA Adapters via Highly-Efficient Pipeline Parallelism in Multiple GPUscited by 12
  2. 2026StaleFlow: Staleness-Aware Data Management for Mitigating Data Skewness in Fully Disaggregated RL Post-Trainingcited by 11
  3. 2025LobRA: Multi-tenant Fine-tuning over Heterogeneous Datacited by 4
  4. 2026tLoRA: Efficient Multi-LoRA Training with Elastic Shared Super-Modelscited by 2
  5. 2026ALTO: Adaptive LoRA Tuning and Orchestration for Heterogeneous LoRA Training Workloadscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026JoyNexus: Service-Oriented Multi-Tenant Post-Training for VLA Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026RL-VLA3: A Flexible and Asynchronous Reinforcement Learning Framework for VLA Training
  2. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments
  3. 2026Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement
  4. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
  5. 2025Periodic Asynchrony: An On-Policy Approach for Accelerating LLM Reinforcement Learning
  6. 2026Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RL
  7. 2026PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR
  8. 2025AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training