mlsanityData to 14 Oct 2026

JigsawRL: Assembling RL Pipelines for Efficient LLM Post-Training

arXiv 2604.23838 ↗First posted 26 Apr 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

We present JigsawRL, a cost-efficient framework that explores Pipeline Multiplexing as a new dimension of RL parallelism. JigsawRL decomposes each pipeline into a Sub-Stage Graph that exposes the intra-stage and inter-worker imbalance hidden by stage-level systems. On this abstraction, JigsawRL resolves multiplexing interference through dynamic resource allocation, eliminates fragmented utilization by migrating long-tail rollouts across workers, and formulates their coordination as a graph scheduling problem solved with a look-ahead heuristic. On 4-64 H100/A100 GPUs across different agentic RL pipelines and models, JigsawRL achieves up to 1.85x throughput over Verl on synchronous RL, 1.54x over StreamRL and AReaL on asynchronous RL, and supports heterogeneous pipelines with moderate latency trade-off.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 51 of its 65 references that are in the map. First posted on arXiv 26 Apr 2026.

Builds on

51 of 65 references.

References · 51

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  8. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  9. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  10. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  11. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  12. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
38 more
  1. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  2. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  3. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  4. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  5. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  6. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  7. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  8. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 392
  9. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 386
  10. 2024The Faiss librarycited by 376
  11. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 250
  12. 2023DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scalescited by 108
  13. 2025Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Librarycited by 101
  14. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 88
  15. 2025In-the-Flow Agentic System Optimization for Effective Planning and Tool Usecited by 72
  16. 2024Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Modelscited by 70
  17. 2025StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generationcited by 61
  18. 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolutioncited by 60
  19. 2025RLinf: Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformationcited by 57
  20. 2025rStar2-Agent: Agentic Reasoning Technical Reportcited by 54
  21. 2025Prosperity before Collapse: How Far Can Off-Policy RL Reach with Stale Data on LLMs?cited by 53
  22. 2024MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Servingcited by 52
  23. 2024Optimizing RLHF Training for Large Language Models with Stage Fusioncited by 43
  24. 2024ReaL: Efficient RLHF Training of Large Language Models with Parameter Reallocationcited by 37
  25. 2025Laminar: A Scalable Asynchronous RL Post-Training Frameworkcited by 37
  26. 2025RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Trainingcited by 35
  27. 2025Seer: Online Context Learning for Fast Synchronous LLM Reinforcement Learningcited by 25
  28. 2025Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooningcited by 24
  29. 2025ReSpec: Towards Optimizing Speculative Decoding in Reinforcement Learning Systemscited by 18
  30. 2025WLB-LLM: Workload-Balanced 4D Parallelism for Large Language Model Trainingcited by 17
  31. 2025RollArt: Disaggregated Multi-Task Agentic RL Training at Scalecited by 14
  32. 2025RLBoost: Harvesting Preemptible Resources for Cost-Efficient Reinforcement Learning on LLMscited by 11
  33. 2025Beat the long tail: Distribution-Aware Speculative Decoding for RL Trainingcited by 10
  34. 2026LLMs Can Learn to Reason Via Off-Policy RLcited by 10
  35. 2025RLHFSpec: Breaking the Efficiency Bottleneck in RLHF Training via Adaptive Draftingcited by 5
  36. 2026OrchestrRL: Dynamic Compute and Network Orchestration for Disaggregated RLcited by 5
  37. 2026TensorHub: Scalable and Elastic Weight Transfer for LLM RL Trainingcited by 4
  38. 2025RollMux: Phase-Level Multiplexing for Disaggregated RL Post-Trainingcited by 3

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026ThunderSyncRL: Lossless Acceleration of Agentic Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AReaL-Hex: Accommodating Asynchronous RL Training over Heterogeneous GPUs
  2. 2025gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
  3. 2025FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
  4. 2026AuroraRL: Fast, Fault-Tolerant, and Cost-Efficient Reinforcement Learning over Decentralized Network
  5. 2026VPP: Virtual Pipeline Parallelism for Efficient Chunked Prefill in Long-Context LLM Inference
  6. 2026PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR
  7. 2026PipeMax: Enhancing Offline LLM Inference on Commodity GPU Servers
  8. 2026MOCAP: Wafer-Scale-Chip-Oriented Memory-Orchestrated Chunked Pipelining Framework for Prefill-Only LLM Inference