mlsanityData to 14 Oct 2026

LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget

arXiv 2607.14952 ↗First posted 16 Jul 2026cs.LG cs.DCcited by 3

Not on the map. Why?

Abstract (arXiv)

Long-context RL post-training is constrained by the lifetime of state and gradients, not attention cost alone. In GRPO, one multi-million-token prompt must serve old-policy and reference scoring plus multiple policy responses, while conventional autograd keeps the prompt graph and all response graphs live alongside model weights, caches, and distributed communication buffers. We present LongStraw, an objective-aware, architecture-aware system for resident-state virtualization, response replay, and distributed-gradient execution. Its transaction captures the shared prompt without autograd, retains only the architecture-required state on explicitly owned pages, restores that state for each group member, scores old/reference branches without a graph, replays one policy response at a time with autograd, and accumulates the resulting gradients before one distributed finalization and optimizer step. This schedule bounds the live training graph by the response suffix while reusing the expensive prompt computation across the complete GRPO group. We instantiate this design for two incompatible model structures. Qwen3.6-27B combines 48 recurrent GDN layers with 16 full-attention layers; LongStraw keeps the compact recurrent state and physically CP8-sharded KV pages, composes global attention through cross-rank LSE/output merging, and performs blockwise response replay. GLM-5.2 combines a 78-layer MLA/DSA attention stack with a 256-expert, top-8 MoE tail. Its implementation keeps CP-sharded MLA latent pages and DSA indexer-key pages in CPU memory, stages one layer at a time, reconstructs IndexShare-aware global sparse selection over CP32, and dispatches routed response tokens over EP32. The two paths share one transaction contract while specializing the retained state, replay operator, and collective communication to the architecture...

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 52 of its 66 references that are in the map. First posted on arXiv 16 Jul 2026.

Builds on

52 of 66 references.

References · 52

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  7. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  8. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  10. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  11. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  12. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
38 more
  1. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  3. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  4. 2020GLU Variants Improve Transformercited by 1,804
  5. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  6. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  7. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  8. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  9. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  10. 2016Training Deep Nets with Sublinear Memory Costcited by 848
  11. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
  12. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  13. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 632
  14. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
  15. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
  16. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 457
  17. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 341
  18. 2022Reducing Activation Recomputation in Large Transformer Modelscited by 302
  19. 2021ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learningcited by 256
  20. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 250
  21. 2023LongLoRA: Efficient Fine-tuning of Long-Context Large Language Modelscited by 244
  22. 2022MegaBlocks: Efficient Sparse Training with Mixture-of-Expertscited by 209
  23. 2022Tutel: Adaptive Mixture-of-Experts at Scalecited by 201
  24. 2021Self-attention Does Not Need O(n2) Memorycited by 154
  25. 2023S-LoRA: Serving Thousands of Concurrent LoRA Adapterscited by 152
  26. 2023DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scalescited by 108
  27. 2023Punica: Multi-Tenant LoRA Servingcited by 83
  28. 2024USP: A Unified Sequence Parallelism Approach for Long Context Generative AIcited by 61
  29. 2025LongCat-Flash Technical Reportcited by 58
  30. 2026LongCat-Flash-Thinking-2601 Technical Reportcited by 34
  31. 2026IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reusecited by 31
  32. 2023DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Trainingcited by 29
  33. 2024LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelismcited by 17
  34. 2025MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Corecited by 17
  35. 2024RankAdaptor: Hierarchical Rank Allocation for Efficient Fine-Tuning Pruned LLMs via Performance Modelcited by 7
  36. 2025Balancing Fidelity and Plasticity: Aligning Mixed-Precision Fine-Tuning with Linguistic Hierarchiescited by 5
  37. 2026AutoQRA: Joint Optimization of Mixed-Precision Quantization and Low-rank Adapters for Efficient LLM Fine-Tuningcited by 5
  38. 2025Large Language Model Compression with Global Rank and Sparsity Optimizationcited by 4

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRAcited by 1
  2. 2026Learning how to Forget: Fine-tuning for Long-Context Sparse Attentioncited by 0
  3. 2026HiLoRe: What to Store, Compress, or Recompute for Efficient GRPO Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026DualKV: Shared-Prompt Flash Attention for Efficient RL Training with Large Rollouts and Long Contexts
  2. 2025Laminar: A Scalable Asynchronous RL Post-Training Framework
  3. 2025RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Training
  4. 2025RLBoost: Harvesting Preemptible Resources for Cost-Efficient Reinforcement Learning on LLMs
  5. 2025AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training
  6. 2026RolloutPipe: Overlapping Pipelined Rollout and Training in Disaggregated On-Policy LLM Reinforcement Learning
  7. 2025Seer: Online Context Learning for Fast Synchronous LLM Reinforcement Learning
  8. 2026Orchestrating Tokens and Sequences: Dynamic Hybrid Policy Optimization for RLVR