mlsanityData to 7 Oct 2026

HiLoRe: What to Store, Compress, or Recompute for Efficient GRPO Training

arXiv 2609.33570 ↗First posted 27 Sep 2026cs.LGcited by 0

New in LLM reasoning; no direction yet. LLM reasoning

Abstract (arXiv)

Group-relative policy optimization (GRPO) makes learner-side activations a major memory-computation bottleneck: gradient checkpointing reduces activation memory through recomputation, but fixed schedules can leave roughly 18 GB unused on a 48-GB GPU despite substantial recomputation overhead. Existing activation-management methods set state fidelity from execution cost, tensor properties, or generic compression sensitivity, without explicitly incorporating GRPO's analytic update structure into state-fidelity allocation. We formalize this dependence as policy-update exposure, linking the current GRPO loss coefficients to state-level approximation sensitivity. These coefficients are available before backward without an additional backward pass. We introduce HiLoRe, which allocates graph-attributed recovery units among high-precision storage, low-precision compression, and deterministic recomputation using measured recovery utility and update-conditioned approximation risk. It combines high-precision storage and deterministic recomputation with low-precision recovery under a calibrated risk budget. Across five model-task settings with 2K responses and memory < 1.10 times GC's per-GPU actor-update peak, HiLoRe's actor-update throughput gains reach 13.5% over GC and 7.9% over the fastest evaluated baseline, with paired mean downstream-score differences below 0.6 percentage points.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 56 references that are in the map. First posted on arXiv 27 Sep 2026.

Builds on

25 of 56 references.

References · 25

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2024Qwen2.5 Technical Reportcited by 4,713
  5. 2023Let's Verify Step by Stepcited by 3,598
  6. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  7. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  8. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 261
  9. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 260
  10. 2023TACO: Topics in Algorithmic COde generation datasetcited by 150
  11. 2024OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Frameworkcited by 70
  12. 2026FP8-RL: A Practical and Stable Low-Precision Stack for LLM Reinforcement Learningcited by 10
13 more
  1. 2025Infinite Sampling: Efficient and Stable Grouped RL Training for Large Language Modelscited by 9
  2. 2026Sparrow: Sparse Rollout for Stable and Efficient Long-context RL of Large Language Modelscited by 4
  3. 2025Adacc: An Adaptive Framework Unifying Compression and Activation Recomputation for LLM Trainingcited by 3
  4. 2026LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budgetcited by 3
  5. 2026Not all tokens are needed(NAT): token efficient reinforcement learningcited by 2
  6. 2026How to Compress KV Cache in RL Post-Training? Shadow Mask Distillation for Memory-Efficient Alignmentcited by 2
  7. 2026DualKV: Shared-Prompt Flash Attention for Efficient RL Training with Large Rollouts and Long Contextscited by 2
  8. 2026OASIS: Online Activation Subspace Learning for Memory-Efficient Trainingcited by 1
  9. 2026Cost-Aware Learningcited by 1
  10. 2026Activation Compression in LLMs: Theoretical Analysis and Efficient Algorithmcited by 1
  11. 2026Schedule-Level Shared-Prefix Reuse for LLM RL Trainingcited by 1
  12. 2026QUADS: Stabilizing NVFP4 Reinforcement Learning for MoE via QUantization-error Alignment across Dual Sidescited by 1
  13. 2026LazyTrain: Limited-resource Allocation toward Zero-waste Yield Optimization in Large Language Model Trainingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic
  2. 2026GRPO Training Dynamics for Small Language Models
  3. 2026How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning
  4. 2026F-GRPO: Don't Let Your Policy Learn the Obvious and Forget the Rare
  5. 2025TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback
  6. 2026Beyond Retraining-Free MoE Compression: A Cost-Normalized Study of Post-Compression Adjustment
  7. 2026ReCo: Reweighting GRPO Against Distributional Concentration
  8. 2025GRPO-Guard: Mitigating Implicit Over-Optimization in Flow Matching via Regulated Clipping