mlsanityData to 7 Oct 2026

Cost-Aware Learning

arXiv 2604.28020 ↗First posted 30 Apr 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

We consider the problem of Cost-Aware Learning, where sampling different components of a finite-sum objective incurs different costs. The objective is to reach a target error while minimizing the total cost. We propose Cost-Aware SGD, which uses a distribution based on gradient norms and costs to sample components. We provide a thorough analysis of this algorithm, including cost-improvement bounds over baselines, a characterization of distribution proxy sub-optimality, and a lower bound. We apply our theoretical insights to reinforcement learning with language models, where the computational cost of sequence-level policy gradients varies with length. We find that the advantage magnitude serves as a high-fidelity proxy for gradient norms, and use this to introduce Cost-Aware GRPO. Empirical results on 1.5B, 4B, and 8B LLMs demonstrate that this algorithm significantly reduces the tokens used in policy optimization while matching or exceeding baseline accuracy.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 37 of its 56 references that are in the map. First posted on arXiv 30 Apr 2026.

Builds on

37 of 56 references.

References · 37

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  10. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  11. 2015Trust Region Policy Optimizationcited by 2,503
  12. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
25 more
  1. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  2. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  3. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  4. 2014Convex Optimization: Algorithms and Complexitycited by 569
  5. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 529
  6. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
  7. 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 335
  8. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 323
  9. 2023Data Selection for Language Models via Importance Resamplingcited by 322
  10. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 288
  11. 2025MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attentioncited by 248
  12. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 226
  13. 2018On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimizationcited by 173
  14. 2025LIMR: Less is More for RL Scalingcited by 136
  15. 2013Stochastic Gradient Descent, Weighted Sampling, and the Randomized Kaczmarz algorithmcited by 132
  16. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rolloutscited by 108
  17. 2018Tight Analyses for Non-Smooth Stochastic Gradient Descentcited by 94
  18. 2014Stochastic Optimization with Importance Samplingcited by 91
  19. 2025The Art of Scaling Reinforcement Learning Compute for LLMscited by 91
  20. 2025Soft Adaptive Policy Optimizationcited by 89
  21. 2025Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learningcited by 84
  22. 2015Variance Reduction in SGD by Distributed Importance Samplingcited by 79
  23. 2025Concise Reasoning via Reinforcement Learningcited by 78
  24. 2025Defeating the Training-Inference Mismatch via FP16cited by 62
  25. 2020Adaptive Region-Based Active Learningcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026HiLoRe: What to Store, Compress, or Recompute for Efficient GRPO Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Relative-Budget Theory for Reinforcement Learning with Verifiable Rewards in Large Language Model Reasoning
  2. 2025Dynamic Policy Induction for Adaptive Prompt Optimization: Bridging the Efficiency-Accuracy Gap via Lightweight Reinforcement Learning
  3. 2026MiniOpt: Reasoning to Model and Solve General Optimization Problems with Limited Resources
  4. 2025Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocation
  5. 2025RiskPO: Risk-based Policy Optimization via Verifiable Reward for LLM Post-Training
  6. 2025Conformal Constrained Policy Optimization for Cost-Effective LLM Agents
  7. 2025COPO: Consistency-Aware Policy Optimization
  8. 2025Learning More with Less: A Dynamic Dual-Level Down-Sampling Framework for Efficient Policy Optimization