mlsanityData to 7 Oct 2026

Does This Gradient Spark Joy?

arXiv 2603.20526 ↗First posted 20 Mar 2026cs.LG cs.AI stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

Policy gradient computes a backward pass for every sample, even though the backward pass is expensive and most samples carry little learning value. The Delightful Policy Gradient (DG) provides a forward-pass signal of learning value: delight, the product of advantage and surprisal (negative log-probability). We introduce the Kondo gate, which compares delight against a compute price and pays for a backward pass only when the sample is worth it, thereby tracing a quality--cost Pareto frontier. In bandits, zero-price gating preserves useful gradient signal while removing perpendicular noise, and delight is a more reliable screening signal than additive combinations of value and surprise. On MNIST and transformer token reversal, the Kondo gate skips most backward passes while retaining nearly all of DG's learning quality, with gains that grow as problems get harder and backward passes become more expensive. Because the gate tolerates approximate delight, a cheap forward pass can screen samples before expensive backpropagation, suggesting a speculative-decoding-for-training paradigm.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 11 of its 16 references that are in the map. First posted on arXiv 20 Mar 2026.

Builds on

11 of 16 references.

References · 11

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  7. 2015Prioritized Experience Replaycited by 1,070
  8. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  9. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 323
  10. 2018Distributed Prioritized Experience Replaycited by 260
  11. 2017Automated Curriculum Learning for Neural Networkscited by 251

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.