mlsanityData to 13 Oct 2026

Learning values across many orders of magnitude

arXiv 1602.07714 ↗First posted 24 Feb 2016cs.LG cs.AI cs.NEcited by 82

Not on the map. Why?

Cited by 30 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Most learning algorithms are not invariant to the scale of the function that is being approximated. We propose to adaptively normalize the targets used in learning. This is useful in value-based reinforcement learning, where the magnitude of appropriate value approximations can change over time when we update the policy of behavior. Our main motivation is prior work on learning to play Atari games, where the rewards were all clipped to a predetermined range. This clipping facilitates learning across many different games with a single learning algorithm, but a clipped reward function can result in qualitatively different behavior. Using the adaptive normalization we can remove this domain-specific heuristic without diminishing overall performance.

Key facts. Cited by 82 papers in the map (data through 13 Oct 2026). First posted on arXiv 24 Feb 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 1; 2021: 16; 2022: 9; 2023: 13; 2024: 11; 2025: 9; 2026: 22.

82 papers.

Sort:

By area:

  1. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 363
  2. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 181
  3. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  4. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 139
  5. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 103
  6. 2024Disentangling the Causes of Plasticity Loss in Neural Networkscited by 85
  7. 2022In Defense of the Unitary Scalarization for Deep Multi-Task Learningcited by 65
  8. 2021Muesli: Combining Improvements in Policy Optimizationcited by 53
  9. 2023Better Zero-Shot Reasoning with Self-Adaptive Promptingcited by 52
  10. 2024From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessonscited by 42
  11. 2018Sample-Efficient Imitation Learning via Generative Adversarial Netscited by 36
  12. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 36
38 more
  1. 2023Composite Motion Learning with Task Controlcited by 26
  2. 2024Streaming Deep Reinforcement Learning Finally Workscited by 24
  3. 2021Return-based Scaling: Yet Another Normalisation Trick for Deep RLcited by 23
  4. 2025MO-GRPO: Mitigating Reward Hacking of Group Relative Policy Optimization on Multi-Objective Problemscited by 21
  5. 2021CoBERL: Contrastive BERT for Reinforcement Learningcited by 20
  6. 2023LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarioscited by 20
  7. 2022ALMA: Hierarchical Learning for Composite Multi-Agent Taskscited by 17
  8. 2024Video-Language Critic: Transferable Reward Functions for Language-Conditioned Roboticscited by 17
  9. 2024Reward Centeringcited by 16
  10. 2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learningcited by 15
  11. 2024A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learningcited by 15
  12. 2021Combining Pessimism with Optimism for Robust and Efficient Model-Based Deep Reinforcement Learningcited by 14
  13. 2022Consistency driven Sequential Transformers Attention Model for Partially Observable Scenescited by 11
  14. 2024The Evolution of Reinforcement Learning in Quantitative Finance: A Surveycited by 11
  15. 2024Monitored Markov Decision Processescited by 10
  16. 2023Reward Scale Robustness for Proximal Policy Optimization via DreamerV3 Trickscited by 7
  17. 2022Comparative analysis of machine learning methods for active flow controlcited by 6
  18. 2022Revisiting Gaussian mixture critics in off-policy reinforcement learning: a sample-based approachcited by 6
  19. 2023Online Reinforcement Learning in Non-Stationary Context-Driven Environmentscited by 6
  20. 2023Replay-enhanced Continual Reinforcement Learningcited by 6
  21. 2025Learning to Ball: Composing Policies for Long-Horizon Basketball Movescited by 6
  22. 2021A Closer Look at Advantage-Filtered Behavioral Cloning in High-Noise Datasetscited by 5
  23. 2025On the Importance of Reward Design in Reinforcement Learning-based Dynamic Algorithm Configuration: A Case Study on OneMax with (1+(λ,λ))-GAcited by 5
  24. 2025RAPTOR: A Foundation Policy for Quadrotor Controlcited by 4
  25. 2021A Deep Reinforcement Learning Framework for Eco-driving in Connected and Automated Hybrid Electric Vehiclescited by 3
  26. 2021Efficient Reinforcement Learning in Resource Allocation Problems Through Permutation Invariant Multi-task Learningcited by 3
  27. 2021Computational Performance of Deep Reinforcement Learning to find Nash Equilibriacited by 3
  28. 2021Adapting to Reward Progressivity via Spectral Reinforcement Learningcited by 3
  29. 2021Learning Emergent Random Access Protocol for LEO Satellite Networkscited by 3
  30. 2022BCRLSP: An Offline Reinforcement Learning Framework for Sequential Targeted Promotioncited by 3
  31. 2023Policy-Based Self-Competition for Planning Problemscited by 3
  32. 2024Maximum Entropy On-Policy Actor-Critic via Entropy Advantage Estimationcited by 3
  33. 2025VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Makingcited by 3
  34. 2021L2PF -- Learning to Prune Fastercited by 2
  35. 2022Demystifying Reinforcement Learning in Time-Varying Systemscited by 2
  36. 2022MAN: Multi-Action Networks Learningcited by 2
  37. 2023Doing the right thing for the right reason: Evaluating artificial moral cognition by probing cost insensitivitycited by 2
  38. 2024Physics-Guided Actor-Critic Reinforcement Learning for Swimming in Turbulencecited by 2

50 most cited of 82 All 82 (JSON)