mlsanityData to 14 Oct 2026

Rainbow: Combining Improvements in Deep Reinforcement Learning

arXiv 1710.02298 ↗First posted 6 Oct 2017cs.AI cs.LGcited by 786

Not on the map. Why?

Cited by 225 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The deep reinforcement learning community has made several independent improvements to the DQN algorithm. However, it is unclear which of these extensions are complementary and can be fruitfully combined. This paper examines six extensions to the DQN algorithm and empirically studies their combination. Our experiments show that the combination provides state-of-the-art performance on the Atari 2600 benchmark, both in terms of data efficiency and final performance. We also provide results from a detailed ablation study that shows the contribution of each component to overall performance.

Key facts. Cited by 786 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 2; 2019: 6; 2020: 6; 2021: 181; 2022: 138; 2023: 134; 2024: 132; 2025: 112; 2026: 75.

786 papers.

Sort:

By area:

  1. 2023Mastering Diverse Domains through World Modelscited by 1,253
  2. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  3. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  4. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  5. 2022Transformers are Sample-Efficient World Modelscited by 328
  6. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  7. 2021Mastering Atari Games with Limited Datacited by 237
  8. 2022Multi-Game Decision Transformerscited by 214
  9. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
  10. 2022Active Example Selection for In-Context Learningcited by 196
  11. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 194
  12. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174
38 more
  1. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  2. 2021Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learningcited by 158
  3. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 147
  4. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  5. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 139
  6. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 139
  7. 2023Controlled Decoding from Language Modelscited by 135
  8. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 131
  9. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 125
  10. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  11. 2021Learning to drive from a world on railscited by 114
  12. 2021APS: Active Pretraining with Successor Featurescited by 112
  13. 2019Benchmarking Batch Deep Reinforcement Learning Algorithmscited by 111
  14. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  15. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 108
  16. 2025Reinforcement Learning with Action Chunkingcited by 107
  17. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  18. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 105
  19. 2021Online and Offline Reinforcement Learning by Planning with a Learned Modelcited by 90
  20. 2023STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learningcited by 89
  21. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  22. 2021Conservative Offline Distributional Reinforcement Learningcited by 79
  23. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 78
  24. 2024Simplifying Deep Temporal Difference Learningcited by 78
  25. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learningcited by 76
  26. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 74
  27. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
  28. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 70
  29. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 70
  30. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 68
  31. 2021Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspectivecited by 65
  32. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 65
  33. 2021Playable Video Generationcited by 64
  34. 2022Federated Reinforcement Learning with Environment Heterogeneitycited by 63
  35. 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Drivingcited by 60
  36. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 60
  37. 2021Learning Task Informed Abstractionscited by 58
  38. 2024Normalization and effective learning rates in reinforcement learningcited by 58

50 most cited of 786 All 786 (JSON)