mlsanityData to 14 Oct 2026

D2RL: Deep Dense Architectures in Reinforcement Learning

arXiv 2010.09163 ↗First posted 19 Oct 2020cs.LGcited by 42

Not on the map. Why?

Abstract (arXiv)

While improvements in deep learning architectures have played a crucial role in improving the state of supervised and unsupervised learning in computer vision and natural language processing, neural network architecture choices for reinforcement learning remain relatively under-explored. We take inspiration from successful architectural choices in computer vision and generative modelling, and investigate the use of deeper networks and dense connections for reinforcement learning on a variety of simulated robotic learning benchmark environments. Our findings reveal that current methods benefit significantly from dense connections and deeper networks, across a suite of manipulation and locomotion tasks, for both proprioceptive and image-based observations. We hope that our results can serve as a strong baseline and further motivate future research into neural network architectures for reinforcement learning. The project website with code is at this link https://sites.google.com/view/d2rl/home.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 11; 2022: 6; 2023: 9; 2024: 5; 2025: 7; 2026: 4.

42 papers.

Sort:

By area:

  1. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  2. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 147
  3. 2023Revisiting the Minimalist Approach to Offline Reinforcement Learningcited by 126
  4. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningcited by 85
  5. 2022Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizescited by 73
  6. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 70
  7. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  8. 2024In value-based deep reinforcement learning, a pruned network is a good networkcited by 39
  9. 2021Training Larger Networks for Deep Reinforcement Learningcited by 38
  10. 2021Hierarchical Skills for Efficient Explorationcited by 38
  11. 2021Recomposing the Reinforcement Learning Building Blocks with Hypernetworkscited by 35
  12. 2023Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlocited by 29
30 more
  1. 2022Policy Architectures for Compositional Generalization in Controlcited by 23
  2. 2021Reinforcement Learning with Latent Flowcited by 19
  3. 2021Towards Deeper Deep Reinforcement Learning with Spectral Normalizationcited by 19
  4. 2022Overcoming the Spectral Bias of Neural Value Approximationcited by 19
  5. 2021Functional Regularization for Reinforcement Learning via Learned Fourier Featurescited by 17
  6. 2021Fast and Data-Efficient Training of Rainbow: an Experimental Study on Ataricited by 10
  7. 2023Estimator-Coupled Reinforcement Learning for Robust Purely Tactile In-Hand Manipulationcited by 9
  8. 2025A Forget-and-Grow Strategy for Deep Reinforcement Learning Scaling in Continuous Controlcited by 9
  9. 2022Transformer in Transformer as Backbone for Deep Reinforcement Learningcited by 6
  10. 2025RAPID: Robust and Agile Planner Using Inverse Reinforcement Learning for Vision-Based Drone Navigationcited by 6
  11. 2025Impoola: The Power of Average Pooling for Image-Based Deep Reinforcement Learningcited by 6
  12. 2023PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learningcited by 5
  13. 2023Optimizing with Low Budgets: a Comparison on the Black-box Optimization Benchmarking Suite and OpenAI Gymcited by 4
  14. 2024Task and Domain Adaptive Reinforcement Learning for Robot Controlcited by 4
  15. 2025Reinforced Multi-teacher Knowledge Distillation for Efficient General Image Forgery Detection and Localizationcited by 4
  16. 2021Honey, I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RLcited by 3
  17. 2021Reinforcement Learning with Formal Performance Metrics for Quadcopter Attitude Control under Non-nominal Contextscited by 3
  18. 2022Leveraging Demonstrations with Latent Space Priorscited by 3
  19. 2023Shrink-Perturb Improves Architecture Mixing during Population Based Training for Neural Architecture Searchcited by 2
  20. 2024Offline Goal-Conditioned Reinforcement Learning for Shape Control of Deformable Linear Objectscited by 2
  21. 2024Offline Model-Based Reinforcement Learning with Anti-Explorationcited by 1
  22. 2025Learning Dexterous Bimanual Catch Skills through Adversarial-Cooperative Heterogeneous-Agent Reinforcement Learningcited by 1
  23. 2023A Central Motor System Inspired Pre-training Reinforcement Learning for Robotic Controlcited by 0
  24. 2024Intelligent Hybrid Resource Allocation in MEC-assisted RAN Slicing Networkcited by 0
  25. 2025Catching Spinning Table Tennis Balls in Simulation with End-to-End Curriculum Reinforcement Learningcited by 0
  26. 2025Online Training and Pruning of Deep Reinforcement Learning Networkscited by 0
  27. 2026M2I2HA: Multi-modal Object Detection Based on Intra- and Inter-Modal Hypergraph Attentioncited by 0
  28. 2026Distributional Value Estimation Without Target Networks for Robust Quality-Diversitycited by 0
  29. 2026Simple Actors and Deep Critics for Scalable Reinforcement Learningcited by 0
  30. 2026Combining Improvements in Uplink AI-RANcited by 0