mlsanityData to 14 Oct 2026

Higher Resolution, Better Generalization: Unlocking Visual Scaling in Deep Reinforcement Learning

arXiv 2605.10546 ↗First posted 11 May 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Pixel-based deep reinforcement learning agents are typically trained on heavily downsampled visual observations, a convention inherited from early benchmarks rather than grounded in principled design. In this work, we show that observation resolution is a critical yet overlooked variable for policy learning: higher-resolution inputs can substantially improve both performance and generalization, provided the network architecture can process them effectively. We find that the widely used Impala encoder, which flattens spatial features into a vector, suffers from quadratic parameter growth as resolution increases and fails to leverage the additional visual detail. We test different modifications to the Impala architecture and conclude that, in particular, introducing a global average pooling layer, as in the Impoola architecture, yields consistent improvements across resolutions and network widths while decoupling parameter count from resolution - at their respective best conditions, visual scaling unlocks a 28 % performance gain for Impoola over Impala. These gains are strongest in environments that require precise perception of small or distant objects, and gradient saliency analysis suggests that the underlying mechanism is a more spatially localized visual attention of the policy at higher resolutions. Our results challenge the prevailing practice of aggressive input downsampling and position resolution-independent architectures as a simple, effective path toward scalable visual deep reinforcement learning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 49 of its 60 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

49 of 60 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Impoola: The Power of Average Pooling for Image-Based Deep Reinforcement Learning7 Mar

Other references · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  7. 2014Going Deeper with Convolutionscited by 4,241
  8. 2022A ConvNet for the 2020scited by 3,579
  9. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  10. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  11. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  12. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
36 more
  1. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  2. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  3. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  4. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  5. 2013Network In Networkcited by 605
  6. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  7. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  8. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  9. 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 461
  10. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  11. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  12. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 349
  13. 2019Challenges of Real-World Reinforcement Learningcited by 291
  14. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  15. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 269
  16. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  17. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  18. 2019Fixing the train-test resolution discrepancycited by 209
  19. 2018A Study on Overfitting in Deep Reinforcement Learningcited by 174
  20. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  21. 2021The Benchmark Lotterycited by 136
  22. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  23. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 125
  24. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  25. 2021Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Researchcited by 102
  26. 2019Observational Overfitting in Reinforcement Learningcited by 91
  27. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 78
  28. 2022The State of Sparse Training in Deep Reinforcement Learningcited by 61
  29. 2021Dynamic Sparse Training for Deep Reinforcement Learningcited by 49
  30. 2024Mixture of Experts in a Mixture of RL settingscited by 21
  31. 2018Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agentscited by 15
  32. 2024Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RLcited by 14
  33. 2025Hadamax Encoding: Elevating Performance in Model-Free Ataricited by 7
  34. 2023Efficient Learning of Urban Driving Policies Using Bird's-Eye-View State Representationscited by 6
  35. 2025Mind the GAP! The Challenges of Scale in Pixel-based Deep Reinforcement Learningcited by 4
  36. 2021Unlocking Pixels for Reinforcement Learning via Implicit Attentioncited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026VIGOR: Zero-Shot Visual Generalization via Latent-Space Consistency in Model-Based Reinforcement Learning
  2. 2026KAGE-Bench: Fast Known-Axis Visual Generalization Evaluation for Reinforcement Learning
  3. 2025Salience-Invariant Consistent Policy Learning for Generalization in Visual Reinforcement Learning
  4. 2024The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning
  5. 2025Pre-trained Visual Representations Generalize Where it Matters in Model-Based Reinforcement Learning
  6. 2025SegDAC: Visual Generalization in Reinforcement Learning via Dynamic Object Tokens
  7. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learning
  8. 2025Reinforcement Learning for Large Model: A Survey