V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Control

arXiv 2608.07870 ↗First posted 8 Aug 2026cs.LG cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Improving sample efficiency remains a core challenge in reinforcement learning (RL), especially in real-world settings like robotics, where data collection is costly. This challenge is pronounced in visual RL, where high-dimensional inputs often obscure learning signals. While prior work in visual RL has focused on algorithmic solutions, such as better dynamics models or exploration strategies, recent advances in state-based RL show that architectural design alone can lead to significant gains in sample efficiency. This raises an important question: Can these architectural principles transfer to visual RL? In response, we introduce V-Simba, a simple yet effective visual RL architecture inspired by the Simba architecture from state-based RL. Built on top of Soft Actor-Critic (SAC) with data augmentation, V-Simba modifies the architecture by adding normalization layers to stabilize training and using pointwise convolutions to reduce computation. Despite its simplicity, V-Simba matches or outperforms the state-of-the-art methods across the DMC, Adroit, and Meta-World benchmarks, while being more computationally efficient than DrQ-v2. We make our code publicly available at https://github.com/DAVIAN-Robotics/V-Simba.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 87 of its 99 references that are in the map. First posted on arXiv 8 Aug 2026.

Builds on

87 of 99 references.

Closest ancestors · 3

Same direction, newest first

Other references · 84

2015Deep Residual Learning for Image Recognitioncited by 52,096
2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
2016Layer Normalizationcited by 5,669
2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
2022A ConvNet for the 2020scited by 3,579
2015Continuous control with deep reinforcement learningcited by 3,407
2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
2018World Modelscited by 1,513
2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
2023Mastering Diverse Domains through World Modelscited by 1,253
2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194

38 more

2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
2018Exploration by Random Network Distillationcited by 993
2018DeepMind Control Suitecited by 956
2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
2017A Distributional Perspective on Reinforcement Learningcited by 765
2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
2020Reinforcement Learning with Augmented Datacited by 468
2022DayDreamer: World Models for Physical Robot Learningcited by 400
2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
2020Planning to Explore via Self-Supervised World Modelscited by 368
2023Efficient Online Reinforcement Learning with Offline Datacited by 363
2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
2020Decoupling Representation Learning from Reinforcement Learningcited by 269
2021Mastering Atari Games with Limited Datacited by 237
2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
2023Understanding plasticity in neural networkscited by 174
2015Deep Spatial Autoencoders for Visuomotor Learningcited by 153
2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 147
2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 144
2017Pointwise Convolutional Neural Networkscited by 127
2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 108
2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
2019CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicitycited by 95
2024Disentangling the Causes of Plasticity Loss in Neural Networkscited by 86
2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 78
2025MuJoCo Playgroundcited by 78
2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learningcited by 76
2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 73
2024Normalization and effective learning rates in reinforcement learningcited by 58
2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 57
2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 54

50 most cited of 84 All 84 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way