mlsanityData to 7 Oct 2026

RRL: Resnet as representation for Reinforcement Learning

arXiv 2107.03380 ↗First posted 7 Jul 2021cs.ROcited by 89

Not on the map. Why?

Abstract (arXiv)

The ability to autonomously learn behaviors via direct interactions in uninstrumented environments can lead to generalist robots capable of enhancing productivity or providing care in unstructured settings like homes. Such uninstrumented settings warrant operations only using the robot's proprioceptive sensor such as onboard cameras, joint encoders, etc which can be challenging for policy learning owing to the high dimensionality and partial observability issues. We propose RRL: Resnet as representation for Reinforcement Learning -- a straightforward yet effective approach that can learn complex behaviors directly from proprioceptive inputs. RRL fuses features extracted from pre-trained Resnet into the standard reinforcement learning pipeline and delivers results comparable to learning directly from the state. In a simulated dexterous manipulation benchmark, where the state of the art methods fail to make significant progress, RRL delivers contact rich behaviors. The appeal of RRL lies in its simplicity in bringing together progress from the fields of Representation Learning, Imitation Learning, and Reinforcement Learning. Its effectiveness in learning behaviors directly from visual inputs with performance and sample efficiency matching learning directly from the state, even in complex high dimensional domains, is far from obvious.

Key facts. Cited by 89 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jul 2021.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 21; 2023: 37; 2024: 19; 2025: 9; 2026: 1.

89 papers.

Sort:

By area:

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  2. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  3. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  4. 2022Masked Visual Pre-training for Motor Controlcited by 246
  5. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
  6. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 180
  7. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  8. 2023Language-Driven Representation Learning for Roboticscited by 163
  9. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  10. 2023Open-World Object Manipulation using Pre-trained Vision-Language Modelscited by 155
  11. 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 125
  12. 2023Policy Expansion for Bridging Offline-to-Online Reinforcement Learningcited by 103
38 more
  1. 2024Foundation Policies with Hilbert Representationscited by 89
  2. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  3. 2023Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulationcited by 78
  4. 2022Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?cited by 70
  5. 2022The State of Sparse Training in Deep Reinforcement Learningcited by 60
  6. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 60
  7. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 59
  8. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 58
  9. 2024Theia: Distilling Diverse Vision Foundation Models for Robot Learningcited by 56
  10. 2022On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baselinecited by 55
  11. 2023Imitation Bootstrapped Reinforcement Learningcited by 54
  12. 2022VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learningcited by 51
  13. 2022Retrospectives on the Embodied AI Workshopcited by 46
  14. 2023Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learningcited by 45
  15. 2023SMART: Self-supervised Multi-task pretrAining with contRol Transformerscited by 44
  16. 2024DINOBot: Robot Manipulation via Retrieval and Alignment with Vision Foundation Modelscited by 43
  17. 2024HRP: Human Affordances for Robotic Pre-Trainingcited by 40
  18. 2022Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretrainingcited by 37
  19. 2023Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learningcited by 37
  20. 2022Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Cloudscited by 31
  21. 2023Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotioncited by 30
  22. 2023Lossless Adaptation of Pretrained Vision Models For Robotic Manipulationcited by 28
  23. 2023Polybot: Training One Policy Across Robots While Embracing Variabilitycited by 28
  24. 2023MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulationcited by 27
  25. 2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 27
  26. 2024SUGAR: Pre-training 3D Visual Representations for Roboticscited by 26
  27. 2023Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modelingcited by 25
  28. 2024CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulationcited by 25
  29. 2022Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?cited by 24
  30. 2023SpawnNet: Learning Generalizable Visuomotor Skills from Pre-trained Networkscited by 24
  31. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
  32. 2024Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representationscited by 23
  33. 2025UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulationcited by 23
  34. 2023For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equalcited by 22
  35. 2023Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robotcited by 21
  36. 2021The Functional Correspondence Problemcited by 20
  37. 2023Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Criticcited by 20
  38. 2023Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Datacited by 19

50 most cited of 89 All 89 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning
  2. 2026Res-HIL: Human-Guided Residual Reinforcement Learning for Sample-Efficient Dexterous Manipulation
  3. 2026Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards
  4. 2025RM-RL: Role-Model Reinforcement Learning for Precise Robot Manipulation
  5. 2021Residual Reinforcement Learning from Demonstrations
  6. 2025Zero-Shot Visual Generalization in Robot Manipulation
  7. 2023REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulation
  8. 2021Manipulator-Independent Representations for Visual Imitation