mlsanityData to 14 Oct 2026

Pre-trained Visual Representations Generalize Where it Matters in Model-Based Reinforcement Learning

arXiv 2509.12531 ↗First posted 16 Sep 2025cs.RO cs.AI cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

In visuomotor policy learning, the control policy for the robotic agent is derived directly from visual inputs. The typical approach, where a policy and vision encoder are trained jointly from scratch, generalizes poorly to novel visual scene changes. Using pre-trained vision models (PVMs) to inform a policy network improves robustness in model-free reinforcement learning (MFRL). Recent developments in Model-based reinforcement learning (MBRL) suggest that MBRL is more sample-efficient than MFRL. However, counterintuitively, existing work has found PVMs to be ineffective in MBRL. Here, we investigate PVM's effectiveness in MBRL, specifically on generalization under visual domain shifts. We show that, in scenarios with severe shifts, PVMs perform much better than a baseline model trained from scratch. We further investigate the effects of varying levels of fine-tuning of PVMs. Our results show that partial fine-tuning can maintain the highest average task performance under the most extreme distribution shifts. Our results demonstrate that PVMs are highly successful in promoting robustness in visual policy learning, providing compelling evidence for their wider adoption in model-based robotic learning applications.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 32 of its 37 references that are in the map. First posted on arXiv 16 Sep 2025.

Builds on

32 of 37 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning15 Nov
  2. 2024Pretrained Visual Representations in Reinforcement Learning24 Jul

Other references · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2016OpenAI Gymcited by 1,736
  5. 2023Mastering Diverse Domains through World Modelscited by 1,253
  6. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  7. 2019Solving Rubik's Cube with a Robot Handcited by 734
  8. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  9. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 622
  10. 2021Extract Free Dense Labels from CLIPcited by 583
  11. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  12. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
18 more
  1. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  2. 2022Masked Visual Pre-training for Motor Controlcited by 248
  3. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
  4. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 218
  5. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  6. 2019What Would Elsa Do? Freezing Layers During Transformer Fine-Tuningcited by 89
  7. 2022Offline Visual Representation Learning for Embodied Navigationcited by 75
  8. 2021Improved OOD Generalization via Adversarial Training and Pre-trainingcited by 66
  9. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 60
  10. 2023An Unbiased Look at Datasets for Visuo-Motor Pre-Trainingcited by 42
  11. 2023What Makes Pre-Trained Visual Representations Successful for Robust Manipulation?cited by 39
  12. 2020Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigationcited by 32
  13. 2023RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalizationcited by 28
  14. 2024HFT: Half Fine-Tuning for Large Language Modelscited by 18
  15. 2022The Benefits of Model-Based Generalization in Reinforcement Learningcited by 17
  16. 2023Partial Fine-Tuning: A Successor to Full Fine-Tuning for Vision Transformerscited by 7
  17. 2025A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learningcited by 7
  18. 2023Visual Encoders for Data-Efficient Imitation Learning in Modern Video Gamescited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equal
  2. 2026VIGOR: Zero-Shot Visual Generalization via Latent-Space Consistency in Model-Based Reinforcement Learning
  3. 2024Pre-trained Visual Dynamics Representations for Efficient Policy Learning
  4. 2022Reinforcement Learning with Action-Free Pre-Training from Videos
  5. 2025Self-Consistent Model-based Adaptation for Visual Reinforcement Learning
  6. 2022Masked World Models for Visual Control
  7. 2025MInCo: Mitigating Information Conflicts in Distracted Visual Model-based Reinforcement Learning
  8. 2025WMPO: World Model-based Policy Optimization for Vision-Language-Action Models