# Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation

arXiv: https://arxiv.org/abs/2011.06698 · HTML: https://mlsanity.com/paper/2011.06698/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2020.

## Built on by (most cited first, 30 of 32)

- [Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning](https://mlsanity.com/paper/2107.09645/) (2021)
- [Masked Visual Pre-training for Motor Control](https://mlsanity.com/paper/2203.06173/) (2022)
- [Reinforcement Learning with Prototypical Representations](https://mlsanity.com/paper/2102.11271/) (2021)
- [Open-World Object Manipulation using Pre-trained Vision-Language Models](https://mlsanity.com/paper/2303.00905/) (2023)
- [The Surprising Effectiveness of Representation Learning for Visual Imitation](https://mlsanity.com/paper/2112.01511/) (2021)
- [RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation](https://mlsanity.com/paper/2306.11706/) (2023)
- [Instruction-driven history-aware policies for robotic manipulations](https://mlsanity.com/paper/2209.04899/) (2022)
- [Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play](https://mlsanity.com/paper/2303.12076/) (2023)
- [Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learning](https://mlsanity.com/paper/2212.08860/) (2022)
- [Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks](https://mlsanity.com/paper/2108.01176/) (2021)
- [CrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Data](https://mlsanity.com/paper/2112.09081/) (2021)
- [The Reality Gap in Robotics: Challenges, Solutions, and Best Practices](https://mlsanity.com/paper/2510.20808/) (2025)
- [Diffusion Dynamics Models with Generative State Estimation for Cloth Manipulation](https://mlsanity.com/paper/2503.11999/) (2025)
- [GOOD: Exploring Geometric Cues for Detecting Objects in an Open World](https://mlsanity.com/paper/2212.11720/) (2022)
- [Accelerating Model-Based Reinforcement Learning with State-Space World Models](https://mlsanity.com/paper/2502.20168/) (2025)
- [Vision-Language Navigation with Embodied Intelligence: A Survey](https://mlsanity.com/paper/2402.14304/) (2024)
- [Learning Continuous Environment Fields via Implicit Functions](https://mlsanity.com/paper/2111.13997/) (2021)
- [Weakly Supervised Correspondence Learning](https://mlsanity.com/paper/2203.00904/) (2022)
- [The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2411.10175/) (2024)
- [Invariance is Key to Generalization: Examining the Role of Representation in Sim-to-Real Transfer for Visual Navigation](https://mlsanity.com/paper/2310.15020/) (2023)
- [Self-Supervised Domain Adaptation for Visual Navigation with Global Map Consistency](https://mlsanity.com/paper/2110.07184/) (2021)
- [Seeking Visual Discomfort: Curiosity-driven Representations for Reinforcement Learning](https://mlsanity.com/paper/2110.00784/) (2021)
- [Image-based Navigation in Real-World Environments via Multiple Mid-level Representations: Fusion Models, Benchmark and Efficient Evaluation](https://mlsanity.com/paper/2202.01069/) (2022)
- [Investigation of Factorized Optical Flows as Mid-Level Representations](https://mlsanity.com/paper/2203.04927/) (2022)
- [Virtual Guidance as a Mid-level Representation for Navigation with Augmented Reality](https://mlsanity.com/paper/2303.02731/) (2023)
- [Visual Forecasting as a Mid-level Representation for Avoidance](https://mlsanity.com/paper/2310.07724/) (2023)
- [Dual-Arm Adversarial Robot Learning](https://mlsanity.com/paper/2110.08066/) (2021)
- [Knowledge Transfer in Deep Reinforcement Learning via an RL-Specific GAN-Based Correspondence Function](https://mlsanity.com/paper/2209.06604/) (2022)
- [Few-Shot Image-to-Semantics Translation for Policy Transfer in Reinforcement Learning](https://mlsanity.com/paper/2301.13343/) (2023)
- [Pre-trained Visual Representations Generalize Where it Matters in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2509.12531/) (2025)

## Abstract (arXiv)

Vision-based robotics often separates the control loop into one module for perception and a separate module for control. It is possible to train the whole system end-to-end (e.g. with deep RL), but doing it "from scratch" comes with a high sample complexity cost and the final result is often brittle, failing unexpectedly if the test environment differs from that of training. We study the effects of using mid-level visual representations (features learned asynchronously for traditional computer vision objectives), as a generic and easy-to-decode perceptual state in an end-to-end RL framework. Mid-level representations encode invariances about the world, and we show that they aid generalization, improve sample complexity, and lead to a higher final performance. Compared to other approaches for incorporating invariances, such as domain randomization, asynchronously trained mid-level representations scale better: both to harder problems and to larger domain shifts. In practice, this means that mid-level representations could be used to successfully train policies for tasks where domain randomization and learning-from-scratch failed. We report results on both manipulation and navigation tasks, and for navigation include zero-shot sim-to-real experiments on real robots.