Data to 7 Oct 2026

Learning Invariant Representations for Reinforcement Learning without Reconstruction

arXiv 2006.10742 ↗First posted 18 Jun 2020cs.LG cs.AI stat.MLcited by 424

Not on the map. Why?

Cited by 47 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction. Our goal is to learn representations that both provide for effective downstream control and invariance to task-irrelevant details. Bisimulation metrics quantify behavioral similarity between states in continuous MDPs, which we propose using to learn robust latent representations which encode only the task-relevant information from observations. Our method trains encoders such that distances in latent space equal bisimulation distances in state space. We demonstrate the effectiveness of our method at disregarding task-irrelevant information using modified visual MuJoCo tasks, where the background is replaced with moving distractors and natural videos, while achieving SOTA performance. We also test a first-person highway driving task where our method learns invariance to clouds, weather, and time of day. Finally, we provide generalization results drawn from properties of bisimulation metrics, and links to causal inference.

Key facts. Cited by 424 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 59; 2022: 90; 2023: 81; 2024: 61; 2025: 63; 2026: 70.

424 papers.

Sort:

By area:

2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
2023A Cookbook of Self-Supervised Learningcited by 248
2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
2022Multi-Game Decision Transformerscited by 212
2022Masked World Models for Visual Controlcited by 169
2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
2023Language-Driven Representation Learning for Roboticscited by 163
2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 108

38 more

2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
2021Model-Based Domain Generalizationcited by 103
2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 102
2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 97
2021Representation Matters: Offline Pretraining for Sequential Decision Makingcited by 90
2021Model-free Representation Learning and Exploration in Low-rank MDPscited by 82
2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 81
2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representationscited by 73
2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
2022Denoised MDPs: Learning World Models Better Than the World Itselfcited by 66
2022Causal Dynamics Learning for Task-Independent State Abstractioncited by 64
2021Learning Task Informed Abstractionscited by 58
2022A Policy-Guided Imitation Approach for Offline Reinforcement Learningcited by 58
2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 58
2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 56
2021High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learningcited by 54
2021Temporal Predictive Coding For Model-Based Planning In Latent Spacecited by 54
2022Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Modelscited by 53
2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 52
2022VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learningcited by 51
2021Towards Robust Bisimulation Metric Learningcited by 49
2022Predicate Invention for Bilevel Planningcited by 49
2022Making Linear MDPs Practical via Contrastive Representation Learningcited by 49
2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 46
2022Retrospectives on the Embodied AI Workshopcited by 46
2023Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learningcited by 45
2021MICo: Improved representations via sampling-based state similarity for Markov decision processescited by 43
2022Reinforcement Learning with Neural Radiance Fieldscited by 43
2022Visuo-Tactile Transformers for Manipulationcited by 43
2023A Survey on Causal Reinforcement Learningcited by 43
2021Learning Markov State Abstractions for Deep Reinforcement Learningcited by 42
2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 42
2022Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Modelscited by 42
2021PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learningcited by 41
2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
2023Proto-Value Networks: Scaling Representation Learning with Auxiliary Taskscited by 41
2025Towards General-Purpose Model-Free Reinforcement Learningcited by 40

50 most cited of 424 All 424 (JSON)